I have a dataframe as seen below: I need two columns now, column A and Column B. Conditions summarized: The required dataframe should be as follows: Answer Try: The !=” will work if you truly have nothing in the cell (as opposed to a NaN etc.). If you have actual NaN values use:
Tag: pandas
Trying to stack several groups of columns into multiple target columns by name
I have the original dataframe like that which contains 1772 columns and 130 rows. I would like to stack them into multiple target columns. id AA_F1R1 BB_F1R1 AA_F1R2 BB_F1R2 … AA_F2R1 BB_F2R2 … AA_F7R25 BB_F7R25 001 5 xy xx xx zy 1 4 xx 002 6 zzz yyy zzz xw 2 zzz 3 zzz I found two different soluti…
Convert object column format “%d-%b-%y” to datetime, where month initial letter is Upper or lowercase and Month can be in italian or english
For this dataframe (link): I want to convert columns [Issue date, Redemption date] to datetime format. The format variations are: Month is lower case: 01-set-09 Month in Italian 01-set-09 (settembre-italian instead of September-English) I have tried using: and this one: however i get the following error: Here…
Remove strings and special characters from multiple columns
I want to remove strings and special characters from multiple columns (which contain nan values). I only want the numbers to be present. Below is one of my columns. Any help will be appreciated! Expected outcome: Answer Start by filling NaN values by empty string, then extract the values by regex, then fill t…
How to get all last rows at second level in MultiIndex DataFrame whose second level has variable length
I have this dataframe: And I want to keep all the last second level rows, meaning that: For thread_id==0 I want to keep the row message_id_in_thread==1 For thread_id==1 I want to keep the row message_id_in_thread==2 For thread_id==2 I want to keep the row message_id_in_thread==1 This can easily be achieved by…
Pandas groupby collapse 1st rows of group
I have a system that lets me export data in a table of this format: where there are many columns like ‘data’ and they can have any values that don’t necessarily follow a pattern. I need to get the data into this format: I’ve tried reading the documentation on gropuby and searching si…
How do I traverse through a dataframe and get polarity score of existing text(transcript) so I have 1 row per id in python?
I am able to traverse through files in a directory with my script but unable to apply the same logic to when all the transcriptions are in a table/dataframe. My earlier script – How do I apply the above to the below table where So as you see here, I have a column interaction id which is unique. I my
Calculate delta in dictionary of dictionary
I have a dictionary of dictionaries which hold list of tuples like this: I would like to calculate the delta of the third items (which their first two items are identical) in each tuple inside of the dictionaries, between each week (e.g., week1 and week2,.. week19 and week20)and put them as new dictionaries i…
Scrape eBay Sold Items Using Selenium Returns []
I have almost no webscraping experience, and wasn’t able to solve this using BeautifulSoup, so I’m trying selenium (installed it today). I’m trying to scrape sold items on eBay. I’m trying to scrape: https://www.ebay.com/sch/i.html?_from=R40&_nkw=oakley+sunglasses&_sacat=0&…
CSV data preprocess
I have a .csv file like this format Then I want to convert it to How can I do it with python pandas Thank you Answer If you load it then you will have DataFrame like Set multi-index usinig year and month Reshape it using stack() You can add name to column with day Reset index to get normal columns