I have two dataframes colored by approximately matching marks: df1: df2: The “marks” are not the same in each of them, but some are close. How can I copy the “Evaluated” value from df2 to df1 based on relevant “name” and “mark”? My code is: Expected result is df…
Tag: dataframe
Is there a better way to increment a timestamp column in a pandas dataframe?
I’m working with a large pandas dataframe and want to add a timestamp column which correlates to the value of another column. For example, the current dataframe looks like this: Server Hour server1 0 server2 0 server1000 0 server1 1 server2 1 and so on, with the hours column at ranging from 0-167, as th…
Pandas: Rolling window to count the frequency – Fastest approach
I would like to count the frequency of a value for the past x days. In the example below, I would like to count the frequency of value in the Name column for the past 28 days. The data is already sorted by Date I found some solutions on StackOverFlow but all of them are neither correct on the dataset
Filter dataframe per ID based on conditional timerange
Hi I will try to explain the issue I am facing. I have one dataframe (df) with the following: ID Date (dd-mm-yyyy) AAA 01-09-2020 AAA 01-11-2020 AAA 18-03-2021 AAA 10-10-2022 BBB 01-01-2019 BBB 01-03-2019 CCC 01-05-2020 CCC 01-07-2020 CCC 01-08-2020 CCC 01-10-2021 I have created another dataframe (df2) with t…
Convert a Pandas DataFrame with true/false to a dictionary
I would like to transform a dataframe with the following layout: into a dictionary with the following structure: Answer IIUC, you could replace the False to NA (assuming boolean False here, for strings use ‘false’), then stack to remove the values and use groupby.agg to aggregate as list before co…
Pandas to read a excel file from s3 and apply some operation and write the file in same location
i am using pandas to read an excel file from s3 and i will be doing some operation in one of the column and write the new version in same location. Basically new version will overwrite the original version. with csv file i am able to achieve using the below code but not sure of excel(.xlsx). Please can someon…
Select specific rows from pivot table in pandas
I have a dataframe which I pivoted and I now want to select spefici rows from the data. I have seen similar questions such as the one here: Selecting columns in a pandas pivot table based on specific row value?. In my case I want to return all the columns but I want to select only specific rows. I have
How to group by month and year from a specific range?
The data have reported values for January 2006 through January 2019. I need to compute the total number of passengers Passenger_Count per month. The dataframe should have 121 entries (10 years * 12 months, plus 1 for january 2019). The range should go from 2009 to 2019. I have been doing: But it doesn’t…
How to set merge and normalize multple dataframes for pd.merge_as_of
I am trying to merge multiple dataframes using pd.merge_asof. They all contain 2 columns with datetime as index column and a variable column with floating values. They are not balanced in their indexes and times so I have to normalize the values. I can succesfully merge the dfs and normalize the values like t…
Ungrouping a pandas dataframe after aggregation operation
I have used the “groupby” method on my dataframe to find the total number of people at each location. To the right of the “sum” column, I need to add a column that lists all of the people’s names at each location (ideally in separate rows, but a list would be fine too). Is there …