I’ve a problem with groupby function of pandas’s library. I’ve the following dataframe. id result date 400001 N 2020-07-03 400001 N 2021-09-09 400001 P 2021-10-27 400002 N 2020-07-03 400003 N 2020-06-30 400003 N 2022-04-27 400004 P 2020-06-30 400004 N 2022-04-27 I need to group by column …
Tag: pandas-groupby
How to use a pandas groupby to filter this dataframe?
Using Python how can you use a group-by to filter this dataset Start How can I make it so that where either the two conditions are accepted, filtering everything else that doesn’t meet these two criteria ID1 – Matches another ID1 and the Last3 are the same ID2 – Matches another ID2 and the F…
Add missing rows in pandas DataFrame
I have a DataFrame that looks like this: What I want to get is: In short, for each id, add the time rows missing with value 0. How do I do this? I wrote something with a loop, but it’s going to be prohibitively slow for my use case which has several million rows Answer Here’s one way using groupby…
Pandas to read a excel file from s3 and apply some operation and write the file in same location
i am using pandas to read an excel file from s3 and i will be doing some operation in one of the column and write the new version in same location. Basically new version will overwrite the original version. with csv file i am able to achieve using the below code but not sure of excel(.xlsx). Please can someon…
Ungrouping a pandas dataframe after aggregation operation
I have used the “groupby” method on my dataframe to find the total number of people at each location. To the right of the “sum” column, I need to add a column that lists all of the people’s names at each location (ideally in separate rows, but a list would be fine too). Is there …
How to get grouped cumulative duration in pandas?
I have the following data: id encounter_key datetime 1 111 2019-04-14 1 111 2019-04-14 1 111 2019-07-18 1 122 2019-09-02 2 211 2019-10-03 2 211 2020-10-03 I want to find the cumulative duration, grouped by id and encounter_key to achieve the following: id encounter_key datetime cum_duration_days 1 111 2019-04…
Pandas cumsum with keys
I have two DataFrames (first, second): index_first value_1 value_2 0 100 1 1 200 2 2 300 3 index_second value_1 value_2 0 50 10 1 100 20 2 150 30 Next I concat the two DataFrames with keys: My goal is to calculate the cumulative sum of value_1 and value_2 in z considering the keys. So the final DataFrame shou…
how to add the value from the next row to the current row
I want to group by id column and add the value from the next row to the current row only for the trip column How can I transform the first data frame to the second data frame shown below? Answer I am not sure if the only thing requested is to concatenate the trip ID of the next row to
How to obtain dataframe from grouped element after using apply
Let’s say this the dataframe: Then the goal is to produce this: The total Val1 is Y as long as one of the instances is Y. My code looks like this: This works except that cumulative has dtype object and I can only access Val1, that is, I cannot access First Name or Last Name (Although when I run print(cu…
Selecting first row from each subgroup (pandas)
How to select the subset of rows where distance is lowest, grouping by date and p columns? Ideally, the returned dataframe should contain: Answer One way is to use groupby + idxmin to get the index of the smallest distance per group, then use loc to get the desired output: Output: