Skip to content

Tag: pandas

pandas – idiomatic way to stash rows filtered out of dataframe

What is the idiomatic way to split a dataframe into two by a condition? Or put another way, to filter a dataframe by a condition but keep the filtered out rows in a new dataframe? I was thinking I could filter the original dataframe for the inverse of the ‘filtered out by condition’ rows using the…

Plot confidence interval of a duration series

I measured the duration of 6000 requests. I got now an Array of 6000 elements. Each element represents the duration of a connection request in milliseconds. [3,2,2,3,4,2,2,4,2,3,3,4,2,4,4,3,3,3,4,3,2,3,5,5,2,4,4,2,2,2,3,5,3,2,2,3,3,3,5,4……..] I want to plot the confidence interval in Python and in…

Create a Nested list from a pandas data frame

I am trying to create a kind of nested list from a pandas data frame. I have this data frame: So I want to create a kind of nested list using the above data frame using ‘id1’ column, and ‘Name1’ and ‘Name2’ columns. For example, if we think about the first row, id1 should b…

Using If statement with string values in Python

I have a df where column A is either blank or has a string in it. I tried to write the if statement (all columns are strings) below. Basically, if there is something (any value) in df[A], then the new column value will be a concatenation of columns A, B and C. If there is no value in df[A], then

Error comparing dask date month with an integer

The dask map_partitions function in the code below has a dask date field where its month is compared to an integer. This comparison fails with the following error: ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all(). What is this error and how to fix it? A…

Type hint Pandas DataFrameGroupBy

How should I type hint in Python a pandas DataFrameGroupBy object? Should I just use pd.DataFrame as for normal pandas dataframes? I didn’t find any other solution atm Answer DataFrameGroupBy is a proper type in of itself. So if you’re writing a function which must specifically take a DataFrameGro…

Pandas pivot_table gives KeyError

(I’m fairly new to Python and completely new to Pandas.) I have software usage data in a tab-separated txt file like this: The DataFrame is imported correctly, and groupby methods like this work all right: However, when I’m trying to create a pivot table with this line: I get where the “key&…