I have a table with top 3 reasons (Table 1) and another table with the category it belongs to for each variable (Table 2). I am trying to match the category bins into the reason table like in table 3. Answer Approach index two data frames in way that works with join() then it’s a pd.concat() of each of …
Tag: dataframe
Resample df to smaller time steps and average the counts
I have a dataframe containing counts over time periods (rainfall in periods of 3 hours), something like this: I need to upsample the dataframe into time periods of 1 hour and I would like to average out the counts for the rain, so that there are no NaNs and the total sum of rain remains the same, means this i…
Create DF Columns Based on Second DDF
I have 2 dataframes with different columns: I would like to add the missing columns for the 2 dataframes – so each one will have each own columns + the other DFs columns (without column “number”). And the new columns will have initial number for our choice (let’s say 0). So the final o…
Transpose dataframe based on column list
I have a dataframe in the following structure: I would like to transpose – create columns from the names in cNames. But I can’t manage to achieve this with transpose because I want a column for each value in the list. The needed output: How can I achieve this result? Thanks! The code to create the…
Resampling with Pandas spline gives strange results. Do I misunderstand, even though the time matches?
I take my dataframe, which is in seconds, and resample it over a period of every n seconds, to properly align all values with even spacing. The seconds are parsed correctly, but the output results are strange, so maybe I’m completely misunderstanding what exactly is being splined over? Gives So where di…
pandas groupby dataframes, calculate diffs between consecutive rows
Using pandas, I open some csv files in a loop and set the index to the cycleID column, except the cycleID column is not unique. See below: This prints the 2 columns (cycleID and mean) of the dataframe I am interested in for further computations: The objective is to use the rows corresponding to the same cycle…
Repeat pattern using python regex
Well, I’m cleaning a dataset, using Pandas. I have a column called “Country”, where different rows could have numbers or other information into parenthesis and I have to remove them, for example: Australia1, Perú (country), 3Costa Rica, etc. To do this, I’m getting the column and I mak…
how to properly apply a vector based function to a pandas dataframe column?
I am trying to apply a function that returns an specific date in an specific format, however I am struggling to apply this function to a new pandas dataframe column. Here’s what I got so far: The next error arises: KeyError: datetime.datetime(2021, 2, 1, 0, 0) Expected output could be a pandas dataframe…
Filter Pandas MultiIndex over all First Levels Columns
Trying to find a way of efficiently filtering all entries under both top level columns based on a filter defined for only one of the top level columns. Best explained with the example below and desired output. Example DataFrame Create filter for multiindex dataframe Desired output: Answer You can reshape for …
Enumerate rows in each group starting from one
I have a dataframe (which is sorted on date, date column is not included in the example for simplicity) that looks like this: I want to create a new column that counts the occurrence of each value in the letters column, increasing 1 by 1 as the value occurs in the letters column. The data frame I want to reac…