I have a dataframe in this format: … though my dataframe is much larger, with more than 500 hundred IDs. I want to convert this 2D – dataframe into a 3D array in this format (num_time_samples, value, ID). Essentially I would like to have one 2D-array for every unique ID. I plan on using the value …
Tag: dataframe
How to check if the value is present in df for multiple columns and append if not present
I have multiple df and i want to compare one with the other if the values are same. And if values are not same i want to append the value to the first data frame. For example: data frame 1 A B C Mm hh jj Kk ll gg Data frame 2 | A | B | C | |—|—|—|
Create pandas dataframe on column name conditions
Python newbie attempting a complex pandas dataframe logic I have multiple dataframes I need to join but I’ll show two below for the example. The dataframe have duplicate columns labelled with suffix ‘_duplicate’. I need to replicate the row instead of having the duplicate column as seen belo…
Python: Pivot dataframe to introduce new columns
I have the following dataframe I would like to pivot it to get the following: Answer
Transformation of time series data arranged as value per column into Pandas 2D Dataframe
I have data where 13000 high frequency time-series data points are mapped as one column per point assosiated with one timestamp, when the data was measured. I read this data from an influxDB and get a Pandas.DataFrame structured as follows: I know the sampling frequency of the high frequency data (10 MHz), so…
Python Pandas, Running Sum, based on previous rows value and grouped
I have a pandas dataframe along these lines, based on where a customer service case sits before being closed. Every time the case is edited and audit trial is captured. I want to generate a counter for each time the Department of a case changes from the department it was previously in. ID Department Start Dat…
Summarizing a pandas DataFrame by group using a custom function results in wrong output
I have a pandas DataFrame that I want to summarize by group, using a custom function that resolves to a boolean value. Consider the following data. df describes 4 people, and for each person the fruits they like. I want to summarize this table to find the people who like both apricot and apple. In other words…
Use list items as column seperators pd.read_fwf
I have text files containing tables which I want to put into a dataframe. Per file the column headers are the same, but the width is different depending on the content (because they contain names of different lengths for example). So far I managed to get the index of the first character of the header, so I kn…
How to join sentences inside data frame column?
I have create a sample data frame and it contains a column Called ‘Body’ and the content of it as below. ‘[‘Former India captains should have shown the maturity to sort out the matter privately’, ‘When egos clash, the results are often disastrous. Ugly too. And the row tend…
Pandas Aggregate Daily Data to Monthly Timeseries
I have a time series that looks like this (below) And I want to resample it monthly, so it has 2019-10 is equal to the average of all the values of october, November is the average of all the PTS values for November, etc. However, when i use the pd.resample(‘M’).mean() method, if the final day for…