I have the following two dataframes: and in df2 I have some missing timestamps compared to df1. I am able to find those timestamps using the following code: I want to populate those missing timestamps in df2 and fill in the values of the columns with the average value of the two previous rows. So the new df2 …
Tag: pandas
Filling empty months in pandas dataframe not working
I have a pandas DataFrame exclusively with dates: Using groupby I get a count for the number of monthly occurrences as seen below: (date is only used for plotting reasons). My issue is, come 09-2021 I have zero monthly counts and I want to obtain my gh dataframe such that the missing rows look something like:…
from dataframe to the body of Email automatically,several formatting issues: thousand separator, color(red for negative number and green for positive)
I have a dataframe look like this I wish to send it as the BODY of the Email with Outlook, it would be great to automate it in the future (as daily report without human intervention) but for the moment I just struggle to achieve some formatting how to get it directly to the body of Email or I have
Convert openpyxl to pandas
After doing some research I understood that pandas can give us better performace comparing to openpyxl. I’m trying to convert this code from openpyxl to pandas: Anyone can give me a suggestion about how to convert this code? Answer Here is the equivalent code using pandas: This code uses pandas to read …
how to transform dataframe into data set/object
I have a data set in a dataframe that’s almost 9 million rows and 30 columns. As the columns count up, the data becomes more specific thus leading the data in the first columns to be very repetitive. See example: park_code camp_ground parking_lot acad campground1 parking_lot1 acad campground1 parking_lo…
Slice Dataframe in sub-dataframes when specific string in column is found
Assume I have the dataframe df and I want to slice this in multiple dataframes and store each in a list (list_of_dfs). Each sub-dataframe should only contain the rows “Result”. One sub-dataframe starts, when in column “Point” the value “P1” and in column “X_Y” t…
Use fields of one dataframe as conditions to fill a field of another dataframe
I have 2 dataframes, the first is a small dataframe (df1) with information to use to fill a field (named Flag) of the second dataframe (df2). I need to write a function that uses each row of df1 as parameters to fill each row of df2 with a certain value (Y or N). df1 = type q25 q75 A 13
can’t replace duplicate values with new values in xlsx with pandas
I have an xlsx file containing too much data. however the data contains duplicate values in column named UniversalIDS which I wanted to replace it with a randomly generated IDS with Pandas. So far I’ve tried different scenarios which I googled but did not work. for example I tried this: also I tried oth…
Null/duplicate check in a column based on another column filter
I am working on pandas with the below requierment I need to check the below conditions if criteria is A, then m shouldn’t be null if criteria is B then n shouldn’t be null I wrote the below code for it or but both are not giving correct result I have also tried but this giving the columns without …
Create Pandas date column from fix starting date and offset days as integer colum
I have the following one-column Pandas data frame: Here the integers are number of days counted from a constant predefined date: Now I want to have a column with dates (calculated as START_DATE + the respective num_days) like this: I have tried this: but no success: Answer