I have the following pandas dataframe df: timestamp version actual pred 2022-01-19 11:00:00.600 1 0 0 2022-01-19 11:00:00.800 1 0 1 2022-01-19 11:00:01.200 1 1 0 2022-01-19 11:00:01.800 1 0 0 2022-01-19 11:00:02.200 2 1 1 2022-01-19 11:00:02.600 2 0 0 2022-01-19 11:00:03.200 3 0 1 2022-01-19 11:00:03.600 3 0 …
Tag: datetime
Remove data time older than specific hours
I want to remove data from my dataframe older than say 2 hours from current time starting with 00 mins (datetime column is in index) when i use below code Current datetime: ’17-03-2022 17:05:00′ Issue: My code keeps all records in df from ’17-03-2022 15:05:00′ to ’17-03-2022 17:0…
pandas pivot_tables doesn’t work with date data (No numeric types to aggregate)
I have the following dataframe: I want to create pivot table to get the following table: I have tried to do this using pivot_table (pandas): but I get this error: DataError: No numeric types to aggregate I have read this post but it seems to be a bit different as I don’t want to change the columns and a…
Get cumulative sum in pandas
Context Datetime Campaign_name Status Open_time 2022-03-15 00:00 Funny_campaign Open 2022-03-15 01:00 Funny_campaign Continue 2022-03-15 02:00 Funny_campaign Continue 2022-03-15 03:00 Funny_campaign Continue 2022-03-15 04:00 Funny_campaign Close 2022-03-15 08:00 Funny_campaign Open 2022-03-15 09:00 Funny_camp…
Python Pandas – Datetime gives wrong output only for certain dates
I have a dataframe with a column of dates in the format MMDDYYY. I want to convert the dates into the format YYYY-MM-DD. This works for most dates. But for dates starting with 1, the wrong output is given. In this example, the last 3 rows are wrong. There are many rows so I cannot hardcode the correct value. …
Is pandas.read_spss misreading datetime into unix?
I have a sav file with a datetime column in %m/%d/%Y string format. When I read it in with pd.read_spss(), which doesn’t seem to have any datetime-related arguments, it ends up in what looks like unix time, except that the time would be a few centuries from now with unique values including 13778726400, …
How to remove datetime values in a row that are within a certain time relative to another row?
If I have a DataFrame as below: Letter Time 0 x 2021-01-01 14:00:00 1 y 2021-01-01 18:00:00 2 y 2021-01-03 14:00:00 How would I delete a row if a value in the Time column(datetime) is within say 14 hours from the time in the row above? I’ve tried using: but I get KeyError 1 in relation to the line if
How to retrieve the week number of the year starting at 1 Jan?
So I have a shell script which returns me the week number using an SQL query : This query returns 1. (Starting at 1st Jan) Python code: I have tried the following however it returns 52 rather than 1 : Week starting from Mon-Sun Is there another way in which I can get the week number of year based on
Pandas: Check each row for condition and insert row below if condition is met
this is my first question here as I really couldn’t figure it out with related answers: I have a list of dataframes “df_list”, for each user I have a dataframe which basically looks like: Data: I would like to go through all the dataframes in my df_list and inside each df I would like to add…
CET timezone strings to datetime
I have a data frame that has columns looking similar to this. This data is imported from SQL into a Pandas data frame, and when I print out the dtypes I can see that the date_cet column is object. Since I need it further on, I want to convert it to a datetime object. However, the stuff I’ve tried just