I have a dataset with multiple variables. I am trying to group these variables according to the end of the name of variable and calculate the mean of each group. Here is an example of my dataset: What I am trying to do is to group the variables that ends with the same number, e.g.: [AST_0-01, AST_1-01, AST_2-…
Tag: pandas
Python Pandas – Datetime gives wrong output only for certain dates
I have a dataframe with a column of dates in the format MMDDYYY. I want to convert the dates into the format YYYY-MM-DD. This works for most dates. But for dates starting with 1, the wrong output is given. In this example, the last 3 rows are wrong. There are many rows so I cannot hardcode the correct value. …
Pandas on Jupyter Notebook VS Code not displaying outputs properly
Does anyone know how to make Pandas outputs display properly in VS Code Jupyter Notebooks? As you can see in the image, the values for individual columns in the output of a Pandas DataFrame aren’t aligned (whereas they are aligned if you use Jupyter Notebooks directly on a web browser). Answer For the a…
Is pandas.read_spss misreading datetime into unix?
I have a sav file with a datetime column in %m/%d/%Y string format. When I read it in with pd.read_spss(), which doesn’t seem to have any datetime-related arguments, it ends up in what looks like unix time, except that the time would be a few centuries from now with unique values including 13778726400, …
Missing value Imputation based on regression in pandas
i want to inpute the missing data based on multivariate imputation, in the below-attached data sets, column A has some missing values, and Column A and Column B have the correlation factor of 0.70. So I want to use a regression kind of realationship so that it will build the relation between Column A and Colu…
Sorting values after transpose of dataframe
My dataframe is: 0 1 2 3 a 1091 347 2164 b 208 284 27647 c 0 8126 22 After transposing 0 a b c 1 1.091 208 0 2 347 284 8126 3 2161 27647 22 But I am getting an error like: I am trying to plot index values to a column but it should be in ascending
How to Eliminate for loop in Pandas Dataframe in filling each row values of a column based on multiple if,elif statements
Trying to get rid of for loop to speedup the execution in filling values in Column ‘C’ based on if, elif conditions involving multiple columns and rows. Not able to find a proper solution. tried applying np.where with conditions, choices and default values. But failed to get expected results as i …
save data to new worksheet, in existing workbook using python
created a new dataframe named df3 and saved to excel workbook named masterdata.xlsx, later created another dataframe named table when I try to write that dataframe to sheet2 of masterdata.xlsx, it overrighted the first created dataframe. masterdata.xlsx have two sheets, sheet1 and sheet2. sheet1 have some dat…
Python Pandas – How to compare values from two columns of a dataframe to another Dataframe columns?
I have two dataframes which I need to compare between two columns based on condition and print the output. For example: df1: df2: I want to write a code which compares ID column and date column between two dataframes is having a conditions like below, if “ID and date is matching from df1 to df2”: …
How to perform addition of two dataframe columns based on condition
I have two dataframes with different sizes and I am trying to perform an addition of values from a common column found in both dataframes based on matching values in another common column. My first dataframe looks like this: df1 df2 looks like this My expected results is How would I go about this please? My f…