I want to merge two dataframes to create a single time-series with two variables. I have a function that does this by iterating over each dataframe using itterows()… which is terribly slow and doesn’t take advantage of the vectorization that pandas and numpy provide… Would you be able to hel…
Tag: dataframe
Return row from a dataframe according to a list of priority values to search
I have a list of values in a sequence from most important to least important, if it doesn’t find a value, it searches for the next one and so on: Is there a more professional way to the same result or is this the correct model? Answer A possible solution involves turning your ‘market_name’ c…
Fastest way to filter csv using pandas and create a matrix
input dict I have large csv files in the below format basename_AM1.csv I have large csv files in the below format basename_AM1.csv Now I need to create a similarity dict like below for the given input_dict by searching/filter the csv files I have come up with the below logic but for an input_dict of 100 sampl…
Retrieving data from multiple parquet files into one dataframe (Python)
I want to start by saying this is the first time I work with Parquet files. I have a list of 2615 parquet files that I downloaded from an S3 bucket and I want to read them into one dataframe. They follow the same folder structure and I am putting an example below: /Forecasting/as_of_date=2022-02-01/type=full/…
Python pandas group by check if value changed then previous value
I’ve a problem with groupby function of pandas’s library. I’ve the following dataframe. id result date 400001 N 2020-07-03 400001 N 2021-09-09 400001 P 2021-10-27 400002 N 2020-07-03 400003 N 2020-06-30 400003 N 2022-04-27 400004 P 2020-06-30 400004 N 2022-04-27 I need to group by column …
Plotly graph is displaying as empty
I have a dataframe with the columns where data about number of users per certain cars in several weeks is shown. I made a pivot table and i’m trying to build a plot with the pivot table in plotly I get no errors but the graph shows no lines. The x-axis and the line names are correct but the values
how do I perform the following operation in python dataframe
below are my two dfs I want to replace the column ‘a’ of df with values in column ‘a’ of dd. Any empty rows are replaced by zero “only” for column ‘a’. All other columns of df remain unchanged. so column ‘a’ should contain 3,3,0,0,0 Answer This is pr…
How can I save multiple dataframes onto one excel file (as separate sheets) without this error occurring?
I have the following Python code: I’m reading the excel file which contains two sheets and then saving those sheets into a new excel file but unfortunately I’m receiving the following error: Any ideas on how I can fix this?. Thanks. Answer Change [0] to 0 in pd.read_excel(path, sheet_name = [0]) w…
How do I square a column from an Excel file with pandas?
I’ve read an Excel file into python using: and I’m trying to square the columns using: I keep getting the error: I’m fairly new to python. Is there any way to easily fix this? Answer Never use apply-lambda for straightforward mathematical operations it is orders of magnitude slower than usin…
Dataframe new columns to tell if the row contains column’s header text
2 columns dataframe as the first screenshot. I want to add new columns (by the contents in the Note column from the original dataframe) to tell if the Note column contains the new column’s header text. Example as the second screenshot. Some lines work for a few columns. When there are a lot of new colum…