Skip to content
Advertisement

pandas concat dictionary problem with too many indices

I am trying to concat a bunch of data frames which I placed into a dictionary. The following is my code:

for file in filepath:   
    df[i] = pd.read_csv(file)
    df[i]['Date'] = pd.to_datetime(df[i]['Date'].astype(str), format='%m/%d/%Y')        
    i += 1
    count += 1
    print(file)
    
new = pd.concat([d.set_index('Date') for d in df.values()], axis=1, ignore_index=True)

I get the following error: Shape of passed values is (1924, 55), indices imply (1904, 55) is there a way to avoid this?

Advertisement

Answer

Based on the comments, what you want is to join (merge) the dataframes not concat them. Without having the dfs handy (for your next question review this please) I cannot test the below, but roughly this is what you want to do

new = None
for file in filepath:   
    df = pd.read_csv(file)
    df['Date'] = pd.to_datetime(df['Date'].astype(str), format='%m/%d/%Y')        
    df.set_index('Date', inplace = True)
    if new is None:
        new = df
    else:
        new = new.join(df)
User contributions licensed under: CC BY-SA
6 People found this is helpful
Advertisement