I have a dictionary that is a list of dataframes that have all the same columns and data structure. I am wanting to essentially ‘union’ all of these into a single dataframe again, where the dictionary keys are converted into another column: df_list{}
JavaScript
x
8
1
{'A' : col1 col2 col3
2
001 val1 val2 val3
3
002 val3 val4 val5
4
5
'B' : col1 col2 col3
6
001 val1 val2 val3
7
002 val3 val4 val5
8
…and so on
but am wanting:
JavaScript
1
6
1
key Col1 Col2 Col3
2
A val1 val2 val3
3
A val4 val5 val6
4
B val1 val2 val3
5
B val4 val5 val6
6
I tried using pd.DataFrame.from_dict() but either I am not using it right or I need something else..
JavaScript
1
2
1
final_df = pd.DataFrame.from_dict(df_list)
2
but get: ValueError: If using all scalar values, you must pass an index
when I try passing the index, I get one column back vs a dataframe.
Advertisement
Answer
This should do it:
JavaScript
1
24
24
1
import pandas as pd
2
3
df1 = pd.DataFrame({
4
"col1":['val1','val3'],
5
"col2":['val2','val3'],
6
"col3":['val3','val5']
7
})
8
9
10
df2 = pd.DataFrame({
11
"col1":['val7','val3'],
12
"col2":['val2','val3'],
13
"col3":['val3','val5']
14
})
15
16
pd_dct = {"A": df1, "B": df2}
17
18
# adding the key in
19
for key in pd_dct.keys():
20
pd_dct[key]['key'] = key
21
22
# concatenating the DataFrames
23
df = pd.concat(pd_dct.values())
24
Alternatively, we can also do this in one line with:
JavaScript
1
2
1
pd.concat(pd_dct, axis=0).reset_index(level=0).rename({'level_0':'key'}, axis=1)
2