Skip to content
Advertisement

Dataframes from dictionnaries in nested lists – Python

I have for example a nested list of 2 lists containing dictionaries like this : [[{},{},{},{}],[{},{},{},{}]]

[[{'Date': 'Dec 16 2020', 'Open': '37.00', 'High': '37.71', 'Low': '36.65', 'Close': '36.67', 'Adj Close': '36.67', 'Volume': '1840194'}, {'Date': 'Dec 15 2020', 'Open': '35.80', 'High': '36.88', 'Low': '35.63', 'Close': '36.79', 'Adj Close': '36.79', 'Volume': '1683816'}, {'Date': 'Dec 14 2020', 'Open': '35.48', 'High': '36.63', 'Low': '35.48', 'Close': '35.74', 'Adj Close': '35.74', 'Volume': '1808091'}, {'Date': 'Dec 11 2020', 'Open': '36.29', 'High': '36.63', 'Low': '34.96', 'Close': '35.08', 'Adj Close': '35.08', 'Volume': '1947461'}, {'Date': 'Dec 10 2020', 'Open': '36.51', 'High': '36.88', 'Low': '35.47', 'Close': '36.35', 'Adj Close': '36.35', 'Volume': '2207529'}],[{'Date': 'Sep 22 2020', : lose': '22.17', 'VolOpen': '22.00', 'High': '22.87', 'Low': '21.89', 'Close': '22.87', 'Adj Close': '22.87', 'Volume': '1997258'}, {'Date': 'Oct 05 2020', 'Open': '21.60', 'High': '
'Open': '21.01', 'High': '21.85', 'Low': '20.74', 'Close': '21.43', 'Adj Close': '21.43', 'Volume': '2064940'}, {'Date': 'Sep 21'O, 'Volume': '2064940 '20.72', 'Close': '21.19', 'Adj Close': '21.19', 'Volume': '2007302'}, {'Date': 'Oct 01 2020', 'Open': '22.35', 'High': '22.XMRClose': '20.88', 'Adj Close': '20.88', 'Volume': '3544256'}]]

I would like 2 dataframes something like :

          Date  Open  High  Low  Close Adj Close Volume 
   Dec 16 2020    37    37   36         36      1840194 
   Dec 15 2020     '     '    '          '            '
   Dec 14 2020     '     '    '          '            '
   Dec 13 2020     '     '    '          '            '

And I obviously can’t use that :

df = pd.DataFrame(data)

with data as a list of dictionaries.

Advertisement

Answer

The solution is simply to flatten your list of lists, then you can pass it to pandas normally

l = [
  [{'date':1, 'value':'1'}, {'date':2, 'value':'2'}],
  [{'date':3, 'value':'3'}, {'date':4, 'value':'4'}]
]
df = pd.DataFrame([item for sublist in l for item in sublist])

Will get you

   date   value
0     1     '1'
1     2     '2'
2     3     '3'
3     4     '4'

Source

While if you need a list of pandas DataFrames you can simply iterate your main list like so:

df_list = []
for x in l:
     df_list.append(pd.DataFrame(x))

Or simply

df_list = [pd.DataFrame(x) for x in l]
Advertisement