Skip to content
Advertisement

how to make json.loads() read a json string with the column names as the first element

I am serializing a datatable from a http get and for performance reasons would prefer to serialize it in a Names, Values structure, so that the first element contains the column names, can json.loads deal with this, if not is there another json parser that will?

{
    "Names" : ["summaryDate","count"],
    "Values" : [["2020-01-15T00:00:00",10],["2020-01-16T00:00:00",12],["2020-01-17T00:00:00",16]]
}

(this reduces the size to 20% of a standard JSON stream with the field names repeated for each ‘row’)

Advertisement

Answer

I did some digging and found ijson. It lets you iterate over a json file and access its objects. you can build you data structur like this(i was lazy and used pd):

import ijson
import pandas as pd
f= open("testjson.txt",'r')
f2= open("testjson.txt",'r')
names=[]
values=[]
names = ijson.items(f, 'Names.item')
values = ijson.items(f2, 'Values.item')

pd.DataFrame(values,columns=list(names))
Advertisement