I am serializing a datatable from a http get and for performance reasons would prefer to serialize it in a Names, Values structure, so that the first element contains the column names, can json.loads deal with this, if not is there another json parser that will?
{ "Names" : ["summaryDate","count"], "Values" : [["2020-01-15T00:00:00",10],["2020-01-16T00:00:00",12],["2020-01-17T00:00:00",16]] }
(this reduces the size to 20% of a standard JSON stream with the field names repeated for each ‘row’)
Advertisement
Answer
I did some digging and found ijson. It lets you iterate over a json file and access its objects. you can build you data structur like this(i was lazy and used pd):
import ijson import pandas as pd f= open("testjson.txt",'r') f2= open("testjson.txt",'r') names=[] values=[] names = ijson.items(f, 'Names.item') values = ijson.items(f2, 'Values.item') pd.DataFrame(values,columns=list(names))