I am serializing a datatable from a http get and for performance reasons would prefer to serialize it in a Names, Values structure, so that the first element contains the column names, can json.loads deal with this, if not is there another json parser that will?
JavaScript
x
5
1
{
2
"Names" : ["summaryDate","count"],
3
"Values" : [["2020-01-15T00:00:00",10],["2020-01-16T00:00:00",12],["2020-01-17T00:00:00",16]]
4
}
5
(this reduces the size to 20% of a standard JSON stream with the field names repeated for each ‘row’)
Advertisement
Answer
I did some digging and found ijson. It lets you iterate over a json file and access its objects. you can build you data structur like this(i was lazy and used pd):
JavaScript
1
11
11
1
import ijson
2
import pandas as pd
3
f= open("testjson.txt",'r')
4
f2= open("testjson.txt",'r')
5
names=[]
6
values=[]
7
names = ijson.items(f, 'Names.item')
8
values = ijson.items(f2, 'Values.item')
9
10
pd.DataFrame(values,columns=list(names))
11