Skip to content
Advertisement

Create a new list of dictionary from the index in dataframe Python with the fastest way

I have a ~200mil data in dictionary index_data:

JavaScript

Key is a value in CustId and Value is an index of CustID in df_data:

I have a DataFrame df_data:

JavaScript

NOTE: If CustID is duplicate, only column Score have different data in each row

I want to create a new list of dict(Total_Score is an avg Score of each CustID, Number is Number2 divide Number1):

JavaScript

My solution is to loop my dictionary and use multiprocessing from multiprocessing import Process, Manager

JavaScript

But It’s not working. Performance is slow and higher memory? Is my setup multiprocess is right? Have another way to do that?

Advertisement

Answer

JavaScript
User contributions licensed under: CC BY-SA
8 People found this is helpful
Advertisement