Skip to content
Advertisement

How to optimize time while converting list to dataframe?(Part II)

I didn’t get any proper answers to my previous question:

How to optimize time while converting list to dataframe?

Let me explain the example more:

Let’s consider the data frame more precisely as

JavaScript

I want the output dataframe when converted to csv as

JavaScript

The character PH,AG, AD,N should not be mapped. It can be any character.

This works fine while looping over this list and mapping and converting to a data frame. But, this took a lot of time for large datasets. The exact code for this process is in the previous question

JavaScript

Advertisement

Answer

Use:

JavaScript

First is defined dictionary for keys by first value in final list with all columns strating with strings:

JavaScript

Then filter DataFrames by lists from dictionary:

JavaScript

For PH is necessary melting for change format:

JavaScript

Last create same columns and join by concat with sorting for correct ordering:

JavaScript

Last create lists with different lengths by remove empty strings:

JavaScript

EDIT: For match Photo with digits is use regex, so instead startswith is used contains with joined values of lists by | for regex OR:

JavaScript

EDIT: Trick is add ^ to start of strings and $ to end of string for exact match values, then is necessary for correct working Photo+ ‘digit’:

JavaScript
User contributions licensed under: CC BY-SA
2 People found this is helpful
Advertisement