Skip to content

Tag: list

Spread List of Lists to Sparks DF with PySpark?

I’m currently struggling with following issue: Let’s take following List of Lists: How can I create following Sparks DF out of it with one row per element of each sublist: The only way I’m getting this done is by processing this list to another list with for-loops, which basically then alrea…

Aggregating dicts within a list based on key value

I’m struggling to wrap my head around this one. I’ve got a list with multiple dictionaries that I would like to aggregate based on two values. Example code: I’m trying to aggregate dicts that have the same age and regex and adding the count key across all instances. Example output would be: …

Preventing reference re-use during deepcopy

Consider the following example: The first print outputs what I would expect because the same reference is duplicated in the list. However, the second print surprised me. I would have expected the deepcopy to end up with two independent references inside the copy list. Instead it maintains the property of a si…