Skip to content
Advertisement

Tag: dataframe

How to extract json from nested column to dataframe

I’m pulling stock data from TD Ameritrade API and I want to store it in a DataFrame. From the API I get a nested JSON object and when I put it in a data frame I get 4 columns: Index, Candles, Empty, Symbol. However inside of candles is a dictionary that I want as separate columns in the dataframe (‘open’,’close’,…)

Does loc/iloc return a reference or a copy?

I am experiencing some problems while using .loc / .iloc as part of a loop. This is a simplified version of my code: basically: I initialize a dataframe with index and columns I populate each row of the dataframe with a for loop I find the index “i_max” finding the maximum value in column ‘A’ I save the row of

Append only matching columns to dataframe

I have a sort of ‘master’ dataframe that I’d like to append only matching columns from another dataframe to The problem is that when I use df.append(), It also appends the unmatched columns to df. But my desired output is to drop columns D and E since they are not a part of the original dataframe? Perhaps I need to

PySpark Dataframe melt columns into rows

As the subject describes, I have a PySpark Dataframe that I need to melt three columns into rows. Each column essentially represents a single fact in a category. The ultimate goal is to aggregate the data into a single total per category. There are tens of millions of rows in this dataframe, so I need a way to do the

Advertisement