Skip to content
Advertisement

Find colums that contains lists

I have a huge pandas dataframe 150000 x 330 and I well find columns that have lists []

I have tried

df[['[]' in x for x in df.values]] 

but it only returns column names.

df['FORM_SECTION.CONTRACT.FD_CONTRACT.CONTRACTING_AUTHORITY_INFORMATION.NAME_ADDRESSES_CONTACT_CONTRACT.SPECIFICATIONS_AND_ADDITIONAL_DOCUMENTS.CONTACT_DATA.E_MAILS.E_MAIL'] 

0         [dsfd@inventura.no]
1                        [---@sivingtt.no]
2                                     None
3                                     None
4                                     None
                        ...               
152463                                None
152464                                None
152465                                None
152466                                None
152467                                None
Name: FORM_SECTION.CONTRACT.FD_CONTRACT.CONTRACTING_AUTHORITY_INFORMATION.NAME_ADDRESSES_CONTACT_CONTRACT.SPECIFICATIONS_AND_ADDITIONAL_DOCUMENTS.CONTACT_DATA.E_MAILS.E_MAIL, Length: 152468, dtype: object

Advertisement

Answer

You can try:

df.applymap(lambda x: isinstance(x, list)).any()

Demo

data = {'Col1': [0, 1, 2], 'Col2': [['a', 'b'], ['a'], ['c']], 'Col3': [1.1, 2.2, 3.3]}
df = pd.DataFrame(data)

print(df)

   Col1    Col2  Col3
0     0  [a, b]   1.1
1     1     [a]   2.2
2     2     [c]   3.3

df.applymap(lambda x: isinstance(x, list)).any()

Col1    False
Col2     True
Col3    False
dtype: bool

You can also find the list of columns with list as follows:

df.columns[df.applymap(lambda x: isinstance(x, list)).any()].tolist()

Output:

['Col2']
User contributions licensed under: CC BY-SA
3 People found this is helpful
Advertisement