I have a huge pandas dataframe 150000 x 330 and I well find columns that have lists []
I have tried
df[['[]' in x for x in df.values]]
but it only returns column names.
df['FORM_SECTION.CONTRACT.FD_CONTRACT.CONTRACTING_AUTHORITY_INFORMATION.NAME_ADDRESSES_CONTACT_CONTRACT.SPECIFICATIONS_AND_ADDITIONAL_DOCUMENTS.CONTACT_DATA.E_MAILS.E_MAIL']
0 [dsfd@inventura.no]
1 [---@sivingtt.no]
2 None
3 None
4 None
...
152463 None
152464 None
152465 None
152466 None
152467 None
Name: FORM_SECTION.CONTRACT.FD_CONTRACT.CONTRACTING_AUTHORITY_INFORMATION.NAME_ADDRESSES_CONTACT_CONTRACT.SPECIFICATIONS_AND_ADDITIONAL_DOCUMENTS.CONTACT_DATA.E_MAILS.E_MAIL, Length: 152468, dtype: object
Advertisement
Answer
You can try:
df.applymap(lambda x: isinstance(x, list)).any()
Demo
data = {'Col1': [0, 1, 2], 'Col2': [['a', 'b'], ['a'], ['c']], 'Col3': [1.1, 2.2, 3.3]}
df = pd.DataFrame(data)
print(df)
Col1 Col2 Col3
0 0 [a, b] 1.1
1 1 [a] 2.2
2 2 [c] 3.3
df.applymap(lambda x: isinstance(x, list)).any()
Col1 False
Col2 True
Col3 False
dtype: bool
You can also find the list of columns with list as follows:
df.columns[df.applymap(lambda x: isinstance(x, list)).any()].tolist()
Output:
['Col2']