I have a dataframe which I query and I want to get only unique values out of a certain column.
I tried to do that executing this code:
JavaScript
x
3
1
database = pd.read_csv(db_file, sep='t')
2
query = database.loc[database[db_specifications[0]].isin(elements)].drop_duplicates(subset=db_specification[1])
3
db_specification
is just a list containing two columns that I query.
Some of the values are NaN
and I don’t want to consider them duplicates of each other, how can I achieve that?
Advertisement
Answer
You can start by selecting all NaN
and then drop duplicate on the rest of the dataframe.
JavaScript
1
3
1
mask = data.isna().any()
2
data = pd.concat([data[mask], data[~mask]])
3