Skip to content
Advertisement

Dropping infinite values from dataframes in pandas?

How do I drop nan, inf, and -inf values from a DataFrame without resetting mode.use_inf_as_null?

Can I tell dropna to include inf in its definition of missing values so that the following works?

df.dropna(subset=["col1", "col2"], how="all")

Advertisement

Answer

First replace() infs with NaN:

df.replace([np.inf, -np.inf], np.nan, inplace=True)

and then drop NaNs via dropna():

df.dropna(subset=["col1", "col2"], how="all", inplace=True)

For example:

>>> df = pd.DataFrame({"col1": [1, np.inf, -np.inf], "col2": [2, 3, np.nan]})
>>> df
   col1  col2
0   1.0   2.0
1   inf   3.0
2  -inf   NaN

>>> df.replace([np.inf, -np.inf], np.nan, inplace=True)
>>> df
   col1  col2
0   1.0   2.0
1   NaN   3.0
2   NaN   NaN

>>> df.dropna(subset=["col1", "col2"], how="all", inplace=True)
>>> df
   col1  col2
0   1.0   2.0
1   NaN   3.0

The same method also works for Series.

Advertisement