Goal
If sub-column min
equals to sub-column max
and if min
and max
sub-column do not equal to each other in any of the column (ao, his, cyp1a2s, cyp3a4s in this case), drop the row.
Example
JavaScript
x
15
15
1
arrays = [np.array(['ao', 'ao', 'hia', 'hia', 'cyp1a2s', 'cyp1a2s', 'cyp3a4s', 'cyp3a4s']),
2
np.array(['min', 'max', 'min', 'max', 'min', 'max', 'min', 'max'])]
3
tuples = list(zip(*arrays))
4
index = pd.MultiIndex.from_tuples(tuples, names=['',''])
5
df = pd.DataFrame(np.array([[1, 1, 0, 0, float('nan'), float('nan'), 0, 0],
6
[1, 1, 0, 0, float('nan'), 1, 0, 0],
7
[0, 2, 0, 0, float('nan'), float('nan'), 1, 1],]), index=['1', '2', '3'], columns=index)
8
df
9
10
ao hia cyp1a2s cyp3a4s
11
min max min max min max min max
12
1 1.0 1.0 0.0 0.0 NaN NaN 0.0 0.0
13
2 1.0 1.0 0.0 0.0 NaN 1.0 0.0 0.0
14
3 0.0 2.0 0.0 0.0 NaN NaN 1.0 1.0
15
Want
JavaScript
1
7
1
df = pd.DataFrame(np.array([[1, 1, 0, 0, float('nan'), float('nan'), 0, 0]]), index=['1'], columns=index)
2
df
3
4
ao hia cyp1a2s cyp3a4s
5
min max min max min max min max
6
1 1.0 1.0 0.0 0.0 NaN NaN 0.0 0.0
7
Attempt
JavaScript
1
4
1
df.apply(lambda x: x['min'].map(str) == x['max'].map(str), axis=1)
2
3
KeyError: ('min', 'occurred at index 1')
4
Note
The actual dataframe has 50+ columns.
Advertisement
Answer
Use DataFrame.xs
for DataFrame
by second levels of MultiIndex
, replace NaN
s:
JavaScript
1
3
1
df1 = df.xs('min', axis=1, level=1).fillna('nan')
2
df2 = df.xs('max', axis=1, level=1).fillna('nan')
3
Or convert data to strings:
JavaScript
1
3
1
df1 = df.xs('min', axis=1, level=1).astype('str')
2
df2 = df.xs('max', axis=1, level=1).astype('str')
3
Compare Dataframes by DataFrame.eq
and test if all True
s by DataFrame.all
and last filter by boolean indexing
:
JavaScript
1
6
1
df = df[df1.eq(df2).all(axis=1)]
2
print (df)
3
ao hia cyp1a2s cyp3a4s
4
min max min max min max min max
5
1 1.0 1.0 0.0 0.0 NaN NaN 0.0 0.0
6