Lets say I have pandas data frame like this:
JavaScript
x
4
1
import numpy as np
2
import pandas as pd
3
my_df = pd.DataFrame(columns=['column 1', 'column 2', 'column 3'], data = [['17.1 g', np.nan, np.nan], [np.nan, '7.0 g', np.nan], [np.nan, '3.7 g', '0.7 g'], [np.nan, np.nan, '1.7 g'], ['1.1 g', np.nan, '1.0 g']])
4
What I would like to have is:
JavaScript
1
7
1
column1 column2 column3 column_new
2
0 17.1 g NaN NaN 17.1 g
3
1 NaN 7.0 g NaN 7.0 g
4
2 NaN 3.7 g 0.7 g 3.7 g
5
3 NaN NaN 1.7 g 1.7 g
6
4 1.1 g NaN 1.0 g 1.1 g
7
I tried using apply
and join
as below but not getting the output as I wish:
JavaScript
1
10
10
1
my_df['column_new'] = my_df[my_df.columns[0:]].apply(lambda x: ' '.join(x.dropna().astype(str)), axis=1)
2
my_df
3
4
column 1 column 2 column 3 column_new
5
0 17.1 g NaN NaN 17.1 g
6
1 NaN 7.0 g NaN 7.0 g
7
2 NaN 3.7 g 0.7 g 3.7 g 0.7 g
8
2 NaN NaN 1.7 g 1.7 g
9
3 1.1 g NaN 1.0 g 1.1 g 1.0 g
10
How can I get output that I want by modifying above code. Thank in advance.
Edit: I am looking solution using above code to get the expected output. Although codes in answers using bfill
and row-wise max
are nice, they don’t work well on the large data that I have.
Advertisement
Answer
We can using bfill
JavaScript
1
9
1
df['New']=df.bfill(axis=1).iloc[:,0]
2
df
3
column 1 column 2 column 3 New
4
0 17.1 g NaN NaN 17.1 g
5
1 NaN 7.0 g NaN 7.0 g
6
2 NaN 3.7 g 0.7 g 3.7 g
7
3 NaN NaN 1.7 g 1.7 g
8
4 1.1 g NaN 1.0 g 1.1 g
9