I have this problem with shifting an entire row based on a column condition if it is NaN. if column 10 is NaN, shift the entire row 3 cells to the right.
initial dataframe:
6 7 8 9 10 11 12 0 2 1 12 3 2 12 4 1 6 2 4 7 NaN NaN NaN 2 4 1 4 3 6 3 2 3 NaN NaN NaN 4 5 3 2 4 3 1 3 2 1 2 6 5 5 2 6 1 NaN NaN NaN 6 NaN NaN NaN 4 5 3 2
I come up with this solution, but the first two columns are gone somehow.
df1_copy[df1_copy[10].isnull()] = df1_copy[df1_copy[10].isnull()].shift(3, axis=1)
Result:
6 7 8 9 10 11 12 0 2 1 12 3 2 12 4 1 NaN NaN NaN NaN NaN 4 7 2 4 1 4 3 6 3 2 3 NaN NaN NaN 4 5 3 2 4 3 1 3 2 1 2 6 5 NaN NaN NaN NaN NaN 6 1 6 NaN NaN NaN 4 5 3 2
Expected:
6 7 8 9 10 11 12 0 2 1 12 3 2 12 4 1 NaN NaN NaN 6 2 4 7 2 4 1 4 3 6 3 2 3 NaN NaN NaN 4 5 3 2 4 3 1 3 2 1 2 6 5 NaN NaN NaN 5 2 6 1 6 NaN NaN NaN 4 5 3 2
Although I tried a similar method for other conditions before this and it’s working, it doesn’t seem to apply to this one. any help would be so much appreciated, thank you. here is the csv
Advertisement
Answer
So turned out I was using Pandas version that is too old and it has a bug in the shift function, I come up with this workaround. Although it’s not elegant it works. Hopefully, someone can provide something better. I ended up just rearranging the order of the columns.
df_shift3 = df1_copy1.loc[df1_copy1.loc[:,10].isnull()][[10, 11, 12, 6, 7, 8, 9]] df_shift3.columns = [6, 7, 8, 9, 10, 11, 12] df1_copy1[df1_copy1.loc[:,10].isnull()] = df_shift3