Trying to compare to values in a pandas dataframe for max value

I’ve got a pandas dataframe, and I’m trying to fill a new column in the dataframe, which takes the maximum value of two values situated in another column of the dataframe, iteratively. I’m trying to build a loop to do this, and save time with computation as I realise I could probably do it with more lines of code.

for x in ((jac_input.index)):
    jac_output['Max Load'][x] = jac_input[['load'][x],['load'][x+1]].max()

However, I keep getting this error during the comparison

IndexError: list index out of range

Any ideas as to where I’m going wrong here? Any help would be appreciated!

Answer

Many things are wrong with your current code.

When you do ['abc'][x], x can only take the value 0 and this will return 'abc' as you are slicing a list. Not at all what you expect it to do (I imagine, slicing the Series).

For your code to be valid, you should do something like:

jac_input = pd.DataFrame({'load': [1,0,3,2,5,4]})
for x in jac_input.index:
    print(jac_input['load'].loc[x:x+1].max())

output:

1
3
3
5
5
4

Also, when assigning, if you use jac_output['Max Load'][x] = ... you will likely encounter a SettingWithCopyWarning. You should rather use loc: jac_outputLoc[x, 'Max Load'] = .

But you do not need all that, use vectorial code instead!

You can perform rolling on the reversed dataframe:

jac_output['Max Load'] = jac_input['load'][::-1].rolling(2, min_periods=1).max()[::-1]

Or using concat:

jac_output['Max Load'] = pd.concat([jac_input['load'], jac_input['load'].shift(-1)], axis=1).max(1)

output (without assignment):

0    1.0
1    3.0
2    3.0
3    5.0
4    5.0
5    4.0
dtype: float64