I would like to get the value of the variables at each iteration of the optimization process when using Python GEKKO.
For example, the following code from the documentation (https://gekko.readthedocs.io/en/latest/quick_start.html#example) solves problem HS71:
JavaScript
x
13
13
1
from gekko import GEKKO2
import numpy as np3
4
m = GEKKO(remote=False)5
x = m.Array(m.Var, 4, value=1, lb=1, ub=5)6
x1, x2, x3, x4 = x # rename variables7
x2.value = 5; x3.value = 5 # change guess8
m.Equation(np.prod(x) >= 25) # prod>=259
m.Equation(m.sum([xi**2 for xi in x]) == 40) # sum=4010
m.Minimize(x1*x4*(x1 + x2 + x3) + x3) # objective11
m.solve()12
print(x, m.options.OBJFCNVAL)13
The output displayed in the console is the following:
JavaScript
1
83
83
1
----------------------------------------------------------------2
APMonitor, Version 1.0.03
APMonitor Optimization Suite4
----------------------------------------------------------------5
6
7
--------- APM Model Size ------------8
Each time step contains9
Objects : 110
Constants : 011
Variables : 1012
Intermediates: 013
Connections : 514
Equations : 715
Residuals : 716
17
Number of state variables: 1018
Number of total equations: - 719
Number of slack variables: - 120
---------------------------------------21
Degrees of freedom : 222
23
**********************************************24
Steady State Optimization with Interior Point Solver25
**********************************************26
27
28
Info: Exact Hessian29
******************************************************************************30
This program contains Ipopt, a library for large-scale nonlinear optimization.31
Ipopt is released as open source code under the Eclipse Public License (EPL).32
For more information visit http://projects.coin-or.org/Ipopt33
******************************************************************************34
This is Ipopt version 3.10.2, running with linear solver mumps.35
Number of nonzeros in equality constraint Jacobian: 1936
Number of nonzeros in inequality constraint Jacobian.: 037
Number of nonzeros in Lagrangian Hessian.: 1038
Total number of variables.: 1039
variables with only lower bounds: 140
variables with lower and upper bounds: 441
variables with only upper bounds: 042
Total number of equality constraints..: 743
Total number of inequality constraints: 044
inequality constraints with only lower bounds: 045
inequality constraints with lower and upper bounds: 046
inequality constraints with only upper bounds: 047
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls48
0 1.6109693e+001 4.00e+001 5.24e-001 0.0 0.00e+000 - 0.00e+000 0.00e+000 049
1 1.6896037e+001 7.28e-001 7.82e-001 -0.9 4.00e+001 - 1.00e+000 1.00e+000h 150
2 1.7121828e+001 1.66e-001 6.51e-001 -1.1 2.15e+000 - 8.23e-001 1.00e+000h 151
3 1.6954231e+001 1.60e-001 6.46e-002 -2.1 6.34e-001 - 1.00e+000 1.00e+000h 152
4 1.7008327e+001 1.68e-002 1.22e-002 -2.9 3.39e-001 - 9.94e-001 1.00e+000h 153
5 1.7013921e+001 3.15e-004 1.47e-004 -4.6 3.56e-002 - 1.00e+000 1.00e+000h 154
6 1.7014017e+001 2.51e-007 4.50e-007 -10.5 5.56e-004 - 9.99e-001 1.00e+000h 155
Number of Iterations.: 656
(scaled) (unscaled)57
Objective: 1.7014017181012623e+001 1.7014017181012623e+00158
Dual infeasibility: 4.5045555432827566e-007 4.5045555432827566e-00759
Constraint violation.: 2.5086595754315365e-007 2.5086595754315365e-00760
Complementarity: 4.8294565936715572e-008 4.8294565936715572e-00861
Overall NLP error.: 4.5045555432827566e-007 4.5045555432827566e-00762
Number of objective function evaluations = 763
Number of objective gradient evaluations = 764
Number of equality constraint evaluations = 765
Number of inequality constraint evaluations = 066
Number of equality constraint Jacobian evaluations = 767
Number of inequality constraint Jacobian evaluations = 068
Number of Lagrangian Hessian evaluations = 669
Total CPU secs in IPOPT (w/o function evaluations) = 0.01470
Total CPU secs in NLP function evaluations = 0.00071
EXIT: Optimal Solution Found.72
The solution was found.73
The final value of the objective function is 17.01401718101262374
75
---------------------------------------------------76
Solver : IPOPT (v3.12)77
Solution time : 0.021100000000000008 sec78
Objective : 17.01401718101262379
Successful solution80
---------------------------------------------------81
82
[[1.0000000339] [4.7429996271] [3.8211500196] [1.3794082228]] 17.01401718183
This includes the value of the objective function at each iteration, but I would also like to get the value of the variables (x) at each iteration.
Advertisement
Answer
The solvers don’t have call-backs to Python. One way to get the values at each iteration is to call the solver multiple times with a max_iter (maximum iteration) limit and report the values until the solver is successful. Use debug=0 to not raise an exception with an unsuccessful solution.
JavaScript
1
22
22
1
from gekko import GEKKO2
import numpy as np3
4
max_iter = 05
while True:6
m = GEKKO(remote=False)7
x = m.Array(m.Var, 4, value=1, lb=1, ub=5)8
x1, x2, x3, x4 = x # rename variables9
x2.value = 5; x3.value = 5 # change guess10
m.Equation(np.prod(x) >= 25) # prod>=2511
m.Equation(m.sum([xi**2 for xi in x]) == 40) # sum=4012
m.Minimize(x1*x4*(x1 + x2 + x3) + x3) # objective13
m.options.MAX_ITER = max_iter14
m.solve(disp=False,debug=0)15
print(max_iter,x, m.options.OBJFCNVAL)16
if m.options.APPSTATUS==1:17
# break loop with successful solution18
break19
else:20
# increment maximum iterations21
max_iter += 122
Here is the iteration summary with iteration number, variable values x, and objective function value. The objective function may get worse (higher) because the equations are not yet satisfied, and the solution may be infeasible.
JavaScript
1
8
1
0 [[1.00999999] [4.960000049] [4.960000049] [1.00999999]] 16.1096929182
1 [[1.1256700056] [4.4664784656] [4.2706611878] [1.1371887857]] 16.8960369953
2 [[1.1003048073] [4.6596383632] [3.9630056459] [1.2300025937]] 17.1218284014
3 [[1.0211678027] [4.7139849665] [3.8710712025] [1.3337143994]] 16.9542314635
4 [[1.0027660895] [4.740529129] [3.8261434464] [1.3737295983]] 17.0083270976
5 [[1.0000616664] [4.7429670916] [3.8212258627] [1.3792900699]] 17.0139207837
6 [[1.0000000339] [4.7429996271] [3.8211500196] [1.3794082228]] 17.0140171818
If you have a large model that you don’t want to redefine multiple times, then just put the m.solve() function in a loop with m.options.COLDSTART=1.
JavaScript
1
24
24
1
from gekko import GEKKO2
import numpy as np3
4
m = GEKKO(remote=False)5
x = m.Array(m.Var, 4, value=1, lb=1, ub=5)6
x1, x2, x3, x4 = x # rename variables7
x2.value = 5; x3.value = 5 # change guess8
m.Equation(np.prod(x) >= 25) # prod>=259
m.Equation(m.sum([xi**2 for xi in x]) == 40) # sum=4010
m.Minimize(x1*x4*(x1 + x2 + x3) + x3) # objective11
12
max_iter = 013
while True:14
m.options.MAX_ITER = max_iter15
m.options.COLDSTART=116
m.solve(disp=False,debug=0)17
print(max_iter,x, m.options.OBJFCNVAL)18
if m.options.APPSTATUS==1:19
# break loop with successful solution20
break21
else:22
# increment maximum iterations23
max_iter += 124