I want to use the python implementation of the optimizer ImFil (Implicit Filtering) using the sckit-quant package. On this website there is a extensive manual on the matlab code on which the python implementation is based on.
The code runs but the output is not as expected. Here is a toy code to demonstrate my problem:
import numpy as np
from skquant.opt import minimize
def f(x):
ifail=0 #function never fails
icount=1 #each call weighted the same
return x[0]*x[0] + x[1]*x[1] , ifail, icount
bounds=np.array([[-10,10], [-10,10]])
x0=np.array([5,5]) #initial point
budget=20
res, hist=minimize(f, x0, bounds, budget, method='ImFil')
print('optimization result: ', res)
print('optimization history: ', hist, ' shape: ', hist.shape)
My expectation: The history of the optimization 'hist' should be a (20)x(7) array, according to the documentation:
histout = iteration history, updated after each nonlinear iteration
= (number of iterations )x(N+5) array, the columns are
[fcount, fval, norm(sgrad), norm(step), iarm, xval]
fcount = cumulative function evals
fval = current function value
norm(sgrad) = current projected stencil grad norm
norm(step) = norm of last step
iarm = line searches within current iteration
= -1 means first iterate at a new scale
xval = transpose of the current iteration
What I instead get:
optimization result: Optimal value: 1.57772e-29, at parameters: [ 3.55271368e-15 -1.77635684e-15]
optimization history: [[ 5.00000000e+01 5.00000000e+00 5.00000000e+00]
[ 5.00000000e+01 -5.00000000e+00 5.00000000e+00]
[ 5.00000000e+01 5.00000000e+00 -5.00000000e+00]
[ 1.25000000e+02 1.00000000e+01 5.00000000e+00]
[ 1.25000000e+02 5.00000000e+00 1.00000000e+01]
[ 2.50000000e+01 0.00000000e+00 5.00000000e+00]
[ 2.50000000e+01 5.00000000e+00 0.00000000e+00]
[ 2.00000000e+02 -1.00000000e+01 -1.00000000e+01]
[ 2.00000000e+02 -1.00000000e+01 -1.00000000e+01]
[ 2.94733047e+01 -3.83883476e+00 -3.83883476e+00]
[ 1.60849571e+01 1.16116524e+00 -3.83883476e+00]
[ 1.60849571e+01 -3.83883476e+00 1.16116524e+00]
[ 9.28616524e+01 -8.83883476e+00 -3.83883476e+00]
[ 9.28616524e+01 -3.83883476e+00 -8.83883476e+00]
[ 1.57772181e-29 3.55271368e-15 -1.77635684e-15]
[ 2.50000000e+01 5.00000000e+00 -1.77635684e-15]
[ 2.50000000e+01 3.55271368e-15 5.00000000e+00]
[ 2.50000000e+01 -5.00000000e+00 -1.77635684e-15]
[ 2.50000000e+01 3.55271368e-15 -5.00000000e+00]
[ 6.25000000e+00 2.50000000e+00 -1.77635684e-15]
[ 6.25000000e+00 3.55271368e-15 2.50000000e+00]
[ 6.25000000e+00 -2.50000000e+00 -1.77635684e-15]
[ 6.25000000e+00 3.55271368e-15 -2.50000000e+00]] shape: (23, 3)
The optimization results are reasonable, but what is it that is returned in the history?
My real life example: Similarly, my code runs and the result of the optimization is reasonable, but the returned history is not what I would expect: In my case I have a 12-dimensional input to my objective function and a budget of 200. I thus would expect histout to have dimensions (200)x(12+5). What I get is (209)x(13). The 209 are ok. A bit overhead is expected because the current number of calls to the evaluation and the budget are only in compared in larger intervals. But the 13 irritates me. Also, there is nothing which could be interpreted as fcount, the number of objective function evaluations, which I expect to be an integer in histout. From the toy problem it looks like the first column in each row is the f(x), where x is the second and third column. The result of the optimization then is the row with the minimal value in the first column. But this is not the case for my actual problem. There values arise in the first column which are smaller than the returned result of the optimization.
Edit: I found out, that the option 'standalone' is hardcoded on False in the code (in skquant/opt/_init.py). When standalone is False the returned history is fval,xval. This does explain the format of the output. The open problem remains: What is fval? The documentation says 'fval, the current value of the objective function'. What exactly does this mean? I'm asking for details, because in my returned array there are values that are outside the range of my function. So, is it some extrapolation or so? I did not really understand how ImFil actually works. Also, what is returned as result is not the minimum of the fval array. How is the result determined?