Julia minimize simple scalar function

user_6435921 0
Viewed 308

How do I minimize a simple scalar function in Julia using Newton's method? (or any other suitable optimization scheme)

using Optim

# Function to optimize
function g(x)
    return x^2
end


x0 = 2.0  # Initial value
optimize(g, x0, Newton())

The above doesn't seem to work and returns

ERROR: MethodError: no method matching optimize(::typeof(g), ::Float64, ::Newton{LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64,Base.RefValue{Bool}}})
julia
2 Answers

The optimize function expects an interval, not a starting point:

optimize(g, -10, 10)

returns

Results of Optimization Algorithm
 * Algorithm: Brent's Method
 * Search Interval: [-10.000000, 10.000000]
 * Minimizer: 0.000000e+00
 * Minimum: 0.000000e+00
 * Iterations: 5
 * Convergence: max(|x - x_upper|, |x - x_lower|) <= 2*(1.5e-08*|x|+2.2e-16): true
 * Objective Function Calls: 6

Concerning available methods I have not read the doc, but you can directly have a look at the source code using the @edit macro:

@edit optimize(g, -10, 10)

Browsing the source you will see:

function optimize(f,
    lower::Union{Integer, Real},
    upper::Union{Integer, Real},
    method::Union{Brent, GoldenSection};
    kwargs...)
     
    T = promote_type(typeof(lower/1), typeof(upper/1))
    optimize(f,
             T(lower),
             T(upper),
             method;
             kwargs...)
end

Hence I think that you have only two methods for unidimensional minimization: Brent and GoldenSection.

By example you can try:

julia> optimize(g, -10, 10, GoldenSection())
Results of Optimization Algorithm
 * Algorithm: Golden Section Search
 * Search Interval: [-10.000000, 10.000000]
 * Minimizer: 1.110871e-16
 * Minimum: 1.234035e-32
 * Iterations: 79
 * Convergence: max(|x - x_upper|, |x - x_lower|) <= 2*(1.5e-08*|x|+2.2e-16): true
 * Objective Function Calls: 80
user_20010170

Optim is designed for vector problems and not scalar ones like in your example. You can adjust the example to be a vector-problem with one variable though:

julia> using Optim

julia> function g(x)        # <- g accepts x as a vector
           return x[1]^2
       end

julia> x0 = [2.0]           # <- Make this a vector
1-element Vector{Float64}:
 2.0

julia> optimize(g, x0, Newton())
 * Status: success
 * Candidate solution
    Final objective value:     0.000000e+00
user_50871360
Related