from Octave's fminunc to Julia's optimize with gradient function with parameters

user_14507208 0
Viewed 140

I am passing my code from octave to julia, in this case a logistic regression. The gradient function takes in addition to the initial theta, X with my features and Y with the sought values.

in Octave works

function [J, grad] = costFunction(theta, X, y)
options = optimset('GradObj', 'on', 'MaxIter', 400);
[theta, cost] = fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);

on Julia first try

optimize(t->CostFunction(t, X1, y), initial_theta, BFGS())

MethodError: no method matching -(::Tuple{Float64,Array{Float64,2}}, ::Tuple{Float64,Array{Float64,2}})

so I separated the function in two: Cost and Gradient

function CostFunction2(theta, X, y)
J = 0;
#m = length(y);
m = size(y,1);
grad = zeros(size(theta));
J = 1/m * sum( (-y .* log.(sigmoid(X*theta))) - ((1.0 .- y) .* log.(1.0 .- sigmoid(X*theta))) );
return J;
end


function Gradient2(X, y, theta)
grad = zeros(size(theta));
grad = (1/m) .* (sum((sigmoid(X*theta).-y) .* X, dims=1))';
return grad;
end

I put only the cost function and it worked, but I don't have the last values of theta. I don't know how to get it

optimize(t->CostFunction2(t, X1, y), initial_theta, BFGS())

I tried this but it didn't work, and I can't find any reference that says how or brings an example

optimize(t->CostFunction2(t, X1, y), Gradient2(X, y, t), initial_theta, BFGS())

UndefVarError: t not defined

How can I obtain the obtained theta values? and How can I include my own gradient function with various parameters?

I hope you can help me, thank you very much

julia gradient octave matlab optimization
1 Answers

I have cleaned typos in your code, added missing definitions, and removed unnecessary operations.

After these changes the following works:

using DelimitedFiles

Data = readdlm("ex2data1.txt", ',', Float64)
X = Data[:,1:2]
y = Data[:,3]
m, n = size(X)
X = [ones(m) X]
initial_theta = zeros(n + 1)

sigmoid(x) = 1 / (1 + exp(-x))

function CostFunction2(theta, X, y)
    Xt = X*theta # do this step outside as it is repeated twice otherwise
    return sum(-y .* log.(sigmoid.(Xt)) .- (1.0 .- y) .* log.(1.0 .- sigmoid.(Xt)))
end

using Optim
optimize(t->CostFunction2(t, X, Y), initial_theta, BFGS())
user_12695670
Related