Plotting gradient approximation of a function

user_11171682 0
Viewed 67

I am attempting to graph the local approximation of a function using the gradient. Here is my code:

using Random
using LinearAlgebra
using Plots
using LaTeXStrings

Random.seed!(0)
x = rand(5,1)
v = rand(5,1)
alphas = LinRange(-1,1,100)

f(x) = log(sum(exp.(x)))
∇f(x) = (1/log(sum(exp.(x))))*[exp(x[1]),exp(x[2]),exp(x[3]), 
                               exp(x[4]), exp(x[5])]

plot(alphas, [f(x+alpha*v) for alpha in alphas], color = :black,
label = L"f(x + \alpha v)")
plot!(alphas, [f(x) + alpha*dot(∇f(x), v) for alpha in alphas], 
    color=:red, label=L"f(x) + \alpha \nabla f(x)^T v")
xlabel!(L"\alpha")

Gradient

From my understanding, the red line should be tangent to the curve at alpha = 0, however this is not the case. I am hoping to identify whether this is an issue with my calculation of the gradient, or if it is something incorrect I did in Julia.

The function in question is f(x) = log(Σe^x_i).

julia gradient
1 Answers

You use an incorrect formula for ∇f(x). It should be:

∇f(x) = (ex = exp.(x); ex / sum(ex)) # precalculate exp.(x) to avoid computing it twice

You could have also avoided calculating it manually by e.g. using the ForwardDiff.jl package and writing:

∇f(x) = ForwardDiff.gradient(f, x)
user_12695670
Related