Why wont this softmax function I have written in Julia change the input data?

user_15006954 0
Viewed 61

I have written the softmax function in Julia. It performs softmax on a matrix row by row and changes the matrix. However, when I call the function in the REPL, it has no effect on the matrix. I do not understand why this is happening and I would really like an explanation.

"""
the following function performs softmax on a design matrix row by row
inputs: X:mxn matrix
output: none
"""
function softmax!(X::Array{Float64,2})
    X = exp.(X)
    for i = 1:size(X,1)
        rowsum = sum(X[i,:])
        X[i,:] /= rowsum
    end
end

And here is an example call in the REPL:

julia> a = rand(2,5)
2×5 Array{Float64,2}:
 0.069014  0.265159  0.489641  0.455672  0.0489479
 0.274386  0.935308  0.41976   0.509558  0.234294

julia> softmax!(a)

julia> a
2×5 Array{Float64,2}:
 0.069014  0.265159  0.489641  0.455672  0.0489479
 0.274386  0.935308  0.41976   0.509558  0.234294

As you can see, there is no change in the matrix. What is extremely strange is if I hardcode what is inside the function in the REPL, I get the intended effect.

julia> a = exp.(a)
2×5 Array{Float64,2}:
 1.07145  1.30364  1.63173  1.57723  1.05017
 1.31572  2.548    1.5216   1.66456  1.26402

julia>     for i = 1:size(a,1)
               rowsum = sum(a[i,:])
               a[i,:] /= rowsum
           end

julia> a
2×5 Array{Float64,2}:
 0.161504  0.196502  0.245957  0.237742  0.158295
 0.158256  0.306475  0.183019  0.200214  0.152037

I know there is something I am not understanding but I am at a loss for what that might be. Any help will be much appreciated :)

julia neural-network
1 Answers

you need to replace X = exp.(X) with X .= exp.(X). Julia is pass by sharing, so when you say X = exp.(X) from that point forward, the X in your function and the X you passed in refer to different memory.

Note also that this method is fairly inefficient since julia uses column major matrices. If you transpose your problem and write,

function softmax!(X::Array{Float64,2})
    X .= exp.(X)
    for j = 1:size(X,2)
        @views rowsum = sum(X[:,j])
        X[:,j] .*= 1/rowsum
    end
end

it will be about 2x faster

user_51413280
Related