Run mapslices in parallel

user_14313394 0
Viewed 186

I use mapslices() to apply the same expression for each row of some array:

function f(x)
      ...
end

a = rand(Normal(), 10000, 50)

b = mapslices(x -> f(x), a, dims = [2])

Is there some elegant way to run a similar procedure in parallel using all the CPU cores? Something similar to how in R you can replace apply() with future_apply(), so that everything is automatically parallelized. Or mapslices() are not suitable here at all and is needed to use some other structure?

julia parallel-processing
1 Answers

You could write this usin @distributed loop.

Consider the code

using Distributed
addprocs(4)
a=collect(reshape(1:12,3,4))

And now:

julia> @distributed (vcat) for col in 1:size(a,2)
           sum(@view(a[:,col]))+1000*myid()
       end
4-element Array{Int64,1}:
 2006
 3015
 4024
 5033

It can be seen that the first column was summed using the first process, second using the second process etc.

Depending on the scenario you might consider using also SharedArrays for storing the data. Finally you might also have a look at pmap and preduce in the Distributed package. Depending on the scenario they might be a good fit.

user_99577100
Related