Julia: spawnat always runs on the same threads even if asked to run for multipleprocesses

user_14830573 0
Viewed 146

I am trying to run the same simulation on different threads in Julia. My computer has 4 cores. I checked that Threads.nthreads() =4, and added processes so that nprocs() = 4. Now I am trying to run the same simulation:simulation() in parallel.

My code is

@spawnat 1 simulation() 
@spawnat 2 simulation()
@spawnat 3 simulation()
@spawnat 4 simulation()

the simulation function displays as well on which threads it runs thanks to a println(Threads.threadid())

The problem is that all processes run on the same thread - thread number 1.

I thought that creating workers and using the @spawnat macro should be enough for the simulation to run on all threads.

Would you have any idea on what to do, and what I did wrong? Thanks in advance

julia multithreading
1 Answers

I think you might be confusing multi-threading and multi-processing? From the docs:

help?> @spawnat
  @spawnat p expr

  Create a closure around an expression and run the closure asynchronously on process p

So your code is executed on one of theworker processes you created with addprocs. Whether or not the code that is being executed uses more than one thread though depends on whether the code itself is multithreaded, so e.g. is using the @threads macro.

The Julia manual discusses multi-threading here and multi-processing here.

EDIT to add: there's a discussion on the Julia Discourse here which includes an example of running multi-threaded worker processes - to do so you need to set an environment variable to ensure worker processes are started with multiple threads.

user_24998920
Related