Random number seeds for Julia cluster computing

user_3979750 0
Viewed 257

I have a Julia code, and I want to submit this code to a remote computing cluster via running a large number of jobs in parallel (i.e., around 10,000 jobs in parallel). The way this code works is that, the main function (call it "main.jl") calls another function (call it "generator.jl") which utilizes random numbers such as rand(Float64) and so on. I submit main.jl via a bash file, and I run N jobs in parallel by including

#PBS -t 1-N

I want to make sure I have a different random number generator for each of the N job submissions, but I'm not sure how to do this. I was thinking of setting a random seed based upon the environment variable; i.e., by setting

@everywhere import Random.Random
@everywhere using Random.Random

Random.seed!(ENV_VAR)

in main.jl. However, I'm not sure how to get the environment variable ENV_VAR. In MATLAB, I know I can get this via

NUM = getenv( 'PBS_ARRAYID' )

But I don't know how to do this in Julia. If I manage to set this new random seed in main.jl, will this generate a different random seed every time the bash script submits main.jl to the cluster? In a similar vein, do I even need to do this in Julia, considering the Julia RNG uses MersenneTwister?

Just in case it matters, I have been using Julia 1.5.1 on the remote machine.

julia random-seed random
1 Answers

There are two issues here:

  1. Getting the job number
  2. Using the job number in random number generation.

Each of those issues has two solutions - one more and other less elegant but also OK.

Ad.1. For managing job numbers consider using PBS together with ClusterManagers.jl. There is a command there addprocs_pbs(np::Integer; qsub_flags="") that will make it possible to manage the run numbers and orchestrate the cluster from within Julia. In many scenarios you will find this approach more comfortable. Here you can use for seeding the random number generator (more on that later) myid() that returns the worker number. Anyway most likely you will in this scenario run your computations using the @distributed loop and you can use that value for seeding RNG. If you are rather orchestrating array jobs externally via bash scripts then perhaps the best thing is to pass the job number via a parameter to the Julia process and read it from ARGS variable or have a setup bash script that exports an environmental parameter that can be read from ENV variable.

Ad.2. There are two approaches here. Firstly you can simply create a new MersseneTwister at each worker and then use it in the streams. For an example (here I assume that you use some variable jobid):

using Random
rnd = MersenneTwister(jobid)
rand(rnd, 4)

This is basically OK and the random streams are known not be correlated. However, you might be worried that this approach is going to introduce you some artifacts to your simulation. If you want to be more careful you can use a single random stream and divide it across processes. This is also perhaps the most state-of-the-art solution:

using Random, Future
rnd = Future.randjump(MersenneTwister(0), jobid*big(10)^20)

This will make all processes share the same huge random number stream (note that the state of Mersenne Twister is 19937 bits and hes the period of 2^19937 – 1 so this size of jump is not big at all and big(10)^20 is the recommended step of the jump because it is already precomputed in the randjump function implementation).

user_99577100
Related