Parallel/multithreading version of linear regression in Julia

user_10797482 0
Viewed 152

I implemented regression model with interactions using Julia GLM package:

Reg = lm(@formula(dep_var ~ var1&var2&var3), data, true).

Fitting this formula requires a lot of RAM (> 80 GB), but I noticed that the calculations are performed on one core, although my OS (x86_64-pc-linux-gnu) has 8 cpu cores.

Is it possible to implement linear regression using multiprocessing/parallelism approaches?

I suppose, it could also improve the model runtime.

julia distributed-computing regression parallel-processing multithreading
1 Answers

Fitting a regression model is basically doing lots of matrix operations. By default Julia is using BLAS and the easiest thing you can do is to try to configure it to be multi-threaded. This requires running Julia in a multi-threaded setting and setting the BLAS.set_num_threads() configuration.

Before starting Julia run:

set JULIA_NUM_THREADS=4

or on Linux

export JULIA_NUM_THREADS=4

Once Julia is started run the command.

BLAS.set_num_threads(4)

You should observe an increased performance of your linear regression models.

user_99577100
Related