furrr with rTorch in multisession

Viewed 126


I want to use rTorch in a furrr "loop". A minimal example seems to be:

library(rTorch)
torch_it <- function(i) {
  #.libPaths("/User/homes/mreichstein/.R_libs_macadamia4.0/")
  #require(rTorch)
  cat("torch is: "); print(torch)
  out  <- torch$tensor(1)
  out
}

library(furrr)
plan(multisession, workers=8) # this works with sequential or multicore
result <- future_map(1:5, torch_it)

I get as output:

torch is: <environment: 0x556b57f07990>
attr(,"class")
[1] "python.builtin.module" "python.builtin.object"
Error in torch$tensor(1) : attempt to apply non-function

When I use multicore or sequential I get the expected output:

torch is: Module(torch)
torch is: Module(torch)
torch is: Module(torch)
torch is: Module(torch)
torch is: Module(torch)

... and result is the expected list of tensors. Uncommenting the first lines in the function, assuming the new sessions "need this" did not help.

Update: I added torch <- import("torch") in the torch_it function. Then it runs without error, but the result I get is a list of empty pointers (?):

> result
[[1]]
<pointer: 0x0>
[[2]]
<pointer: 0x0>
[[3]]
<pointer: 0x0>
[[4]]
<pointer: 0x0>
[[5]]
<pointer: 0x0>

So, how do I properly link rTorch to each of the multisessions? Thanks in advance!

Info:

> R.version
               _                           
platform       x86_64-redhat-linux-gnu     
arch           x86_64                      
os             linux-gnu                   
system         x86_64, linux-gnu           
status                                     
major          4                           
minor          0.3                         
year           2020                        
month          10                          
day            10                          
svn rev        79318                       
language       R                           
version.string R version 4.0.3 (2020-10-10)
nickname       Bunny-Wunnies Freak Out
0 Answers
Related