Can i save multiple torch tensors in a list in R?

Viewed 83

I'm creating a neural network and i want to use the library torch for its autograd function. Now i can convert my data to a torch_tensor, but as soon as i then add that tensor to a list of other tensors they seem to lose their torch properties (which are needed to calculate the gradient at the end of the feedforward loop).

The reason i want to put the tensors in a list is because i want the number of hidden layers and neurons per hidden layer to be customizable by the end-user. Previously, before i started using torch, i accomplished that by making a list of all the separate weight matrices, the amount of which is determined by a user-provided variable.

This is the main learning function for my neural network:

train <- function(x, y, hidden = 4, layers = 3, rate = 0.01, iterations = 10000) {
  
  d <- ncol(x) + 1
  
  x <- torch_tensor(x)
  
  Wn <- list()
  Wn[[1]] <- torch_randn(d, hidden[1], requires_grad = T)
  
  if(layers > 1){
    for(j in 2:layers){
      Wn[[j]] <- torch_randn(hidden[j-1] + 1, hidden[j], requires_grad = T)
    }
  }
  
  Wn[[layers + 1]] <- torch_randn(hidden[length(hidden)] + 1, 1, requires_grad = T)
  
  for (i in 1:iterations) {
    ff <- feedforward(x, Wn)
    Wn <- backpropagate(y, Wn, ff, learn_rate = rate)
  }
  return(Wn)
}

My feedforward function looks like this:

feedforward <- function(x, Wn) {
  
  h <- list(x)
  
  for(k in 1:length(Wn)){
      Zn <- cbind(1, h[[k]]) %*% Wn[[k]]
      h[[k + 1]] <- relu(Zn)
  }
  
  return(h)
}

with relu <- function(x) { max(0,x) }

Is there any way of making this work? Or should i try to find a different method to do the feedforward function?

0 Answers
Related