How to avoid unnecessary for loops in R

Question

I have two vectors drawn from a t-distribution, i.e. X and epsilon and I generate Y from these to vectors based on a condition. I aim to simulate multiple samples. If I simulate 10.000 samples, it will take a long time for the computer to complete. I want to reduce the computation time by avoiding the for loop. I've tried a few things but it didn't work. How to avoid the for loop and reduce the computation time for this specific for loop? The code is as follows

X <- rt(1250,5)
eps <- rt(1250,5)
Y <- replicate(1250,0)

for(i in 1:1250) {
  if(X[i]>quantile(X, 0.5)){
    Y[i] = X[i] + eps[i]
  }
  else { 
    Y[i] = 1.5*X[i] + eps[i]
  }
}

user_7795396 · Answer

You can use the C-level for loops that exist for many R functions. This is called 'vectorisation' and it is a powerful concept in R. Function ifelse is vectorised as are the + and * functions. Hadley Wickham explains it here.

X <- rt(1250,5)
eps <- rt(1250,5)

Y <- numeric(1250)
for(i in 1:1250) {
  if(X[i]>quantile(X, 0.5)){
    Y[i] = X[i] + eps[i]
  }
  else { 
    Y[i] = 1.5*X[i] + eps[i]
  }
}

Y_vectorized <- ifelse(X > quantile(X, 0.5), X + eps, 1.5*X + eps)

With the result:

> identical (Y,Y_vectorized)
[1] TRUE

How much faster is the vectorised approach (using r2evans suggestion to keep the quantile calculation out of the loop)?

library(microbenchmark)
Y <- numeric(1250)
med <- quantile(X, 0.5)
microbenchmark("for-loop" = {
  for (i in 1:1250) {
    if (X[i] > quantile(X, 0.5)) {
      Y[i] = X[i] + eps[i]
    }
    else {
      Y[i] = 1.5 * X[i] + eps[i]
    }
  }
}, 
"vectorised" = { Y_vectorized <- ifelse(X > med, X + eps, 1.5 * X + eps) },
times = 100)

Unit: microseconds
       expr      min       lq       mean    median        uq      max neval
   for-loop 120488.2 123000.6 131055.758 125508.95 131246.40 247101.6   100
 vectorised     30.2     36.1     48.955     51.15     53.75    139.6   100

For the vector length of 1250 the vectorised approach is ~2670 times faster.

user_3358272 · Answer

Don't recalculate quantile(X,0.5) each and every time: it never changes, calculate it once and reuse the stored value.
Use vectorized operations, knowing that comparisons and assignments can happen a whole vector at a time. I suggest that you can reduce that to:
```
X <- rt(1250,5)
eps <- rt(1250,5)
med <- quantile(X, 0.5)
Y <- ifelse(X > med, 1, 1.5) * X + eps
```

A quick walk-through of vectorized operations demonstrated here:

set.seed(42)
X <- rt(10, 5)
eps <- rt(10, 5)
med <- quantile(X, 0.5)
X
#  [1]  1.9151  0.0878 -0.0773 -0.0618 -0.0480  5.0230  1.0924  0.8423  1.5165
# [10] -0.2601
eps
#  [1]  0.712 -1.048  2.233 -0.737 -1.273 -0.890  0.395 -1.828 -0.601 -0.392
med
#   50% 
# 0.465

If we compare a vector with a scalar (or a vector with the same-length vector), then we get a vector of logical/boolean:

X > med
#  [1]  TRUE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE  TRUE FALSE

From this, ifelse is a vectorized conditional. While if (...) {...} else {...} can deal with a single length-1 logical at a time, ifelse works on a vector at a time. For instance:

ifelse(c(T, F, F, T), 1:4, 11:14)
# [1]  1 12 13  4

Back to the example, we continue with ifelse and add math operations, which also work just as well on a vector as a scalar.

ifelse(X > med, 1, 1.5)
#  [1] 1.0 1.5 1.5 1.5 1.5 1.0 1.0 1.0 1.0 1.5
ifelse(X > med, 1, 1.5) * X
#  [1]  1.9151  0.1318 -0.1160 -0.0926 -0.0720  5.0230  1.0924  0.8423  1.5165
# [10] -0.3902

How to avoid unnecessary for loops in R

2 Answers