How can I avoid writing nested for loops for large data sets?

This is combinations with repetitions. rcppalgos is likely your best out of the box but at n = 1000L, that's just over 500 million combinations to go through which will take up ~ 2GB of ram.

library(RcppAlgos)
n = 1000L
mat <- comboGeneral(n, 3L, repetition = TRUE)

Now there are two routes to go. If you have the RAM and your function is able to be vectorized, you can do the above very quickly. Let's say if the sum of the combination is greater than 1000 you want the means of the combination, other wise you want the sum of the combination.

res <- if (rowSums(mat) > 1000L) 
  rowMeans(mat)
else
  rowSums(mat)

## Error: cannot allocate vector of size 1.2 Gb

Oh no! I get the dreaded allocate vector error. rcppalgos allows you to return the result of a function. But note that it returns a list and is a lot less fast because it is going to have to evaluate your R function instead of staying in c++. Because of this, I changed to n = 100L because I do not have all day...

comboGeneral(100L, 3L, repetition = TRUE,
                        FUN = function(x) { 
                          if (sum(x) > 100L)
                            mean(x)
                          else
                            sum(x)
                        }
)

If I had a static set where I was always choosing 3 combinations out of n, I would likely use Rcpp code directly depending on what foo(a,b,c) and bar(a,b,c) are but first I would like to know more about the functions.

My previous function lazyExpandGrid is not a perfect match, but I think it addresses your concern about memory-exhaustion. Other languages have the prospect of a lazy iterator; R has it in the iterators package, and since I'm not proficient with it, some time ago I wrote this gist to address an itch.

One problem with lazyExpandGrid is that it expects the factors to be pre-defined. This can be handled with a quick condition, so it'll be memory-efficient though admittedly not space-efficient. I don't think it'd be a quick fix to implement conditionals in the method, since its mechanism for lazily dealing with the expansion is knowing mathematically which index attaches to which combination of factors ... and conditions will bust that.

Here's how that function can work here:

n <- 3
it <- lazyExpandGrid(aa = 1:n, bb = 1:n, cc = 1:n)
while (length(thistask <- it$nextItem())) {
  if (with(thistask, bb > aa || cc > bb)) next
  print(jsonlite::toJSON(thistask))
}
# [{"aa":1,"bb":1,"cc":1}] 
# [{"aa":2,"bb":1,"cc":1}] 
# [{"aa":3,"bb":1,"cc":1}] 
# [{"aa":2,"bb":2,"cc":1}] 
# [{"aa":3,"bb":2,"cc":1}] 
# [{"aa":3,"bb":3,"cc":1}] 
# [{"aa":2,"bb":2,"cc":2}] 
# [{"aa":3,"bb":2,"cc":2}] 
# [{"aa":3,"bb":3,"cc":2}] 
# [{"aa":3,"bb":3,"cc":3}] 

### to demonstrate what an exhausted lazy-expansion looks like
it$nextItem()
# NULL
it$nextItem()
# NULL

(Note how the conditional with next skips those combinations.)

That would translate to your flow as:

n <- 1000
it <- lazyExpandGrid(aa = 1:n, bb = 1:n, cc = 1:n)
it
# lazyExpandGrid: 4 factors, 1e+09 rows
#   $ index : 0

while (length(thistask <- it$nextItem())) {
  if (with(thistask, bb > aa || cc > bb)) next
  with(thistask, {
    if (foo(aa, bb, cc)) bar(aa, bb, cc)
  })
}

(Or without the with, using thistask$aa, etc.)

Note: I'm not going to lie, though, this simplifies the flow, it does not make it fast. In this case, doing something 1e+09 times is going to take time, and I don't know of anything that will help with that besides parallel operations and perhaps a friendly cluster of R hosts. (I started running an empty no-op while loop as above and it took 268 seconds to get through 822K of them. I hope you have a lot of processing power.)

It is important to point out why using rcpp is both easy and recommended for this.

When we refer to r, under the hood is a bunch of code compiled in c. Thus far, the R developers have not needed to develop compiled code to allow arbitrary functions foo() and bar() to be used in combinations with repetitions. So as users, we can use an r loop to have the flexibility of r or, when we have a lot of iterations to loop through, look at some alternatives.

The R loop is trivial to make into an Rcpp loop. I have included arbitrary functions just so we can return something (it would have been nice if the OP post had included something to return as well...):

#include <Rcpp.h>
using namespace Rcpp;

bool foo(int x, int y, int z) {
  return((x + y + z) > 50);
}

int bar(int x, int y, int z) {
  return(x - y + z);
}

// [[Rcpp::export]]
double manual_combos_w_reps(const int n) {
  double ans = 0;

  for (int i = 1; i <= n; i++) {
    for (int j = 1; j <= i; j++) {
      for (int k = 1; k <= j; k++) {
        if (foo(i, j, k)) {
          ans += bar(i, j, k);
        }
      }
    }
  }

  return(ans);
}

Here's the counter-part in R which is just your code with foo(...) and bar(...) added.

r_foo = function(x, y, z) {
  return((x + y + z) > 50L)
}

r_bar = function (x, y, z) {
  return(x - y + z)
}

r_loop = function (n) {
  ans = 0;
  for (i in 1:n) {
    for (j in 1:i) {
      for (k in 1:j) {
        if (r_foo(i, j, k)) {
          ans = ans + r_bar(i, j, k)
        }
      }
    }
  }
  return(ans)
}

Now this is the magic part. Rcpp flies through these iterations. For n = 1000L, it takes R code 360 seconds to run. It takes Rcpp only 0.5 seconds to run.

n = 10L
bench::mark(manual_combos_w_reps(n)
            , r_loop(n)
            )

### A tibble: 2 x 13
##  expression                 min median `itr/sec` mem_alloc
##  <bch:expr>              <bch:> <bch:>     <dbl> <bch:byt>
##1 manual_combos_w_reps(n)  4.7us    5us   178048.    2.48KB
##2 r_loop(n)               1.63ms 1.68ms      505.        0B

n = 100L

### A tibble: 2 x 13
##  expression                min median `itr/sec` mem_alloc
##  <bch:expr>              <bch> <bch:>     <dbl> <bch:byt>
##1 manual_combos_w_reps(n) 467us  469us   2064.      2.48KB
##2 r_loop(n)               627ms  627ms      1.60        0B

n = 1000L

### A tibble: 2 x 13
##  expression                   min   median `itr/sec`
##  <bch:expr>              <bch:tm> <bch:tm>     <dbl>
##1 manual_combos_w_reps(n) 459.29ms 459.39ms   2.18   
##2 r_loop(n)                  6.04m    6.04m   0.00276

You should absolutely look into rcpp for this - there is no truly canonical answer in base R that will provide high performance for the task you are doing. The only concern is what your actual foo() and bar() functions are as they may be difficult to implement in rcpp.

purrr solution with .filter also works :

library(purrr)

n <- 10L
levels <- 3L

# keep only elements below diagonal
isdesc<- function(...){all(diff(unlist(list(...)))<=0)}
# some extra filtering
foo <- function(...) { sum(unlist(list(...)))==27}

filter <- function(...) {!isdesc(...)|!foo(...)}

cross_list <- cross(rep(list(1L:n),levels),.filter = filter)

bar <- function(...) ( unlist(list(...))) 

cross_list %>% map(bar)

Unfortunately, like grid.expand, it doesn't scale nicely because cross first allocates the complete cartesian product before filtering it.