How do I iterate over a range of numbers defined by variables in Bash?

Viewed 1573949

How do I iterate over a range of numbers in Bash when the range is given by a variable?

I know I can do this (called "sequence expression" in the Bash documentation):

 for i in {1..5}; do echo $i; done

Which gives:

1
2
3
4
5

Yet, how can I replace either of the range endpoints with a variable? This doesn't work:

END=5
for i in {1..$END}; do echo $i; done

Which prints:

{1..5}

20 Answers
for i in $(seq 1 $END); do echo $i; done

edit: I prefer seq over the other methods because I can actually remember it ;)

The seq method is the simplest, but Bash has built-in arithmetic evaluation.

END=5
for ((i=1;i<=END;i++)); do
    echo $i
done
# ==> outputs 1 2 3 4 5 on separate lines

The for ((expr1;expr2;expr3)); construct works just like for (expr1;expr2;expr3) in C and similar languages, and like other ((expr)) cases, Bash treats them as arithmetic.

discussion

Using seq is fine, as Jiaaro suggested. Pax Diablo suggested a Bash loop to avoid calling a subprocess, with the additional advantage of being more memory friendly if $END is too large. Zathrus spotted a typical bug in the loop implementation, and also hinted that since i is a text variable, continuous conversions to-and-fro numbers are performed with an associated slow-down.

integer arithmetic

This is an improved version of the Bash loop:

typeset -i i END
let END=5 i=1
while ((i<=END)); do
    echo $i
    …
    let i++
done

If the only thing that we want is the echo, then we could write echo $((i++)).

ephemient taught me something: Bash allows for ((expr;expr;expr)) constructs. Since I've never read the whole man page for Bash (like I've done with the Korn shell (ksh) man page, and that was a long time ago), I missed that.

So,

typeset -i i END # Let's be explicit
for ((i=1;i<=END;++i)); do echo $i; done

seems to be the most memory-efficient way (it won't be necessary to allocate memory to consume seq's output, which could be a problem if END is very large), although probably not the “fastest”.

the initial question

eschercycle noted that the {a..b} Bash notation works only with literals; true, accordingly to the Bash manual. One can overcome this obstacle with a single (internal) fork() without an exec() (as is the case with calling seq, which being another image requires a fork+exec):

for i in $(eval echo "{1..$END}"); do

Both eval and echo are Bash builtins, but a fork() is required for the command substitution (the $(…) construct).

You can use

for i in $(seq $END); do echo $i; done

I've combined a few of the ideas here and measured performance.

TL;DR Takeaways:

  1. seq and {..} are really fast
  2. for and while loops are slow
  3. $( ) is slow
  4. for (( ; ; )) loops are slower
  5. $(( )) is even slower
  6. Worrying about N numbers in memory (seq or {..}) is silly (at least up to 1 million.)

These are not conclusions. You would have to look at the C code behind each of these to draw conclusions. This is more about how we tend to use each of these mechanisms for looping over code. Most single operations are close enough to being the same speed that it's not going to matter in most cases. But a mechanism like for (( i=1; i<=1000000; i++ )) is many operations as you can visually see. It is also many more operations per loop than you get from for i in $(seq 1 1000000). And that may not be obvious to you, which is why doing tests like this is valuable.

Demos

# show that seq is fast
$ time (seq 1 1000000 | wc)
 1000000 1000000 6888894

real    0m0.227s
user    0m0.239s
sys     0m0.008s

# show that {..} is fast
$ time (echo {1..1000000} | wc)
       1 1000000 6888896

real    0m1.778s
user    0m1.735s
sys     0m0.072s

# Show that for loops (even with a : noop) are slow
$ time (for i in {1..1000000} ; do :; done | wc)
       0       0       0

real    0m3.642s
user    0m3.582s
sys 0m0.057s

# show that echo is slow
$ time (for i in {1..1000000} ; do echo $i; done | wc)
 1000000 1000000 6888896

real    0m7.480s
user    0m6.803s
sys     0m2.580s

$ time (for i in $(seq 1 1000000) ; do echo $i; done | wc)
 1000000 1000000 6888894

real    0m7.029s
user    0m6.335s
sys     0m2.666s

# show that C-style for loops are slower
$ time (for (( i=1; i<=1000000; i++ )) ; do echo $i; done | wc)
 1000000 1000000 6888896

real    0m12.391s
user    0m11.069s
sys     0m3.437s

# show that arithmetic expansion is even slower
$ time (i=1; e=1000000; while [ $i -le $e ]; do echo $i; i=$(($i+1)); done | wc)
 1000000 1000000 6888896

real    0m19.696s
user    0m18.017s
sys     0m3.806s

$ time (i=1; e=1000000; while [ $i -le $e ]; do echo $i; ((i=i+1)); done | wc)
 1000000 1000000 6888896

real    0m18.629s
user    0m16.843s
sys     0m3.936s

$ time (i=1; e=1000000; while [ $i -le $e ]; do echo $((i++)); done | wc)
 1000000 1000000 6888896

real    0m17.012s
user    0m15.319s
sys     0m3.906s

# even a noop is slow
$ time (i=1; e=1000000; while [ $((i++)) -le $e ]; do :; done | wc)
       0       0       0

real    0m12.679s
user    0m11.658s
sys 0m1.004s

This works fine in bash:

END=5
i=1 ; while [[ $i -le $END ]] ; do
    echo $i
    ((i = i + 1))
done

There are many ways to do this, however the ones I prefer is given below

Using seq

Synopsis from man seq

$ seq [-w] [-f format] [-s string] [-t string] [first [incr]] last

Syntax

Full command
seq first incr last

  • first is starting number in the sequence [is optional, by default:1]
  • incr is increment [is optional, by default:1]
  • last is the last number in the sequence

Example:

$ seq 1 2 10
1 3 5 7 9

Only with first and last:

$ seq 1 5
1 2 3 4 5

Only with last:

$ seq 5
1 2 3 4 5

Using {first..last..incr}

Here first and last are mandatory and incr is optional

Using just first and last

$ echo {1..5}
1 2 3 4 5

Using incr

$ echo {1..10..2}
1 3 5 7 9

You can use this even for characters like below

$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z

if you don't wanna use 'seq' or 'eval' or jot or arithmetic expansion format eg. for ((i=1;i<=END;i++)), or other loops eg. while, and you don't wanna 'printf' and happy to 'echo' only, then this simple workaround might fit your budget:

a=1; b=5; d='for i in {'$a'..'$b'}; do echo -n "$i"; done;' echo "$d" | bash

PS: My bash doesn't have 'seq' command anyway.

Tested on Mac OSX 10.6.8, Bash 3.2.48

Related