Repeating with `for`

The basic shape is this.

for A in aa bb cc; do  echo [$A]done

Run it and you get this.

[aa][bb][cc]

You read it like this.

for A in the row of things  | take one out and put it in Ado  do the insidedone  on to the next one

A is a name you choose. F or COLOR or anything. Usually you pick a name that says what is inside.

for F in *.txt      F for filefor N in 1 2 3      N for number

It closes with done. The same job as the fi of if.

if   ->  fifor  ->  done

You can pack it onto one line.

for A in aa bb; do echo $A; done

Separated with ;. One line is fine when short, but once there are two things inside, writing it down the page reads better.

The commonest use is this.

for F in *.txt; do  echo "--- $F ---"  wc -l < "$F"done

The *.txt expands first, and each one goes into F.

for F in *.txt  | the glob expands (chapter 10)for F in blue.txt red.txt yellow.txt

The order of expansion from chapter 10, at work as it is. The shell rewrites it first, remember.

One thing to watch. When nothing matches, this happens.

~ $ for F in none*.txt; do echo [$F]; done[none*.txt]

The pattern itself goes round once. Not zero times.

So checking inside is the manner.

for F in none*.txt; do  [ -f "$F" ] || continue  echo "$F"done

continue means "skip this go and move on". Skip what is not there.

And do not forget the "$F". With a name containing a space, unwrapped it splits. As in chapter 10 and the last lesson.

You can go round things besides file names.

Round numbers

for N in 1 2 3; do echo $N; donefor N in $(seq 1 10); do echo $N; done

seq lays out numbers (you used it in chapter 2). Do it ten times takes this shape.

Round the contents of a file

for COLOR in $(cat colors.list); do  echo "$COLOR turn"done

The contents of colors.list go in a line at a time. The list of what to do can be kept in a file.

This is powerful.

rewrite colors.list and what gets done changes

You change the behaviour without touching the script. Keeping the procedure and the targets apart.

It has a weakness, though.

a line with a space in it splits

The word splitting of chapter 10. When you want lines with spaces, you use while read. That is the next lesson.

Going round the arguments matters too.

for F in "$@"; do  echo [$F]done

Wrapped as "$@", the joins between arguments are kept, as in the last lesson.

Think back to the xargs of chapter 4.

ls *.txt | xargs wc -l

The same thing writes with for too.

for F in *.txt; do wc -l "$F"; done

Which should you use? Here is a guide.

xargs hands them over together, so it is fast.

xargs   wc -l a.txt b.txt c.txt      one gofor     wc -l a.txt / wc -l b.txt    three runs

With ten thousand files the difference shows.

But inside a for you can write what you like.

for F in *.txt; do  [ -s "$F" ] || continue  echo "--- $F ---"  head -1 "$F"done

Conditions, and procedures of any length. You cannot do this with xargs.

So remember it this way.

simple, and you want speed   ->  xargsyou want to think inside     ->  for

There are two tools because the purposes differ.

Two tools for inside a repetition.

continue — skip this go and move on

for A in aa bb cc; do  [ "$A" = bb ] && continue  echo $Adone

aacc          <- bb was skipped

break — stop the repetition

for A in aa bb cc; do  [ "$A" = bb ] && break  echo $Adone

aa          <- it stopped at bb

Where you use them.

continue   skip what does not fit (a sieve)break      stop when you find it (a search)

continue gets a lot of use. The shape on the earlier slide is that.

for F in *.txt; do  [ -f "$F" ] || continue     skip what is not there  [ -s "$F" ] || continue     skip the empty ones too  the real workdone

Conditions to skip at the top, the real work below. The same shape as "returning early" from the last lesson. Nothing is nested, so it reads well.

You can nest them.

for A in aa bb; do  for B in 1 2; do    echo $A$B  donedone

Out comes aa1 aa2 bb1 bb2. The inner one goes all the way round before the outer moves on. Deeper gets hard to read, so two levels is a good limit.

The output of a for can go into one file together.

for F in *.txt; do  echo "--- $F ---"  wc -l < "$F"done > summary.log

The > goes after the done. You have pointed the way out of the whole repetition.

Compare with writing it inside.

for F in *.txt; do  wc -l < "$F" > summary.log      <- overwritten every time!done

That way only the last one is left, because > blanks it each time.

To add from inside, use >>.

  wc -l < "$F" >> summary.log     it grows

Here are the three side by side.

done > f is the tidiest. It opens the file once so it is fast, and there is no accident of forgetting.

The > and >> of chapter 4, combined with repetition, change the result entirely. This is a place to move your hands and see.

Here are the shapes for this lesson.

for A in aa bb cc; do ... done      a row you wrotefor F in *.txt; do ... done         file namesfor N in $(seq 1 5); do ... done    numbersfor L in $(cat list); do ... done   the contents of a filefor F in "$@"; do ... done          the argumentscontinue                            skip this gobreak                               stop the repetitiondone > summary.log                  put it all out together

One knack for making them.

> Go round with echo first and see what comes

for F in *.txt; do echo "$F"; done

Put out just the names before the real work. The same thinking as "echo before anything dangerous" from chapter 10.

for F in *.txt; do echo rm "$F"; done     see what would be removedfor F in *.txt; do rm "$F"; done          really remove it

With repetition, a mistake happens to everything, so checking matters especially. Removing one file by mistake and removing a hundred are not the same kind of accident.

The next lesson is while.

for     when the number is settledwhile   while a condition holds

The choice will come into view. Let us go round.