The shell environment

INPUT · Slides

The order things expand in (`~`, `$`, `*`)

01 / 07

From the return key to the running

What this chapter has said three times now gathers into one thread.

> The shell rewrites the characters before it runs the command

And the rewriting has an order. Roughly, this is the flow.

1. turn ~ into the home place        tilde expansion2. turn $name into the contents      variable expansion3. turn $(...) into its output       command substitution4. turn $((...)) into the result     arithmetic expansion5. split on spaces                   word splitting6. turn * ? [] into file names       globbing7. take off the quotes               quote removal8. hand the finished list to the command

It looks like a lot, but the only order to learn is 2, then 5, then 6. Every accident of the last lessons is in there.

split on spaces after expanding the variable  ->  hence "$F"expand the * after the splitting              ->  hence a * inside a variable expands too

Know the order and you can predict the result. Predict it and you can stop before something dangerous. That is what this lesson is for.

02 / 07

~ only at the front

Start with something light. ~ expands into the home place.

~ $ echo ~/home/learner~ $ echo ~/memo.txt/home/learner/memo.txt

It does not expand everywhere, though.

~ $ echo a~a~            <- no expansion

It expands only at the front of a word. So a ~ in the middle is just a character.

That is worth knowing. Some tools use ~ in the names of backup files.

memo.txt~     the copy an editor makes (just a name)

And ~ has other uses.

~         your own home~learner  learner's home

Write a name it does not know and it stays as it is, unexpanded.

~ $ echo ~x~x

You may think of ~ as the same as $HOME, but strictly they differ a little.

echo "~"       ->  ~ (wrapped, no expansion)echo "$HOME"   ->  /home/learner (expands even wrapped)

In a script, $HOME is the surer one, because it works wrapped.

03 / 07

Put a command's output in with $(...)

This one is powerful.

~ $ echo "I am $(whoami) here"I am learner here

It runs the command inside $(...) first and puts its output there. It is called command substitution.

echo "I am $(whoami) here"  | run whoami -> learnerecho "I am learner here"

You can use a pipe inside it.

~ $ N=$(ls *.txt | wc -l)~ $ echo $N3

Now you can keep the result of a command in a variable. That is what lets you write the scripts of chapter 11.

You see an older way of writing it too.

N=`ls | wc -l`     backquotes (the old shape)N=$(ls | wc -l)    this is the current one

Same meaning, but use $( ), because it nests.

echo $(( $(ls *.txt | grep -c .) * 2 ))

With backquotes, nesting needs piles of \ and becomes dreadful to read. The newer shape is the plainer one.

04 / 07

Calculate with $((...))

The shell can add up as well.

~ $ echo $((2+3))5~ $ N=4~ $ echo $((N*N))16

Inside $(( )) you need no $ on a variable, it being known as a calculation.

One thing to watch: whole numbers only.

~ $ echo $((10/3))3            <- not 3.33

When you need decimals, lean on another tool.

awk 'BEGIN{print 10/3}'

The awk of chapter 5. What the shell cannot do, hand to a tool that can.

Here are the symbols you can use.

SymbolMeaning
+ - * /the four operations (/ rounds down)
%the remainder
> < ==compare (1 or 0 comes back)

Put it together with command substitution and you can do this.

~ $ echo $(( $(ls *.txt | grep -c .) * 2 ))6

Count something and then calculate with it. Only small tools stacked up, and yet quite a lot gets done.

05 / 07

A * inside a variable expands too

The most important experiment of this lesson.

~ $ P='*.txt'~ $ echo $Pa1.txt a2.txt b1.txt      <- it expanded!~ $ echo "$P"*.txt

You assigned it wrapped in singles and it expanded when you used it. Can you see why? Think back to the order.

echo $P  | 2. variable expansion  ->  echo *.txt  | 6. globbing            ->  echo a1.txt a2.txt b1.txt

Because globbing runs after the variable is expanded. The quotes at assignment time only apply at assignment time.

Wrap it and it stops.

echo "$P"  | no globbing inside quotes*.txt

Here the manner of the last lesson turns up again.

when in doubt, wrap it as "$variable"

Wrapping stops two things.

5. word splitting (being cut on spaces)6. globbing (the * expanding)

Both "happen after the replacing". So the wrapping tidies up after the replacing — an easy way to remember it.

06 / 07

What this environment does not have

Some things you see in books do not work here.

~ $ echo {a,b}{a,b}            <- no expansion~ $ echo {1..3}{1..3}           <- no expansion

The feature is called brace expansion, and bash has it while this environment's shell (busybox ash) does not.

in bash:  echo {a,b}    ->  a b          mkdir dir{1,2,3}  ->  dir1 dir2 dir3 at once

It is handy, so do try it when you use bash on a real computer.

Here is the important part.

> There are kinds of shell

sh    the minimum. everywhereash   busybox's sh (this environment)bash  the commonest on Linuxzsh   the default on a Mac

The foundations of the writing are the same; what differs is which conveniences exist.

So when you write a script, think like this.

you want it to run anywhere    ->  write with sh features onlyyour own machine is enough     ->  use the bash conveniences

What this course gives you is the former, the foundation. Write something that runs in the narrowest place and the wide places give no trouble.

07 / 07

Now have a go

Here are the shapes for this lesson.

echo ~                     the home place (at the front only)echo $((2+3))              calculate (whole numbers only)echo $(whoami)             put a command's output inN=$(ls | wc -l)            keep the result in a variableecho ?1.txt                match exactly one characterecho [ab]1.txt             any one of theseP='*.txt'; echo $P         it expandsP='*.txt'; echo "$P"       it stops

Here is one tool for checking. Put echo in front and look first.

~ $ echo rm $Prm a1.txt a2.txt b1.txt      <- three would go

The result is visible before you type it. Always do this before an operation you cannot undo. The professionals do it before rm -r as well.

And the three of the order, once more.

2. $name becomes the contents5. split on spaces6. * expands

Know that sequence and you can explain eight in ten of your own "why did that happen?" moments. Now let us type.