Shrink it (`gzip`)

Here is the most surprising thing about gzip.

~ $ lsorigin.txt~ $ gzip origin.txt~ $ lsorigin.txt.gz        <- origin.txt has gone!

It replaces the original file. It does not make a copy the way cp does.

To put it back there is gunzip.

~ $ gunzip origin.txt.gz~ $ lsorigin.txt           <- back (and the .gz has gone)

Both are replacements, so you just go back and forth.

When you want to keep the original, add -k (keep).

~ $ gzip -k origin.txt~ $ lsorigin.txt   origin.txt.gz     <- both there

Three ways of writing it are enough.

gunzip and gzip -d are exactly the same thing: one program called by two names. The same as busybox having many names, from earlier in the chapter.

Uncompressing something every time you want to read it is a bother, so there is a tool.

~ $ zcat access.log.gzok /index.htmlerror /missing.html...

A relative of cat. It gives the contents while they stay compressed, and the file stays as .gz.

Being on the way out, it naturally goes into a pipe.

~ $ zcat access.log.gz | grep error | wc -l3

This is the shape you use most in practice. Old logs on a server are nearly always kept compressed.

/var/log/messages         today's log/var/log/messages.1.gz    yesterday's (compressed)/var/log/messages.2.gz    the one before

So an investigation looks like this.

zcat /var/log/messages.*.gz | grep error

With -c you can send gzip itself to the way out.

gzip -c origin.txt > copy.gz     keep the original, compress under another name

-c is used in many commands to mean "to the way out, not to a file". Worth knowing.

That familiar name should be readable to you now.

docs.tar.gz     ^   ^     |   \- shrunk with gzip     \----- packed with tar

The name records the two tools it went through, in order. So making one takes two moves.

~ $ tar cf docs.tar docs    pack it~ $ gzip docs.tar           shrink it~ $ lsdocs.tar.gz

Undoing it goes the other way round.

~ $ gunzip docs.tar.gz      unwrap it~ $ tar xf docs.tar         open it

The order reverses in the same way that taking clothes off reverses putting them on.

To do it in one move, join them with a pipe.

~ $ gunzip -c docs.tar.gz | tar tf -

The - at the end is the mark for "not a file, take it from the way in". Something like it came up when you learned xargs in chapter 4.

The tar here has no z mark, so splitting it is the only way. On a real Linux this is one command.

tar czf docs.tar.gz docs    make ittar tzf docs.tar.gz         see ittar xzf docs.tar.gz         extract it

With a compressed file there is this worry.

the transfer was cut off partwaythe copy stopped halfway

You would not notice until you tried to look inside, so there is a mark for checking.

~ $ gzip -t docs.tar.gz~ $ echo $?0                    <- all well

Broken, it goes like this.

~ $ gzip -t fake.gzgzip: invalid magic~ $ echo $?1                    <- no good

The -t is the t of test. Finishing without a word means it is fine, which is the command line's manner. "Quiet means success", from chapter 4.

The magic in invalid magic is the mark at the head of the file saying "I am a gzip".

read the first few bytes and you know the kind

The file command in chapter 2 guessed the kind from the contents by reading exactly this mark. The extension is decoration for people; the real kind is written inside.

So text merely renamed to .gz is seen through by gzip at once.

Here are the shapes for the lesson.

gzip name               shrink it (the original goes)gzip -k name            shrink it (keep the original)gzip -c name > other.gz to the way out, saved under another namegunzip name.gz          put it backgzip -d name.gz         put it back (the same thing)zcat name.gz            give the contents while compressedgzip -t name.gz         check it is not broken

You also use the trick for making a biggish file to practise on.

~ $ yes hello | head -2000 > origin.txt

yes is the command that gives out the same word for ever. You used it in place of "heavy work" in chapter 8. Here head -2000 cuts off two thousand lines of it.

Being nothing but repetition, it is material that shrinks well. You can make material that does not, too.

~ $ head -c 12000 /dev/urandom > ran.bin

/dev/urandom is the device that gives out random values, a sibling of /dev/zero. With no repeats in it, it does not shrink. Measure both and see the difference with your own eyes.