Reshape text

INPUT · Slides

Sort it (`sort`)

01 / 08

This chapter grows the set of things to join to

You learned pipes in the last chapter. You know how to join, so now we grow what there is to join to.

Every tool in this chapter has the same shape.

Take text in, work on it, give text back.

Which is why they all take a pipe. A tool that sorts, a tool that drops duplicates, a tool that pulls out a column — combine them and the sort of adding up you would do in a spreadsheet fits on a line.

First up is sort, the tool that puts things in order.

02 / 08

`sort` puts the lines in order

The simplest possible form.

sort names.txt

That puts the lines into alphabetical order. The original file does not change; it only puts the sorted result out.

To keep it, add a >, as in the last chapter.

sort names.txt > sorted.txt

~ $ sort names.txtsatosuzukitanaka

03 / 08

Numbers do not sort as they are

Here is the biggest trap in this lesson, up front. sort a set of numbers and the order comes out strange.

~ $ sort scores.txt1002539

100 at the top, and 3 before 9. That is odd.

The reason is that sort is comparing them as text. It looks at the first character and orders 1 < 2 < 3 < 9. It is only comparing the 1 of 100 against the 2 of 25.

Same idea as a phone book, where "Ab" comes before "Ax".

04 / 08

`-n` sorts them as numbers

When you want them sorted as numbers, add -n (the n of number).

~ $ sort -n scores.txt3925100

That is what you meant.

When there are numbers in it, -n. Forget it and you will get it wrong every time, so let it into your fingers. Remember sort -rn from the log summarising? That n is this one.

05 / 08

Sorting on a column

With data laid out as a table, you want to say which column to sort on.

sort -t, -k2 -n grades.csv

-t, — the separator is a comma
-k2 — sort on the second column
-n — as numbers

grades.csv is in the shape name,score, so that puts it in order of score.

Without -t the separator is whitespace. For whitespace separated data, -k2 on its own is enough.

~ $ sort -t, -k2 -n grades.csvsato,72tanaka,80kimura,95suzuki,95

06 / 08

`-u` drops the duplicates

Add -u (unique) and it squashes identical lines into one as it sorts.

sort -u fruit.txt

That does the same thing as the uniq of the next lesson, but sort -u is one command, so it is shorter.

There are things only uniq can do, though (such as counting with -c). So you learn both.

07 / 08

The order of capitals and lower case

By default sort puts capitals before lower case, so Apple, then Banana, then apple.

Surprising, but it is ordering by character code number. Inside the computer A (65) is smaller than a (97).

When you want the order a person expects (Apple, apple, Banana), add -f (fold, as in fold the cases together).

Add -f when you are making a list for a person to read. It is worth knowing if you want kind output.

08 / 08

Now have a go

There are five files.

names.txt — three names
scores.txt — four lines, 100 9 25 3
grades.csv — four lines of name,score
fruit.txt — six lines of fruit, with duplicates
caps.txt — Apple apple Banana

This lesson leans on | head -1 (the top) and | tail -1 (the bottom) to check the order. The quickest way to see whether an order is right is to look at the ends.