Commands to learn:

sort (1)		- sort and collate lines
uniq (1)		- remove or report adjacent duplicate lines

8. The sort command (filter)
The sort command sorts data as well as read files containing previously sorted
data and merge them into a large, sorted file.

        sort [-bdfiMnru] [-o outputfile] [-tx] [inputfile...]
or
        sort -m [-o outputfile] sortedfile...
or
        sort [ [+n1][bdfMnr] [-n2] ... ] inputfiles

The last form sorts by key fields and is interpreted:
"Skip the 1st n1 fields and sort from to n1+1 and stop at the end of field n2"
Note that field 0 is the first field.

This last form is being replaced by the posix version of sort, which uses
the form:
	sort [ -k(n1+1)[bdfMnr],n2 ] ... ] inputfiles

Examples:

$ who | sort

$ cat phonebook
Jack Briner     919-555-1212
Jan Keppler     212-555-8999
Janet Adams     814-555-8888
Jim Jones       713-555-8772
John Doe        201-555-7893
$ sort phonebook
Jack Briner     919-555-1212
Jan Keppler     212-555-8999
Janet Adams     814-555-8888
Jim Jones       713-555-8772
John Doe        201-555-7893
$ sort +1 -2 phonebook		#or sort -k2,2 phonebook
Janet Adams     814-555-8888
Jack Briner     919-555-1212
John Doe        201-555-7893
Jim Jones       713-555-8772
Jan Keppler     212-555-8999
$ sort +1 -2 +0 -1 phonebook    #or sort -k2,2 -k1,1 phonebook
Janet Adams     814-555-8888
Jack Briner     919-555-1212
John Doe        201-555-7893
Jim Jones       713-555-8772
Jan Keppler     212-555-8999
$ sort +2n phonebook            #or sort -k3n,3 phonebook 
John Doe        201-555-7893
Jan Keppler     212-555-8999
Jim Jones       713-555-8772
Janet Adams     814-555-8888
Jack Briner     919-555-1212

9. the uniq command (filter) works with sort. It displays unique and/or
duplicated lines in files. It requires that the files be presorted. The
field and character specifications represent how many fields or characters
to skip on each line before considering uniqueness or duplication.

   uniq [-cdDiu] [-f skipfields] [-s skipchars] [-w maxchars] [infile [outfile]]

Examples:

$ who | sort | uniq -c

$ cat data
5	57.3
8	87.9
9	88.7
53	88
53	88
5	57.3
9	88
$ sort -n data | uniq
5       57.3
8       87.9
9       88
9       88.7
53      88
$ sort -n data | uniq -c
2 5       57.3
1 8       87.9
1 9       88
1 9       88.7
2 53      88

Questions? Robert Katz: rkatz@ned.highline.edu
Last Update July 16, 2002