View on GitHub

Commandline Tools Workshop

Course site for PiCSciE/RC bootcamp workshop

The philosophy of Unix utilities

Moving text around

All shells, not just Bash, include a basic set of facilities: a stream called stdout that prints program output, a stream called stderr that prints program error messages, a stream called stdin that receives input and the ability to pipe those streams to a different program or file.

Modular programs

Rather than having one monolithic program (think Microsoft Word or the Office suite as a whole), all of the basic utilities that tend to be standard on a POSIX based OS tend to do one thing, and then be capable of invoking another utility based on their output.

Take for example, ls, which lists directory output. Its purpose is very simple: write a stream of text listing the contents of a specified directory to stdout.

It has flags to get more detail information about files, but no particular ability to search for a specific file.

grep can search a file or a stream for lines matching a particular set of characters. It can also recurse through files. However, sometimes you want to match a file name.

A quick and easy way to use these together is ls | grep target where target is the text you want to look for in file names. Now, there are probably even better ways to do this search (find), but for a quick, low syntax way to do a search, it’s quite handy.

The | (pipe) character takes the output of ls and pipes it to grep which can take a stream as its second argument.

This is often how you can do incredibly complicated tasks without needing to pull out a separate scripting language like Python, Perl, etc.