Finding Things

One of the most common things you may need to do is to find files or directories.

Maybe you need to find something in your own files, or maybe you need to see if the current system has a C/C++ header in one of the usual locations, or has a library where you expect it to be (or not!)

Some systems may have locate, which can be magical, but it’s not common on HPC systems. So, you’re better off learning to exploit find and grep to get what you want.

find

find is a very powerful utility that recursively lists all files and folders starting from the path you pass as its first argument.

find . recurses down from your current directory to all subfolders.

The files/directories are then listed on stdout. find also has a plethora of flags that can shape how far you search (-mindepth, -maxdepth), what type you’re looking for (-type d or -type f for directories or files), and case sensitive (-name) or insensitive (-iname) searching using Bash wildcards, and even regexes with (-regex)

This would be pretty useful in and of itself, but the real power of find comes with its -exec flag , which passes {} as the name of the file or dir to any arbitrary command you would like.

Let’s say you wanted to remove all files in a series of directories that ended in .dud (a made up extension for purposes of the exercise):

find mydir -type f -name '*.dud' -exec rm -f {} \;

This would find all files in mydir and its subdirectories, ending in .dud and execute rm on them without asking for confirmation. {} is the path of the file and \; is an escaped ; that terminates the command.

grep

Sometimes you need to search the contents of files. grep is the standard tool for this. You can use find to execute it, but grep can happily recurse through files itself.

N.B. Not all greps are created equal. Different OSes have slightly different versions and your mileage may vary.

-E flag lets you use extended regexes for more complex queries (this is the equivalent of running egrep)

A basic example would be to search all files in a directory for the phrase foobar:

grep -Rn foobar mydir/

The -n flag will display the line number, in addition to the default behavior of showing the file and the matching line.