Searchdown: Find and Grep Redux

Note: there's a public repository containing the source code and any formal documentation for the program that this post is about.

Searchdown provides a simpler, pithier and (arguably) more standard interface for find … and find … | xargs … commands, with additional support for find … | xargs grep … commands.

If you need to list or operate on some or all of the files under one or more directories (where for our purposes a file is under a directory if it's directly in that directory or if it's under one of that directory's subdirectories) it's hard to beat the standard find command. Among its advantages are:

• it comes preinstalled with most UNIX-like operating systems (as do xargs and grep),
• it's reasonably fast in most cases and allows the user to improve its speed and efficiency based on their knowledge of the files being searched, and
• its options for selecting which files to find are all but comprehensive: I don't think I've ever had a set of criteria that I couldn't translate into find options.

It does have some disadvantages when used interactively, though:

• many of its options - including some of the most commonly used ones - are both long and nonstandard (e.g. -file instead of --file or -f),
• you always have to specify at least one directory: there's no default (and can't be, as a consequence of the command's syntax)
• by default it doesn't properly handle pathnames containing spaces or other special characters, and
• it can be overly persnickety about the ordering of some of its options: order does often affect the results and efficiency of its search, but in some cases - the -mindepth and -maxdepth options are where I've most often encountered it - find (or at least the GNU version of find) will complain about where an option appears in a command but will then carry on running that command, so presumably it wasn't confused about what was intended, just displeased.

The main reason that I wrote searchdown was that find commands - including ones combined with xargs - were among the ones that I used the most, and were also some of the longer commands to type. I didn't need the implementation to change, just the interface, at least for the sorts of find commands that I used frequently. So I wrote a Python program that

• provides a simpler, more standard and less verbose command line interface, and then
• builds and runs the corresponding find command.

The options and arguments that searchdown takes are all described in its usage message (with some delegation to the find command's documentation), so I won't repeat that here. Instead I'll give some examples of how I use it, which is often by way of the shell aliases and functions defined in the searchdown-aliases.sh file that's included in the searchdown git repository.

But first things first. The name of the searchdown command itself is pretty long: it's more than twice as long as find. Not an auspicious start! Personally I feel that short command names should be reserved as much as possible for use by the user for (aliases for or symbolic links to) the commands that they use the most often, so I tend to give my programs longer names and then define shorter aliases for them.

In the case of searchdown I give it the alias sd, which is how it'll appear in the examples below. More accurately, sd is an alias for a shell function (defined in searchdown-aliases.sh) that sorts the results and runs them through the default pager program, and also assumes that when it's given a single non-option argument that it's a regular expression to search for in all of the matching files. So for example the command

sd Som.thing

is equivalent to the find command

find -O2 . -print0 | xargs -0r grep -I -D skip -d skip -H -n 'Som.thing'

(where most of the grep command options just cause directories and binary and device files to be ignored). Note that pathnames containing spaces or any other special characters - including newlines - are handled properly.

Sometimes you just want to list all of the files of a given type: the command

sd -tl

will list the pathnames of all of the symbolic links under the current directory by running the find command

find -O2 . -type l | sed 's_^\./__'

(where the sed command just removes ./ from the start of any pathnames that start with it). The sdl alias from searchdown-aliases.sh does the same thing, and the sdf and sdd aliases do the same thing but for regular files and directories, respectively, instead of symbolic links. And all three aliases work like sd does in that if they're given a single non-option argument then they assume it's a regular expression to search for in all of the files that are found. I use commands of the form sdf foo quite a bit.

Another type of searchdown command that I find useful finds a subdirectory when I can only remember the start of its name

sdd -p Pre

the end of it (sdd -s Suf) or some fragment of it (sdd -c Part). And of course the sdf, sdl and sd commands can be used in place of sdd to search for files of other types, or of any type.

Emacs users might find the -fb option handy: for example

sd -D src -D doc -fb

will list the pathnames of all of the backup files - that is, files that end with a tilde (~) - that are under either the src or the doc subdirectories of the current directory. All of the files that are found can then be deleted by rerunning the command and adding -- rm to the end of it:

sd -D src -D doc -fb -- rm

which is equivalent to running the find command

find -O2 'src' 'doc' -name '*~' -print0 | xargs -0r rm

(The fb alias can be used in place of sd -fb.)

And as some final examples I often use searchdown's -e option to only match files with a given extension (where the extension can be specified with or without a leading '.'): sd -e mp3 (or sde mp3) will find all of the MP3 files under the current directory and (like the sd, sdf, etc. commands) the command sde txt Som.thing will find all of the lines that match the regular expression 'Som.thing' in all of the text files (that is, files with the .txt extension) under the current directory, ignoring any other files. If you often grep or operate on files with a given extension - markdown files, for example - then it can be worthwhile to define a shell function or alias for that type of file. There are already several defined in searchdown-aliases.sh that you can base yours on.

I hope this gives you some idea of what searchdown can do, and whether it's something you'd find useful. I've been using it for a few years now and I find that I go for months at a time (or even longer) between direct uses of the find command. In fact, if I hadn't used the find command for so many years before that - and if I didn't use it from time to time in shell scripts - I'm not sure how well I'd remember it's syntax! But you don't have to worry about that either: just add the -n option to any searchdown command and it'll output the equivalent find command without running it. (Or use the -v option to both output the command and run it.) This can be useful for generating find commands for use in shell scripts where you don't want to add a dependency on another script (i.e. searchdown).