Any suggestions for a bash scripting tutorial to help me better connect the dots? Over the years I've learned some nifty one-liners, but I'm not really familiar enough with the patterns and art of of command-line thinking to solve my own problems in bash. Most of the bash resources I've found online are just loosely organized collections of magic spells. I'd really like to get better at it though.
The resource that has been instrumental for me to move up from beginner-that-generally-can-work-out-things-with-a-lot-of-random-tries, to someone who often has things right on the first try, and if not, have a clue of what is going on , has been the following book:
Classic Shell Scripting
Arnold Robbins, Nelson H.F. Beebe
Recommend it to anyone with a basic understanding and a will to learn more. It also really doubles as a rich information resource. Cannot recommend it enough if books are your thing.
I would recommend two things: 1. Read the "bash" man page about once a year, and 2. Learn Perl or Python or something for the harder tasks.
It's helpful to learn some more about bash, and there's some good stuff in there. But shell scripts above a few dozen lines tops are a bad idea, and even those "few dozen" lines need a surprising amount of armoring with "set -e" and such.
I don't think this is because bash is bad. I've come to the conclusion that there's a fundamental tension between interactive use and safe programmatic use, and anything really good at the one is not going to be good at the other. In particular, interactive users more-or-less want the shell to accept inputs that are a bit sloppy, because they can interactively recover from the vast bulk of misinterpretations immediately, whereas a program becomes very risky with the same level of sloppiness because it will quite likely just keep going, doing very bad crazy stuff. (Which is why all my shell scripts tend to start with "set -e", which amounts to "If you get confused... STOP, instead of trashing an arbitrary amount of state.")
And again let me emphasize #1. By splitting my world up into interactive use and programmatic use, I've made my peace with shell, and I don't hate it or anything. I use a lot of the features interactively. For instance, I do a lot of little loops on the command line. Often I'll build them up with some "echo"s done to make sure that it's about to do what I think it will do. There's a lot of great ways to save time in interactive bash (or zsh or whatever), and the best part is, in interactive use you can be as crazy efficient as you want and there are no software engineering implications. Writing bash scripts with every crazy sophisticated trick in the books starts raising software engineering issues fast, and worst of all, some of them are quite invisible ("what if you run this in a directory with a file with a space in it? a newline? a file called '-rf'?"), to say nothing of whether the next person working on the script will know what all the fancy operations do. And I've got a decent-sized collection of 5-line bash scripts lying around doing various things. I just don't let them get much longer.
One of the most useful things to do is just load up an index in your head of what bash can do, then know you can look it up if you ever want it.
(Substitute your choice of shell for bash throughout.)
[Shameless plug] Not too long ago, I wrote a small book that tries to address those kinds of questions, i.e. how to use Bash to solve problems instead of 1-liners. You may find it useful: http://adventuresindatascience.com
