Any recommended reading for Bash? I'm somewhat new to it and it's interesting ways of getting things done. I've used it minimally in the past, but have found myself writing a 100> LOC script, which I can't help but feel I'm likely over-complicating certain bits and pieces.
I would generally describe a GNU/Linux distro as being a "giant pile of shell
scripts". That's a little less true with init scripts generally now being systemd units. But that's where I'd start: Look at the code that distros write, that isn't part of some other upstream software.
Gentoo also has a lot of good shell scripting to look a, but it's mostly either POSIX shell, or targets older Bash (their guidelines http://devmanual.gentoo.org/tools-reference/bash/index.html say to avoid Bash 3 features). I tend to believe that changes made to the Bash language over the years are enhancements, and that they let you write cleaner, more robust code.
There used to be this great resource called Linux Documentation Project. It's not as active as it used to be but it produced some really book-quality documents, including the Advanced Bash Scripting Guide at https://www.tldp.org/LDP/abs/html/ .
Read it! It's great. But know that a lot of bash scripting isn't really in bash, it's really required to be proficient with grep, sed, cut, dc and a few other text processing utilities. Learn those! Then are are a few other tricks to be mindful of: be mindful of spaces (wrap your variable substitution in double quotes, mostly), be mindful of sub-shells (piping to something that sets variables can be problematic), and a few other things that can really only be learned by reading good code.
But it's also good to know when you shouldn't venture further. My rule of thumb is when I'm using arrays or hash maps, then it's a good idea to move to another language. That's probably Python nowaways. A lot of people use tons of awk or perl snippets inside their bash scripts, that can also be a sign that it's time to move the whole script over.
It can't be done. If you want to write reliable code, and actually notice all of the possible error conditions instead of silently ignoring them, your code needs to get more verbose and complicated than it would be to just use a more capable tool like Python or Node, and it still won't be as reliable.
If you have more logic than a couple of string comparisons, Bash is not the right tool for the job.
I recommend Greg's Bash Wiki ... https://mywiki.wooledge.org/BashGuide. See general notes, then at the bottom of the page are many links to additional considerations.
Like others say, "bash" is a hard tool to get right (and I'm not saying I do it right either, necessarily, but Greg's Wiki was real helpful!). I'm building a hybrid bash/python3 environment now (something I'll hopefully open-source at some point), and bash is just the "glue" to get things set up so most aspects of development can funnel through to python3 + other tools.
But ... things that make bash real useful:
* it's available everywhere (even in Windows with Ubuntu-18.04/WSL subsystem)
* it can bootstrap everything else you need
* it can wrap, in bash functions, aliases, and "variables" (parameters), the
real functionality you want to expose ... the
guts can be written in python3 or other tools
Without a good bash bootstrap script you end up writing 10 pages of arcane directions for multiple platforms telling people to download 10 packages and 4 pieces of software per platform, and nobody will have consistent reproducible environments.
EDIT: I think there's a revised version of Greg's Bash Wiki in the works.
It is availbale, almost, everywhere but be careful with the version, different Linux diatros are at different versions, the last time I used OSX it was stuck on a very old version, and I expect the different BSD OSes to run fairly new versions.
BSDs don't ship Bash as part of the base system - you have to install it from packages or ports. And that one is the most recent that the maintainer bothered to package. E.g. FreeBSD is on 4.4.23 right now, which actually appears to be newer than e.g. Debian unstable.
Python, sure, but replacing Bash with Node just seems like replacing a language crippled by its need to be backward compatible with (Bourne) sh with a language crippled by its need to be backward compatible with what Brandon Eich came up with in two weeks in 1995.
It wouldn't be my first choice either, but modern JavaScript isn't backwards-compatible, it is at least possible to write reliable and robust code if you take some care and know what options to enable, and it's mostly consistent across platforms. Last month, I spent a week trying to handle all of the possible error conditions in an overgrown bash script. It was immeasurably worse.
Python is the more direct substitute, though: it's built-in to almost every platform, and the Python 3 stagnation has even given us a consistent version: 2.7.
Sure it can. Treat it as a functional and immutable language, and you can go pretty far with it, pretty safely.
You're forgetting that bash really just calls other code. So you can combine two language's stdin/out functionality (eg, Python or Node) to pair code together. Sure, it won't be fast, but it can do quick wonders as an ad hoc data pipeline.
By far my best advice is to write auto-complete scripts if possible: https://iridakos.com/tutorials/2018/03/01/bash-programmable-... . Many of mine will do things like look up database values, so "command<tab><tab><tab>" is the workflow 99% of the time.
Keep them short. There are always exceptions but the "do one thing" mantra is handy, they can always be wrapped into a more complex workflow with a bigger script. None of my frequently used ones are over 100 LoC.
Write them for you and you alone when possible, start off simple and iterate. Fight that developer urge to solve a generalized problem for everyone.
Embrace the environment and global environment variables. We're trained to avoid global variables like the plague but they are really useful in scripts. My auto complete scripts that I mentioned above, they know which database to connect to based off the environment variable and there are separate commands to switch environment.
Make sure you aren't using it where things like make would be more appropriate.
I've recently picked up a subscription to Destroy All Software and he has a few really good demos of how to architect simple and powerful shell scripts using bash. I'd really recommend it for anyone who doesn't need help with the semantics and instead needs help with the organization of a script.
I've got a subscription too, watching him go in some of the videos as he casually does Bash blew my mind, which is another reason I ask this question. I'm trying to at least watch all his videos, I intend to work through them after watching them all at least once, and then I intend to rewrite some of the solutions (the Ruby ones) in Python and Go just to be sure I understand the concepts more, and I'm not just copying and pasting.
From the google Shell Style Guide: "If you are writing a script that is more than 100 lines long, you should probably be writing it in Python instead. "
LOC alone doen't mean much.
I have a backup script in fish of 400+ lines of code because it tries to dump data out of different sources like MySQL, PGSQL, InfluxDB, /etc files and others. Just use what feels comfortable achieving the task.
Readline is the library bash uses for editing the input line, and it has some nice movement keys. For example Alt-b moves the cursor back a word, Ctrl-u deletes to the beginning of the line, Ctrl-w removes one word behind the cursor.
They work in a bunch of other programs, like the Python interpreter's interactive mode for example.
That's true, so many times I have used shellcheck and started to improve a bash script, only to realize my time was better spent rewriting the script in Ruby.
I would highly recommend BashGuide[2] and ryanstutorials[3] as a starting point. After that, go through rest of the wooledge site for FAQs, best practices, etc
shellcheck[4] is awesome for checking your scripts for potential downfalls and issues
The full manual is here. https://www.gnu.org/software/bash/manual/bashref.html
GNU projects tend have info pages that are a lot more complete and thorough then their man pages, so that'd be a good place to check too.
Because they're structured differently. A man page for an app is just that, a page, with some formatting and sections. Texinfo is more like your typical website - lots of small cross-linked pages with a common index. Consequently, a man page is usually an extended take on --help, while info pages are more like product manuals.
This is the theory. The practice is that GNU mandates Texinfo for its projects, and because documentation tends to be the weak spot for all open source projects, manpages end up the most neglected as a result, which can be quite annoying. Especially since pretty much nobody else uses Texinfo - man pages are good enough for most console apps, and those that need more detailed documentation use Docbook, Markdown etc.
Historical difference in style and preferences. GNU’s Not UNIX and all that Jazz, so they have different conventions, some of which would not translate to a man page that well. Plus Texinfo is a tool for generating multiple output formats from a single source.
Typically (unless this changed at some point) the info pages are the canonical reference for GNU projects and the man pages are to accommodate Unix hackers.[1]
I usually do programming in Python and other languages. This particular case it makes sense since I'm taking advantage of command line utilities, otherwise I'm usually just making software myself, e.g. web services or daemons.
You could learn bash to understand existing scripts but you could also learn fish, which can be a more sane looking script than bash, if you're writing your own.
