Help Message for Shell Scripts (opens in new tab)

(samizdat.dev)

457 pointsreconquestio5y ago123 comments

123 comments

The power is out on your boat, again. It’s 3am. You suspect that, again, the alternator housing has come loose.

You duct tape a flashlight to the bulkhead so you can work hands free and actually see what you are doing. All you have on you is a broken pocket knife but it’ll do because all you need to accomplish right now is to tighten the housing screws enough. You know this for a fact because you’ve done it three times already in the last 24 hours.

It’s not even a documented procedure — you’ll replace the housing mounts entirely when you’re back at port in three days’ time. You guarantee it — this is the first thing you’ll do even, when you get back to shore. You have my word on that, captain!

The duct tape came unstuck. It was damp and doesn’t work so well (at all) when it’s wet. The flashlight survived the fall. More tape this time should do the job. Tape mount version 2 will still unstick of course, eventually. Nothing stops the damp at sea, but if you use enough tape then you’ll have fixed the power by the time the tape fails. That’s your plan B and you’re sticking to it.

Sure, you could do this job better if you had an impact driver with an automatically illuminated bit chuck, but buying one of those is further down the todo list than fixing the power on the boat, making it back to port, and ensuring the power doesn’t fail this way again, as promised. Or at least won’t fail for the next few shifts.

On your days off you relax by programming in Bash.

chrisweekly5y ago

+1 for the sailing analogy!

"Necessity is the mother of invention" must have been coined by a sailor.

adrianmonk5y ago

You can also use a "here document"

    help() {
      cat <<'EOH'
    my-script — does one thing well
    
    Usage:
      my-script <input> <output>
    
    Options:
      <input>   Input file to read.
      <output>  Output file to write. Use '-' for stdout.
      -h        Show this message.
    EOH
    }

If the indentation bugs you, you can use a simpler sed trick to remove leading space so that you can indent it as desired:

    help() {
      sed -e 's/    //' <<'EOH'
        my-script — does one thing well
        
        Usage:
          my-script <input> <output>
        
        Options:
          <input>   Input file to read.
          <output>  Output file to write. Use '-' for stdout.
          -h        Show this message.
    EOH
    }

pavon5y ago

Or just a multiline string:

  #!/bin/bash
  USAGE="my-script — does one thing well
    
    Usage:
      my-script <input> <output>
    
    Options:
      <input>   Input file to read.
      <output>  Output file to write. Use '-' for stdout.
      -h        Show this message.
  "

  help() {
    echo "$USAGE"
  }

This is my standard approach which is cleaner for putting the documentation at the very top of the file like the linked article.

sillysaurusx5y ago

Thank you! I had no idea that multiline strings were valid bash.

1 more reply

GordonS5y ago

Woah, I had no idea multiline strings were even a thing in Bash, I've been using heredocs for help messages since forever!

Do you know if these are portable?

1 more reply

Anthony-G5y ago

That's exactly what I do. For others who were not aware that multi-line strings can be used, this is POSIX-compatible (most of my shell scripts are executed by `dash`).

tuatoru5y ago

This is one (useful!) interpretation of "code should be self-documenting". Just put the documentation into strings.

psophis5y ago

You can also use add a hyphen ( <<-EOF ) to suppress leading tabs but not spaces. https://linuxhint.com/bash-heredoc-tutorial/

smichel175y ago

I've always avoided that for fear that my or someone else's editor will accidentally replace the tabs with space. Mixed is not a common configuration, these days.

1 more reply

yjftsjthsd-h5y ago

Oh, that's nice! And in spite of the labeling on that page, it doesn't seem to be a BASHism; it at least works in dash, too.

1 more reply

amelius5y ago

Seriously, bash has too many obscure features.

3 more replies

koala_man5y ago

I believe the point of the article's method is that it allows you to document your script with code comments, and then reuse the same text for help output

jolmg5y ago

The neat thing about not indenting it is that you can make use of your editor's text-width auto-wrapping. For example, if you have it set to 80 columns, indenting it would make the text-width of the help text 76 in your case.

Also, putting the help text in code like this instead of a comment allows one to expand $0 so that the command name in the help text always matches the filename and path used in the invocation.

amelius5y ago

But now the sed line bugs me ;)

ElCampechano5y ago

Or just write it in Python.

hedora5y ago

Python is terrible at one liners. The sed is much more compact.

1 more reply

j1elo5y ago

I learnt the same trick some years ago, from an article called Shell Scripts Matter:

https://dev.to/thiht/shell-scripts-matter

So I took some of the advice and tips offered in there, and wrote a template file to be used as a baseline when writing scripts for any project that might need one:

https://github.com/j1elo/shell-snippets/blob/master/template...

Other resources that I link in the readme of that repo, because they were a great guide to write better and more robust scripts, are:

- Writing Robust Bash Shell Scripts: https://www.davidpashley.com/articles/writing-robust-shell-s...

- Common shell script mistakes: http://www.pixelbeat.org/programming/shell_script_mistakes.h...

- Bash Pitfalls: http://mywiki.wooledge.org/BashPitfalls

- The Bash Hackers Wiki: https://wiki.bash-hackers.org/

EDIT: -for anyone who would like to read some actual examples- I have to manage a bunch of scripts so actually a slightly more up to date version of the template is put into practice by means of a common bash.conf file that then gets sourced by all scripts: https://github.com/Kurento/adm-scripts/blob/master/bash.conf...

themodelplumber5y ago

Thank you for this really helpful comment. It's like an encyclopedia's worth of bash information in one go--much appreciated.

j1elo5y ago

You're welcome! Shell scripting has a weird language, unsafe by default, and very prone to mistakes... but knowing it well pays off.

People say that for complex things it's better to write Python, but that doesn't fly in embedded or Docker environments. Python is not even present in the default Ubuntu Docker images. Also if all you want to do is really write glue code between CLI programs, shell scripting is the way to go.

Happy coding!

1 more reply

HellsMaddy5y ago

A bash pitfall which I have experienced but didn’t see mentioned is the behavior of the `set -e` (errexit) option when using command substitution. If you expect failures within the command substitution to cause the script to exit, you’re gonna be confused.

https://twitter.com/hellsmaddy/status/1273744824835796993?s=...

Tl;dr use `shopt -s inherit_errexit`

Anthony-G5y ago

Thanks. I wasn't aware of that option. I've added the following to my Bash-specific shell scripts:

    # Cause command substitution to inherit the value of the `errexit` option.
    # Introduced in Bash 4.4
    if [ "${BASH_VERSINFO[0]}" -gt 4 ] ||
      { [ "${BASH_VERSINFO[0]}" -eq 4 ] && [ "${BASH_VERSINFO[1]}" -ge 4 ]; }; then
        shopt -s inherit_errexit
    fi

mey5y ago

Handling of arguments is one of the reasons I reach for Python or Powershell instead of a bash script when writing my own stuff.

https://docs.python.org/3/library/argparse.html is great.

Powershell has the Param keyword that functions like argparse in Python

https://docs.microsoft.com/en-us/powershell/module/microsoft...

Spivak5y ago

But handling args isn't that bad in bash.

    while [[ $# -gt 0 ]]; do
      case "$1" in
         -h|--help)
           do_help
           exit
           ;;
         -v|--version)
           do_version
           exit
           ;;
         -d|--debug)
           debug=true
           shift
           ;;
         -a|--arg)
           arg_value=$2
           shift 2
           ;;
      esac
    done

mey5y ago

    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument('-v','--version',action='version', version='demo',help='Print version information')
    parser.add_argument('-d','--debug', help='Enable Debug Mode')
    parser.add_argument('a','arg', help="Argument Documentation")
    args = parser.parse_args()

Personally I feel like this is more readable code, gets me better validation, and help docs for "free". That's the attraction.

1 more reply

jolmg5y ago

To expand on that pattern:

    while (( $# )); do
      case "$1" in
        -h|--help)
          usage
          exit
          ;;

        -v|--version)
          do_version
          exit
          ;;

        -d|--debug)
          debug=true
          ;;

        -a|--arg)
          arg_value="$2"
          shift
          ;;

        *)
          if [[ ! -v pos1 ]]; then
            pos1="$1"
          elif [[ ! -v pos2 ]]; then
            pos2="$1"
          else
            >&2 printf "%s: unrecognized argument\n" "$1"
            >&2 usage
            exit 1
          fi
      esac

      shift
    done

1 more reply

dragonwriter5y ago

> https://docs.python.org/3/library/argparse.html is great

Argparse is okay (and being in stdlib makes it always-available), but it's no click. https://click.palletsprojects.com/en/7.x/

mey5y ago

This is why I love the HN community, learning something new every day. (Happy 10,000) I use argparse because it is stdlib, but will checkout click!

40four5y ago

Thumbs up for 'Click'. I used it for a project once, and I was really happy with it. Easy to use, good docs. Would use it again.

2 more replies

wahern5y ago

The original argparse is available in every Unix shell: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/g...

nickysielicki5y ago

A great option when you're stuck with an old crusty script that you don't want to completely rewrite in python, but do want to clean up enough so that you can call it with `-h` and remember how to use it a few months in the future.

Unfortunately, this won't help you if you're on embedded where python isn't in the base system.

sidpatil5y ago

PowerShell also has comment-based help, which is like a manpage embedded as a comment within the script. It's like OP's suggested help format, but better.

https://docs.microsoft.com/en-us/powershell/module/microsoft...

hnarn5y ago

I also really like the "flag" package for Go, it generates the help text for you and easily lets you set defaults as well as helps you type-check inputs.

PenguinCoder5y ago

I like https://github.com/jpillora/opts for Go arg parsing. It has better, sane defaults with long/short opts.

1 more reply

acdha5y ago

My rule for a long time has been that any time you have more than one screen's worth of code and/or are using any of arrays, functions, or more than simple positional arguments you'll save time by rewriting it in Python. shellcheck has helped soften that a bit but the usability difference is pretty noticeable even with multiple decades writing shell scripts.

daitangio5y ago

Try pyhon click: from flask& jinja2 author, it is very nice

diablerouge5y ago

This seems like a neat sed trick, but I'm not sure that it's useful for this particular case?

When I write a shell script, I often write a help function if it's not a totally trivial script, but there's no need for this cryptic sed expression, right? You can just call `echo` a few times and do it the obvious way. That works better for maintainability and if you put it at the top of the file then it's immediately visible when opening or using `head` on it.

Neat trick though - sed is super useful for all kinds of things. I had a co-worker who bound a keybinding to a sed one-liner that would automagically reconfigure some files that needed to be changed regularly. I ended up using that trick to allow for changing my terminal console colorscheme with a single command.

e12e5y ago

It's a little sad that standard shell here documents only support elliding leading tabs (it wouldn't be so sad if the record separator hadn't been thrown under the bus - having a character for indentation distinct from space is good... In theory).

But at any rate my typical usage() is generally along these lines (warning watch out for expansions):

  usage()
  {
    cat - <<-EOF
    `basename ${0}`: demonstrate here docs
     Usage:
     `basename ${0}` <required argument> [-o|--optional-param] 

       Etc. Possibly referencing
       default value: ${DEFAULT_VALUE}
    EOF
  }

jolmg5y ago

I think it'd be better without using basename, just $0. That way it matches the way it was called, which is how the user chose to access it for whatever reason. The bare filename might refer to a different command, even. Also, if you include examples in the help text, they'll also be able to copy and paste, instead of having to manually insert what basename stripped away.

1 more reply

zzzcpan5y ago

Right, it's a bit useless trick. I guess the author is just exploring how to organize help, maybe thinking about larger scripts or maybe doing it for the article. Either way it's hard to see it going anywhere with sed. If you were to explore parsing and organizing help, I'd suggest starting with a simple pure shell loop like this:

   while read -r line; do
      case "$line" in 
      "###"*) 
         echo "${line#\###}" ;;
      esac
   done <"$0"

dougdonohoe5y ago

We do this for Makefile entries - looking for '##' that we put before each make command.

  ## help: prints this help message
  help:
     @echo "Usage: \n"
     @egrep -h "^## [a-zA-Z0-9\-]*:" ${MAKEFILE_LIST} | sed -e 's/##//' | column -t -s ':' |  sed -e 's/^/ /'

  ## build: builds JAR with dependencies
  build:
     mvn compile

xvolter5y ago

I also posted to the github gist, this the sed command here is not cross-plataform friendly. You can accomplish the same thing with an awk command though:

awk '/^###/' "$0"

ddevault5y ago

The bit at the end isn't POSIX sh, either. Fixed version:

if [ $# -eq 0 ] || [ "$1" = "-h" ]; then help exit 1 fi

Relevant spec bits:

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/t...

yesenadam5y ago

This also strips the leading hashes + optional space:

    awk 'sub("^### ?","")' "$0"

pwdisswordfish25y ago

sed -n '/^###/p' is cross-platform friendly.

u801e5y ago

This will also work:

  sed '/^###/!d'

1 more reply

jpxw5y ago

Nice, this is much cleaner

account425y ago

> $0 means a filename of a file that is being executed.

This is only a convention and is entirely up to the calling program.

For example in bash scripts you can use `exec -a name ...` to pass "name" as the 0th argument.

If you are already using #!/bin/bash you might as well use ${BASH_SOURCE[0]} to get the path to the current script.

xelxebar5y ago

This reminds me of a nice sed one-liner I recently happened to craft.

Do you ever collect families of functions in your shell scripts under different sections? Here's a nice way of printing out all the functions under a given section:

    funs(){ sed -n '/^## /h;x;/'"$1"'/{x;s/^\(\w\+\)().*/\1/p;x};x' "$0";}

Where "sections" are delimited by comments of the form "## Section Name" at the beginning of a line. A particularly nice use case is when you write scripts that expect "subcommand" arguments, like

    $ foo.sh bar baz

and wish to keep track of the available subcommands in the help documentation. Simply collect all your subcommands under the heading "## Subcommands" and stick a funs call in your documentation:

    usage=$(cat <<USAGE
    Usage: foo <subcommand>
    Subcommands: $(funs Subcommands)
    USAGE
    )

The sed one-liner above uses the oft-ignored "hold space" which lets you store data that persists between lines. Here's the same sed but expanded with comments:

    funs(){ sed -n '/^## /h  # Store header line in hold space
        x               # Swap out current line with header in hold space.
        /'"$1"'/{       # Run block if last encountered header matches $1
            x           # Return to processing current line (instead of header)
            s/^\(\w\+\)().*/\1/p    # Print function names
            x           # Whether or not this block runs, we want to return to
                        # processing the current line. If the block does not
                        # run, then the hold space contains our current line
                        # with the active line being our header. So we must
        }               # return to that state as whell when the block is run.
        x               # Restore current line from hold space' "$0"
    }

fomine35y ago

I'm particular about: If I run a command and its arguments is wrong, it should output error and help messages to STDERR. But if I run a command with --help argument, it should output help messages to STDOUT.

ahnick5y ago

I like the idea of combining the header with the help documentation to reduce the number of areas to maintain in smaller scripts. For larger scripts though, I think I'd still prefer to have a separate function, so that the help documentation doesn't overwhelm the initial viewing of the actual code.

I also like to feed a heredoc directly into man, which allows you to achieve nicer formatting for the help documentation. Something like this...

  man -l - << EOF
  .\" Manpage for encpass.sh.
  .\" Email contact@plyint.com to correct errors or typos.
  .TH man 8 "06 March 2020" "1.0" "encpass.sh man page"
  .SH NAME
  encpass.sh \- Use encrypted passwords in shell scripts
  ...
  EOF

See encpass.sh for a working example of this -> https://github.com/plyint/encpass.sh/blob/master/encpass.sh

sicromoft5y ago

Note that this doesn't work on macOS, where the builtin `man` command doesn't support the `-l` option.

ahnick5y ago

Ah interesting, is there any workaround for Mac? Otherwise, I may just have to fallback to stripping the man page formatting and sending it to less.

3 more replies

lolsal5y ago

> so that the help documentation doesn't overwhelm the initial viewing of the actual code.

That is a very strange argument to me. You find that more cumbersome than jumping around to random functions?

ahnick5y ago

No, I think the readability of the help documentation itself is largely the same whether it is placed in the header or in a dedicated function. When I'm viewing the shell script code though, often I want to jump right in and see the actual code, not look at the help documentation.

By having the help documentation in a function in the middle or towards the end of the file, I don't have to page down through the help documentation to get to the code that is actually doing things. If I'm really interested in the help documentation, then I'd prefer to look at the nicely formatted version output by the script (<script> --help or whatever) rather than looking in the actual script code anyway.

Admittedly, this may be more of a subjective personal preference item.

hansdieter13375y ago

Even better: Don’t use bash. I started using Python instead of bash. It’s way better to read and more maintainable. If I need the performance of native-Unix commands, I can still use them using subprocess.

ChAns0n5y ago

Python is not as handy/efficient as bash when you want to utilize some existing Unix commands. And normally it’s crucial to stop bleeding fast if something bad is happened to you server/cluster

argonauts125y ago

I've been migrating shell scripts to python and have found the 'sh' library invaluable for pulling in pure CLI commands from the bash script and adding to the python implementation. http://amoffat.github.io/sh/ It's essentially an abstraction layer above subprocess. Quick example - to use ifconfig natively in python:

from sh import ifconfig print(ifconfig("wlan0"))

ben5095y ago

Python is surprisingly bad at managing subprocesses, though. Read the subprocess docs closely, it's quite easy to get deadlocks if you try to compose operations the way you can in bash.

aflag5y ago

There are libraries that fix that: https://pypi.org/project/sh/ if you are in position to use external libraries.

drewbug5y ago

bash is everywhere

nhumrich5y ago

So is python

2 more replies

raggi5y ago

I used this strategy in "fx" which is a development helper frontend for fuchsia builds and tools. I used four # for the "short description" and three for the long description. The reason I used the strategy there is that lot of our scripts delegate arguments to native commands, and so adding help purely to --help wasn't really a good ROI. Implementation: https://fuchsia.googlesource.com/fuchsia/+/refs/heads/master...

jandrese5y ago

Hmm:

% sed -rn 's/^### ?//;T;p' testfile

sed: 1: "s/^### ?//;T;p": invalid command code T

Looks like it might need GNU Sed or something. But honestly if I want to read the top of the file less works just as well.

adrianmonk5y ago

Yeah, "man sed" on my machine says, "This is a GNU extension."

You could do the same thing with awk instead:

    awk '{ if (sub("^### ?", "")) { print; } else { exit; } }'

jandrese5y ago

Well sure, there are tons of ways to do this in other languages. :)

Perl for example was made for problems like this.

   perl -ne 'print if ( s/^### ?// )'

bewuethr5y ago

    sed -rn 's/^### ?//p'

1 more reply

heinrichhartman5y ago

I like to do:

    help() { cat $0 }

"May the source be with you." : )

OliverJones5y ago

Cool. I did this on 7th Edition UNIX in 1977. I forget how. It's interesting that ....

* NIX makes people do creative things

All that Bell Labs stuff still works the way it always did.

* People are still reinventing this particular wheel, and

* This embedded help stuff still somehow hasn't made it into the infrastructure along with autocomplete.

jeffrom5y ago

Something I've wanted to do for a while is build a library to parse and generate use lines / documentation that adheres to the posix useline spec (can't find the link at the moment) while also being able to idempotently (de)serialize and be descriptive enough to define arguments and flags in a way a human could easily understand. iirc the spec seemed probably too vague to just work with all the currently existing man pages, but it would be nice to have a spec all programs can follow that machines can parse on my os.

layoutIfNeeded5y ago

http://docopt.org/ ?

andsens5y ago

My time to shine! I built an argument parser that uses a POSIX compliant help message as the input. It's a parser generator really. It generates minified bash that is inlined in your script, so no dependencies. The work is based off of docopt (and is docopt compliant). Check it out: https://github.com/andsens/docopt.sh

flaxton5y ago

Didn't work on macOS (multiple sed errors - switches differ from Linux) but prompted me to write a help function ;-)

gfosco5y ago

I have a project that makes heavy use of sed.. On macOS, I `brew install gnu-sed` and use `gsed` instead, so it works like other platforms.

flaxton5y ago

Nice! I will check that out.

ivansavz5y ago

here is a gist that works on macOS (using `awk` instead of sed)

https://gist.github.com/ivanistheone/0454191800c9caad77a52e0...

owenshen245y ago

Unrelated: Is there any connection between the author and the other sam[]zdat who writes about society and other intriguing topics?

https://samzdat.com/

krick5y ago

I wouldn't know, but there is no reason for me to be thinking something like that. "Samizdat" is not really a name or something, it's a transliteration of "самиздат", which is a short/colloquial for "самостоятельное издательство", which literally means "self-publishing" (this was a thing during the USSR, where "self-publishing" was basically opposed to "real, official government-approved publishing"). I believe it's just a "clever" domain somebody was able to acquire, nothing more.

owenshen245y ago

Ah, thanks for the explanation.

77866555y ago

Pretty sure this won't work if the script is called via $PATH

pwdisswordfish25y ago

    x=$(command -v $0 2>/dev/null)
    sed -rn 's/^### ?//;T;p' $x

Personally I would not use the author's chosen sed commands.

    exec sed -n '/^###/p' $x

would work fine.

sneak5y ago

Why is “./script.sh -h” better than “less script.sh”?

soraminazuki5y ago

If you use zsh, you can use the help output to generate completions for your own scripts.

https://github.com/zsh-users/zsh-completions/blob/master/zsh...

mttpgn5y ago

I just spent three days explaining a bash script I authored to a tech lead who "doesn't know bash that well". If I had written in more help messages I probably could have preserved more of my time.

stkai5y ago

Handy! Now, can it author the help text as well? ;)

thangalin5y ago

There's an endless variation on how shell scripts can present help information. Here's another, consider this array:

    ARGUMENTS+=(
      "a,arch,Target operating system architecture (amd64)"
      "b,build,Suppress building application"
      "o,os,Target operating system (linux, windows, mac)"
      "u,update,Java update version number (${ARG_JRE_UPDATE})"
      "v,version,Full Java version (${ARG_JRE_VERSION})"
    )

The lines are machine-readable and alignment is computed by the template:

https://github.com/DaveJarvis/scrivenvar/blob/master/build-t...

When install script[0] help is requested, the following is produced:

    $ ./installer -h
    Usage: installer [OPTIONS...]

      -V, --verbose  Log messages while processing
      -h, --help     Show this help message then exit
      -a, --arch     Target operating system architecture (amd64)
      -b, --build    Suppress building application
      -o, --os       Target operating system (linux, windows, mac)
      -u, --update   Java update version number (8)
      -v, --version  Full Java version (14.0.1)

Using an array reduces some duplication, though more can be eliminated. Scripts typically have two places where the arguments are referenced: help and switch statements. The switch statements resemble:

https://github.com/DaveJarvis/scrivenvar/blob/master/install...

Usually parsing arguments entails either assigning a variable or (not) performing an action later. Introducing another convention would allow hoisting the switch statement out of the installer script and into the template. Off the cuff, this could resemble:

    ARGUMENTS+=(
      "ARG_ARCH,a,arch,Target operating system architecture (amd64)"
      "do_build=noop,b,build,Suppress building application"
      "ARG_JRE_OS,o,os,Target operating system (linux, windows, mac)"
      "ARG_JRE_UPDATE,u,update,Java update version number (${ARG_JRE_UPDATE})"
      "ARG_JRE_VERSION,v,version,Full Java version (${ARG_JRE_VERSION})"
    )

The instructions to execute when arguments are parsed are thus associated with the arguments themselves, in a quasi-FP style. This approach, not including the FP convention, is discussed at length in my Typesetting Markdown series[1].

[0]: https://github.com/DaveJarvis/scrivenvar/blob/master/install...

[1]: https://dave.autonoma.ca/blog/2019/05/22/typesetting-markdow...

oweiler5y ago

This is not elegant, this is an ugly hack at best.

j / k navigate · click thread line to collapse

123 comments

gorgoiler5y ago

The power is out on your boat, again. It’s 3am. You suspect that, again, the alternator housing has come loose.

On your days off you relax by programming in Bash.

chrisweekly5y ago

+1 for the sailing analogy!

"Necessity is the mother of invention" must have been coined by a sailor.

adrianmonk5y ago

You can also use a "here document"

    help() {
      cat <<'EOH'
    my-script — does one thing well
    
    Usage:
      my-script <input> <output>
    
    Options:
      <input>   Input file to read.
      <output>  Output file to write. Use '-' for stdout.
      -h        Show this message.
    EOH
    }

If the indentation bugs you, you can use a simpler sed trick to remove leading space so that you can indent it as desired:

    help() {
      sed -e 's/    //' <<'EOH'
        my-script — does one thing well
        
        Usage:
          my-script <input> <output>
        
        Options:
          <input>   Input file to read.
          <output>  Output file to write. Use '-' for stdout.
          -h        Show this message.
    EOH
    }

pavon5y ago

Or just a multiline string:

  #!/bin/bash
  USAGE="my-script — does one thing well
    
    Usage:
      my-script <input> <output>
    
    Options:
      <input>   Input file to read.
      <output>  Output file to write. Use '-' for stdout.
      -h        Show this message.
  "

  help() {
    echo "$USAGE"
  }

This is my standard approach which is cleaner for putting the documentation at the very top of the file like the linked article.

sillysaurusx5y ago

Thank you! I had no idea that multiline strings were valid bash.

1 more reply

GordonS5y ago

Woah, I had no idea multiline strings were even a thing in Bash, I've been using heredocs for help messages since forever!

Do you know if these are portable?

1 more reply

Anthony-G5y ago

That's exactly what I do. For others who were not aware that multi-line strings can be used, this is POSIX-compatible (most of my shell scripts are executed by `dash`).

tuatoru5y ago

This is one (useful!) interpretation of "code should be self-documenting". Just put the documentation into strings.

psophis5y ago

You can also use add a hyphen ( <<-EOF ) to suppress leading tabs but not spaces. https://linuxhint.com/bash-heredoc-tutorial/

smichel175y ago

I've always avoided that for fear that my or someone else's editor will accidentally replace the tabs with space. Mixed is not a common configuration, these days.

1 more reply

yjftsjthsd-h5y ago

Oh, that's nice! And in spite of the labeling on that page, it doesn't seem to be a BASHism; it at least works in dash, too.

1 more reply

amelius5y ago

Seriously, bash has too many obscure features.

3 more replies

koala_man5y ago

I believe the point of the article's method is that it allows you to document your script with code comments, and then reuse the same text for help output

jolmg5y ago

Also, putting the help text in code like this instead of a comment allows one to expand $0 so that the command name in the help text always matches the filename and path used in the invocation.

amelius5y ago

But now the sed line bugs me ;)

ElCampechano5y ago

Or just write it in Python.

hedora5y ago

Python is terrible at one liners. The sed is much more compact.

1 more reply

j1elo5y ago

I learnt the same trick some years ago, from an article called Shell Scripts Matter:

https://dev.to/thiht/shell-scripts-matter

So I took some of the advice and tips offered in there, and wrote a template file to be used as a baseline when writing scripts for any project that might need one:

https://github.com/j1elo/shell-snippets/blob/master/template...

Other resources that I link in the readme of that repo, because they were a great guide to write better and more robust scripts, are:

- Writing Robust Bash Shell Scripts: https://www.davidpashley.com/articles/writing-robust-shell-s...

- Common shell script mistakes: http://www.pixelbeat.org/programming/shell_script_mistakes.h...

- Bash Pitfalls: http://mywiki.wooledge.org/BashPitfalls

- The Bash Hackers Wiki: https://wiki.bash-hackers.org/

themodelplumber5y ago

Thank you for this really helpful comment. It's like an encyclopedia's worth of bash information in one go--much appreciated.

j1elo5y ago

You're welcome! Shell scripting has a weird language, unsafe by default, and very prone to mistakes... but knowing it well pays off.

Happy coding!

1 more reply

HellsMaddy5y ago

https://twitter.com/hellsmaddy/status/1273744824835796993?s=...

Tl;dr use `shopt -s inherit_errexit`

Anthony-G5y ago

Thanks. I wasn't aware of that option. I've added the following to my Bash-specific shell scripts:

    # Cause command substitution to inherit the value of the `errexit` option.
    # Introduced in Bash 4.4
    if [ "${BASH_VERSINFO[0]}" -gt 4 ] ||
      { [ "${BASH_VERSINFO[0]}" -eq 4 ] && [ "${BASH_VERSINFO[1]}" -ge 4 ]; }; then
        shopt -s inherit_errexit
    fi

mey5y ago

Handling of arguments is one of the reasons I reach for Python or Powershell instead of a bash script when writing my own stuff.

https://docs.python.org/3/library/argparse.html is great.

Powershell has the Param keyword that functions like argparse in Python

https://docs.microsoft.com/en-us/powershell/module/microsoft...

Spivak5y ago

But handling args isn't that bad in bash.

    while [[ $# -gt 0 ]]; do
      case "$1" in
         -h|--help)
           do_help
           exit
           ;;
         -v|--version)
           do_version
           exit
           ;;
         -d|--debug)
           debug=true
           shift
           ;;
         -a|--arg)
           arg_value=$2
           shift 2
           ;;
      esac
    done

mey5y ago

    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument('-v','--version',action='version', version='demo',help='Print version information')
    parser.add_argument('-d','--debug', help='Enable Debug Mode')
    parser.add_argument('a','arg', help="Argument Documentation")
    args = parser.parse_args()

Personally I feel like this is more readable code, gets me better validation, and help docs for "free". That's the attraction.

1 more reply

jolmg5y ago

To expand on that pattern:

    while (( $# )); do
      case "$1" in
        -h|--help)
          usage
          exit
          ;;

        -v|--version)
          do_version
          exit
          ;;

        -d|--debug)
          debug=true
          ;;

        -a|--arg)
          arg_value="$2"
          shift
          ;;

        *)
          if [[ ! -v pos1 ]]; then
            pos1="$1"
          elif [[ ! -v pos2 ]]; then
            pos2="$1"
          else
            >&2 printf "%s: unrecognized argument\n" "$1"
            >&2 usage
            exit 1
          fi
      esac

      shift
    done

1 more reply

dragonwriter5y ago

> https://docs.python.org/3/library/argparse.html is great

Argparse is okay (and being in stdlib makes it always-available), but it's no click. https://click.palletsprojects.com/en/7.x/

mey5y ago

This is why I love the HN community, learning something new every day. (Happy 10,000) I use argparse because it is stdlib, but will checkout click!

40four5y ago

Thumbs up for 'Click'. I used it for a project once, and I was really happy with it. Easy to use, good docs. Would use it again.

2 more replies

wahern5y ago

The original argparse is available in every Unix shell: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/g...

nickysielicki5y ago

Unfortunately, this won't help you if you're on embedded where python isn't in the base system.

sidpatil5y ago

PowerShell also has comment-based help, which is like a manpage embedded as a comment within the script. It's like OP's suggested help format, but better.

https://docs.microsoft.com/en-us/powershell/module/microsoft...

hnarn5y ago

I also really like the "flag" package for Go, it generates the help text for you and easily lets you set defaults as well as helps you type-check inputs.

PenguinCoder5y ago

I like https://github.com/jpillora/opts for Go arg parsing. It has better, sane defaults with long/short opts.

1 more reply

acdha5y ago

daitangio5y ago

Try pyhon click: from flask& jinja2 author, it is very nice

diablerouge5y ago

This seems like a neat sed trick, but I'm not sure that it's useful for this particular case?

e12e5y ago

But at any rate my typical usage() is generally along these lines (warning watch out for expansions):

  usage()
  {
    cat - <<-EOF
    `basename ${0}`: demonstrate here docs
     Usage:
     `basename ${0}` <required argument> [-o|--optional-param] 

       Etc. Possibly referencing
       default value: ${DEFAULT_VALUE}
    EOF
  }

jolmg5y ago

1 more reply

zzzcpan5y ago

   while read -r line; do
      case "$line" in 
      "###"*) 
         echo "${line#\###}" ;;
      esac
   done <"$0"

dougdonohoe5y ago

We do this for Makefile entries - looking for '##' that we put before each make command.

  ## help: prints this help message
  help:
     @echo "Usage: \n"
     @egrep -h "^## [a-zA-Z0-9\-]*:" ${MAKEFILE_LIST} | sed -e 's/##//' | column -t -s ':' |  sed -e 's/^/ /'

  ## build: builds JAR with dependencies
  build:
     mvn compile

xvolter5y ago

I also posted to the github gist, this the sed command here is not cross-plataform friendly. You can accomplish the same thing with an awk command though:

awk '/^###/' "$0"

ddevault5y ago

The bit at the end isn't POSIX sh, either. Fixed version:

if [ $# -eq 0 ] || [ "$1" = "-h" ]; then help exit 1 fi

Relevant spec bits:

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/t...

yesenadam5y ago

This also strips the leading hashes + optional space:

    awk 'sub("^### ?","")' "$0"

pwdisswordfish25y ago

sed -n '/^###/p' is cross-platform friendly.

u801e5y ago

This will also work:

  sed '/^###/!d'

1 more reply

jpxw5y ago

Nice, this is much cleaner

account425y ago

> $0 means a filename of a file that is being executed.

This is only a convention and is entirely up to the calling program.

For example in bash scripts you can use `exec -a name ...` to pass "name" as the 0th argument.

If you are already using #!/bin/bash you might as well use ${BASH_SOURCE[0]} to get the path to the current script.

xelxebar5y ago

This reminds me of a nice sed one-liner I recently happened to craft.

Do you ever collect families of functions in your shell scripts under different sections? Here's a nice way of printing out all the functions under a given section:

    funs(){ sed -n '/^## /h;x;/'"$1"'/{x;s/^\(\w\+\)().*/\1/p;x};x' "$0";}

Where "sections" are delimited by comments of the form "## Section Name" at the beginning of a line. A particularly nice use case is when you write scripts that expect "subcommand" arguments, like

    $ foo.sh bar baz

and wish to keep track of the available subcommands in the help documentation. Simply collect all your subcommands under the heading "## Subcommands" and stick a funs call in your documentation:

    usage=$(cat <<USAGE
    Usage: foo <subcommand>
    Subcommands: $(funs Subcommands)
    USAGE
    )

The sed one-liner above uses the oft-ignored "hold space" which lets you store data that persists between lines. Here's the same sed but expanded with comments:

    funs(){ sed -n '/^## /h  # Store header line in hold space
        x               # Swap out current line with header in hold space.
        /'"$1"'/{       # Run block if last encountered header matches $1
            x           # Return to processing current line (instead of header)
            s/^\(\w\+\)().*/\1/p    # Print function names
            x           # Whether or not this block runs, we want to return to
                        # processing the current line. If the block does not
                        # run, then the hold space contains our current line
                        # with the active line being our header. So we must
        }               # return to that state as whell when the block is run.
        x               # Restore current line from hold space' "$0"
    }

fomine35y ago

ahnick5y ago

I also like to feed a heredoc directly into man, which allows you to achieve nicer formatting for the help documentation. Something like this...

  man -l - << EOF
  .\" Manpage for encpass.sh.
  .\" Email contact@plyint.com to correct errors or typos.
  .TH man 8 "06 March 2020" "1.0" "encpass.sh man page"
  .SH NAME
  encpass.sh \- Use encrypted passwords in shell scripts
  ...
  EOF

See encpass.sh for a working example of this -> https://github.com/plyint/encpass.sh/blob/master/encpass.sh

sicromoft5y ago

Note that this doesn't work on macOS, where the builtin `man` command doesn't support the `-l` option.

ahnick5y ago

Ah interesting, is there any workaround for Mac? Otherwise, I may just have to fallback to stripping the man page formatting and sending it to less.

3 more replies

lolsal5y ago

> so that the help documentation doesn't overwhelm the initial viewing of the actual code.

That is a very strange argument to me. You find that more cumbersome than jumping around to random functions?

ahnick5y ago

Admittedly, this may be more of a subjective personal preference item.

hansdieter13375y ago

ChAns0n5y ago

Python is not as handy/efficient as bash when you want to utilize some existing Unix commands. And normally it’s crucial to stop bleeding fast if something bad is happened to you server/cluster

argonauts125y ago

from sh import ifconfig print(ifconfig("wlan0"))

ben5095y ago

Python is surprisingly bad at managing subprocesses, though. Read the subprocess docs closely, it's quite easy to get deadlocks if you try to compose operations the way you can in bash.

aflag5y ago

There are libraries that fix that: https://pypi.org/project/sh/ if you are in position to use external libraries.

drewbug5y ago

bash is everywhere

nhumrich5y ago

So is python

2 more replies

raggi5y ago

jandrese5y ago

Hmm:

% sed -rn 's/^### ?//;T;p' testfile

sed: 1: "s/^### ?//;T;p": invalid command code T

Looks like it might need GNU Sed or something. But honestly if I want to read the top of the file less works just as well.

adrianmonk5y ago

Yeah, "man sed" on my machine says, "This is a GNU extension."

You could do the same thing with awk instead:

    awk '{ if (sub("^### ?", "")) { print; } else { exit; } }'

jandrese5y ago

Well sure, there are tons of ways to do this in other languages. :)

Perl for example was made for problems like this.

   perl -ne 'print if ( s/^### ?// )'

bewuethr5y ago

    sed -rn 's/^### ?//p'

1 more reply

heinrichhartman5y ago

I like to do:

    help() { cat $0 }

"May the source be with you." : )

OliverJones5y ago

Cool. I did this on 7th Edition UNIX in 1977. I forget how. It's interesting that ....

* NIX makes people do creative things

All that Bell Labs stuff still works the way it always did.

* People are still reinventing this particular wheel, and

* This embedded help stuff still somehow hasn't made it into the infrastructure along with autocomplete.

jeffrom5y ago

layoutIfNeeded5y ago

http://docopt.org/ ?

andsens5y ago

flaxton5y ago

Didn't work on macOS (multiple sed errors - switches differ from Linux) but prompted me to write a help function ;-)

gfosco5y ago

I have a project that makes heavy use of sed.. On macOS, I `brew install gnu-sed` and use `gsed` instead, so it works like other platforms.

flaxton5y ago

Nice! I will check that out.

ivansavz5y ago

here is a gist that works on macOS (using `awk` instead of sed)

https://gist.github.com/ivanistheone/0454191800c9caad77a52e0...

owenshen245y ago

Unrelated: Is there any connection between the author and the other sam[]zdat who writes about society and other intriguing topics?

https://samzdat.com/

krick5y ago

owenshen245y ago

Ah, thanks for the explanation.

77866555y ago

Pretty sure this won't work if the script is called via $PATH

pwdisswordfish25y ago

    x=$(command -v $0 2>/dev/null)
    sed -rn 's/^### ?//;T;p' $x

Personally I would not use the author's chosen sed commands.

    exec sed -n '/^###/p' $x

would work fine.

sneak5y ago

Why is “./script.sh -h” better than “less script.sh”?

soraminazuki5y ago

If you use zsh, you can use the help output to generate completions for your own scripts.

https://github.com/zsh-users/zsh-completions/blob/master/zsh...

mttpgn5y ago

I just spent three days explaining a bash script I authored to a tech lead who "doesn't know bash that well". If I had written in more help messages I probably could have preserved more of my time.

stkai5y ago

Handy! Now, can it author the help text as well? ;)

thangalin5y ago

There's an endless variation on how shell scripts can present help information. Here's another, consider this array:

    ARGUMENTS+=(
      "a,arch,Target operating system architecture (amd64)"
      "b,build,Suppress building application"
      "o,os,Target operating system (linux, windows, mac)"
      "u,update,Java update version number (${ARG_JRE_UPDATE})"
      "v,version,Full Java version (${ARG_JRE_VERSION})"
    )

The lines are machine-readable and alignment is computed by the template:

https://github.com/DaveJarvis/scrivenvar/blob/master/build-t...

When install script[0] help is requested, the following is produced:

    $ ./installer -h
    Usage: installer [OPTIONS...]

      -V, --verbose  Log messages while processing
      -h, --help     Show this help message then exit
      -a, --arch     Target operating system architecture (amd64)
      -b, --build    Suppress building application
      -o, --os       Target operating system (linux, windows, mac)
      -u, --update   Java update version number (8)
      -v, --version  Full Java version (14.0.1)

https://github.com/DaveJarvis/scrivenvar/blob/master/install...

    ARGUMENTS+=(
      "ARG_ARCH,a,arch,Target operating system architecture (amd64)"
      "do_build=noop,b,build,Suppress building application"
      "ARG_JRE_OS,o,os,Target operating system (linux, windows, mac)"
      "ARG_JRE_UPDATE,u,update,Java update version number (${ARG_JRE_UPDATE})"
      "ARG_JRE_VERSION,v,version,Full Java version (${ARG_JRE_VERSION})"
    )

[0]: https://github.com/DaveJarvis/scrivenvar/blob/master/install...

[1]: https://dave.autonoma.ca/blog/2019/05/22/typesetting-markdow...

oweiler5y ago

This is not elegant, this is an ugly hack at best.

j / k navigate · click thread line to collapse