Show HN: Tail out log files from multiple remote hosts with one command (opens in new tab)

(github.com)

28 pointsNickC_dev11y ago18 comments

18 comments

So it's basically:

    tail -f <(ssh -t host 'tail -f /var/log/sth') <(ssh -t host2 'tail -f /var/log/sth')

+ I need to install nodejs?

voltagex_11y ago

I have no problem with people writing cool little tools like this in whatever language, but there seems to be a chronic lack of understanding of what existing tools can do.

SixSigma11y ago

Henry Spencer 11/15/87

    Those who do not understand Unix are condemned to reinvent it, poorly.

robbles11y ago

The interface to this tool is far nicer to work with. Can you replicate that interface with your command?

Serious question - I actually would like to see how you'd do it.

vlad00311y ago

You can have something like this in your .bashrc:

    multitail() {
        com="tail -f "

        for arg; do
            IFS=':' read -a array <<< "$arg"
            host="${array[0]}"
            file="${array[1]}"

            com="$com <(ssh -t $host 'tail -f $file')"
        done

        eval ${com}
    }

This will take the host:file syntax of remtail and turn it into Plugawy's example. I haven't tested it, but if you echo "$com" it'll spit out Plugawy's code.

1 more reply

rakoo11y ago

For another display, there's also multitail (http://www.vanheusden.com/multitail/) that you can run with a custom command, such as

  ssh host tail -f /path/to/log

(which is the command that remtail seems to be doing)

jfroma11y ago

I use pssh [1].

  pssh -H h1 -H h2 -P "tail -n 1000 /var/log/x.log"

It is very easy to install on every platform since is on most package managers.

[1]: http://linux.die.net/man/1/pssh

lobster_johnson11y ago

pssh is great.

You'll want -i to avoid the buffered output that would make the log output hard to read.

And if your command doesn't contain anything that you need to escape from the shell, you don't need to quote the command.

And finally, since it implements BSD-style short flags correctly, so you can do "-Pi" instead of "-P -i".

Handy with a pre-defined list of hosts, too:

    pssh -Pi -H "hosts.txt" tail -f /var/log/haproxy.log

or a specific set of hosts:

    pssh -Pi -H `cat hosts.txt | grep prod` tail -f /var/log/haproxy.log

jfroma11y ago

Nice tips, thank you!

ghuntley11y ago

    > You can optionally use a credentials file in ~/.remtail.json of this format:

What ever happened to just using .netrc?

http://www.gnu.org/software/inetutils/manual/html_node/The-_...

kjjw11y ago

or ssh config?

NickC_devOP11y ago

ssh config support is the first thing on the roadmap. I'd be happy to take a pull request for it.

1 more reply

fsniper11y ago

Good implementation but not for my tastes. node.js for this purpose? I think it's a bit heavy in this case.

mssh or cluster ssh is also effective in this manner, with the added flexibility of ssh terminals.

typicalbender11y ago

I've also used dsh[1] to accomplish the same thing. Never tried to use username and password for this though so that actually may be more of a pain with dsh. Also tailing multiple files on the same host is pretty cool.

[1] - http://www.tecmint.com/using-dsh-distributed-shell-to-run-li...

fidz11y ago

In my company, we have hundred machines and tailing done with ansible. If we want customize the log view, we can simply edit the playbook. I think it is very handy compared to we need additional npm package (and not to mention additional effort for customization).

dugmartin11y ago

If you want to send that tail output to your browser here is a little app I released last week:

http://dougmart.in/projects/pagepipe

j / k navigate · click thread line to collapse

18 comments

Plugawy11y ago

So it's basically:

    tail -f <(ssh -t host 'tail -f /var/log/sth') <(ssh -t host2 'tail -f /var/log/sth')

+ I need to install nodejs?

voltagex_11y ago

I have no problem with people writing cool little tools like this in whatever language, but there seems to be a chronic lack of understanding of what existing tools can do.

SixSigma11y ago

Henry Spencer 11/15/87

    Those who do not understand Unix are condemned to reinvent it, poorly.

robbles11y ago

The interface to this tool is far nicer to work with. Can you replicate that interface with your command?

Serious question - I actually would like to see how you'd do it.

vlad00311y ago

You can have something like this in your .bashrc:

    multitail() {
        com="tail -f "

        for arg; do
            IFS=':' read -a array <<< "$arg"
            host="${array[0]}"
            file="${array[1]}"

            com="$com <(ssh -t $host 'tail -f $file')"
        done

        eval ${com}
    }

This will take the host:file syntax of remtail and turn it into Plugawy's example. I haven't tested it, but if you echo "$com" it'll spit out Plugawy's code.

1 more reply

rakoo11y ago

For another display, there's also multitail (http://www.vanheusden.com/multitail/) that you can run with a custom command, such as

  ssh host tail -f /path/to/log

(which is the command that remtail seems to be doing)

jfroma11y ago

I use pssh [1].

  pssh -H h1 -H h2 -P "tail -n 1000 /var/log/x.log"

It is very easy to install on every platform since is on most package managers.

[1]: http://linux.die.net/man/1/pssh

lobster_johnson11y ago

pssh is great.

You'll want -i to avoid the buffered output that would make the log output hard to read.

And if your command doesn't contain anything that you need to escape from the shell, you don't need to quote the command.

And finally, since it implements BSD-style short flags correctly, so you can do "-Pi" instead of "-P -i".

Handy with a pre-defined list of hosts, too:

    pssh -Pi -H "hosts.txt" tail -f /var/log/haproxy.log

or a specific set of hosts:

    pssh -Pi -H `cat hosts.txt | grep prod` tail -f /var/log/haproxy.log

jfroma11y ago

Nice tips, thank you!

ghuntley11y ago

    > You can optionally use a credentials file in ~/.remtail.json of this format:

What ever happened to just using .netrc?

http://www.gnu.org/software/inetutils/manual/html_node/The-_...

kjjw11y ago

or ssh config?

NickC_devOP11y ago

ssh config support is the first thing on the roadmap. I'd be happy to take a pull request for it.

1 more reply

fsniper11y ago

Good implementation but not for my tastes. node.js for this purpose? I think it's a bit heavy in this case.

mssh or cluster ssh is also effective in this manner, with the added flexibility of ssh terminals.

typicalbender11y ago

[1] - http://www.tecmint.com/using-dsh-distributed-shell-to-run-li...

fidz11y ago

dugmartin11y ago

If you want to send that tail output to your browser here is a little app I released last week:

http://dougmart.in/projects/pagepipe

j / k navigate · click thread line to collapse