Show HN: An SSH based utility to transfer pipes across machines – beam (opens in new tab)

(github.com)

28 pointsksdme91y ago31 comments

Hey HN,

At work, I was annoyed about having to download logs by writing them out to files and transferring them over to my local over HTTP (admittedly, this is because of a rather novel architectural situation).

So, I built a tool in Go that lets you pipe contents into an SSH server for sending and reading them out on another connection. You can also use this tool to transfer files. Just cat the file on the sender end and redirect the output of the receiver end to a file.

Please note that the transfer rate is heavily dependent on your connection and proximity to the public ssh.beam.camp host. This is because there is no simple way to auto connect to the closest host using a regular SSH client.

Hope you find beam useful. Have a nice weekend.

31 comments

Darkskiez1y ago

This is a great way to send all of your files to the author of the utility / operator of the website.

ksdme9OP1y ago

From the readme,

> Beam cannot support end-to-end encrypted buffers. While data is encrypted during transfer to and from the Beam host, it’s decrypted temporarily before being re-encrypted and forwarded. The host only holds a small buffer (typically 1 kB) of unencrypted data at any time and never stores the full stream. For extra security, you can encrypt your files or pipes before sending them through Beam.

With a little effort, beam is as trustable as any (if not more) of its alternatives. And, that extra effort is a result of the design goal of not having to force a binary installation.

Plus, you can always self host beam, it's not that complicated.

moralestapia1y ago

Yeah, as with literally every other service in the web for the past 20 years.

f-az1y ago

After reading the readme I’m confused what the advantage is over just regular piping. ‘’’ command-on-server1 | ssh user@server2 'command-on-server2' ‘’’

unsnap_biceps1y ago

I think a fully equivalent would be to use socat

On Receiving Server

    ssh user@proxy socat UNIX-RECVFROM:/tmp/foobar -

And then on sending server

    ./foobar | ssh user@proxy socat - UNIX-SENDTO:/tmp/foobar

ksdme9OP1y ago

Beam lets you do what you described even when both the machines are isolated and not reachable from one another. All you need is ssh clients on both the machines.

hnlmorg1y ago

That’s not clear in the readme. It might be worth explicating stating that this uses the SSH protocol but is designed to work on hosts that either dont have OpenSSH enabled or environments where you don’t control the OpenSSH config.

Given the current readme, I was left wondering the same as the GP too.

1 more reply

alanfranz1y ago

But if you have ssh clients on both machines, you can use port forwarding to allow machine2 to connect to machine1 via ssh. Or you can connect both machines to the "SSH stun server" (which is something like the beam host if I understand correctly) and let them connect through that.

I mean, I'm not bashing beam, but it's mostly syntactic sugar, not something that provides some actual new functionality, is that right?

1 more reply

fulafel1y ago

The remote ssh server used in the demo seems to be a proxy service so apparently it's for cases where you can't ssh in to your example's server2.

remram1y ago

I think you can turn this (or any equivalent service like ssh-j.com) into an end-to-end encrypted option by relaying an SSH connection over which you send the data, instead of relaying the data.

Example:

  # receiver
  ssh top-secret@ssh-j.com -N -R ssh:22:localhost:22
  socat UNIX-RECV:/tmp/foobar - | my command

  # sender
  my-command | ssh -J top-secret@ssh-j.com ssh socat - UNIX-SENDTO:/tmp/foobar
  (log in with your creds to receiver host, enjoy end-to-end encryption)

lathiat1y ago

Maybe better to use something like magic wormhole? https://github.com/magic-wormhole/magic-wormhole

moralestapia1y ago

No, because you have to install more crap.

This thing is "already there" and it takes two commands. I like it a lot, tbh.

ksdme9OP1y ago

Yup, wormhole and portal are better in most aspects. Beam is helpful when you have novel constraints like not being able to install arbitrary binaries, or if you have the need to transfer streams and not just files, or maybe even pub key based white-listing of access.

remram1y ago

This seems very similar to ssh-j.com (open source: https://bitbucket.org/ValdikSS/dropbear-sshj)

rzzzt1y ago

Is "ssh.beam.camp" an example hostname, or an instance that you run?

ksdme9OP1y ago

It's ssh.beam.camp and is a public host that I run.

rzzzt1y ago

Argh, an extra letter got in. Edited my original comment to avoid confusion.

moralestapia1y ago

@kdsme9 I think this is great and you've done a great work putting it together and sharing it with us for free.

Using the public key to match streams on both ends is super clever. I will start using this since it fills a need I have almost every day. I work with many small AWS instances and sometimes I just need to copy a small file or something to them without having to install a whole new OS like some short sighted people here are suggesting.

You also shipped your product, which is something only ~1% of developers actually do, so congrats.

Forget about the naysayers and thank you for creating this, it will make my work easier :).

Edit: Downvote me all you want, I still like this thing.

ksdme9OP1y ago

Thanks for the kind words. I am so glad you found it useful.

ilyagr1y ago

I have used https://pipe.pico.sh/ for this. pico.sh also has a bunch of other SSH-accessible services, e.g. pastes, and paid (or free if self-hosted) port tunneling.

eqvinox1y ago

I don't see any advantage over plain "-o 'ProxyJump box.in.the.middle'"? I guess this is supposed to buffer things? But then again, that's what "tail -f" is for…

More confused than anything else…

CGamesPlay1y ago

It would be great for beam.ssh.camp to implement this (which would require no software, just configuring OpenSSH).

rakoo1y ago

This is for cases where the server in the middle can't ssh into the target server

pacifika1y ago

Could you please give a complete example for those not aware? Ty

CGamesPlay1y ago

Given a local machine and two remote machines: ssh remote2 -J remote1. This will connect to remote1 as a normal SSH connection, but then port forward a new connection to remote2 via that connection. Traffic leaving localhost is double-encrypted, and remote1 cannot see the data.

Disadvantage: remote1 must be able to connect to remote2. In this (ssh.beam.camp) software, remote2 can be in a place where accepting incoming connections is not possible.

whatever11y ago

Huh? Why I cannot just ssh from machine 2 to machine 1 and just type `tail`?

ksdme9OP1y ago

This is meant for cases when the machines are isolated, for example, when you attach to a remote container and want to download a file from it to your machine.

philipwhiuk1y ago

Isolated but still able to access (and SSH!) a random server on the internet?

New meaning of the word isolated

1 more reply

j / k navigate · click thread line to collapse

31 comments

Darkskiez1y ago

This is a great way to send all of your files to the author of the utility / operator of the website.

ksdme9OP1y ago

From the readme,

With a little effort, beam is as trustable as any (if not more) of its alternatives. And, that extra effort is a result of the design goal of not having to force a binary installation.

Plus, you can always self host beam, it's not that complicated.

moralestapia1y ago

Yeah, as with literally every other service in the web for the past 20 years.

f-az1y ago

After reading the readme I’m confused what the advantage is over just regular piping. ‘’’ command-on-server1 | ssh user@server2 'command-on-server2' ‘’’

unsnap_biceps1y ago

I think a fully equivalent would be to use socat

On Receiving Server

    ssh user@proxy socat UNIX-RECVFROM:/tmp/foobar -

And then on sending server

    ./foobar | ssh user@proxy socat - UNIX-SENDTO:/tmp/foobar

ksdme9OP1y ago

Beam lets you do what you described even when both the machines are isolated and not reachable from one another. All you need is ssh clients on both the machines.

hnlmorg1y ago

Given the current readme, I was left wondering the same as the GP too.

1 more reply

alanfranz1y ago

I mean, I'm not bashing beam, but it's mostly syntactic sugar, not something that provides some actual new functionality, is that right?

1 more reply

fulafel1y ago

The remote ssh server used in the demo seems to be a proxy service so apparently it's for cases where you can't ssh in to your example's server2.

remram1y ago

I think you can turn this (or any equivalent service like ssh-j.com) into an end-to-end encrypted option by relaying an SSH connection over which you send the data, instead of relaying the data.

Example:

  # receiver
  ssh top-secret@ssh-j.com -N -R ssh:22:localhost:22
  socat UNIX-RECV:/tmp/foobar - | my command

  # sender
  my-command | ssh -J top-secret@ssh-j.com ssh socat - UNIX-SENDTO:/tmp/foobar
  (log in with your creds to receiver host, enjoy end-to-end encryption)

lathiat1y ago

Maybe better to use something like magic wormhole? https://github.com/magic-wormhole/magic-wormhole

moralestapia1y ago

No, because you have to install more crap.

This thing is "already there" and it takes two commands. I like it a lot, tbh.

ksdme9OP1y ago

remram1y ago

This seems very similar to ssh-j.com (open source: https://bitbucket.org/ValdikSS/dropbear-sshj)

rzzzt1y ago

Is "ssh.beam.camp" an example hostname, or an instance that you run?

ksdme9OP1y ago

It's ssh.beam.camp and is a public host that I run.

rzzzt1y ago

Argh, an extra letter got in. Edited my original comment to avoid confusion.

moralestapia1y ago

@kdsme9 I think this is great and you've done a great work putting it together and sharing it with us for free.

You also shipped your product, which is something only ~1% of developers actually do, so congrats.

Forget about the naysayers and thank you for creating this, it will make my work easier :).

Edit: Downvote me all you want, I still like this thing.

ksdme9OP1y ago

Thanks for the kind words. I am so glad you found it useful.

ilyagr1y ago

I have used https://pipe.pico.sh/ for this. pico.sh also has a bunch of other SSH-accessible services, e.g. pastes, and paid (or free if self-hosted) port tunneling.

eqvinox1y ago

I don't see any advantage over plain "-o 'ProxyJump box.in.the.middle'"? I guess this is supposed to buffer things? But then again, that's what "tail -f" is for…

More confused than anything else…

CGamesPlay1y ago

It would be great for beam.ssh.camp to implement this (which would require no software, just configuring OpenSSH).

rakoo1y ago

This is for cases where the server in the middle can't ssh into the target server

pacifika1y ago

Could you please give a complete example for those not aware? Ty

CGamesPlay1y ago

Disadvantage: remote1 must be able to connect to remote2. In this (ssh.beam.camp) software, remote2 can be in a place where accepting incoming connections is not possible.

whatever11y ago

Huh? Why I cannot just ssh from machine 2 to machine 1 and just type `tail`?

ksdme9OP1y ago

This is meant for cases when the machines are isolated, for example, when you attach to a remote container and want to download a file from it to your machine.

philipwhiuk1y ago

Isolated but still able to access (and SSH!) a random server on the internet?

New meaning of the word isolated

1 more reply

j / k navigate · click thread line to collapse