Unicode-based scientific plotting for working in the terminal (opens in new tab)

(github.com)

136 pointsdelib5y ago36 comments

36 comments

Every time I see these things, I wish Sixel graphics[0] were better supported, or else some more modern alternative.

Being able to use GNUPlot with Sixel output or tools like lsix[1] is really cool! I think if support / libraries / knowledge about Sixels was more widespread, we would see some really nifty terminal applications cropping up.

0 - https://saitoha.github.io/libsixel/

1 - https://github.com/hackerb9/lsix

dheera5y ago

I wrote a generic terminal graphics library that uses Unicode Braille with the option of ASCII fallback: https://github.com/dheera/python-termgraphics

I was thinking about adding sixel support to this to have a universal graphics library that intelligently uses the best method available., but never got around to it yet.

fishmaster5y ago

This looks pretty cool. Why isn't this more popular?

dheera5y ago

Most of the "default" terminal apps (e.g. gnome-terminal) don't support sixel.

andi9995y ago

What terminal do I need to see these images. I am confused.

charlesdaniels5y ago

The libsixel website lists several that support it. Xterm can support this in VT430 emulation mode, but support needs to be enabled at compile time.

sprash5y ago

I recently found out that XTerm supports sixels natively if started with the "-ti 340" option. For example gnuplot can make use of that feature using the command "set terminal sixlegd".

It turns out because of the way sixels are encoded (six vertical pixels in a row using bitplanes) it packs better than png when used in conjunction with the bzip2 packer. Every image can simply be encoded to sixel with imagemagick using "convert image.png sixel:- > image.six"

Personally I think sixels are genius should be used much more often.

brutt5y ago

Sixels will be added to libvte (gnome-terminal): https://gitlab.gnome.org/GNOME/vte/-/issues/254

iso-8859-15y ago

Great news! I think we may finally see a resurgence. It will be well supported on mainstream Linux, and given that the web is getting so heavy-weight, this will offer a solid alternative for graphics that you can rely on. Libvte is very popular.

petepete5y ago

GNUPlot and XTerm also support Tektronix, there's lots of info in this older discussion.

https://news.ycombinator.com/item?id=7776904

mongol5y ago

The Kitty terminal author also published a "terminal graphics protocol". I don't know what other terminal supports it, however.

https://sw.kovidgoyal.net/kitty/graphics-protocol.html

dTal5y ago

>it packs better than png when used in conjunction with the bzip2 packer.

Cute! But is it fair? I don't think it's lossless - the color depth is much reduced.

sprash5y ago

I took the reduced color depth into consideration. However on second glance it seems like Imagemagick does not compress PNG very efficiently with default settings. After recompressing with zopflipng PNG was easily able to beat bzip2 packed sixels. Results for 256 color 512x512 Lenna:

Sixel + bzip2: 245K

PNG Imagemagick: 467K

PNG Zopfli 10 Iterations: 188K

iso-8859-15y ago

I used your ImageMagick command, but the resulting image is very bright and almost invisible. I tried disabling Unicode using +u8, but that didn't work. I tried mlterm, that worked.

sprash5y ago

I forgot to tell that you have to enable 256 colors as well. Put this into your .Xresources:

    XTerm.*.decTerminalID: vt340
    XTerm.*.numColorRegisters: 256

And refresh with

    xrdb -merge $HOME/.Xresources

It should work now.

leephillips5y ago

That should be "sixelgd", of course. And there is the classic "dumb" terminal, that plots to any text terminal, optionally using ansi escape codes for color.

dorfsmay5y ago

As somebody who grew up during the semi-graphic era, it makes me sad that we still use a 30 year old standard for terminals and cannot do direct plot in the main terminal (as opposed to the Tektronix separate window).

KenoFischer5y ago

Several terminals (kitty, iTerm2) do support plotting right into the terminal, but this is still sometimes useful if you don't want to load a while graphics stack just to do a simple plot.

dorfsmay5y ago

Are they using a similar API/standard?

1 more reply

tempodox5y ago

I was thinking something similar. Looking at this makes me wish a terminal app could just embed a PNG or whatever in its output.

kps5y ago

The OSC 1337 protocol pioneered by iTerm2 does exactly that. You'll have to check whether your preferred terminal supports it (I known mlterm does), either from documentation or just trying it¹.

Also, some terminals support the DEC SIXEL protocol² (again, I know mlterm does).

¹ https://www.iterm2.com/utilities/imgcat

² https://en.wikipedia.org/wiki/Sixel

Edit: I couldn't find a simple SIXEL sample — you'd think one of the popular sixel libraries would have some in unit tests, but no. So for the purpose of this thread (but maybe not forever, hence /tmp/),

    curl https://datatravelandexperiments.com/kps/tmp/example.sixel
    curl https://datatravelandexperiments.com/kps/tmp/example.osc1337

Those are such that output directly to the terminal will produce an image if that format is supported (and probably nothing if not, but I'm not making promises).

z3t45y ago

I heard they have something similar at Cern and it also has clickable links and universal resource locators.

visviva5y ago

This is one of my favorite Julia packages! A fast plotting utility that outputs right to the terminal is one of those things that I didn't realize I needed until I found it.

ViralBShah5y ago

Yes. Some people think this is a gimmick. But it is actually quite a bit useful. Especially, when in the middle of a session, I often do not want to move my hands from the keyboard to the mouse to close windows that plotting tools open, or move my gaze. It's not often that I work like this, but I like it that I can for the times I need to.

Having inline plots in the terminal - especially things like histograms which just work - is often quite handy. And it just works over an ssh connection.

Also, recently, Julia changed its sparse matrix display to show a picture of the sparse matrix (spy plot) using braille characters!

  julia> a = sprand(20,20,0.2)
  20×20 SparseMatrixCSC{Float64,Int64} with 67 stored entries:
  ⠀⠀⠁⠐⠄⠋⠡⠀⠐⡀
  ⢳⠁⠡⠈⠂⠀⠀⠁⢡⠑
  ⡠⠋⢠⡀⢄⢀⢀⠄⠈⠈
  ⠩⠄⠐⠨⠈⠀⠆⠂⠨⠤
  ⠄⠄⢀⢂⠠⢀⠀⠂⠆⠂

vok5y ago

I also like that I can incorporate plots into my plain-text notes files.

lsllc5y ago

This is awesome. When I build my spaceship, it's going to have a command line interface and I'll use this to display pertinent spaceship-data, such as gravitational pull of nearby black holes, tachyon levels, and warp signatures etc.

Can't safely use a touchscreen or mouse when all hell breaks loose in space!

antirez5y ago

Trivia: Redis uses a similar technique (Braille unicode points) to show the Redis 5 LOLWUT. http://antirez.com/news/123

michaelhoffman5y ago

This is great! Anyone know of anything like this for Python or R?

ianhorn5y ago

I made a dead simple, somehow surprisingly useful one for python back in 2014. I still use it for quick investigations all the time.

hipsterplot: https://github.com/imh/hipsterplot

dheera5y ago

I also wrote a more generic Python terminal plotting library: https://github.com/dheera/python-termgraphics

For robotics people -- the above is used in rosshow: http://github.com/dheera/rosshow

metaodi5y ago

I guess plotille (https://github.com/tammoippen/plotille) is the closest match for python. And maybe termgraph (https://github.com/mkaz/termgraph) is kind of similar.

foob5y ago

It's not unicode, but there's also sixelplot [1] which allows matplotlib plotting in a terminal using sixel. The underlying pysixel library [2] unfortunately doesn't support Python 3.7+ however.

[1] https://github.com/kktk-KO/sixelplot

[2] https://github.com/saitoha/PySixel

1 more reply

SubiculumCode5y ago

Honest question: Why would I ever want/need to plot data in the R console, and not in a R graphics window?

tejtm5y ago

the "work" you are doing is via ssh into a big box behind gateway/jumphost with peculiar limitations on ports & access from the data gravity well which is your bog standard asymmetric home isp ; annoying but you need to do anyway because the remote hardware has more bandwidth, memory, cores and storage than a stack of laptops taller than you... I could go on but lets just acknowledge not every task fits in a prepackaged shiny solution.

j / k navigate · click thread line to collapse

36 comments

charlesdaniels5y ago

Every time I see these things, I wish Sixel graphics[0] were better supported, or else some more modern alternative.

0 - https://saitoha.github.io/libsixel/

1 - https://github.com/hackerb9/lsix

dheera5y ago

I wrote a generic terminal graphics library that uses Unicode Braille with the option of ASCII fallback: https://github.com/dheera/python-termgraphics

I was thinking about adding sixel support to this to have a universal graphics library that intelligently uses the best method available., but never got around to it yet.

fishmaster5y ago

This looks pretty cool. Why isn't this more popular?

dheera5y ago

Most of the "default" terminal apps (e.g. gnome-terminal) don't support sixel.

andi9995y ago

What terminal do I need to see these images. I am confused.

charlesdaniels5y ago

The libsixel website lists several that support it. Xterm can support this in VT430 emulation mode, but support needs to be enabled at compile time.

sprash5y ago

I recently found out that XTerm supports sixels natively if started with the "-ti 340" option. For example gnuplot can make use of that feature using the command "set terminal sixlegd".

Personally I think sixels are genius should be used much more often.

brutt5y ago

Sixels will be added to libvte (gnome-terminal): https://gitlab.gnome.org/GNOME/vte/-/issues/254

iso-8859-15y ago

petepete5y ago

GNUPlot and XTerm also support Tektronix, there's lots of info in this older discussion.

https://news.ycombinator.com/item?id=7776904

mongol5y ago

The Kitty terminal author also published a "terminal graphics protocol". I don't know what other terminal supports it, however.

https://sw.kovidgoyal.net/kitty/graphics-protocol.html

dTal5y ago

>it packs better than png when used in conjunction with the bzip2 packer.

Cute! But is it fair? I don't think it's lossless - the color depth is much reduced.

sprash5y ago

Sixel + bzip2: 245K

PNG Imagemagick: 467K

PNG Zopfli 10 Iterations: 188K

iso-8859-15y ago

I used your ImageMagick command, but the resulting image is very bright and almost invisible. I tried disabling Unicode using +u8, but that didn't work. I tried mlterm, that worked.

sprash5y ago

I forgot to tell that you have to enable 256 colors as well. Put this into your .Xresources:

    XTerm.*.decTerminalID: vt340
    XTerm.*.numColorRegisters: 256

And refresh with

    xrdb -merge $HOME/.Xresources

It should work now.

leephillips5y ago

That should be "sixelgd", of course. And there is the classic "dumb" terminal, that plots to any text terminal, optionally using ansi escape codes for color.

dorfsmay5y ago

KenoFischer5y ago

Several terminals (kitty, iTerm2) do support plotting right into the terminal, but this is still sometimes useful if you don't want to load a while graphics stack just to do a simple plot.

dorfsmay5y ago

Are they using a similar API/standard?

1 more reply

tempodox5y ago

I was thinking something similar. Looking at this makes me wish a terminal app could just embed a PNG or whatever in its output.

kps5y ago

The OSC 1337 protocol pioneered by iTerm2 does exactly that. You'll have to check whether your preferred terminal supports it (I known mlterm does), either from documentation or just trying it¹.

Also, some terminals support the DEC SIXEL protocol² (again, I know mlterm does).

¹ https://www.iterm2.com/utilities/imgcat

² https://en.wikipedia.org/wiki/Sixel

    curl https://datatravelandexperiments.com/kps/tmp/example.sixel
    curl https://datatravelandexperiments.com/kps/tmp/example.osc1337

Those are such that output directly to the terminal will produce an image if that format is supported (and probably nothing if not, but I'm not making promises).

z3t45y ago

I heard they have something similar at Cern and it also has clickable links and universal resource locators.

visviva5y ago

This is one of my favorite Julia packages! A fast plotting utility that outputs right to the terminal is one of those things that I didn't realize I needed until I found it.

ViralBShah5y ago

Having inline plots in the terminal - especially things like histograms which just work - is often quite handy. And it just works over an ssh connection.

Also, recently, Julia changed its sparse matrix display to show a picture of the sparse matrix (spy plot) using braille characters!

  julia> a = sprand(20,20,0.2)
  20×20 SparseMatrixCSC{Float64,Int64} with 67 stored entries:
  ⠀⠀⠁⠐⠄⠋⠡⠀⠐⡀
  ⢳⠁⠡⠈⠂⠀⠀⠁⢡⠑
  ⡠⠋⢠⡀⢄⢀⢀⠄⠈⠈
  ⠩⠄⠐⠨⠈⠀⠆⠂⠨⠤
  ⠄⠄⢀⢂⠠⢀⠀⠂⠆⠂

vok5y ago

I also like that I can incorporate plots into my plain-text notes files.

lsllc5y ago

Can't safely use a touchscreen or mouse when all hell breaks loose in space!

antirez5y ago

Trivia: Redis uses a similar technique (Braille unicode points) to show the Redis 5 LOLWUT. http://antirez.com/news/123

michaelhoffman5y ago

This is great! Anyone know of anything like this for Python or R?

ianhorn5y ago

I made a dead simple, somehow surprisingly useful one for python back in 2014. I still use it for quick investigations all the time.

hipsterplot: https://github.com/imh/hipsterplot

dheera5y ago

I also wrote a more generic Python terminal plotting library: https://github.com/dheera/python-termgraphics

For robotics people -- the above is used in rosshow: http://github.com/dheera/rosshow

metaodi5y ago

I guess plotille (https://github.com/tammoippen/plotille) is the closest match for python. And maybe termgraph (https://github.com/mkaz/termgraph) is kind of similar.

foob5y ago

It's not unicode, but there's also sixelplot [1] which allows matplotlib plotting in a terminal using sixel. The underlying pysixel library [2] unfortunately doesn't support Python 3.7+ however.

[1] https://github.com/kktk-KO/sixelplot

[2] https://github.com/saitoha/PySixel

1 more reply

SubiculumCode5y ago

Honest question: Why would I ever want/need to plot data in the R console, and not in a R graphics window?

tejtm5y ago

j / k navigate · click thread line to collapse