your_command_here | less
It will capture the output and let you scroll up and down.
Otherwise, this would really be a feature of the terminal, and not necessarily required to be supported (it makes sense when you consider that many early real terminals literally printed the output to paper -- who needs scroll-back, just look up the tape! And repeating previously shown output would result in a confusing print-out).
However, if your terminal emulator doesn't have a scroll-back, you could try something like tmux, or alternatively gnu screen. These add lots of little features to the terminal (a buffer to scroll back, split terminals, persistence so you can detach and reattach) (tmux is more modern feeling, IMO).
Assuming you tried moving the cursor vertically, have you tried the alternative approach of Store Cursor Position / Restore Cursor Position cf https://stackoverflow.com/questions/28986776/ansi-escape-seq...
The scrollback buffer may be a limitation, so you should try with enabling or disabling the alternate screen (ti/te)
If that fails, you may have to tweak your terminal emulator to "hook" the SCP/RCP to a specific point of the scrollback buffer, to allow this scrollback.
You'd be better off piping into less / more / most. These are called "pagers" and are designed to do this. eg
cat large-file | grep common-phrase | less -F or --quit-if-one-screen
Causes less to automatically exit if the entire file can be displayed on the
first screen.
(Also handy: add in `-X` to leave the contents on the screen when you exit less)
You don't necessarily need ncurses to have a portable smart terminal experience, but you do need terminfo.
ncurses magic relies on the lore stored in terminfo, which contains all of the obscure escape sequences and other information about the disparate world of terminals.
They're maintained as a combined package, but you don't need ncurses to drive the screen. You can get the codes yourself if that's what you want to do. You can use tput to make colorful labels in shell scripts.
While much of the world has moved on to the One Terminal running on the One OS, not everyone has.
> being compatible only with ANSI-capable terminals is a feature, not a bug
Driving old hardware terminals (as opposed to terminal emulators) is a fun stunt, not something of actual modern value. Every modern terminal emulator supports ANSI escape sequences. Some features are supported by a subset of terminals, but you can either 1) probe for those features by asking the terminal, which some terminals support, or 2) try them and have graceful degradation if they're not supported, or 3) have configuration options to use them, or 4) don't use them.
For every one case in which terminfo allows you to support some obscure non-ANSI terminal, there are many many more cases where terminfo won't happen to have a definition of the user's terminal (or won't know all the capabilities of the user's terminal) and you'll have less functionality than if you just used ANSI escapes. This is especially true over SSH and similar.
Color ansi codes might be everywhere today, but that's no reason to disgrace terminfo, there are a ton of other features that need checking and are not universal.
And even truecolor was added to it about 10 years after terminals started gaining support.
And many terminals just claim to be "xterm-256color".
>2) try them and have graceful degradation if they're not supported
Note: There are many cases where degradation isn't graceful. Many terminals on windows currently spew garbage on your screen if you send bracketed paste (alacritty, for instance).
>1) probe for those features by asking the terminal, which some terminals support
This requires waiting for a reply, which often isn't useful. E.g. if you want synchronized output, you want it from the very first paint (because that's when the terminal is most likely to still be resized, e.g. by a tiling window manager). So you would have to delay your startup until you've either gotten a reply or "enough" time has passed that you believe it's not supported.
Frankly, this is all a big mess and terminfo isn't very helpful, but we don't have a good alternative either.
Is this true on Windows? My understanding is that quite a few TUI libraries don't have windows support.
> being compatible only with ANSI-capable terminals is a feature, not a bug, go the fuck away. terminfo is a fucking joke. nobody needs to target obscure dumb terminals (or smart terminals, for that matter) from 1983 anymore.
I don't have the expertise to opinion on this. But AFAICT most terminals that people still use[1] will self-report as xterm or xterm-256color so they'll only use the xterm escape sequences anyway.
[1] kitty is a notable exception. But it's rarely supported on preinstalled terminfo's, and manually adding them to all the remote servers that you access is a PITA.
Terminfo is not a de jure standard, only de facto. It is not in POSIX.
POSIX specifies a tput command with exactly three operations: clear, init and reset.
Right here: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/t...
There is a Rationale section giving reasons for why terminfo wasn't standardized.
Bottom line: ECMA/ANSI is the real standard that can be used to drive a conforming terminal from any OS, without any special library other than basic I/O.
Also not just ECMA. The standard was ratified by ANSI X3.64-1979, ISO/IEC 6429, and FIPS-86 in addition to ECMA-48. The whole civilized world wants these codes (e.g. \033[A for UP) to be a standard. Because it makes console development blissful when you can ignore all the accidental complexity of curses and terminfo when you can just assume the terminal behaves sanely and isn't the leftover byproduct of bygone commercial rivalries.
Definitely worth checking out the full landscape these days if you're going to dive into making your console programs prettier.
[1] My words, I just coined that name (although I wouldn't be surprised if others had said it too). I mean that in the last ~decade there has been a lot of TUI work in the background, with lots of new programs and some pretty stunning results. I blame unicode - once the web folks realized they could use something like font-awesome instead of sprite-sheets (or maybe as a pre-build sprite sheet?), and the "icons in a font" movement took hold, terminal programs got a lot prettier too. It makes sense, its the same font/font renderer provided by the system no matter if the app is a browser or terminal emulator.
This so much! Dank, Jart, Yatli and HPA are 4 people whose work is single handedly pushing TUIs forward.
There are others, but these 4 are the poggers, and I don't think the likes of me (or Christian, or Autumn, etc) can stand the comparison.
Notcurses is a unique work of love and art worth all the praises in the world.
The title should be changed to (2019) because this post came out at about the same time as the teletypewriter aristocracy kicked off the renaissance.
https://github.com/migueldeicaza/gui.cs
(Disclaimer, I maintain Terminal.gui).
I'm suddenly really excited for what can be done on TUIs. Like I was some 40 years ago.
Edit: and you can write CSS instead of curses? That's seriously cool.
How does it work with buffered stdout? For example, if you wanted to colour a single word?
I was also surprised when I discovered this. IMO this should be used everywhere but I've rarely seen it.
> How does it work with buffered stdout? For example, if you wanted to colour a single word?
This is not affected by buffering. What matters is what the output device/file/stream gets eventually.
How coloring a single word works depends on how you use it.
You can do
write_and_flush_to_dev_tty(BEGIN_RED_TEXT);
write_and_flush_to_dev_tty("word");
write_and_flush_to_dev_tty(END_RED_TEXT);
Redirection won't have any effect on these lines of code. You always get a red "word" on the terminal. The redirection target doesn't get anything.
write_and_flush_to_dev_tty(BEGIN_RED_TEXT);
write_and_flush_to_stdout("word");
write_and_flush_to_dev_tty(END_RED_TEXT);
write_and_flush_to_dev_tty(BEGIN_RED_TEXT);
write_and_flush_to_stdout("word");
write_and_flush_to_stdout(END_RED_TEXT);
When stdout is a tty, /dev/tty is the same as stdout, so a write that goes to one goes to the other. In this case, even if you don't flush after write immediately it's likely not a problem. Still it's something that the developer should pay attention to.
-------------
Edit:
Please ignore all the pseudocode examples. They don't convey what I wanted to express.
Just use /dev/tty in the same way as how you would use stdin/stdout/stderr that is connected to a terminal. Only difference is that it is a rw device that can be used for input and output at the same time.
Yes I have. Using /dev/tty doesn't stop an application's author from adding a --color that lets their program send color codes to stdout. But if the author doesn't use /dev/tty either stdout or stderr of their application can't be redirected.
We're basically recreating Flash for the terminal, where we got a fancy UI but we lose the original text functionality, i.e. output is non pipable anymore. Not ideal when working on UNIX systems.
I wonder if Chrome considers the upside down "LATIN SMALL LETTER TURNED K" potentially deceptive.
They've apparently got a whole document about what Chrome will and won't display
https://chromium.googlesource.com/chromium/src/+/main/docs/i...
punicode was a terrible idea to allow in DNS.
This is a very western-centric viewpoint. Most of the world does not use the Latin alphabet as its primary writing system.
I'd recommend that over the wikipedia link in the article, since wikipedia seems to be missing some things.
The VT510 manual is also useful since it was designed to be used with a PC keyboard so it covers newer escape codes that aren't on older DEC terminals https://vt100.net/docs/vt510-rm/chapter8
Either way, I wouldn't use it as an authoritative source uncritically.
As a father trying to teach two small children to read and write, I've begun to think it's kind of annoying and unnecessary to have two different forms. It's easy enough to teach when they are just bigger glyphs, e.g. O -> o, C -> c, S -> s, etc. But having uppercase and lowercase that are entirely different is just a pain in the ass, e.g. A -> a, D -> d, G -> g, R -> r, etc.
I do wonder if literacy might be improved if we got rid of uppercase, to little to no detriment elsewhere. (I'm sure someone here can probably sight some data on that!)
Much like all capitals is seen as shouting.
It’s a way of saying, you are not worth talking to properly.
[1] apparently an "overcompensation" coming from a language in which all nouns are capitalised.
Is that correct?
I seem to recall having a hell of a time trying to figure out which key combinations Emacs could understand and why (keyboard -> OS -> terminal emulator -> protocol -> program.)
Are there any proposals? Workarounds? Proofs-of-concept?
Yes, this is correct. In particular, there are certain key combinations which cannot be distinguished (for example, Control+I is indistinguishable from Tab) and some keys aren't affected by modifiers (such as Shift and Control on Space).
There's an extended discussion of this issue, and a proposed solution, at: http://www.leonerd.org.uk/hacks/fixterms/
$ wget https://justine.lol/ttyinfo.com
$ chmod +x ttyinfo.com
$ ./ttyinfo.com
"\001" is CTRL-A a.k.a. ^A
"\000" is CTRL-@ a.k.a. ^@
"\002" is CTRL-B a.k.a. ^B
"\f" is CTRL-L a.k.a. ^L
"\r" is CTRL-M a.k.a. ^M
"\033\001" is ALT-CTRL-A
"\033[A" is UP
"\033\033[A" is ALT-UP
It'd be great if the standard bodies could promulgate some kind of well-defined solution to this problem. Having read the ECMA standards documents, I think they really could be doing more to focus on the people who use it for terminals!
It's not exactly like that - it's more like there are competing "standards" and interpretations of these standards, and sometimes supporting one means not supporting the other.
Fortunately, such things are rare, and can be addressed by GUI options.
Take for example SGR1 for "bold/intense" text: read the whole issue that came to Microsoft Terminal team in 2018 up to its most recent discussions on https://github.com/microsoft/terminal/issues/109 then check the "simpler" version in wikipedia https://en.wikipedia.org/wiki/ANSI_escape_code#SGR_.28Select... and you will realize there's no "right" answer between bold and bright (or both!), just different preferences and interpretations of the standards.
Are there standards to represent, for example, Ctrl+Alt+* (I.e. Ctrl+Alt+Shift+8) ? Additionally with cursor keys, Insert, Delete, Backspace, F1 - F12, etc.
I’ve searched several times for such standards before but never had any luck…
keyboard -> OS -> terminal emulator -> protocol -> shell -> program
And that’s for local IO…
(Too late to edit my above comment.)
* The ESC [ command start sequence is actually a compromise for 7 bit systems. The [ character is not chosen by accident. It has an obvious positional relationship to ESC in the ASCII code which is why, informally, Ctrl-[ is the same as ESC.
* If you have an 8-bit-clean channel to the terminal, only a single character is required: the "upper escape" from the C1 control character set (0x80 to 0x9F). This character, 128 + 27 or 0x9B is called CSI: control sequence introducer, which is basically its role in these terminal control sequences. Thus ESC [ is just an alternative way of encoding CSI for 7 bit. E.g instead of CSI 4 A, you use ESC [ 4 A.
I'd avoid using this. It conflicts badly with UTF-8's use of 0x80 through 0xBF as continuation characters.
Of course, if the terminal is ignorant of UTF-8 then this is a nonstarter, because whenever CIS occurs as a continuation byte, it will be misinterpreted. If the terminal is ignorant of UTF-8, why would you send UTF-8 to it, though? If it's going to be interpreting UTF-8 as some branch of ISO Latin, the display will be a mess.
If the terminal and host do handle UTF-8, then this CSI signaling is just an extension of the state machine. It's also nice and simple that, in the absence of any data loss or synchronization error, the CSI code is unambiguously not part of any valid UTF-8 character (except as the second or subsequent byte where the receiver is in the right state to interpret it that way).
In my experience, it's the terminal->host direction where you get mixups, whereby the terminal generates some escape sequence like for an arrow key, but the host is not in the right state, and interprets part of it as data. This is exacerbated by a situation in which the host supports ESC as a UI command. CSI solves the abmbiguity between the control sequence start and ESC just being ESC.
Putting a terminal into C1 mode is a great way to have it do unpredictable and weird things unless printable input is constrained to the lower seven.
If they are being encoded, this is still useful. Though space isn't saved, any ambiguity between the control bytes and UTF-8 bytes is eliminated. The advantage is still present that CSI is different from ESC, and so in the terminal->host direction, you don't have the ambiguity between ESC as a UI command character versus control sequence signal byte.
Actually, yes, understanding the tty in detail seems to become a dark art.
However it's the best way to do complex things quickly: I did use some of these tricks like storing and then restoring the cursor position to have the time at which a command stop executing ABOVE the command itself and next to the time it started executing in https://github.com/csdvrx/bash-timestamping-sqlite
I had to, because I was using MSYS2 and the time to execute a command was a limiting factor in Windows before WSL2.
No, WYSIWYG: I don't bother much more than that with explaining how it's done: the source code + the comments + the github page already give all the details away to whoever wants to dig deeper.
I'm not much into social media or self promotion either: if people like what I do, they'll use it - I don't care much more than that, as everything was written for myself and my selfish needs first.
The tty/sixel world is very small world anyway, we generally know and recognize eachother, so we know where to look for cool new stuff :)
If you keep an accounting of how many lines you have displayed since then, you could alter the response to RCP by also doing the appropriate amount of scrolling: it should only take one variable, the deltaY to scroll.
If you want to test the idea, I think you could even use tmux and send commands to control the scrollback cf ahttps://superuser.com/questions/209437/how-do-i-scroll-in-tm...
I've used similar tricks with RCP/SCP but for simpler things: the only slight difficulty is the deltaY accounting, like when you are executing commands near the bottom of the screen because you must take into account that scolling will happen - but it's essentially similar to your idea.
So check https://github.com/csdvrx/bash-timestamping-sqlite/blob/main... and make sure you understand both how the __notbottom function works, and why PS0 needs an extra Esc[2a
Actually, now that I think more about your idea, it would be sweet to keep a SCP/RCP stack with multiple values, where you can push values with each SCP then pop them with RCP, say in sequence, or maybe just access the nth value with a different command that wouldn't pop them? That could be done nicely by augmenting RCP.
Also you could augment SCP with an optional flag to specify whether the terminal should scroll back upon RCP of this nth entry, and you'd have a great function that would be quite useful (ex: SCP with a jump bool when the return is non 0: you could make a shortcut to jump to the commands that have returned errors)
There's no reason to stop adding cool features to terminals: we're in a terminal renaissance!
anyway the content looks good so will try to find time to sit down and read this "essay" soon
The final example is both a total treat and a beast. I encourage anyone with any interest to chuck the code into GCC and behold the interactive experience for themself.
It's interesting, because I sometimes also find colors distracting, so for my Vim and other terminal tools I make an intensive use of text attributes (like bold, italic, underline and their various combinations)
Your comment may incentivize me to release my monochrome vimrc: it looks quite good on mintty/msys2 or Windows Terminal, and still very decent with xterm!
In fact they mention various issues with their approach that nurses would fix automatically and without the user having to configure stuff in the app:
> sadly, true color isn't supported on many terminals, urxvt tragically included. for this reason, your program should never rely on it, and abstract these settings away to be configured by the user.
Apparently they don't care about the great terminal-independence ncurses offers because these ANSI sequences will only work fully on terminal emulators. And as they mention even those differ in supported features. Ncurses was created precisely to abstract these differences. It's not just good for "obscure dumb terminals from 1983". It would also be quite a pain having to deal with terminal resize events etc when building a TUI. And no, doing a full screen rewrite for each minor update does not make for smooth TUIs. Yes modern hardware is fast but sometimes you're on a slow SSH connection.
Personally I still use real terminals too at times (I own a real VT520 and love it) and apps that are totally ignoring termcap/terminfo are super annoying. But I know this is niche.
PS: Also from the article:
> also, i have effectively zero pull in the tech community and am also kind of a controversial character who is liable to give projects a bad reputation, which i don't normally care about
Gee, I wonder why...
The tone and presentation seem calculated to annoy HN readers (nice) but this article has a bunch of difficult-to-uncover details with just enough context to apply them if you actually have a use for them and a desire to.
From the tone of this piece I gather that the ANSI escape codes are actually standard enough to target. Cool! Thanks for the heads up.
termfo[1] comes with a "termfo" CLI utility which – among other things – can group terminals by escape code; for example "termfo find-cap save_cursor" shows that almost all terminals use "\x1b7", with just a few very old ones using something different (full output is a bit long, but it's at [2]).
It's useful to check "can I safely hard-code this escape code?" But like you said: for ANSI it's pretty safe to just hard-code most codes, especially the common ones, but never hurts to check.
Correct. Hardware terminals are extinct in the wild, and essentially all software terminals (including the Windows terminal!) now support a reasonable subset of "extended VT100" terminal control sequences. Some of the weirder features of the VT100 (like double-high/double-wide text or VT52 compatibility mode) are usually omitted, and some features which were added in later DEC terminals (like color) are often added.
Even now there's big differences between terminal emulators. Especially when you take modern features like images into account, there's like 6 different ways of doing it as some terminals have invented their own (kitty for one) and there's more standard ones like sixels which come originally from the Dec VT340 series. Though they were ridiculously slow on that hardware so it didn't take off until much later.
Only since Windows 10 version 1511 (released 2015) which may or may not be too recent for you to assume support.
Sure, if all you care about is coloring text on recent terminals, I do think just using ansi codes is fine.
However, I completely disagree with the assertion that hardcoding ansi codes is somehow “more readable” than using a well-named API call that abstracts such detail away from the user. Maybe I’m alone in this, but I really don’t want to waste brain space memorizing ansi codes. Yeah I can write my own library or little macros to do this but…why? If I’m doing anything more involved than coloring text why wouldn’t I just pull in the lib I need? There’s no way in hell I want to maintain a custom implementation of terminal cursor movement for kicks. I’d much rather reach for a battle tested library that scores of other devs have already used and improved upon, whether it’s ncurses or one of the more modern TUI libs.
Also, it’s become cool recently to hate ncurses for some reason. Are there things ncurses should do differently from today’s perspective? Sure. Is it’s api awkward to use in the face of modern programming features and paradigms that we’re now used to? Sure. But on the whole it’s design is really solid and it has plentiful documentation. Whenever I see rants like this I get the impression the author is a person that lacks an appreciation for history and the way technical development proceeds in general and imagines they are our technical savior, come down from the heavens to enlighten all us blindered fools about how terrible X is.
If you want to educate people about ansi codes, great. A really inefficient way to do that is to wrap your advice in an obnoxious rant.
/s
Sure, going with VT102-family codes probably covers somewhere between four and seven 9s of cases, but...
And terminfo, pty, ctty, process groups, and the rendering width of a utf-8 string, and, and and…
> i might be one of a handful of people left on earth who actually has the knowledge to
Uh, no. Anyone on BBSes in the 90s is very aware of ANSI, thank you. And we've not died off yet.
And honestly it's really not that hard at all. Even on the topic of terminals this is not even the hardest aspect of it.
> i have effectively zero pull in the tech community
If you think there's all there's to terminals, yet you act as if you know it all, then you shouldn't have any pull.
I would also not accept any pull request that looked even remotely like their example code.
Here's a more interesting article about a less understood aspect of terminals: http://www.linusakesson.net/programming/tty/index.php
https://hn.algolia.com/?query=The%20TTY%20demystified&sort=b...
You guys should be more vocal. When you have great knowledge like that, you can't just keep it to yourself as a fond memory. People wouldn't be saying what the OP said if more of the oldskool crowd was out there blogging and mentoring the younger generation.
It's not obscure.
One thing I actually agree with the author about is that these escape codes are the only relevant thing. Outside of retro computing nobody should care about supporting anything else.
Other programs doing this are not exactly in short supply. Anyone can do "ls --color /bin/ls | hexdump -C" and see the secret sauce.
I blog about various things, partly to help out people who are less experienced. But I don't pretend what I write about is some sort of lost art, that "only a handful of people know".
Like "how do I make bold text in linux terminal" gives as second result this:
https://askubuntu.com/questions/528928/how-to-do-underline-b..., which also links to https://misc.flogisoft.com/bash/tip_colors_and_formatting.
Searching on youtube I immediately got this, for those who prefer in video form: https://www.youtube.com/watch?v=OL21-EnsNjQ
And it manages to do this without the pure arrogance of "i might be one of a handful of people left on earth who actually has the knowledge".
Seriously, this whole article could just have been just:
"Q: How do I output more than just print black&white text to the terminal? A: ANSI escape codes. See https://en.wikipedia.org/wiki/ANSI_escape_code"
Not "everything you wanted to know about terminals".
Like, "how does Ctrl-C work? What's flow control?". No, this post is entirely about ANSI codes.
What's extra frustrating is that you too are calling this "oldskool crowd". I'm just not that old, and this just isn't forgotten. It's simply another tool that people use when they need to solve the problem of colors, etc.
Just because many people don't know how a malloc()/new becomes a mmap or sbrk doesn't make it "oldskool". It's simply a thing that many people have not learned yet, because they haven't needed to. If and when they need to it's quite documented and many others know it, if they want more hand holding.
Like say I didn't know how garbage collectors worked. I don't go "Oh you older generation of lisp programmers, you need to blog more and teach us younglings, so that we can understand the languages that we use". Sure, blogging etc about GCs is good, but who would be arrogant enough to just write an article about "old gen and new gen" and call it "everything you ever wanted to know about GC" and claim that they are one of the handful of people who understand GCs.
Then I noticed Pygments, a Python-based library for pretty formatting source code, has an option to output an ANSI formatted file. I quickly found a bunch of libraries that could render ANSI formatted text to a print canvas.
In the end, I put the original source code file through 'pygmentize -16m -o tempfile.an` (`16m` is the 16M color terminal ANSI formatter) and pipe the `tempfile.an` through a print-optimized renderer to actually print the source code.
ANSI escapes FTW!
[1] WinPrint - https://github.com/tig/winprint [2] https://pygments.org/
Nothing like as powerful as your app and entirely tangential to the topic of ANSI escapes, but my preferred way to generate HTML from source code is simply:
vim +TOhtml +wq +q path/to/source/file.ext
I’m not even geriatric.
<meta name="viewport" content="width=device-width, initial-scale=1">
The author additionally uses the writing technique of "enallage", defined as a "slight deliberate grammatical mistake that makes a sentence stand out," [2] from the article excerpt: "in other words, we need to use termios. the ugly side of termios."
In other words: the author writes in a highly informal, colloquial tone with a stream of consciousness narrative style. The article is notable for its vulgarity and usage of enallage to achieve its exceptionally high degree of informality. The author is reminiscent of James Joyce as an experienced C programmer.
[0] https://liberalarts.oregonstate.edu/wlf/what-stream-consciou...
[1] https://english.stackexchange.com/questions/29720/whats-the-...
Also, it will include the null terminator. Probably won't do any harm but quite silly if you're redirecting to a file for example. I'd subtract one, or use strlen which would cover the pointer case above and I'd hope a modern compiler would elide the call on a string literal anyway.
[1] #define szstr(str) str,sizeof(str)
Almost every day the mandatory prayer "I wish they had used curses"
It's not hard to find yourself with an incorrectly configured terminal on a modern Linux distro. For example, try running emacs inside tmux inside rxvt-unicode and find out how Ctrl/Shift/Ctrl+Shift with arrow keys are bound.
It seems less and less people want to use the terminal.
All the new developers I notice want to stay in the IDE and almost never want to get their “hands dirty”.
Thanks for writing that great resource.
No, terminal emulators differ all over the place, especially for new features. Do you want to lock yourself in and only use the ancient ANSI codes? In, like you say, “THE TWENTY FIRST FUCKING CENTURY”? Great, you do that.
What’s that you say? You want to use modern features on terminals which support it? Do you then write “if getenv("TERM") == "spiffy-terminal"”? Congratulations, you’ve just begun implementing your very own ad hoc, informally-specified and bug-ridden terminal UI library.
> my hope is that this tutorial will curtail some of the more egregiously trivial uses of ncurses and provide others with the knowledge needed to implement a 21st-century terminal UI library
If you don’t like curses specifically, then don’t use it, but there are now myriads of alternatives.