I'm not certain if it's exactly what you're getting at, but less can be instructed to parse ANSI color sequences, and output them, with the -R flag. (If you forget to put -R on the command line, or some utility automatically starts less for you, you can also type "-R" while in less to trigger that behavior.) That said, less has to be fed the sequences, and most programs will not output color sequences when piped to less, as they detect that the output isn't a TTY (this is normally the right thing to do, it just kinda sucks in less's case) so you might need to, e.g., that_prog --color=always | less.
> There's just no way for exa to know that it's being piped into less while keeping the above rule true too.
Yeah, this is the real problem. I kind of wish there was some sort of content negotiation between piped utilities, s.t. less could inform the upstream utility of "hey! I speak ANSI color codes!".
What follows is definitely well into the "WTF too clever" column and I do not recommend it, but, this is HN: I presume you could riff through /proc trying to find the other end of the pipe that you're writing to (I think /proc on Linux has that) and then, if the executable's name is "less", well, you see where I'm going…