What do you mean by "UNIX shells suck"? I'm fairly sure most people would consider UNIX shells to be one of the bigger revolutions in computing.
Things are better now with builtin tests, but god help you if you need to use /bin/[ and you end up with idioms like 'if ["x$foo" = "x$bar"]' so that bad things don't happen if $foo starts with a dash, or either variable is empty.
It gets even more fun when you look at the amount of cruft around terminal emulators themselves, and the support for legacy terminals in the command line. Why do modern versions of OSX still ship with /usr/lib/libtermcap.dylib?
Don't get me wrong -- UNIX shells are amazingly powerful tools, but they're just stuck in the 70s and we can do much better today.
The security ceremony required around PS is also an obstacle to getting started. And of course Microsoft have missed the opportunity to fix the path separator.
I admit that Bourne shell is not great for anything over a few lines, and find it better to drop into perl or python where filename-separator issues are mostly eliminated.
Sounds like you are looking for Tee-Object[0].
"Saves command output in a file or variable and also sends it down the pipeline."
And of course Microsoft have missed the opportunity to fix the path separator.
Windows supports both path separators in pretty much all cases (open files, changing directories, etc)
PS C:\> d:
PS D:\> cd /projects/node_modules/immutable/node_modules/uglify-js/node_modules
PS D:\projects\node_modules\immutable\node_modules\uglify-js\node_modules>
I guess this depends on what you want out of the system. Some options are:
"format-table -auto" which provides a nicely formatted table view of the data
"format-list" which provides each property on a separate line with a blank line between objects
"format-custom" which allows you to create your own views of objects (with a very readable default), which has ultimate flexibility
"converto-csv" which output the object as a CSV (or with custom delimiters such as TSV)
"converto-html" which outputs the object as an html table (with somewhat customizable markup)
"convertto-xml" similar to the above
But really, if it's possible, then it makes sense to use the object pipeline until you're done (so that different parts of the pipeline can use different parts of the object). Then spitting out CSV, TSV, XML, etc should get interop with basically every other tool that's going to process text in some way.
[0] - https://technet.microsoft.com/en-us/library/hh849937.aspx
I feel like the interface for these commands is exactly what you'd want from a collection of executables that do only one thing.
As for shell scripting, you don't have to use bash/sh/whatever. You could write your script or executable in any language that will let you call system commands, so it's really a non-issue if you absolutely can't stand the syntax. I write plenty of bash scripts and I do feel your pain, they're not usually very pleasant if you have to go beyond trivial use cases- but it's also (generally) not impossible.
Hadn't thought of chaining that with a csv-to-json workflow, that's a neat idea -- but if that's the easiest way out, that's clearly a symptom of the problem I'm describing, isn't it?
But as soon as you want to do anything moderately interesting, it's better to grab a real programming language, because UNIX shells suck. Escaping failures, whitespace handling, almost-real-arrays, fail-and-continue by default, and many many other reasons you'll fail by accident one day is why they absolutely suck for almost anything past basics.
This is, in my opinion, the beauty of UNIX shells. They're simply enough to use like DOS but powerful enough to use like a REPL. It's not perfect; far from it. But few other tools support such a wide demographic.
The shell languages are not much better.
There are some useful concepts, but in general Unix shells are used despite their user unfriendliness.
The shells of the various Lisp Machines were quite different. The Symbolics shell, later called Dynamic Lisp Listener was quite nice on the GUI side and the management of commands, completions, defaults, interactive help, etc..
See for example: https://www.youtube.com/watch?v=o4-YnLpLgtk
The interactive help of that Lisp Machine OS is quite a step up from what any typical shell offers. Though it was mainly developed for single user machines with powerful GUIs.
The problems of that approach: it wasn't very sophisticated on the text terminal - actually it was quite bad. The whole UI mostly assumed a GUI. For development one needed an extended Common Lisp (or Zetalisp), which is a bit too complex for many users.
See also a video I made long ago about the user interface of Symbolics Genera, the operating system of the Symbolics Lisp Machine line of personal workstations.
In both cases, for small uses, you can stretch your shell or REPL or programming language over to the other case, but for long term use it's just not practical. After all, in theory many languages have REPLs that could have long since replaced shell, but only a handful of very dedicated people actually replace their shell with a REPL for some language. I've tried, and I can't do it, even with shell support libraries in the relevant language.
The suckiness of UNIX shells is mitigated if you view them as "the tool optimized for interactive usage". And once you let them be that, they really aren't so bad. It's only if you try to force them to be programming languages, especially past a couple dozen lines or when they try to do something other than just run lots of commands pretty much blindly, that they become really bad.
Also complicated quoting is a pain, but as a percentage of the number of commands I run in my shell, they aren't that large, they just loom large in my memory. Run history and really look at what you're doing day in, day out with your shell. Unless you've got a really crazy usecase, there's a lot of things like "ls" followed by a "cd dir" and such. It's part of why I can't get into REPL replacement; replacing "cd tmp" with even "cd('tmp')" is, percentage-wise, a huge increase in keystrokes. (And don't forget, the space is much cheaper than left paren; the space bar is huge and I basically have an entire digit dedicated to it, whereas left paren is two not-on-home keys at the same time.)
I had been juggling the idea of using a modified REPL for shell work, but I haven't written much code. However, someone has written such a shell, namely Avesh[0]. Check out the Reddit comments as well [1].
I haven't used it yet, but the examples seem to fit what I would want in a Common Lisp shell: * Simple sh-like commands with flags (e.g. ls -o, grep -i) * Handling commands and piping as native CL functions (e.g. | (| ls (grep c))(grep r) ).
[0] = https://gitlab.com/ralt/avesh [1] = https://www.reddit.com/r/lisp/comments/48r70b/avesh_the_supe...
Everything's an object. Instead of passing lines of text, you're passing objects, with properties, types, etc. This becomes important because of
Library files are first class citizens. Because the shell is string based, you have to wrap everything up in functions, creating helpers in order to use any sort of function. Powershell makes this trivial, such that you can use functions you wrote elsewhere in your scripts as if they were builtins. Hell, it's trivial to write a webserver that forks multiple processes in powershell.
Yes, the unix shell was revolutionary when it was released, but it's been hamstrung by tradition. The shell is long overdue for letting in more interesting abilties.
I get why Powershell is great for sysadmins (and especially on Windows, where so many system-level utils weren't built with the idea that you'd pipe their output in mind), but for me, it's a step back.
Having said that, I can't vouch for your scripts, but my 10 line shell scripts are unlikely to handle 'edge' cases such as spaces, quotes and backticks in file names.
Also, "incomprehensible to anyone who doesn't know .net" may be true, but your average shell script that builds on tools such as sort, grep, ls, cut, cat, sed, awk, etc. isn't that particularly comprehensible to people who don't know UNIX, either.
Finally, I think it is likely that people who grow up on Powershell feel differently about this.
Hybrids like ipython shell are quite interesting for personal use though.
If I'm spending twice the time typing on average, even more advanced features won't win me over when I get to use them occasionally (while, again, typing way more on average). It should be saving you time, in the end.