Recently discovered CMus and am a big fan of Pandoc and youtube-dl.
This has me wondering what else I'm missing out on.
history | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl | head -n10
1 167 16.7% git
2 130 13% ll
3 94 9.4% clear
4 90 9% make
5 80 8% cd
6 57 5.7% e
7 52 5.2% cat
8 45 4.5% man
9 38 3.8% valgrind
10 37 3.7% rm
1 401 16.04% git
2 337 13.48% fg
3 241 9.64% cargo
4 234 9.36% cd
5 197 7.88% ls
6 195 7.8% tig
7 165 6.6% task
8 81 3.24% vim
9 38 1.52% imag
10 31 1.24% pwd
1 93 18.6% l
2 75 15% exit
3 60 12% zsh
4 53 10.6% cd
5 47 9.4% nvim
6 20 4% ssh
7 14 2.8% lt
8 14 2.8% ls
9 14 2.8% la
10 10 2% sudo 1 178 35.5289% gi
2 90 17.9641% ls
3 77 15.3693% vim
4 51 10.1796% fg
5 28 5.58882% cd
6 20 3.99202% jobs
7 10 1.99601% date
8 6 1.1976% pass
9 5 0.998004% docker
10 4 0.798403% sudo
but when I try to save it as an alias method.. I keep getting error:
awk: cmd. line:1: {CMD[]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;} awk: cmd. line:1: ^ syntax error awk: cmd. line:1: error: invalid subscript expression
I thought I would just escape the double quotes ", and things would work. :/ Any ideas?
alias mostused='history | awk '\''{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}'\'' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl | head -n10'Use a shell function instead of an alias, that's the better tool here.
clear && make all && ./a.out - fd: A simple, fast and user-friendly alternative to 'find'
https://github.com/sharkdp/fd
- ripgrep: ripgrep recursively searches directories for a regex pattern
https://github.com/BurntSushi/ripgrep
- exa: Replacement for 'ls' written in Rust
https://github.com/ogham/exa
- bat: A 'cat' replacement. I recommend following the customizations.
https://github.com/sharkdp/bat
- asciinema: Terminal session recorder
https://github.com/asciinema/asciinema
- httpie: Modern command line HTTP client
https://github.com/jakubroztocil/httpie
- visidata: A terminal spreadsheet multitool for discovering and arranging data
https://github.com/saulpw/visidata
- youtube-dl: Command-line program to download videos from YouTube.com and other video sites
https://github.com/rg3/youtube-dl
- pgcli: Postgres cli with autocomplete and syntax highlighting
https://github.com/dbcli/pgcli
- jq: Command-line JSON processor
https://github.com/stedolan/jq
- tmux: a terminal multiplexer
https://github.com/tmux/tmux
- pspg: postgres pager (you can combine it with pgcli)
https://github.com/okbob/pspgExactly! And we all know how good most programmers are at UI design ;-)
Written in Rust
- fd: A simple, fast and user-friendly alternative to 'find' https://github.com/sharkdp/fd
- ripgrep: ripgrep recursively searches directories for a regex pattern https://github.com/BurntSushi/ripgrep
- exa: Replacement for 'ls' written in Rust https://github.com/ogham/exa
- bat: A 'cat' replacement. I recommend following the customizations. https://github.com/sharkdp/bat
Written in Python
- asciinema: Terminal session recorder https://github.com/asciinema/asciinema
- httpie: Modern command line HTTP client https://github.com/jakubroztocil/httpie
- visidata: A terminal spreadsheet multitool for discovering and arranging data https://github.com/saulpw/visidata
- youtube-dl: Command-line program to download videos from YouTube.com and other video sites https://github.com/rg3/youtube-dl
- pgcli: Postgres cli with autocomplete and syntax highlighting https://github.com/dbcli/pgcli
Written in C
- jq: Command-line JSON processor https://github.com/stedolan/jq
- tmux: a terminal multiplexer https://github.com/tmux/tmux
- pspg: postgres pager (you can combine it with pgcli) https://github.com/okbob/pspg reply
- rq: A tool for doing record analysis and transformation (replacement for jq, supports yaml, json, avro, protocol buffers, etc.) https://github.com/dflemstr/rq
Agree on links tho.
because I never had to lookup how to use it
EDIT: hm it doesn't even have a man/info page. Where can you learn about "cd" or "cd -", then?!
Some systems, like OS X and CentOS, map the cd man page to builtin which lists all the shell builtins and lets you know you should look at your shell's man page.
See 1.4.1.7 here for how bash searches for commands: http://www.tldp.org/LDP/Bash-Beginners-Guide/html/sect_01_04...
It does on Unix and BSD systems.
/r/commandline https://www.reddit.com/r/commandline/
general commands
- awk/grep/sed/wc - text file processing
- convert (image magick) - image processing tool
- cpulimit - limit the cpu usage of a process
- fdupes - check for duplicate files and optionally remove them
- pdftk - concatenate pdf files and more - https://www.pdflabs.com/tools/pdftk-the-pdf-toolkit/
- rsync - back up tool
- tmux - terminal multiplexer - almost always my first install
- tree - view directory contents in a tree format
- zcat/zmore - display a file that is compressed (e.g. zmore /var/log/syslog.2.gz)
security/reverse engineering:
- binwalk - search binary images for embedded files
- gdb + peda - debugger with extensions - https://github.com/longld/peda
- nmap - network scanning tool
- radare2 - command line dissassembler - https://github.com/radare/radare2
- scapy - network packet crafting - https://github.com/secdev/scapy
- tshark - network traffic analyzer
- tcpdump - network traffic analyzer
- xxd - hexdump tool
Bonus:
- find - find files - I often use this to find files containing a string:
$ cat bin/findstring
#!/bin/sh
find . -type f -exec grep -i -l "$@" {} \;
Pdftk is a must. I use pdflatex also a lot of course. And R and psql.
- git-standup http://github.com/kamranahmedse/git-standup
- z https://github.com/rupa/z
- ZSH https://github.com/robbyrussell/oh-my-zsh
- curl-size https://github.com/egoist/curl-size
- ripgrep https://github.com/BurntSushi/ripgrep
- jq https://github.com/stedolan/jq
- asciinema https://github.com/asciinema/asciinemaUnless you belong to the Prezto or Antigen sects. :)
It does show the commits from each repo sorted by time. Repositories are not sorted though, if that is what you are asking for.
Just ensure you're running a 256-bit color terminal emulator, and for most uses it doesn't even differ that much from the GUI version. Recently I've been doing a lot of work by SSH-ing to my desktop machine from a small netbook and using the terminal Emacs client. If you run Emacs in server mode, the transition is so smooth that at this point, I don't bother getting out of bed if I need to work on something late in the evening, I just take the netbook and SSH over.
du -sh *
Bonus tip: if you're using btrfs, it has a built-in du that keeps track of cow statistics, so you can see which files/folders are deduped and which aren't. It's invoked with
btrfs filesystem du -s *
btrfs fi du -s *Some webservices are still only available in xml (e.g. uv levels). I currently use XMLStarlet for xml in the terminal. There are extra annoyances in xml that you can't avoid (like namespaces), but it could be a lot closer to jq.
pup (https://github.com/ericchiang/pup/blob/master/README.md) which apparently inspired xpup does seem to have this, e.g. chaining, but unfortunately it uses css selectors instead of xpath.
I changed to 'ack' six months ago after having used 'grep' for two decades, and already its successor 'ag' has been made redundant in favor of 'rg'.
My only peeve with 'ack' is that I can't make it switch nicely between paging and not paging:
https://stackoverflow.com/questions/50135983/less-waits-when...
I doubt any of 'ack's competitors have a fix for this, since it seems to be a pager feature that I'm looking for.
This shell snippet lists all the directories in $PATH, using less, so you can page through them:
for dir in `echo $PATH | sed 's/:/ /g'`
do
echo Listing dir: $dir
ls -l $dir | nl
done | less
A Python version of the Linux watch command:
https://jugad2.blogspot.in/2018/05/a-python-version-of-linux...
I've also been writing Python versions of some other Linux commands, and will post about them here some time later.
Also, for saving (and then viewing) man pages (about commands or other topics), without the formatting characters (such as ^H), on some Linux distros, I find this useful:
m, a Unix shell utility to save cleaned-up man pages as text:
https://jugad2.blogspot.in/2017/03/m-unix-shell-utility-to-s...
^H is one of the two backspace characters (the other being ^?) depending on which VT standard you're configured to use. So you shouldn't really need a dedicate script to do that as it's just a problem between your pseudo-TTY (Linux console) and your terminal emulator (PuTTY / iTerm / xterm / etc).
Thankfully both are easily configurable. However without knowing which terminal emulator you're using I couldn't walk you throw configuring that. In terms of the TTY; you can change the backspace character via the `stty` command:
stty erase [hit backspace on your keyboard]
It isn't because the terminal is mis-configured and/or mis-interpreting those characters, it's because these characters have historically been used in a special way by troff's ascii output driver, as used by the man command.
Bold characters are "emulated" by outputting the character itself, followed by ^H, then by repeating the character, and underline is emulated by outputting _ (the underscore character) followed by ^H, followed by the character to be underlined. This is the same way that bold/underline was achieved on a manual typewriter or by old teletype terminals with paper output.
Pagers like "more" & "less" have in-built behaviour that knows how to interpret these sequences and render bold or underline appropriately, and if you "cat" the file your terminal would probably ignore them, but if you open a file with those sequences in a text editor, you're going to see a bunch of unnecessary ^H characters. The OP's script uses the "col" command to remove the unnecessary ^Bs (and the preceding character) that troff has output.
By default, GNU groff doesn't actually output those sequences anymore and uses ANSI escape codes instead. AFAIK many (most?) distributions actually compile out the ANSI behaviour in favour of the old way though (because "more" and "less" don't actually behave correctly with the ANSI characters by default), but some don't, which breaks the script (it's broken in Cygwin, for instance).
FWIW, if you need a fool-proof way to convert a man page to plain old ASCII with no escapes at all, it's easiest just to redirect the output of "man" to a file:
man ls > ls.txt
zcat /usr/share/man/man1/ls.1.gz | groff -man -Tascii -P -cbdu > ls.txtI first created that script called m years ago on some Unix boxes that I was using. Could have been HP-UX or other version. And I've used it over the years on many Unix and Linux versions that I worked on. It could be that in some cases, the script is needed due to a tty or TERM or other configuration issue. But I'm pretty sure that I've had the need for it (to remove those control characters) on at least some systems where such config was okay. I know this because, while I do tweak env. var., stty and other settings now and then, I do not always (need to) do so, and have still found control chars [1] in the man output, even when redirecting to file. That is why I created the script, because when working on a C, Python or any other project where I need to read man pages, I often like to redirect the man output to a file (in my ~/man dir) and then read them using vi/vim.
[1] See what I say in that post (about m) about nroff and troff.
Also, I do know about stty, have used it for years, although there is less need for it these days. Used to do a lot of tweaking and experimenting with the erase option (for which, BTW, instead of backspace key in your example, we can also write a literal ^h, i.e. caret, then letter h), intr option, onlcr, ocrnl and others. Used to be good fun and sometimes frustrating too, because docs for this areas were somewhat lacking then.
slrn http://slrn.sourceforge.net/ for reading newsgroups
My old fave (which I didn't even remember the name of) was
... which had too steep a (re)learning curve for reading just the one newsgroup.
And `z` (the zsh plugin) to cd into your most common directories
Fuzzy Finder for files, history and more. Integrates with Vim (and maybe other editors). It is wicked fast.
beets http://beets.io/ for library management
mpd https://www.musicpd.org/ for serving music &
ncmpcpp https://rybczak.net/ncmpcpp/ for playing music (which I alias to `music`)
- tmux
- mutt - email client
- wyrd + remind - calendar
- ledger - double-entry accounting tool
- weechat + weeslack plugin - fully featured, super fast slack from the CLI
I used screen, remind+wyrd, mutt, and irssi (and even ledger) in school; I got a job at a bigco that pretty much required me to use gmail and gcal, and I stopped using console things as much. Lately I've been toying around with moving my mail flow back into console / console-ish inside emacs.
This brings back all kinds of memories: https://www.roaringpenguin.com/wiki/index.php/Remind_use_cas...
I heavily use git, but via Magit in Emacs.
[1]: My own little utility to use bare bones Emacs as less/pager: https://eless.scripter.co
fortune: prints something silly or thought-provoking. You can get extra fortune cookie files or add your own.
cowsay: prints an ascii-art cow (or other creature) with a speech-bubble filled with the text of your choice. You can pipe fortune into cowsay for extra fun.
$ fortune | cowsay
______________________________________
/ Publishing a volume of verse is like \
| dropping a rose petal down the Grand |
\ Canyon and waiting for the echo. /
--------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||https://github.com/erkin/ponysay
My latest find is to use a combination of weechat and wee-slack for most of my slack usage (it's note quite there when it comes to editing, or in-line reactions). But works well for the chat part:
https://github.com/wee-slack/wee-slack
As a former BitchX and sometime irssi user, weechat feel cozy. And it doesn't eat all your ram just for chat.
alias dl-audio='youtube-dl --extract-audio --audio-format mp3'
If you have ffmpeg installed, just drop the video ID after dl-audio, and you'll get an mp3 of the highest quality audio stream available for the video.
zsh: Stepping up your terminal game - https://github.com/robbyrussell/oh-my-zsh
tmux: A terminal multiplexer, aka turns your terminal to a minimalist workspace (the only sane way I found to copy terminal output without using a mouse)
howdoi: Search stachoverflow/stackexchange from the CLI - https://github.com/gleitz/howdoi/
nmtui: WIFI/Networking from the terminal - https://github.com/heftig/NetworkManager
bluetoothctl: Bluetooth from the terminal
___________________
Lists for exploration:
- https://en.wikipedia.org/wiki/List_of_Unix_commands
- https://wiki.archlinux.org/index.php/List_of_applications
- https://github.com/alebcay/awesome-shell
- https://github.com/agarrharr/awesome-cli-apps
- https://github.com/herrbischoff/awesome-command-line-apps
(The github links seems similar, but there are some differences indeed)
Some other useful CLI programs that I use and are not part of POSIX are: strace, perl, webalizer, rcs. RCS might be a mostly-forgotten VCS tool, but it is very suitable for version-controlling projects that consist of single files. I mostly use it for plain-text or Org-mode documents.
If you meant terminal-UI (TUI) programs, like top/htop, vi, etc., I don't really use them. But I've used these in the past:
- mutt: a very nice mail client
- newsbeuter: RSS/ATOM feeds client
- vi: I do occasionally use vi in some situations
In the past years I've started to use Emacs tools instead of these because Emacs is a very rewarding, composable and comfortable working environment. Still, I use CLI programs through Bash/shell-mode all the time, because it's the most efficient way to do many tasks.
Great way to give you a progress bar on a long-running process from the command line.
entr is useful for doing stuff on file change:
GNU parallel is like xargs, but in parallel:
I switched to ripgrep for my grepping needs, I find it fast and the syntax and output coloring handy.
Just found fastmod recently, https://github.com/facebookincubator/fastmod. Very handy for doing bulk substitutions of strings in project directories - it can prompt with a diff for every change and do full regexes, so you can try out tricky things and make sure it isn't doing anything crazy before you let it make 500 changes throughout your repo.
Also a fan of gitsh, https://github.com/thoughtbot/gitsh. Helps out with using Git on the command line just the right amount IMO - just avoiding the git prefix, tab completion, and a little syntax highlighting, instead of trying to redo the whole interface.
I don't use it that much, but jq, https://stedolan.github.io/jq/ is really handy for slicing and dicing json data without diving into the complexity of writing actual programs.
And on OS X, homebrew is indispensable for actually managing all of your CLI programs.
I also have found this Google Drive cli handy, https://github.com/odeke-em/drive. If you want to sync some but not all of your Google Drive files, and only push and pull on command instead of continuously, it's great.
For news, newsboat is quite nice:
For parsing and select actions over url links you can use urlview (it works perfectly with pdftotext, newsboat, mutt, vim, w3m, etc.):
https://github.com/sigpipe/urlview
You can download webpages with httrack:
rsync is a fundamental tool to move files between computers or do backups, forget about scp:
You can work with most pdf and ps (those that are not scanned images) using your usual text tools,
There are even ready to go pdf + grep commands:
Somewhat meta, when you live in the terminal you need to be sure that you are a bit POSIX compliant. shellcheck is a good tool for that:
Finally, there are a two packages that should be in any distributions (include commands like vidir, parallel, watch, etc.):
https://joeyh.name/code/moreutils/ https://gitlab.com/procps-ng/procps/
>This blog is intended as a quick look at terminal-based applications available in Linux, and in some similar operating systems. Not only are these ideal for modern, low-power and space-saving computers, but they go a long way toward invigorating out-of-date hardware.
$ time locate -0 /{opt,usr}/\*.gitignore |
xargs -0 stat --format=%s | wc
387 387 1225
real 0m0.087s
$ time find /{opt,usr} -name '.gitignore' -print0 2>/dev/null |
xargs -0 stat --format=%s | wc
387 387 1225
real 0m0.748s
$ sudo updatedb
A collection of tools for slicing and dicing csv files on the command line.
export MARKPATH=$HOME/.marks
function jump {
cd -P $MARKPATH/$1 2> /dev/null || (echo "No such mark: $1" && marks)
}
function mark {
mkdir -p $MARKPATH; ln -s $(pwd) $MARKPATH/$1
}
function unmark {
rm -i $MARKPATH/$1
}
function marks {
ls -l $MARKPATH | sed 's/ / /g' | cut -d' ' -f9- && echo
}
_jump()
{
local cur=${COMP_WORDS[COMP_CWORD]}
COMPREPLY=( $(compgen -W "$( ls $MARKPATH )" -- $cur) )
}
complete -F _jump jump
Local documentation:
wget --recursive --level=5 --convert-links --page-requisites --wait=1 --random-wait --timestamping --no-parent $1
- googler: Google from the terminal
- Buku: Bookmark manager like a text-based mini-web
- nnn: The missing terminal file browser for X
- bcal: Storage expression calculator
- imgp: Multi-core batch image resizer and rotator
- keysniffer: Linux kernel mode debugfs keylogger
- ddgr: DuckDuckGo from the terminal
- pdd: Tiny date, time diff calculatorhttps://github.com/basic-gongfu/cixl/blob/master/examples/ci...
PNG viewer:
https://github.com/basic-gongfu/cximage/blob/master/examples...
Linux/64 binaries may be found here:
- htop - a graphical top
- xonsh: A shell written in Python. You can now write your shell scripts in Python (with access to any Python library)
- mplayer: Video player
https://gist.github.com/jamestomasino/9d912b409e5b6067ac233e...
Also of interest, my custom functions:
https://github.com/jamestomasino/dotfiles/tree/master/bash/....
And finally, my OSX & Linux Mint install scripts which include a lot of great app packages:
https://github.com/jamestomasino/dotfiles/blob/master/osxset... https://github.com/jamestomasino/dotfiles/blob/master/mintse...
glances https://nicolargo.github.io/glances/
rtv https://github.com/michael-lazar/rtv
rainbowstream https://github.com/orakaro/rainbowstream
discurses https://github.com/topisani/Discurses
telegram-cli https://github.com/vysheng/tg
https://github.com/dwwmmn/z.ps1
It's a little simplistic for now but I hope it helps someone.
Often, I need to do a test run of something which involves slightly rerouting network traffic and I'll spend half a day hacking together some horrible "solution" that I could literally have done in 3 lines of bash code on a UNIX-like system. Maybe throw hexdump for looking at binary stuff in there too.
Edit: Oh, and of course GDB! If only I could have the accumulated days of log forensics back that were done just because it's still faster to do that instead of getting the visual studio debugger to work remotely. Whatever happened to KISS? -_-
https://news.ycombinator.com/item?id=17019197
It's a bit like a mix of (x)stow and apt for Windows. Or the good parts of arch aur with binary distribution.
Should probably also mention that I'm experimenting with asdf as a version manager - to avoid a long list of setup in bashrc to adjust paths to various interpreters. It follows a lot of best practices that various dedicated version managers seem to pick up eventually:
* dcraw - RAW photo processing tool
* imagemagick - image processing library
* pandoc - convert one file into another
* diceware - random word generator (useful for creating passwords)
* csvtomd - convert csv files to markdown tables
* newsbeuter - RSS reader
* zathura - pdf viewer
* weechat - irc client
* ncdu - interactive disk usage
* ranger - file manager
* mpd - music player daemon
* vimpc - vim inspired client for mpd
> What else I'm missing out on
echo 'Currently there are ' $(ver=(/bin /sbin /usr/bin /usr/sbin) && ls ${ver[*]} | wc -l) 'commands installed and available in the system'
This prints the man page of 3 random commands. One by one. Is a way to discover a few new commands each day. To go to the next command and exit press q. I hope you enjoy it.
commandoftheday=(/bin /sbin /usr/bin /usr/sbin) && ls ${commandoftheday} |sort -R |head -3 |xargs man '{}'
Why?
As web applications are being split in micro services and independant entities, it becomes harder and harder to setup a local dev environment and you have to handle dozen of shell terminals.
GCE aims to take that pain away by offering you the ability to configure the setup of your projects once and run it in the blink of an eye.
Network mgt: ipcalc, mtr, wireshark, nc
Comms & Info: mutt, w3m, irssi, screen (though if you're just starting: tmux), swaks (smtp swiss army knife),
Code & Tasks: make, gawk, sponge (moreutils generally), xclip (pbcopy OSX), at, batch, and cron,
Calculators: units (GNU, not BSD), bc, & dc,
Editing & Docs: pandoc, vim, less ("&" filters!),
Files: curl, wget, scp, rsync, & lftp, mc, column, tree,
Images: mogrify / convert (ImageMagick),
Sysadmin: apt, grep-dpkg, apt-file, auto-apt, pstree,
tmux, neovim are great together (after some configuration.)
Mplayer seems better than mpv. They need configuration though.
htop/top are pretty useful for figuring out which applications are taking up cpu resources.
cpufreq-utils are useful to set max/min cpu speeds if you're running low on laptop battery.
meld is an amazing diff tool though not command line. vimdiff is useful too.
- `ffmpeg` one utility for video manipulation to rule them all.
- `ping` to check whether I am connected to the internet or not.
- `curl -vvv maktabkhooneh.org` to check where is the connection problem happens.
- `howdoi` search StackOverflow for snippets to solve your problem from the command line.I'm biased, though, because I'm the author. https://github.com/ironarachne/heraldry
- subliminal: Find & Download subtitles https://github.com/Diaoul/subliminal
$ alias | grep lpass
lp=lpass
lpls='lpass ls'
lppw='lpass show -c -G --password'
lpsh='lpass show -G'
lpst='lpass status'
cat file.txt | grep -v secret | sponge file.txtI used to enjoy "orpheus", and MP3 player. That's ncurses, which I think counts. http://thekonst.net/orpheus
- mc
- git
- ssh
screen
notmuch, isync
powertop
parallel, ag
A python cli tool for twitter
Parse CSV with SQL
https://www.google.co.in/search?q=decorate+sort+undecorate+i...