---
[0] HISTFILE="${HOME}/Sync/Dotfiles/history/$(date -u +%Y-%m-%d.%H.%M.%S)_${HOSTNAME_SHORT}_$$"; export HISTFILE
[1] https://github.com/junegunn/fzf
[3] __fzf_history__() {
sift --no-color -e "^[^#]" --files "_${HOSTNAME_SHORT}_" -N --no-filename $HOME/Sync/Dotfiles/history | sort | uniq -c | sort | $(__fzfcmd) +s --tac +m -n1..,.. --tiebreak=index --toggle-sort=ctrl-r | sed "s/ [0-9] *//"
}
Had to tweak mine slightly though:
${HOSTNAME_SHORT} -> ${HOSTNAME}
$HOME/Sync/Dotfiles/history -> $HOME/Sync/Dotfiles/history/*
sift searches a path/folder so the star you have is not needed.
But yeah, HOSTNAME_SHORT is something I derive from HOSTNAME.
export HISTTIMEFORMAT='%Y-%m-%d %H:%M.%S | '
$ history | tail
50198 2016-05-31 10:15.57 | cd docker
50199 2016-05-31 10:16.03 | cd rpms/
50200 2016-05-31 10:16.11 | scp docker* omniscience:/tmp/
50201 2016-05-31 10:14.06 | screen -ls
50202 2016-05-31 08:33.34 | screen -x
50203 2016-05-31 19:06.53 | grep HIST .bashrc
50204 2016-05-31 19:07.46 | history | tail
50205 2016-05-31 19:08.10 | task ls
50206 2016-05-31 19:08.23 | docker ps -qa
50207 2016-05-31 19:08.30 | history | tail
$ grep HIST .bashrc
HISTCONTROL=ignoredups:ignorespace
# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
export HISTFILESIZE=99999
export HISTSIZE=99999
export HISTTIMEFORMAT='%Y-%m-%d %H:%M.%S | '
export HISTIGNORE="ls:exit:history:[bf]g:jobs"
TLDR: Overwriting history is a feature.
It does depend on what one does on their computer, obviously, but in my opinion for developers having a more or less permanent history with a fuzzy matcher (fzf is what I use) is extremely important.
I personally have bash set up so it saves a new history file each month, and a custom fzf alias to search on all of them, and I wouldn't want to have it any other way, I also treat these history files as confidential and make sure they are not in git etc. and if I have to input any passwords on the command line I just prepend the command with a space so it's not saved
If you have administrative stuff you need to do more than once, write a little script or alias for it. Depending on history for this is just lazy.
Also, silly idea for a DOS attack vector: script-spam enough commands to have the audit history consume all available space on server.
I work around the security issues by not backing up history and having encrypted file systems on all of my Linux laptops. I don't save history on my servers.
Having an unlimited history is great. Sometimes there'd be a tool or resource I'd vaguely remember skimming (e.g., "oh yeah, I was working on ___, so it must have been on (day +/- 24 hours)". I'd find a command that more or less is chronologically synced with the command, then go into Chrome's SQLite history db in my User Profile to find it.
It's one thing to use porn as an example of "things that may be awkward if found in my bash history", it's something else to get judgemental about it. Unnecessary and off-topic.
export HISTFILE=
If you're using zsh, a tip I picked up from [0] was to alias all your common commands like
'cd', 'ls' 'fg'...
' cd' ' ls' ' fg' ...
setopt HIST_IGNORE_SPACE
[0] http://chneukirchen.org/blog/archive/2012/02/10-new-zsh-tric...
EDIT: formatting
It would be really clever if a series of cd/ls rolled into a final absolute path cd so history shows context for the subsequent batch of commands.
Instead of doing:
ls
cd foo/
ls
cd bar/
ls
cd baz/
ls
cd foo/<tab><tab>bar/<tab><tab>baz/
You can frequently avoid even more `cd`/`ls` shenanigans by judiciously using `pushd`.
function cd () { builtin cd "$@" && ls; }
HISTIGNORE="history:ls:[bf]g:exit:htop:ncdu:pwd:reset:clear:mount:umount:&:^[ \t]*"
Mine might be flawed though.
export HISTIGNORE="ls:exit:history:[bf]g:jobs"That is then an increased risk of breach because it is kept around for a long time. YMMV. Defense in depth, don't use production secrets in development, etc, etc.
Too bad the author didn't Grep "password" plus a few lines on either side. Even if you sudo <stuff> and type outside the prompt once every few hundred attempts it's still gonna turn up a lot after a year or more.
Even the most novice of adversaries would have a field day with the bash history of a lower level IT admin.
precmd() {
eval 'if [ "$(id -u)" -ne 0 ]; then echo "$(date "+%Y-%m-%d.%H:%M:%S") $(pwd) $(history | tail -n 1)" >>! ~/Dropbox/Logs/Bash/Macbook/bash-history-$(date "+%Y-%m-%d").log; fi'
}
I use `unset HISTFILE` to make sure when I exit the history file doesn't get all the cruft in. Every now and then I might do some housecleaning, including going through the history file then `cp ~/.bash_history{,.2}` to keep a clean copy around.
As far as saving, heres a handy function: `getlast() { fc -ln "$1" "$1" | sed '1s/^[[:space:]]*//' }`
Moreover, bash history doesn't work nicely across different terminal tabs or computers.
Does anybody know a tool providing better command line history?
There's several controls to make the bash history do what you want. The defaults are just a bit odd.
2. I remember a HN submission for a piece of software (or collection of softwares?) that provided a centralized log of your command history across multiple machines. In this way, if you are working on multiple machines in a cluster, you can later search back through your histories from those machines in a single location. My Google Fu is failing me now though.
I started this in response to a HN post some years ago.
There is one aspect in which it's significantly worse, though: it's fiendishly difficult to configure and understand. I just use configs provided by others, which is not ideal but works.
export HISTFILESIZE=5000000 export HISTSIZE=5000000 export HISTTIMEFORMAT="%F %T " shopt -s histverify shopt -s histappend
It's actually an insecure default - 'insecure' as in, it's a minor form of data loss.
Then again, so is bash launching scripts without pipefail, variable expansion fail, etc.
It'd be worth fixing all this stuff in the next major bash version.
histappendOne time it saved me from a crontab -r that wiped out a 100+ line crontab. I had viewed it recently so I just copied it out of my history.
On a day to day basis it's more about looking up old queries I typed out, the results of those queries at that time, bash commands and their results, the state of a file I edited at a certain time, a stack trace, the output of an ls -l command, etc. Anything I ever do in a session.
Some of those have their own logs or ways to capture history, but a way to capture everything at once is much more comprehensive and less sensitive to forgetting to set the size of the bash history on a given machine, archiving the bash/psql history files when a machine goes away, etc.
And besides not having to worry about the availability of distributed history/log data, being able to grep everything at once is invaluable.
http://unix.stackexchange.com/questions/25639/how-to-automat...
I have been wondering about doing something like this as well
script
I believe iTerm2 has some support for logging remote commands, but I'm not sure it quite does the trick. Anybody know of another tool that will log both local and remote commands?
At the moment, I get around this by storing as much as I can in searchable scratch files, but this relies on me knowing what to copy. I'm sure there's a better way.
So, basically, in combination with sp332's and raldi's points, the answer is that it is completely not the same.
## SANE HISTORY DEFAULTS ##
# Append to the history file, don't overwrite it
shopt -s histappend
# Save multi-line commands as one command
shopt -s cmdhist
# Record each line as it gets issued
PROMPT_COMMAND='history -a'
# Huge history. Doesn't appear to slow things down, so why not?
HISTSIZE=500000
HISTFILESIZE=100000
# Avoid duplicate entries
HISTCONTROL="erasedups:ignoreboth"
# Don't record some commands
export HISTIGNORE="&:[ ]*:exit:ls:bg:fg:history"
# Useful timestamp format
HISTTIMEFORMAT='%F %T '
That way I have control of when I log and what gets logged. And way easier to find than hunting for dot files.
it makes me sad when everybody doesn't see the same things the same way. I'm just waiting now for systemd to disable bash history in favor of logging in binary form all mouse rolls intermixed with key clicks till it wipes it all out when you log out in case yours is a student account on a shared university machine. Don't worry, a GUI editor for it all is on the wishlist.
I find screen logging a bit more useful than shell logging. Shell only records the commands sent and not the result. Screen records everything including interactive like nano.
I've been trying to leverage bash's history, and wanted to avoid putting a file write into my prompt hook. But I have a big kludgy system for exporting bash history separately for each day, de-duping the overlapping history, and trying to make sure bash always logs the command I typed.
I'm still fighting with bash sometimes not saving my history commands. I've done everything recommended in all corners of the Internet, but certain keyboard combos will still cause history lines to not get saved. It's maddening, and the solution here avoids it completely.
I wish bash history wasn't so broken. I suppose once upon a time it was ok to assume people only had one shell session at a time.
As far as I can tell, bash deals with multiple shell sessions exactly how I'd expect (and want) it to: an individual terminal doesn't have access to the live session from other terminals, but when you close it it appends it's session to the shared history file, so each session ends up a continuous block in that file.
On Mac OS X and iTerm2 I've noticed a thing where pasting in commands means they don't get added to the history and using up arrow or search to find them doesn't work. Anyone seen that? It's one of those things that annoys me every time it happens, but I'm always in the middle of a task when it occurs so I never get a chance to figure it out.
I know Linux BASH will ignore the command from history if that's the case, and I think that OSX is similar.
A colon-separated list of values controlling how commands are saved on the history list. If the list of values includes ignorespace, lines which begin with a space character are not
saved in the history list. A value of ignoredups causes lines matching the previous history entry to not be saved. A value of ignoreboth is shorthand for ignorespace and
ignoredups. A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved. Any value not in the above list
is ignored. If HISTCONTROL is unset, or does not include a valid value, all lines read by the shell parser are saved on the history list, subject to the value of HISTIGNORE. The
second and subsequent lines of a multi-line compound command are not tested, and are added to the history regardless of the value of HISTCONTROL.Some of the features of my solution are that it creates a sub folder in the user's home directory called ~/.bash_history and underneath this it will create sub folders for each month (YYYY-MM) and under each of those sub folders will reside a daily audit log file of all the bash command history (YYYY-MM-DD). The audit script logs both login and log outs, as well as each command executed in bash.
---
[0] https://github.com/onelittlehope/bash-prompt [1] https://github.com/rcaloras/bash-preexec
^ Here is my comment on a previous post for how I am logging all of my bash history in a logical manner (keeping track of terminals and timestamps, etc.) to an sqlite database.
Here are a few other projects:
https://github.com/thenewwazoo/bash-history-sqlite # simple shell script to store Bash history into SQLite3 database
https://gist.github.com/pklaus/1e381be8592426568df9 # simple Python script to store Bash history into Pandas dataframe
https://gist.github.com/pklaus/26925cfa1fc6b370e043 # simple Python script to store Bash history in SQLite3 database
https://github.com/fredsmith/history-db # store Bash history in PHP/MySQL web server
https://github.com/joshuacronemeyer/shellsink # store Bash/Zsh history in Python web server
I'm still not sure why it's helpful to have three and a half years of your bash shell logs.
I love this idea because you can easily analyse your commands then modify or automate your workflow where it makes sense. I've done this in the past to create a two letter aliases to frequently used bash commands.
[0] export PROMPT_COMMAND='if [ "$(id -u)" -ne 0 ]; then echo "$(date "+%Y-%m-%d.%H:%M:%S") $(hostname) $(tty) $(pwd) $(history 1)" >> ~/.logs/bash-history-$(date "+%Y-%m-%d").log.`hostname`; fi'
It stores commands in a sqlite database. For each command it also saves the time, the user and the host. It permits global search with some useful features (grep A/B/C switches, regexp, etc).
I wrote it to scratch an itch, thus the documentation isn't user friendly. If anyone is interested, shameless plug: https://github.com/andmarios/bashistdb
Storing the entire history would no doubt be helpful, but I feel similarly to what other people have said so far, which is that it may not be the safest thing to do? Probably depends on usecase.
Have cron job to signal bash session to flush history with command timestamps/dates to disk daily.
Handy for recalling commands/command switches you ran months ago.
The only way round this that I can see is to grep out the password - but then my password (or a substring of it) is now in plaintext in my .zshrc/.bashrc
I can't think of a solution to this. Does anyone have anything?
At regular intervals, especially when I have important commands to save I do a:
cp ~/.bash_history ~/bash_history_bak_2016-05-31.txt
cat ~/bash_history* | grep <important-command>
At some point, you have to clean things up and create a script for repetitive tasks, or write wrappers to tame complicated command line syntax. I think curl and groff are my favorite examples of commands that have too many options.
It's not obvious, but with this mechanism commands are only logged once they complete. So the log file does not include commands while they are still running. Run into that from time to time.
sudo chattr +a ~/.bash_history
:-[
Instead better would've been to use a custom method; internally the `require.resolve()` API can still be used to get the same semantics as `require()`
export HISTSIZE=50000