Incredibly frustrating. I have trouble when I have to use other PCs or devices. I can no longer use the "arrow keys" and get momentarily confused when hjkl are showing up in the textarea instead of giving me movement. And I can't tell you how many times I hit escape in LibreOffice/Calc/etc.
This is one reason why I don't use online editors like Google Docs. They don't have great vim keybindings. Even extensions like Vrome and Vimperator fall just a little short. And for apps that do have a subset of the bindings--they are almost worse than having no bindings at all; evince has hjkl, but no gg/G, etc. It drives me mad.
I've configured AwesomeWM to be vim-like, but how I wish there was window toolkit-level support for vim bindings! In fact, I wish I could run everything from vim.