I’m a die-hard vim user and a fish developer but (gasp!) I don’t use vi mode in fish or any other shell. Alt-V will open $EDITOR with the command line contents preloaded into the buffer and update the prompt on exit, I find that to be good enough without the frustration of trying to convert the prompt into a text editor when it’s not.