The options that I see for using $EDITOR are: (1) simply ignore the line option for non-vim editors, (2) add implementation for specific editors, each with their own "jump to line" argument if it exists, or (3) add an additional command line argument that passes args/flags on to the editor. Option (3) seems best to me but I do worry about the number of supported flags - it is quite a lot already! More seriously, this option could be a vector for malicious code execution.
Thanks again, really appreciate the feedback! I'd love to hear your thoughts.
[EDIT: add comment on malicious code execution]