I've fallen out of using it, but for a while I was using dtach to do similar without the virtual terminal multiplexing. Much much more direct.
I'd just run a vim session. If I needed terminals, they were in my vim! Even wrote a short shell-script to automate creating or re-attaching to a project specific vim session. https://github.com/jauntywunderkind/dtachment
Haven't looked into it, but I'm love a deeper nvim + atuin (shell history) integration.