I’m a Colemak layout user. I don’t bother remapping anything for Vim because I want it to be consistent with other apps with vi keys.
I have done that briefly years ago but I find it a dead end.
After a while you just get used to the cursor keys being in a different place and if you’re touch typing it doesn’t mater much.
Also what you loose to cursor key placement you gain back on other navigation.
So in short just support qwerty and it’ll be fine.