Your single-core performance is the most important thing for a Minecraft server, especially for Forge mods.
For vanilla, I’d try Paper (a high performance spigot fork) and see if you still have problems. If you’re lagging with 6 people while running Paper, you simply need a better CPU.
Tangentially related, I had to move a web app off of DigitalOcean because the CPU performance was terrible compared to what you get for $80/mo from a dedicated server from OVH. I know Linode and DigitalOcean compete. Get a OVH dedicated server, try the CPU passmark bench or whatever and compare. AWS + Linode + DigitalOcean are all VPS and pretty slow.
