Unless your site is made of 4 files/modules, you simply must bundle. ESM will download the files in series, as each dependency is discovered. Then compression doesn’t work as well as it will be per-file. Then of course you can’t tree-shake dependencies from npm, so good luck downloading the whole of lodash on your client.
In short you lose the advantages of only download what you need pretty quickly.