>hackers are probably still motivated enough to try it.

The amount of actual exploit crafting that is needed to actually do something meaningful with a hack is pretty much not worth doing for any financial reason. The only time this happens now is when state funded actors or prominent groups with lots of manpower really want to take down an individual person.

Linux on its own isn't even 0.1% visitors normally. We're talking multiple orders of magnitude less for disabled mitigations. And on top of all that, it's possible that exploiting on that machine is going to be harder due to custom software with uncommon memory layout - i.e. it's probably not a stock Ubuntu. And finally, for accessing data outside of the page, you really want to have some specific target, so they'd have to guess that too.

> If just visiting a webpage with some JS will let them do ACE on even 0.1% of visitors

Spectre is not an arbitrary code execution kind of attack.