I don't want to say people who rely on screen readers don't matter and I think it can be fixed for them. However people who disable JS really shouldn't matter to you unless you are Facebook, Google, Apple, Amazon, etc. Even then some of those sites don't work without JS which could (Like google drive). It's simply not worth the resources to support a small group of people who have knocked out the foundation of a house and are expecting it to stand up just fine. "Screen readers don't play nice with js frameworks" (which is not true across the board) are not an argument for no-js, it's an argument for building better screen readers.
When it comes to the web you need to optimise for the 98% not the 2% (I'm being generous with that 2% number BTW) especially when that 2% CHOOSE to create this problem for themselves.