The problem is not the CDN, it's the last mile. You seriously overestimate public Wi-Fi and shit cellular networks; there are times where a simple HTTP request returning dozens of bytes of text will fail. In these moments every byte matters and personally if I'm going to make my user download 5kb of data I will make sure it's content they care about, not a JS framework.

Let's take the example of London Underground's Wi-Fi. It's only available at platforms, not while the train is travelling between stations. When the train arrives into the station it will stay there around 10 seconds. 5 seconds of those are wasted by the phone finding the network and authenticating to it, which means you've already got only 5 seconds before the train goes away and the connection drops off. During those 5 seconds you have lots of apps in the background (push, e-mail sync, background app refresh, etc) that are all fighting for around 1Mbps of bandwidth.

There's also the issue of failing gracefully. If CSS or image loading fails the browser will still display the bare HTML and the user will still have their content in a consumable format. If JS fails to load in an SPA the page is unusable and often shows just a blank screen. If SPA-initiated requests fail it's up to the developer to handle the exception and offer the user a way to retry instead of just giving up and waiting for a page reload.