No, the hashing either when generating or checking is very fast like you said. Hashing itself isn't the culprit, but the battle between browsers and those fingerprinting users.
Originally the point of using a shared CDN like this was that if others used it too the file would already be cached on the user's computer and make it even faster. But, this feature was used for fingerprinting users by checking which files from other websites were cached and browsers have isolated the caches in response which makes it impossible to get the speed benefits from before.
So if you're not getting that speed benefit, and only really getting a tiny bandwidth reduction, the risks of serving the file from a 3rd party (which could be mitigated by the hashes) aren't worth it compared to simply vendoring the file and serving it yourself.
So it's not that hashing prevents caching or lowers response times, but that the risk it is mitigating isn't worth the effort. Just 'err on serving the file yourself.