Many times the 100KB of JavaScript is faster to load when minified and combined with other site code and served compressed over a single HTTP request or streamed via HTTP/2. It's almost always faster to use an existing connection than to start a new one.
Also there isn't one canonical version of jQuery. There's dozens of potential versions available[1]. So it's not immediately clear that a user will have the version a site depends on.