What you want is a dedicated repository format. Libraries can still be hosted by whoever maintains them on their own repository (which can be their own piece of software as long as it follows the standards), or in the community central repository. But either way, if you depend on those libraries and want to lower your risks, it's trivial to set up a local mirror and make sure all your third party dependencies come in via this mirror. That way if their repository goes down temporarily or permanently it's no problem, and you ensure your builds remain reproducible.
The most infuriating part is, the software to do this already exists. If you want to start a new language, great. But please, use maven; otherwise you are doomed to re-invent it, poorly.
(context: I make https://circleci.com - a continuous integration company for web apps, often Rails. We occasionally get support requests that allow/ask us to look at Gemfiles, so I've seen an above average number of Gemfiles. However, I more often see the stdout of the `bundle install` command, which shows GitHub being accessed).
gem "rails", :github => "rails/rails", :tag => 'v3.2.11'
You can also specify multiple gem sources (http://gembundler.com/v1.2/gemfile.html), but usually only rubygems.org is used unless you need a private geù repository.