Then the mirror became too slow and couldn't handle the amount of package data and was shut down: https://github.com/RubenKelevra/pacman.store
It worked rather well and even automatically mirrored the packages on my LAN. Maybe it'll be back some day.
I did try doing something fun with network-interface scripts and avahi to make this more dynamic - the idea was my desktop and laptop both ran squid proxies, and then dynamically set peer proxies if they detected each other on the network. It didn't work great given the overhead.
It really feels like there should be a better way to do this in general, and without having to break TLS directly for things which are not really "secret".
Therefore to use torrents requires one of two choices, neither of which are really viable when you think about it. Either:
* Require everyone who uses the distribution to torrent releases of the full set of packages for the distribution. (And remember that packages regularly get updated even in LTS releases, due to backporting security fixes etc.) It means everyone using the distro really needs to be running dozens or hundreds of torrents.
* Everyone torrents only the packages they need, so access to packages becomes up to popular vote instead of actual importance.
Torrents are fine for isos but intractible for ongoing package management.
I wish more systems had this kind of feature. I have a fat fiber connection, I'd be thrilled to pop up an unofficial mirror for something like a Linux distro. I've tried mirroring Linux ISO torrents, but it seems almost nobody ever downloads from a torrent, so I end up never actually uploading any of these images.
I'm sure this will happen once the Linux community standardizes on a package format /s.
AWS is just being ignorant. If I were in charge of Fedora infrastructure I'd block them and send them instructions on how to setup a mirror.
I suspect there are only a handful of vendors with popular enough AMIs or container images that could account for this, though.
A user with "five million additional systems" on AWS?
XML schemas have had a similar history, tanking w3c.org servers.
If someone blocks you it becomes an incident, a post mortem and you learn your lesson.
If someone blogs about it, or e-mails you, it gets added to a todo list and might get fixed in a few weeks by a disinterested intern.
Isn't that sensible thing to do?
You now get the stock epel mirror listing, not a custom package Amazon rewrote, and they'd need to add a custom mirror that provides an override for that package to make it work that way.
So I would _guess_ one or more large companies migrated a bunch of things onto cloud hosted EL7-based system after EOL meant god help you finding new hardware support.
(I would also guess either EPEL doesn't have a mirror on Amazon or it should have auto-picked it, or the auto-picker is bad, and either way, Amazon will probably go "oh our bad", since I believe they host local mirrors of things anyway...)
Just stop support brand's and corporation's if you don't get paid?
> The massive uptick in Fedora/EPEL activity puts additional pressure on Fedora web proxies for mirror data and then the mirrors themselves that tend to be volunteer run. Much of this new traffic is coming from the Amazon/AWS cloud.
But every tool has its own config to set to use the Artifactory. One setting for the OS package manager (which is different for different Linux distributions), another for PyPI, another for NPM (or Yarn or whatever), another for Maven/Gradle, something else for Go, then I need to download this Postgres extension and build it from source - the list goes on. So almost inevitably something gets missed and one ends up not being as good a citizen as one ought, and then one day some random Jenkins job is failing because some external dependency could not be downloaded.
I wish there was an easier way. Like some standard mechanism for saying “for this URL use this proxy”.
I guess one could just use a proxy server (http_proxy environment variable) but with most things on HTTPS it needs to MITM the TLS which then means you need that certificate installed in the build process - which is another one of those “everything can do it but everything does it differently” problems. And in any event, MITM is a bad smell.
Tools like Artifactory are a hack built upon a hack built upon a hack; redirecting certain HTTP requests to proxies would just be piling even more crap on top.
Content-addressing is a much cleaner option: identify files by their hash, rather than as the result of a HTTP request to some particular URL (many tools will already use these hashes to verify the result anyway; that's what "lock files" are for!). Content-addressed data is agnostic about how its retrieved, which makes caching trivial. There's no need to care about the data format, whether it's RPM, Deb, a source tarball, a patch, a Python "egg", or whatever.
For example, I lead a transition to Nix at a previous employer. We had a bunch of projects with various build processes (Maven, SBT, Gradle, PyPI, NPM). After wrapping these in Nix, the whole lot could be cached on S3 by simply copying files around (see https://nix.dev/manual/nix/2.22/store/types/s3-binary-cache-... )
Pulp is perfect for this but the demands on my time make it hard to see around the corner.
The problem is most clients don’t do GET https://, because in your old-school corporate web proxy use case, the proxy server is remote, and sending HTTPS requests to it over HTTP eliminates the security of HTTPS.
If only there was some standard environment variable like artifact_proxy, which had to be a localhost http URI, and which tools would understand as meaning “send HTTP GET to this proxy, even for https://, delegating all the TLS stuff to it, but only if you are trying to download a build artifact, not for any runtime use”
The hard part wouldn’t be implementing this idea (the local proxy server and the environment variable), the hard part would be getting all the different tool developers to agree to support it
An astounding milestone for the English language.
Imagine what this sentence possibly could have meant in 1990.
I thought this title referenced cars driving in a south american rain forest.
A decade ago, each company used to have its own cache of all public packages for CI/CD, but it looks like that bo one cares anymore.
Even worse, many of those processes are running an 'apt-get update' (or equivalent), so there's no way to know what packages they'll get each time!
