Public S3 buckets are not a good choice for that because of the "anyone could bankrupt you" reason above. That is generally NOT the use case for S3, and it's the reason why private is the default and there are alerts for public buckets. For public access, within AWS, you'd want to put CloudFront in front of the bucket and only allow external users to access CloudFront. However, there's a better option...

Outside of AWS, Cloudflare has a service called R2 which is just like S3, except you DON'T pay egress! It's the same thing, but without the "anyone could bankrupt you" aspect. You pay for the storage but you don't pay per download. I highly recommend this for hosting open source binaries. You can still keep a copy in a private S3 bucket for safe keeping if you don't trust Cloudflare long-term.