For the past week, I've been working on writing a little static site generator and putting it on S3 and I thought I was pretty damn clever for finally getting around to doing that... except now it turns out I'm clueless yet again. I'm looking at Cloudfront now, but I'm still not sure if has all the features that I was expected out of S3 alone (someone already mentioned Route 53 integration).
S3 is for storing files, Cloudfront is for serving cached versions of them really quickly out of edge locations (i.e. the closest CDN datacentre to the user that requests it).
In your usecase, your best bet is to use them in combination. Set up a Cloudfront distribution to point to your S3 bucket, then setup DNS to point to your Cloudfront distribution.
One big point to note is that you need to either:
* Configure Cloudfront to expire objects after a TTL (time-to-live) that is reasonable to you (e.g. 1 hour, 1 day etc). You can do this from the Cloudfront 'new distribution' wizard.
OR
* Let Cloudfront respect HTTP headers and then make S3 (or whatever your custom origin is) set cache-control headers that make sense for how often you update your site. Not sure if/how you can do this with S3, with a custom origin its your app so you can set whatever http headers you like.
To be clear: if you don't do this, I'm pretty sure cloudfront caches things forever, or at least a very long time.
Personally, I think triggering cache invalidations should only be for emergencies (e.g. someone has uploaded questionable content to serve to other users and it's cached in at and edge). Rather than screwing around with that, save yourself some headaches: pick a sensible TTL and wait a little longer to have things up to date at your edges.
Note that by using Cloudfront in this manner, you get the performance benefit of serving static files. If performance at the expense of convenience was your main reason for going with static site generation, you might want to rethink that decision (there are other perfectly good reasons for wanting to use a static site generator, security being my favourite).
Do feel free to ping me over email if you have any questions on the above.
Great comment. I didn't get this part though. What would be a better alternative?
Performance is one of the main reasons I considered this (uptime is another). Let's just say I've had the same shared hosting for over 5 years and the speed/uptime have been a disappointment for a long time. When I was working on a site and noticed a 500kb background image was taking 2 seconds to load and around the same time I saw that the spotify homepage was streaming a fullscreen video instantly, that was kind of the last straw.
So I thought the idea was that skipping dynamic generation, using distribution (well, I think my assumption was S3 did have edge locations), and just having a better host was a big win.
In that case, you get the benefit of both worlds.
On paper the Rackspace one looks like a great performance/price alternative.
Depending on the nature of the content, Cloudfront is not usable for video, particularly the kind where people seek around a bunch, like instructional videos and tutorials. Also, people with slow connections expect the video to buffer while paused. This doesn't happen if you serve the videos from cloudfront.
If that is the only problem, flipping one bit in every response seems like a really simple solution. Why hasn't Cloudfront fixed it yet, do they know about this?
/co-founder at Advection.NET
"Rackspace uses 213 of Akamai's edge locations, selected especially for our customers' typical usage patterns, and designed to cover all major areas of the globe." http://www.rackspace.com/cloud/files/
Akamai has approximately 5 gazillion edge locations, so yes, it is a cut down version. It is still a lot more POPs than just about every other CDN, though this doesn't necessarily translate into performance.
1) POST (and so are PUT, DELETE, OPTIONS and CONNECT)
are not yet supported on CloudFront.
2) HTTPS/SSL for your own domain is not yet supported
on CloudFront.You're not making the web faster, you're shilling for your employer by comparing their apples to a competitor's oranges and proclaiming "our competitor's oranges make bad apple sauce!"
Failure to mention CloudFront is disingenuous.
To be honest, your post feels like spam for MaxCDN.
Pro-tip: If you're using Rails, just create a distribution with your app as the origin server and in production.rb, set your asset host to your distributions host. You get the asset cache without having to do the precompile step. Tastes great with Heroku.
I used to do the same until cloudfront rolled out custom origins.
> Since Heroku will precompile by default, what do you do to disable it.
I misspoke in my comment, what I meant was you can skip the synch with S3 step. I've never actually bothered to stop heroku precompiling assets (though I may as well). This question on SO looks promising:
http://stackoverflow.com/questions/8953360/preventing-heroku...
If you do that though, as you mentioned, you will definitely need to flip serve_static_assets on.
Taking Vine for example, I picked a random vine from twitter (https://vine.co/v/bE3YI365gxd) and a popular vine from twitter (https://vine.co/v/bEFFxdwjK9x). The popular video and thumbnails are served from a CDN, where the new one with no traffic is served directly from S3.
YouTube did the same thing in it's early days. CDNs are not required for all traffic, and blindly recommending them is a bad prescient. They're really great for content that is frequently accessed, but their value greatly decreases on long-tail content.
While I can't compare CloudFront to other CDN's, I do know it works well for my clients that have been using it (and certainly better than serving directly from S3).
$ curl -I http://phaven-prod.posthaven.netdna-cdn.com/uploads%2F2013-05-17%2F20%2F3128%2FErQE0vKlNMIeNvaxbneY75nWy
HTTP/1.1 403 Forbidden
Date: Sat, 18 May 2013 20:31:08 GMT
Content-Type: application/xml
Connection: keep-alive
x-amz-request-id: 41706FB9149898AF
x-amz-id-2: d5F1JMIBLaQzNG5A
Boo. :)PostHaven cut off the URI:
curl -I http://phaven-prod.posthaven.netdna-cdn.com/uploads%2F2013-0... HTTP/1.1 200 OK Date: Sat, 18 May 2013 20:34:13 GMT Content-Type: binary/octet-stream Content-Length: 52958 Connection: keep-alive x-amz-id-2: NO6o51/19JsQJN9YHc+T/sraZSGNT+f3R+1GWl2QL3aD4SubqazjbMURb4VYaZyS x-amz-request-id: E640348D2D6EDA7B Last-Modified: Sat, 18 May 2013 00:47:10 GMT ETag: "f95534e9752b560f4acdda20228f90ba" Server: NetDNA-cache/2.2 X-Cache: HIT Accept-Ranges: bytes
I have my static assets on a sub-domain, so I just set cloudflare to cache everything on that subdomain (and left it off on everything else).
I switched to CliudFront and, of course, downloads improved dramatically. I had to go with CF because we needed signed downloads. Would be nice to have alternatives but I'm happy with CF.