By default, Rails' page caching persists to disk. An immense advantage of this solution is that you can use your web server to serve these pages directly, without ever hitting the Rails stack.
You're saving the page generation time, but you're still hitting the Rails stack + Redis for every single page request, both of which are entirely unnecessary.
http://www.holovaty.com/writing/django-two-phased-rendering/
and there's a permissive license (BSD, I think) implementation:
http://codysoyland.com/projects/django-phased/documentation/...
You can still get away with caching large parts of your pages though.
Check out the code that instantiates it at https://github.com/rails/rails/blob/master/railties/lib/rail...
require 'action_controller/integration'
app = ActionController::Integration::Session.new
module CurrentUserHack
def current_user
@user ||= User.look_up_my_user_here
end
end
ApplicationController.send :include, CurrentUserHack
script/console production < worker/cache_page.rb
?
Our slow endpoint was on a back-end administrative page only; faking the session data in curl would have been annoying. Also, it was exceeding the timeout limits of our production server
I think you have a bigger problem here.
You're right this is very hacky, it makes me itch, but I'm not sure I have better solution. Why not just use wget to load and cache the page (passing in a unique parameter that you use to expire the cache and skip the filters)?
What you should be doing is caching that big time consuming job (I guarantee you aren't timing out someone's browser with template code) at the model level, and then generating pages off of that cached result. Feel free to further cache the HTML too, but the bulk of the win is from decoupling long running jobs from your display code.
If you want to ensure that the user sees the absolute latest, use an AJAX call to pull the more recent result after serving the previously cached result.
(though in this case, I think it's safe to assume the second hard thing is solved - "kludge" seem to fit just fine)
If your page takes long than a second or two to load when uncached you should really be moving towards a batch processing model.
https://github.com/stucchio/Stiletto
It's nearly always better for nginx to serve up pre-gzipped content than for nginx to ask django/rails to ask memcached for the same content. It reduces your CPU load as well, so you need fewer servers to scale up.
On EC2, make absolutely sure you are storing the pre-rendered files in ephemeral storage (/mnt, not /var) - instance storage is slow.
We eventually moved to decoupled services, which were able to do much smarter caching.
saves you a few keystrokes...