Static Website Generators Are the Next Big Thing (opens in new tab)

When I was heading up reliability at Netflix, we considered, and even began evaluating, turing the whole thing into one big static site. Each user had a custom listing, but generating 60+ million static sites is a very parallelizeable problem.

At the time, the recommendations only updated once a day, but an active user would have to dynamically load that content repeatedly, and at the same time, the recs were getting updated for users who hadn't visited that day. By switching to static, we could generate a new static site for you every time you watched something (which could change your recommendations), and increase reliability at the same time, so it would have been a much better customer experience. Unfortunately we couldn't get enough of the frontend engineers to buy into the idea to get it off the ground, and also they were already well along the path to having a data pipeline fast enough to update recs in real time.

I'm not entirely sure they're the next big thing. More likely, is that 15 years ago, it was people who were used to static sites began moving to dynamically generated ones as sites became more complex. They were the new thing then. Now we have a load of people who have grown up with dynamically generated sites and are suddenly discovering the benefits of static sites - thanks in part to the proliferation of tools that are easy to use.

It's the usual boomerang cycle of discovery and adoption.

Both types of sites have their benefits and it's a balancing act to use the right tool for the job. It's getting this right that comes with experience and an understanding of the current pitfalls of each. It's the rough edges that push people in the other direction and without the experience of the pitfalls of each, it's inevitable that people start predicting that one solves all the problems facing the other.

I fully expect the usual over reliance on the wrong type of tech for the sake of it being the current hotness and then an over correction in the other direction the moment we have a new generation of developers.

Every time static generation rears its head, I'm reminded of Yahoo!'s ... unique... take.

Back in 2006 when I worked for Yahoo!, and they had a CMS / template management system called Jake that statically generated templates for the PHP-based frontend servers to evaluate at request time. The idea was that you put as much of your logic as possible into the template generation layer, leaving the request-time logic to handle the stuff that changed request by request.

Now, that all sounds quite reasonable, but the two layers were written in different languages. The pre-template-generation logic was written as inline Perl (plus a little custom syntax, because why not), while the dynamic frontend logic was written in PHP. Perl was frequently used to generate chunks of PHP code to be executed by the frontend servers, and sometimes this PHP code wrote chunks of inline JavaScript. To say that debugging said JS was fun would be an understatement.

The reason I like static site generators so much is because it allows me to treat my website as a program that outputs a website. Take some posts in whichever format you prefer, write a program to parse them as a tree of HTML nodes, insert them into templates for HTML pages, Atom feeds, etc. It's all just plain text code and markup, no stateful database with a dynamic web application in front doing everything.

It's an interesting idea, and I see the appeal if the site is fairly basic, but I think there's one thing people are forgetting here.

You're outsourcing half your site to third parties, and basically letting them do whatever the hell they like with it. Disqus comments? Better hope the people behind that system don't decide to outlaw comments about the thing your website is about. Javascript embedded shop system? Good, so long as you don't need to modify the look very much and don't mind all your data being hosted in a different part of the world (like, the US for people in other regions).

And if they decide that all your data needs to be shared with the NSA or some other government organisation, then tough luck. If they hacked... well, tough luck again.

Without hosting such systems yourself, you're relying on a lot of third parties to be transparent, honest and respectful of your privacy (and that of your visitors). It's basically like a return to the days of free hosting and services like Bravenet.

I've been a fan of static websites for a long while now.

In addition to page load issues, they also more or less completely solve the Slashdot effect (aka the Reddit Hug Of Death, these days). A competently-configured Nginx server on a 512mb VPS, serving static-only content, will handle ridiculous amounts of traffic without flinching.

Ever since a front-page mention on BoingBoing took down my feature film BloodSpell's site immediately after release in 2007, avoiding server load on a site has been high-priority for anything I'm launching that is likely to have bursty traffic.

It's nice to see usable tools for managing larger sites with a static generator developing and becoming popular.

People are finally starting to realize that CMSes are horrible. They got us through a time period where the front-end developer is a scare resource, but now there's enough of them everywhere that we don't need WYSIWYG web content anymore. CMSes are eventually going to be relegated to the sole proprietor who doesn't want to learn HTML but still needs to maintain his / her own content-oriented website.

My company is handcuffed to a legacy custom CMS that's still using Rails 2.3.8. We have no need for it, as we have a front-end guy who would be perfectly comfortable using git, as he has me to ask whenever there's a problem. When you use the database to store content, you lose a great number of useful properties. I had to build a custom tool to search through the entire database to find encoding errors. And I had to keep re-adjusting it every time I found some hiding somewhere. It was annoying and painful. Content is code, not data, it needs to be managed like code.

The thing that always pulls me back to wordpress is the ease of creation and hosting. Drag an image into your text, and pop it's uploaded, hosted, and linked to. Write something in the interface and bam it's online and reliable. Every static site generator I've played with (And there have been TONS), solves 80% of the problem.

I'd love an app that gave me that ease of hosting and creation and generated a static site from there (hook it up to an S3 bucket or something). I'd pay for sure.

Currently I'm using Nginx w/ SSIs for lightly dynamic sites. Works well enough and is very very simple.

FYI: I've put together a showcase of the world's greatest static sites [1] (using the Jekyll machinery). Examples include: Bootstrap, Google Polymer, Facebook React, Open Data Handbook v2, PHP: The Right Way and many more. Cheers. [1]: http://planetjekyll.github.io/showcase

I love pelican as a generator, it's great

The one static generator I wish there was (unless there is one and I just haven't found it) is one that would take a tree of code files and display it kinda like github does, in a browsable file browser hierarchy with syntax highlighting when drilling down to individual files.

Kind of like a precompiled static file browser, there are several dynamic file browsers around but they all require server-side code (php usually) to do the directory listing and so on, but I think it should be possible to precompute all the directory display pages with symlinks to the individual files, and do the highlighting in those in JS/CSS

I might end up writing this at some point as it's definitely an itch I'd like scratching unless it does exist already and somebody kindly points me to it

The only problem I feel this really solves is caching, and for that specific problem, generating static pages may be a (work-aroundy) solution to consider, but in general I don't think static website generators are going to be the next big thing...

When you look at page speed, not much of the slow-down is from servers delivering pages, but browsers having to digest HTML/CSS/IMG/JS

I guess it all depends on what you are trying to achieve. Thanks as always for the article, interesting to see someone taking this with both hands.

I can't wait for ghost's API.

I've been following their repo for a while, hopefully by next year they'll be far along for it to be usable.

Ghost has an excellent editor and it would be awesome to have a static site built in any way you like that links straight to your blog posts via API calls.

They have a trello card that mentions it https://trello.com/c/QEdjRlgK/67-open-public-api-via-oauth-a...

and are working on it as we speak :) https://github.com/TryGhost/Ghost/issues/4004

I know there's a wordpress API as well but I find wordpress too bloated IMO.

I've only done static websites for a long while now. I created ThinCMS as a browser-based tool for building and publishing static web sites. It came about after I learned XSLT well enough to bend it to my will. I used it to build several public and private web sites, including two iterations of longwoodgardens.org (they've since moved to Drupal) and pittsburghtoday.org (where it is still used). XSLT ( and probably other static templating engines) is perfectly capable of generating complex nested navigation. The templates themselves are nested three layers so as to keep things DRY.

The PittsburghToday site is representative of the idea that a static web site is only static in the technical sense of the back-end content serving. The front-end is still dynamic since the data for the charts is being obtained from Google Docs and the Twitter feed from Twitter, etc.

I always felt like the odd man out, so I am glad to see strong interest in static web sites nowadays.

As the primary maintainer of Pelican, a Python-based static site generator, I've given several talks on the resurgence of static site generators and in which cases they are (and are not) a good fit.

My most recent talk was two weeks ago at PyCon Japan. Following are my slides for that talk, in case that's useful for anyone who wants to get a better understanding of SSG history and advantages/disadvantages: http://justinmayer.com/talks/

Also happy to answer any questions here, of course. (^_^)

I've been using static site generators a lot for client work over the last two years. I started off with Jekyll, but unless you are having under 100 pages, the build process gets painfully slow (trust me, I have micro-optimised).

I've since started using http://gohugo.io

It's lightning fast with 1000s of pages, and quite easy to pick up.

I would be equally interested if the article was titled "Why Static Website Generators Are Awesome". They are not the "next big thing", they've been around for years.

I am working on a project, http://blogful.me, that combines a hosted static blog generator with a solid admin backend including post syndication, an embedded analytics dashboard, authoring tools, and an API if you don't want to use the frontend.

There definitely appears to be a lot of interest in this space because you get the best of all worlds. Static site generators definitely seem like the way to go for all but actual web applications.

I'm still looking for a static site generator that's as easy to use as Wordpress (so with a UI, not markdown). Anyone know of something that fits the bill?

When I worked on a video search engine for UK football (soccer) clubs, we made the whole thing static.

We figured that inputs to the search was from a static range, i.e. these players, those games, that league, this type of incident (foul, goal, celebration, etc).

Then we pre-calculated all possible combinations and fired them through what we called a "cache cannon".

It was highly parallelizeable, simple to store on disk (we stored JavaScript files whose names were the form inputs), and worked extremely well.

Even for something like a search engine, unless you're doing full text search over a very wide corpus, you can look at pre-populating a cache and that cache actually being stored on your web servers and being directly addressable.

The design above, allowed that search engine to work over the weekend peak of 2 million users. That's where it shone... we just did not have to worry about the thundering herd with a pre-populated cache.

... because Javascript and external APIs can now do so much.

If I can do everything with JS on the visitors browser, why not host some shell HTML on S3 and never worry about a server? Maybe hit AWS Lambda if need be for one specific thing? Dunno. The age of the do everything server seems to be coming to a close.

FYI: Join the "movement" and start a static site user group. For example, I've started the 1st one in Europe, that is, Vienna.html [1]; others in the U.S. include Static Web Tech in San Francisco [2] and {Static is} The New Dynamic in New York City [3]. Cheers. [1]:http://viennahtml.github.io [2]:http://www.staticwebtech.com [3]:http://www.meetup.com/The-New-Dynamic

Ok, "static" here means no RDBMS-backed website. But you can still use statically generated JSON resources from a db once off. These resources can then be "filtered" and "combined" without the need for databases (not using the words JOIN or WHERE carefully).

Sounds like a great idea to overcome the need to obsess about connection multi-plexing.

That's the cool tool they used to test the performance of the site:

https://performance.sucuri.net/

You know what I've always wanted, but my searches have led me to believe that it (inexplicably) doesn't exist?

I want a simple site generator. I don't want markdown, I don't want a fancy templating engine. I want some simple templating system that takes in normal HTML and generates pages from simple templates I define. I want to shove in some arbitrary HTML and have it spit out a site using some base templates.

To the best of my knowledge, that doesn't exist. It would be perfect for someone like me who wants to keep a website updated, but doesn't always want to run PHP on the server for something as simple as that.

I implemented a shoddy version of it on my own, but it's far from ideal. I'm pretty astounded there's not a well thought out version of it out there, considering how useful it seems it would be.

Anyone remember Noah Grey's "Grey Matter"?

It was an inspiration to what became WordPress

https://en.wikipedia.org/wiki/Greymatter_(software)

Most of the static site generators have serious usability problems. They are simply not usable for many users using WordPress today.

I think that Static Website Generators, aren't by them self the next big thing. Mostly is about how the codebase influence on the content of the website, and use the the power of the existing versioning tools is indeed an advantage. Instead by them self are quite limited.

IMO, they are the next big thing if their are contextualised in Micro Service Structure, so that's why I build Monera - http://github.com/Ideabile/monera

What you think?

Haven't done any benchmarks myself, but I'd be keen to find out if a static site loaded up with JS design elements, a product store, comments and analytics code would load any quicker than a CMS with PHP caching, system caching and microcaching to handle bursts on a lightweight webserver such as Nginx.

The article is frustratingly biased in this regard. Static sites should just play to their strengths, otherwise you probably want a CMS that will act like a static site when it needs to.

I agree with the article and think static web sites is the way to go if the read/write ratio is high, and where the view is not unique to the user.

However, quote > "The static version is more than six times as fast on average!"

This must be an engineering problem, especially on easily cached content. Serving static web sites Does require computation. But the current tools are very well made and optimized for it, witch is not the case with most CMS systems.

I am working on Open Source Static Website Generator/Manager for College Professors to create/manage course webpages. Its actually a Desktop App which runs on windows, linux & Mac and can directly push the webpages created to Github-Pages, SFTP Server. Do check out the project at https://github.com/navya/Kalam .

"With the maturation of browsers, many features that used to require dynamic code running on a server can be moved entirely to the client. Want comments on your website? Add Disqus, Isso or Facebook comments. Want social integration? Add Twitter or Facebook’s JavaScript widget to your website. Want real-time data updating live on your website? Add a squirt of Firebase. Want search? Add Swiftype. Want to add live chat support? Olark is there. Heck, you can even add an entire store to a static website with Snipcart.

The list goes on and on, as a whole ecosystem of purely browser-based add-ons to websites is emerging. Apart from that, modern web apps built with Ember.js, AngularJS or React are often deployed entirely as static websites and served directly from a CDN with a pure API back end that’s shared between the website’s UI and the mobile client."

--

I'm not sure I understand. It doesn't seem to me that a fully single-page, AJAX web site is truly "static". If much of the utility and content must be paged in via client-side JS calls, that too will contribute to load time and the same problems that are attributed to dynamic document generation. It may be all asynchronous and fancy, but from a UX point of view, the content isn't there until the data's retrieved. How's this any different than arguing for a grid of IFRAMEs?

After all, if your page is a minimal HTML DOM harness for a bunch of JS, can one really be said to have "loaded" the page simply in virtue of having loaded the stub HTML?

Or is this argument based mainly on either the implicit premises that (1) not all the functionality and components are used at once? or (2) that much of any given site's functionality can be off-loaded to third-party components (e.g. Disqus) which can all be loaded in parallel from different network sources?

I host my company's web site on AWS Cloudfront using my homemade static website publisher which minifies my JavaScript's ,css and html and gzips them before pushing to Cloudfront . The pages load fast but the downside is making changes to page like fixing typos is not as simple but that probably an issue with my generatorvas well as Cloudfront cache configuration. I don't use Jekyll or Hyde or other static publishers because I wanted to write one in clojure and I figured I could write one in clojure faster than what it would take me to learn jekyll etc. You can check out and (critique) my website by visiting https://www.videocloudmanager.com. I run business video hosting service.

One static site CMS that I know of is: http://www.webhook.com/

It's open source. Design and content is edited collaboratively and it deploys a static site.

Are there any other CMS systems designed to deploy static sites?

I think you guys are thinking small when thinking about a static site. Change the content dynamically with Javascript only is still considered a static website, this doesn't make the frontend static at all, it's very dynamic! But no more need for PHP, Perl, etc.

Best Regards

I do something I call Dyna-Static.

I can run a static page off an Apache (or any wwwserver) instance. Just chuck files in /var/www/ where you want them.

Now where it gets interesting is I use Node-red to generate the pages; content and all. I want headers? It's a variable. I want ads? It's another variable Google provides. I want chat? Easy ( I can do it with nodered or 3rd party). I can bridge that webchat with my or someone else's IRC room.

Now, I can script it so the pages are updated from nod-red server to webserver. They can easily sit on the same box, as node-red takes few resources.

And the kicker is that I could get that done in an hour or so. Check out Node-Red . It really is that amazing.

I think static site generation will simply be a feature of a build tool like Webpack or Gulp, maybe as a plugin, and either way there will be an api for developers. Or it will simply be part of a larger build chain / automation system somehow.

Static site generator doesn't mean there's no backend. A website is called 'dynamic' when its operation depends on communicating with a server. The JS logic delivered to the client can range from animation to async http requests.

The distinction between "static site generator" and Webpack / Gulp is very gray. It all depends on what you want to do with your client-side JS logic.

Sadly, I think it will never work for end users. I myself also don't update my blog very often, partly because writing new posts is still too cumbersome (with Jekyll). I am a developer, but I don't use Markdown and Liquid (or what was the name) on a daily basis, so I still have to look things up when writing.

Now I tend to think that the best way would be to use a random CMS (like Wordpress) and mirror it's output as a static site. That way, the dynamic part the end user uses for creating content could be behind some secure login-wall, and the public site would just be static.

As for comments, I guess they just won't be supported.

I think the simplest static site generator is a command that visits all of your routes and saves the response html into a `build/` folder.

That way you can use whichever framework/stack/templating/database you're already familiar and productive with, and in the end you're just deploying a static build folder from localhost.

I started doing this when it came down to hacking Jekyll to implement something that's trivial to do in a microframework, so I went with microframework + page caching. I do the build and deploy with a gulp task that I'd like to generalize into a gulp module.

To an extent I agree I think we are just getting better at recognizing the security and performance optimizations that are staring us in the face when content doesn't change rather than having complex websites completely static as static site gens do. I think we will see an increase in static elements mixed with dynamic elements in a more comprehensive way.

------

I host 10s of more or less static sites (Contact forms being the most dynamic elements) which are generated on the spot from one PHP (laravel) installation.

Anyone know the best way to cache the html/css statically to serve?

Partition template parameters into request-dependent and request-independent sets. Memoization with persistence of the request-independent template instantiation. "static" then is the special case of no request-dependent template parameters.

I think I'll write an article "pure functions with memoization are the next big thing!" Except they are not "new" because they've been around for decades. The only difference is the Web 2.0 uberkids haven't "discovered" the concepts yet.

... Next Big Thing For Minimally Dynamic Sites

If your content doesn't change frequently and/or the costs of regenerating the static content is minimized for you, great.

At what point do we see static sites take a fair share of the top-X-trafficked sites? Top 100? 1000? 1,000,000?

This is probably great for a small corp's info site... but then the client asks for a contact form or members/admin secured area, and there we go down the rabbit hole again.

Think of web app architectures like data structures. Most websites should be optimized for reading, not writing. Static site generators are very optimized for reading, not writing. Wordpress sites, on the other hand, are more optimized for writing. It makes sense that most websites should choose static site generators - it's just a better "data structure" for the needs.

My favorite kind of static website generator is lazy, just in time, and supports TTL based regeneration on a per page level.

I'd love to use static site generators for my client work but they usually ask for features like e-commerce that completely rule out it's use.

Usability is also an issue. Wordpress is a far more friendly environment for them to make changes or create a new post than creating a text file with specific formatting and running a script.

I'm currently (slowly) working on a static blog that utilizes only HTML, JS, and CSS. I liked jekyll, but wanted something with no backend technology requirement:

https://github.com/milge/lilblog

Does this mean that developers and admins will get paid less, because their jobs just got a lot simpler?

I use MiddleMan as a static site generator for a page hosted on my raspberry pi: http://pi.tafkas.net

As I am more of a python guy I wonder if there is a similar (as in not primarily for blogging) generator for python?

I have been doing this since 1998 with perl and cron for my own systems and I can say it works great. Combine that with a ram disk and I can handle the load with a couple small machines that would typically require a big farm using dynamic content for everything.

I'm currently building my own static site CMS. Basically. Word press like interface (at some point). Spits out a static site. My blog http://adnanissadeen.com runs on it. I'm building the CMS over at https://github.com/spartakode/static-cms . Been inactive for a while because I'm just a horrible procrastinator. I'll be finishing up (read: making it usable by public) end of this year hopefully. Will try and offer a hosted service a little later.

Considering a lot of content seems to be increasingly generated and handled via Javascript, what are some of the downsides to statically generating a website or webapp?

I remember this custom CMS written in Perl mason that we used at JupiterMedia ( formerly internet.com ) back in 2004. It worked well but had some complexity.

How many more articles about static sites being the "Next Big Thing" before they just become "The Big Thing"?

A bit of web history, mention some static site generators, end by saying their tool is the babel fish of static site generation.

Movabletype anyone? Its older than wordpress.

just an ad for Netifly

I'd be happy with a wordpress plugin that rendered static pages. 95% of client sites I've ever done have little to no reason to be dynamic. I know there are valid reasons for a dynamic site however IMHO those are rare.

Next to geosites.com, I suppose.