The beginning of a standard for browser automation (opens in new tab)

(Disclaimer: I'm chair of the W3C working group working on the automation spec, and co-chair of the W3C testing effort.)

There are several efforts converging at one point here.

First, W3C is writing a bunch of specs. HTML, CSS, DOM and so on. To ensure that implementations of said specs are interoperable, we need good test suites. I guesstimate that we will need at least a million test cases to properly test the web platform. These tests need to be able to run automatically in any browser.

Testing things that can be evaulated with JavaScript is easy, and we have a decent tool for that (testharness.js). Testing things that require a visual inspection is slightly more difficult, but is essentially a solved problem (refests).

Then we get to the difficult parts. Testing things that require some sort of user interaction. How can you, automatically, cross-browser, test the assertion "when hovering your pointing device above the blue box below, it should turn green"? (That's a basic test for the :hover psuedo-class i CSS.)

It turns out - someone already created a tool that does just this. (That "someone" includes the author of the blog post above. :) The WebDriver API allows one to instruct a browser to click, type, navigate around, query the DOM and take screenshots. Its initial audience was web application authors, but you might as well turn it upside down and use it to test web browsers. It works in most major browsers, and is now officially supported by Google, Mozilla and Opera.

(I initiated the project at Opera back in 2008 because we needed better test automation internally. I no longer work for them.)

But the W3C can't just tell its member organizations that they must use this random open source tool to run the official W3C test suites. "Just reverse engineer this tool" won't cut it. We need a spec to build a level playing field here, and to get patent worries out of the way. (The W3C patent policy is great.)

The second effort here is the Selenium/WebDriver open source project. It's a wonderful piece of software I've personally used to test several different web applications and browsers. This project has a few challenges:

1) Some inconsistency between implementations

2) No official support from Microsoft and Apple

Turning the de facto standard for browser automation into a de jure standard will hopefully help the open source project in both respects.

A standard is a good thing, because people build tools around standards, that will inter-operate.