By that I mean, evaluating each new surface to verify that it provides value to all parties — the end users, the developers, and to Google (in roughly that order, but all are essential) — and that this value scales as the popularity of an API or platform increases.
Over past years, we've launched a handful of APIs that perhaps met one or two of those criteria, but not all, and learned through practice that we're unlikely to achieve a sustainable ecosystem without first identifying a "virtuous cycle" such that success for each party is complementary to the others.
Nothing pains me more than seeing an API surface go away, so we've been working at getting consistently better at making sure each new launch has strong potential for longevity. Launching a new surface just because it's interesting (or to find out if it's even possible) is obviously tempting, and can be fun and even rewarding in the short-term, but each new API is also a promise being made to developers, and that promise is a commitment not to be taken lightly.
Now sometimes, the underlying products go away for good reasons, and we need to turn down the corresponding API surface (e.g., when the Buzz API was shut off when Buzz was sunsetted in favor of Google+), sometimes we grossly underestimated the potential for abuse (e.g., the old translate API, the first version of which didn't have quotas or require any real form of registration), sometimes we were able to declare victory and move on (e.g., Gears -> HTML5), and sometimes API functionality was never officially available to begin with, and as such when the underlying product changes and it's impossible to keep the unofficial surfaces intact (e.g., Reader), but in every case the decision to turn one of these down is a difficult one. Whenever possible we've gone through great lengths to keep the surfaces stable and available as long as practical to give developers time to find new solutions (6 months to a full 3 years is the norm).
Going forward, I think the additional up-front discipline is a win for all of us. It's a bit more work on our side to play out all of the possible scenarios (what if no one uses it? what if everyone uses it? what if our fiercest competitors start using it?), but in doing so I have even more confidence that when we launch new platforms we're going to be able to stand by them for the years ahead.
Personally I'd still bet (heavily) on third-party services, both from big established companies like Google, Amazon, Microsoft, etc., and also on smaller and upcoming startups. Though I'd certainly ask myself the question: is there the virtuous cycle in the API that I depend on? Can the service provider reasonably sustain it for the long-haul? Does the API provider benefit when my usage increases? Do I benefit if my own user numbers grow? Etc. Contingency plans are still a must — betting your entire company on getting something free from someone else forever is probably a bad bet — but I'm bullish on the future of web services in general, and plan to personally stay in this space for a long time to come.
Some day I should give a talk on lessons learned the hard way...
https://plus.google.com/112678702228711889851/posts/eVeouesv...
However high the odds are that the founders of a given startup will leave their company within five years, the odds that the people who built your in-house widget will leave your company within five years may well be higher. However high the odds that a given API will evolve away from your use case in five years, the odds that your use case will be obsolete in even less time may well be higher.
The underlying truth is that all APIs everywhere, in-house and out, have bugs and missing features and finite half-lives, unless you control them and continuously expend resources maintaining them. As your project grows older and more stable you may eventually discover that its expected half-life is longer than that of your provider APIs, and that's when it's time to start moving stuff in-house. This is when you realize, if you haven't already, that open-source software is your friend.
I suppose you're right that any given code is unlikely to outlast its dependencies. But the corrollary to that is that successful code is, by definition, almost always going to outlast its dependencies.
Dependencies are hugely expensive. And it makes me happy to see someone try to explain that to the modern world of import-happy coders. Reuse isn't free. Solving simple problems with external dependencies can often be more work than it's worth.
Even with the worst bugs I can use and reverse engineer the oldest API if it runs on my own hardware. I can turn on and access my Amiga 500, or my Apple II "APIs". There are uncountable companies running legacy software, just sometimes they need to reboot the machines or "play" a known trick to continue working.
Yes, this means that the word API no longer connotes "true platformness". That's a branding problem for the "true platforms", but I'm sure Microsoft and Oracle can handle it. They have resources.
I'm working with pretty much every type of API through my startup Zapier.com and some of these API's I wouldn't wish upon anyone except maybe the deeply masochistic coder. They suck. I suppose that's good for us. However, we aggregate data from many dozens of APIs for the end user, so losing an unpopular service's API here or there doesn't hurt us in the least.
While I've got an API pedestal and a crowd of developers: please no SOAP or XML. And do NOT go off the OAuth spec. Bad documentation is the gravest of sins, which can mean lack of examples or, worse, out of date info/examples.
Good APIs: Github, Wufoo, Strip. All of them are more or less self documenting, use JSON, and RESTful. The docs are awesome. A joy. In and out.
Bad (or just annoying) APIs: Salesforce, 37signals, Google Contacts and co, Docusign... While they work, you'll struggle every inch of the way to make the simplest things happen. XML (or some derivative) and SOAP. No thanks.
There are also many visual XML mappers (mapforce, biztalk mapper, stylus studio, IBM and Oracle integration products)... but truly, these are enterprise-expensive tools which I think are only used in the enterprise (with its more complex data and, some say, unnecessary corporate inflexibility).
While it's true that disruptive tools tend to start at the bottom (e.g. with startups) and bubble up, the complex data needs of the enterprise have to be addressed in some way, either by sticking with conventional tools, or the new tools growing in ability. The common consensus seems to be that if JSON's ecosystem developed a schema language, a "JSLT" etc etc, it would be just as ugly as the XML ecosystem (i.e. ugliness is partly due to irreducible complexity) ... and we already have one.
I wonder if the technology development path might be similar to Twitter, iterating quickly with agile, featherweight Ruby, then switching to Java only when scaling demanded a heavyweight solution: start with light JSON when the needs are straightforward; move to XML when complexity demands a heavy solution. For many wildly successful startups, that might not happen, and JSON might remain perfectly fine; just as many startups stay with Ruby.
I suppose it boils back down to the most appropriate tool. Are you selling to enterprises? Use SOAP. Everyone else? Use REST/JSON.
Before you cast stones, consider this all started as a side-project, we merged with Stockpoint after they noticed our scraping and arranged a meeting.
Anyway, with APIs it would seem useful to have an 'API watchdog' automated product that alerts when an API changes behavior. Similar to our watchdog for webpage changes when our scraping would fail. Or server-down watchdogs. etc.
It's quite pleasing when it works, and a hell of a lot easier to deal with as you just read the web page and scrape away.
I think when web pages become more clearly marked up we'll start seeing less and less of the API mess we've got ourselves into. If the websites did what we would hope they would, there wouldn't be a great need for a RESTful aspect of them.
And having several choices is a great thing.
When I integrate with an external API, I look at 2-3 other APIs as well. I then write an interface based on the common features / constructs available across similar APIs. Then I write program to that interface.
If the external provider dies, at least I have an easy path to switch to another provider.
Model the API design after the leader... similar to how some have done with Delicious, AWS, etc.
I like your strategy.
"Perl is another example of filling a tiny, short-term need, and then being a real problem in the longer term. Basically, a lot of the problems that computing has had in the last 25 years comes from systems where the designers were trying to fix some short-term thing and didn’t think about whether the idea would scale if it were adopted. There should be a half-life on software so old software just melts away over 10 or 15 years." -ACM Queue A Conversation with Alan Kay Vol. 2, No. 9 - Dec/Jan 2004-2005
Without specifics on which ideas Perl has that you feel don't scale, and how ... this is merely trolling by appeal to authority.
And Perl 5 is certainly on its way out: http://www.tiobe.com/index.php/paperinfo/tpci/Perl.html
When Alan Kay made his remark, Perl had just hit its highest mark in popularity. And it just this year hit its lowest mark. So I think you've been holding your graph upside-down.
Of course, it's not the right tool for everything - far from it - but it is a Swiss army knife.
Ok, then call me back when you've purged the world of C, lisp, Smalltalk and UNIX, and I'll give you a hand with perl :)
I can't think of any industry where your vendor / supplier guarantees it will be there forever and never change.
Even with contracts and commitments, the world moves forward.
We are just moving forward at faster and faster rates. I think there is a lot of opportunity here for API owners to provide copycat services for redundancy.
Also lots of opportunity to really keep interfaces dead simple and easy to integrate, adapt and change.
Then, the problem is the difficulty of integration: how hard it is to switch to another provider? In a mature industry, standards eventually arise that make switching easier, but not in the early days.
