Padmapper made craigslist apartment searches much better, instead of making their own apartment searches better craigslist shut Padmapper's access to craigslist data down. It shows that they value their own business more than they value the experience of their users, which to a lot of people here isn't exactly "good".
Companies are free to do with their data whatever they want and they're free to restrict access to it however they like, sure, but it's still lame when a company does it because they don't want to improve their product and don't like competition.
This implies that part of the quality of a product is NOT the ability to gather data. This would be like me walking into a Walmart, setting up a cash register next to theirs and start ringing up customers and arguing that Walmart can't compare with my customer service. Ignoring that customer service is just one small part of the business. Walmart had to find the property, buy the property, do the legal work to get the product on the shevles, the negotiation, the supply chain management, etc... I can't just focus on the last mile and then argue that I'm entitled to everything else they did.
Craigslist built its name, it's reputation, it's backend, it's API, vetted a business model for its services, scouted out regions to focus on etc...
Furthermore, people are arguing the data is the users, not CLs. True, but the user gave it to CL, not to you. When I loan books to a neighbor I assume they aren't going to create a library of my loaned books (unless they specifically told me so).
I'm not sure if it's a sense of entitlement fostered by all the seemingly easy money around tech startups or simply sheer laziness.
By charging for only a small number of listing, CL accomplish two things:
1. They create a large marketplace, which by virtue of its size provides more value than if it were a smaller, more diversified set of markets.
2. It "sucks the oxygen out of the room" for any competitor attempting to create a similar service. There are a few other commerce-related online sites -- eBay, Amazon, iTunes store, etc., and even a few competing specialized listings services (jobs, apartments, etc.) -- but none that specifically serve the general classifieds market.
I do suspect that there's a genuine component to CL's community service mission. But that doesn't mean the self-serving aspects of its behavior are any less valid.
Maybe they realized the path to greatest profitability was to act like they don't care about profits.
Just a thought.
If you value users, don't hurt yourself to show it. Rather, don't shut off sites that save them hours of their limited and precious lifetime.
In the past, do you think anybody ever looked at the classifieds page in a newspaper and thought "this data is owned by the Times"?
In the PadMapper case, who are the losers? Renters benefit from a radically different UI and de-cluttering of daily reposts. Owners benefit from the increased traffic to their listing. Craigslist benefits because it's more of a 1-stop-shop for owners: post to craigslist (and pay them), and get some level of syndication to different services and apps. They also aren't losing out from reduced mindshare of renters: when you use PadMapper you know you're using Craigslist. It's always obvious and all links point there.
How about the convenience and enjoyment of their users?
I kid, I kid.
Padmapper is the Mozilla.
If I place a classified ad it is to find a buyer for the services or product I'm trying to move. CL currently does that better than the competition. Yet the comments I see being handed around are about UI, design, automation and other items that might, but don't actually increase my chance of selling. To replace CL those sites need to provide the person posting the ad with a chance of finding a buyer that's equal to or greater than CL.
If those new start-ups can't provide that, but only have a shinier UI to offer, then they're competing on irrelevant details that aren't the main drivers behind people's decisions to post/view CL.
Others have mentioned marketing approaches and techniques to help improve those aspects. But most of what I've read is summarized as: "we'll provide a better app/ui/api" and that's a noble goal, but the reason for existing of these sits is to connect seller with buyer, what do those things bring to the table if a new app lacks a sufficiently large viewing audience? That's the problem that needs solving, not the app/ui/api one.
That's human nature. Users in marketing communities conclude that it's the best marketing that wins. If you're an executive you think the best management team is all that matters. Designers think it's design. PR folks think it's press. Financiers think it's money.
I have bad news for everyone. The world does not work in the way our echo-chambers would seem to model it. It's almost impossible for the human mind to process such a fact. Yet, it's true.
It's true that if they do something highly awful, then that might create enough demand for a competitor to take over. But it would have to be a high degree of awfulness to overcome their existing momentum. There are many shades of not-quite-awful, where those companies can make decisions with market impunity.
So considering that we're kinda stuck with those guys, I think the question becomes, why shouldn't we complain if they are doing something that we don't like? Doesn't matter who owes who what. It's our world, we should try to improve it.
The consumers absolutely have a right to express that they are dissatisfied with the exchange being made.
I might be inclined to agree, if it weren't for network effects.
Once a company reaches a certain size and market share, like Craigslist, would-be competitors are at a competitive disadvantage. Lots of people have created objectively better sites than Craigslist, but they fail because of network lock-in. Normal capitalistic competition is failing.
This is the same thing that happened with railways, utilities, and other natural monopolies. In these cases, it is necessary and proper for the government to step in and regulate access and interoperability.
Sites like Craigslist and LinkedIn have become the new natural monopolies. And demanding that they open up their data is the natural, and right, action to bring back natural market competition that benefits consumers and the world at large.
There is no lock in. A buyer or seller is free to move place their listing or make their purchase from any number of alternatives regardless and Craigslist can't stop them.
> Sites like Craigslist and LinkedIn have become the new natural monopolies.
Mark Zuckerburg would probably disagree with that statement. In 2005 MySpace would have been the equivalent of Craigslist, massive network, shitty design and no data portability. If the value provided by alternatives to CL were as superior as most HN readers believe them to be then END USERS would see that as well and migrate just like MySpace users migrated to FB. But they don't so CL retains its dominance, the failure of alternatives is not indication that CL is cheating or doing anything even remotely monopolistic.
> And demanding that they open up their data is the natural, and right, action to bring back natural market competition that benefits consumers and the world at large.
The data is open, just not in a format you think you are entitled to it in. It's open in that the same poster that put their listing on CL can post the same listing on alternatives, they just choose not to. Convince the seller or buyer of the value you bring to the table and they will open their data to you as well.
Many HN readers have reached the conclusion that convenience and competition are the same thing. They aren't.
If the value provided by alternatives to CL were as superior as most
HN readers believe them to be then END USERS would see that as well and
migrate just like MySpace users migrated to FB.
This type of dilemma is known as a collective action problem. When these problems arise the best way forward is often to have a superauthority such as the government step in for everyone's benefit. For example, even though it is generally accepted that free trade increases the world's prosperity when it is practiced around the world, individual countries occasionally see immediate advantage in adopting mercantilist or protectionist economic policies. The GATT is an attempt to provide a superauthority to stop such practices, because the immediate gains possible when a country embargoes an enemy are not as valuable as a system where the majority of actors behave in open and unbiased ways.
The free market doesn't work on its own. When there are systemic problems in the way actors conduct business, judiciously applied regulation can help promote the public interest.
Craigslist disrupted a huge classified industry without any legislator saying that papers had to "open" up their data. Markets work over time.
Craigslist disrupted the classifieds industry because it had first-mover advantage in a new medium, the Internet. It now abuses that advantage in order to not innovate or even compete.
The market is not working over time -- proof is how everyone complains about how crappy Craigslist is.
But a solution, for example, could be something like: legislators could force companies with over, say, 25% market share, that are mainly based on user-generated content, to not be able to sue other companies if those companies re-display information originally input by users, that is publicly visible on the site.
This wouldn't force Craigslist to alter its behavior one bit. But it would allow entrepreneurs to actually fight once again on the basis of their ideas, as opposed to whoever had first-mover advantage and subsequent network lock-in.
I'm not so sure (for reasonable values of "forever"). From what I see, once some software hits a critical mass, it's nearly inextricable.
Look at Microsoft. Before they became a major player, the dominant OS and computer system would seem to shift every couple of years; Amiga, Commodore, Apple, etc., but once Microsoft became a major player they never really lost their grip. They continue to maintain that grip simply because so many people were familiarized on their systems and so much software was written for those systems. Does anyone really believe that someone looks at OS X and Windows these days and decides that Windows is objectively superior? Most people, even technical people who hate Apple's MO, would admit that OS X is better on its merits alone.
The reason that MS has dominated and continues to dominate is because of the software available for the platform. It's all about lock-in. People don't use Windows because it has more compelling features than OS X, they use Windows because they want to play games, buy software off the shelf and know it will run without any problems, or run their company's internal software.
Let's take Facebook as another example. For years there was a pretty constant, roughly bi-annual shift in the most prominent "social networking" site; LiveJournal, Friendster, Xanga, etc., until Facebook. Around 2008 Facebook began eating MySpace's lunch and Facebook remains undisputed social networking champ today with no signs of slowage.
I'm concerned that Facebook is the new Microsoft -- it has become the de-facto standard, and despite any number of superior features that may be developed in other systems, it doesn't look like anyone is going to be able to challenge Facebook seriously. Some would say that Google was perhaps the most worthy challenger, but even they have failed to really make a dent despite the fact that most of the world visits their search and email services daily.
I'm not sure exactly where that critical mass gets hit but I think it's hard to reverse once it happens. Maybe it's something like, "Once your grandparents use a certain program because everyone else does, it's relatively safe to call it 'ingrained'." I know that my grandma had no compulsion to create a MySpace profile even though she constantly [attempts to] check[s] Facebook now, and I also know that it'd be extremely difficult to train her to use a non-Windows interface.
1. The goal of almost every rational business is to achieve a monopoly in a certain space. If the public or the government had the power to systematically break up every company who comes close to success, there would be little incentive to start a business. Perfectly-competitive industries are awful for business owners. You have to draw a line in the sand, before-which monopolies as tolerable. I don't think Craigslist is anywhere near that line.
2. A huge part of anti-trust laws means that companies engage in anti-competitive practices. To the best of my knowledge, Craigslist isn't doing this. It's simply refusing to willingly participate in its own destruction.
3. I question your usage of the words "normal" and "natural". First you claim that monopolies indicate the failure of normal capitalistic competition. Then you claim that monopolies naturally occur. Then you claim that we need to act to bring back natural market competition. So which is it? Are monopolies "natural" or is competition? And why does something being "natural" make it good?
So, so many people have forgotten this. Thanks for pointing it out.
1. No, the goal of most businesses is not generally to achieve a monopoly, the goal of most businesses is market share or just profit. A company can have 100% market share and not be a monopoly, if the barriers to entry for new market participants are comparable to the original barriers to entry for the current winner (so there aren't network effects, or prohibitive capital investment relative to divided market size).
2. I never talked about anti-trust laws or anti-competitive practives. I talked about natural monopolies [1], which have nothing whatsoever to do with those. You're thinking of artificial monopolies, something totally different.
3. I said "normal capitalistic competition" in the sense of competition outside of special cases (like monopolies). And "natural monopoly" is an economics term [1]. I used neither in a normative sense.
Consumers do not get to choose alternative railways. They can chose alternative classefeids services; those services are just inferior (maybe they have a better UI but they aren't as good at selling things)
Internet is still relatively young, but I think eventually we're going to have to have laws that regulate how companies that collect user details allow access to 3rd parties.
Monopolies are illegal for a reason. Companies who build infrastructure have to manage it fairly.
And for that data, they provide you with services. That doesn't mean they should be required to hand that data out to any service that requests it?
I mean, are you really asking that companies like LinkedIn allow free access to it's api's and user data to any 3rd party that requests it?
> Monopolies are illegal for a reason.
Well, no. Monopolies just require additional oversight. Microsoft, for example, didn't get in trouble for being a monopoly, it got in trouble for unfair business practices as a monopoly. More importantly, a monopoly doesn't exist with any of the companies presented in this article.
Really though, your comment fits with the picture at the top of the article.
It does, however, mean that it's up for negotiation, if the consumers want it to be.
APIs are nice, but legal hassles aren't worth dealing with.
Legal hassles are totally worth dealing with if the cost of managing them doesn't exceed your profit. If you're the right company, legal red tape can be a very effective barrier to entry.
It would be great if there was regulation over certain types of data that would require companies to post back to a central database if they take certain kinds of information. It would be great if there was a device that could help level the playing field.
The point of my post was to highlight that complaining, or being shocked by these companies behavoir is sort of missing the point.
How can we unify and provide access to what we feel to be 'universal' data types?
My god, I hope not. The last thing we need is legislation that determines, because I went through the work of building up and creating a successful product and now have a massive amount of valuable data, I have to share it in the interest of 'leveling the playing field'.
I've been on the side of being a content creator with a very successful site, that was promptly scraped by a competitor because I had ammassed a large amount of very useful data. This data that was, through my site, freely available to the public, and the ad revenue helped pay the bills.
So I really don't have a lot of sympathy for someone who had, as part of their business model, the practice of scraping content from someone else's site, nor do I agree that we, as developers, should have free and unrestricted access to someone else's API... they are the ones gathering/storing the data, building the network, etc.. and any use they decide to allow us is at their discretion and a gift. Nothing wrong with that.
The idea of mandating any such rule is not the solution. It's antithetical to how markets should work.
There's been no company that's managed to dominate forever. LinkedIn, CL, Netflix will be no different. Someone or ones will attack them (perhaps orthogonally) and ultimately they will lose their dominance. That's what happens to incumbents. The market takes care of them. Note: this disruption may or may not be quick.
I'm not so sure. Communication tools are fundamentally different than most other products, because their primary value is the number of other people reachable through the service. This isn't true of most other industries, where you can drive a Toyota and I can drive a Ford and everyone is happy.
When your a whole market is based on the ability to interact with other people in some way, and a single company can flip interoperability on and off like a light switch, it has the potential to be a dangerous monopoly. The government stepped in and broke up Ma Bell because of this.
"I'm from the government and I'm here to help!" - music to every Internet entrepreneur's ears.
I mean that PodMapper was a service for Craigslist's users, first and foremost. And banning PodMapper means that Craigslist doesn't give a shit about us, its users. Do you really believe that PodMapper did any damage to Craigslist?
You seriously think Craigslist doesn't give a shit about users? Think about what you are saying. Think about the world before Craigslist. It's a free service that even my mother uses. This is the _exact_ sense of entitlement that the OP is talking about.
So for me personally PadMapper provided 50% of the total CL positive experience.
I've used ebay over 100 times. Ebay is ripe for disruption, imho, but it's still okay.
You sound very nervous.
Whether or not we think PodMapper is a better service for "us", it's not our choice to make on behalf of Craigslist.
I really would like to stop hearing the whine "but they have a right to do X" every time a business's practices are criticized here on HN.
My point wasn't so much that they have the right to do that, but more about when your playing with another company's API don't expect to disrupt them.
Choose a start-up that is already successfully and profitably using a company's API and make your service even better. AKA don't compete with the mothership.
If Microsoft decides tomorrow to say "if you want to write Free/Open Source Software, you can't use our API's" do they have a right to do so? What if it is competing with Microsoft software, i.e, "no use of our API's to build competing web browsers and office software" or the like?
I am genuinely asking because I think it's as clear a line as one might think.
Your own linked articles about that topic are a user only complaining about the legal takedown of a useful tool [1] and another complaint about UX with an included blurb asking startups and users to put their time and data into something better [2]. Nothing about API changes or right to a company's proprietary data.
[1] http://blog.garrytan.com/save-padmapper-craigslist-is-wrong-... [2] http://pandodaily.com/2012/05/30/craigslist-is-squashing-inn...
Wouldn't we all better off as a whole if technology wasn't solely based on profit only?
I think that's the real question, and what started the whole "API" "open XXX", not "OMG I WANT IT /CRY/CRY" that the image suggests. Remember that most devs nowadays participate in open source projects and start to have different ethics than they used to have (which was, money > everything, and justifies all actions)
Or the dev-became-CEO, too, I guess. Not all of them of course.
I just wish it would keep on being "more open" for these reasons, and it would be sad to go back to an "all proprietary, closed, etc" world.
As a provider of data from which these companies make their profits, I believe it is entirely reasonable to demand that they make those data accessible to other services I'd like to use. Sadly the trend of the Internet has been away from open systems and toward siloed, proprietary data stores.
What you're upset about is that you've already given them your data, and now you want more for it. You've upped your price after handing over the product, and them having delivered the product.
Am I somehow mystically bound to never ask for more in return for the data I provide them?
Of course we can ask to change the terms. It's a business transaction.
Google: man up and build your own database of everything. Stop indexing our data when we already offer our own search mechanism.
I get that everybody wants to build ajax web apps and ios apps and android apps and so on, but what's the point of the internet except for being the default client-server protocol if we can't create mashups? The whole point of the internet is that data is public and linkable and interconnected. If companies don't want to allow scraping and they don't provide an API for their non-user-specific data then they're defeating the whole point of the internet.
The vast majority of their users don't give a hoot about whether or not other parties have a way to wrap up craigslist data and do stuff with it.
The whole point of the Internet is that it's a collecation of independently run networks that can communicate. The point of the WWW is to serve hyeprlinked pages to people. There IS NO POINT to the internet - it just happened, and here we are.
I was watching Ghost Dog: Way of the Samurai last night. In it there are a number of Samurai codes that are highlighted in title cards. One of them was this:
"If one were to say in a word what the condition of being a samurai is, its basis lies first in seriously devoting one's body and soul to his master. Not to forget one's master is the most fundamental obligation for a retainer."
Other companies' data, or other companies' users' data?
And, if you go against that approach you shouldn't be surprised or indignant when you get shut down. You will have a much better chance building a truly complimentary product, rather then rallying users to boycott a service or demand changes in policy.
To me it's sort of like someone who always drives over the speed limit by at least 50mph, and one day they finally get pulled over and are given a ticket. But, because they were never 'caught' before, they just feel like they were entitled to always drive that fast, and instead of just paying the ticket, they try to get the speed limit laws changed, or to have a judge throw out their ticket and let them keep driving however they want.
When you use another company's API, you are driving on someone else's road, and for better of worse you need to play by their rules.
So if you want to drive fast with no consequences, build your own road. And if your interested in 'exploiting' or piggybacking on someone else infrastructure, don't do something to call too much attention to your self.
PadMapper - started providing other listings that were not from craigslist. GoodFilms - providing information and data to other movies services besides netflix Pealk - were undercutting the price point for LinkedIN premium features.
And yeah, these companies don't "owe" us anything, but consumers (even the subset of developer-consumers) can still get pissed off at them and say shit and that's the fucking market. And maybe it won't mean anything, maybe it'll all fade away... and maybe it won't. That's the risk those brands are taking. It's our market-based prerogative to bitch about whatever the hell we want to bitch about, using whatever hyperbole we want, with any sense of entitlement that we've acquired.
If someone starts sending mail bombs to these companies because they aren't tending their APIs as we'd like, then that's coercion. But there's no coercion here, this is just people offering their opinions. So quit yer whining about our whining, the internet has entitled us to whine as much as we fucking want to, and has offered market-based ways to lift or bury that whining.
Sure, they don't owe third-parties anything, and personally I think it's folly to base all or even a significant portion of your business on data or functionality that you don't control.
But that doesn't mean that the community corporations do business in must always just shut up about it when, for instance, one company pretends to support a third-party's efforts for two months, just long enough to get a good hard look at their user experience, financial standing and business model, and then summarily shut them down without recourse.
"It's just business" is fine, but it's not a get-out-of-shame-free card. Just as third-parties can't complain when an API is yanked out from under them like Lucy with a football, so corporations shouldn't get butt-hurt when the community they work with looks at them and says, "the way you did that really stinks."
I think less people would be concerned about the issue if it was just Craigslist putting up technical measures (e.g. blocking API access) to competitors, but competitors were free to rent VPSes or pay people for use of their IP address to access the data for scraping. If there was no law against scraping information from websites that is available to arbitrary members of the public (provided that you comply with copyright laws post-scraping), then there would be no issue.
So the real issue is not that Craigslist owes anyone anything, but rather, that the government is enforcing laws on their behalf to entrench the network effects of existing businesses against new entrants to the market.
If there is API access it should be enforced with more care or not made available at all. Personally, I was skeptical that PadMapper would go anywhere (expecting CL to kill the scraping early on). Much to my surprise it stuck around and furthermore was really useful. I'm not mad that it is gone: I read the TOS from CL - it's totally their right to do that. It just hurts end users which is the problem.
And sure, I do believe it's dangerous (and can be misleading to users [like when your data access gets pulled]) to build a product entirely on a platform, but it's almost unavoidable. (That's why you are the platform). Companies like PadMapper are going to try to improve where you have left off. (They probably didn't even register on CL's logs until they queried a lot which might explain the delayed C&D).
Let our entitlement / disappointment be a message to these companies on how to improve their services (ideally they could improve in a way that makes them more money). Or hopefully fuel for others to build competing products.
That's a two-way street.
As crazygringo pointed out, a very large part of the value of these firms comes not from their technology but simply from their market position, dominating a specific niche, and generating network effects as a consequence of size.
In a market in which there's a tendency toward monopoly: utility services, telecommunications, desktop/personal software, business office software, business systems software, advertising networks, publishing, broadcasting, pop music/entertainment, agricultural middleman, major chip foundries, pharmaceuticals, a large part of the value accruing to the firm/organization is a consequence of the interest / business which society as a whole has invested in the monopoly.
At the same time, twin abuses of monopoly pricing (which accrues a greater marginal profit to the monopolist) or anti-competitive actions (driving out competition by way of locally (time or region) undercutting prices, contractually discouraging or prohibiting customers from doing business with competitors) exacts an additional cost on society as a whole.
I've long been a fan of Craigslist. I think that their business exemplifies some of the best of how to conduct an online business (leverage technology massively, but also rely on community goodwill to conduct many business operations, including removing of bad/fraudulent listings). While I'm not a particular critic of CL's page format -- they've held to the KISS principle in the extreme -- it's become increasingly clear that for many types of listings, there are some very evident improvements which could be made.
My feeling is that CL are squandering an opportunity in their dealings with Padmapper. The Padmapper interface, for apartment and real-estate listings, is vastly superior to CL's existing listing format. While there is a business risk to CL in allowing another party to utilize its comprehensive listings (for which CL would reasonably be able to assert compilation copyrights, answering some here), I really believe it would be to CL's benefit to find a reasonable business relationship under which it could utilized Padmapper's interface. Whether that's a data exchange agreement, a purchase or leasing of the technology, or simply a reinvention of it, I don't particularly care (I also owe Padmapper nothing in particular).
And if that's not the case, then, well, CL show a case of a market which, if a feasible business case can be made, is ripe for disruption.
Moral is if you quit, you are a quitter. If you are useful to people find some other solution implement it asap and keep rolling. I am not yet living in the sates but if I truly believe that my service is useful to people I'd just go and try to do it differently and move on with that. Never give up, especially if you are doing something that is good. If you're causing rucks and are being useful maybe that might be an opportunity of a lifetime...
my 2c
This is also how one destroys markets. Say for example dumping cheap corn on markets to destroy the livelihood of local famers, then when they are gone jacking up the prices far above what would have been possible when the local farmers were still operating. This and other monopolistic practices are illegal and generally accepted to be morally reprehensible in most of the world.
There is a implicit deal that is being struck between suppliers and consumers. That what we get today we can get tomorrow. TQM style supply contracts often call this out explicitly building long term positive relationships between supplier and consumer. Unilaterally changing the deal is bad for buisnes by leaveing the overall environment unable to reliably plan or predict future actions.
Both sides had a point, and while my bias is with [Bidder's Edge], I don’t mean to deny the plausibility of a different regime. What I do deny, however, is that the answer to this question is obvious.
One is "a website like website x does anything it does to generate profit". That would be the side the OP takes. The other is "a website like website x does anything it does to make the world / the internet a better place", something like that. I think an open source enthusiast would take this position.
Obviously people value these ideals differently (otherwise this discussion wouldn't be there) and therefore voice different opinions. With that in mind, I think it's easier to evaluate the opinions of others on this matter.
On the other, I think that when you have a businesses creating industries around themselves, that a significant amount of that autonomy has to be abridged.
Take for example Ebay and how they strongly encourage Paypal. If you run into a problem with Paypal, and your business has a significant presence on Ebay, that's a pretty heavy cost, and I don't think Ebay should be able to say "so long, tough luck."
While unfortunate, the move by Craiglist isn't surprising. They have done this repeatedly. The situation PadMapper is in is inevitable for any entity that isn't self-sustaining..
PadMapper should be working as quickly as possible to capitalize on the added attention this has caused and start letting users input their listings directly..
If you got a C&D as well and isn't happy about it, you should go ahead and make a better product. Otherwise you can go to work for one of their competitors. That's the beauty of being a hacker.
Just make it sure that online classifieds is your passion.
Craiglist is available by RSS and would not be too hard to scrape. You want "API access", you got it. Unlike Facebook, I haven't heard of anyone tossed off Craigslist for using a script to access the data Craigslist is happy to give out for free.
Ah, but you don't want "API access", you want a license to resell the data - and for free! You want different terms of use. Etc. Wah, wah, wah.
Guess what? That's different. Obviously.