Rapgenius.com Traffic and Demographic Statistics (opens in new tab)

Penalties are blunt instruments.

The whole point of this penalty is that Google cannot trust the signals that are in its index when it comes to Rap Genius's site. And it sure isn't Google's fault that they can't.

Can Google subtract out all the "naughty stuff" and decide if Rap Genius deserves to even be on the front page for "rap genius" anymore? I doubt it.

If Google's search had a category for "obvious searches" then yes, maybe they could apply a penalty as delicately as possible. But delicateness is not really the point. Dropping them down 50 spots is blunt and simple. I think everything about this approach makes sense.

>De-weighting RG is one thing, nuking it into the ground is entirely different, and is in my opinion unwarranted. If I type "Rap Genius" into Google, I am justified in expecting Rap Genius to be the first result. Period. When Google goes out of its way to effectively expunge a site from existence (to the extent that Google can expunge a site), they are doing a massive disservice to their users and are degrading the quality of their search results.

Necessary reminder that Google does whatever the fuck they want with their service, and that no one is in a position to expect anything. Does it suck? Yes. Is it a predictable outcome of the free market on which Silicon Valley and the tech industry in General is built? You bet.

> Moreover, I think many people are upset with the scope of Google's punishment. De-weighting RG is one thing, nuking it into the ground is entirely different, and is in my opinion unwarranted.

If the impact wasn't significant, it wouldn't be a disincentive to engage in black hat SEO.

> There is no reason that a response of this magnitude by Google is justified

Yes, there are reasons. The reasons are "specific deterrence" and "general deterrence".

Cutts has said before their goal is to "break the black hatters spirit" (http://searchengineland.com/googles-matt-cutts-on-search-spa...). I assume that's what they're trying to do here.

The response was designed to send a message. And it seems to have accomplished that.

Interesting point!

Google's search team has been famously resistant to any influence by Google's sales/marketing teams on ranking decisions. Search quality gave rise to the 'golden goose', after all, and the self-conception of the team, and its individual members, is based on independence from purely money-grubbing considerations.

But I wonder, what happens when Google's antitrust lawyers came to the ranking team, and say: "soften this penalty [or otherwise tweak these quality-rankings], because otherwise we'll be in more trouble with regulators or leave too much evidence of market-power."

Could the search team still say, "buzz off, we're sticking to what we know from the numbers is what's best for our users"? Or does keeping regulators happy, and avoiding smoking-gun fact-patterns, trump everything? I suspect it often will, because of the unique risks of state enforcement and appearing unlawful. It's also easier to rationalize "we're sacrificing our own idea of what's best because the law seems to require it" (even if the actual risk is fuzzy before losing in a formal legal process) than "we're doing this to make the guys over in ads/mobile/etc a bit more money this quarter".

you are not using that word correctly. If Google was the only company doing search and prevented others from entering the market you would be correct. Being the best through a superior product and innovating is not a monopoly.

> This topic is interesting because it shows how much in a monopoly status google really is in in controlling where traffic goes on the net.

Does it? I mean, is there any reason to believe that Rap Genius is typical in their degree of dependence on Google here?

I'd replace the word monopoly with popular.

I think people - foreigners and valley outsiders for the most part - are often genuinely clueless about the workings of the valley.

They are quite apprehensive of the degree of even-handedness that is at play in the valley. They suspect an air of nuanced in-group favoritism, within startups here, that help propel some and sink others.[1]

Hopefully these kind of actions (on behalf of Google) help dispel such notions.

[1] http://www.slate.com/blogs/future_tense/2013/09/13/silicon_v...

I personally know of a number of legitimate sites that were invariably affected due to content and blog farms harvesting their content when Google's panda algorithm rolled out.

Those penalties hurt a lot of good people!

Anyways, this one is a bit more complex than how much the penalty hurts, rather it's because, a well known and well funded start-up adopted shoddy traffic building practices that were unacceptable years ago, got caught and then had swift and targeted judgement bestowed upon them.

However much people understand that sometimes business needs to be conducted in a grey area, there is a certain amount of satisfaction when the system catches up and behaves fairly.

I think this is exactly it.

Their affect is another example of cultural appropriation of "urban" culture by three very affluent guys (Stanford & Yale grads), which I think turns a lot of people off, including myself. As someone who genuinely grew up in that culture, it is painfully cringe worthy and just sad to see people who have no background in it adopt it. Their Disrupt interview was a genuine display of the generally awkward douchebaggery that comes from that appropriation.

So when I see folks like this do poorly because of arrogance, it is indeed schadenfreude.

Beware trying to assess "what HN thinks" about a topic from informal, personal scans of items. Neither the top stories, nor the longest/loudest threads, are necessarily representative of 'most readers' or even 'most commenters' – just those that show up at those times/places.

Where the especially peeved choose to spend their attention (and comments and upvotes) can make lots of people or companies seem like community bete noires – even if most of the HN audience thinks the targets are fine or even admirable, but just has less energy for anti-carping.

why is that? i assumed HN would like then since i remember a bunch of articles about RG being big RoR proponents

Google just proved all that when someone breaks their laws they punish him, even if it's a YC company.

Who knows maybe because they are a well funded YC company Google decided to make an example out of them and won't forgive them so quickly.

If you're right, and were this actually intentional too, well, that would be genius.

But given that their traffic is maybe off two-thirds now and that it is unknown how long google will leave the lights on (weeks, maybe months), during which time this will be long-flushed out of the news cycle and it becomes harder and harder to recover, and harder to secure things like more capital when you look this stupid, that's a high-risk bet. I'd say sabotage is more likely.

I don't think organic links matter much at this point when you can't even Google their name and find them on the front page.

>there's a very blurry line between 'growth hacking' and 'being annihilated by Google' and no-one's really sure where it is. That's what's interesting here.

Except in the cases where the line isn't blurry, i.e. the well-defined set of rules about which G has repeatedly said "we'll penalize you for ignoring these!" Unfortunately the tactic RG used wasn't a blurry-line grey-area tactic, it was one G clearly said don't use.

>Can you explain why you care if other people discuss a topic you don't think is "worthy" of discussion?

I thought there might be a piece to this story that I was missing. I gained a lot of value from the replies to my question (esp. the conversation spawned from @snowwrestler's comment.)

>You know you can ignore the story and look at the entire rest of the front page, right?

You didn't really need this in your comment. It was a bit of snark that devalued your otherwise justified response.

Also, nobody seems to mention the fact that this story was created out of an HN thread... I imagine that has a lot to do with everyone's interest in it.

#3 definitely is why there is so much enthusiasm running around regarding their punishment

On 3, I wonder if they think being cool is the most important part, or the reality is that they are executing on everything AND ALSO trying to be cool to differentiate themselves AMD get noticed, but the critics only see the PR front and ignore the rest.

Yeah, you're right, missing this makes them incompetent.

Does that word have any meaning at all anymore? It seems like everyone who isn't an amazing brilliant self-made billionaire is incompetent. Oh wait, I forgot, we were discussing the incompetence of a bunch of amazing brilliant self-made billionaires... So I guess it's everybody now.

Was about to write a similar response. In the end they will walk away with the penalty removed, and hundreds of links from bloggers talking about this story. While the links may not be the highest of quality in terms, they will still be links from sites like TechCrunch, The Verge, and many others with high PR scores. The story will die down over the next couple days and then I am sure we will see another influx of stories as their ranking increases again, providing more links to their domain again.

either way, 1 data point isn't a trend. wait and see is the best course of action.

Ah, cool. I did not see that. That makes much more sense.

Actually no, in this case it's directly measured. Quantcast only shows exact stats for sites that include their JS tracker. Sites normally do this for advertising purposes (advertisers don't have to wonder about your traffic).

its way too early to be friends as well, lets wait a few days.

lol its too early...friends = threads :)