undefined | Better HN

story

0 pointsdsacco11y ago0 comments

Hi, I'm a security engineer. I work at one of the largest private infosec firms and I've done research on bug bounties that Google, Facebook, Twitter, CERT and 50+ other companies participated in.

Now that I've sufficiently named my experience, allow me to give my side:

1. You will never receive $100,000 for selling a vulnerability in PayPal. You probably couldn't even find a buyer for it on the "black market." I have explained why repeatedly on Hacker News before, so I'm just going to link this: http://breakingbits.net/2015/04/01/understanding-vulnerabili...

2. Bug bounties are not always a net positive for an organization. They are also not a cornerstone of good security posture. A foundational focus on robust software security would start with various other things until the financials are worked out and there is someone knowledgable to read incoming reports.

Only 7% of submitted reported to companies for a responsible disclosure program are valid. This is especially true for paid programs, where the validity percentage often drops to 3% or 4%. Loads of people who know nothing about software security try to find bugs, desperate for the gold rush of bounties they see headlining places like HN. They submit spurious reports and as a result the signal to noise ratio of responsible disclosure is fantastically bad. What this means practically is that the average organization spends between 50 and 300 hours a month investigating incoming security reports.

You can quickly see how the cost adds up here. I'm not an executive or manager trying to cut costs - I've managed bug bounties for plenty of startups and Fortune 500 companies. I've also reported bugs that loads of people tell me I could have sold for "millions" of dollars - and received nothing for it.

I love bug bounties. I run them, I participate in them. But they can be a frivolous waste of time for development teams without a solid enough grasp of security to review incoming reports, and a waste of money in the worst case.

3. I'm sorry, but you lose credibility by claiming most security reports can be qualified in a minute or less. You can certainly throw out many in a similar time frame, perhaps five minutes, but real vulnerabilities? No.

If I report a server-side request forgery in your API that requires a very specific set of actions to occur in an obscure, undocumented application function, you will not qualify this quickly. Unless you are literally verifying a CSRF issue, it is completely unrealistic to assume this.

A race condition will not be qualified in a minute. A buffer overflow will not be qualified in a minute. Budget an hour per report, and be happy when you come across the reports that take you a few minutes. XSS and CSRF are comparatively simple to verify with a good report, yes, but most other classes aren't.

Let's add to this the folks who can find great vulns but write bad reports. No exploit code, but he found something real? Good luck verifying. I spoke to a fellow infosec engineer the other day and he told me he spent an entire morning out of his work day verifying a report that came in. Not patching or even triaging mind you - verifying. Most security teams do not have the olympic level efficiency and skillset diversity that Google's and Facebook's do - it is unreasonable to assume a report can or even should be verified quickly.

This is all to say that I believe your outlook is not consistent with reality, with all due respect. Bug bounties are not a simple decision to make. I've seen development teams swamped, overwhelmed and jaded from the reports they receive.

0 comments

FeatureRush11y ago

So... about that gold rush - how much work it takes to be good at it? Could it reliably provide OK income for someone or is it like other gold rushes?

nmjohn11y ago

> 3. I'm sorry, but you lose credibility by claiming most security reports can be qualified in a minute or less. You can certainly throw out many in a similar time frame, perhaps five minutes, but real vulnerabilities? No.

That was exactly what I was trying to convey: real vulnerabilities can easily be separated from non-issues quite quickly because the later mostly entail things which can be checked in a matter of minutes.

> You will never receive $100,000 for selling a vulnerability in PayPal. You probably couldn't even find a buyer for it on the "black market."

$100,000 may have been slightly inflated, but with a bit of creativity it isn't that hard to believe. Orchestrated correctly one could walk away with a few million dollars from exploiting such a vulnerability.

> This is all to say that I believe your outlook is not consistent with reality, with all due respect. Bug bounties are not a simple decision to make. I've seen development teams swamped, overwhelmed and jaded from the reports they receive.

Or perhaps you and I simply have experienced different realities - as I have not seen development teams swamped from them and have seen major security improvements come about as a direct result to a bug bounty program.

Of the perhaps 15-20 companies (albeit all < $1b market cap) I've spoken/worked with in regards to bug bounty programs or security in general - none of them were receiving more than a handful of reports a week which took up perhaps 2 hours of an engineer's time.

MichaelGG11y ago

>That was exactly what I was trying to convey: real vulnerabilities can easily be separated from non-issues quite quickly because the later mostly entail things which can be checked in a matter of minutes.

What about the non-issues that are reported with complicated conditions but don't actually work? Just because you can throw out the obviously bad items doesn't mean the rest are real.

>Orchestrated correctly one could walk away with a few million dollars from exploiting such a vulnerability.

Exploiting it is rather different from selling it, though, right? And since a vuln in a website can literally be closed immediately, and PayPal's got whole divisions dedicated to preventing and undoing the damage you can do even with "account takeover", it'd be rather much a risk to pay someone cash for a vulnerability. At the first slip, the value drops to $0. Plus all the issues of verifying the bug and establishing trust for both parties. Seems rather difficult.

nmjohn11y ago

> What about the non-issues that are reported with complicated conditions but don't actually work? Just because you can throw out the obviously bad items doesn't mean the rest are real.

Yes. There will be some which don't fit into the overly simplistic categories I provided.

However in what I've seen the complicated condition requiring reports which turn out to not actually be bugs are rare enough where they aren't relevant to the discussion.

> Exploiting it is rather different from selling it, though, right? And since a vuln in a website can literally be closed immediately, and PayPal's got whole divisions dedicated to preventing and undoing the damage you can do even with "account takeover", it'd be rather much a risk to pay someone cash for a vulnerability. At the first slip, the value drops to $0. Plus all the issues of verifying the bug and establishing trust for both parties. Seems rather difficult.

You are hung up on what was an arbitrary example.

My point simply is if the reward for serious vulnerabilities is orders of magnitude higher if the researchers chooses the black hat instead of the white one - the overall result is a huge net negative for the world.

MichaelGG11y ago

The arbitrary example is a good one though, because it nicely illustrates why a bug in a website just isn't worth a whole lot to sell. No matter what the issue, from a PayPal account issue to a Facebook privacy bypass, the ops teams are monitoring for this kinda thing and will shut it down quick.

Do you have first hand knowledge of selling such an exploit?

j / k navigate · click thread line to collapse

0 comments

FeatureRush11y ago

So... about that gold rush - how much work it takes to be good at it? Could it reliably provide OK income for someone or is it like other gold rushes?

nmjohn11y ago

> You will never receive $100,000 for selling a vulnerability in PayPal. You probably couldn't even find a buyer for it on the "black market."

MichaelGG11y ago

What about the non-issues that are reported with complicated conditions but don't actually work? Just because you can throw out the obviously bad items doesn't mean the rest are real.

>Orchestrated correctly one could walk away with a few million dollars from exploiting such a vulnerability.

nmjohn11y ago

> What about the non-issues that are reported with complicated conditions but don't actually work? Just because you can throw out the obviously bad items doesn't mean the rest are real.

Yes. There will be some which don't fit into the overly simplistic categories I provided.

However in what I've seen the complicated condition requiring reports which turn out to not actually be bugs are rare enough where they aren't relevant to the discussion.

You are hung up on what was an arbitrary example.

MichaelGG11y ago

Do you have first hand knowledge of selling such an exploit?

j / k navigate · click thread line to collapse