We ran a security bounty for our tiny bootstrap startup. Here's what happened

We're a team of five, bootstrapped, building BCMS, a headless CMS. A year ago we launched a security bounty. Rewards were between $100 and $700... basically what we could afford. Surprisingly, that didn't stop people. Hunters still flooded in. Several a day. The issue wasn’t the size of the payouts, it was the kind of behaviour it attracted.

I expected people to try breaking into user data, finding crazy privilege escalations, or discovering ways to dump entire databases. What actually happened was… different.

We got reports about: - Creative ways to inject JavaScript into HTML through weird embedding edge cases. - Strange file upload pitfalls we hadn't thought of. - Tons of tiny UX bugs filed as "security." - And the most time-consuming one: things that worked exactly as designed, but looked like bugs to outsiders.

Example: in our CMS, you can use an "entry pointer" to select related content. A blog editor with no permissions for “products” should still be able to select products to recommend, but not open/edit/delete them. To make that UX work, our API returns product titles even if you don’t technically have product access. This is by design. We had to spend hours explaining this again and again to bounty hunters convinced it was a data-leak.

What we hoped: our existing users would feel encouraged to report issues. What we got: bounty hunters who’d never heard of us, hammering the API.

Another surprise: the program didn’t really motivate our existing users to report bugs. Instead, we suddenly got a wave of people who literally searched Google for

inurl:security "reward" "cms" bug bounty bug bounty reward $100

and landed on us. No genuine conversations, just transactional hunting. For a few months, 30% of all site visitors came from that one Google query. Our Sentry lit up from random API fuzzing, exceptions everywhere, onboarding metrics tanked.

So yes, the bounty helped. We did patch dozens of real issues that would have been painful later. We ended up paying out ~$5k total, and several reports led to meaningful patches. So even with small rewards, we got serious contributions. But it also forced us to “fix” things that weren’t broken (just to stop getting the submissions about them haha), drained time explaining design decisions, and buried real user issues.

We eventually shut it down. For us right now, it’s more valuable to understand how new users onboard, where they get stuck, and what makes them stay.

So, if you’re considering creating a bug bounty, be ready for bounty hunters to outnumber your real users. If you want signal from your actual users, maybe wait until you can afford the noise.