How many people do you think are browsing with a weird enough config (eg. custom browser like OP, or some weird config like firefox with fingerprinting protection on a raspeberry pi) to trip cloudflare's protection?
We had all of our Devs Pixels get blocked, and after talking to CF, it was because Internet archive was rebooted their scraping farm, all the devices stampeded and overwhelmed the known bot safeguards, and those tags were added across the board. CF gives sites the tools to tune what is getting blocked, we bumped the sensitivity down to 25 and haven't had many complaints (despite having a very vocal community)
The most common complaint is users' IP address getting blocked because of compromised devices
And no, it wasn't due to a country-based block selected by site operator.
In my experience what really makes it loop every single time though is JShelter. CF doesn't like having your fingerprintable data bits messed with.
There are legitimate uses for non-instrusive, ethical and legal scraping, but some of us have had to resort to extreme measures:
[1] - https://blawg.nochan.net/b/Internet-Crap/20260522-Maybe-AI-B...
At least let me complete a "prove you are human" challenge or something, but don't outright ban my IP address?
It takes very little for CF to consider you "weird".
I know some actual users get blocked. But the amount of spam we get without it, the amount of bot traffic simply overwhelming the server... It is just too much.
Recently I also hard blocked all IPs from china Singapore India Pakistan Russia and whole of africa. Do I want to do it? No. But the amount of bot traffic and corresponding spam is a bigger problem :(
At least for China, I imagine most of the real humans might use a VPN anyway
> I know some actual users get blocked. But the amount of spam we get without it, the amount of bot traffic simply overwhelming the server... It is just too much.
One of the core tenets of system design is Availability. If your service is not available - if your forms are blocking legitimate users - then why are you pretending to have a form submission feature at all? Just to frustrate users?
The service won't be available to anybody because of overwhelming unwanted traffic. Now it's available for most potential users. You're speaking econ 101 when everyone else has played out iterated prisoner's dilemmas.
If you are getting blocked by CloudFlare, you are most likely not our demographic.
And there's always email address given in form submission, so a couple of users (like less than 5), emailed about the block and I added rules for each of them.
Better than taking down the whole thing because of bots scraping the site 5x more rate than humans.
Turns out that people have a tolerance for a non-zero amount of work, but still have a limit.
Suggesting "turn off your website" is does not account for the desire to also provide some access.
Treat people who host content as humans, just as we must treat users as humans. There are tradeoffs, suggesting "shut down your website unless you provide access everywhere" is worse on all fronts for everyone.
Yesterday cloudflare blocked me from visiting the MX-Linux site ... including an old browser with -no- protections ...
I have to wonder - assuming these sites are paying CF for this 'service' - are they getting a list of all the fejected IPs?