However these errors are generated... in error. The page links to a DMCA complaint which lists about a half dozen unrelated YouTube (and Piped) links, none of which are being accessed when the error is generated. In fact, viewing the video on YouTube plays back fine. There appears to be a glitch in Cloudflare's URL filtering. It's been happening so frequently that Piped is often unusable.
I really want to just see us get to the point where we don't have to rely on such services. I refuse to use them or any other for services I run, DDoS be damned.
It goes to show the flaws of centralised services where you are not the customer. Not only is there no one to complain to, you can’t even take your money/traffic elsewhere as the competitors probably use cloudflare too
Specifically, there's a ticket in progress to improve how bot mitigation handles requests for certain types of static content (including RSS feeds).
Lots of spam hosted on cloudflare these days.
Is it easier/better to block by IP than to block unknown free domains?
If CF had a simple way to get (verified!) customer details, much of the crime using CF would go away while the pure DDOS-protection and CDN-usage wouldn't be impacted. Legitimate companies have their legal info on their websites anyhow, they don't care if you also can query CF about who they are.
I believe that once you put your website behind cloudflare it's really hard, if not imposible to get content using requests. Don't know about scraping tho.
Also, I think it's better to block unknown free domains, because (public) IPs can have thousands of devices asociated with them. Once you block a domain, the "scammer" has to buy a new one.
Wouldn't that be a dream world for Cloudflare? "We protect spammers and if you wanna be as well protected against said spammers, sign up for our firewall"
Ha-ha, only serious.
If you're scraping with Python, try cloudscraper—among other things(!), it supports JS rendering (basically the bare-minimum check cloudflare does), without needing to run a full browser in the background. It's built on requests, so integration was pretty easy.
In this case, you're moving the trust you put in your ISP or anyone who resolves your DNS queries to Cloudflare. Depending on where you are in the world, or how your threat profile looks, this might be good or bad, or degrees of good/bad.
That everyone is starting to tunnel more and more of their traffic to one single entity (Cloudflare or not) is overall not that good. But certainly not 100% bad.
in other parts of the world ISPs give your DNS data to the not so secret police and compared to that Cloudflare is a huge improvement
in the parts where ISPs don't sell your DNS data you should switch to a different DoH provider
And even knowing the technical differences, one may want to dissociate with a stupid domain name like yewtu.be.
Edit: I showed https://yewtu.be/channel/<channel_id> to a content creator friend just now. Predictably, the reaction is "WTF, am I being impersonated? What should I do?"
TL;DR: Apparently there's a Hong Kong dude living in Germany that didn't like his videos being on Youtube, so he sent DMCA takedown requests to Piped instead and Cloudflare did a takedown on the whole domain, which only appears if sent as a referral from outside piped.kavin.rocks (or using the redirect extension for firefox).
However, being on both sides of this, both operating a bot for my search engine, and operating a web service that is aggressively targeted by bots. They're not actually bad to deal with.
The big unanswered question is how they'll manage to stay good given the obvious incentive of abusing this setup. Maybe this CEO has a moral backbone, but will the next, and when they're acquired by the Meta-Amazon-Alphabet group in 15 years, will they still stick to these principles?
It was true twenty years ago too, the only difference I can see between then and now is that you can outsource that task for a (relatively) small amount of money if you want to.
Then again, the last time I dealt with a site under DDoS, something in their stack was leaking the underlying IP (never did figure out what) but it turned out that "finding a provider who'd sell them a decent sized server and charge them for the bandwidth" was perfectly economical for their use case because their haters' firepower was insufficient compared to their revenue.
(I'd love to be less vague here but I'm sure readers can see the obvious professional ethics issues with doing so)
Why do you think they're still "good"? CloudFlare has chosen to abandon sites that held free speech (abhorrent speech, but still free speech) while still protecting forums upon which credit cards and methamphetamine were listed for sale on the front page.
To me, that's not a sign of a "good" actor.
RE bots: TikTok has incredible bot protection that comes from engineering (webmssdk) instead of network-based filtering. I'm not even sure if they use Cloudflare.
Exactly what you do _not_ want protecting the neutral internet. They've done better being neutral than some might have, but that's in reality more insidious because clearly there are points they will bend on and those points will change over time and almost certainly continue to erode.
But from some basic calculations I get that R2, Workers and egress bandwidth beyond a few terabytes costs just as much as Oracle cloud / Alibaba.
But what I dislike the most is how little control you have over what's going on there. Like: If you haven't setup TLS on your webserver, why do they allow unencrypted traffic to flow between the server <-> Cloudflare and encrypt it to the end users and pretend that is secure?
Why can't they forward all my server's headers? Why <XYZ> ?????????
Read some horror stories on Hackernews and you'll quickly find out what their "unmetered bandwidth" really means. You get very little if any transparency about the pricing, which I would except from tiny cloud companies, but this is supposed to be a major one!
1. It's probably better than nothing. 2. It's a legacy thing.
A company like Cloudflare has to make a choice - how frequently do we break users who've set up their site in a way that is no longer in line with security best practices? It looks like the decision they've made is to break infrequently. Certainly the site I set up in 2014 when their free TLS was new still runs, and I haven't made changes.
I believe that you can set up strict TLS between Cloudflare and the end host if you choose, but it's up to you. I think in that instance, your 'little control you get' is actually more control, no?
And, if you look back even a few years, TLS was both uncommon and expensive. Cloudflare was a pioneer by offering free TLS certificates in I think 2014 (only 8 years ago!). LetsEncrypt started in 2015 and was niche for quite some time. I think even now you can find Linux distros preferring to ship their data over HTTP with GPG-keys recommended for the security. Of course in 2022 even simple sites should be TLS'd, but Cloudflare's existed for a while.
And, TLS to the client but plaintext from CDN to site is still better than cleartext the whole way, because it (generally) stops the ISP from snooping on its customers.
Some competition exists, but it's both more expensive and less reliable and convenient.
I don’t get the issue here. The traffic between client and Cloudflare is secure. SSL is terminated at Cloudflare. You can choose to have end to end security if you want.
If you set up your own frontend that terminates SSL, but choose not to secure the traffic to your backend, the end client will still see the connection as secure.
Sorry if I am missing something here. Cloudflare gives flexibility to their customers. That seems right.
Cloudflare enterprise is pretty transparent if you've gone through the sales process. They tell you exactly what the limits are. For average person, on free plan, they are not obligated to provide details of where the limits are. That's no different than BackBlaze unlimited storage plan.
I Really Can't Think of Any Reason
One problem was that if the code was TOO aggressive in protecting from a denial of service attack, you could actually help an attack or be the culprit yourself by denying legitimate traffic.
I think this is what cloudflare is doing. They are imprecise and they are denying legitimate traffic.
Years back their DNS service also stopped honouring ns_t_any requests (for reasons of DDOS amplification apparently).
I do tend to agree with you about centralisation, gatekeepers particularly.
https://blog.cloudflare.com/introducing-cryptographic-attest...
while they are main reason (in my browsing at least) the "verification pages" happen.
I think they probably realized that maybe they don't want to be known as the reason these pages are showing up everywhere and inconveniencing legitimate traffic.
It's probably true that some VPNs are used for nefarious stuff, but it's also lame that Cloudflare is such an anti-privacy warrior.
It would be interesting to know what percentage of bots are actually nefarious.
The "checking your browser" isn't a default CF thing btw, that's up to the site owner and how paranoid they are (with or without reason). It's annoying me too, but we have sites on CF and practically nobody sees any checks when they access our sites.
(I did write a support request message to Gitlab, but their support clearly sucks. What do I know what kind of subscription my employer has? I don't care! They are paying for me, so Gitlab should offer a modicum of support, if I cannot even log in on their shitty site any longer, because of their changes. But they stonewalled with something like: "We need to know your subscription level blablabla before we can continue the process." kinda automated e-mail. Well, duh! Check your friggin database for my subscription level. Oh but then you would actually have to work. Ah that's a problem of course. Better stonewall a paying (paid for) customer.)
Additionally you can set up a permanent redirect with a browser addon like the Redirector to always be sent from piped.kavin.rocks to piped.video.
Coupled with the hypocrisy of an open web and freedom of speech, it makes CloudFlare arguably one of the worst threats to the web as we know it.
Whereas the freedom of speech ala Cloudflare stops as soon as it can generate cheap PR, because then a website is quickly blocked after a few media reports.. or in case of Piped as soon as the content mafia is complaining.
I doubt that people actually need something like Cloudflare.
It's not about traffic costs, but processing power.
Was at a startup that paid 5k/mo for Cloudfront and moved to Cloudflare and paid just 200/mo. DNS performance improved as we switched over to Cloudflare as well. Saw a decrease in bot traffic. No complaints about usability or being blocked.
So yes, Cloudflare was useful and helped saved $ for us
If this particular instance is them getting DMCA'd then it's not really their fault, but I'm confirmation biasing it with a pattern I see of them making more and more judgement calls about what to host and becoming more like a standard 100% profit-driven megacorp hosting provider.
?????
Those guys... not to mention their pesky "browser verification" which is does not work with noscript/basic (x)html browsers.
They have on multiple occasions had long and public campaigns talking about how important it is to fight censorship in all its forms except a random DMCA troll in Hong Kong?
I don’t think Cloudflare really love “free speech” as much as they pretend in their public messaging.
