OtmaneBenazzou

Hi, I’m a pretty experienced scraper/automator and I was thinking about sharing my knowledge. I originally thought about writing a blog, but I'm not sure that would be the best format to transmit everything I know. Maybe I'll do both.

There would first be a quick introduction to the legacy ways of scraping(i.e GET request+parser), why they’re often not enough today and when can they be still used.

Then I’d delve into more advanced techniques, I usually try to find the cheapest/free solutions to the problems I’m solving but I’ll also compare paid options:

-how to make your scrapers/bots as stealthy as possible

-everything you need to know about fingerprinting

-how to bypass datadome, cloudfare, captchas…

-how to scale to millions of requests

-how to make your own free rotating proxies

-how to do automation on mobile apps and desktops

-how to reverse engineer APIs on the web or mobile

-every tips and tricks I know to scrape better

I may add a section on GPT/LLM, either about how to fasten the development of a scrapers or about how to make self healing scrapers.

Is this a topic that would interest a lot of you?

Would some of you be ready to pay a small amount for an ebook/courses on those subjects?

Thanks

Hi, due to an (yet) unidentified bug on our Firebase storage/Google Cloud infra/buckets, we've lost hundreds of files in the last few days(it was spotted just a few hours ago).

There is no backups of all the files(I know...I did not decide that)

I'd guess Google would probably be legally required to not wipe everything instantly for legal/forensics purposes, right? So is there any way to somehow recover those files?

Does any Googler or GCP user here knows if there is any kind of way of recovering those files? They are pretty critical to our core product.

Thanks in advance

I've noticed that it's much easier to make stealthy bots while using virtual android phones. I've made bots for tons of apps without getting blocked(when I was getting blocked on the PC website).

Would anybody be interested by a platform (kinda like Phantombuster) but for mobile?

You could automate any task on one or multiple virtual android phones.

You could easily click on the right buttons using OCR(character recognition) and extract data.

On almost any apps.

It would cost maybe 50 cents an hour by phone.