undefined | Better HN

0 pointstheshrike792mo ago0 comments

I had to download file attachments from a specific niche web forum that's behind login. I could've went looking for a browser plugin or a 3rd party tool to do it.

Once again, it took me about an hour while watching my shows to get a custom one made.

The first version operated by me downloading the pages one by one to a directory, the Python app parsed the html, downloaded the files and renamed according to thread name.

After a few iterations the tool just grabs a cookies.txt file exported from Firefox and can take any thread URL, browse through it, skipping existing files and determining if everything is already downloaded

I could easily have it just watch a set of threads for new content and download automatically, but the current system is fine =)

0 comments

chasd002mo ago

That’s pretty great, I’ve done something similar by hand many moons ago. It was very tedious.

I need a simple S3 compatible API to store some files with basic auth and ssl certs using let’s encrypt. Nothing crazy, Garage is overkill, Minio is overkill. I may see if Claude code can handle that for me using python or something.

/btw, I work in consulting and the above project would have a budget of probably $100k and a schedule of 3 months. I see a lot of change for swe consultants coming.

jppope2mo ago

As the saying goes: "for us to be replaceable, clients would need to know what they want and how to articulate it... we're safe"

j / k navigate · click thread line to collapse