n=1, I have been in tech for 25+ years, and a recruiter has always been my preferred entry into an org when I don't have a network connection. Our incentives are aligned; I want the work, they get paid if I get hired and stay. Their sales commission depends on me succeeding. Without a recruiter, a company is trying to hire the best candidate at the lowest comp offered possible. The greater rate at which workers change jobs for better comp, the more likely comp is to go up (this is why companies pulled remote work and are trying to create geographic stickiness for jobs in the US, to slow wage gains and reduce labor mobility). I would suggest reconsidering your view on recruiters. Some suck, some are worth their weight in gold. If the job turns out to be suboptimal, do your best to find out before you take the role, or live your life in a way you can bail for the next job without much hassle.

When you have success with recruiters, connect and keep in touch with them. A career is long, and its good to have options, as you never know when you'll need them. Optimize for optionality in this context.

Source: I've been a programmer for 25 years, and ran a recruitment company for 8.

This happens, but it's unusual. It's normally only really something you'd bother with as a recruiter if you were doing CV marketing, that is, reaching out to people who aren't your clients saying "this is the kind of person I could get you!". They're not really meant to do it, but recruitment regulations aren't strongly enforced in most of the Anglosphere.

To fill a role with one of your clients, they've signed T&Cs that mean they can't really cut you out, and assuming they don't hate you they also don't want to lose you as a recruiter. Fucking candidates absolutely will try and occasionally cut you out of the process -- usually out of incorrectly thinking it will help them land the job because the employer won't have to pay a commission.

There are many shitty recruiters, but finding a good one will absolutely help you find good roles, and can do all sorts of useful things like make sure you're asking for enough money, get feedback that you wouldn't directly get as a candidate, harass the hiring manager about your application on your behalf, and engage in a dialogue with the hiring manager about your application that virtually no hiring manager would be willing to do directly with you.

Not everyone is Linus Torvalds, and companies generally want to hire mid-level players too - they’re just somehow unable to even email them back

There is a certain amount of job interviewing that people do to gather intelligence. I've went to numerous job interviews where I was trying to find out what was going on and not particularly interested in changing jobs. Companies sure do interview people for the same reason.