There's also a ton of opinion and taste involved in software engineering, which can bias the interviewer's view of someone's technical ability. Not to mention all the unconscious biases against people who don't "fit the description" of a typical software engineer demographically.

The Googles of the world limit their own potential by rejecting many talented people who couldn't make it through their hiring filter for reasons that have nothing to do with their actual ability to build software. Then they express angst over how difficult it is to fill positions. It's a self-inflicted problem, and a hard one to solve. It's probably impossible to design a theoretically perfectly fair hiring process without also solving all causes of inequality in society generally. But it can be made better and more fair than it is, and many companies and applicants aren't satisfied with the existing hiring processes, so they are making efforts to change the situation, as they should.