The problem is that Google is engaged in a war with SEO optimizers, and losing (or at least, barely breaking even). No business model will fix that. It's a hard problem that any successful search engine will struggle with.
For many search terms, the entire above-the-fold and sometimes multiple scroll heights are filled with:
- Ads
- Info cards
- People Also Ask
- Map/Business info
- Videos
- Related Products
- More Ads
The organic search results start after that and on mobile they may as well not exist.
If everyone is clicking the directly or indirectly sponsored/algorithmic content then what incentive does Google have to give you good, straightforward links to relevant webpages?
So why would Google serve the more direct and helpful result from the edu domain!
I think they are genuinely struggling in an adversarial environment. Imagine every hacker in the world specifically focused on your system and the payoff is millions or even billions of dollars.
A typical employee at any large company is incentivized to:
- Maximize their salary
- Maximize their future earnings potential (e.g. CV)
- Have fun
- Have work-life balance
... and so on. That goes all the way up to the CEO. Pretending a corporation thinks and responds to incentives is folly. Running a corporation like Google is like herding cats. If everyone marched in the same direction, Google would cut through SEO optimizers like a chainsaw through butter. There is no wayh to do that with 100+k people.
Google fundamentally has no financial incentive to fix any problem, since organizations aren't intelligent, sentient beings which act on incentives. They have dynamics from the interactions of 100k+ individual agents.
There is no magical fairy which makes large organizations act in their own best interests or even try to survive. The only reason they do survive is that's as true of their competitors as themselves.
Corporations are intelligent, sentient entities which act on incentives. They don't have a conscious sense of self, but then neither do many natural organisms. That doesn't keep them from responding to their environment as if they're aware of it, and acting in incentive-driven ways.
Corporations seeking to maximise profits are exactly like humans attempting to maximise income. The problem is that corporations - like humans - have the wrong incentives, further diminished by limited predictive ability and poor heuristics.
So maladaptive behaviours emerge and stick. Both humans and corporations find a nice local minimum and don't move from it.
More intelligent behaviour would be able to predict and avoid existential threats without having to experience them first. But it's a question of predictive ability, not a binary absence of all intelligence.
So it's not that Google can't do better because it has too many employees. It can't do better because its cultural heuristics - which include it internal culture and the external culture it operates in - don't allow it to.
I can set up "incentives" for an anthill, in the form of a mixture of sugar and borax. Emergent behavior takes over.
The behavior of corporations, at the scale of Google, Facebook, or similar, is almost entirely emergent dynamics. Same thing for countries, for that matter.
