We're missing a large component of the discussion when we pretend that the content of the letter is the principal issue here. The thing the C-levels are thinking about is liability, because that represents the most direct threat to the company.
Allowing Damore to remain on payroll could be interpreted as a tacit endorsement of his letter, which means in a lawsuit, a complainant can claim that Google has already proven itself to accept illegal anti-woman hiring practices by allowing an employee who espouses these things on company time and with company resources to stick around.
It can further be argued that their failure to address this bias constitutes a hostile workplace, and will greatly strengthen any potential argument that a female Googler was intentionally and/or actively discriminated against either now or in the past.
On the other hand, the consequences of terminating Damore are, essentially, limited to bad press, which is not really a large cost in itself. Google can counteract Damore's complaints with the relevant labor boards by pointing out that they are merely attempting to comply with the law that compels them to create a non-hostile work environment for women.
So why are so many CEOs so quick to jump on these diversity/inclusion bandwagons? Because a lawsuit will cost the company millions of dollars in lawyer time alone, and if they lose, potentially many millions of dollars in damages, especially if it's class action.
The factual validity of Damore's memo is immaterial. All that matters is that Google risks much more money, more aggressive regulatory oversight, and puts itself in peril of other onerous legal sanctions by keeping Damore on board, and by terminating him, they don't.
Anyone who is upset about this should look at the root cause, which is not only the set of laws that may compel such specific behaviors, but also the arcane configuration of the legal system as a whole. It is frequently wielded as a weapon, and that should not be a thing.
IANAL