The accusation is not that he sympathized with the Nazis. Quite the contrary, the accusation is that he has no sympathy for anyone. The accusation is that his worldview, was (and is) that he is responsible only for himself and he has no effect on affairs because if he didn't do [whatever] someone else would. He claims, on essence, that the events of the world is an unchangeable script, and the only control we have is which role we play. For an expert in markets, he claims stunning ignorance of the concept of marginal effects, and a non-stunning awareness of how to profit from amorality.