Admittedly, the one reference he made to parties at the beginning was unnecessary (somehow, I hadn't even heard it). But he isn't calling out any party; he is calling out a particular
policy, and the way of thinking that pre-dates it, that he feels is counter-productive. There is no way to criticize the ideas he does without being "political".
To put the shoe on the other foot: let's say someone had data which demonstrated that welfare and affirmative action are bad for the people they purport to help (just an example, I'm not making a case for or against either). How could one make that argument in a neutral, non-partisan way, when those issues have been politicized to death for decades?
You couldn't. You'd have to either state your case bluntly and take the heat, or shut up and go away. Though the talk could definitely have been better (more data, please), I'm glad the speaker chose the former, and I think he made the best attempt possible at being neutral.