It's not an appeal to authority, it's an appeal to good ideas. Dyson's points are valid. If you dismiss them without even thinking deeply about them, you're pretty much doing what the Inquisition did to Galileo.
Moreover, Dyson's points are based on good Science. He focuses on measurements and data, which are hard evidence. Dyson does not appeal to emotion or fear.