"Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances."
They could have asked him to leave. Then demanded he leave. Then arrested him if he refused to leave. That would have been legal.
They removed him because they don't allow you to petition the government, you know, within earshot.
http://www.huffingtonpost.com/rob-kall/former-cia-agent-ray-...
http://www.youtube.com/watch_popup?v=N-Vy8fFnz18#t=0m20s
If I am understanding this correctly, all this guy did was stay standing with his back to the speaker.
That's some insane abuse if true.