Add to that the fact that the pope has a cultural influence that goes further than only the catholic audience (lots of Protestant see the pope as important even if that’s not something dictated by Protestantism, a bunch of not really religious people see him as a sort of spiritual leader, etc)
Heck, it's a struggle to convince many of them that Catholics are Christian at all, and "the Pope is the antichrist" used to be a normal, mainstream comment in American newspapers.
It is somewhat a piece of irony that the Pope generally holds a more favorable reputation in the minds of the irreligious in America than the religious. Even the average Catholic likely does not have as much respect for the Pope as some of the commenters here.
