But let's think about it: The actors you're talking about who have the agent on the lowest rung are still professional actors, i.e. they do acting to earn a living and nothing else.

However that set of actors is a small sub-set of those actors who get a role every now and then but don't earn a full time living. So there is an entire class of actors who get an industrial spot, do summer stock, play an extra or do voice overs who don't have an agent.

I know this because I've hired those actors for various projects. However if those actors had a name that you could recognize they'd have an agent.