There is no such thing as "no PM". Such a concept can't exist (for the same reason that "no developers" can't exist). There's actual work there, that someone has to do, or nothing gets built.
Now, it's totally fine if you say "I think it's best if all of our team members spend 75% of their time as developers, and 25% of their time as project managers". That's what you've just described above, and (in my opinion) there's nothing inherently wrong with that.
But your misleading yourself if you claim "PM's aren't needed because Agile", or "PM's aren't needed because the one's I've met 'silo' us off". All you've done is shift the PM burden from a specific person, to some/all of your developers. Your developers are now actually "part-time developers, part-time PM's"
There's also some weird assumptions that your making around PM's. PM's do not have to silo developers away from the client (and good ones don't). PM's do not have to estimate for devs (good ones always use team estimates).
I've seen developers write buggy code, or brittle tests. But I would never say "developers aren't needed". You've clearly encountered bad PM's, and that's very unfortunate. But that doesn't mean "PM isn't needed", and as you yourself just described, you already have to pick up that work on your own since you lack one on your team.