Regarding spam, there is some (seemingly inevitable in any community involving humans). But frankly Mastodon, right from the beginning, focused heavily on Moderation tools. Figuratively speaking, no expense has been spared to make it easy and convenient to block bad actors (or instances) from your account (or your whole instance).
With USENET people expected to see every posts in the newsgroups they were subscribed to, and a lot of people would complain loudly if anything was missing. With a network like this nobody expects to read everything. At the same time receiving everything is easier (still) - if you federate with a couple major endpoints you get most stuff. There will be challenges with this as the network grows but I spent too much time ensuring we got messages from newsgroup X within Y hours (less was unfeasible, as there were still sites exchanging on a schedule via dialup).
With respect to spam, once you accept you're not likely to see everything, things get a lot simpler and you can apply a lot more aggressive filtering.
Personally I'm working on my own ActivityPub implementation, and one of the things I've used on Twitter in the past very effectively via the Twitter API was a simple Bayesian network used to rank things to surface interesting stuff, but lots of room to apply more sophisticated machine learning there too.
Maybe there could be a variant of NNTP which allowed some amount of advertisement posts in between. Then let the maintainers keep the proceeds from those?