The argument that Schopenhauer is espousing a fantasy probably does have merits, and we've the benefit of another century and a quarter of publishing (the novel really only emerged into mass culture during the 19th century). There's the challenge of recognising greatness as it first appears (it almost always takes at least some time for that awareness to dawn), and the sheer arbitraryness of assessments as well.

That said ...

We're still left with the fact that what financial compensation promotes is at the very least appeal to a minimum common standard. That lesson has emerged again and again in the history of mass-media, beginning with street carnivals and players, the penny press (both newspapers and "penny-dreadful" fiction), the mass media of radio and television (Murrow's "Wires and Lights in a Box" and Minow's "Vast Wasteland"), and the Web, mobile media, Reddit, and Facebook (once Literally Harvard, now ... not so much).

In the case of online content and services, it seems that true gems are virtually always underfunded. (The constant gripes aimed at Wikipedia's apparent mostly sufficient endowment are a rare exception.) Sites such as LWN eke out an existance, Linux Journal ultimately folded --- for all that adtech supposedly pays for the Internet, it certainly failed there.

Looking at collections of great books, what strikes me is how many of them predate not only recent history (say, the past 50 years), but all of modernity. How much of this is measurement bias and a varianty of the Lindy Effect, and how much is a well-placed assessment on whatever truth there may be in merits is of course very hard to say. But it's quite persistent.

As I read through works (fiction and non), what I'm struck by is how little of what is recent is truly novel. The refrain from Ecclesiastes, that there is nothing new under the sun, isn't entirely accurate, but it's far more so than it has any right to be. I suspect it's a combination of pressures to publish and an ignorance (often cultivated through deliberate presentism and deprecation) of earlier literature that leads to this.

On your "difficulty* point: part of the cause is also the haste and rush to publish leading to just plain sloppy work. That's not entirely new, and the practice can even be an art form (Kerouak's On the Road). But far too many leading works --- bestsellers and the like --- are riddled with poor editing, rambling structure, typings and misspellings, and poorly-checked facts. There are of course exceptions, but again it seems that the pressure to publish and transact leads to poor results.

(Self-published works can of course exhibit this to a far greater extent, but they're also produced under profit pressures, and with far fewer available resources than traditionally-published works, for the most part.)

I suspect that underlying this discussion are two questions that haven't been asked yet, so I'll ask them:

1. What makes a work "great"?

2. What are the circumstances in which such works emerge?

If you could provide any examples of "recent" (I'll give you anything published since Schopenhauer wrote, so 1891, though more recent would be more compelling) books meeting both the "great" and "not commercially motivated" criteria, either fiction or not, I'd be interested in seeing what you come up with.

(Others can contribute as well.)

... I actually think that would be more interesting than continuing the debate above. I think your argument has some merits though I'm not fully convinced.