I agree the industry would have a hard time surviving off library sales alone, in the same way that most businesses rely on multiple revenue streams to make ends meet, but I think library revenue is much more significant than you're making it out to be!

There's plenty of incentive for most people to buy the real book rather than wait for the queue.

(I've also found libraries a useful way to discover lesser-known authors, since you can quickly sample/browse books on the shelves. But they wont have all of the books published by those unknown authors.... so I end up buying/ordering other things by them)