story
I don't see how collapsing them all to a linear discussion would help this: you simply get one massive topic overtaking the entire conversation, making it impossible to even have the second discussion at all... if the content doesn't get posted because it is off-topic in the linear model, no amount of scrolling will find it.
Again, though, this is why almost no sites are actually a linear model; the closest you seem to get to a linear community currently is in the medium of real-time chat, where you still have servers divided into channels, but the channels are relatively fixed and difficult to browse.
The result is that you have communities existing within a single server/channel; as an example, there is a community which I started on an IRC server (not iPhone hacking, btw: a bunch of random friends and acquaintances that has grown and shrunk at various points over time, mostly concentrated with people form the small college that I attended) that has existed for years with a single linear thread.
Real-time chat, though, is a drastically different kind of medium: handling multiple real-time conversations is simply hard in a way that having multiple asynchronous conversations is not. I maintain, then, that this is why almost no discussion community has ever had a truly linear model: content tends to fall into 4-5 levels of hierarchy, with at least two of them being under direct user control.
Once you expand the scope of where you look for the tree in this way, you then note that StackOverflow is actually more threaded than classic discussion forums: of the five levels it has (sites, tags--which I will again explicitly point out is just another hierarchy level where the entries in lower levels might be seen in multiple places--, questions, answers, comments), four of them are under immediate user control, and the fifth is decided by a largely democratic process.
In practice, Hacker News threads sometimes get really deep (and I again agree that they can get confusing down there), but the very top-level is sufficiently shallow that you need some of that depth below to have any interesting conversations: if there were only one linear conversation per posted link, things would rapidly get really boring when only one of the politics/economics/etc. angles was able to be covered at a time (by which point the others would have missed their window).
Hacker News, does, though, actually have a form of thread limiting; I didn't go into it in detail in my earlier post, but I'll go ahead and do so now: as you get deeper into a thread, the "reply" button takes longer and longer to appear. This mechanism is actually defeat-able (which is funny, and some might argue a bug: you click "link", and on the comment's own page you can always reply immediately), but it still does help slow the rate at which a thread can get very deep.