I don't know if this applies to VLC, too, but raw audio output (or something that isn't PulseAudio) is necessary wherever low (or at least constant) latency is required.
Edit: oh - now that I look at your username, I believe congratulations are in order re. the subject of this thread :)?
That said, I don't think a video player needs low latencies. It can generally have pretty large buffers.
The entire world of music production uses JACK, which got the design right well before PulseAudio even existed. It can provide synchronous operation to prevent drifting between audio tools. I include mplayer2 as one of those tools for s few things. More importantly, JACK guarantees a reliable very, very low latency audio path.
How "low latency" do I mean? Consider that in printed music, while somewhat uncommon, quite a few pieces have been written that use 128th notes. That would be anywhere from ~80ms to ~10ms note lengths, depending on the tempo. Larger latencies mean notes being heard as a different note from what was played, and I keep JACK down to about 5ms latency, and would set it lower if I could afford new hardware. Notes are not the only feature of musinc, and some headroom is needed. Any latencies higher than about 10ms-20ms I would consider broken and unusable.
(Incidentally - I cannot play GuitarHero/RockBand on most TVs either because of horrible latench. A (60Hz) video, a two frame delay is a "miss" on the harder songs. those 16.6ms latencies are nasty regardless which direction the latency is being added)
Beyond that, JACK is a great API. It makes it trivial to do a LOT of the common needs (i.e. "Just give me the audio samples..." or "just let me write new audio data and not have to worry about anything else... such as timing")
Of course, there are cases where you do need lower latencies, such as games where interactive feedback cause the sound to change, or when you pause a movie. In those cases you adjust the latency (possibly temporarily in the case of the pause).
That said, PulseAudio has a different set of requirements and thus design decisions from Jack. It is not meant to be used in music production, but then again, I doubt the music producers will use some sandboxed app to do music production either. So, I don't see the problem here.
I don't know what PulseAudio considers "very low latency" and there aren't any numbers on that page to indicate it, but in my experience, PA's best latency possibilities are about an order of magnitude away from being useful for any kind of hardcore real-time processing. This isn't a problem as far as PA is concerned since it's not what it was conceived for, but it won't cut it for applications which need that kind of capability.
PA also doesn't really provide latency guarantees, either. In fact, it's mentioned on that page:
> So in summary: tell PA what latency you want, but program defensively so that you can deal with getting both lower or higher measured latencies.
I don't know if that has improved though. The last time I had to poke PulseAudio was a few months before that page's "Last Edited" timestamp.
Edit: For what it's worth: I never managed to get PA to consistently keep latency below 25-30 ms, which is at least 10-15 ms away from being adequate for real-time audio processing. I guess it is a cornercase though, and probably outside the Gnome project's interests. Reading the other comments in this thread, I assume VLC's problems with PA aren't related to latency.
The only place where low-latency matters is real-time encoding situations. For example, video conferencing, composing music, streaming things live, etc but even in those situations "low" is a relative term. Low-latency video conferencing is anything under 200-500ms (depending on who you ask). Low-latency music composition is <20ms (again, depending on who you ask).
