(1) The ability to add context via a local apps integration into OS level resources is big. With Claude, eg, I hit Option-SPC which brings up a prompt bar. From there, taking a screenshot that will get sent my prompt is as simple as dragging a bounding box. This is great. Beyond that, I can add my own MCP connectors and give my desktop app direct access to relevant context in a way that doesn't work via web UI. It may also be inconvenient to give context to a web UI in some case where, eg, I may have a folder of PDFs I want it to be able to reference.
(2) Its own icon that I can CMD-TAB to is so much nicer. Maybe that works with a PWA? Not really sure.
(3) Even if I can't use an LLM when offline, having access to my chats for context has been repeatedly valuable to me.
I haven't looked at provider-agnostic apps and, TBH, would be wary of them.