I'd actually be pretty happy if browsers chose to implement those APIs with the same restriction in mind - that is, only for explicitly installed PWAs. I said this elsewhere in the past, and I still think that's a reasonable restriction that could provide a path forward.
> or mitigating one of the two critical problems of WebUSB (that the computer trusts peripherals too much, so that one that’s hijacked can more easily become a remote code execution vulnerability)
I'm not very well-versed in the details, but I believe that's also the reason why WebUSB (or Chromium implementation of WebUSB?) doesn't allow certain classes of devices to be ever accessed via that API.