The initial WebUSB proposal included the notion of "allowed origins", but it was eventually removed in favor of user freedom.
The device can provide a landing page URL for the browser to show when the device is plugged in, but otherwise any site meeting the secure origin requirements and that has user permission can access a USB device, same as any other website.