Don't worry.
I don't mean we should allow the fetch API to mess with the browsers trust configuration. It should only allow a temporary override of trust rules, similar to DANE TLSA-RRs, but provided by JavaScript instead of DNSSEC-verified DNS lookups.
Imagine e.g. combining this with an SPA bootloader contained in a data-url (like a bookmarklet), which the user scans via a QR-code or receives via text-based messaging.
CORS would still be in-play, and maybe the insecure nature of the caller is communicated to the API.
The benefit of this pinning would be e.g. allowing direct communication with IoT hardware, or even just prevention passive content analysis.
You could talk to IPs directly and still use TLS without weird wildcards like *.deviceid.servicedevices.com where the dns just has these zone entries:
deviceid.servicedevices.com DNAME has-a.name
, but that's ugly and leaks the device's IP through a DNS lookup.