With today's browser implementations of WebRTC, you can proxy through a TURN server while still maintaining end-to-end encryption, but you can't proxy through any kind of customized endpoint/server, because each endpoint necessarily has the encryption keys as part of the session negotiation.
Chrome implements experimental user-space media stream processing APIs that allows you to build "end-to-end encryption" at the javascript level. But, to me at least, it's a bit hand-wavy to call that "end-to-end encryption" because the keys are created, managed, and accessible from user-space. And neither Safari nor Chrome yet support these APIs.
There's ongoing work on this: https://datatracker.ietf.org/wg/perc/documents/
