Your description is very simplistic, but yes, you have to trust the code that's delivered to you. For example, no Android/iOS user would check every single update to E2EE apps they install for backdoors. However with web, there's an opportunity for a backdoor in every single request and the server can ship different code to different users. In my opinion, using web cryptography is still worth it, but it's definitely more risky than native apps.