> The server side is still proprietary
Open sourcing it would make no difference. Signal's server is open source, yet the sources are always released late. For a whole year, Signal was running a totally different server code than the one they had made public, they even injected some crypto stuff and not a single person knew what the server was running.
This is the nature of servers. Backend is always unverifiable, even if it's got the latest code available to the public. The only thing open source backend is useful for is self-hosting, not verification.