For these specific purposes, the JMAP working group at IETF is the place to start, because it’s the hub where this stuff is being incubated (though some of the work happens in other working groups). See <https://datatracker.ietf.org/wg/jmap/documents/>; the drafts JMAP for Calendars (integrating JSCalendar, published as RFC 8984 six months ago by the same group of people through the calext WG) and JSContact are particularly relevant here.

Like Matrix, JMAP is commonly misunderstood, with people thinking it’s about email (it’s replacing IMAP, which is Internet Mail Access Protocol, right?), but it’s actually fundamentally an object synchronisation protocol, with the core in RFC 8620 being domain-neutral, and the email model completely separate in RFC 8621. (IMAP has also shifted a bit over time from being about mail with proper synchronisation being an unsound and secondary concern, to nailing down sound object synchronisation in extensions. But JMAP does it better.)

Matrix is about decentralisation while JMAP is client-server, but you can probably reuse much or all of the underlying data models, just as JMAP for Calendars and JMAP for Contacts are establishing JMAP-independent data models models first.