The Matrix Holiday Update 2023

Hi all,

2023 has been a pivotal year for Matrix, with huge changes landing both organisationally and technically to prepare the protocol for future generations. The ecosystem has once again gone from strength to strength, with active users (based on Synapse opt-in phone-home reporting) doubling across the public network, and more projects building on Matrix than we can count (look out for the “This Year in Matrix” community wrap-up blog post) - and more organisations than we can track joining Matrix for all their secure decentralised communication needs.

On the governance side, we are in an incredibly exciting new era with Josh joining the Matrix.org Foundation as its first ever Managing Director (and employee!), with a mandate to cement sustainable funding for Matrix as an independent foundation, governed by the forthcoming elected open Governance Board. Now, Matrix needs funding more than ever - but rather than turning the entirety of this post into a plea for donations, I’m going to let Josh fly the flag in the coming weeks. Meanwhile, if you want Matrix to keep existing (especially if you’re an organisation who builds on Matrix) please join the Foundation and donate.

On the technical side: the theme of the year has been one of focus: extreme, overdue, focus.

Over the years, it’s fair to say that the core team has tried to strike a balance between building the core foundational technology of Matrix (the spec, a stable server implementation, client SDKs, end-to-end encryption, VoIP, etc)... and long-term forward-looking projects designed to futureproof Matrix (e.g. Account Portability, P2P Matrix, Dendrite, Hydrogen) and/or inspire developers to build on Matrix for more than just chat (e.g. Third Room, Applications Beyond Chat). In retrospect, this was wildly optimistic: we underestimated the amount of remaining work needed to polish the foundational tech to mainstream quality - and despite Matrix uptake going through the roof, this hasn’t translated into sufficient funding to have the luxury to support folks to proactively work on next-gen projects (or foundational projects, for that matter).

So, this year, we’ve ended up focused on one thing: getting the foundational Matrix featureset to better-than-mainstream quality, performance and stability. We’ve dubbed the overall initiative Matrix 2.0, and kicked it off at FOSDEM 2023 with our Matrix 2.0: How we’re making Matrix go vooooom main-stage talk.

The Road to Matrix 2.0

Matrix 2.0 isn’t (yet) an actual versioned release of the Matrix specification - instead, it describes the various foundational projects needed to get quality, performance and stability up to and beyond that of today’s mainstream messaging apps. These projects are:

• Sliding Sync (MSC3575): the ability to instantly log in, launch and sync Matrix clients no matter how large or busy the account.
• Native E2EE Group VoIP (MSC3898 + TBA): scalable video and voice conferencing and calling built natively on Matrix and so benefiting from Matrix’s end-to-end encryption.
• Native OIDC (MSC3861): replacing Matrix’s historical authentication mechanisms with industry-standard Open ID Connect (giving us two factor authentication, multi-factor auth, passkeys, and radically simplifying auth implementations for both client and server developers).
• Faster Remote Room Joins (MSC3902): letting servers rapidly join rooms on other servers by incrementally participating in the room.

Over the course of the year Matrix 2.0 has gone from the initial demo on stage at FOSDEM to concrete implementations which users can play with today as announced in our Matrix 2.0: The Future of Matrix post in September. Since then, we’ve been busy polishing away. On Sliding Sync, the proxy has pretty much stabilised - although the protocol itself can and should be simplified before we think seriously about native implementations (in practice, having the server track room list ordering gets very fiddly when only clients can really determine the final ordering, due to E2EE). Element X and matrix-rust-sdk has been the main implementation driving forwards Sliding Sync and much of the other Matrix 2.0 work, for those itching to play with it.

On Native Group VoIP: we’ve gone through many iterations over the year - starting off with Full Mesh calling (good for ~7 users per call); then switching to the experimental waterfall Selective Forwarding Unit (SFU) to provide scalable but not-E2EE conferencing; and then switching to a hybrid solution using LiveKit to provide an E2EE-capable scalable SFU, but with the signalling and encryption all handled by Matrix. Element Call is the main implementation driving forwards the underlying Matrix work here, and Element Call Beta 3 showed off the new LiveKit based implementation in July - which was then integrated with Element X complete with end-to-end encryption in November. There’s still some polishing remaining here, with a new layout engine in the wings for Element Call, and enabling full encrypted-per-sender conferencing by default in both Element Web and Element X, but it really feels like the hardest work is behind us now: the core team has been successfully doing all of its collaboration on Element Call for months now, like so:

On Open ID Connect: things are also shaping up well. This will be the first time that we’ve replaced a large chunk of the Matrix spec with something else, and in order to manage your account in Matrix 2.0-native clients like Element X homeserver admins will need to migrate their authentication to the new OIDC World using matrix-authentication-service (MAS). There’s a great blog post from September which explains what this will entail - and since then, we even have the beginnings of syn2mas: a migration script to migrate from Synapse-managed accounts to MAS-managed accounts (warning: still experimental). The Matrix.org homeserver hasn’t been migrated yet (as we need to support social login first), but an increasing number of standalone Matrix servers are going OIDC-native, so arguably the migration has already begun! We’ll keep https://areweoidcyet.com updated as the project progresses.

Finally, the core of Faster Remote Room Joins (FRRJ) shipped in Synapse back in February. There’s still some major speedups that FRRJ could unlock, but the other tracks of Matrix 2.0 have been taking priority.

So: Matrix 2.0 is palpably on the horizon - all that remains is polish on the example clients (Element X & Element Call), full support for migrating to OIDC, and landing the MSCs into the spec. For instance, Element X just added read receipts and (early) E2EE backup support in the last few days - the gap is closing! It’s worth noting that significant amounts of this work has been funded by BWI for BwMessenger and BundesMessenger: huge thanks to BWI for supporting core Matrix development by contracting Element.

Levelling up on Encryption

Encryption stability received a huge amount of attention this year. It turns out that reliable end-to-end encryption is surprisingly tricky in a decentralised environment, and historically we’ve been playing on hard mode by implementing three entirely separate implementations of the Matrix layer of encryption between matrix-js-sdk, matrix-ios-sdk and matrix-android-sdk2, each with their own bugs - more than tripling the costs of development, audits, and maintenance by the Foundation (quite ignoring the independent implementations from the community in mtxclient, libquotient, matrix-dart-sdk, trixnity etc).

So a huge project has been underway to converge on a single auditable codebase for the core team’s E2EE implementation so that any bugs or future features can be resolved in a single place. That codebase is matrix-rust-sdk’s matrix-sdk-crypto crate (and our underlying vodozemac double ratchet implementation) - and we’re proud to say that we are using it for encryption in matrix-rust-sdk itself (as showcased by Element X and Fractal 5), matrix-ios-sdk and matrix-android-sdk2 (as used in the old Element iOS & Android apps), and have now merged it in matrix-js-sdk too (available for new logins on develop.element.io). The process of rustifying the encryption in Element Web and the old iOS & Android apps has been nicknamed “Element R”.

The process of switching matrix-js-sdk to use Rust encryption has been particularly gruelling, requiring compiling matrix-sdk-crypto down to WASM as matrix-rust-sdk-crypto-wasm and then doing heart surgery to replace the old JS crypto implementation… while also needing to extensively loop from WASM back into the browser to use IndexedDB for storage, all while outperforming the old implementation. It’s tantalisingly close now: while develop.element.io has it turned on by default for new logins, there are still a few remaining performance edge cases to be chased down related to online backup before we migrate everyone to it. The remaining blocking issues can be found on GitHub for those interested in tracking progress.

matrix-crypto-sdk is already manifestly more reliable than the old implementations (in terms of the chances of hitting infamous Unable To Decrypt errors) - and now that we are so close to converging on it everywhere, the race is on to ensure that any remaining defects get flushed out for once and for all. One of the new initiatives here is called complement-crypto - a full end-to-end torture testing suite specifically for matrix-crypto-sdk. You can read all about it in the announcement post a few weeks ago, but suffice it to say it’s a super exciting project which stress-tests both matrix-rust-sdk and matrix-js-sdk (with the new rust crypto implementation) against federated Synapse containers in order to test E2EE under the most horrible failure modes imaginable. It’s already picked up some elusive bugs which have plagued us for literally years, and it looks set to be the main framework by which we will hunt down and kill any remaining issues. See the Test hitlist for the full scope we’re targeting.

Now that everyone’s (almost) converged on matrix-sdk-crypto, the next big project for the Crypto Team is going to be improving the E2EE usability (at last!). The big news here is that we’re shifting to Trust On First Use (TOFU) for user trust. Specifically: this means that we will only encrypt messages to devices whose owner has explicitly cross-signed them (essentially trusting the owner by default). You will still be able to explicitly verify that other users are not being impersonated (via QR scan or emoji comparison), but this should improve the default behaviour to be much more secure. Alongside TOFU will come other radical simplifications of the E2EE UX (both around login, self-verification, cross-verification and backup), so watch this space: the game is afoot to finally fix Matrix’s E2EE usability, now we can make all the changes in one place!

Finally, work continues to progress at matrix-dmls on supporting a decentralised dialect of Messaging Layer Security (MLS, RFC9420) on top of Matrix as an alternative to our normal Olm/Megolm encryption, with recent work focused on making it play nice with matrix-sdk-crypto. https://arewemlsyet.com is the place to track updates (although it’s a bit overdue for an update).

In other news

Faced with limited funding and the decision to focus exclusively on stability, reliability and performance, there have inevitably been some major changes impacting the core team.

One of the biggest changes is that Element (the company formed by the core Matrix team back in 2017 to try to fund our work on Matrix) can no longer financially afford to donate its work on Synapse and other server components to the Matrix Foundation under the permissive Apache licence. Instead, Element is continuing development under the copyleft AGPLv3 licence at github.com/element-hq/synapse going forwards. This is to let Element sell AGPL exceptions to commercial Matrix vendors in order to fund their underlying Matrix development: you can read more about it at Element’s announcement - or you can listen to this week’s Matrix Live for a firsthand explanation:

The other major change is that we’ve had no choice but pause development on the majority of the core team’s next-generation Matrix projects. We had high hopes of being able to secure dedicated funding for Third Room (especially after the awesome Tech Preview 2: Creator Update in June), but the interested parties did not come through, and the team has now disbanded. Meanwhile, P2P Matrix and Low Bandwidth Matrix is on hiatus until there’s dedicated funding - and Account Portability work is also temporarily paused in favour of commercial Element work, despite the fantastic progress made recently with Pseudo IDs (MSC4014) and Cryptographic identifiers (MSC4080). Given P2P Matrix and Account Portability were the main projects driving Dendrite development recently, this may also cause a slow-down in Dendrite development, although Dendrite itself will still be maintained.

Needless to say, this is far from an ideal situation: we sent up distress flares loud and clear at the beginning of last year’s holiday update; and we’ve now had to shrink to focus exclusively on the core projects. However, we’re optimistic that the tighter focus in the medium term will help us get back to the point where we can resume the longer-term projects - assuming that organisations (and individuals) dependent on Matrix sign up to support the project.

Conclusion

Despite the downsides of 2023, right now we’re feeling distinctly optimistic: Matrix 2.0 clients like Element X already outperform the best proprietary mainstream options by many metrics - and focusing purely on improving the foundations is only going to improve that. We may not have taken the most direct route to get to where we are today, but it genuinely feels like 2024 will be the year where Matrix overtakes the incumbents.

Talking of which, there’s just one last thing to mention: amidst the economic challenges of 2023, one future-facing core team Matrix project has survived: our work around the EU Digital Markets Act (DMA). The DMA is leading antitrust legislation from the European Commission, which aims to stop big centralised tech companies (so called ‘gatekeepers’) from carelessly suppressing innovation, competition and consumer choice by trapping users inside their walled gardens.

Needless to say, we’ve been tracking the DMA closely throughout its gestation, and we’re now in the final sprint: in March 2024, messaging services which have been identified as gatekeepers will have to open their networks to allow interoperability with requesting messaging services (while preserving E2EE, if they’re encrypted). So far, only WhatsApp and Facebook Messenger have been identified as gatekeeper services (Apple is doing everything they can to wriggle out of it). However, it looks like WhatsApp is taking it seriously, which could prove very interesting indeed.

As Matrix, we’ve taken a two-pronged approach: on one side, showcasing how Matrix as it stands today can already bridge existing chat systems together as a highest-common-denominator protocol (including preserving E2EE, if they happen to already use the Double Ratchet). On the other side, we’ve also been contributing significantly to MIMI, the IETF Working Group dedicated to standardising a lowest-common-denominator protocol specifically for DMA interoperability.

2023 has involved a lot of work on MIMI, participating in the Design Team alongside Phoenix, Wire, Cisco, Google and Wickr - and at IETF 118 in Prague in November we collectively proposed the first draft of the protocol (see also the architecture presentation for context). MIMI has ended up taking some inspiration from Matrix (and Linearized Matrix, a simplified dialect we proposed which does away with full-mesh decentralisation), while focusing very tightly on the specific problems of server-to-server interop between existing communication services - leaning on MLS (where available) for synchronising state across the services, while also leaving the door open for using the Double Ratchet to provide an on-ramp for pragmatic bridging to today’s services (including Matrix!).

MIMI’s tight focus means that it doesn’t currently provide conversation history, arbitrary state events, or decentralised conversations - and is focused purely on server-server communication. However, there’s certainly a world where Matrix could evolve to be compatible with MIMI - such a hypothetical Matrix 3.0 would effectively layer Matrix’s richer existing semantics (decentralised conversations, extensible state events, state resolution, group VoIP etc) on top of MIMI’s subset of functionality. It’ll be interesting to see how this plays out. Meanwhile, we’ll continue to provide both Matrix for pragmatic DMA interoperability for today - and participate in MIMI for IETF-track interoperability.

Anyhow: hopefully it’s clear that 2024 is going to be a super interesting year for Matrix - whether that’s simply by nailing Matrix 2.0, or whatever excitements DMA interoperability brings. And if you’re relying on Matrix: please donate.

Meanwhile - have a fantastic end of the year; and thanks once again for flying Matrix.

- Matthew, Amandine, Josh & the whole team.