Matrix Live

Continue reading…

Matrix Live

Dept of Status of Matrix 🌡️

Thib announces

Let me start right away with no less than a sneak peek at Matrix 2.0. Matthew walks us through the most exciting MSCs currently in flight with working implementations, that will be drastically changing the experience on Matrix.

One of those MSCs makes Matrix support OIDC. But Synapse already supports OIDC, right? If you want to learn more about the difference between OIDC and OIDC (sic) and how it's going to impact you, the auth team and I wrote a post for you.

Continue reading…

TL;DR: If you want to play with a shiny new Matrix 2.0 client, head over to Element X.

Matrix has been going for over 9 years now, providing an open standard for secure, decentralised communication for the open Web - and it’s been quite the journey to get to where we are today. Right now, according to Synapse’s opt-in usage reporting, in total there are 111,873,374 matrix IDs on the public network, spanning 17,289,201 rooms, spread over 64,256 servers. This is just scratching the surface, given we estimate that 66% of servers in the public network don’t report stats, and there are many enormous private networks of servers too. We’ve come a long way from creating Matrix HQ as the first ever room on today’s public network, back on Aug 13th 2014 :)

Meanwhile, the Matrix ecosystem has continued to grow unbelievably - with huge numbers of independent clients, bots and bridges maturing into ecosystems of their own, whole new companies forming around the protocol, and organisations ranging from open source projects to governments, NGOs and Fortune 100 companies adopting Matrix as a way to run their own secure, decentralised, standards-based self-sovereign communication.

The world needs Matrix more than ever. Every day the importance of decentralisation is more painfully obvious, as we concretely see the terrifying risks of centralised Internet services - whether that’s through corporate takeover, state censorship, blanket surveillance, Internet shutdowns, surveillance capitalism, or the spectre of gigantic centralised data breaches. It’s been amazing to see the world pivot in favour of decentralisation over the time we’ve been building Matrix, and our mission has never been more important.

On one hand it feels we’re creeping ever closer to that goal of providing the missing communication layer for the open Web. The European Union’s Digital Markets Act (DMA) is a huge step in that direction - regulation that mandates that if the large centralised messaging providers are to operate in the EU, they must interoperate. We’ve been busy working away to make this a reality, including participating in the IETF for the first time as part of the MIMI working group - demonstrating concretely how (for instance) Android Messages could natively speak Matrix in order to interoperate with other services, while preserving end-to-end encryption.

On the other hand, Matrix has often got stuck in focusing on solving the Hard Problems of decentralisation, decentralised end-to-end encryption, and the logistical complexities of supporting a massive heterogeneous public communication network and its surrounding heterogeneous ecosystem. It’s fair to say that in the early days our focus was on making something that worked at all - and then later, we shifted to focusing on something that worked and scaled correctly… but we hadn’t managed to focus on ensuring that Matrix provides the building blocks necessary to create blazingly fast, hyper-efficient communication apps which has potential to outperform the centralised mainstream messaging services…

…until now!

Matrix 2.0

Back at FOSDEM we announced the idea of Matrix 2.0 - a series of huge step changes in terms of Matrix’s usability and performance, made up of Sliding Sync (instant login/launch/sync), Native OIDC (industry-standard authentication), Native Group VoIP (end-to-end encrypted large-scale voice & video conferencing) and Faster Joins (lazy-loading room state when your server joins a room).

Now, we’re excited to announce that as of today everyone can start playing with these Matrix 2.0 features. There’s still some work to bring them formally into the specification, but we’re putting it out there for folks to experience right now. Developers: watch this space for updates on the spec front.

Practically speaking, this means there are now implementations of the four pillars of Matrix 2.0 available today which you can use to power a daily-driver Matrix 2.0 client. The work here has been driven primarily by Element, using their new Element X client as the test-bed for the new Matrix 2.0 functionality and to prove that the new APIs are informed by real-world usage and can concretely demonstrably create an app which begins to outperform iMessage, WhatsApp and Telegram in terms of usability and performance… all while benefiting from being 100% built on Matrix.

matrix-rust-sdk and Element X

The mission of Matrix 2.0 has been to provide a huge step forwards in real-world performance, usability and stability - and that means using a real client codebase as a guinea pig to ensure the new protocol is fit for purpose. matrix-rust-sdk has been the main vehicle for this, with Element X as the app primarily driving the new features (although other clients built on matrix-rust-sdk such as Fractal 5 can then automatically benefit from the work should they wish).

To see what all the fuss is about, your best bet is probably to head over to the Element X launch blog post and read all about it! But from the Matrix perspective, this is a flag day in terms of the existence of a Matrix client which empirically outperforms the mainstream clients both in terms of usability and performance: it shows that Matrix is indeed viable to power communication for billions of users, should we get the chance.

From a client perspective: this has meant implementing Sliding Sync (MSC3575) in matrix-rust-sdk - and then creating the entirely new matrix-sdk-ui crate in order to expose higher level APIs to help apps efficiently drive their UI, without each app having to keep reinventing the wheel and risking getting it wrong. The new UI crate gives APIs for efficiently managing a lazy-loaded room list, lazy-loaded room timelines (including edits, reactions, aggregations, redactions etc), and even when the app should show a sync spinner or not. As a result, the vast majority of the heavy lifting can be handled in matrix-rust-sdk, ensuring that the app layer can focus on UI rather than Matrix guts - and performance improvements (e.g. roomlist caching and timeline caching) can all be handled in one place to the benefit of all clients using the SDK.

This is a huge breakthrough relative to the old days of Matrix where each client would have no choice but burn significant amounts of time hand-carving its own timeline and encryption glue logic (although of course clients are still very welcome to do so if they wish!) - but for those wanting higher-level building blocks, matrix-rust-sdk now provides an excellent basis for experimenting with Matrix 2.0 clients. It’s worth noting that the library is still evolving fast, though, and many APIs are not long-term stable. Both the Sliding Sync API and the UI crates are still subject to significant change, and while the crypto crate and its underlying vodozemac E2EE implementation is pretty stable, features such as E2EE Backup are still being added to the top-level matrix-rust-sdk (and thence Element X).

In order to hook matrix-rust-sdk up to Element X, the Element team ended up contributing cancellable async bindings to uniffi, Mozilla’s language binding generator, so you can now call matrix-rust-sdk directly from Swift, Kotlin and (in theory) other languages, complete with beautifully simple async/await non-blocking semantics. This looks to be a pretty awesome stack for doing modern cross-platform development - so even if you have a project which isn’t natively in Rust, you should be able to lean on matrix-rust-sdk if you so desire! We hope that other projects will follow the Rust + Swift/Kotlin pattern for their extreme performance needs :)

Sliding Sync

The single biggest change in Matrix 2.0 is the proposal of an entirely new sync API called Sliding Sync (MSC3575). The goal of Sliding Sync is to ensure that the application has the option of loading the absolutely bare essential data required to render its visible user interface - ensuring that operations which have historically been horribly slow in Matrix (login and initial sync, launch and incremental sync) are instant, no matter how many rooms the user is in or how large those rooms are.

While matrix-rust-sdk implements both Sync v2 (the current API in Matrix 1.8) as well as Sliding Sync, Element X deliberately only implements Sliding Sync, in order to focus exclusively on getting the fastest UI possible (and generally to exercise the API). Therefore to use Element X, you need to be running a homeserver with Sliding Sync support, which (for now) means running a sliding-sync proxy which bolts Sliding Sync support on to existing homeservers. You can check out Thib’s excellent tutorial for how to get up and running (or Element Server Suite provides packages from the Element team)

Now, implementing Sliding Sync in matrix-rust-sdk has been a bit of a journey. Since we showed off the very first implementation at FOSDEM, two big problems came to light. For a bit of context: the original design of Sliding Sync was heavily inspired by Discord’s architecture - where the server calculates an ordered list of large numbers of items (your room list, in Matrix’s case); the client says which window into the list it’s currently displaying; and the server sends updates to the client as the view changes. The user then scrolls around that list, sliding the window up and down, and the server sends the appropriate updates - hence the name Sliding Sync.

Sliding Sync was originally driven by our work on Low Bandwidth Matrix - as it makes no sense to have a fancy line protocol which can run over a 2400 baud modem… if the first thing the app tries to do is download a 100MB Sync v2 initial-sync response, or for that matter a 10MB incremental-sync response after having been offline for a few days (10MB takes 9 hours to shift over a 2400 baud modem, for those who missed out on the 80s). Instead, you clearly only want to send the absolute essentials to the client, no matter how big their account is, and that’s what Sliding Sync does.

The first minor flaw in the plan, however, is that the server doesn’t necessarily have all the data it needs to order the room list. Room ordering depends on what the most recent visible events are in a room, and if the room’s end-to-end encrypted, the server has no way of knowing which events are going to be visible for a given client or not. It also doesn’t know which rooms have encrypted mentions inside them, and we don’t want to leak mention metadata to the server, or design out keyword mentions. So, MSC3575 proposed some complicated contortions to let the client tweak the order client-side based on its superior knowledge of the ordering (given most clients would need to sync all the encrypted rooms anyway, in order to index them and search for keyword notifications etc). Meanwhile, the order might be ‘good enough’ even without those tweaks.

The second minor flaw in the plan was that having implemented Sliding Sync in Element X, it turns out that the user experience on mobile of incrementally loading in room list entries from the server as the user scrolls around the list is simply not good enough, especially on bad connectivity - and the last thing we want to do is to design out support for bad connectivity in Matrix. Users have been trained on mobile to expect to be able to swipe rapidly through infinite-scrolling lists of tens of thousands of photos in their photo gallery, or tens of thousands of emails in their mail client, without ever seeing a single placeholder, even for a frame. So if the network roundtrip time to your server is even 100ms, and Sliding Sync is operating infinitely quickly, you’re still going to end up showing a placeholders for a few frames (6 frames, at 60fps, to be precise) if the user starts scrolling rapidly through their room list. And empirically that doesn’t look great - the 2007-vintage iOS team have a lot to answer for in terms of setting user expectations!

So, the obvious way to solve both of these problems is simply to pull in more data in the background, to anticipate the user scrolling around. In fact, it turns out we need to do that anyway, and indeed pull in all the room data so that room-search is instantly responsive; waiting 100ms or more to talk to the server whenever the user tries to search their roomlist is no fun at all, and it transpires that many users navigate their roomlist entirely by search rather than scrolling. As a result, the sliding sync implementation in matrix-rust-sdk has ended up maintaining an ‘all rooms’ list, which starts off syncing the roomlist details for the most recent N rooms, and then in the background expands to sync all the rest. At which point we’re not really sliding a window around any more: instead it’s more of a QoSed incremental sync.

So, to cut a long story short: while the current Sliding Sync implementation in matrix-rust-sdk and Element X empirically works very well, it’s ended up being a bit too complicated and we expect some pretty significant simplifications in the near future based on the best practices figured out with clients using it. Watch this space for updates, although it’s likely that the current form of MSC3575 will prevail in some respect in order to support low-bandwidth environments where roomlist ordering and roomsearch latency is less important than preserving bandwidth. Critically, we want to figure this out before we encourage folks to implement native server implementations - so for now, we’ll be keeping using the sliding-sync proxy as a way to rapidly experiment with the API as it evolves.

Native Matrix Group VoIP

Another pillar of Matrix 2.0 is that we finally have native Matrix Group VoIP calling (MSC3401)! Much like Sliding Sync has been developed using Element X as a testbed, Element Call has been the guinea pig for getting fully end-to-end-encrypted, scalable group voice/video calling implemented on top of Matrix, building on top of matrix-js-sdk. And as of today, Element Call finally has it working, complete with end-to-end encryption (and integrated in Element X, for that matter)!

Much like Sliding Sync, this has also been a bit of a journey. The original implementations of Element Call strictly followed MSC3401, using full mesh conferencing to effectively have every participant place a call to every other participant - thus decentralising the conference and avoiding the need for a conferencing ‘focus’ server… but limiting the conference to 7 or 8 participants given all the duplication of the sent video required. In Element Call Beta 2, end-to-end encryption was enabled; easy, given it’s just a set of 1:1 calls.

Then the real adventure began: to implement a Selective Forwarding Unit (SFU) which can be used to scale up to hundreds of users - or beyond. The unexpected first move came from Sean DuBois, project lead of the awesome Pion WebRTC stack for Golang - who wrote a proof-of-concept called sfu-to-sfu to demonstrate the viability of decentralised heterogenous cascading SFUs, as detailed in MSC3898. This would not only let calls on a single focus scale beyond hundreds of users, but also share the conferencing out across all the participating foci, providing the world’s first heterogeneous decentralised video conferencing. Element took the sfu-to-sfu implementation, hooked it up to Element Call on a branch, and renamed it as waterfall.

However, when Sean first contributed sfu-to-sfu, he mentioned to us that if Matrix is serious about SFUs, we should take a look at LiveKit - an open source startup not dissimilar to Element who were busy building best-in-class SFUs on top of Pion. And while waterfall worked well as a proof of concept, it became increasingly obvious that there’s a lot of work to be done around tuning congestion control, error correction, implementing end-to-end encryption etc which the LiveKit team had already spent years doing. So, Element reached out to the LiveKit team, and started experimenting with what it might take to implement a Matrix-capable SFU on top of the LiveKit engine.

The end result was Element Call Beta 3, which is an interesting hybrid between MSC3401 and LiveKit’s existing signalling: the high-level signalling of the call (its existence, membership, duration etc) is advertised by Matrix - but the actual WebRTC signalling is handled by LiveKit, providing support for hundreds of users per call.

Finally, today marks the release of Element Call Beta 4, which adds back end-to-end encryption via the LiveKit SFU (currently by using a shared static secret, but in the near future will support full Matrix-negotiated end-to-end encryption with sender keys) - and also includes a complete visual refresh. The next steps here include bringing back support for full mesh as well as SFU, for environments without an SFU, and updating all the MSCs to recognise the hybrid signalling model that reality has converged on when using LiveKit. Meanwhile, head over to https://call.element.io to give it a go, or read more about it in the Element X Ignition blog post!

Native Open ID Connect

Finally, last but not least, we’re proud to announce that the project to replace Matrix’s venerable existing authentication APIs with industry-standard Open ID Connect in Matrix 2.0 has taken a huge leap forwards today, with matrix-authentication-service now being available to add Native OIDC support to Synapse, as well as Element X now implementing account registration, login and management via Native OIDC (with legacy support only for login/logout).

This is a critical step forwards in improving the security and maintainability for Matrix’s authentication, and you can read all about it in this dedicated post, explaining the rationale for adopting OpenID Connect for all forms of authentication throughout Matrix, and what you need to know about the transition.

Conclusion

There has been an enormous amount of work that has gone into Matrix 2.0 so far - whether that’s implementing sliding sync in matrix-rust-sdk and sliding-sync proxy, matrix-authentication-service and all the native OIDC infrastructure on servers and clients, the entirety of Element Call and its underpinning matrix-js-sdk and SFU work, or indeed Faster Joins in Synapse, which shipped back in Jan.

It’s been a pretty stressful sprint to pull it all together, and huge thanks go to everyone who’s contributed - both from the team at Element, but also contributors to other projects like matrix-rust-sdk who have got caught in the crossfire :) It’s also been amazing seeing the level of support, high quality testing and excellent feedback from the wider community as folks have got excited about the promise of Matrix 2.0.

On the Foundation side, we’d like to thank the Members whose financial support has been critical in providing bandwidth to enable the progress on Matrix 2.0 - and for those who want to help accelerate Matrix, especially those commercially building on top of Matrix, please consider joining the Foundation as a member! Also, in case you missed it, we’re super excited to welcome Josh Simmons as Managing Director for the Foundation - focusing on running the Foundation membership programme and generally ensuring the growth of the Foundation funding for the benefit of the whole Matrix community. Matthew and Amandine continue to lead the overall project (alongside their day jobs at Element), with the support of the other three independent Guardians - but Josh is working full time exclusively on running the non-profit foundation and gathering funds to support Matrix.

Talking of funding, we should mention that we’ve had to pause work in other places due to lack of Matrix funding - especially while focusing on successfully shipping Matrix 2.0. Major next-generation projects including Third Room, P2P Matrix, and Low Bandwidth Matrix have all been paused unless there’s a major shift in circumstances - so, if you have money and you’re interested in a world where the more experimental next-generation Matrix projects progress with folks working on them as their day job, please get in touch with the Foundation.

What’s next?

While this is the first usable release of Matrix 2.0 implementations, there’s loads of work still to be done - obvious work on Matrix 2.0 includes:

• Getting Native OIDC enabled on matrix.org, and providing migration tools to Native OIDC for existing homeservers in general
• Reworking Sliding Sync based on the lessons learned implementing it in matrix-rust-sdk
• Actually getting the Matrix 2.0 MSCs stabilised and matured to the point they can be approved and merged into the spec
• Adding encrypted backups to matrix-rust-sdk
• Reintroducing full-mesh support for Native Matrix Group VoIP calling
• Having a big Matrix 2.0 launch party once the spec lands!

Outside of Matrix 2.0 work, other big items on the horizon include:

• Adding Rust matrix-sdk-crypto to matrix-js-sdk, at which point all the official Matrix.org client SDKs will (at last!) be using the same stable performant E2EE implementation
• Continuing to contribute Matrix input to the MIMI working group in IETF for Digital Markets Act interoperability
• Working on MLS for next-generation E2EE
• Next generation moderation tooling and capabilities
• Account Portability and Multihomed accounts
• …and much much more.

So: welcome to our brave new Matrix 2.0 world. We hope you’re excited about it as we are - and thanks to everyone for continuing to use Matrix and build on it. Here’s to the beginning of a whole new era!

Matthew, Amandine and the whole Matrix team.

Before we explain all about our new authentication system - don't panic! This change is part of Matrix 2.0, the next step for Matrix that will be introduced in a blog post later today.

Nothing breaks

Matrix 2.0 is a reference to Matthew’s FOSDEM talk Matrix 2.0, itself a reference to the name he coined in the Matrix Summer Special 2022: evolutions of the spec that are making the Matrix experience so much better it almost feels like a different protocol. One of the major changes in this bundle is the introduction of OpenID Connect-native Matrix and its implementation matrix-authentication-service (MAS).

Don’t panic: your current setup is not at risk, you don’t urgently need to deploy the matrix-authentication-service to keep using Matrix.

Let’s walk together through what OIDC-native Matrix, what changes as compared to the Matrix we know and love, and what’s the impact for your deployment.

From OIDC-ish to OIDC

One of the most popular Matrix server implementations, Synapse, supports authentication via OIDC. This is the only mainstream server implementation to support authentication via an OpenID Connect Provider. OpenID Connect, SAML, and more generally Single Sign On (SSO) are a requirement for most companies past a certain size, as well as a security best practice.

As it is today, Synapse is more OIDC-compatible than OIDC-native. This means that it’s possible to use SSO (typically OpenID Connect) to connect to Synapse, but that’s about where the OpenID Connect adventure stops. Once you’re authenticated, Synapse generates a Matrix Access Token for your client, but that is not an OAuth2/OIDC Access Token. From your client point of view, it is doing the “Matrix SSO dance” defined by the spec, but the fact that Synapse does a login via OIDC, SAML or CAS is irrelevant to the client. Whenever your client asks Synapse for anything that requires being logged in, it sends that Matrix Access Token in the Authorization header of the http requests.

While this approach has served Matrix well, the Matrix Access Token method is reimplementing some concepts of OpenID Connect without all the benefits of thousands of developers battle testing and fixing every edge case they meet.

MSC3861, which proposes the adoption of OIDC in Matrix, is all about embracing the best of OIDC to make Matrix even better. By adopting the standard OIDC flows we allow Matrix to stand on the shoulders of another battle-tested industry standard. Not only does it improve security overall, it also unlocks new use cases for Matrix.

Unlocking use cases

The Matrix Access Tokens Matrix currently relies on are secure, but not very flexible. A unique token is granted to each and every client during user login. This Matrix Access Token is only revoked when the user manages their sessions and logs a device out. A Matrix Access Token gives full access to a Matrix Account. Session management based on Matrix Access Tokens is quite nuclear.

Using OpenID Connect for Matrix comes with three major areas of improvements:

• Standard authentication process on every client, including on devices without a keyboard
• Improved security with token rotation
• Better granularity of permissions (which improves security too!)

Authenticate like you want

When relying on OIDC, the client delegates authentication to the OpenID Provider. This means that the client redirects the user to a web page on the OpenID Provider, where the user needs to authenticate. This authentication can happen through a username and password, WebAuthn (passwordless login), additional steps if you want to add MFA… and much more.

Since the authentication happens on the OpenID Provider, the client doesn’t need to support any authentication method other than “redirect the user to the OpenID Provider, and handle the result”. Once the user has authenticated against the OpenID Provider, it is going to be redirected back to its client, to allow it to retrieve an Access Token on their behalf.

During the whole process, no password was given to the client, and the client doesn’t even know how the user authenticated. All that matters to the client is that it now has an Access Token it can use to perform the regular Matrix API calls.

From the user perspective, the authentication is a very familiar process very well integrated in their password manager regardless of the client they use. They can use any client that supports OIDC, without having to worry about whether it supports every particular step of their authentication process. It’s also worth noting that matrix-authentication-service has a compatibility layer to support the m.login.password flow. This means compatibility with older clients will not break!

From the administrator perspective, it’s possible to force authentication flows (e.g. MFA), making sure the login is secure and matches their organisation’s policies. It also makes it possible to have a central management of all users' devices, including the ability to enforce policies upon them as provided by the OpenID Provider. For example, it becomes possible to the re-authentication of the user if they leave a trusted network.

From the developer perspective, neither clients nor servers have to add support for every new authentication method, existing or to come.

Element has been spearheading the effort to implement MSC3861 and matrix-authentication-service, and Element X is the first client to support them. Here is how the authentication flow and account management look like as seen from Element X iOS.

Token rotation

While current Matrix Access Token can expire thanks to MSC2918, this best practice didn’t make it to most of the clients of the ecosystem. If the access token of a user was leaked for one reason or another, they wouldn’t notice since this wouldn’t create a new session, and the leaked token would be valid forever. Please note that thanks to E2EE, attackers who gain access to a Matrix Access Token would not be able to read the encrypted content in encrypted rooms.

OpenID Connect relies on two tokens:

• an Access Token, that is used to perform the authenticated API calls, and that expires regularly,
• and a Refresh Token that is used in conjunction with the Access Token to rotate both.

This ensures that if the Access Token was leaked during an API call, its validity would be very limited in time. Leaking a Refresh Token will also have limited impact, since the server would notice that two different clients are trying to consume the same Refresh Token and would log the user out from devices using this token.

Principle of least privilege

Matrix is not just about apps and servers. It’s also a vibrant ecosystem of widgets, bots and various CLI to toy with. One of the strengths of Matrix has been its hackability: log into a client, dump the matrix access token, and you can hand it to a CLI or a bot for it to do things on your behalf.

While this approach is particularly convenient for development, it comes with its own lot of issues. Most of the time you don’t want the CLI or bot to be able to do everything on your behalf.

Widgets are another case where OIDC can help. While widgets are not yet part of the spec, in practice they are already commonly used. In essence, a widget is a tiny app that has access to an (unspecified) widgets API that allows it to do a limited amount of things on behalf of the user, such as sending a message in the rooms it’s in. While this has served several clients well, this can be improved. Indeed, it’s up to the client to enforce the restrictions mandated by the widgets API. This means you can’t be assured that all the clients will consistently enforce them.

With OpenID Connect, widgets can eventually become tiny embedded Matrix clients on their own, and they can get their own Access Token with explicit and granular permissions of their own. Those restrictions will be enforced by the server, who will not even be aware it’s a widget calling the API, making the experience consistent across clients. This behaviour still needs to be added to the specification, but a more OIDC-native Matrix paves the way for it.

Upgrading to use matrix-authentication-service

MSC3861 (and friends) define how the OIDC flows and concepts (e.g. token exchange, dynamic client registration) should be used in Matrix, as well as the requirements for things which need to be coordinated with the homeserver (e.g. session lifecycle management).

The goals of matrix-authentication-service (MAS) are two-fold: firstly, it provides an implementation of the MSCs to support the Matrix Specification Change (MSC) process; secondly, it provides capabilities to facilitate the transition of the Matrix ecosystem to OIDC.

Because MSC3861 is changing the technical mechanism by which your client gets access to your account it is not something that can be done by pushing a single, big button.Instead we need a transition period where both OIDC and the existing legacy Matrix authentication methods coexist so that different clients and homeservers can migrate at their own pace.

One way in which MAS facilitates this transition is by allowing deployments without an external Identity Provider (IdP) to use OIDC out-of-the-box. This means that MAS will support some of the current flows that Synapse supports, primarily in the form of password-based login. Please note those flows are not supported yet: MAS requires an external IdP for now.

Another way is that MAS allows for existing sessions and access tokens to be imported (just from Synapse so far) so that there is no need for clients to re-authenticate when a homeserver is migrated.

Because of these migration capabilities we expect MAS to be widely used, however, in keeping with the open philosophy of Matrix, we expect other implementations to exist either standalone or integrated with other OIDC providers like Keycloak or Dex.

MAS is not intended to be a fully fledged IdP solution: we suggest using a dedicated software for this if you want to do more than what is included.

Currently MAS is deployable as a sidecar service alongside the homeserver. While larger setups shouldn’t see this as a problem, it might be seen as cumbersome in smaller self-hosted setups. For the sake of convenience, the Synapse and MAS team are investigating how to bundle MAS along with Synapse natively using PyO3. We anticipate such a bundle to exist by the end of Q2 2024.

You can browse the matrix-authentication-service docs to get all the technical details and installation steps.

Matrix Live

Dept of Spec 📜

TravisR says

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

• MSC4056: Role-Based Access Control (mk II)

MSCs proposed for Final Comment Period:

• MSC2702: Specifying semantics for Content-Disposition on media

MSCs in Final Comment Period:

• No MSCs are in FCP.

Accepted MSCs:

• No MSCs were accepted this week.

Closed MSCs:

• No MSCs were closed/rejected this week.

New MSCs in detail

In this new segment, we aim to give a bit more context as to why an MSC was opened, beyond what is available in the MSC's introduction.

MSC4056 stems from a conversation held back at IETF 117, where members of the Spec Core Team (SCT) were attempting to make RBAC work in Matrix. Thankfully, there was prior art in the form of MSC2812, but a problem with decentralization (and specifically state resolution) was discovered. Thoughts were had about how to fix it, and MSC4056 is the result of those thoughts. Implementation work is eventually planned for this MSC, but in the meantime it should see forwards movement with the SCT's involvement in the MIMI working group at the IETF.

If you have thoughts or suggestions about the very Discord-centric approach, please leave them on the MSC :)

Continue reading…

Matrix Live

Dept of Status of Matrix 🌡️

Matthew announces

Josh joined the Matrix Foundation as its first ever Managing Director!

Continue reading…

Hi all,

Today is a big day! As you know, over the last few months we’ve been searching for a Managing Director to join the Matrix.org Foundation full-time, focused on managing the Foundation’s finances, organising the Foundation’s membership programme, helping raise funding to support Foundation work, working with the Guardians to ensure the Foundation stays on mission, and ensuring the Foundation can operate successfully as a fully independent entity.

Continue reading…

Matrix Live

Dept of Status of Matrix 🌡️

Thib says

We’d like to thank everyone for their patience as we continue to work toward restoring the Libera.Chat bridge, and apologize for the continued inconvenience. We’ve heard from many people and communities who are impacted, who have confirmed that operating this bridge is an important service and we remain committed to getting it back online.

It’s been a month since our last update and folks have been reaching out, so we wanted to take this opportunity to provide a brief update.

The bridge team at Element is still actively working on the issues that led to the bridge being disabled in the first place. You can see some of the work that’s in flight through GitHub PRs: #1757, #1766, #1764, #1734.

We’re also looking into a way to transition responsibility for the bridge from Element to being directly run by The Matrix.org Foundation over the coming months - more details as we have them.

Unfortunately, we do not yet have a clear timeline for bringing the bridge back online. We’ll continue providing regular updates and will share more information as soon as we can. Thank you again for your patience! Please do not hesitate to reach out at #libera-matrix:libera.chat if you have any questions or concerns.

Continue reading…

Dept of Spec 📜

TravisR announces

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

• MSC4049: Sending events as a server or room
• MSC4048: Signed key backup
• MSC4047: Send Keys
• MSC4046: Make & send PDU endpoints

MSCs in Final Comment Period:

• No MSCs are in FCP.

Accepted MSCs:

• MSC3930: Polls push rules/notifications

Merged MSCs:

• MSC4040: Update SRV service name to IANA registration

Matrix 1.8 is here!

If you haven't yet seen the blog post, check it out. Room version 11 is new in this release, and we've already got an idea for what Matrix 1.9 looks like :)

New MSCs in detail

In this new segment, we aim to give a bit more context as to why an MSC was opened, beyond what is available in the MSC's introduction.

MSC4049 is highly experimental investigative work into what it would take to support making messages as appearing to be sent by a room or server instead of a user. There are some use cases highlighted in the MSC itself, but the primary driving factor is a point of relatively minor feedback from the MIMI working group: "does sender really need to be a user ID?". The spike-shaped experiment overlaps heavily with both crypto IDs and pseudo IDs by accident, but might help inform those two projects via MSC4047 and MSC4046. Currently there is not a plan to push any of the 3 MSCs towards FCP, though feedback is very much welcome on how the stack feels.

MSC4048 is part of the crypto team's mission to improve encryption across all of Matrix, with this particular MSC looking to improve the trustworthiness of key backups. Watch this space for updates as the MSC progresses, and please provide feedback on the proposal itself.

Continue reading…

Hey all,

Matrix 1.8 is out now! The last release, Matrix 1.7, was full of features and laid out a plan for what Matrix 1.8 was expected to become. We spent most of our time focusing on the MSC3995-related MSCs from that original plan, but made an effort to get the other stuff looked at as well.

With this release we see a total of 9 MSCs achieve their formally adopted status. The full changelog at the bottom has all the details, but please read on for what’s new in room version 11, and Matrix 1.9’s roadmap :)

Room version 11

Matrix 1.8 features a new room version! Normally a room version wouldn’t have a particular theme, but for v11 we aimed to clean up the different algorithms and event format details. After 10 prior room versions there were some artifacts of the past sticking around (but not causing problems necessarily) - many of them are cleaned up here.

Specified originally as MSC3820, v11 introduces the following changes:

• MSC2174 - Move redacts to the content of m.room.redaction events.
• MSC2175 - Remove creator from m.room.create events (use sender instead).
• MSC3989 - Remove origin from events.
• MSC2176, MSC3821 - General updates to redaction algorithm.

Alongside being a cleanup room version, v11 is the initial base we used for our efforts in the IETF world. It provides an easier starting point for new server implementations, particularly when paired with Linearized Matrix (described as both an IETF Internet-Draft and MSC3995).

In future room versions the cleanup effort will continue, alongside additional features for supporting the IETF use cases. Watch this space for updates.

Upcoming in Matrix 1.9

We’re continuing the trend of planning ahead and have the following themes planned for work in Matrix 1.9:

• Anything required by the MIMI/IETF efforts we’re undertaking. This currently includes:
  • Extensible Events
  • Role-Based Access Control (RBAC; new MSC expected)
  • Next generation of Linearized Matrix (already)
  • MSC4044 and MSC4045
  • Binary-encoded events (maybe, but probable)
  • Canonical DMs (maybe)
  • Sending events as rooms/servers instead of just users (maybe)
  • Further room version cleanup as needed
• Merging MSC3939.
• Accepting a combination of MSC2545, MSC1951, MSC3892, and MSC4027. This may involve creating a new MSC to cover the functionality of each, plus the needful for IETF and Extensible Events efforts.
• Merging MSC3391 (time permitting).

A lot of this stuff might take the shape of opening MSCs or thinking about the problems, but we’re also very optimistic about getting them through FCP before November 2023. Watch this space for updates :)

The full changelog

There’s so many more things than what we covered in this blog post - flip through the changelog below for a full idea of what’s landed.

Client-Server API

Backwards Compatible Changes

• Require callers to be joined to the room to report its events, as per MSC2249. (#1517)

Spec Clarifications

• Fix missing type property in the JSON schema definition of the m.reaction event. Contributed by @chebureki. (#1552)
• Make sure examples types match schema in definitions. (#1563)
• Allow null in room_types in POST /publicRooms endpoints schemas. (#1564)
• Fix broken header formatting. Contributed by @midnightveil. (#1578)
• Render binary request and response bodies. (#1579)
• Fix description of MAC calculation in SAS verification. (#1590)
• Update link to SAS emoji definition data. (#1593)
• Fix various typos throughout the specification. (#1597)

Server-Server API

Deprecations

• Deprecate matrix SRV lookup steps during server discovery, as per MSC4040. (#1624)

Backwards Compatible Changes

• Add matrix-fed SRV lookup steps to server discovery, as per MSC4040. (#1624)

Spec Clarifications

• Document why /state_ids can respond with a 404. (#1521)
• Fix response definition for POST /_matrix/federation/v1/user/keys/claim. (#1559)
• Fix examples in server keys definition. (#1560)
• Make sure examples types match schema in definitions. (#1563)
• Allow null in room_types in POST /publicRooms endpoints schemas. (#1564)
• Fix broken header formatting. Contributed by @midnightveil. (#1578)
• Remove spurious mention of a "default port" with respect to SRV record lookup. (#1615)
• Switch to ordered list for server name resolution steps. (#1623)

Application Service API

Spec Clarifications

• Fix type of custom fields in thirdparty lookup queries. (#1584)

Identity Service API

Spec Clarifications

• Make sure examples types match schema in definitions. (#1563)

Push Gateway API

No significant changes.

Room Versions

Backwards Compatible Changes

• Add room version 11 as per MSC3820. (#1604)
• Move redacts from top level to content on m.room.redaction events in room version 11, as per MSC2174. (#1604)
• Remove creator from m.room.creator events in room version 11, as per MSC2175. (#1604)
• Remove remaining usage of origin from events in room version 11, as per MSC3989. (#1604)
• Update the redaction rules in room version 11, as per MSC2176 and MSC3821. (#1604)

Appendices

Backwards Compatible Changes

• Allow + in Matrix IDs, as per MSC4009. (#1583)

Spec Clarifications

• Clarify spec re canonical JSON to handle negative-zero; also, give an example of negative-zero and a large power of ten. (#1573)

Internal Changes/Tooling

Backwards Compatible Changes

• Upgrade Swagger data to OpenAPI 3.1. (#1310)
• Create @matrix-org/spec npm package to ship the SAS Emoji data definitions & translations. (#1620)

Spec Clarifications

• Update the CI to validate the file extension of changelog entries. (#1542)
• Disclosure sections now only display their title when collapsed. (#1549)
• Fix the sidebar in recent versions of Hugo. (#1551)
• Bump jsonschema to validate JSON Schemas against Draft 2020-12. (#1556)
• Use Redocly CLI to validate OpenAPI definitions. (#1558)
• Use tag name as the OpenAPI definition version. (#1561)
• Make sure version in x-changedInMatrixVersion is a string. (#1562)
• Clarify usage of ABNF for grammar in the documentation style guide. (#1582)
• Remove unnecessary oneOfs in JSON schemas. (#1585)
• Update the version of Hugo used to render the spec to v0.113.0. (#1591)
• Fix rendered changelog with new version of towncrier. (#1598)
• Improve the layout of tables on desktop displays. Contributed by Martin Fischer. (#1601)