This Week in Matrix 2023-03-31

Matrix Live

Dept of Status of Matrix 🌡️

uhoreg announces

Messaging Layer Security approved by the IETF

The IETF has approved Messaging Layer Security (MLS) for publication. MLS is an end-to-end encryption method designed for group messaging. We have been working on integrating a variant of MLS into Matrix. Keep an eye out for demos in the near future.

Dept of Spec 📜

Andrew Morgan (anoa) [GMT-8] says

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

• MSC3987: Push actions clean-up
• MSC3985: Break-out rooms
• MSC3984: Sending key queries to appservices

MSCs in Final Comment Period:

• MSC3980: Dotted Field Consistency (merge)

Accepted MSCs:

• MSC3923: Bringing Matrix into the IETF process
• MSC2677: Annotations and reactions

Closed MSCs:

• No MSCs were closed/rejected this week.

Spec Updates

Last week we mentioned that we'd have more to share this week about an easier API for accessing rooms in Matrix, and while it was our intention to have an MSC out by now, our priorities shifted slightly after the MIMI working group session at IETF 116. Some exciting news on that front though: our proposed easier API, called Linearized Matrix (apologies to all the mathematicians), is very much in line with what the working group is thinking about. So much so that we're going through the effort of writing up Matrix as a series of proper Internet-Draft specifications.

We don't currently have an up-to-date document which covers Linearized Matrix completely, but the short version is it allows a server to support individual rooms being linear arrays instead of DAGs. This doesn't prevent a DAG-capable server from joining the room and speaking full-blown DAG either, which is particularly important for compatibility with the existing Matrix network. Currently our efforts on Linearized Matrix are in implementation rather than documentation, though once things are slightly more stable we'll be getting an MSC out there for everyone to review more easily. Watch this space for news.

Following IETF 116, we have an immense amount of work ahead of us to define Matrix as the standard for interoperable chat, but we're well on our way on getting through it all. Namely, we're going around and mapping Matrix's functionality onto MIMI's concepts, defining Matrix as a proper IETF standard along the way. The expected outcome of this for the implementation authors of Matrix is a spec that is significantly easier to follow, finally.

For an idea of what's ahead, here's what the SCT will be looking at over the next several months:

• Linearized Matrix (implementation & MSC)
• Extensible Events (at least the core types) - this will serve as the basis for an interoperable messaging format in our IETF drafts
• Decentralized MLS & interoperability of crypto
• Clarification gaps and bugs in the current spec
• Pseudonymous user IDs and account portability
• Almost certainly something that was missed when writing this list

Considering the above list, the Matrix 2.0 objectives (sliding sync, OIDC, native VoIP conferencing, and faster room joins), and the core team's work around mentions, abuse reporting, and more, the SCT will be a bit busy. That said, if you have MSCs you think we should be looking at, let us know in the #sct-office:matrix.org room. We've recently started doing our weekly planning in that room too, which should help give an idea for what the SCT is expecting to look at each week.

Random MSC of the Week

The random MSC of the week is... [WIP] MSC2966: Usage of OAuth 2.0 Dynamic Client Registration in Matrix!

This MSC provides a mechanism for implementing Dynamic Client Registration (RFC 7591) for OAuth 2.0 between Matrix clients and homeservers. Without this, homeserver admins would need to manually configure OAuth metadata (Redirect URIs, application names, client secrets, and more) for every Matrix client that wanted to connect to the homeserver. Since that doesn't really scale in an environment that allows anyone to use any client they like, dynamic registration is critical! Dynamic registration allows clients to communicate this metadata to the homeserver at the login/registration step.

MSC2966 is part of a series of MSCs that add first-class OpenID Connect (OIDC) support to Matrix. You can see an overview of the related MSCs (here) and https://areweoidcyet.com/ for the latest progress on integrating OIDC into the Matrix spec!

Dept of Servers 🏢

Synapse (website)

Synapse is a Matrix homeserver implementation developed by the matrix.org core team

Shay says

It's yet another Friday which means it's TWIM day. This week the backend team released Synapse 1.80.0. Notable highlights include:

• Fixed a bug in which the POST /_matrix/client/v3/rooms/{roomId}/report/{eventId} endpoint would return the wrong error if the user did not have permission to view the event. This aligns Synapse's implementation with MSC2249
• Fixed a bug introduced in Synapse 1.75.0rc1 where the SQLite port_db scriptwould fail to open the SQLite database
• Stabilized support for MSC3966: event_property_contains push condition
• Fixed a long-standing bug in which the user directory would assume any remote membership state events represent a profile change

and much more! You can learn more here in the release notes: https://github.com/matrix-org/synapse/releases.

Nico says

It has been quiet for a while, but mostly because I was busy or I felt like there was nothing worth talking about. But this time I fixed a bug, that I am actually so proud of, that I need to share it!

I regularly ban users in my rooms temporarily, if they are behaving like an asshole and I want to show them, that such a behaviour is not accepted, even if they usually contribute useful stuff. This tactic has in general worked pretty well and led to a noticeable improvement in the conversation culture in those rooms.

HOWEVER one bug has plagued us for ages and made this method way more destructive, than it should be. Everytime you banned someone on a single user homeserver or even just if they were the only user in the room from that server, you couldn't unban them. That is because there is no member in the room anymore to send the unban event to, so the other server never received an unban. Now, you might think, why is that a problem, you can join the room initially as well. And that was my thought as well, but considering that this bug was transferred to Github in 2016 and several people looked at it, I thought it was just way more complicated to fix. But since I hit this a few times a year and sometimes even multiple times a month, I spent a lot of time looking at the issue on Github. And I did see a few times, that there was a Complement test for it and every server passed that test, but only Synapse got blocklisted. So it couldn't be a fundamental problem...

So I thought, okay, this is annoying me regularly now, I will just get my hands dirty this time and fix it. Even if it takes me weeks or months. So I set up Complement, allowed the test to run on Synapse and watched it fail. Sadly, the complement logs were so long, that I couldn't find the actual error, but a few tries (and hours of runtime) later I got useful output, searched for the error message in synapse and looked at the code. It looked more complicated than Dendrite, all of the membership transitions were in 1 function instead of different ones. But apart from that, the obvious thing that jumped out was that Synapse verified against the room state if the user was allowed to join before checking if it was in the room and then attempting a remote join. Now usually it makes sense to verify the state locally, but not if you are not in the room, so I just moved the in room check up a bit and moved the local verification behind that check, so that the local state is only used for verification, if we are in the room. And it worked! So apart from indentation changes, this was a 1 line change in Complement, a 1 line change in Sytest (to make it fail on the right error) and a 3 line change in synapse (+ 50 lines or so of indentation changes...). This bug has been bothering me for ages and that was all it took to fix!

Now, bugs often look easy in hindsight, so don't use this to judge the Synapse devs or anyone. I just had the right motivation and a bit of luck finding the problem quickly (and help from Synapse devs). Possibly that change broke something else deep down in Synapse and we get to deal with demons soon. But I am happy for now!

Context links: https://github.com/matrix-org/synapse/pull/15323 https://github.com/matrix-org/synapse/issues/1563 (look, they both start with 15!)

Construct (website)

A performance-oriented homeserver with minimal dependencies.

Jason Volk announces

This week we focused on build system improvements. First, a huge thank you to Giovanni Bottaro for digging into the Ubuntu docker image and slimming it down to nearly half its size. It's still not as compact as our Alpine image, which is the smallest of any of the Matrix servers, but it's the one I would say is the most stable at this time. On the heels of this, I decided to overhaul the docker image matrix. Construct now offers several dozen combination docker images from Ubuntu 22.04, 22.10, Alpine 3.16, 3.17, gcc-10, 11, 12, clang 14, 15, and on arm64 with neon, amd64 with avx, avx2, and avx512.

Check out #test:zemos.net today!

Dept of Clients 📱

Nheko (website)

Desktop client for Matrix using Qt and C++17.

Nico reports

While I was mostly spelunking in CI land, some people did some really cool stuff!

Dtelsing implemented screenshare support on Wayland. This requires a proper pipewire + xdg-desktop-portal ScreenCast interface setup, but it means Nheko will properly ask your OS for a screenshare and your OS then gets to decide, what screen or window you are allowed to share. Especially considering how little documentation there is for this, I am pretty impressed how quickly they got this working and it seems to work quite nicely on the platforms I have tested, but feel free to test it yourself and report issues you find!

LorenDB tweaked the design of our switches, because they bothered him. Now, I have no eye for that, so I can't tell you what changed, but I assume people who can tell if something is pretty will be happy! Ryandesign also fixed the macOS build on case sensitive filesystems, a pseudonymous contributor added querying your own status message to the dbus interface and Nheko now linkifies improperly encoded matrix.to links again.

I think that is all, thank you all for the cool contributions!

Ement.el (website)

Matrix client for Emacs

alphapapa announces

Ement.el, a Matrix client for the GNU Emacs text editor and Lisp environment, has been released at version 0.8. Changes since the last release include:

Additions

• Command ement-create-space creates a new space.
• Command ement-room-toggle-space toggles a room's membership in a space (a way to group rooms in Matrix).
• Visibility of sections in the room list is saved across sessions.
• Command ement-room-list-kill-buffer kills a room's buffer from the room list.
• Set device_id and initial_device_display_name upon login (e.g. Ement.el: username@hostname).

Changes

• Room-related commands may be called interactively with a universal prefix to prompt for the room/session (allowing to send events or change settings in rooms other than the current one).
• Command ement-room-list reuses an existing window showing the room list when possible.
• Command ement-tag-room toggles tags (rather than adding by default and removing when called with a prefix).
• Default room grouping now groups "spaced" rooms separately.

Fixes

• Message format filter works properly when writing replies.
• Improve insertion of sender name headers when using the "Elemental" message format.
• Prompts in commands ement-leave-room and ement-forget-room.

Feel free to join us in the chat room: #ement.el:matrix.org.

Element Web/Desktop (website)

Secure and independent communication, connected via Matrix. Come talk with us in #element-web:matrix.org!

Danielle reports

• This week we’ve been making some great progress on all notifications fronts:
  • Our stuck notifications MSC is in progress, other work on these bugs is also on-going
  • The new notifications settings pages are designed and being worked on
  • The Intentional Pings MSC is also making progress; hoping to remove those pesky pings and allow us to stop using l33t speak!
• The cloud packaging improvements we’ve been making are nearly at an end - here’s to quicker, more reliable releases for our teams! 🎉
• Other product improvements include updates to the permalinks behaviour - try sending a matrix.to link to a message and let us know what you think.
• Our Product and Design teams have been taking a closer look at community PRs and are putting processes in place to reduce the time PRs are blocked by either department.

Element iOS (website)

Secure and independent communication for iOS, connected via Matrix. Come talk with us in #element-ios:matrix.org!

Ștefan reports

This week in Element X on iOS:

• We’ve been fixing some issues with the timeline including updates to edits and messages that failed to send. You’ll also see the timeline ordering is more reliable also.
• Started to look into OIDC and how we integrate the new systems there
• Room details are making forward progress also and we’ve recently added “Ignore user” right to the DM details screen.
• We have also started working on the media upload flows and are making good progress

Element Android (website)

Secure and independent communication for Android, connected via Matrix. Come talk with us in #element-android:matrix.org!

benoit says

On Element X for Android this week we:

• Introduced push notifications!
• We’re also looking at how we share translations across mobile platforms.
• And! The Room Details screens are continuing to grow; now you can see the room member list.

Release of Element Android get delayed because we want to fix an issue on the timeline first.

Dept of VoIP 🤙

Element Call (website)

Native Decentralised End-to-end Encrypted Group Calls in Matrix, as a standalone web app

Dave reports

We have a new Element Call release -> 0.3.6. This release principally contains a fix for a bug where participants could see video from other users but not be shown themselves (matrix-org/matrix-js-sdk@da03c3b). Feature wise you can use Element Call now for screen sharing only (without any media device attached to the computer). Here is the full changelog

Dept of SDKs and Frameworks 🧰

matrix-nio (website)

Paarth Shah announces

Hello all! I'm a new maintainer for this project, and I'm happy to say that we've released a new version, 0.20.2, already available on Pypi!

This release was mainly for upgrading dependencies and fixing bugs, (see the full changelog for details!) but I'm hopeful that I'll be able to dedicate some time to advancing some new features, and being able to help get new contributions rolling in!

matrix-rust-sdk (website)

Next-gen crypto-included SDK for developing Clients, Bots and Appservices; written in Rust with bindings for Node, Swift and WASM

Jonas Platte reports

• Lots of additions to our bindings in the matrix-sdk-ffi crate
  • We've been working on adding bindings to the tracing library to matrix-sdk-ffi such that Kotlin or Swift code like ElementX that calls into the Rust SDK can benefit from the same structured logging infrastructure
  • We switched matrix-sdk-ffi to use the new sqlite crypto store
  • We added Client::ignore_user, Client::get_profile
• We fixed a problem where gappy syncs (which happen mostly when a process using the SDK is suspended and resumed a while later after many events arrived) would result in an invalid timeline item ordering
• We updated the read marker code for the timeline to never insert the marker at the very end

Matrix in the News 📰

Oleg announces

It became a habit to talk about the new Matrix stuff after FOSDEM in the German Podcast Das Duumvirat. 🙂

Feel free to check out the Recording (in 🇩🇪)

Dept of Ping

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	matrix.m0aws.co.uk	433
2	tchncs.de	509
3	kohlernet.de	611
4	aesyc.co	742.5
5	ilmari.org	860
6	nognu.de	917
7	kitsunet.info	928
8	zemos.net	2006
9	matrix.org	2276
10	aguiarvieira.pt	4264.5

#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	skladka.net	227.5
2	herkulessi.de	608
3	matrix.org	785.5
4	zemos.net	835.5
5	frai.se	11939

That's all I know

See you next week, and be sure to stop by #twim:matrix.org with your updates!