Matrix v1.8 release

23.08.2023 16:25 — Releases Travis Ralston

Hey all,

Matrix 1.8 is out now! The last release, Matrix 1.7, was full of features and laid out a plan for what Matrix 1.8 was expected to become. We spent most of our time focusing on the MSC3995-related MSCs from that original plan, but made an effort to get the other stuff looked at as well.

With this release we see a total of 9 MSCs achieve their formally adopted status. The full changelog at the bottom has all the details, but please read on for what’s new in room version 11, and Matrix 1.9’s roadmap :)

Room version 11

Matrix 1.8 features a new room version! Normally a room version wouldn’t have a particular theme, but for v11 we aimed to clean up the different algorithms and event format details. After 10 prior room versions there were some artifacts of the past sticking around (but not causing problems necessarily) - many of them are cleaned up here.

Specified originally as MSC3820, v11 introduces the following changes:

  • MSC2174 - Move redacts to the content of m.room.redaction events.
  • MSC2175 - Remove creator from m.room.create events (use sender instead).
  • MSC3989 - Remove origin from events.
  • MSC2176, MSC3821 - General updates to redaction algorithm.

Alongside being a cleanup room version, v11 is the initial base we used for our efforts in the IETF world. It provides an easier starting point for new server implementations, particularly when paired with Linearized Matrix (described as both an IETF Internet-Draft and MSC3995).

In future room versions the cleanup effort will continue, alongside additional features for supporting the IETF use cases. Watch this space for updates.

Upcoming in Matrix 1.9

We’re continuing the trend of planning ahead and have the following themes planned for work in Matrix 1.9:

  • Anything required by the MIMI/IETF efforts we’re undertaking. This currently includes:
    • Extensible Events
    • Role-Based Access Control (RBAC; new MSC expected)
    • Next generation of Linearized Matrix (already)
    • MSC4044 and MSC4045
    • Binary-encoded events (maybe, but probable)
    • Canonical DMs (maybe)
    • Sending events as rooms/servers instead of just users (maybe)
    • Further room version cleanup as needed
  • Merging MSC3939.
  • Accepting a combination of MSC2545, MSC1951, MSC3892, and MSC4027. This may involve creating a new MSC to cover the functionality of each, plus the needful for IETF and Extensible Events efforts.
  • Merging MSC3391 (time permitting).

A lot of this stuff might take the shape of opening MSCs or thinking about the problems, but we’re also very optimistic about getting them through FCP before November 2023. Watch this space for updates :)

The full changelog

There’s so many more things than what we covered in this blog post - flip through the changelog below for a full idea of what’s landed.

Client-Server API

Backwards Compatible Changes

  • Require callers to be joined to the room to report its events, as per MSC2249. (#1517)

Spec Clarifications

  • Fix missing type property in the JSON schema definition of the m.reaction event. Contributed by @chebureki. (#1552)
  • Make sure examples types match schema in definitions. (#1563)
  • Allow null in room_types in POST /publicRooms endpoints schemas. (#1564)
  • Fix broken header formatting. Contributed by @midnightveil. (#1578)
  • Render binary request and response bodies. (#1579)
  • Fix description of MAC calculation in SAS verification. (#1590)
  • Update link to SAS emoji definition data. (#1593)
  • Fix various typos throughout the specification. (#1597)

Server-Server API

Deprecations

  • Deprecate matrix SRV lookup steps during server discovery, as per MSC4040. (#1624)

Backwards Compatible Changes

  • Add matrix-fed SRV lookup steps to server discovery, as per MSC4040. (#1624)

Spec Clarifications

  • Document why /state_ids can respond with a 404. (#1521)
  • Fix response definition for POST /_matrix/federation/v1/user/keys/claim. (#1559)
  • Fix examples in server keys definition. (#1560)
  • Make sure examples types match schema in definitions. (#1563)
  • Allow null in room_types in POST /publicRooms endpoints schemas. (#1564)
  • Fix broken header formatting. Contributed by @midnightveil. (#1578)
  • Remove spurious mention of a "default port" with respect to SRV record lookup. (#1615)
  • Switch to ordered list for server name resolution steps. (#1623)

Application Service API

Spec Clarifications

  • Fix type of custom fields in thirdparty lookup queries. (#1584)

Identity Service API

Spec Clarifications

  • Make sure examples types match schema in definitions. (#1563)

Push Gateway API

No significant changes.

Room Versions

Backwards Compatible Changes

  • Add room version 11 as per MSC3820. (#1604)
  • Move redacts from top level to content on m.room.redaction events in room version 11, as per MSC2174. (#1604)
  • Remove creator from m.room.creator events in room version 11, as per MSC2175. (#1604)
  • Remove remaining usage of origin from events in room version 11, as per MSC3989. (#1604)
  • Update the redaction rules in room version 11, as per MSC2176 and MSC3821. (#1604)

Appendices

Backwards Compatible Changes

Spec Clarifications

  • Clarify spec re canonical JSON to handle negative-zero; also, give an example of negative-zero and a large power of ten. (#1573)

Internal Changes/Tooling

Backwards Compatible Changes

  • Upgrade Swagger data to OpenAPI 3.1. (#1310)
  • Create @matrix-org/spec npm package to ship the SAS Emoji data definitions & translations. (#1620)

Spec Clarifications

  • Update the CI to validate the file extension of changelog entries. (#1542)
  • Disclosure sections now only display their title when collapsed. (#1549)
  • Fix the sidebar in recent versions of Hugo. (#1551)
  • Bump jsonschema to validate JSON Schemas against Draft 2020-12. (#1556)
  • Use Redocly CLI to validate OpenAPI definitions. (#1558)
  • Use tag name as the OpenAPI definition version. (#1561)
  • Make sure version in x-changedInMatrixVersion is a string. (#1562)
  • Clarify usage of ABNF for grammar in the documentation style guide. (#1582)
  • Remove unnecessary oneOfs in JSON schemas. (#1585)
  • Update the version of Hugo used to render the spec to v0.113.0. (#1591)
  • Fix rendered changelog with new version of towncrier. (#1598)
  • Improve the layout of tables on desktop displays. Contributed by Martin Fischer. (#1601)

This Week in Matrix 2023-08-18

18.08.2023 00:00 — This Week in Matrix MTRNord

Dept of Spec 📜

TravisR announces

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

MSCs in Final Comment Period:

Accepted MSCs:

Closed MSCs:

  • No MSCs were closed/rejected this week.

Spec Updates

This last week the SCT has largely been preparing for the spec release happening on August 23rd, 2023 and working on getting some of the IETF/MIMI work into MSC shape. It's largely business as usual at the moment for the SCT :)

Matrix 1.9's planned work will be finalized on Monday as well, ahead of the Matrix 1.8 release on Wednesday. Please raise any MSCs or general feature areas to the SCT before Monday in #sct-office:matrix.org for them to be considered. The SCT will have limited/no bandwidth to look at things not raised for consideration.

Continue reading…

This Week in Matrix 2023-08-11

11.08.2023 00:00 — This Week in Matrix Hubert Chathi

Matrix Live

Dept of Spec 📜

TravisR says

Spec

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

MSCs proposed for Final Comment Period:

MSCs in Final Comment Period:

Accepted MSCs:

  • No MSCs were accepted this week.

Closed MSCs:

  • No MSCs were closed/rejected this week.

Spec Updates

We have a release date planned for Matrix 1.8! We're looking at Wednesday, August 23rd, 2023, and tracked as issue #1614. Currently the only release blocker is room version 11, which should land well in advance of August 23rd. If there's other things we should be considering please raise them ASAP in #sct-office:matrix.org.

August 23rd also begins the Matrix 1.9 cycle where we'll be sticking to our MSC review plan more strongly. Stay tuned to TWIM for news on the exact MSCs/features we'll be looking at for that cycle, and let us know in #sct-office:matrix.org if you think we should consider something in our planning.

The SCT has otherwise been thinking a lot about the MIMI working group at the IETF and how the protocol layering works there. About half of the SCT is going to take a break from MSC review over the next few weeks to ensure the protocol we're designing for MIMI will be fully compatible with Matrix - this will mean that some MSCs will move slower through FCP, sorry.

As always, if you have questions, concerns, complaints, etc then let us know in #sct-office:matrix.org 🙂

TravisR also announces

port 8448 has formally been registered by IANA 🎉

Continue reading…

Libera.Chat bridge temporarily unavailable.

04.08.2023 22:30 — Bridges Neil

Following a series of stability issues, the Libera.Chat team has requested that the Matrix <> Libera.Chat bridge be disabled until we can resolve the stability issues.

From 14:00 UTC on Saturday 5th August the bridge will be unavailable. We will be working to get the bridge back up as soon as we can, however, given the severity of the situation we do not expect immediate resolution.

We send our sincere apologies to anyone caught up in this decision and unable to reach folks on the Libera side.

We’ll get you back as soon as we can.

This Week in Matrix 2023-08-04

04.08.2023 18:19 — This Week in Matrix MTRNord

Matrix Live

Dept of Spec 📜

Andrew Morgan (anoa) says

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

  • There were no new MSCs this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

Accepted MSCs:

  • No MSCs were accepted this week.

Closed MSCs:

  • No MSCs were closed/rejected this week.

Spec Updates

No movement through the process on the surface for any MSCs according to the above chart, but some things have been happening! Other than the usual background hum of IETF work, conversations across many MSCs have been moving along. We also saw MSC3930 (Polls push notifications) have FCP proposed! The latter would stop a notification from being generated every time someone voted in a poll, which is sorely needed.

A reminder that in keeping with the spec's quarterly release schedule, Matrix v1.8 is due to release this month and Matrix v1.9 is due for November. We want to plan well ahead for the v1.9 release though, so if you would like to see anything in particular land in v1.9, please raise that concern in the Office of the Spec Core Team room!

See this message in the same room for more information including the currently planned v1.9 spec changes.

Random MSC of the Week

The random MSC of the week is... Refine and clarify how presence works!

This is a very old "MSC" (still on google docs), but it's come up and I've seen folks taking a look at revamping presence recently, so I figured it may be interesting to share.

The document lists a number of confusing behaviours that come with the current presence spec (at the time, though it hasn't moved much since then). There is also a bullet-point list of what a redesigned presence could look like.

Given the conversation on the GitHub issue, this document appears lost to time. But perhaps someone will find it useful today.

Continue reading…

Disclosure: Bridges security issues

04.08.2023 10:30 — Bridges Integrations Team , Matrix Security Team

Hi folks. As previously mentioned on Monday, we’re now disclosing the vulnerabilities patched for the IRC, Slack and Hookshot bridges. If you have not already done so, please ensure you are running the patched versions.

Today we are disclosing the 3 vulnerabilities.

matrix-appservice-bridge doesn't verify the sub parameter of an openId token exchange (CVE-2023-38691)

GHSA-vc7j-h8xg-fv5x / CVE-2023-38691

The POST /v1/exchange_openid endpoint did not check that the servername part of the sub parameter (containing the user's claimed MXID) is the same as the servername we are talking to. This could allow a malicious actor to spin up a server on any given domain, respond with a sub parameter according to the user they want to act as and use the resulting token to perform provisioning requests.

This is now patched so that the server part of the sub / user ID is checked against the server used to make the request.

Discovered and reported by a community member.

IRC command injection via admin commands containing newlines (CVE-2023-38690)

GHSA-3pmj-jqqp-2mj3 / CVE-2023-38690

When the IRC bridge attempted to parse an admin command from a Matrix user, it would only split arguments by a literal space. For example, sending “!join #matrix\nfoobar” would treat the channel name as “#matrix\nfoobar”. This could then be exploited to inject any IRC command into the bridge to be run. Since the !join command first joins via the bridge bot user, it could be used to execute commands as the bridge bot.

This is now patched so that both the command handler is more strict about its arguments, as well as channel names being explicitly validated when provided by users.

Discovered and reported by Val Lorentz.

Events can be crafted to leak parts of targeted messages from other bridged rooms (CVE-2023-38700)

GHSA-c7hh-3v6c-fj4q / CVE-2023-38700

The IRC bridge caches recent timeline messages in memory, so that when a reply is seen for a message it doesn’t need to request the event content from the homeserver. However the room ID was not validated when accessing this cache, so a malicious actor could craft a reply event in another room referencing any event ID (so long as it was still in the bridge cache) to trick the bridge into posting the message content into a bridged reply.

Discovered and reported by Val Lorentz.

If you have further questions, please reach out on [email protected]

Bridges Security Update

31.07.2023 11:40 — Bridges Integrations Team , Matrix Security Team

Today we are announcing security updates for several of our bridges.

In addition we have released matrix-appservice-bridge 9.0.1 (and backported to 8.1.2) which patches GHSA-vc7j-h8xg-fv5x.

All mentioned bridges are affected by a vulnerability in the provisioning interfaces of these bridges. If you are unable to upgrade, please disable provisioning for now (which should be documented in the relevant bridge sample config).

Continue reading…

This Week in Matrix 2023-07-28

28.07.2023 18:00 — This Week in Matrix Thib

Matrix Live

Dept of Spec 📜

Andrew Morgan (anoa) says

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

MSCs in Final Comment Period:

Accepted MSCs:

  • No MSCs were accepted this week.

Closed MSCs:

  • No MSCs were closed/rejected this week.

Spec Updates

We've been quite busy at IETF 117 this last week discussing MLS and MIMI in several contexts, meetings, and sessions. Overall things have moved pretty fast in the last week, but the short summary is we're working with MIMI to get (Linearized) Matrix used as the new-found "signalling layer". This layer delegates membership of the room to the crypto layer when the crypto layer (namely MLS) supports being used as such, and is responsible for enforcing all policies. Policies in the context of MIMI are things like join rules, history visibility, and power levels, but with an added twist: we're looking at supporting Role-Based Access Control (RBAC) in combination with power levels in MIMI, which should also bring RBAC to Matrix in the form of a currently-unwritten MSC.

All told, we've got several new documents to write and MSCs to draft, but we'll get there in time. The MIMI working group is expecting solutions in place by about September, so watch this space for more news as we progress. An architecture draft is also in progress on the MIMI side to further explain what all of these new layers mean. In the meantime, if you have questions then please visit the matrix-spec room on Matrix!

We're also looking for more Matrix 1.9 candidates. Currently we have just custom emoji and anything to do with MIMI on the agenda - if you'd like to add more, let us know in the Office of the Matrix Spec Core Team room on Matrix.

Random MSC of the Week

The random MSC of the week is... MSC3062: Bot verification!

This MSC describes a method for verifying (cross-signing) the devices of a bot user, and how verification of that sort could be done. Obviously it wouldn't make much sense to verify emoji with a bot. Instead, this MSC suggests that the bot provide a URL to present to the user. If the URL appears trustworthy (those who would control this URL should also be in charge of this bot), then the user can choose to continue the verification.

The user's Matrix client would then make a request to the URL with details of the verification. If the server responds successfully, some cryptographic magic happens, and your client will consider the bot verified!

This is essentially tying a bot's verification with control of a domain's DNS, which I think is a smart way to do things. But you do need to watch out for those pesky UTF-8 control characters when asking the user to verify the URL!

Continue reading…

Postponing the Libera.Chat deportalling

28.07.2023 14:00 — Bridges Thib

We have recently announced that we will be honouring Libera Chat’s request to turn off portalled rooms on the Libera.Chat bridge maintained by the Matrix.org Foundation. The changes were originally scheduled to be effective on 31st July. In the meantime, we posted instructions for people to turn their portalled rooms into plumbed ones so the bridge keeps working for them.

Some stability issues on the bridge have prevented people from turning their portalled rooms into plumbed ones. We have been actively working on resolving those issues since the first reports and the situation is gradually improving. However, at this point, we do not believe the plumbed mode can be considered sufficiently stable yet.

Continue reading…

This Week in Matrix 2023-07-21

21.07.2023 00:00 — This Week in Matrix Hubert Chathi

Dept of Spec 📜

Andrew Morgan (anoa) says

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

MSCs in Final Comment Period:

  • No MSCs are in FCP.

Accepted MSCs:

Closed MSCs:

  • No MSCs were closed/rejected this week.

Spec Updates

This week we have been preparing for IETF117, Matrix 1.8, Matrix 1.9, and Messaging Layer Security (MLS) for Matrix. Most of our work on globally interoperable communications is ongoing through the More Instant Messaging Interoperability (MIMI) working group at the IETF, and will be making significant strides in the coming days as we head to the IETF117 hackathon and meetup.

Over the last few months we've been working on a version of Linearized Matrix which supports the simplicity of linear event history while being fully compatible with today's Matrix network, and while we think that the 03 draft we wrote up accomplishes a lot of this, there's further work to be done to make it cleaner and easier to use. We've also been writing implementations of it to prove the semantics (and find areas which need improvement), starting with our cleanroom eigen-server TypeScript implementation and interoperating it with a branch of Synapse. During IETF117 we expect more implementations to sprout and have their interoperability tested - watch this space for updates on how that goes.

Aside from IETF117, we're continuing to look at the previously-selected Matrix 1.8 MSCs for release in mid-late August 2023. This might be slow over the next couple of weeks while half of us are at IETF117, but expect more forward progress when we get back. Matrix 1.9 is scheduled to be released sometime in November 2023, and a few months ago we said we were aiming to plan ahead for releases a bit more deliberately. Starting this week, we're accepting submissions for ideas and specific MSCs which need our attention in Matrix 1.9. If you have an MSC (current or future) which will need Spec Core Team (SCT) attention between August 2023 and November 2023, let us know in the SCT Office room. Once Matrix 1.8 is released (exact date TBD) we will have limited availability to add things to the Matrix 1.9 target - please raise your MSCs & themes as soon as possible. The current set of MSCs up for consideration can be found on the SCT Intake Board.

If you've made it this far in our weekly update, congratulations, and thank you. We expect things will rapidly start to happen with IETF117 kicking off tomorrow (July 22, 2023), and we will do our best to keep folks updated. Next week's TWIM in particular will have a post-IETF117 debrief for your reading enjoyment :)

As always, if you have any questions or concerns about what we're working on, visit the SCT Office and let us know. We can't promise a prompt reply (particularly during IETF117), but we will take a look when we can.

Random MSC of the Week

The random MSC of the week is... MSC3105: Previewing UIA flows!

This MSC addresses a shortcoming in the current User-Interactive Authentication (UIA) mechanism where attempting to deduce the required authentication flows for an action will result in that action being carried out if it turns out no flows were required. This makes it tricky for a client to present a "are you sure you want to do X?" as a final step in completing an action that requires authentication.

The proposals aims to allow an OPTIONS pre-flight HTTP request to the same endpoint in order to retrieve the flows necessary, without actually carrying out the action. The proposal does note that using OPTIONS for this case is a bit non-standard though, and some clients may treat the typical 401 error code returned during User-Interactive Auth as a fatal error.

While this does address a flaw in the UIA system, it's worth noting that many other flaws exist! Matrix is planning to move over to an OpenID Connect-based authentication system in the not too distant future, which will likely have far fewer edge cases than our traditional, home-grown one. You can visit https://areweoidcyet.com/ for more information and to track the current progress on that front.

Continue reading…