Hey all,
It’s finally here: threads, edits, and private read receipts. v1.4 has been a little later than usual in the quarter because we wanted to make sure we nailed down all the core MSCs for threads before publishing the spec itself, but we’ve done that now and we’re excited about it.
Hey all,
It’s another quarter and therefore another spec release! Matrix 1.2 was released back in February, and while a bit late in the quarter this time around we’ve still got some exciting additions coming to you in this release.
Like last time, the speed of these releases might feel a bit quick for developers: fret not if you’re still working on v1.1 or v1.2 implementations. We’re not expecting that implementations update as soon as a new spec release is published, but rather that the ecosystem gradually update over the course of the next few months. Implementations should be aiming for as close to realtime as they can get though, particularly considering v1.4 is expected to have some large features (more on that later on).
Matrix 1.3 sees 14 MSCs get merged, but we can’t possibly go into detail on them all here. We’ve picked some notable highlights and recommend the full changelog at the bottom for a complete idea of what’s been going on.
It’s no secret that MSC1849-style server-side aggregation of related messages have been in the review backlog for a while. We ended up splitting MSC1849 down into more reviewable chunks like MSC2675, allowing us to finally land the first pieces into Matrix 1.3 today.
In the spec there aren’t currently any defined relationships which make use of aggregations or even the rel_type described by MSC2674, but we do expect that v1.4 of the spec will have support for at least threads (MSC3440) and edits (MSC2676), filling the gap. For now, we’ve decided that holding aggregations back until v1.4 doesn’t make a lot of sense, so we have launched them into the world as-is for custom relations or for clients & servers to prepare for what’s expected to be coming in v1.4 of the spec.
To further prepare for threads, we’ve also removed some restrictions of rich replies through MSC3676, thus allowing replies to be constructed with non-text messages like images. Check out the new rich replies module for more information.
When we launched restricted rooms in Matrix 1.2 we noted that we forgot to handle a case where someone might want to support both knocking and restricted rooms at the same time. We’ve fixed that with a stop-gap join rule from MSC3787 in room version 10.
The new knock_restricted join rule allows the room to keep its desire to be restricted whilst also allowing members who do not meet the criteria to knock on the room instead. We’ll likely expand on this sort of mixing of join rules in proposals like MSC3386 down the line, however for now this should cover the gap in support. Next up: figuring out how to make encrypted room history available to these new joiners in a safe way.
Originally planned for this release, MSC3440-style threads are anticipated to be ready for v1.4 (next quarter) with support for notifications and, of course, a whole new way to communicate in a room.
Threads are one of the more complicated features we’ve tried to land in recent history as it has a large impact on a wide variety of the spec: everything from event relationships (fixed in this release) to read receipts to push notifications needs to be worked out to build a system that not only users can understand, but also the developers trying to build it. With this massive surface area we just weren’t comfortable with adding threads to v1.3 as-is, given problems like notifications aren’t yet fully solved.
Alongside threads, we also anticipate that MSC2676-style message editing will be landing in v1.4, finally specifying how to update an event’s contents without having to redact & re-send. Although message editing has been supported for a long time in some clients, we're excited that it will finally become part of the official specification, meaning it can be implemented more widely. Messaging clients are encouraged to give the proposal an early read and possibly even attempt implementation if not already done to help us ensure the system is in a reasonable state for inclusion in the spec, though we do note that feature requests for edits will likely be deferred to a future MSC.
Keep an eye on This Week In Matrix for updates on what v1.4 is expected to include, and how things are progressing.
MSCs are how the spec changes in the way it does - adding, fixing, and maintaining features for the whole ecosystem to use. The blog post can’t cover them all, but that doesn’t make them any less important! Check out the full changelog below, and the Spec Change Proposals page for more information on how these MSCs got merged (hint: they submitted a proposal, which anyone can do - take a look at the Matrix Live episode where Matthew covers the proposal process).
Deprecations
sender_key and device_id on m.megolm.v1.aes-sha2 events, and the sender_key on m.room_key_request to-device messages, as per MSC3700. (#1101)Backwards Compatible Changes
from optional on GET /_matrix/client/v3/messages to allow requesting events from the start or end of the room history, as per MSC3567. (#1002)knock_restricted join rule in supported room versions, as per MSC3787. (#1099)Spec Clarifications
m.room.avatar events is not required. (#987)type in user-interactive authentication can be omitted. (#989)Flow information is explicitly defined when the client-server API is rendered. (#1003)invite where it is not specified in an m.room.power_levels event. (#1021)matrix-doc github repository. (#1032)m.room.message.feedback per MSC3582. (#1035)@ are in fact reserved. Regressed from #3658. (#1100)name field of m.room.name events. (#3669)room_id field from examples of m.read, m.typing in /sync and m.fully_read in room account data. (#3679)event_match in push rule conditions. (#3690)m.login.appservice login identifier, instead using m.login.application_service. (#3711)invite->knock and external->leave are valid transitions. (#3730)Backwards Compatible Changes
Spec Clarifications
origin field from PDUs. (#998)matrix-doc github repository. (#1032)valid_until_ts is in milliseconds, like other timestamps used in Matrix. (#1055)/send_join response. (#3703)content for X-Matrix signature validation is the parsed JSON body. (#3727)Backwards Compatible Changes
No significant changes.
No significant changes.
Backwards Compatible Changes
knock_restricted join rule supported by room version 10 as per MSC3787. (#1099)Spec Clarifications
m.federate to the authorization rules, as originally intended. (#1103)join_rule is knock. (#3737)No significant changes.
Hey all,
Welcome to the second installment of our quarterly spec releases. If it feels like it hasn’t been long since our last release, you’re not alone! Our last release was just 3 months ago, introducing the new platform and world we build within.
This improvement in speed might seem too fast, but fret not: implementations are not expected to update as soon as the new spec is published. Rather, it is more realistic to expect that the ecosystem gradually update over the course of the following few months/year. Particularly after the massive release that was v1.1.
So, what’s new in v1.2? With 18 MSCs merged there’s a lot to cover - we’ve picked some notable highlights and recommend the full changelog at the bottom for a complete idea of what’s been going on.
Spaces launched into beta last May, redefining how we can use rooms on Matrix to represent different data structures. Described mostly as MSC1772, Spaces are simply rooms with a specific type in their m.room.create event. With state events being used to define which other rooms (meaning Spaces too) are part of that Space, the possibilities for tree-like structured data become endless.
There’s still quite a lot of work to do in the Spaces space (hah), though we’re excited to see it all land. For instance, MSC3216 and MSC2962 target power level syncing, MSC3219 aims for flair, and MSC3089 looks at file structures using Spaces. We might even be able to replace the public room directory with a server-wide space, making writing clients a little bit easier.
Restricted rooms are new in this release from MSC3083. In these rooms (and therefore Spaces too!), the join rules can be configured such that a member of another room can join without needing an invite. In a typical setup, this means that a Space could be set as invite-only, but all the rooms and Spaces underneath that Space could allow joins for members of the parent Space. When someone wishes to explore the universe you’ve laid out in your Space they can simply join the interesting rooms without having to ask for invites constantly.
This feature changes how membership events are authorized, so is only available in room versions 8 and 9 (both introduced in this release). Room version 9 fixes a relatively rare bug from version 8, so we’d recommend using v9 if you’re planning an upgrade for this functionality.
Further work in this area involves figuring out how to keep membership lists perfectly in sync between the rooms, which is currently done by external tooling. For example, evicting someone from the parent Space could (optionally) evict the user from all the subsequent rooms and Spaces too.
We also need to figure out how to support both knocking and restricted rooms at the same time (oops). MSC3613 and MSC3386 both tackle this problem in different ways and timescales.
A massive shoutout goes to kitsune and the whole community for working on MSC2312, giving us a URI we can pass around outside of Matrix to bring us back in. The early work on this dates all the way back to 2014, the very beginning of Matrix’s development, and has since been marked Provisional by the IANA.
The full spec is available here - feel free to discuss it at matrix:r/matrix-spec:matrix.org ;)
MSCs are how the spec changes in the way it does - adding, fixing, and maintaining features for the whole ecosystem to use. The blog post can’t cover them all, but that doesn’t make them any less important! Check out the full changelog below, and the Spec Change Proposals page for more information on how these MSCs got merged (hint: they submitted a proposal, which anyone can do - take a look at the Matrix Live episode where Matthew covers the proposal process).
Breaking Changes
prev_content field is now returned inside the unsigned property of events, rather than at the top level, as per MSC3442. (#3524)aliases property from the chunks returned by /publicRooms, as per MSC2432. (#3624)New Endpoints
GET /_matrix/client/v1/rooms/{roomId}/hierarchy) as per MSC2946. (#3610)/_matrix/client/v1/register/m.login.registration_token/validity as per MSC3231. (#3616)Backwards Compatible Changes
/_matrix/client/r0/login to accept a m.login.appservice, as per MSC2778. (#3324)restricted rooms as per MSC3083, MSC3289, and MSC3375. (#3387)is_guest to /account/whoami as per MSC3069. (#3605)m.set_displayname, m.set_avatar_url, and m.3pid_changes capabilities as per MSC3283. (#3614)Spec Clarifications
AesHmacSha2KeyDescription consistent with KeyDescription in marking name as optional. (#3481)m.location events (#3492)403 M_FORBIDDEN error code to /profile/{userId} as per MSC3550. (#3530)/sync API, including a fix to an ASCII art diagram. (#3543)base_url in client well_known may or may not include trailing slash. (#3562)m.olm.v1.curve25519-aes-sha2 messages. (#3573)New Endpoints
GET /_matrix/federation/v1/hierarchy/{roomId}) as per MSC2946. (#3610, #3660)Backwards Compatible Changes
Spec Clarifications
GET /_matrix/federation/v1/event_auth/{roomId}/{eventId} does not return the auth chain for the full state of the room. (#3583)Spec Clarifications
GET /_matrix/app/v1/thirdparty/protocol/{protocol}. (#3675)Backwards Compatible Changes
Spec Clarifications
Spec Clarifications
Backwards Compatible Changes
Spec Clarifications
unsigned data of a Federation PDU. (#3522)knock -> leave in v7, v8, and v9. (#3694)Backwards Compatible Changes
Hey all,
Once again it's been a little while since we've done a spec release (sorry; we're aiming for quarterly releases from here on out), but this time we have some pretty big news: we've got an all-new spec platform and a new versioning scheme. The new spec platform has been needed for a long time to make better sense of what Matrix is, and as part of getting that out the door we reduced the number of "Matrix versions" to just one.
Huge thanks to Will Bamberg for building it out for us, anoa for working out the deployment details, and everyone for testing it all. They talk at length about what this specification even is and about the platform itself on Matrix Live S6E19. It's the single greatest improvement to the spec we've seen to date.
The new versioning scheme presents the whole specification as a single document, making it easier to check compatibility between implementations and the spec itself. Previously, a grid would have to be drawn to show whether Server-Server r0.1.4 is compatible with Client-Server r0.6.1 - while obvious at release time, the historical context gets lost quite easily. Now, with a single version number, the entire specification is compatible across a single version number: servers implementing Matrix 1.1 will be compatible with clients implementing v1.1, and vice versa. For the specification itself, this means we no longer have to carefully point and update links between the APIs, as they'll instead be versioned together.
Changing the versioning of the specification does have an impact on the Client-Server API in particular, however. You may have noticed that Client-Server APIs are currently versioned at "r0". By removing rX.Y.Z versioning, all of the endpoints suddenly don't have a version to reference. All endpoints across the specification are now versioned individually to allow for breaking changes at the endpoint level. They no longer require the whole specification to be listed as a breaking change: a v1 endpoint can get additions/changes which are backwards compatible, but if we need to change format (for example) then it'll get bumped up to v2, leaving v1 in its last known state.
For the Client-Server API, a slight complication is that v1 and v2 (alpha) are already versions familiar to those that have been around for a while - to avoid confusing those people, existing Client-Server API endpoints will start at v3. New endpoints introduced after v1.1 will start at v1 instead.
It's been well over a year since the last release, which means there's a whole lot of features and changes to cover. Here we'll try to cover the things most clients/servers will have to worry about, but we do still recommend reading through the changelog included below. Overall, 28 MSCs have been merged through this release, but some have had to be excluded in the interest of getting the spec release out there. Many of them are expected to be in the next anticipated release (which should hopefully be next quarter).
In Matrix 1.1, client developers get all sorts of new features to play with. Clients which support end-to-end encryption should explore key backups, cross-signing, SSSS, and breaking changes to verification. Quite a lot of this stuff has existed for a while, but has made it into the specification formally now. As an added bonus, the emoji for SAS verification have been translated (contribute here).
Knocking has also landed in the spec (thanks Sorunome for leading the charge on this!), opening up the ability for rooms to optionally allow people to request invites to join. This can be helpful for semi-private rooms where it can be easier to approve/deny requests compared to finding someone's MXID and manually inviting them. This does require a v7 room to work, however.
Thanks again to Sorunome, Message Spoilers have been officially included in Matrix. People can now safely discuss the ending to the latest movie without being banned for spoilers. Though, as a new feature, there's a chance that the spoiler text still gets included in the message: clients should update as soon as possible to avoid their users accidentally getting banned for spoiling the conclusion to the Spaces saga ;)
There's a few other smaller improvements/additions, and of course the regular "various clarifications and bug fixes" to take a look at. For a sample checklist, check out element-web's issue on the subject.
Room version 7 has landed, bringing forth the ability for users to knock on rooms (requesting an invite to join). The changes are largely scoped to using the reserved keywords for join rules and membership, and are described through the auth rules. Thankfully, the changes over v6 are minimally invasive so should be quick to implement.
Additionally, the cross-signing bits have been included in the API responses and EDU definitions. Matthew's blog post from last year (it really has been that long...) describes cross-signing and the history of its implementation.
As per usual, there's also various specification errors corrected to aid understanding. Synapse has an exhaustive issue to detail what servers might need to do.
PS: Room versions 8 and 9 are also out there, but will be included in a future spec release.
We haven't mentioned identity servers, bridges, etc in this post but they have changes too! Below is the whole changelog, the entire year and a bit of it. Thank you to everyone who has submitted MSCs, and congratulations on getting them released. If we forgot yours, please mention it in #matrix-spec:matrix.org so we can apologize and correct.
PS: The MSC process is how changes to Matrix are made, and you (yes, you) can propose those changes too. Check out the Matrix Live episode where Matthew talks about how this process works, and how we avoid blocking clients from implementing their proposals behind the relatively slow spec release cycles.
Breaking Changes
curve25519-hkdf-sha256 key agreement method for SAS verification, and deprecate old method as per MSC2630. (#2687)m.key.verification.ready and m.key.verification.done to key verification framework as per MSC2366. (#3139)Deprecations
m.key.verification.request as per MSC3122. (#3199)New Endpoints
/room_keys/*) endpoints as per MSC1219. (#2387, #2639)POST /keys/device_signing/upload and POST /keys/signatures/upload as per MSC1756. (#2536)/knock endpoint as per MSC2403. (#3154)/login/sso/redirect/{idpId} as per MSC2858. (#3163)Removed Endpoints
m.login.oauth2 and m.login.token user-interactive authentication mechanisms as per MSC2610 and MSC2611. (#2609)Backwards Compatible Changes
POST /keys/query as per MSC1756. (#2536)device_id parameter to login fallback as per MSC2604. (#2709)reason on all membership events and related endpoints as per MSC2367. (#2795)M_NOT_FOUND error to push rule endpoints as per MSC2663. (#2796)reason and score parameters optional in the content reporting API as per MSC2414. (#2807)color attribute as per MSC2422. (#3098)<details> and <summary> to the suggested HTML subset as per MSC2184. (#3100)m.login.sso as per MSC2858. (#3163)device_id to /account/whoami response as per MSC2033. (#3166)FAIL_PROMPT as per MSC2284. (#3169)v3 as a starting point instead of r0 as per MSC2844. (#3421)Spec Clarifications
age and unsigned being shown in the wrong places. (#2591)room_id from /sync examples. (#2629)titles. (#2647)m.key.verification.accept and secret storage. (#2653)highlight tweak for consistency. (#2670)state for /sync with lazy-loading. (#2754)m.room.redaction event. (#2814)messages as a required JSON body field in PUT /_matrix/client/r0/sendToDevice/{eventType}/{txnId} calls. (#2928)client_secret request body parameters so that they do not include invalid characters. (#2985)m.presence. (#3091)Access-Control-Allow-Headers recommendation to fit CORS specification. (#3225)replacement_room is a room ID in m.room.tombstone events. (#3233)/sync, /rooms/{room_id}/messages, /initialSync, /rooms/{room_id}/initialSync, and /notifications. (#3353)redacted_because is meant to work. (#3411)mimetype from EncryptedFile examples, as per MSC2582. (#3412)/versions endpoint. (#3420)threepid_creds. (#3471)New Endpoints
Backwards Compatible Changes
GET /user/keys and GET /user/devices/{userId}, m.device_list_update EDU, and a new m.signing_key_update EDU as per MSC1756. (#2536)Spec Clarifications
GET /_matrix/federation/v1/make_join/{roomId}/{userId} can return a 404 if the room is unknown. (#2688)/_matrix/federation/v1/user/devices/{userId} response which actually returns "self_signing_key" instead of "self_signing_keys". (#3312)<hostname> TLS certificate is needed rather than <delegated_hostname> for SRV delegation. (#3322)prev_events. (#3340)Spec Clarifications
New Endpoints
Removed Endpoints
Spec Clarifications
/versions endpoint. (#3459)Spec Clarifications
Hey all,
It's been a little while since we've done a spec release, so here we are with Room Version 6, Client-Server r0.6.1, and Federation r0.1.4.
Room Version 6 (and the associated Federation r0.1.4 release) is largely something for implementations to worry about. It contains new event authorisation rules, changes to the redaction algorithm, and stricter compliance for JSON.
Client-Server r0.6.1 contains a number of clarifications as well as SSO support for authorisation, "soft logout" to avoid needlessly destroying e2e history, and new ways to publish aliases within rooms.
If you're wondering where all the E2E-by-default related MSCs are - we're doing final iterations based on the real-world feedback from the E2E-by-default launch a few weeks ago, and they are then expected to land in the upcoming Client-Server r0.7.
Here's all the MSCs that got merged since the last release:
query_auth federation endpoint/rooms/{roomId}/aliases for retrieving local aliases for a room. (#2562).m.rule.contains_user_name default push rule to set the highlight tweak. (#2519)event_id is returned when sending events. (#2525)/createRoom. (#2571)POST /publicRooms endpoint for filtering the room directory. (#2305)/send_join and /send_leave endpoints per MSC1802. (#2547)prev_events and auth_events for PDUs. (#2538)/send endpoint. (#2560)/send_join. (#2575)Hey all,
For the last several months the team has been working on tightening up privacy in Matrix, and with the 1.4 release of Synapse and Riot quite a lot has been done in the area. One of the remaining pieces was to release all the specification changes to help other client/server implementations achieve the same goals, and now we've done that.
The Client-Server r0.6.0 and Identity Service r0.3.0 spec releases both cover the privacy improvements added through a number of MSCs in the last few months. Of particular note is that identity servers are now expected to support terms of service endpoints, which requires authentication that clients might need to worry about - check the spec changelogs for details.
The full changelog for the Client-Server r0.6.0 release is:
Breaking Changes
New Endpoints
POST /account/3pid/unbind for removing a 3PID from an identity
server.
(#2282)Backwards Compatible Changes
M_USER_DEACTIVATED error code.
(#2234)bind_msisdn and bind_email from /register now that the
identity server's bind endpoint requires authentication.
(#2279)m.identity_server account data for tracking the user's
preferred identity server.
(#2281)id_server and make it optional in several places.
(#2310)Spec Clarifications
m.room.message$m.notice schema.
(#2125)url field of certain
m.room.message msgtypes.
(#2129)m.key.verification.start and its
m.sas.v1 variant.
(#2132).m.rule.room_one_to_one push rule.
(#2152)/rooms/:roomId/event/:eventId returns a Matrix error.
(#2204)state_key check on .m.rule.tombstone.
(#2223)m.room_key_request action value, setting it from
cancel_request to request_cancellation.
(#2247)submit_url field is without authentication.
(#2341)/context.
(#2344)/sync.
(#2345)The full changelog for the Identity Service r0.3.0 release is:
New Endpoints
Backwards Compatible Changes