Matrix Live 🎙
Dept of Spec 📜
Andrew Morgan (anoa) says
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
- MSC3861: Matrix architecture change to delegate authentication via OIDC
- MSC3860: Add media download redirect proposal
- MSC3859: Add well known media domain proposal
- MSC3857: Welcome messages
MSCs in Final Comment Period:
- No MSCs are currently in FCP.
- MSC2832: HS -> AS authorization header
- MSC3818: Copy room type on upgrade
- MSC3844: Remove unused policy room sharing mechanism
- MSC3786: Add a default push rule to ignore
This week saw the merge of three MSCs from first-time MSC authors! MSC3818 (Copy room type on upgrade) from @Mikaela and both MSC3786 (Add a default push rule to ignore
m.room.server_aclevents) and MSC3827 (Filtering of
/publicRoomsby room type) from @SimonBrandner. Nice work to both of them!
Random MSC of the Week
The random MSC of the week is... MSC2499: Fixes for Well-known URIs!
A bundle of small quality-of-life changes to the .well-known server and client endpoints. Removing some ambiguities, replacing old and deprecated endpoints as well as a potential size cap suggestion.
Looks like a good contender for one to push over the line given a bit of review and response!
TWIM: someone in #matrix:matrix.org complained that the image in the spec section is always the same, so I made this gif (actually webp) out of all those images:
Dept of Servers 🏢
We've started work on our HG HomeServer written in pure TypeScript, compilable as a single JS file, with no dependencies except NodeJS. It's intended for a special use cases when Matrix is used as a backbone for custom apps. It's lightweight, minimal and for the moment isn't even planned to support full Matrix spec. We might make it possible to run it on browser later. https://github.com/heusalagroup/hghs
Synapse is a Matrix homeserver implementation developed by the matrix.org core team
Another week, another release! Synapse 1.64.0 was released this week, featuring a host of new features, bugfixes, and internal changes aimed at reducing memory usage, increasing performance, and improving the developer experience. Check out the full list of changes here. In addition, work continues on faster room joins. The goal gets closer every day!
Second generation Matrix homeserver
This week we released Dendrite 0.9.0 and Dendrite 0.9.1. There are quite a few big changes, including an all-new caching model and several optimisations. This release also moves our baseline supported Go version up to Go 1.18.
The following changes are included across both releases:
- Dendrite now uses Ristretto for managing in-memory caches
- Should improve cache utilisation considerably over time by more intelligently selecting and managing cache entries compared to the previous LRU-based cache
- Defaults to a 1GB cache size if not configured otherwise
- The estimated cache size in memory and maximum age can now be configured with new configuration options to prevent unbounded cache growth
- Added support for serving the
/.well-known/matrix/clienthint directly from Dendrite
- Configurable with the new configuration option
- Refactored membership updater, which should eliminate some bugs caused by the membership table getting out of sync with the room state
- The User API is now responsible for sending account data updates to other components, which may fix some races and duplicate account data events
- Optimised database query for checking whether a remote server is allowed to request an event over federation without using anywhere near as much CPU time (PostgreSQL only)
- Database migrations have been refactored to eliminate some problems that were present with
gooseand upgrading from older Dendrite versions
- Media fetching will now use the
/v3endpoints for downloading media from remote homeservers
- HTTP 404 and HTTP 405 errors from the client-facing APIs should now be returned with CORS headers so that web-based clients do not produce incorrect access control warnings for unknown endpoints
- Some preparation work for full history visibility support
- Upgrades a dependency which caused issues building Dendrite with Go 1.19
- The roomserver will no longer give up prematurely after failing to call
- Removes the faulty room info cache, which caused of a number of race conditions and occasional bugs (including when creating and joining rooms)
- The media endpoint now sets the
Cache-Controlheader correctly to prevent web-based clients from hitting media endpoints excessively
- The sync API will now advance the PDU stream position correctly in all cases (contributed by sergekh2)
- The sync API will now delete the correct range of send-to-device messages when advancing the stream position
- The device list
changedkey in the
/syncresponse should now return the correct users
- A data race when looking up missing state has been fixed
/send_joinAPI is now applying stronger validation to the received membership event
- Fixes a crash that could occur during event redaction
/membersendpoint will no longer incorrectly return HTTP 500 as a result of some invite events
- Send-to-device messages should now be ordered more reliably and the last position in the stream updated correctly
- Parsing of appservice configuration files is now less strict (contributed by Kab1r)
- The sync API should now identify shared users correctly when waking up for E2EE key changes
- The federation
/stateendpoint will now return a HTTP 403 when the state before an event isn't known instead of a HTTP 500
- Presence timestamps should now be calculated with the correct precision
- A race condition in the roomserver's room info has been fixed
- A race condition in the sync API has been fixed
As always, please feel free to join us in #dendrite:matrix.org for more Dendrite-related discussion.
Homeserver Deployment 📥️
Helm Chart (website)
Matrix Kubernetes applications packaged into helm charts
This week has seen the usual updates to my Helm Charts - with element-web being updated to 1.11.2 and matrix-synapse to 1.64.0. Additionally, the matrix-synapse chart now also allows for adding entirely custom .well-known data - along with an example on how to use that for MSC1929.
Dept of Bridges 🌉
This bridge connects users on Matrix and Discord – or other platforms, if combined with other bridges. Earlier this year the community bridge has been adopted by the Matrix.org bridge team to give it some attention. Its most recent update dated back to December 2020 and some fixes waited for a new release.
Well, here it is! v2.0.0 Its breaking changes are the requirement of NodeJS 14 or newer and the usage of
npm install. Furthermore, the update introduces a changelog and rolls out the guidelines we use for developing other matrix.org bridges.
Dept of Clients 📱
Element Web/Desktop (website)
Secure and independent communication, connected via Matrix. Come talk with us in #element-web:matrix.org!
- Version 1.11.2 is out!
- „Start DM only on first message“ has landed on develop. Changing the DM flow from invite → message to message & invite at the same time. If you want to see it live, join our testing session at 12:00 BST on Tuesday
- New user onboarding experience is getting closer to release!
- You can also now disable sending read receipts on develop, coming out on 16 August in a release
- Finished porting end to end tests to Cypress! Also started looking into adding support for accessibility testing
- Final comment period for our new code style is over. Next up is updating our contribution guidelines to include changes
- We finished the task around improving tracking of unused code to reduce maintenance burden
Element iOS (website)
Secure and independent communication for iOS, connected via Matrix. Come talk with us in #element-ios:matrix.org!
- Version 1.8.24 is available on the App Store with our new Sign Up and Sign In flows. The update is rolling out slowly and should be available to everyone by Monday.
- The work on our new app layout is coming along nicely with much of it merged into the repo (but disabled behind a build flag).
- In-app notifications will now also be delivered to Notification Centre.
- Continuous improvements are being made to the Live Location sharing feature.
- Integration tests are now run for every PR on matrix-ios-sdk more than doubling the reported test coverage!
Element Android (website)
Secure and independent communication for Android, connected via Matrix. Come talk with us in #element-android:matrix.org!
- 1.4.31 is rolling out which includes the new and improved FTUE onboarding experience along with fixes for markdown lists no longer always starting from 1 and html entities showing up in messages.
- The new app layout is starting to materialise with PRs available on github for anyone interested in having a sneak peek!
- We’re continuing to make improvements to Live Location sharing and cross signing verification as well as investigating performance issues.
Dept of Non Chat Clients 🎛️
Development marches forward, with the focus on getting chat into a usable state and task's up as the first MVP app on top: Effektio now support sending multiple images at once in chat, has seen fixes to the chat in widescreen mode, and now support markdown for chat messages. Meanwhile device verification is drawing to a close with a few remaining UI fixes being implement and the state machine for tasks being fleshed out slowly but steadily.
Populus Viewer (website)
A Social Annotation Tool Powered by Matrix
After a short vacation, I've done some new work on populus-viewer. Most of this has been UX and bugfixes, but I've added one new feature that I wanted to share. MSC3775: Markup Locations for Audiovisual Media gives a spec for annotating audio and video on matrix, but it also allows you to annotate images. So, for completeness sake, I've added support for annotating image files in Populus! This might be useful for discussing publication layouts, product designs, or for teaching art history. So populus can now annotate: video, audio, images and pdfs.
As always, if you want to learn more, follow populus development, or discuss the future of decentralized social annotation on Matrix, come join us at #opentower:matrix.org.
Dept of Widgets 🧩
matrix-widget-api v1.0.0 was released yesterday, reflecting the fact that we're not expecting any big changes to the library's architecture for the foreseeable future, and that it's more or less ready for wider use.
It also comes with a couple of new features ✨ for widget authors: Sending and receiving to-device messages with MSC3819, and getting TURN servers from the client with MSC3846. Together, these features enable some new complex use-cases, such as doing VoIP from inside a widget. See Matrix Live for a preview of what that looks like with Element Call! 📹️
This is as good a time as any to mention that matrix-widget-api doesn't have to just be for web-based apps. If you're building mobile apps with Matrix and want to make use of widgets, running matrix-widget-api inside a web view to liaison with the actual widget can make it a lot simpler to start supporting widget API features. If you're curious, watch Element iOS+Android for upcoming examples of this.
Dept of VoIP 🤙
Element Call (website)
- This week we've been building on the initial work on the pion call server (SFU) from Sean, and have made our very first call through it with Element Call! It's still very early and there's still lots of work to do, but this will allow Element Call to scale up to much higher numbers of people.
- The VoIP team are also looking after widgets now, and in our quest to embed Element Call into the Element apps, we've hit the milestone of releasing matrix-widget-api 1.0.0. We're also making great progress towards embedding into Element Web.
Dept of SDKs and Frameworks 🧰
Matrix Client-Server SDK for Rust
This week saw a few major improvements on the dev-ex of the Rust SDK (among others): support for refresh tokens landed; we have a setup for integration tests against an actual synapse instance now (and started writing tests); our docs for
mainnow contain the js bindings, too; event handlers can now be unregistered and we've added an API for room specific event handlers. Further more, we've refactored the examples (now to be found in
/examplesrather than some sub-sub-directory no one finds) and created two more:
getting-startedcombines the autojoin and command-example with a lot more additional inline comment to explain what is going on and
custom-eventsshowcases how to use the sdk and ruma types to send any custom events (in that case a ping-ack example) over matrix!
This was also a week of fighting the ci. After the integration tests landed, coverage reporting broke and an investigation was kicked off to check whether llvm-cov is in a usable state for us by now (it is not), but a fix was found. We've also started regretting adding
npm-based workflows in our repo, as we've found ourselves at the end of an upgrade bug and the saw CI failing without us changing anything :shakes_fist_at_sky: . A fix was found quickly by pinning a sub-dependency. On a similar note, we've postponed merging the kotlin bindings until the android team is back on it and available to answer some questions we have, and the wasm-js bindings showed build failures on CI, which the original author will have to take a look at after coming back from vacation - thus delaying
crypto-jsa bit further.
👉️ Wanna hack on matrix rust? Go check out our
help wantedtagged issues and join our matrix channel at Matrix Rust SDK.
Elixir libraries related to the Matrix communications protocol.
It's been a while since I've made an update on Polyjuice Client Test. Since the last update:
- several more tests have been added
- there have been some some front-end improvements
- I've also improved the documentation, done some refactoring, and added some helper/utility modules to make things clearer. My goal is to make it easier for people who don't know Elixir to be able to read and write tests without running away screaming.
- compatibility with some Matrix clients has been improved
- added some functions to make it easy for tests to completely override Matrix endpoints
I'm also happy to say that Polyjuice Client Test has helped find bugs in some clients, which have since been fixed.
Dept of Services 🚀
Matrix Public Archive (website)
madlittlemods (Eric Eastwood) announces
This week I'm showing off an early look at the Matrix public archive. As the name suggests, it acts as an archive of history for your world-readable Matrix rooms. This allows you to view historical content day-by-day and jump back years ago to see what your Matrix room was up to.
More importantly, it also allows Google to do the same thing so you’ll probably start finding Matrix content from your favorite search engine and be able to harness the massive knowledge base stored in Matrix. Imagine seeing Matrix logs instead of Stack Overflow answers when googling questions! The new portal into the Matrix ecosystem 🌌
You can also see this demoed in this week's Matrix Live video (at the top of the page).
Under the hood, we use the MSC3030
/timestamp_to_eventendpoint to fetch the messages for a given day and then we sever-side render the events with the Hydrogen SDK. Re-using Hydrogen gets us pretty and native(to Element) looking UI and keeps the maintenance burden of supporting new event types in Hydrogen.
If you want to follow what’s going on and see how it's coming along, you can checkout the project on GitHub, https://github.com/matrix-org/matrix-public-archive
Dept of Bots 🤖
Matrix Registration Bot (website)
moanos [he/him] says
Thanks to @lifeofbrian:justprojects.de and @ben:rs485.network the Docker image of the bot was shrunk from 406.28 MB to 69 MB (nice)! You can get the new image from Dockerhub or install it via the matrix-docker-ansible-deploy.
Dept of Guides 🧭
I was able to to host a matrix-synapse server on termux app on an android device. I documented the process on termux-synapse github repo And started working on a script that automates the process (final commit is being currently tested before pushing). It was a "for fun" type of thing. But I can see it being useful for people who do not have access to a raspberry pi (such as myself at the moment) to use as a small homeserver. It can hold up in 1 to 1 Direct Messages and in small rooms.
Room of the Week 📆
ssorbom ⚡️ reports
Have you ever felt lost in the Matrix world? Too many rooms and spaces to manage? Well, back by popular demand (with Timo's blessing), I present, The Room of the Week! Every week we strive to highlight a room or a space that we believe deserves attention for discussing interesting goings on across the Matrix Network.
This week, we are highlighting: #libregaming-games:tchncs.de
A matrix space dedicated to finding all of the free open source games, engines, and assets in the Matrix world so that you don't have to. Helpfully organized, and well maintained, it is the Premier stop for open source gaming on The Matrix Network!
If you know of a room that you would like to see highlighted, please visit https://matrix.to/#/!bIyiUUnriVoHtYzuPS:fachschaften.org to let us know of the room that you would like to spotlight.
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!