This year, the Matrix.org Foundation is excited to host the first ever Matrix.org Foundation and Community devroom at FOSDEM. A full day of talks, demos and workshops around Matrix itself and projects built on top of Matrix.
Matrix is the open source project that publishes the Matrix open standard for secure, decentralised, real-time communication, and its Apache licensed reference implementations.
We encourage people working on the Matrix protocol or building on it in an open source project to submit a proposal! Note that companies are welcome to talk about the Matrix details of their open source projects, but marketing talks are not welcome.
We want this devroom to be a space where the Matrix community can show its work, where developers can talk about the challenges they faced and how they overcame them, and where people can get a glimpse of the future of the Matrix protocol and ecosystem.
Talk Details
The talks will be pre-recorded in January. They will be played during FOSDEM, followed by a session of live Q&A depending on the format. During the playback of the talk, people will be able to comment and ask questions in the chat (via Matrix!).
The talks can follow one of three formats:
5 min lightning talk, ideal to showcase your project and make people want to have a look at it
20 min talk + 10 min Q&A, for topics that can be covered briefly
50 min talk + 10 min Q&A for more complex subjects which need more focus
We strongly encourage you to prepare a demo when it makes sense, so people can actually see what your work looks like in practice!
Of course, the proposal must respect the FOSDEM terms as well:
The conference language is English. All content must relate to Free and Open Source Software. By participating in the event you agree to the publication of your recordings, slides and other content provided under the same licence as all FOSDEM content (CC-BY).
We expect to receive more requests than we have slots available. The devroom organisers (two community members and one core team rep) will be reviewing the proposals and accepting them based on the potential positive impact the project has on Matrix (as defined in by the Mission section of https://matrix.org/foundation).
If a project proposal has been turned down, it doesn't mean we don't believe it has good potential. Maintainers are invited to join the #twim:matrix.org Matrix room to give it some visibility.
NOTE: Synapse 1.49, due out on December 14th, will be the last release of Synapse to support Python 3.6 or PostgreSQL 9.6 per our platform dependency deprecation policy. Accordingly, we will remove support for Ubuntu 18.04 LTS (Bionic) at the same date, as it ships with Python 3.6.
Password resets and identity servers
This release removes the long-deprecated trust_identity_server_for_password_resets configuration option. This option was initially deprecated in Synapse 1.4.0 back in October 2019.
Admins of servers still using this configuration option will need to update their Synapse configuration to send password resets through an SMTP server directly rather than relying on identity servers to send them on their behalf.
New admin APIs and improved alignment with Matrix 1.1
This release also introduces a handful of new admin APIs, allowing administrators to un-shadow-ban users, block a room, and run specific background updates (but we'll talk about this last one a bit later on). The delete room API has also been updated to be able to run in the background or to block a room pre-emptively, even if the server doesn't know about it yet.
This release also brings Synapse into greater alignment with version 1.1 of the Matrix specification by adding support for API paths beginning /_matrix/client/v3 and /_matrix/media/v3.
Background updates
When Synapse updates from one version to another, it might need to run large scale updates on its database. In order to avoid blocking startup for too long while waiting for these updates to run, Synapse runs them in the background after starting.
Lately the Synapse team has been doing some work to improve the performance of these background updates. More specifically, this release includes a performance fix for a background update introduced in Synapse 1.47.0, as well as a new admin API to let admins rerun specific updates.
This release also includes some improved support of MSC3440 to help threading. It also adds support for the stable identifiers from MSC2778, bringing Synapse closer to supporting end-to-end (or end-to-bridge) encryption support for application services.
We also now publish a Docker image, matrixdotorg/synapse:develop, which tracks the development head of Synapse.
Please see the Synapse Release Notes for a complete list of changes in this release.
Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including Dirk Klimpel, Stanislav Motylkov, Tulir Asokan and Neeeflix.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.
MSC2675 (serverside aggregations) is getting lots of updates from Bruno in order to align the proposal with what is currently implemented in the wild (as it's easier to iterate on incremental improvements from a starting point grounded in reality). This MSC is a bit of a special case though, as it was implemented with stable prefixes before the MSC landed (in the before times...).
Regardless, thank you very much to Bruno for going through and finally untangling and help land aggregations in the spec! This MSC is one of four which describe how aggregations should work in Matrix, and it's great to see them finally being properly spec'd, especially as further features start to be built on top of them (such as threading!).
This proposal aims to allow appservices to get ever greater visibility into what is happening on the homeserver, while still maintaining full process separation.
A lot of the time solutions to complex problems require knowing when a user has registered or logged in, with what name/3pids etc, when users change their emails, etc. Hooking into these non-room-based actions can help with developing useful features. Maybe you want an appservice that plays a sound effect whenever a user signs up to your homeserver!
These days, that is often achieved by homeserver implementation-specific solutions, such as modules in Synapse. Being able to notify of these events using a standard API shape would be hugely beneficial to generalisation of projects.
So give the proposal a look over and review if that interests you!
Some exciting MSCs this week, I've been waiting for server-side aggregations for long!
As predicted last week, we released Synapse 1.47.1 on Tuesday. This is a security release which fixes an issue with Synapse's built-in media repository. Admins are strongly encouraged to upgrade.
Otherwise, relatively slow week: a handful of of the team have been away (Happy Thanksgiving, Americans!), but we did release Synapse 1.48.0rc1. Most importantly, this release candidate includes changes to improve the efficiency of large background updates from past releases, which should significantly reduce database load when upgrading. It also adds support for the /v3 APIs defined in version 1.1 of the Matrix specification.
We'll talk more about 1.48 when it's formally released next week, but as always, we appreciate folks trying out the release candidates and letting us know how they behave.
Administrators, keep your users safe: update as soon as you can!
A bit of an earlier update this week, but I wanted to make sure to note that my Hem Charts have been updated to matrix-synapse 1.47.1 for the security fix - and element-web has also been bumped to 1.9.5
mautrix-googlechat has seen lots of improvements over the past few days. New features include:
Bridging edits, deletions, reactions, formatting and read receipts in both directions (even /rainbow somewhat works from Matrix)
Bridging typing notifications and any types of files from Matrix to Google Chat
Bridging Google Meet links from Google Chat to Matrix
Syncing group members from Google Chat
(edits and deletions are only available on Google Workspace accounts, not normal accounts. I have no idea why they did that, but that's just how Google Chat works 🤷)
There's still a bug where it sometimes silently stops receiving messages, which I'm currently trying to solve (or work around). After that I'll make a v0.3.0 release. Backfilling history may also happen in the near future
An add-on for the matrix-appservice-webhooks bridge. Webhooks are essentially web interfaces for applications to "push" data to. The bridge can receive messages in a certain format, which is nice if the notifying app can be configured. Often it cannot.
Do you like to receive notifications in matrix?
Matrix Webhook Receiver (MWR) is an add-on for the matrix-appservice-webhooks bridge. Webhooks are essentially web interfaces for applications to "push" data to.
The bridge can receive messages in a certain format, which is nice if the notifying app can be configured. Often it cannot.
This is where MWR comes in:
It can receive any (JSON) content, optionally reformat it nicely (customizable!), and forward it to the webhooks bridge which will post it to a room for you. If you are running any software service, there is a good chance it can notify you via webhooks!
Right now, several example configurations exist, ready for you to use:
Hey folks! Some exciting new news on the bridge front: I've renamed matrix-github to matrix-hookshot to better reflect it's not-just-GitHub-ness. That's not all though, as there are new features too:
The bridge now supports Rust as a companion language (we're aiming to rewrite critical sections in rust). Some parts of the formatting code have already been rewritten.
The bridge now supports JIRA (full puppeting!)
The bridge now supports generic webhooks too, with the ability to write custom handling code inside the state event to process these hooks into pretty messages.
Basic support for GitHub discussions.
In the works:
A provisioning API to hook into integration managers
More GitLab support
Better GitHub discussions support
We're not quite ready for a 0.2.0 release, but please check us out at https://github.com/Half-Shot/matrix-hookshot.
An interesting update, and Half-Shot even demoes it in today's Matrix Live!
After the 0.9.0 release last week, we have of course been busy fixing all the bugs different people reported. Messing around with the sticker pack editor and then leaving room should not make Nheko crash anymore. The problems where the flatpak has issues starting on Gnome systems are still under investigation. We thought we had a solution, but that seems to have broken other stuff! 💥
Apart from that we have been doing some after release party cleanup. Apart from some refactorings, you can now filter your rooms on whether they are a direct chat or not in the sidebar. This is in addition to the filters we already had for favourites, spaces and your other personal tags. User colors should also now be much less biased towards blue and jdenticons should have more variance. Expect the next release to be a much more colorful experience!
Speaking of colors, Twily made this awesome ZX Spectrum inspired logo after we changed our Gitlab bot to be more colorful! Check it out:
We're still distracted with SDK work and other things less visible for users, but this week we've also released 0.2.22 that fixes login on Element One (and other servers using SSO login and not yet supporting the experimental dehydrated devices).
simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio. Version 2.4.0 provides several new features and a fix.
New Features:
Newlines are now supported when sending markdown messages.
The msgtype of text and markdown messages can now be specified. Text and markdown messages can now optionally be sent as "m.notice" to avoid alerting everybody of the new message. The default msgtype will continue to be "m.text".
New Fixes:
Fixed issue where the homeserver was hardcoded in an http request.
Example usage is shown below:
import simplematrixbotlib as botlib
creds = botlib.Creds("https://home.server", "user", "pass")
bot = botlib.Bot(creds)
PREFIX = '!'
@bot.listener.on_message_event
async def echo(room, message):
match = botlib.MessageMatch(room, message, bot, PREFIX)
if match.is_not_from_this_bot() and match.prefix() and match.command(
"echo"):
response = " ".join(arg for arg in match.args())
await bot.api.send_text_message(room.room_id, response, "m.notice") ## Uses the msgtype of m.notice instead of m.text
bot.run()
A thank you to HarHarLinks for their contributions to version 2.4.0!
This week saw three releases of jOlm which fix a native memory management issue, an Olm API (buffer) issue and add a few other improvements. Everyone is strongly encouraged to update to the latest release.
maubot v0.2.0 was released last weekend. Highlights:
Enabling encryption should be much easier: the device ID can be entered in the web UI or you can just do mbc auth --update-client to automatically log in and store the access token and device ID in maubot.
mbc auth can now log in with SSO.
The standalone mode for running a single plugin with a static config is now mostly functional and somewhat documented.
Also, I finally took a day to figure out Sphinx/autodoc and made some decent-looking autogenerated docs for mautrix-python. I'll probably extend that to generate maubot-specific API references too eventually.
Federated sign-in component for your web app (using Matrix)
This week's update:
Gained 180 stars on GitHub since release (thanks!)
Added login states, accessible from the API
Sign out
Added CSS styling via variables
Updated demo
more on https://github.com/mishushakov/signin-with-matrix
As last week, a note to keep in mind that this is a community project and that there is a MSC to make Matrix more OAuth2 friendly. More on that very soon!
Today is a good day for those calling me a spy, someone not wanting to care about privacy and for those who did publicly harass me for server_stats.
People using the API likely already noticed it wasn't reachable for a while. Effective immediately I am currently leaving all rooms the bot is part of. This will take days or even months considering this are 6397 rooms at the time of writing. I am not going into the motives of why I am shutting it down. It comes down to personal reasons.
There won't be any dump of the data. The source will be kept public. Note though if anyone ever tries to run it that you need about 600GB of space for synapse, a lot of CPU, a lot of RAM and plenty of workers as this can easily crash synapse.
Server_Stats was an incredibly useful project. It pains me a lot to see it go, but it pains me even further that its author got harassed. This is not an acceptable behaviour, and we are better than that as a community. Thanks for this incredible project MTRNord, it's been both exciting and useful.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Today we are releasing Synapse 1.47.1, a security update based on last week's release of Synapse 1.47.0. This release patches one high severity issue affecting Synapse installations 1.47.0 and earlier using the media repository. An attacker could cause these Synapses to download a remote file and store it in a directory outside the media repository.
Attackers cannot control the exact name or destination of the stored file.
To quote from the advisory:
GHSA-3hfw-x7gx-437c / CVE-2021-41281: Path traversal when downloading remote media.
Impact
Synapse instances with the media repository enabled can be tricked into downloading a file from a remote server into an arbitrary directory, potentially outside the media store directory.
The last two directories and file name of the path are chosen randomly by Synapse and cannot be controlled by an attacker, which limits the impact.
Homeservers with the media repository disabled are unaffected. Homeservers configured with a federation whitelist are also unaffected.
The advisory has full details, including workarounds.
This issue was discovered and fixed by our internal security team.
Friday already? Did this week already happen? It looked like the spacetime continuum was broken and we didn't know who did it. We needed witnesses to solve the case: did things really happen this week? And the witnesses showed up! A huge thanks to everyone in the Matrix community who reported their progress, and to everyone currently working on making awesome projects around Matrix!
The case is closed: the week has not been stolen from us. Time appears to have wings, and flies faster than we had anticipated.
Matrix Live 🎙
A very very dense and exciting wrap up of what's happening these days in the Matrix space by Matrix Foundation co-founders Matthew & Amandine! Matrix is stepping up a gear with blazing fast Sync v3, Threading Support, VoIP, VR, a new release of the Spec, always more monthly active users, a full security audit and progress on P2P.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.
This week we finally, finally had one of the aggregation-related MSCs, MSC2674 (event relationships) enter final comment period! This MSC, along with several others, document the stuff that powers message edit, reactions, the upcoming threading and polls MSCs, and much more! So it's really great to see the MSCs start to actually land.
Speaking of threading MSC3440 has had a good amount of review from the Spec Core Team last week. Threading in Matrix has been a long awaited feature for chat applications - as well as helping extend the flexibility of Matrix as a data structure even further. The MSC relies on both MSC2674 and MSC2675 (or a modified version of it), so the latter will be an area of focus for review for next week.
This is definitely a feature that I would love to have for chat. Note that this MSC proposes marking a room as unread, rather than a specific point in the room's timeline. This is intentional as noted in the document, as the latter is more complicated, as it intersects with sending out read receipts to other users.
When we release the fix, the changes will be publicly known and bad actors will have the ability to deduce the vulnerability. Most servers don't run release candidates, so releasing the fix in an RC will mean there's a larger window for an attacker to exploit the problem. Internally, there will be an RC deployed to test homeservers and eventually matrix.org. This means we'll be able to confidently recommend the upgrade to server administrators.
And that's the very reason we have dedicated security releases, instead of just rolling the security fixes into a feature release. The security release doesn't contain anything new apart from the security fix so it minimizes the chance of things going wrong.
Keep your servers up to date, and your users safe, administrators!
Implement "best effort" basic IRC moderation in plumbed rooms if bot has ops on IRC
Allow configuring topic sync for plumbs (IRC<->Matrix or one way)
Allow using forward slash (/) as MXID separator for IRC ghosts
Bump max mautrix version to <0.12
Small fixes
Plumb moderation! If the bridge bot has ops on IRC it will do its best to map kicks and bans (regarding IRC users) from Matrix. This definitely isn't perfect and is meant as a convenience.
Topic synchronization is now configurable for plumbs as well to make it possible to share the same topic between an IRC channel and a plumbed Matrix room. Default is still off and it requires the bridge bot to have enough PL to work.
The separator for IRC ghosts can now be changed to forward slash (/) from the default underscore (_). This happens by modifying the regex in the registration file. Only do this for new installations and it will cause all IRC users to duplicate in rooms who you can't remove and probably other bad side effects as well. The default may be changed in the future.
There were lots of refactoring issues so I hope I fixed all of them 🙈.
So, Nheko has a small little release this morning! Okay, that's a lie, it was actually pretty big! You can find the full changelog and some of our binaries here: https://github.com/Nheko-Reborn/nheko/releases/tag/v0.9.0
As always, thank you everyone, who contributed. There were over 30 authors this release! If you haven't tried Nheko in a while, give it a whirl. Lots of stuff changed, some things might not even have been mentioned in TWIM! I put the first few lines of the changelog below for your convenience:
Highlights
Somewhat stable end to end encryption 🔐
Show the room verification status
Configure Nheko to only send to verified users
Store the encryption keys securely in the OS-provided secrets service.
Support online keybackup as well as sharing historical session keys.
Crosssigning bootstrapping 🔄
Crosssigning is used to simplify the verification process. In this release
Nheko can setup crosssigning on a new account without having to use a
different client.
Nheko now also prompts you, if there are any unverified devices and asks you to verify them.
Room directory (Manu) 📂
Search for rooms on your server and other servers. (Prezu)
If their topic interests you and it has the right amount of members, join
the room and the discussion!
Custom sticker packs 🐈⬛
Add a custom sticker picker, that allows you to send stickers from MSC2545.
Support creating new sticker (and emote) packs.
You can share packs in a room and enable them globally or just for that
room.
Token authenticated registration (Callum) 🎫
Sign up with a token to servers, that have otherwise disabled registration.
This was done as part of GSoC and makes it easier to run private servers for
your family and friends!
Features
Support email in registration (required on matrix.org for example)
Warn, if an @room would mention the whole room, because some people don't like that.
Support device removal as well as renaming. (Thulinma)
Show your devices without encryption support, when showing your profile.
(Thulinma)
Move to the next room with unread messages by pressing Alt-A. (Symphorien)
Support jdenticons as a placeholder for rooms or users without avatars.
(LorenDB)
You will need to install https://github.com/Nheko-Reborn/qt-jdenticon
Properly sign macOS builds.
Support animated images like GIF and WebP.
Optionally just play them on hover.
Support accepting knocks in the timeline.
Close a room when clicking it again. (LorenDB)
Close image overlay with escape.
Support .well-known discovery during registration.
Limited spaces support.
No nice display of nested spaces.
No previews of unjoined rooms.
No way to edit a space.
Render room avatar changes in the timeline. (BShipman)
Support pulling out the sidebar to make it wider.
Allow editing pending messages instead of blocking until they are sent.
(balsoft)
Support mnemonics in the context menus. (AppAraat)
Support TOFU for encryption. (Trust on first use)
Right click -> copy address location.
Forward messages. (Jedi18)
Alt-F to forward messages.
A new video and audio player, that should look a bit nicer.
As always, come check us out and chat about Nheko in #nheko:nheko.im ♥
That's one massive update for Nheko! Thanks Nheko contributors!
We fixed regressions reported on our previous release candidates. Sorry for the delay but the current release candidate 1.6.8 should be available on the App Store on Monday
The work to replace Matomo by PostHog has been resumed
We are still working on making the MatrixKit obsolete
Space creation / invites: will start design and code review starting next week
Start space management integration in rooms
Element Android
Secure and independent communication for Android, connected via Matrix. Come talk with us in #element-android:matrix.org!
Voice message draft is currently under active development. We want the feature to work well before we release it, and it was an opportunity to rework the whole feature, to improve its architecture.
after the Miounne update posted a minute ago here is another one, and it's about time: Time To Matrix (ttm) got v1.4.0 release!
Time To Matrix is a time-like command that will send end of an arbitrary command output and some other info (like exit status) to matrix room.
With new release, following things were added:
arch linux AUR package
automatic room alias resolving, so you can use #ttm:etke.cc instead of !XODRhTLplrymaFicdK:etke.cc
help message and human-readable errors
option to change message type (m.text or m.notice)
option to omit plaintext and send only html-formatted message (to get some more space for log)
option to override message type to m.notice if the command exits with non-zero exit code (by default m.text is sent, so you will get m.notice on failure)
simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio. Version 2.3.0 adds support for additional configuration via config files and other methods. Currently, there is only one setting that can be changed, however many existing and future features will be able to be enabled or disabled via this config.
Example usage is shown below:
"""
random_user
!echo something
echo_bot
something
"""
import simplematrixbotlib as botlib
creds = botlib.Creds("https://home.server", "user", "pass")
config = botlib.Config()
config.load_toml("config.toml")
bot = botlib.Bot(creds, config)
PREFIX = '!'
@bot.listener.on_message_event
async def echo(room, message):
match = botlib.MessageMatch(room, message, bot, PREFIX)
if match.is_not_from_this_bot() and match.prefix() and match.command("echo"):
await bot.api.send_text_message(room.room_id,
" ".join(arg for arg in match.args()))
bot.run()
New feature: if a user on your homeserver reports abuse, Mjölnir may now show the abuse report in your moderation room and offer you two-click moderation options. This feature is considered a preview for the time being.
Performance improvements for protections that need to lock back in the history of a room, decreasing the number of cases in which we could end up timing out.
Many improvements to testing.
Note: Any rumor of a v1.2.0 Docker image borked by yours truly is sadly true. There should be no risk in 1.2.0 but, to be on the safe side, if you have updated to 1.2.0, please update to 1.2.1.
That's one feature I've wanted for a while, and it's going to make moderation a lot easier! Thanks Mjolnir teams for keeping us safe!
Today I want to showcase you MinesTRIX. MinesTRIX is a decentralized social media based on matrix.
The goal is to create a privacy respectful social media using the power of matrix while trying to be as simple as possible.
Two Objectives
Showing that matrix could be used to build such a system.
There was the "Sign in with Matrix" project recently
I tried to do something similar with https://matrix-login.lyc.fi / https://gitlab.com/ptman/matrix-login
An important note on the interesting projects using Matrix for the login: those are community projects, and there are MSCs in the works to "do it right" at the Spec level!
The direction we're headed in the Matrix spec core team is instead towards replacing Matrix's current auth mechanisms with normal Open ID Connect (rather than wrapping our own OIDC-like thing, as we do today) - as per https://github.com/sandhose/matrix-doc/blob/msc/sandhose/oauth2-profile/proposals/2964-oauth2-profile.md The common login flow would then be for users to be authed by their server using a trusted OIDC identity provider, rather than ever trusting arbitrary clients with their credentials.
I have compiled a list of public homeservers available for registration, since previous such efforts to make these homeservers more discoverable fell through. This list serves as a sanitized version of the asra.gr list, with only homeservers intended for public consumption included. It is a static list and does not include pings, but rather than focusing on the technical aspect, my list has an emphasis on the written rules of a homeserver, which I believe to play a larger role in the Matrix experience. Hope this can spark other efforts in maintaining a better list!
That's one very useful list of hand curated servers! The transparency about the inclusion criteria is very much appreciated. Good job!
Hi everyone! Did you ever feel lost in the Matrix world? The room directory is big, but it's still hard to find something you like. Or are you a room moderator, but there is not much activity in your room because it doesn't have enough users?
This is why I want to share rooms (or spaces) I find interesting.
On Tuesday, 23rd November we plan to release Synapse 1.47.1 at 12:00 UTC to address a single high severity issue. This vulnerability was discovered internally by our security team. Synapse is a Matrix homeserver implementation developed by the matrix.org team and the wider Matrix community.
If you're a server administrator running Synapse, please be prepared to upgrade as soon as the patched version is released.
We will be reaching out to downstream packagers to ensure they can prepare patched versions of affected packages at the time of the release. The details of the vulnerability will be disclosed in a blog post on the day of the release. There is so far no evidence of the vulnerability being exploited in the wild.
Thank you for your patience while we work to resolve this issue.
Edit, 2021-11-19: The opening paragraph was amended to note that the vulnerability was discovered internally.
Edit, 2021-11-22: The opening paragraph was amended to include a release time of 12:00 UTC.
NOTE: We anticipate publishing a security release, Synapse 1.47.1, on the coming Tuesday, the 23rd of November.
Synapse 1.47.1 will contain a fix for a high severity issue.
Synapse 1.47.0 features additions to the admin and module APIs, a plethora of fixes for long-standing bugs, and a raft of internal improvements. Server administrators should note that this release removes a deprecated API for deleting a room and deprecates a module callback. Consult the upgrade notes for more details.
We fixed a variety of different bugs in this release, many of which were long-standing. It's good to see them dealt with! Worth mentioning in particular:
Improvements to thehandling of the device_inbox table, which trim redundant data to reduce database bloat.
Fixes related to restartingworkers, to ensure a more reliable upgrade process.
Additionally, work continues on improving type-checking coverage, both inSynapse and inSygnal.
Sydent 2.5.1
This week also saw the release of Sydent 2.5.1, the reference implementation of a Matrix Identity Server. This is a minor release which mainly tidies up error handling to reduce the amount of noise in logs. It should also make it easier for us to diagnose some outstanding bugs which remain to be squashed.
Everything Else
In the background, we're still working away at implementing MSC3440 to facilitate threading. Early tests are promising. We're also exploring MSC2775 as a means to speed up room joins. Both will be long term projects that should hopefully reap major rewards for the Matrix ecosystem. Lastly, there's support for MSC3228 to allow identity servers to provide bespoke invites to spaces. We mentioned this last time in Sydent release notes; now we've got support for it on the Synapse side.
Please see the Synapse Release Notes for a complete list of changes in this release.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.
Matrix v1.1 was released! Read the blog post here if you missed it; it summarises everything that's new in v1.1, as well as plans for the future. Now that the new spec build pipeline and release infrastructure is in place, we're aiming for roughly quarterly releases going forward. Thank you all for being so patient in the meantime!
Extensible events is something that has been a long time coming in Matrix. It unlocks so much potential, and is even currently being built on (see MSC3381 (polls)). Definitely one of the next big ticket items to tackle in the medium term.
Dan (aka callahad) is away this week, so let me report on his behalf.
We cut a release candidate for Synapse (1.47.0rc2, but see the changelog for rc1). It exposes new functionality for pluggable modules and new endpoints to the Admin API. We've fixed a bunch of long-standing bugs and continued to drive forward efforts to improve documentation and code quality. Thank you to all of our contributors!
With future releases in mind, we've been continuing work to support threading and E2EE application services. We've also been prototyping a new Admin API to remove users from all rooms belonging to a certain space.
Elsewhere, we've been doubling down on our effort to improve reliability and maintainability of our services as a whole. Sydent and Sygnal have a number of PRs in flight for both, aimed at improving type coverage and driving down error noise in the logs. We've drafted a blog post to summarise the process of type annotating mypy (keep your eyes out for that one). We also worked to make the matrix.org database more resilient, and made changes to improve the experience of rolling out upgrades to Synapse en masse.
Vermicularis is a script for forwarding messages from Päikky to Matrix. Päikky is a parent-teacher communication website and mobile app for daycare and pre-school. git clone https://k2c42.dy.fi/git/vermicularis.git and join #paikky:ellipsis.fi to give feedback
Päikky was created by a group of parents who wanted to make communication with daycare staff easier. They started a company and later sold it to Abilita. Päikky is currently used in 40+ municipalities in Finland and also in some private daycare companies too.
Päikky is also used for reserving daycare times for kids.
Sensitive flag for MSG and NICKSERV to hide it from network room (for AUTOCMD)
Improved STATUS for admin room and simple STATUS for network rooms
Small fixes
Not much going on this week. Hopefully the reconnect refactor doesn't cause any breakage as it fixed multiple issues that have been around since the inception of Heisenbridge.
I guess not. For that reason bug fixing in Nheko for the next release continues. We fixed a super annoying issue, where loading keys from the online backup could make Nheko stuck in a flickering mode with no way to interact with it anymore. You can now also click anywhere on a read receipt to open someones profile instead of just their avaitar, edited messages now also show as redacted, if only the original message got redacted and don't lose the reply in encrypted rooms, if the edit was sent by a client, that is not Nheko. You also used to get logged out after registration, which should finally be resolved. There were also a bazillion translation updates! Thank you, everyone who contributed to those.
We also finally merged the prettier video player, which also fixes a video playback issue on macOS. Try it out and give us feedback on it!
We are still hunting down some last bugs, but expect the release soon now. In the meantime, I did start summarizing the changes, if you want to see what will be in the next release: https://github.com/Nheko-Reborn/nheko/blob/master/CHANGELOG.md#090----unreleased
Message threading is coming to Element! If you haven’t yet, head to develop.element.io and ensure ‘Enable threading’ is turned on in Labs to test on the web.
We’re spinning up development on iOS & Android as we speak.
We’ll also be running the first Threads community testing session next Thursday (18th Nov) at 17:00 GMT. Come join us in #element-community-testing:matrix.org!
As well as the community testing on Threads, we’ll also be hosting our next testing session for Android on Wednesday (17th), with the time to be confirmed. If you’d like to be involved, join us in #element-community-testing:matrix.org.
Element Web
In the background, if you’ve been paying close attention to our issue tracker, you might have noticed lots of changes to our triage & issue workflows over the past couple of months. We’re continuing to iterate on these, extending the best learnings to iOS & Android too.
We recently started a project to improve the info architecture/layout of our apps, starting on the web first. We’re merging our first tweaks and experiments soon, so expect exciting things to be landing on develop!
We’re continuing to implement more Space creation & management support on iOS. We’ve added in Space creation screens (implemented in SwiftUI!) and we’re polishing inviting people to Spaces.
We’ve also been conducting user research to see what parts of the app are tripping users up, and working on fixes to various issues. Expect tweaks to land soon!
Otherwise, we’re also merging & testing release candidates for upcoming releases, merging several branches.
Element Android
We’ve been conducting the same user research on Android, watch this space for more tweaks & improvements coming soon!
Otherwise, we’re also fixing up some smaller issues specific to Android 12.
Full UIAA implementation means now you can register an account on any hs that allow
Configure default homeserver with options to add more than one.
Can use http when looking for homeserver
Enhanced invite list UX
Added logout in loading screen
Hide pinned space notification from home icon
Add option to select role on roomCreation
Added Invite/disinvite option in profile viewer
Bugs
Fix commands activating anywhere in the input
Fix duplicate and minus notification count
Fix links splitting across line mid-word
Find more about Cinny at https://cinny.in/
Join our channel at: #cinny:matrix.org
Github: https://github.com/ajbura/cinny
Twitter: https://twitter.com/@cinnyapp
A time-like command that will send end of an arbitrary command output and some other info (like exit status) to matrix room. Useful when you need to run something in terminal and get a ping when it's done.
Since the last ttm news in TWIM new options were added:
matrix auth with access token - useful for SSO when you don't have actual login/password pair
skip time info
skip html formatting (doubles allowed log size in message)
post full log output to matrix (with auto-shrinker to avoid "message is too big" error)
Version 2.2.0 adds support for authentication via access_tokens.
In addition to username/access_token, it is possible to authenticate using username/password and login(SSO) token.
Example usage is shown below:
"""
Example Usage:
random_user
!echo something
echo_bot
something
"""
import simplematrixbotlib as botlib
creds = botlib.Creds(
homeserver="https://example.org",
username="echo_bot",
access_token="syt_c2...DTJ",
)
bot = botlib.Bot(creds)
PREFIX = '!'
@bot.listener.on_message_event
async def echo(room, message):
match = botlib.MessageMatch(room, message, bot, PREFIX)
if match.is_not_from_this_bot() and match.prefix() and match.command(
"echo"):
await bot.api.send_text_message(room.room_id,
" ".join(arg for arg in match.args()))
bot.run()
Hello again! Halcyon is a Matrix bot library created with the intention of being easy to install and use.
This release brings minor non-breaking features and some bug fixes:
Added
change_presence() now allows you to set if you are online, idle, or away. Status message support
Roadmap and documentation updates
Fixed
A fix for retrying on 5xx errors
A better catch for bad server syncs
Windows support, fixing the NotImplementedError reported by @bhuitt (Thanks!)
More info at on the project at https://github.com/WesR/Halcyon . Come by and chat with us over in https://matrix.to/#/#halcyon:blackline.xyz
Heads up for those in Berlin. You're welcome to join us Tuesday, 16th Nov at 7:00 PM chatting about Matrix development and hosting. We're going to meet in person at c-base. In compliance with the hackerspace's house rules this is a strict 2G event.
We'll talk about everyone's Matrix projects, test the P2P demo via Bluetooth in person and plan the presence of Matrix at the rC3 event happening in the c-base shortly before New Years.
"Sign in with Matrix" is a web component, which developers can use to build a web login using Matrix account
it is similar to those "Sign in with Google" and "Sign in with Facebook" buttons you see on the internet
but now it's Matrix!
i invite you to take a look at the repository: https://github.com/mishushakov/signin-with-matrix
and experience a demo: https://mishushakov.github.io/signin-with-matrix/
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Once again it's been a little while since we've done a spec release (sorry; we're aiming for quarterly releases from here on out), but this time we have some pretty big news: we've got an all-new spec platform and a new versioning scheme. The new spec platform has been needed for a long time to make better sense of what Matrix is, and as part of getting that out the door we reduced the number of "Matrix versions" to just one.
Huge thanks to Will Bamberg for building it out for us, anoa for working out the deployment details, and everyone for testing it all. They talk at length about what this specification even is and about the platform itself on Matrix Live S6E19. It's the single greatest improvement to the spec we've seen to date.
The new versioning scheme presents the whole specification as a single document, making it easier to check compatibility between implementations and the spec itself. Previously, a grid would have to be drawn to show whether Server-Server r0.1.4 is compatible with Client-Server r0.6.1 - while obvious at release time, the historical context gets lost quite easily. Now, with a single version number, the entire specification is compatible across a single version number: servers implementing Matrix 1.1 will be compatible with clients implementing v1.1, and vice versa. For the specification itself, this means we no longer have to carefully point and update links between the APIs, as they'll instead be versioned together.
Changing the versioning of the specification does have an impact on the Client-Server API in particular, however. You may have noticed that Client-Server APIs are currently versioned at "r0". By removing rX.Y.Z versioning, all of the endpoints suddenly don't have a version to reference. All endpoints across the specification are now versioned individually to allow for breaking changes at the endpoint level. They no longer require the whole specification to be listed as a breaking change: a v1 endpoint can get additions/changes which are backwards compatible, but if we need to change format (for example) then it'll get bumped up to v2, leaving v1 in its last known state.
For the Client-Server API, a slight complication is that v1 and v2 (alpha) are already versions familiar to those that have been around for a while - to avoid confusing those people, existing Client-Server API endpoints will start at v3. New endpoints introduced after v1.1 will start at v1 instead.
It's been well over a year since the last release, which means there's a whole lot of features and changes to cover. Here we'll try to cover the things most clients/servers will have to worry about, but we do still recommend reading through the changelog included below. Overall, 28 MSCs have been merged through this release, but some have had to be excluded in the interest of getting the spec release out there. Many of them are expected to be in the next anticipated release (which should hopefully be next quarter).
Clients: There's a lot of stuff
In Matrix 1.1, client developers get all sorts of new features to play with. Clients which support end-to-end encryption should explore key backups, cross-signing, SSSS, and breaking changes to verification. Quite a lot of this stuff has existed for a while, but has made it into the specification formally now. As an added bonus, the emoji for SAS verification have been translated (contribute here).
Knocking has also landed in the spec (thanks Sorunome for leading the charge on this!), opening up the ability for rooms to optionally allow people to request invites to join. This can be helpful for semi-private rooms where it can be easier to approve/deny requests compared to finding someone's MXID and manually inviting them. This does require a v7 room to work, however.
Thanks again to Sorunome, Message Spoilers have been officially included in Matrix. People can now safely discuss the ending to the latest movie without being banned for spoilers. Though, as a new feature, there's a chance that the spoiler text still gets included in the message: clients should update as soon as possible to avoid their users accidentally getting banned for spoiling the conclusion to the Spaces saga ;)
There's a few other smaller improvements/additions, and of course the regular "various clarifications and bug fixes" to take a look at. For a sample checklist, check out element-web's issue on the subject.
Servers: Knock knock, it's a new room version
Room version 7 has landed, bringing forth the ability for users to knock on rooms (requesting an invite to join). The changes are largely scoped to using the reserved keywords for join rules and membership, and are described through the auth rules. Thankfully, the changes over v6 are minimally invasive so should be quick to implement.
Additionally, the cross-signing bits have been included in the API responses and EDU definitions. Matthew's blog post from last year (it really has been that long...) describes cross-signing and the history of its implementation.
As per usual, there's also various specification errors corrected to aid understanding. Synapse has an exhaustive issue to detail what servers might need to do.
PS: Room versions 8 and 9 are also out there, but will be included in a future spec release.
The full changelog
We haven't mentioned identity servers, bridges, etc in this post but they have changes too! Below is the whole changelog, the entire year and a bit of it. Thank you to everyone who has submitted MSCs, and congratulations on getting them released. If we forgot yours, please mention it in #matrix-spec:matrix.org so we can apologize and correct.
PS: The MSC process is how changes to Matrix are made, and you (yes, you) can propose those changes too. Check out the Matrix Live episode where Matthew talks about how this process works, and how we avoid blocking clients from implementing their proposals behind the relatively slow spec release cycles.
Client-Server API
Breaking Changes
Document curve25519-hkdf-sha256 key agreement method for SAS verification, and deprecate old method as per MSC2630. (#2687)
Add m.key.verification.ready and m.key.verification.done to key verification framework as per MSC2366. (#3139)
Deprecations
Deprecate starting verifications that don't start with m.key.verification.request as per MSC3122. (#3199)
New Endpoints
Add key backup (/room_keys/*) endpoints as per MSC1219. (#2387, #2639)
Add POST /keys/device_signing/upload and POST /keys/signatures/upload as per MSC1756. (#2536)
Add User-Interactive Authentication fields to cross-signing APIs as per MSC1756. (#3331)
Mention that a canonical alias event should be added when a room is created with an alias. (#3337)
Add an 'API conventions' section to the Appendices. (#3350)
Clarify the documentation around the pagination tokens used by /sync, /rooms/{room_id}/messages, /initialSync, /rooms/{room_id}/initialSync, and /notifications. (#3353)
Remove the inaccurate 'Pagination' section. (#3366)
Clarify how redacted_because is meant to work. (#3411)
Remove extraneous mimetype from EncryptedFile examples, as per MSC2582. (#3412)
Describe how MSC2844 affects the /versions endpoint. (#3420)
Fix documentation errors around threepid_creds. (#3471)
Server-Server API
New Endpoints
Add /make_knock and /send_knock endpoints as per MSC2403. (#3154)
Backwards Compatible Changes
Add cross-signing information to GET /user/keys and GET /user/devices/{userId}, m.device_list_update EDU, and a new m.signing_key_update EDU as per MSC1756. (#2536)
This week we have demos from andybalaam, showcasing high contrast in Element Web/Desktop, Bruno walking us through device dehydration in Hydrogen and Kegan giving us a glimpse of Sync v3! A lot of exciting work going on here!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.
The spec release mentioned last week is happening on Tuesday - look out for a long blog post explaining all the changes, and check out the new platform (as it stands in its current, unstable state) before it goes live. Exciting stuff!
Other than this sounding like something out of a workout routine, soft kicks are proposed as a hint to clients that the user has been kicked from the room, but that it was not out of moderation, but for a technical reason (the example given disconnection from an IRC network). The client can then present a different UX as opposed to a normal, "hard" kick.
Give it a read and provide feedback if the topic interests you!
This week we released Synapse 1.46! This release includes a fix for the performance regression from 1.44 which could cause smaller hosts to lock up when clients reconnected after some time offline.
Synapse 1.46 also includes experimental support for MSC3440: Threading via m.thread relation, a big step toward threaded messaging in Matrix. It's otherwise mainly a bugfix release, as we've spent much of our time reviewing infrastructure and making plans to improve our resilience should we experience hardware failures similar to what took out matrix.org two weeks ago.
We continue working toward a prototype implementation of MSC2775: Lazy loading over federation, starting with setting up automated infrastructure to repeatedly measure the join speeds of assorted large, public rooms. Our goal is to have a rough proof of concept for lazy room joins by the end of the year.
We also released Sydent 2.5.0! In addition to fixing bugs, this release passes mypy --strict, uses Jinja2 for templates, and supports the room_type field from MSC3288 to better differentiate between invitations to rooms and invitations to Spaces.
Threads! Awesome! And lazy loading over the federation looks really promising, I can't wait for it.
Soru has not found any time to work on the mx-puppet-* project in very long. At this point, it makes sense to look for someone / multiple people to inherit the project to. So, if you are willing to pick it up, or part of it (like a single bridge, only the library, etc.), then please look by #mx-puppet-bridge:sorunome.de. Thanks!
It's always sad when maintainers don't have time for former projects, but you can take over!
mautrix-whatsapp v0.2.0 was released. The main change is switching to the new WhatsApp multidevice API, which means the phone doesn't need to be connected 24/7 anymore. The release also includes a new backfilling system using MSC2716, and making the relaybot mode less special-cased and more like what mautrix-signal does.
Allow opening multiple simultaneous connections to the same network
New STOP/STAHP ROOM command to end a long paste flood
Clear channel send queue if the user gets kicked to prevent continuing flood on rejoin
New KB ROOM command to do a proper kick & ban (contributed by GitHub user @blastrock, thanks!)
The most useful feature this week is having the ability to open multiple connections to the same network. This can be used to plumb channels with a secondary connection while using the bridge as yourself with the main one. You can also STAHP long pastes from being actually sent out completely to a channel if you weren't careful enough on Matrix side.
Upgrading from previous version to v1.5.0 will cause a small migration for every channel/pm that is open so it might take a little longer for the first startup to complete. If no multiple network connections are opened it is safe to downgrade but if someone uses the feature it will cause shenanigans for the duplicate networks in the older version.
We are preparing for a release! That means, if you are translator, we could use your help to get our translations updated and if you want to have a bug fixed before release, now is the time to report it or complain about it! Please come and see us in #nheko:nheko.im for complaints and questions!
We saw a lot of translation updates by Linerly, Priit and Thulinma already. The click ghosting issue in the room list should be fixed as well as the crash when accepting an invite from your browser. You can now register on matrix.org again, after fighting with some... reality deforming wording in the spec. Notifications on macOS now use the "modern" API for notifications, including grouping notifications by room. (Expect more fancy improvements there in the future!) The warning about mentioning the whole room now vanishes properly, hiding space children from the roomlist now doesn't hide them from the space itself anymore, the loading spinner shouldn't show in a fully loaded room again, the room directory search bar is now focused by default and a few smaller fixes.
We also completed the bootstrapping UI now. This means you get prompted for unverified devices, when your device isn't verified or when cross-signing isn't configured for your account yet and you are given ways to remedy that.
We have a few more bug fixes we want to land before the release, so don't expect it tomorrow, but we have the featureset down now. And if you thought the changes listed in this snippet are long, wait for the full release changelog. It will probably take a few days for me to tidy that up and I think this will be our biggest release yet! Happy testing, everyone!
Device dehydration support got released this week in 0.2.20 (see matrix live demo!), allowing you to decrypt messages sent to you while not logged in on any device. Expect another release this evening (0.2.21) with support for multiple lines in the composer and some other bug fixes.
After being away for a few month, Kévin Commaille came back with a blast. No less than three MR already, and I have a feeling he’s only getting started:
We’re continuing to flesh out space creation screens on iOS, improving the invite flows post creation for private and public spaces.
We’re continuing to iterate on the overall app layout on Web, to fix fundamental usability issues.
Expect to see some changes on develop soon, rearranging the top left corner of the app (user avatar, space names, context menu).
Expect to see larger experiments landing on develop soon too. We’ll share more info on these when they’re live, and everything will be opt out to not break existing workflows while we’re testing and experimenting.
Community testing session ran successfully on RC with a focus on voice messages and VoIP calls. We found significantly fewer issues and regressions than in the previous sessions. It was great to see 9 newcomers - thank you to all of our testers who helped out!
Improved our sentry analytics data to provide more useful detail.
Redesigned the device management section in the user settings. The new view makes it clearer how to rename devices and lets you verify them.
This week in labs (features in labs can be accessed at develop.element.io and nightly desktop builds):
Threads now has a more polished UI and users can now link to events in a thread. Notifications for threads messages are being worked on and should land very soon.
Release 1.3.6 is finally live on the PlayStore! Release 1.3.7 is expected for the beginning of next week.
We have started to implement threading of messages.
Draft for voice message has been delayed, we want to ensure that it’s working well before releasing it.
We are doing lots of internal migrations: Dagger to Hild, MvRx to Mavericks2, Rx to coroutines Flow.
And as always, we are trying to keep the app as stable as possible, and as reliable as possible
It’s also worth noting that the SDK 1.3.6 has some improvements in the login API, not needed for Element Android, but requested by other users of the SDK.
Godot-Matrix is a C++ module for connecting to Matrix from the Godot Game Engine.
I refurbished it and the changes made it upstream now ^_^, thx vurpo! :
https://www.matrix.org/docs/projects/sdk/godot-matrix/
Changes:
Fixed compilation for 3.x Godot branch
Support plain http too for local tests with Conduit
Respect port numbers (as per Matrix spec)
Substantial rewrite of the example so it works
Refurbishment made with love for the participants of FediJam 4, a weeklong gamedev competition with the focus on fediverse and federated multiplayer (via Matrix or any other means ^_^).
It starts this Sunday at 18:00 UTC. Chat
Unlike other "overlay" style crates it does not follow the sdk versions using its own version number or using features. So please make sure you use the right version of the SDK with it.
Also it now has support for rusttls, the ! at the start of a command now gets correctly checked and is updated to the latest sdk.
You can check it out at https://crates.io/crates/mrsbfh and https://github.com/MTRNord/mrsbfh
New release of the Commander of etke.cc backoffice is coming here!
New version brings new neat features and extends arsenal of the antispam tricks:
buymeacoffee listings integration - pretty basic, but that's just a first step! So now you can list subscribers, supporters and extras purchases directly in matrix room (spolier: next releases will bring notifications about new subscribers, supporters and purchases)
rate limiter for http forms - no one can bother you if you don't want that! So now you can configure specific rate limits per form
enhanced email validation (mostly used in the forms module) - you had spam hosts and emails list before, but now you have MX records checks, regular expressions check and some other useful tricks to hold the door stop spammers
... and some small changes, like doc updates and auto-shrink of long messages to fit into matrix payload limits
I created a Vacation Responder for Matrix. While running, it responds to any DMs that come in to your account with a preset message. It can be configured to only respond once in a given period of time (defaults to 24 hours) so that the person contacting you can queue up questions for you when you return. Other ways to use the bot include responding to DMs to previously active accounts (for example if you move homeservers). Join the discussion about this bot at #matrix-vacation-responder:nevarro.space.
SeaGL (the Seattle GNU/Linux Conference) is being held on Matrix Friday and Saturday, November 5th and 6th, from 9am-5pm Pacific Time. The conference is running on a customized version of element-web and a regional Matrix homeserver, found at https://attend.seagl.org
Founded in 2013, this is the conference's ninth year overall and second virtual event. Last year, the all-volunteer team built their portal on-top of KiwiIRC and Jitsi for attendees, and BigBlueButton for speakers. This year they have been relying heavily on Spaces, Subspaces, and the built-in features that have been rapidly developing within the Matrix protocol.
From their website:
SeaGL (the Seattle GNU/Linux Conference) is a free—as in freedom and tea—grassroots technical summit dedicated to spreading awareness and knowledge about free / libre / open source software, hardware, and culture.
SeaGL strives to be welcoming, enjoyable, and informative for professional technologists, newcomers, enthusiasts, and all other users of free software, regardless of their background knowledge; providing a space to bridge these experiences and strengthen the free software movement through mentorship, collaboration, and community.
Following a question from someone about getting started with Matrix bots, I wrote a super quick getting started guide to writing a simple bot with opsdroid. I will probably integrate this into the opsdroid docs, or somewhere more accessible in the future, but for now here it is: https://gist.github.com/Cadair/2c64cb46fdce7499ff1c0bbb116ce29e
Being an innocent child, I asked:
are you aware of projects relying on opsdroid to make a Matrix bot? Other than half of the very famous TWIM bot 😇
I have written a few (https://github.com/Cadair/skill-motw, https://github.com/Cadair/skill-intersphinx, https://github.com/Cadair/skill-jitsi) I know that Oleg has also written a number, including many which interact with homeassistant.
I didn't know what I had started! We almost became This Week in Opsdroid! Fortunately the Matrix community is very active and helped us keeping sovereignty over the blog. Phew!
There are too many to list here, but I strongly encourage you to blend in with the opsdroid community. Not only are these kind people, they are also very active and develop fancy and useful bots!
