This audit was a bit of a whirlwind, as while we were clearly overdue an audit of Matrix’s E2EE implementations, we decided quite late in the day to focus on bringing vodozemac to auditable production quality rather than simply doing a refresh of the original libolm audit. However, we got there in time, thanks to a monumental sprint from Damir and Denis over Christmas. The reason we went this route is that vodozemac is an enormous step change forwards in quality over libolm, and vodozemac is now the reference Matrix E2EE implementation going forward. Just as libolm went live with NCC’s security review back in 2016, similarly we’re kicking off the first stable release of vodozemac today with Least Authority’s audit. In fact, vodozemac just shipped as the default E2EE library in matrix-rust-sdk 0.5, released at the end of last week!
The main advantages of vodozemac over libolm include:
Native Rust - type, memory and thread safety is preserved both internally, and within the context of larger Rust programs (such as those building on matrix-rust-sdk). This is particularly important given the memory bugs which libolm sprouted, despite our best efforts to the contrary.
Performance - vodozemac benchmarks roughly 5-6x faster than libolm on typical hardware
Better primitives - vodozemac is built on the best practice cryptographic primitives from the Rust community, rather than the generic Brad Conte primitives used by libolm.
Also, we’ve finally fixed one of the biggest problems with libolm - which was that the hardest bit of implementing E2EE in Matrix wasn’t necessarily the encryption protocol implementation itself, but how you glue it into your Matrix client’s SDK. It turns out ~80% of the code and complexity needed to securely implement encryption ends up being in the SDK rather than the Olm implementation - and each client SDK ended up implementing its own independent state machine and glue for E2EE, leading to a needlessly large attack & bug surface.
To address this problem, vodozemac is designed to plug into matrix-sdk-crypto - an SDK-independent Rust library which abstracts away the complexities and risks of implementing E2EE, designed to plug into existing SDKs in any language. For instance, Element Android already supports delegating its encryption to matrix-sdk-crypto; Element iOS got this working too last week, and we’re hard at work adding it to Element Web too. (This set of projects is codenamed Element R). Meanwhile, Element X (the project to switch Element iOS and Element Android to use matrix-rust-sdk entirely) obviously benefits from it too, as matrix-rust-sdk now leans on matrix-sdk-crypto for its encryption.
Therefore we highly recommend that developers using libolm migrate over to vodozemac - ideally by switching to matrix-sdk-crypto as a way of managing the interface between Matrix and the E2EE implementation. Vodozemac does also provides a similar API to libolm with bindings for JS and Python (and C++ in progress) if you want to link directly against it - e.g. if you’re using libolm for something other than Matrix, for example XMPP, ActivityPub or Jitsi. We’ll continue to support and maintain libolm for now though, at least until the majority of folks have switched to vodozemac.
In terms of the audit itself - we recommend reading it yourself, but the main takeaway is that Least Authority identified 10 valid concerns, of which we addressed 8 during the audit process. The remaining two are valid but lower priority, and we’ll fix them as part of our maintenance backlog. All the issues identified are excellent valid points, and we’re very glad that Least Authority have added huge value here by highlighting some subtle gotchas which we’d missed. (If you write Rust, you’ll particularly want to check out their zeroisation comments).
So: exciting times! Vodozemac should be landing in a Matrix client near you in the near future - we’ll yell about it loudly once Element switches over. In the meantime, if you have any questions, please head over to #e2ee:matrix.org.
Thanks again to gematik for helping fund the audit, and to Least Authority for doing an excellent job - and being patient and accommodating beyond the call of duty when we suddenly switched the scope from libolm to vodozemac at the last minute ;)
Next up: we’re going to get the Rust matrix-sdk-crypto independently audited (once this burndown is complete) so that everyone using the matrix-sdk-crypto state machine for Matrix E2EE can have some independent reassurance too - a huge step forward from the wild west of E2EE SDK implementations today!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
The Spec Core Team has decided to not include MSC3440 (threading via m.thread relations) specifically in the upcoming Matrix v1.3 release. We previously stated that it would be in v1.3, but there's a few unproven MSCs (namely MSC3773 (notifications for threads)) which make it feel improper to put threads into the spec at this stage. We're aiming for a future (very near) spec release to contain threads, assuming we can get enough of the feature landed. This is a very similar approach to what we did with Spaces: while the core feature existed as FCP-concluded for a while, we wanted to give some other MSCs a chance to land before writing it up formally.
As a follow-on from this, any implementations that are currently assuming "v1.3" spec support translates to threading support should note the following:
Clients which are pre-emptively checking for v1.3 in /versions should stop doing that. Look for org.matrix.msc3440.stable as an "unstable" feature flag instead.
Servers which offer MSC3440's behaviour should expose it on stable endpoints and expose org.matrix.msc3440.stable for clients to detect.
Optionally, clients and servers can drop support for the org.matrix.msc3440 (truly unstable) flag, given clients & servers should have already upgraded.
At the spec level, we'll still be writing up MSC3440's dependencies (aggregations and friends), but will notably be missing MSC3440 specifically in v1.3. If folks can update their clients urgently, it would be massively appreciated!
A simple change to the auth rules to help prevent an edge case where one is able to invite a user, but not disinvite them as the latter requires a separate permission. The MSC attempts to solve this by always allowing the cancellation of invitations you've sent.
A small MSC, but one with some open threads still. Perhaps a good opportunity to go in and clean it up!
In parallel, Olivier's been doing groundwork to reduce replication traffic for workerised Synapse deployments, Andrew (anoa) has been looking at media retention policies and work continues on faster room joins.
This week we've actually made two releases — Dendrite 0.8.4 and Dendrite 0.8.5 — which are both primarily targeting performance and bug fixes.
We've also released our new living documentation, which is available at https://matrix-org.github.io/dendrite/, which features new installation instructions and documents that will be helpful to Dendrite server administrators. We will be writing more documentation and expanding this over time.
Changes include:
The built-in NATS Server has been updated to version 2.8.2
Monolith deployments will no longer panic at startup if given a config file that does not include the internal_api and external_api options
State resolution v2 now correctly identifies other events related to power events, which should fix some event auth issues
The latest events updater will no longer implicitly trust the new forward extremities when calculating the current room state, which may help to avoid some state resets
The one-time key count is now correctly returned in /sync even if the request otherwise timed out, which should reduce the chance that unnecessary one-time keys will be uploaded by clients
The create-account tool should now work properly when the database is configured using the global connection pool
Fixes a regression introduced in the previous version where appservices, push and phone-home statistics would not work over plain HTTP
Adds missing indexes to the sync API output events table, which should significantly improve /sync performance and reduce database CPU usage
Building Dendrite with the bimg thumbnailer should now work again (contributed by database64128)
As always, feel free to join us in #dendrite:matrix.org for more Dendrite-related discussion.
Ement.el, a Matrix client for [GNU Emacs](https://www.gnu.org/software/emacs], received some more updates this week:
Additions
New Transient-based command menu for room buffers, bound to ?.
New command ement-list-members, which lists all members in a room.
Member completion now completes from all members in a room (rather than only those whose messages have been seen).
Power-level events are displayed.
Grouped membership events have tooltips showing each member's individual event.
When usernames are colorized, message bodies may optionally be colorized in a less intense shade of the same color, so they are easily distinguished yet easy on the eyes (and, of course, the shading is customizable).
Changes
When displaying usernames in the margin, names that are wider than the margin are abbreviated, with the full name in the tooltip.
Fixes
Typing notifications (recently broken--oops!).
Sending direct messages marks new direct rooms as direct rooms.
Event notifications for rooms not displayed in a buffer.
As always, feel free to join us in #ement.el:matrix.org.
The newest version of the app (1.8.14) was released on Monday! In this release we’ve made a bunch of improvements and bug fixes, check it out here
💊 We also merged an initial version of mention pills and are working on ironing out edge cases. Exciting!
There’s good progress on the new authentication flows and live location sharing, both of which we’re looking forward to sharing and testing with you soon.
On the ElementX front we're nearly done with the DevX setup and are working towards using the latest Rust SDK
unit and UI test runs on the CI and Codecov.io coverage reports
swiftlint, danger-swift and sonarcloud.io checks for code quality
automatic generation of PR builds, published on diawi
scripts for importing and reusing localizable strings from Android
Cinny v2.0.0: Session verification and other e2ee features
Features
Redesign app/user settings
Add support to manage cross-signing and key backup
Add support to manage sessions
Add Session verification by emojis
Add recently used section to emoji board
Add LaTeX / math input and rendering support
Add notification sounds
Add sound on incoming invites
Sort DM's by activity
Sort search results by activity
Improve jump to unread button
Replace confirm and prompt with custom dialogs
Adapt to different device widths
Native application using Tauri
Bugs
Fix loading on older browsers
Fix crash on load and room creation on dendrite
Fix can't open spaces from public room list
Fix max power level in room permissions
Fix incorrect custom html crashing app
Fix crash on unable to getContent of tombstoned room
Don't enable e2ee for bridged platform
See release at: https://github.com/ajbura/cinny/releases/tag/v2.0.0
Cinny for desktop
Since this release, we are also shipping a desktop app of Cinny for Windows, MacOS and Linux. You can download the app for your platform from https://github.com/cinnyapp/cinny-desktop/releases
Find more about Cinny at https://cinny.in/
Join our channel at: #cinny:matrix.org
Github: https://github.com/ajbura/cinny
Twitter: https://twitter.com/@cinnyapp
The Rust Matrix team is very excited to announce the next major release Matrix SDK 0.5 "stores and native crypto", now available for everyone by adding matrix-sdk = "0.5.0" to their dependencies in their Cargo.toml. This first major release in about 8 months brings significant changes many have been waiting for:
Better, safer, native-er crypto: matrix-sdk 0.5 ditches libolm in favor of Vodozemac, the completely fresh rust-native re-implementation of the crypto crate. It has also been audited, the report of which is scheduled to be released soon.
Store revamp: the entire cache storage infrastructure for crypto and state has been restructured to allow for full plug-ability, with even the two default supported backends, sled and indexeddb both being separate crates now. And as a matter of fact, a community-driven sql implementation has already been published as beta (with postgres and sqlite support) by charlotte 🦝.
WebAssembly; both of these changes also allow us to finally make WASM a Tier 1 supported target for the SDK. Not only does it build and even use the browser native indexeddb for permanent storage, we also have CI tests ensuring nothing will break this support going forward.
Features, Features and Process: there's plenty more features and bug fixes in this release, e.g. native thumbnail generation support, as well as changes on our processes going forward.
Buscarron is the new backoffice of the etke.cc and its main feature is to receive web forms (HTML/HTTP POST) and convert them into (encrypted) matrix messages.
So, what's new?
Email confirmation after form submissions with Postmark
If you use Matrix with the Signal bridge (or any bridge) with relay mode
enabled, your guests have probably experienced the confusion where they don't know who else (on Matrix) is in the chat as only Matrix can double-puppet.
When this bot is added to a Matrix room, it will announce:
Who the room members are when a new Signal user joins;
When a Matrix user joins or leaves, who that Matrix user was.
I built it just this afternoon using the excellent matrix-bot-sdk, but it's in Typescript and has plentyful unit tests!
PRs for adding support for other chat apps (basically identifying which bridge from the puppeted username) are very welcome! https://github.com/jakecoppinger/whos-in-this-room-matrix-bot
The videos from the HYTRADBOI conference a few weeks back are now publicly available, in case anyone was curious but couldn't make the conference. I gave a talk about building collaborative, open software on Matrix. The whole conference was great, so I recommend browsing the schedule for videos that may interest you. 😄
Loomio, an open source collaborative decision making tool, now supports integrating with Matrix rooms using their chatbot. They previously supported other (lesser) chat systems, but have recently added support for Matrix.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
"Wowza, that's a lot of new MSCs!" I hear you say. This is actually a listing of the last 3 weeks, as it was pointed out to me that the last 3 editions of TWIM have not all features an MSC status update - which means some MSCs may not have necessary made it into the limelight.
As such, here's those from this past week, as well as the ones that were missed!
Spec Updates
It's that time of the year again - a new Spec release is on the horizon! Matrix v1.3 is expected to land sometime this month, and will include:
Any other MSCs which have so far been accepted, but not yet made it into the spec.
MSCs which land between now and v1.3 might get incorporated if easy enough, though chances are they'll slip to v1.4 (expected next quarter).
On top of that, the other bit of news is that the proposal-in-review label has now been removed from the matrix-spec-proposals repo and thus all MSCs. This doesn't mean your MSC is no longer in review! We simply found the label redundant along with the other labels in the process (it was attached to all MSCs which hadn't been accepted/merged yet, which means very little) and thus we decided to remove it to reduce noise.
This MSC has actually been potentially(?) superseded by MSC3382. In short, both look to provide a solution to the problem of attaching (multiple!) images to messages, such that clients can display an album of images in a single event in the timeline.
Check it out if you're interested in introducing such a feature!
Synapse v1.58.0 was released this week! Check out the v1.58 blog post for the full scoop, but the highlights are:
Our Docker images and Debian packages are now built from a dependency lockfile courtesy of Poetry. Props to David for leading the charge on this multi-week epic.
An experimental implementation of MSC3383 has been contributed by @Bubu and @jcgruenhage at Famedly, which will help with proxying federation traffic, amongst other uses.
An experimental implementation of MSC2815 has been merged, which allows room moderators to view redact event content for the purposes of moderation. This was contributed by @tulir at Beeper.
Device list updates are now asynchronous thanks to Erik, leading to much faster login times for accounts in big rooms.
plus many other bug fixes and improvements!
We also released a v1.58.1 patch shortly afterwards to address an issue with the Debian packages on some configurations. Server admins not using the Matrix.org Debian packages do not need to install this patch release.
Otherwise, ongoing work from the core team continues with Sean and Rich working on faster room join performance, Olivier pushing forwards on testing Synapse with workers in Complement, while Erik has been tackling general performance work.
Hey folks! We're announcing our 1.6.0 release for matrix-hookshot today. More features in this release: tadzik has gotten involved and given us support for RSS feeds in this release. You can check out how to get going with those in our hosted documentation.
Add support for GitLab in the widgets configuration UI. (#320)
Send a notice when a GitLab merge request gets some review comments. (#314)
Add new !hookshot gitlab project command to configure project bridges in rooms. See the docs for instructions. (#321)
New hookshot_connection_event(_failed) metrics for tracking successful event handling.
Reinstate matrix_* metrics which were previously not being recorded. (#312)
This week brings another round of big improvements & new features!
Features & fixes include:
Fixes to Docker support & the documentation for setting it up
Changes to the login flow
The bridge can now be configured to enforce a login flow that more closely matches the official KakaoTalk desktop client's, and to better warn you when a new login will kick out another one of your KT sessions. This will hopefully prevent the bridge from getting KT accounts banned!
Backfilling for KT->Matrix read receipts
Commands for adding/removing KakaoTalk friends
Commands to edit your KakaoTalk ID & mark it as shared or hidden
Matrix->KT DM creation, i.e. the ability to create a new DM with a KakaoTalk user from within Matrix
This might fail on the first attempt at creating a DM. If it does, try re-inviting the KakaoTalk puppet to the room.
Many stability & correctness improvements
With the combination of friends list curation & DM creation support, it should now be possible to connect with KT contacts entirely from Matrix! Simply add some friends, list them, and invite a listed friend to a 1:1 room!
This week we released Thunderbird Beta 101.0b1. It includes a bunch of bug fixes for the Matrix implementation and shows reactions as system messages, so you know when someone gives you a 👍️.
Someone on Twitter complained that no chat client works like MSN anymore, where you could have separate windows for each chat. So for MSN compatibility we added that. (It is a bit experimental still, expect some stuff to not work or crash)
Bulbu also went through a lot of effort to add a second set of emoji shortcodes to Nheko. Those should be more familiar to people coming from other platforms or clients while the old shortcodes are available still as well. We also saw a lot of progress on the Finnish, Chinese, Indonesian and Russian translations, animated images should work properly in the image viewer again, buttons for opening the member list are now clearly marked with a people icon next to them. The memberlist is also now search- and sortable.
Anyway, have a look at how important multiple windows are:
FluffyChat 1.4.0 has been released with a lot of bug fixes, performance improvements and new designs. Take a look at the new designed sign up screen, which also now supports recommended homeservers from joinmatrix.org
Ement.el has received several updates since the last time it was mentioned in TWIM:
Additions
A new room list command, ement-taxy-room-list, shows rooms grouped "smartly" (and programmably). (It will be made the default soon; until then, call the command to use it.)
The new room list shows unread-notification/highlight counts, similar to Element.
Basic support for spaces:
Rooms (and spaces) can be added to and removed from spaces.
Rooms are listed under their spaces in ement-taxy-room-list.
Reactions can now be toggled off (i.e. redacted) by clicking them, like in Element.
Command ement-room-send-file allows uploading non-image files to rooms, and file messages are now displayed usefully, so the files can be downloaded or viewed.
Command ement-ignore-user ignores and unignores users.
Command ement-tag-room adds and removes favourite, low-priority, or custom tags on rooms.
Command ement-room-occur allows searching messages in a room buffer, similar to Emacs's occur command (i.e. it locally searches already-received messages).
Membership events are now coalesced by default, similar to Element (especially helpful in large public rooms, like bridged IRC rooms, where join/part spam can be overwhelming).
"In reply to" links in replies, as well as other matrix.to links, can be activated in a room buffer to go to the linked message.
Mentions in outgoing messages are HTML-linkified (aka "pilled"), like Element.
Generated avatars for rooms that have none, similar to Element. (Only displayed in ement-taxy-room-list for now.)
Option ement-shr-use-fonts (disabled by default, which prevents HTML messages from being displayed in a different font).
Faces for favourite/low-priority rooms.
Redacted messages that are still known locally are now displayed with a strikethrough (after reconnecting, their content will be unknown).
When images are disabled, image messages are presented as clickable URLs, so they can still be viewed.
Some events, like membership events and images, are displayed alongside a vertical bar to help distinguish them from normal messages.
Even "hotter"-looking timestamps for rooms updated in the last hour.
Changes
Refactored notification system. Now easier to configure (see M-x customize-group RET ement-notify RET).
Fixes
Improved error handling when looking up homeserver hostnames and .well-known URIs.
Improved error handling when processing malformed events.
"Elemental" display style (not used by many users, apparently).
In this months demos (also shown off at the top of the TWIM post), Eric showed off a proof of concept to anonymize your screenshots so you can easily include more context when submitting logs or bug reports without worrying where the bug occurred. The current concept will color block all of the text on the page to still maintain the structure of the app but completely hide the content itself.
This still has a ways to go before it's integrated in Element under the click of a button so if anyone wants to take over or give feedback, go comment on the issue, https://github.com/vector-im/element-web/issues/9615. If you want to use the anonymization now, you can manually run the JavaScript snippet from the issue in the browser devtools.
This mode could even be further extended further to include event_id labels in the corner of each message to easily correlate the logs to the screenshot.
We are testing out Sonarcloud, you’ll see comments from the sonarcloud bot in new PRs about bugs, vulnerabilities and other inconsistencies that it has found
Our first time user experience is getting some attention in Web. We’re working on simplifying a user’s first session as we know it can be pretty daunting
We’re thinking about new ways to give flexibility on the Home screen - watch this space!
The new Create Account flow is being implemented, currently we’re working on the captcha and T&Cs pages
Fix several crashes like when searching or when clicking bottom bar twice
Improve interaction with spaces which are in a space
ElementX-iOS
We are going to share more and more about ElementX-iOS, a complete rewrite of the Element-iOS app using matrix-rust-sdk. We made a prototype last quarter. The result was so promising that we want to go full steam on it
Work is moving forward with our proposals for a new app layout, we’re excited to share it with you
Our Create Account experience is still being updated, we’re currently working on ensuring that the Sign-in pieces work as expected
Fixed a number of crash bugs around calls, launching app and space list
Notifications have have been improved so that there is a noise for every notification which is received when they are enabled, in line with how web and iOS work
On the SDK side, we are working to improve / clarify / document the public API. It’s still in progress, but the generated API documentation is already visible here: https://matrix-org.github.io/matrix-android-sdk2/
Over the last two weeks, the theme in populus-viewer development has been permissions:
The settings menu for rooms has been redesigned
It's now possible to assign and remove standard roles (admin and moderator) from populus viewer
It's now possible to set power levels for standard events and actions from populus viewer
It's now possible to ban and unban using the membership management modal
Redacting the messages of others is now exposed as an action for those with that permission.
Besides that, I've been having some fun on the css-overhaul branch reorganizing some design elements, and centralizing the color CSS in a few variables, opening the way for things like a dark mode and user-definable themes. Here's a peek at the new milder and more welcoming default look:
The permissions work now has us at one thousand commits. Here's to the next thousand! 🎉 As always, if you want to learn more, follow project progress, or talk about the future of social annotation on matrix, come join us at #opentower:matrix.org.
I'm working on a few features for Wrench's anniversary on 13th June.
Here's a bug fix for storing identities and a new button to check your membership status in a room.
Added: A "Am I a member?" button tells your membership state in a room.
Fixed: Unchanged identities got deleted from localStorage when editing identities twice.
After a few release candidates and weeks later, we released Trixnity 2.0.0 today! It contains many breaking changes due to a large refactoring, which allows us to share a lot code between server and client implementations of the Matrix APIs.
This means, that Trixnity can be used to implement matrix servers in addition to application services and clients!
Trixnity finally has support for Kotlin/JS. You can implement a browser client or even better: multiplatform client. Support for Kotlin/Native will follow soon (e. g. for iOS clients). Right now, there is no trixnity-client-store implementation for JavaScript (e. g. IndexedDB), so you need to stick with InMemoryStore at the moment.
Besides notification support (push rules are evaluated), we introduced helpers to get the complete timeline as Kotlin-Flow (no more complicated loops to get the timeline). You can even subscribe to all timeline events, which is really helpful to implement bots with e2e encryption support. Speaking of e2e encryption: libolm is bundled into trixnity-olm jars, so you don't need to build olm by your own anymore.
These are the most important features. For the complete changelog, have a look at the release candidates.
This week the team released version 0.9.0 which contains new features and some refactoring as many deprecated endpoints were removed so make sure to use the new ones when doing the migration.
Support for the Element recent emoji (io.element.recent_emoji) was added. A new method to get the last read marker was added.
Lastly, the timeline logic was enhanced, and it's now possible to load the timeline at a specific event. So you can load the room to the last event and then paginate through the new events. When reaching the last event, the timeline will automatically allow new events to be added in the timeline when received.
I have spent the day ignoring more important things and trying to have something to TWIM. I wrote (yet another) GitHub Action for sending build status to matrix rooms. This action looks at all the completed jobs in your workflow and posts a combined status for the workflow as a message and reactions for all the individual jobs, inspired by the maubot GitLab plugin.
Hello together. I made a generic bot, named Safed Sundarata, or Sundar in short, that should make it easy to get basic special functionality working. It has a lot of different functionality so you have an example code to work off of. The main motivation for this bot was to provide an !error <code> command for the developers in the ReactOS Matrix Rooms, which translates error codes to human readable text. Then of course i added other useful or fun commands to the bot. You want to read the Readme to find out more.
This is the first time i ever wrote a bot, so feedback and corrections to the Source Code are highly appreciated. It is written in Go using the mautrix-go framework (thanks tulir and everyone who worked on this making it easy!). Even easier is to run and build it with go, no insane build system with broken dependencies required, which was the reason why i made it from scratch.
We have released v1.4.2 of Mjolnir https://github.com/matrix-org/mjolnir/releases/tag/v1.4.2 which includes:
Bot:
A new JoinWaveShortCircuit protection, thanks to Jonathan with https://github.com/matrix-org/mjolnir/pull/280. This protection can be used to detect a mass-join scenario and set the room to invite-only.
Change of behaviour: Mjolnir will now apply server ACL and member bans to the most recently active rooms first (while syncing). The order was random before.
The causes of errors at startup (e.g. via misconfiguration) have been made more clear.
The image with the latest tag on dockerhub is now correctly in sync with the main branch.
I am releasing a brand new moderation bot, graim! This bot allows you to moderate communities that are bridged with Discord via matrix-appservice-discord. You can tie a Matrix account to a Discord account, and moderate both platforms seamlessly from either end! Graim supports one Discord server and as many Matrix rooms as you want, per instance. :)
You can run it yourself from the Github - https://github.com/luphoria/graim
You can join the matrix space, which is bridged to the Discord server
I'm streaming every week live coding Rust and Matrix at https://twitch.tv/andybalaam and https://andybalaam.uk.to (Owncast). Feel free to join me as I write a Rust Matrix bot. I'm currently on a side quest to add Redis support to the matrix-rust-sdk crypto store. You can watch past streams at diode.zone/a/andybalaam.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Synapse 1.58 is
out! Let's dive into this new release.
Poetry
If you've been reading the past few Synapse release announcements, you might
have already heard about our efforts to integrate
Poetry into Synapse. Poetry is a tool which
manages dependencies of a Python project. Its killer feature is its lockfile,
which explicitly pins all dependencies (direct and transitive) to a fixed
version, even across multiple Python versions and platforms. See more about why
we choose Poetry here.
During the past few months, we've been hard at work converting Synapse and its
CI to use Poetry. The goal is to make our Docker images and Debian packages more
reproducible and to provide a fixed, known-good environment for CI. This will
help us to ensure the stability and security of Synapse installations.
Synapse 1.57's Docker image was the first to use Poetry; after a successful
trial, this was extended to Synapse 1.58's Debian packages. Installations from
PyPI using pip will not use the locked environment, though everyone is
welcome to install from source using our
lockfile.
Huge props to David from the Synapse team for
leading the work on this front.
Everything else
This release of Synapse also includes performance improvements around sharing
device list updates, which should greatly improve login times for large Matrix
accounts.
Synapse 1.58 also features the implementation of two MSCs:
MSC3383 to
include a destination parameter in federation authentication headers, and
MSC2815 which
(if enabled) allows room moderators to see the content of a redacted event (as
long as it hasn't already been deleted by the homeserver).
See the full
changelog for a
complete list of changes in this release. Also please have a look at the
upgrade
notes
for this version.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper, Dirk
Klimpel, Famedly and
Sami Olmari.
We've released updates to matrix-appservice-irc and our forked node-irc that it depends on to patch a High security vulnerability.
It's advised to update to 0.34.0 as soon as possible.
The vulnerability allows an attacker to manipulate a Matrix user into executing IRC commands
by having them reply to a maliciously crafted message.
Incorrect handling of a CR character allowed for making part of the message be sent to the IRC server verbatim
rather than as a message to the channel.
If you are currently a matrix-appservice-irc user, exercise caution when replying to messages from untrusted participants
in IRC bridged rooms until your bridge instance has been upgraded.
The vulnerability has been patched in node-irc version 1.2.1 and matrix-appservice-irc 0.34.0.
You can get the release on Github.
The bridges running on the Libera Chat, OFTC and other networks bridged by the Matrix.org Foundation have been patched.
Taking my GNOME Foundation hat on for this news: the GNOME Foundation is adopting Matrix! Is this TWIM worthy? It probably wouldn't have been if it wasn't for all the great work of Gwmngilfen! We published a joint blog post (though in all fairness Greg did all the heavy lifting) to explain how the decision was taken, and transparently share the data we had at hand.
In practice that means after years of Matrix being around in the GNOME community and being the most popular platform, it's finally moving out of the grey zone of "everyone is using it but some older contributors really like their current set up, so it exists but it's not officially our primary platform". For now we're keeping both our Matrix instance and the bridge to GIMPnet (the IRC network GNOME people have been using for years). The major change is that IRC is now our legacy, so if tradeoffs have to be made Matrix will keep the premium experience.
Again, huge shout out to Gwmngilfen who gave meaning to the data I handed to him.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
Hello again everyone! Your regularly-scheduled TWIM Spec update has returned, along with graph-based media!
Quick news to point out this week is that a new maubot plugin has been written to simply note the details of MSCs after their IDs ("msc1234") have been posted in a room. The bot supports multiple MSC IDs in a message, message edits, threads and end-to-end encrypted rooms.
A copy of the bot is currently running under @mscbot:matrix.org, so feel free to invite it to a room if you'd find that useful!
This MSC serves as an alternative to MSC2962, and essentially allows for user permissions to be simultaneously enforced across a set of rooms in a Space. This would allow functionality to users of platforms like Discord, where a user can be a moderator in a guild, and thus in all of that guild's channels.
A few operational issues have taken up quite a bit of the team's time this week, but we've still found time to cut a couple of release candidates of v1.58. Highlights include:
Locked python dependencies for our Docker images and Debian packages, thanks to David's poetry work.
We're also excited to see quite a few contributions from outside the core maintenance team: thanks to Dirk, Beeper, Famedly and everyone else in the community who has made pull requests!
Meanwhile on the core team, Sean has been continuing to lay groundwork for faster room joins, and Olivier has been doing some exciting work so that we will (finally) be able to test Synapse in worker mode under Complement.
This week we announced Dendrite 0.8.2 with a bunch of usability fixes. As always, join us in #dendrite:matrix.org for more discussion and updates.
Features
Lazy-loading has been added to the /sync endpoint, which should speed up syncs considerably
Filtering has been added to the /messages endpoint
The room summary now contains "heroes" (up to 5 users in the room) for clients to display when no room name is set
The existing lazy-loading caches will now be used by /messages and /context so that member events will not be sent to clients more times than necessary
The account data stream now uses the provided filters
The built-in NATS Server has been updated to version 2.8.0
The /state and /state_ids endpoints will now return M_NOT_FOUND for rejected events
Repeated calls to the /redact endpoint will now be idempotent when a transaction ID is given
Dendrite should now be able to run as a Windows service under Service Control Manager
Fixes
Fictitious presence updates will no longer be created for users which have not sent us presence updates, which should speed up complete syncs considerably
Uploading cross-signing device signatures should now be more reliable, fixing a number of bugs with cross-signing
All account data should now be sent properly on a complete sync, which should eliminate problems with client settings or key backups appearing to be missing
Account data will now be limited correctly on incremental syncs, returning the stream position of the most recent update rather than the latest stream position
Account data will not be sent for parted rooms, which should reduce the number of left/forgotten rooms reappearing in clients as empty rooms
The TURN username hash has been fixed which should help to resolve some problems when using TURN for voice calls (contributed by fcwoknhenuxdfiyv)
Push rules can no longer be modified using the account data endpoints
Querying account availability should now work properly in polylith deployments
A number of bugs with sync filters have been fixed
A default sync filter will now be used if the request contains a filter ID that does not exist
The pushkey_ts field is now using seconds instead of milliseconds
A race condition when gracefully shutting down has been fixed, so JetStream should no longer cause the process to exit before other Dendrite components are finished shutting down
Let us know your feedback as we continue to work on this feature.
The next step for Threads is improving our notifications and push rules on threaded messages. We’ve drafted some MSCs and are excited to move them forwards.
Community testing
Thank you all who joined the async community testing sessions.
The leaving space experience is now in line with web: you can leave all rooms, none of the rooms or only selected ones
Our first time user experience and create account flow is being improved.
Started prototypes for IA experiments. We’re hoping to make the app much easier to use and will test out new interactions and patterns to see what works for users.
Hard at work fixing bugs - we’ve squashed one where the UI would freeze after incremental syncs and we’ve also made improvements to the read line and read markers.
Our team is still working on the sign up flow. It’ll be ready soon and we hope that new-to-element users will find it much easier (and less scary) to create an account and get chatting.
We’ve started designing and building prototypes for a new home experience; we’re excited to start experimenting with ways we can simplify our app.
A set of Rust library crates for working with the Matrix protocol. Ruma’s approach to Matrix emphasizes correctness, security, stability and performance.
Our last update included the words "if things go to plan", and even though that's usually calling for trouble, not so this time!
We released Ruma 0.6.0 yesterday and apart from a tiny missing feature that was fixed and released as 0.6.1 today, everything went smooth 🙂
The biggest breaking changes are
new owned identifier types – calling .to_owned() on &UserId now gives you an OwnedUserId instead of Box<UserId>
a new event enum hierarchy, where redacted events are better taken into account
Also, with this release we can now claim practically 100% coverage of Matrix v1.2, that is you can send and receive requests for all specified endpoints of the various APIs, as well as send and inspect all specified event types, all with a strongly-typed Rust API.
There were no news about miounne for a while because we worked on a new "iteration" that can handle matrix encryption (thanks to the mautrix-go).
So, here is it - Buscarron v1.0.0 with e2e encryption and multiarch (arm32, arm64, amd64) support. It can handle any HTTP POST/HTML form and send it to a (encrypted) matrix room. Of course, Buscarron has small neat features like spam checker and rate limiter.
etke.cc is migrating to Buscarron and deprecating Miounne.
I wrote a small, extensible bot that replays GPX recordings as a location sharing stream.
This is an implementation of MSC3489: Sharing streams of location data with history and may help you debug the feature or see it with real world data.
Join my Space to see this in action (with the Element Web Labs feature): #geo-trackers:iot-staging.ems.host
or run the bot with your own rooms and GPX files. https://gitlab.com/jaller94/matrix-location-sharing
Opsdroid 0.26 has been released this week. Included in this release are some improvements to the Rasa NLU parser if you want natural language training for your matrix bots as well as some small documentation improvements to the matrix connector and other bug fixes. Big thanks to Oleg for many of the changes as well as all the other contributors.
Matrix Community Manager has seen some work in the features department over the past week.
With the new Ansible playbook I think it's possible for someone with a bit of Ansible experience to get this bot up and running in under 10 minutes. I did a fresh install in 4 minutes. That is a fresh VPS and creating a new matrix account. Instructions for the Ansible deploy can be found here.
The major features are:
An Ansible playbook that should simplify the installation process for new users.
Edits to the message that triggered the bot will now be reflected in the bots message. This bot now has a feature Twitter doesn't have :laughing:.
Hashtags are now called filters. Filters are more customizable allowing for custom delimiters and now show the originating room.
The AutoJoin feature has filters to control who or from where invites will be excepted.
Announcements can have more than one output room. Using a comma separated list of room ids.
A few bugs were fixed as well:
Some text characters were causing messages to be classed incorrectly thus causing crashes.
Exotic characters specifically in links were causing crashes.
We're meeting on Fri, 6th May at 6:30 PM in the c-base.
Please join us, if you're looking for locals who work with Matrix and use it regularly.
Our Matrix room is Matrix Meetup Berlin.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
This week was a little on the quieter side again, with some of us away on holiday and others away with illness. Nevertheless, this week develop saw a variety of activity. Some personal highlights:
Shay's effort to document Synapse's numerous config optionslanded. This is a huge and comprehensive effort that'll help administrators to understand all the options they have to configure Synapse.
A subtle bugfix from Erik which corrects incorrect counts in one of the Admin API endpoints.
The remainder of the poetry work has landed. There were a few last-minute complications in CI to smooth out, but we've now finished the migration. Synapse's docker images and debian package now come with a fixed set of dependencies (even transitive ones) in a way that lets us support multiple Python versions and interpreters.
We're looking forward to shipping these in Synapse 1.58, whose RC is scheduled for Tuesday the 26th of April.
I'm also keen to see how PR 12522 from Rich and Olivier progresses. It aims to fix a nasty memory bug which can cause Synapse to get stuck in an OOM loop.
As ever, I'd like to express thanks to our community of users, server operators and contributors. Thanks are also due to @callahad: today is his last day of managing the Synapse team. We're all going to miss him and wish him well: thank you for everything you'd done over the last year and a half.
As a follow on from our previous launches, the team is building “live location sharing”. This will allow you to automatically share where you are in the world with others for a period you choose (15 minutes, 1 hour, or 8 hours), instead of sending a single static location.
Community testing
We ran our first async community testing session this week. Thank you all who joined in! We found 13 issues so far, which is good going.
We have some new features landing for the search experience on Web, join us for testing from Thursday!
Threads is in Beta! The team is hard at work smashing bugs and improving notifications so that we can remove the beta label and get everybody access to our new feature!
If you’re using Threads, don’t forget to give us your feedback.
This week the team discussed how our Web product looks and feels to new users. While we use our own products all the time, it can be good to take a step back and ensure that our product decisions benefit all users, including the newbies. Watch this space for more info in the coming weeks as we continue to trial things that we hope will make onboarding to Element easier.
In labs (you can enable labs in settings on develop.element.io or on Nightly)
Video rooms are coming together nicely, with the addition of lobby screens and room list stickiness to wrap up the design implementation
The newest version of the app (1.8.13) is ready! In this release we’ve made a bunch of improvements and bug fixes, check it out here.
Our favourites include: Force touching room previews no longer update your read receipts, there’s a congratulations screen for new users when creating an account, and we’ve reduced the number of home page reloads.
Also new in product are presence indicators! From your home page or DMs you’ll be able to see if a user is present or away. Note; this is only supported if your homeserver has it enabled.
Upcoming; We’re working on adding ‘mention pills’ when tagging users in a room.
Please note; We’ll soon be removing support for iOS 12 and iOS 13
The team is working on fixing bugs and making improvements to flows like “ignore user”. We’ve also made some changes to the way we format code.
Element Android 1.4.12 is ready to be released but currently on hold in the Play Store due to rules around User Generated Content (UGC). We’re chatting with Google and the new update should be available soon.
We've added two major user-facing features to populus viewer:
A two-up view in the pdf viewer, allowing you to view and annotate two pages at once on a wider screen
An audio viewer, complete with audio annotation, so that you can annotate temporal segments of an audio recording
I'm especially excited about the latter - being able to discuss audio recordings will be a great teaching tool, and I'm hoping it might also be useful for things like people who want to talk about podcasts together (for purposes of editing, or just for fun), and people who want to annotate recordings of public talks of various kinds. The last use case would also be pretty well served by video annotation - so that's now rising on the agenda. I've created an MSC, here where I'll be filling in the details of a proposed format for interoperable audiovisual media annotations. As always, if you've got questions, want to follow development progress, or just want to talk about the future of social annotation on Matrix, come join us at #opentower:matrix.org.
nheko-krunner has finally been released! Version 0.1.0 contains several handy features such as searching your rooms, joining new rooms, and starting and switching to direct chats. Furthermore, if a room has unread notifications, nheko-krunner will let you know by tacking on a notification count to the room name (e.g. This Week in Matrix (TWIM) (1). Currently this feature is not disable-able, but I plan to dive into writing KCMs (KConfig Modules) soon so that I can add a toggle for this.
Hopefully a few people find this handy, and remember that if you don't use KDE, nheko still has a handy D-Bus API that you can use to write integrations with other software.
We haven't done any regular posts about it in a while, but the SDK sees almost weekly releases. (We did release version 0.8.20 last week) If you forgot about it, the Matrix Dart SDK is the SDK FluffyChat, the Famedly clients and a few others use. Let me summarize some of the changes for those who haven't kept up:
One of the killer features is probably WebRTC support 🚀. It handles all the signalling for you, which makes integrating call support into your Flutter or Dart Matrix client fairly simple. Currently it supports version 1 of the call protocol, which is still an MSC and implements a few of the other MSCs. Experiments with group calls are still in progress.
We also switched databases a few times and tuned it more (we posted about FluffyBox last year). The database backend should now be a lot more stable and more performant, while also using less memory.
Continuing with stability, we worked a lot on how the timeline is stored and structured, leading to a much more robust experience. The SDK is also now completely nullsafe 🎉, which gets rid of a lot of annoying error modes. There were some fixes to E2EE here and there, but mostly the experience has been really solid and didn't need much work.
Some of the more recent developments include widget support as well as improving the file upload experience. The SDK now inserts a dummy event for a pending upload, images are thumbnailed and resided in a different isolate and uploads are automatically retried after a connection failure. We are also working on improving the push support, making it easier to fetch events from a push.
And I'm currently working on bringing fragmented timeline support. Nico told me it would be pretty (not) easy to do... and I have to say, he was right. Good first issue 🙃
I've created a NixOS module, nix-matrix-appservices, that can simplify the bootstrap and maintenance of matrix appservices on nixos. With it, appservices can be spun up easily with registration and tokens handled by the module. Most mautrix and mx-puppet bridges have been tested to work and they can be setup in a couple lines of nix code. There are plenty of options you can make use of to configure more complex appservices.
You can find the the code, matrix room(gitlab badge), and usage information here: https://gitlab.com/coffeetables/nix-matrix-appservices
If you are looking for a docker container with just the auto compressor for linux amd64/arm64 in it, her you go: https://gitlab.com/mb-saces/rust-synapse-compress-state
PS: You can also help us to design the underlying MSC, especially, we need some opinions on how to name things (the profile rooms) the room !!
Dept of Ping 🏓
We know you missed it, here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Hey everyone! Guess what? Synapse
1.57 has just been
released! Let's see what's new in this version!
Application services
Application services are piece of software that have privileged access to some
of the homeserver's features. For example, they can create and manage multiple
users at the same time, which is especially useful for bridges.
This release of Synapse changes the way Synapse manually manages transaction
identifiers when talking to application services (a transaction being a group of
events the application service should know about). While this doesn't have much
impact on the everyday life of a server administrator (besides fixing a
bug and paving the way for
future performance improvements), this change means server admins should take
extra care when updating Synapse.
More specifically, if you are running a dedicated Synapse worker for handling
traffic related to application services, this worker must be stopped when
upgrading the main Synapse process to ensure the update is performed safely. See
the upgrade
notes
for more information about this change, and instructions on recovering from an
incorrect upgrade.
This release of Synapse also continues work on bringing end-to-end capabilities
to application services, which I was already telling you about in the Synapse
1.50 release blog
post. More
specifically, Synapse now supports sending device list updates to applications
services, as part of implementing
MSC3202. This
is still very experimental and definitely not production ready, but also very
exciting!
Improvements to the module system
Modules allow third-party developers to expand Synapse with extra features that
wouldn't necessarily fit into the Matrix specification and/or ecosystem. In the
past release of Synapse, we have been improving this system to add more
functionality to it, and this one is no exception!
Synapse modules can now implement new callbacks to react to account data
updates,
as well as to react to new 3PID (email address, phone number)
associations.
On the latter, note that this callback will only be called after a 3PID has been
validated on the homeserver, and does not trigger when the validation happens on
an identity server (e.g. when publishing a 3PID so that other users can look it
up).
The ModuleApi (which is the Python class enabling modules to interact with
Synapse) has also been updated to allow module to read and write global account
data. This can be done by using the new
AccountDataManager
class, which can be accessed as api.account_data_manager (where api is an
instance of ModuleApi).
The module API has also been updated with a new
method,
to allow modules to promote an existing user to server administrator (or demote
a server administrator to a normal user). This follows up on an improvement
introduced in Synapse
1.56 allowing modules
to promote users to server administrators when registering them.
Everything else
This release also includes a performance improvement for workers handling
/sync requests. While this change makes starting this kind of workers slightly
more heavy performance-wise, it aims at improving the load associated with the
first /sync requests hitting it right after starting. See this
comment
for more details.
Synapse 1.57 also now includes bundled aggregations in message search results by
default, as
MSC3666 has
been accepted and has finished its final comment period.
See the full
changelog for a
complete list of changes in this release. Also please have a look at the
upgrade
notes
for this version.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper, Dirk
Klimpel, Famedly and
Jorge Florian.
Plus a number of bugfixes within Synapse and fixes for spec compliance issues.
As ever, our warmest thanks to to our contributors for sharing their time, effort and expertise with the project. We look forward to seeing your work in the wild, come the 1.57 release proper on Tuesday the 19th of April.
On develop, we have been powering through a number of different tasks:
another spurt of work to get complement (finally!) testing Synapse in worker mode.
Speaking personally, my efforts to manage Synapse's dependencies using poetry are coming to a close. As I write this, I've just submitted my last (planned!) PR for this project. It's been several months in progress now, and there were many moving parts; some unexpected bugs in poetry and even one in pip; plus a heck of a lot of CI config to change and understand. My sincere thanks to the broader Synapse team for their help in getting the last batch of changes made, reviewed and merged. Once this is all said and done I hope we'll all reap the benefits of
a stabler CI pipeline,
reproducible virtual environments in our docker and debian images, and
Hey webhook-consuming-people! The hookshot webhook bridge 1.5.0 update has now been released. This release is a bit softer than 1.4.0, containing some smaller features and bugfixes. Thanks to everyone who contributed!. The highlights are:
Allow specifying msgtype for generic webhook transformations. (#282)
Add new GitHubRepo config optionnewIssue.labels which allows admins to automatically set labels on new issues. (#292)
Allow priority ordering of connection command handling by setting apriority: number key in the state event content. (#293)
After a long and arduous journey, LorenDB's dbus interface for Nheko got finally merged. Currently this allows you to list rooms as well as switch between them. In the future we might add simple messaging support, so that you can automatically send messages into specific rooms using scripts. The API is disabled by default, since it is not authenticated. Enable it at your own risk!
Another long standing PR, that finally got merged, was MTRNord (they/them)'s support for pretty power level formatting. Nheko can now tell you what changed in a power level instead of just telling you that it changed. This should make them much more accessible to users, that don't like looking at the raw event source all the time (weird, eh?).
The image viewer should also be more reliable now and notification counts should be correct after a restart.
We have also been playing around with qt6 support. At this point we have a working Qt6 branch, with a few functional regressions:
No voip (gstreamer has no Qt6 support yet)
No drop shadows on buttons
No switchable color schemes
So far Qt6 looks pretty great and seems to fix a lot of the minor annoyances.
We are also still looking for some student or interested individuum to take part in the Google Summer of Code and improve the call support in Nheko! Deadline is drawing close, so if you intend to apply, better work on your proposal quickly! If you need any feedback for your proposal, feel free to DM me and ask me to review it/questions. I'm looking forward to someone applying! <3
Anyway, merry christmas everyone or whatever you are celebrating this weekend!
We will inform users in the next release that we will no longer support iOS12 and 13. It will be effective the release after, i.e. April 25th.
We had a big drop on UTDs (Unable to Decrypt errors) since 1.8.8.
We are setting up the ElementX-iOS project to rewrite the Element-iOS app. It will be based on SwiftUI and the Matrix Rust SDK. The roadmap is available here.
We may experience some delays in shipping the app through the Android PlayStore as we are in discussions about whether not decrypted messages should be reportable
Cleanup to the SDK API has landed on develop - this will help with generating API documentation and migrating to the Rust SDK
Exciting news this week; the Threads Beta is live on all platforms! If you haven’t seen Threads yet, head to Settings > Labs and check it out.
Please note; If your homeserver doesn’t yet support threads functionality you may not see the beta, or your experience will be degraded. If this applies to you, you’ll see a warning message before you activate Threads.
The team is continuing to squash bugs and improve performance, we’ve also started working on the next improvements…
Notifications and badges are top of mind for us next. We need to ensure accuracy of badge counters and consistency between platforms (if you use more than one). We’re looking at this now so keep your eyes peeled for our upcoming MSCs.
libolm 3.2.11 has been released. This release mainly features improvements to the Objective-C and Java bindings, but also fixes building of the documentation, which was broken for quite a while. For more details, see the release notes.
A set of Rust library crates for working with the Matrix protocol. Ruma’s approach to Matrix emphasizes correctness, security, stability and performance.
Over the last two weeks, we have been busy working towards the 0.6.0 crates.io release of Ruma. If things go to plan, it should be published in the coming weeks (but no promises). That means mostly finishing some large refactorings, but we also got some bug fixes in, as well as a number of convenience functions for working with certain events:
HASSkbot can be used to control an action in Home-Assistant via Matrix reactions.
It's an Opsdroid bot, which reacts to a home-assistant entity state change (like person leaving a zone) and asks for confirmation in Matrix if it should run additional home-assistant automation.
Matrix reactions are used for confirming or canceling an action.
benpa is back in town and I can't keep silence: here is a small update on Honoroit - v0.9.6 comes with "stable spaces" support that will work with new versions of element (yes, even on Element Android from F-Droid).
DWeb Camp is a gathering hosted by Internet Archive and volunteers dedicated to Decentralized/Distributed web projects. Previously it happened in 2019, and Matrix had some presence there.
Maybe people from Matrix projects and/or Matrix.org will be there this year?
That's all I know 🏁
Come back next week for a much restored TWIM! With Matrix Live, Spec updates, Dept of Ping and so on. :D
See you next week, and be sure to stop by #twim:matrix.org with your updates!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, we've been largely looking at proposals which are at or near FCP in an effort to get them through the last stages of the process. We're also thinking about what the next release (v1.3) looks like and when we'll end up putting it out into the world. If you have MSCs which you'd like included, please stop by the #sct-office:matrix.org on Matrix with your suggestions - if they're near enough, we'll try to get them in.
Random MSC of the week
The script has elected MSC3391: Removing account data as the random MSC this week. It's a small but interesting MSC which helps clean up account data on the server when it's no longer needed, though how this sort of removal gets represented to clients can be a challenge. It's currently missing an implementation if someone is looking for a medium complexity contribution this weekend 😉
The Chart
So many MSCs are in the open state, which is why we're continually looking at merging MSCs which are ready to go.
This week, we've released Synapse 1.56! It includes only a couple of new features but quite a few bug fixes and internal improvements. One of the main changes included in this version is that Synapse will now refuse to start if configured with open registration with no verification (e.g. email, recaptcha, etc). This is an attempt at reducing the likelihood of spam across the federation, as most cases of abuse we've observed over time usually involves the attacker(s) finding homeservers with open registration and automatically creating a lot of accounts on them in order to evade sanctions.
This version of Synapse also deprecates the groups/communities feature of Matrix. This is a feature we introduced back in 2017, and was the predecessor of Matrix spaces. But now that it has been mostly replaced by spaces, we have decided to retire this feature, which we thank dearly for its 4 years of good and loyal service to the federation.
Read all about this, and more, in the release announcement on the Matrix.org blog! 🙂
This week we released Dendrite 0.8.0, which is primarily a feature release, and then Dendrite 0.8.1 which fixes an emergency bug. It's also a recommended upgrade if you are running a Dendrite deployment. It includes:
Support for presence has been added
Presence is not enabled by default
The global.presence.enable_inbound and global.presence.enable_outbound configuration options allow configuring inbound and outbound presence separately
Support for room upgrades via the /room/{roomID}/upgrade endpoint has been added (contributed by DavidSpenler, alexkursell)
Support for ignoring users has been added
Joined and invite user counts are now sent in the /sync room summaries
Queued federation and stale device list updates will now be staggered at startup over an up-to 2 minute warm-up period, rather than happening all at once
Memory pressure created by the sync notifier has been reduced
The EDU server component has now been removed, with the work being moved to more relevant components
It is now possible to set the power_level_content_override when creating a room to include power levels over 100
/send_join and /state responses will now not unmarshal the JSON twice
The stream event consumer for push notifications will no longer request membership events that are irrelevant
Appservices will no longer incorrectly receive state events twice
Hello webhook fans! As teased last week, configuration widgets have landed in hookshot! 1.4.0 now contains all you need to setup your very own webhooks without having to leave the comfort of your GUI. The plan is for widgets to be greatly expanded over the new few releases to support more services. Eventually, this work is going to propagate out to other matrix.org bridgey projects 🌉.
The full feature list for this release looks a bit like:
Add support for configuring generic webhooks via widgets. (#140)
Show the closing comments on closed GitHub PRs. (#262)
Webhooks created via !hookshot webhook now have their secret URLs sent to the admin room with the user, rather than posted in the bridged room. (#265)
Automatically link GitHub issues and pull requests when an issue number is mentioned (by default, using the # prefix). (#277)
Small exception: Matrix->KT replies don't yet work in KT "open channels" yet.
Ability to create a portal by inviting a KT puppet to a DM
Note that this currently only works for KT direct chat channels that already exist & have been active recently.
Connection resilience between the Python and Node components of the bridge
i.e. If the Node component ever exits & restarts, the Python component will reconnect to it automatically. This helps both with deployment (since it allows the components to be started in any order) and crash tolerance (since a Node crash & restart no longer requires a manual restart of the Python component)
Clear warnings when receiving a KT message that the bridge doesn't yet support
At this point, the bridge should be fairly usable now. Very soon I'll open a Matrix-bridged KT channel to act as a public stress-test!
Heisenbridge is a bouncer-style Matrix IRC bridge.
Release v1.11.0 🥳
Fixed retry behavior on startup to wait for HS startup
Ignore TAGMSG messages from IRC server
Fixed HTML messages not working as commands
Fixed room aliases in messages dropping the message completely
Upgrade to Mautrix 0.15
Just your typical bug fix release but this release also breaks support for homeservers not supporting the "v3" API path so if you run Synapse 1.47 or older the bridge will not start. Sorry.
Thunderbird is a free open-source email, calendar & chat app.
The latest Thunderbird beta finally has Matrix support enabled by default. Get Thunderbird beta now to try it out.
There have been many improvements to the Matrix implementation since the last update, including:
Syphon is a Matrix client with heavy emphasis on privacy and ease of use; currently in open alpha.
Hi all 👋.
We released 0.2.13 this week mainly fixing an annoying bug that could cause messages sent while a configured proxy server was down to be re-sent multiple times once the proxy came back online. If you use a proxy with Syphon, this update is highly recommended!
In addition, this release will only show the option to use hidden read receipts if the feature is supported by your server.
Finally, a range of translation updates and improvements are included in this release.
More changes are coming soon, including a (currently work-in-progress) implementation of MSC2228 (self destructing events). As far as we know, we're on track to be the first user-facing implementation of this MSC, putting Syphon on the bleeding edge of Matrix!
Thanks to the awesome polyjuice client, Nheko now supports MSC3700, which slightly improves privacy in encrypted rooms. It also lead to us fixing issues with the secure symmetric secret storage, where some clients use a different base64 encoding than recommended in the spec, which could make unlocking the secrets with a recovery key or passphrase fail. And we also improved the key queries on initial login, which would sometimes fail to_device messages with a warning, that the device is unknown.
As a small feature, you can now close the currently open room using Ctrl-W, spaces are not treated as DMs under some circumstances anymore, you should get a less confusing error message than 500 when entering an invalid alias now and lots of fixes to the translations.
Thank you, LorenDB, Apurv and Mikaela for the contributions!
Have released the SDK, v0.0.10 with custom tiles support. Calls and theming are getting closer, the latter we were planning to release this week but hit a blocker for theming support in develop mode, so we'll have to postpone to next week.
Considering dropping support for iOS12 - this will impact 0.9% of sessions. Requiring iOS 13 or newer will allow us to use SwiftUI libraries.
You will be able to opt in to threads in the next release (currently in testing), alongside updates to room preview on long press in room list, ability to share any location and support for more languages
Release candidate 1.4.11 is currently available on the PlayStore if you are a tester. Should be pushed to production next Monday! F-Droid publication is in progress too. Learn more about the full release content here: https://github.com/vector-im/element-android/releases
Add banner to timeline when location sharing is running. Live Location Sharing (a.k.a. LLS) is still a work in progress and not available in the Element app yet.
Improved unit test coverage (especially around login with MXID)
Improved how threads look in the main timeline
Add notification for users to opt in to threads
Polishing around spaces to bring them into line with latest designs
Hotfix for leaving all rooms in a space without leaving the DMs. The hotfix is included in the release candidate 1.4.11.
We are considering modifying our rules to format source code. We will try to limit the impact on forks, but it will not be easy.
Since last time, we've made a lot of small quality-of-life improvements, but a few changes that stand out are:
We've improved support for offline PWA usage.
We've improved caching of space contents, reducing the number of times that we need to hit the spaceHierarchy endpoint and improving performance.
We've moved to a more in-the-spirit-of-the-spec way of handling hidden annotations: these are now represented by rooms with an m.space.parent event, and no correpsponding m.space.child event in the resource-space.
We've added a modal for viewing image messages at full-size.
Number 4 works nicely with my teaching-assistant-bot (built with matrix-bot-sdk, mathjs, and chartjs), which helps me visualize information about student activity.
Matrix highlight saw some "under the hood" changes this week, in particular a refactor to rely less on the Chrome/Firefox extension API. This should make it possible (in theory, and with some more work) to run Matrix Highlight on pages without installing anything! Aside from the obvious, I think that there are additional use cases opened up by this change; one such case I have in mind is as a commenting system on a site (a la cactus comments, but with the ability to highlight page snippets!).
In the process of all of this, I've spent some time running Matrix Highlight in Firefox. I've encountered no issues during this time, so it seems like the tool is usable from FF, too.
Trixnity 2.0.0-RC1 has been released.
This release candidate contains many breaking changes due to a large refactoring, which allows us to share a lot code between server and client implementations of the Matrix APIs.
Yes, that means Trixnity can be used to implement a matrix server!
We also made some progress to make the client module (with all the high level logic) multiplaform. This is the only module, which does not support Kotlin/Native and Kotlin/JS yet.
There are many other features (like client-side notifications!), which has been added. See the changelog for more details:
features/improvements:
clientserverapi-server: new module for server-side REST endpoints of the Client-Server-API (Server-Server-API will follow soon)
olm: libolm is bundled into trixnity-olm jars
client: push notification support (push rules are evaluated)
client: introduce helpers to get complete timeline as flow (no more complicated loops to get the timeline)
client: allow subscribing to all timeline events -> really helpful for bots with e2e support
client: allow to sync once (e. g. for push notifications)
client: content field of TimelineEvent gets also set for unencrypted events
client: public access to keys
clientserverapi-model: allow custom field in pusher data
Polyjuice Client Test is a testing tool for Matrix clients. Since the last TWIM update,
two new tests have been added: key history sharing (MSC3061) and no plaintext sender key (MSC3700).
more clients endpoints have been implemented or stubbed. This has improved compatibility with some Matrix clients, and reduced noise in the logs.
the deployment at https://test.uhoreg.ca/ now automatically runs the latest version from git. This Matrix-based continuous deployment is powered by another personal side-project, which will be revealed in the future.
the UI is now significant less ugly (unless you hate purple, in which case you may find it more ugly).
#polyjuice:uhoreg.ca now exists for discussing anything related to the Polyjuice project
Dept of Ping 🏓
Dept of Ping will return!
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
