Hey all, Synapse
1.62 is out! Let's
have a look inside.
Spam checker improvements
In the past few weeks, the Synapse team has been working with the Matrix.org
Trust & Safety team to help module developers build more efficient protections
against spam. As a consequence of this work, Synapse 1.62 introduces new ways
for modules to communicate the result of actions taken against a specific
message or operation through the spam checker module
callbacks.
Previously, most spam checker callbacks would be expected to return a boolean
indicating whether a specific operation should be qualified as spam. Starting
from Synapse 1.62, modules are now expected to return either
synapse.module_api.NOT_SPAM (to indicate an action is not spammy), or an error
code that is part of synapse.module_api.errors.Codes.
Note that the previous behaviour is still supported but is now deprecated, and
will be removed in a future version of Synapse.
See the upgrade
notes
for a list of the affected callbacks and an example of this change. Note that on
top of the list described there, the check_event_for_spam callback was also
updated with a similar
change
in Synapse 1.61.
Everything else
This release of Synapse includes important performance improvements around
syncing, specifically around handling device lists and notifications.
Synapse 1.62 also introduce a changes of its optional dependency on the LDAP3
authentication provider
module to
v0.2.1
in order to fix an issue with usernames that include uppercase characters.
See the full
changelog for a
complete list of changes in this release. Also please have a look at the
upgrade
notes
for this version.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper, Sami
Olmari, Daniel
Aloni,
Thumbscrew and Hannes
Lerchl.
I had the chance to have Ryan with me for a special edition of Open Tech Will Save Us on Thunderbird to celebrate their 102 release, which includes Matrix support for the first time in a stable release.
We covered many interesting topics, such as the importance of specifying the expected behaviour of clients and servers in a protocol to deliver the best experience to end users (wink, wink, reminds you of something?), why Thunderbird was more dormant and is now vibrant as ever, what they plan for the future. I had good fun and I hope attendees did too! Next episode is going to be at the end of July, but you can already join the
Open Tech Will Save Us room
Audrey Tang, Digital Minister for Taiwan is offering sponsorship for full localisation of Element and other leading Matrix clients into zh_Hant_TW - see https://twitter.com/audreyt/status/1542296087310258176. If you're interested in helping out, please get in touch with Thib and we'll coordinate.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
This MSC is a relatively simple it. Currently widgets are only allowed to specify their capabilities (what the widget is allowed to do, like reading the logged-in user's display name) when the widget is loaded. This MSC attempts to allow that model to be extended to let widgets ask for additional (or fewer) permissions over time.
I can't help but be reminded of the shift in permissions on iOS and Android here in how they shifted from "ask all permissions up front" to "ask for permission for each thing when it's used in the app" :)
New Matrix Intern Usman has just started on his project to prototype Favourite messages in Element Web. He's written a blog post introducing himself and the project: https://yaya-usman.hashnode.dev/outreachy-blog-introducing-myself
This week the Synapse team released Synapse 1.61.1! This is a security release which addresses a high severity vulnerability in URL preview feature. Server administrators are encouraged to update as soon as possible! We have published a blog post explaining the vulnerability and detailing a few workarounds that can be implemented on homeservers which can't be updated right away: https://matrix.org/blog/2022/06/28/security-release-synapse-1-61-1
Other than that we have published the first RC for Synapse 1.62.0 (which was followed today by a second bugfix RC). Synapse 1.62 will feature an improved spam checker API for modules, performance improvements around device lists, more customisation for .well-known client files and much more. Watch this space next week for the full rundown 🙂
This week we released Dendrite 0.8.9 which contains a number of improvements around backfilling room history, amongst other things.
Features
Incoming device list updates over federation are now queued in JetStream for processing so that they will no longer block incoming federation transactions and should never end up dropped, which will hopefully help E2EE reliability
The /context endpoint now returns "start" and "end" parameters to allow pagination from a context call
The /messages endpoint will no longer return "end" when there are no more messages remaining
Deactivated user accounts will now leave all rooms automatically
New admin endpoint /_dendrite/admin/evacuateUser/{userID} has been added for forcing a local user to leave all joined rooms
Dendrite will now automatically attempt to raise the file descriptor limit at startup if it is too low
Fixes
A rare crash when retrieving remote device lists has been fixed
Fixes a bug where events were not redacted properly over federation
The /invite endpoints will now return an error instead of silently proceeding if the user ID is obviously malformed
As always, please feel free to join us in #dendrite:matrix.org for more Dendrite discussion.
Thunderbird 102 with Matrix support is now available for download at https://thunderbird.net/. The Matrix implementation reflects what's been previously discussed in TWIM with some additional bug fixes. You can read more about what's new in our blog post.
Ryan discussed the Thunderbird 102 release as well as the project in general in this week's Open Tech Will Save Us, give it a listen for the latest inside scoop.
Have you ever noticed that some people are just plain @*%&!§%"+? Well, to quickly deal with what they wrote, Nheko now has a /redact @userid:server.name command, so you can redact everything they wrote (as long as it is in the currently cached section of the timeline). Note that you will run into rate limits when using that and Nheko is not yet applying an appropriate backoff in that case.
Similarly, q234rty fixed a lot of cases where icons in Nheko were either blurry or the wrong size. We fixed a few crashes, the room list should now not sometimes store the wrong order of rooms, brausepulver made large avatars cropped locally (since servers don't guarantee any size over 96x96 when cropped and synapse doesn't properly save the full image size in that case) and added a menu entry to copy a roomlink. You can also now define new powerlevels for users instead of using the existing levels in the room and Jason fixed some compiler warnings when a private member struct doesn't have an explicit constructor on some compilers. Nheko also now downloads the full online key backup when you explicitly toggle the switch.
A more noticeable topic might be, that Nheko now requires servers which support the v1.1 API or later and will not allow you to login or register otherwise. At the same time we also enabled support for the shiny and new knock_restricted rule and all remaining groups code was removed (you know, the feature from before spaces were cool).
I also updated Nheko on my work laptop now and since I was quite surprised by how fast it starts now (while my room count is over 900 !), I attached a video of that below for your pleasure.
This week is an exciting time in Element land! We have one feature coming out of beta and another going in: the new search experience is going live with the next release, while video rooms will become available for testing. You can already preview them in the release candidate!
Threads
After investigating ranged read receipts, we are looking at per thread read receipts again as a more practical approach.
Community testing
RC testing done:
Web - MD/HTML support in Space/Room topics
iOS - regression testing of messages, reactions and rooms
Because sometimes you just have to do something else and it might be interesting to some people:
I fixed calls not working when trying to use Element Web with a Conduit server: https://github.com/matrix-org/matrix-react-sdk/pull/8931
I fixed Element Android still using the unstable endpoint (i.e. the one from the MSC, that proposed it) to list aliases: https://github.com/vector-im/element-android/pull/6288
Sometimes fixing something that annoys you isn't even that hard, and since it annoys you, you have some motivation to fix it. I call this Anger Driven Development (ADD). If you have some things that annoy you, I encourage to try fixing them for a bit. It might help out some other people too. If that makes you want to contribute to Nheko, don't hesitate to join #nheko:nheko.im and ask me to help you out with whatever fix you are trying to contribute.
This time I just decided to clean out some issues, that came up in the Conduit development channel and that weren't fixed yet. Turns out it was just a few lines to change and it hopefully makes the Matrix experience better for everyone in the long run. The changes were also quickly reviewed and are merged now. Do you have any such little fixes that no one hears about, but you are proud of them? Please post about them, I'd love to see them!
Are you enjoying Location Sharing but have tired thumbs from tapping every time you need to update your location? Then fret no more! Element 1.8.20 features our new Live Location Sharing feature, now available in Labs.
Live Location Sharing: Now in Labs, we’ve added the ability to send your location in real time to the people and rooms of your choice!
Mark as Read: Quickly mark a single room as read by long-pressing it on the home screen.
Annoying bugs fixed: It’s much easier to tap the names and avatars of room members, voice messages now work better with VoiceOver enabled and it is significantly faster to create a new room.
Over the last few weeks, a significant push on rust-crypto-support for Nodejs has taken place, e.g. this PR. Major parts are available, the infrastructure to create releases ready, however, a pretty rare huge memory allocation that causes a crash in Nodejs on the CI has blocked progress of the team recently. Debugging this costs a lot of time and it is yet unclear, how much of a problem this really is (as the cause stays mysterious), however it has also been seen in the wild on Windows now-- a concerning development.
This week, the team released version 0.10.2. In this version, we added experimental support for native group call signaling MSC3401. We have been playing with it in our internal clients, and we have to say that we are pretty happy with the result! There are still some bugs, as sometimes a new call is creating instead of joining the existing one. But it's looking great!
We also took the opportunity to fix some bugs in call handling. Now the ringtone is properly properly when rejecting a call. Also, when opening the client, we don't trigger a call event anymore if the call end event is in the sync response.
We also did some refactoring for the sync handler, reducing the usage of JSON objects and adding some try catch to handle issues when handling event room updates. We switched to custom CachedStreamController so even if the stream has already been listened to, you can still access the last sent value.
Finally, when sending messages when under unreliable connection, messages could be sent out of order. This is now fixed thanks to the implementation of a message sending queue.
This week's release completed support for resolving, creating, and deleting room aliases. Get the code and contribute at https://git.thisisjoes.site/joe/axon.sh (supports GitHub and GitLab login) and join #axon:matrix.thisisjoes.site for discussion and updates!
Two is better than one! Ascurius joined the synadm team and helps maintain the github repo, code new features and support users on github and #synadm:peek-a-boo.at.
Thanks to all synadm users, feature requesters and contributors, issue submitters and #synadm:peek-a-boo.at members. It's fun to maintain a project and see its community grow each day!
Some features from the latest releases we'd like to highlight:
The local part of an MXID can now be used as the <user_id> argument in various synadm commands. Of course, there is still the possibility to use the full MXID if desired.
We have added a shadow-ban command so that admins now can more easily deal with abusive users: synadm room shadow-ban
The room state API is now supported, try synadm room state
And with the help of that API we created a command to easily generate a list of rooms and corresponding admins and mods: synadm room power-levels
The synadm room search command was adapted to make better use of current Synapse versions possibilities.
Documentation chapter around using synadm together with Synapse instances deployed with matrix-docker-ansible-deploy.
The magic around "retrieval of the own homeserver name" got a massive overhaul. This affects several user and media subcommands.
Have a look at the releases list for more details: https://github.com/JOJ0/synadm/releases
Dept of Events and Talks 🗣️
Matrix Summit Conference Berlin calls for your participation.
With the first Matrix Summit Conference we try to showcase the matrix-eco-system in its whole width and depth.
We are looking for talks and workshops around matrix-related projects and products. We are interested in all aspects of those: From the past to the future, from the moment of the idea, the story of the creation or the vision of the future.
We’d like to understand the principles as well as the technology.
The conference is from people for people, so if you’d like to talk about yourself, your community, your organization, please do.
Showcase yourself and your relation to the Matrix world.
We try to compile a versatile program. We are open to contributions of any length, from 5 minutes(lightning talks) to presentations and talks to workshops and hacksessions up to 5 hours.
We’ll come together to discover, celebrate and enjoy the world of matrix. Also, if you have any arty, cultural or playful contribution in mind, please offer it.
You can enter proposals until 2022-07-22 22:22 (Europe/Berlin), 3 weeks from now in our pretalx
Matrix-summit-berlin-2022 matrix summit space
The #matrix-summit-berlin-2022 space will contain all rooms and subspaces related to the event.
For the organization of the summit we have orga-room. Just come by if you like to know something or help with the summit.
There is a gitlab organisation which contains our codebase and issuetrackers.
Sponsors?
We try to make everything low cost. But we need some money for food, drinks, merch, travel, accommodation. Please contact Yan or write mail, when you can sponsor the event.
Note this is a community event that is not organised by the Matrix.org Foundation.
Meet other matrix users, chat about Matrix, the rest, and everything else, discuss your Matrix ideas, sign each other in persona, and maybe spice the evening with a good mate or beer.
Also when the bbq is lit you may wish you brougth your favorite item :)
Every first Wednesday of the month in the c-base at 8pm ('til the next pandemic).
An oversimplified, easy to implement, embedded live chat widget that allows your website's visitors to send messages seamlessly to your Matrix account.
The widget is Built using Svelte, so everything goes in one nice bundle, its fast and awesome to embed. No need for external libraries or big framework.js files. Just import into your website's structure one JS and one CSS files.
The server uses Golang, the Binary is only 3.2MB in size. Make sure you configure the .env file.
A demo and more information about livematrix can be found here:
https://github.com/osousa/livematrix
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Today we're exceptionally releasing Synapse
1.61.1, which comes
as a security release. Server administrators are encouraged to update as soon as
possible.
This release fixes a vulnerability with Synapse's URL preview feature. URL
previews of some web pages can lead to unbounded recursion, causing the request
to either fail, or in some cases crash the running Synapse process.
Homeservers with the url_preview_enabled configuration option set to false
(the default value) are unaffected. Instances with the enable_media_repo
configuration option set to false are also unaffected, as this also disables
the URL preview functionality.
Server administrators who are unable to update Synapse should disable URL
previews by setting url_preview_enabled: false in their configuration file.
They can also delegate URL preview to a separate, dedicated worker to ensure the
process crashing does not impact other functionality of Synapse.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
The big news from the spec this last week is the release of Matrix v1.3 🎉 (read the blog post if you haven't already)! Roughly three months since the release of Matrix v1.2, this release brings improvements such as knocking on rooms, room version 10, reduced metadata in encrypted messages and the first pieces of aggregations finally landing in the spec proper. And more! See the blog post for the full changelog.
Most of the Spec Core Team has been away this week, thus there has not been much moving forwards. But we do have two new MSCs from @duxovni and @Johennes, which you can view above.
This MSC allows for differentiating between different incoming streams of media coming from a single user by adding a sdp_stream_metadata dictionary to Voice over IP (VoIP)-related events. This is a relatively simply addition with useful functionality, such as allowing a single user to share both their camera feed and screen share at the same time!
1.4.24 released to beta testers which includes support for UnifiedPush and fixes for voice recordings and duplicated messages in the timeline
We're making it easier to opt in to Live Location Sharing by displaying the labs setting within the location sharing flow, no need to hunt down the setting anymore!
We have also fixed some outstanding crashes around opening large images in the timeline and signing out
added previews of room name and topic in tooltips for room icons
added handling for an eventId component of annotation URLs
Along with a number of other minor bugfixes and UX improvements. And, we've added one neat new user-facing feature: inline previews of video and audio annotations. This one is a little hard to explain, but a video is worth ten thousand words:
A set of Rust library crates for working with the Matrix protocol. Ruma’s approach to Matrix emphasizes correctness, security, stability and performance.
Released Ruma 0.6.4 with a bug fix for rich reply fallback generation
Added support for pretty much everything from Matrix 1.3, which was to a large extent just the removal of feature flags for previously-unstable functionality
Thanks to @zecakeh for both implementing most of these features and now stabilizing them in Ruma!
Hey everyone! Guess what? Synapse
1.61 is out! Let's
have a look at it.
Farewell, groups
If you are new to Matrix, you might have not heard of the feature referred to as
"groups" or "communities" (depending on the context). This feature allowed
grouping rooms and users to better represent a community, one of which being
+matrix:matrix.org which used to represent the Matrix community. This may
sound similar to Matrix
Spaces, and it would
make sense since Spaces are meant to be a more powerful replacement for groups.
In Synapse 1.56,
support for groups was deprecated, with a plan to fully remove it in a
later release of Synapse. This has now been done as of Synapse 1.61, and most of the
code supporting this feature has now been removed.
Note that this means that administrators of homeservers using workers can remove
endpoints related to groups from their reverse proxy configuration. See the
upgrade
notes
for more information.
Media retention
A common issue we see homeserver administrators struggle with is managing the
disk space used by Synapse. A non-negligible part of that disk space usage is
dedicated to storing files uploaded by Matrix users, both local and remote.
Up until now Synapse would only provide administrators with limited, manual ways
to manage the media store of their homeserver, via the admin
API.
As of this release, Synapse now allows administrators to define retention
lifetimes for local and remote media. This allows media that hasn't been
accessed in a long time to be automatically deleted, therefore freeing up disk
space. Server administrators wishing to control media retention more finely can
also define different policies for remote and local media.
This feature can be enabled by configuring the media_retention setting, see
the configuration
guide
for more information.
Everything else
This release of Synapse introduces a change in the return value of the
check_event_for_spam spam checker module callback, in order to allow modules
more flexibility in communicating to users why their messages are rejected. This
is part of ongoing improvement works around spam checker callbacks, watch this
space next time for more information!
See the full
changelog for a
complete list of changes in this release. Also please have a look at the
upgrade
notes
for this version.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper, Dirk
Klimpel and Jacek
Kuśnierz.
Hi there! I'm Marco Melorio and I'm participating in this year's Google Summer of Code, under the GNOME Foundation. I'm working on Fractal, the GNOME matrix client, with the help of my mentor Julian Sparber. More specifically I'm working on implementing a media history viewer to the app.
To follow my progress on the project you can check out my blog here. I've already published a small introduction post about me and a first update post which includes a mockup and milestones about the project.
This week the team released Synapse 1.61, which main new feature is media retention. That's right, you can now control how long Synapse keeps media files around, which should help server admins manage Synapse's disk space usage more efficiently. On a different note, this release of Synapse removes support for groups/communities (which was deprecated back in Synapse 1.56), as it has now been replaced by Spaces. Farewell groups, you have served your users well.
See the full Synapse 1.61 release announcement on the matrix.org blog here: https://matrix.org/blog/2022/06/17/synapse-1-61-released
Aside from this, the team is as always hard at work on making Synapse better and more efficient.
New version of Quadrix (1.0.6) available on desktops and mobiles. Mainly bug fixes, plus the addition of a button to deactivate the account on the server (this apparently will be soon mandatory for iOS and MacOS messaging apps). The new desktop version should offer better support for Wayland (tested on Fedora 36, Ubuntu 22.04 and Mobian/Phosh). Repo at https://github.com/alariej/quadrix, project room at #quadrix:matrix.org :-)
Since I skipped the last update, this one is a bit longer :3
First of all there have been lots and lots of updates to the translations! Finnish is now at 100% thanks to the tireless work of Aminda and Lurkki.
Nheko now also shows a nice badge on Unity compatible desktops (like KDE) for unread messages (although that doesn't work properly for multiple profiles being open at the same time due to limitations in that desktop protocol. Thank you d42 for contributing this, may you role a natural 42 every dice throw!
Syldra fixed up the paste behaviour (which didn't properly tie into undo in some edge cases), cleaned up how we find some of our dependencies and made cusor movement more consistent across systems.
Finally we fixed our glare resolution when verifying other sessions, which will be especially noticeable when verifying Cinny, since that responds to verification requests in a different way than I tested before! This should get rid of a whole range of verification issues people might have experienced. As part of our stabilization for the next release, we also fixed a crash on logout because I fatfingered and deleted a return statement, we now send an Element Android compatible height attribute on emotes, properly compile when the C++ version is set to 20, we once again support the current version of the hidden read receipts MSC (so that others can't tell what you read), edits now properly update in replies again, you can close the image overlay again by clicking outside, cancelled uploads properly get removed again, logging out and back in now doesn't mess up your configuration anymore and pinned messages now properly refresh once the events are loaded.
Phew, that was a mouthful. And we are not even done yet!
I spent some time on making Nheko compilation a bit faster again as well as improve startup speed. This might order your room list a bit weirdly for a few days after updating, but should normalize when receiving a few messages in some rooms. With this we now don't need to load the last message in all rooms on startup. This makes Nheko startup now only take 7 seconds on my old laptop (when not doing something CPU heavy at the same time). The remaining startup time is 40% building up the font database and 40% loading the powerlevels for each room. So we do have the chance to speedup startup by probably another 60%. It is unlikely that is necessary though.
When I discovered Matrix, Element was still called Riot. At the time some of the big design changes started happening to make it the Element you know today. One of the sideeffects was that the roomlist was consistently taking up 20% of my CPU on my Laptop, which I used at the time (and am forced to use now again, since I broke my newer laptop). It also used a lot more RAM than it does today. So for that reason I started shopping around for what other clients there are and I found Nheko. Clearly because it isn't a webclient, it had to be faster and use less RAM. Well, it did maybe a bit, but frankly the difference wasn't that much. Especially since at the time it loaded and rendered ALL your messages on startup (kinda). It never removed messages from memory, so it would just continuously render more and more parts of your timeline, which clearly increased RAM usage. It did however never resort the full roomlist, which made it not use a lot of CPU.
Since I didn't know any web development at the time, but I knew some C++, I started contributing to Nheko. At some point I made the roomlist constantly resort, which used quite some amount of CPU, but I quickly fixed that. At the time the most noticeable difference was that my fans didn't spin when using Nheko, but Element did (because of the roomlist sorting, iirc). RAM usage was pretty bad though. So that was one of the next projects, removing all events from RAM and only pulling them from the database as needed. While this means some additional load when switching rooms, it did decrease RAM requirements by quite a bit. Some new features made us still require loading data for every room from the database on startup, which causes quite some noticeable startup delay. The latest changes however got rid of most of that. We now don't need to load the messages from the database anymore. The only thing we still load is a small info object with roomname, notification and member count as well as the power levels of the room.
Since I recently broke my new and fast laptop, I decided to checkout how things changed on my old and slow laptop. Nowadays I am not in 15 rooms anymore, but I am in 900 rooms, but Matrix, servers as well as clients, has also gotten a lot more performant. So in the end I decided to do a little benchmark of where things stand. DISCLAIMER: This is completely unscientific and unfair, so please take those numbers with a grain of salt. Almost no one runs such a slow laptop as I do, so likely your measurements will completely different. Even more, I was video recording the benchmark, which makes both clients a lot slower. Nonetheless it does somewhat mirror my personal experience.
I came to the conclusion that with 900 rooms, Nheko takes about 10-20 seconds to load and be ready for use on my system, while Element takes about 3 to 4 minutes. So basically Element handles 60 times as many rooms about 2x slower than it did back in the day, while Nheko got a bit faster or about the same speed on the same hardware (but still 60x as many rooms). I've attached a sped up video to this post, so that you can compare it for yourself. But since a lot of people ask, I guess the reason is that I wanted to see how fast you can make a Matrix client. I think I somewhat achieved that in the startup time department, but switching rooms still has a loooooong way to go. Also it is just fun to implement whatever you want in a client, since you are the maintainer and none can tell you how bad of an idea it is. That's probably the reason a lot of people start their own clients? (Although I didn't start Nheko, I just wrote too much code and people didn't want to review it anymore.)
That's it, I hope your eyes didn't glaze over with me babbling on about things. See you next time! :3
There have been many improvements to clickable texts in various parts of the application (e.g. when you see “Show more” in the UI) and many other UI tweaks (thank you luixxiul!). As well as UI improvements to colours and layouts in the settings dialog
Fixing a pagination regression because of a synapse update to conform the matrix spec. The fix will be included in the coming release 1.4.22.
Merged a selection of FTUE improvements, including simplifying login if you specify your server as part of your username. This work is not visible yet in the UI.
UnifiedPush is coming! Will be available in Element Android 1.4.22. In the meantime, you can test the feature using a nightly build . This technology will let F-Droid version of the app be able to receive Push and Element Android can stop polling your homeserver in the background, which was consuming lots of battery.
When you send a rageshake, screenshots will not be included by default any more
We also have set up Flipper in the app, to help developers to debug the application. Inspecting the Realm databases, or the network traffic will help a lot when developing new features or fixing issues.
As you can guess by the title, I've started Matrix Wrench one year ago.
Here are the most notable changes of today's birthday release v0.8.0 (2022-06-13):
Added: All pages have URLs to quickly navigate to.
Added: Bulk actions to invite and kick users
Added: The About page now shines some light on the application's features and security.
On other major news, the first UniFFI macro PR, submitted by our very own Jonas, was merged by Mozilla earlier this week 🎉. This is an important milestone as this PR is the opening gate for getting more and more macro-support into UniFFI, eventually replacing the entire UDL-in-between-action. Congrats for this major step forward
The there is a new release that avoids crashing of the bug when the synapse admin is not correctly exposed. For technical reasons (on layer 8) the bug fix is made in two releases. The latest version is now v1.1.7
Four days of Barcamp sessions, presentations, meeting others and Berlin culture!
The Matrix Summit 2022 brings people together who work with Matrix.
Thu, 25th to Sun, 28th August at the iconic hacker space c-base in Berlin.
This event is organised by the local Matrix Meetup community.
While people of every skill level are welcome on all days, we're going to highlight real world use cases of Matrix on Sunday. We're inviting newcomers and other communities to learn about decentralised communication software. In return we're learning how Matrix is already used by the German education and health sectors.
Our Call for Presentations starts next week. The event is going to be bilingual: English and German.
See the schedule:
https://summit2022.matrixmeetup.de/
UnifiedPush is an open protocol for push notifications on Android, independent from Google Services. It is currently supported by FluffyChat, SchildiChat, and now nightly builds of Element.
If you are new to UnifiedPush:
Installing the ntfy app (F-Droid, Play Store) is the easiest way to start receiving notifications for your matrix messages without Google Services.
And if you want to go one step further and self-host a ntfy server, the latest release makes it simpler than ever to set it up on your own server!
For persons already self-hosting ntfy:
The brand new version v1.26 of ntfy server includes a built-in matrix gateway: your homeserver can talk directly to ntfy. If you were self-hosting ntfy with a personal matrix gateway (common-proxies or with a Nginx setup), you can remove the gateway after upgrading to v1.26.
An early-stage home server prototype that communicates in a fully decentralized manner: every user runs their own server locally!
This is achieved with the libp2p library which provides decentralized node discovery, NAT traversal and gossip-based communication capabilities.
The server implements the Matrix API specifications (with many advanced features still missing, like client-side end-to-end encryption), but did little modification on federation side so that it doesn't rely on the centralized domain name system to setup and authenticate other nodes. On the other hand, this means it cannot federate with existing home servers yet, because of the different trust model (domain name/HTTPS vs self-owned cryptographic keys).
Try out the latest Webview client on windows, or on other OSs with Docker&Browser!
anarc.at published a long and thoughtful critique of Matrix over at https://anarc.at/blog/2022-06-17-matrix-notes/ - interesting reading, even if we don't agree with all the points.
It’s another quarter and therefore another spec release! Matrix 1.2 was released back in February, and while a bit late in the quarter this time around we’ve still got some exciting additions coming to you in this release.
Like last time, the speed of these releases might feel a bit quick for developers: fret not if you’re still working on v1.1 or v1.2 implementations. We’re not expecting that implementations update as soon as a new spec release is published, but rather that the ecosystem gradually update over the course of the next few months. Implementations should be aiming for as close to realtime as they can get though, particularly considering v1.4 is expected to have some large features (more on that later on).
Matrix 1.3 sees 14 MSCs get merged, but we can’t possibly go into detail on them all here. We’ve picked some notable highlights and recommend the full changelog at the bottom for a complete idea of what’s been going on.
Aggregations and the relationships made along the way
It’s no secret that MSC1849-style server-side aggregation of related messages have been in the review backlog for a while. We ended up splitting MSC1849 down into more reviewable chunks like MSC2675, allowing us to finally land the first pieces into Matrix 1.3 today.
In the spec there aren’t currently any defined relationships which make use of aggregations or even the rel_type described by MSC2674, but we do expect that v1.4 of the spec will have support for at least threads (MSC3440) and edits (MSC2676), filling the gap. For now, we’ve decided that holding aggregations back until v1.4 doesn’t make a lot of sense, so we have launched them into the world as-is for custom relations or for clients & servers to prepare for what’s expected to be coming in v1.4 of the spec.
To further prepare for threads, we’ve also removed some restrictions of rich replies through MSC3676, thus allowing replies to be constructed with non-text messages like images. Check out the new rich replies module for more information.
Join if you can, or just knock
When we launched restricted rooms in Matrix 1.2 we noted that we forgot to handle a case where someone might want to support both knocking and restricted rooms at the same time. We’ve fixed that with a stop-gap join rule from MSC3787 in room version 10.
The new knock_restricted join rule allows the room to keep its desire to be restricted whilst also allowing members who do not meet the criteria to knock on the room instead. We’ll likely expand on this sort of mixing of join rules in proposals like MSC3386 down the line, however for now this should cover the gap in support. Next up: figuring out how to make encrypted room history available to these new joiners in a safe way.
A thread for next time
Originally planned for this release, MSC3440-style threads are anticipated to be ready for v1.4 (next quarter) with support for notifications and, of course, a whole new way to communicate in a room.
Threads are one of the more complicated features we’ve tried to land in recent history as it has a large impact on a wide variety of the spec: everything from event relationships (fixed in this release) to read receipts to push notifications needs to be worked out to build a system that not only users can understand, but also the developers trying to build it. With this massive surface area we just weren’t comfortable with adding threads to v1.3 as-is, given problems like notifications aren’t yet fully solved.
Alongside threads, we also anticipate that MSC2676-style message editing will be landing in v1.4, finally specifying how to update an event’s contents without having to redact & re-send. Although message editing has been supported for a long time in some clients, we're excited that it will finally become part of the official specification, meaning it can be implemented more widely. Messaging clients are encouraged to give the proposal an early read and possibly even attempt implementation if not already done to help us ensure the system is in a reasonable state for inclusion in the spec, though we do note that feature requests for edits will likely be deferred to a future MSC.
Keep an eye on This Week In Matrix for updates on what v1.4 is expected to include, and how things are progressing.
The full changelog
MSCs are how the spec changes in the way it does - adding, fixing, and maintaining features for the whole ecosystem to use. The blog post can’t cover them all, but that doesn’t make them any less important! Check out the full changelog below, and the Spec Change Proposals page for more information on how these MSCs got merged (hint: they submitted a proposal, which anyone can do - take a look at the Matrix Live episode where Matthew covers the proposal process).
Client-Server API
Deprecations
Deprecate the sender_key and device_id on m.megolm.v1.aes-sha2 events, and the sender_key on m.room_key_request to-device messages, as per MSC3700. (#1101)
Backwards Compatible Changes
Make from optional on GET /_matrix/client/v3/messages to allow requesting events from the start or end of the room history, as per MSC3567. (#1002)
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
The release date for Matrix v1.3 has been set in stone for Thursday, June 16th 2022! Expect a blog post on the day detailing all of the new additions.
The bot that posts MSC updates in #matrix-spec:matrix.org and elsewhere has been switched from @mscbot:amorgan.xyz to @mscbot:matrix.org. The backend of the bot uses https://github.com/Informo/specs-bot/.
This is a proposal that adds a mechanism for homeserver administrators to define a password policy for users. This policy can decide rules such as a minimum password length, whether a digit is required, etc. This policy can be enforced by the homeserver when a user registers an account or changes their account password and is communicated to the client so that it can also be enforced locally.
Astute readers will possibly note that authentication in Matrix will eventually be replaced by OAuth2 (MSC2964). This will move operations like password policy enforcement from the homeserver to a separate authentication service, essentially removing the need to reinvent-the-wheel in a homeserver.
Registration, login and managing passwords and connected third-party IDs is often a complex part of a Matrix homeserver. Moving these out to a separate authentication service will both unlock new features (log in with Matrix!) as well as reduce the resources required to implement a Matrix homeserver.
Hey there, I am Rohit. I'll be participating in GSoC this summer, under the Matrix organization.
For my project, I'll be working on the desktop client, Nheko. Specifically, I aim to work on its VoIP Library and also on implementing and updating some of the features to concur to specification changes to the Matrix protocol, which include
Improved VoIP Signalling
VoIP Call Transfers
Muting Calls
I plan on starting a blog and hopefully will do so soon. In the meanwhile, if you want to know more about the project you can visit here and join the #nheko:nheko.im to participate in discussions.
Looking forward to learning and interacting with the community.
Work continues on speeding up federated room joins, improving testing, and reducing database i/o. In addition, some lovely features have been added:
Add new media_retention options to the homeserver config for routinely cleaning up non-recently accessed media.
Experimental support for MSC3772: Push rule for mutually related events.
Update to the check_event_for_spam module callback: Deprecate the current callback signature, replace it with a new signature that is both less ambiguous (replacing booleans with explicit allow/block) and more powerful (ability to return explicit error codes).
Add storage and module API methods to get monthly active users (and their corresponding appservices) within an optionally specified time range.
Support the new error code ORG.MATRIX.MSC3823.USER_ACCOUNT_SUSPENDED from MSC3823.
Add a configurable background job to delete stale devices.
Improve URL previews for pages with empty elements.
Allow updating a user's password using the admin API without logging out their devices. Contributed by @jcgruenhage.
This is in addition to quite a number of bugfixes!
This week we released Dendrite 0.8.8, which contains the following improvements:
The performance of state resolution has been increased significantly for larger rooms
A number of changes have been made to rate limiting:
Logged in users will now be rate-limited on a per-session basis rather than by remote IP
Rate limiting no longer applies to admin or appservice users
It is now possible to configure additional users that are exempt from rate limiting using the exempt_user_ids option in the rate_limiting section of the Dendrite config
Setting state is now idempotent via the client API state endpoints
Room upgrades now properly propagate tombstone events to remote servers
Room upgrades will no longer send tombstone events if creating the upgraded room fails
A crash has been fixed when evaluating restricted room joins
As always, please feel free to join us in #dendrite:matrix.org for more discussion.
Hello friends. This week we bring you a security release for matrix-hookshot. Please ensure you have upgraded to at least 1.7.2, and have read the security advisory. Thanks!
The changes are as follows:
Add support for GitHub enterprise. You can now specify the URL via enterpriseUrl in the config file. (#364)
Add ability for bridge admins to remove GitHub connections using the admin room. (#367)
FluffyChat 1.5.0 has been released and will soon be in all stores. This release comes with a bunch of bugfixes and introduces the first iteration of a Material You based design. We also try to support Android 12 accent colors (while this does not seem to work yet). On other platforms, our own purple shape will stay the default.
I like the new Material You design while we needed to tweak it at some points. Also the dropdown menu misses some elevation but this is a known issue in the Flutter repo which will be fixed soon.
This is also the first build with Flutter 3 which should improve the performance a little bit. Unfortunately it brings some regressions. We are forced to ship a little bug with it: Sharing on iPads seems to be broken. I'm very unhappy with this situation but otherwise we would have ended up in different releases for different platforms. As there afaik are not that many iPad users out there, we decided to live with this compromise and ship a bugfix release asap.
Yeah... Flutter has a lot of pros but also a lot of cons. Every new release of this framework leads to the fear of new regressions. Shipping a new major release where basic stuff like "Sharing" is just broken, is totally stupid... but that's the decision of Google. :-P
We didn't want to wait to ship the new design and also we needed a new release to come back to F-Droid where FluffyChat wasn't available in the last weeks because of a ProGuard problem, which should be fixed now.
Threads is in Beta and progressing. We’re working hard setting up the foundational work needed to improve read receipts and notifications cross-platform.
Keep sending feedback and rageshakes as we’re also continuing to improve the UI and fix any bugs that are raised.
Community testing
We’re moving closer to getting the new search experience out of beta, thanks for all your help on testing so far.
Next up: Big regression testing session on Android, after the removal of communities/groups (Wednesday or Thursday TBC)
We’re continuing our bug fixing spree and the latest release has fixed some of the more significant problems we had
We’re working towards adopting a new notification filtering entitlement and we will soon be able to silence those pesky empty notifications
Our “Edit Home Screen Layout” experiment is running well and we hope to be sharing the results of the diary study and prototypes soon.
The new first time user experience is continuing to make good progress and is quickly approaching finalisation
We are now allowing account deactivation for users that signed up through SSO
On ElementX we have merged the new crash reporting service and we are starting to see reports come in. We’ve also successfully integrated an initial version of the DesignKit and added room filtering
1.4.19 is available and fixes some small bugs, including a fix for the regression surrounding space switching performance.
Our prototype study is nearly at a close - we’ve received a lot of great feedback on our suggested changes to the home screen and space switching interactions.
We’ve been working on integrating UnifiedPush for both Google Play and FDroid versions of our app.
The team is finalising the updates to the create account flow. It will be ready for testing soon.
This week we landed Element Call beta 2 (https://call.element.io) including a bunch of nice updates. First of all, everything is now end-to-end encrypted by default: not only the WebRTC streams but also the matrix signaling (It was consciously disabled in beta 1 for debugging purposes). Moreover, we have experimental support for spatial audio rendering. Give it a try — you can find it in the settings section. It's a lot of fun to play with and really supports immersion during a video call. This release also introduces a whole new experimental way of communicating: Walkie-talkie mode. In that mode, videos are disabled, and everyone is muted by default. To speak, press the ‘push-to-talk’ (PTT) button — takes me back to my childhood 🙂
For further details follow this blog post: https://element.io/blog/element-call-beta-2-encryption-spatial-audio-walkie-talkie-mode-and-more/
This week, the team released version 0.9.12. In this version, we updated our Matrix API Lite package, in which we fixed the issue with the room hierarchy endpoint with some improper parsing and wrong type coming from the generation of the spec Open API documentation.
We also did some housekeeping and renamed some methods and getter to make them explicit. For example, the futureSender getter is now an asynchronous method called fetchEventSender() to make it clear that calling this function may trigger an API call.
Possibility to override the supported Matrix spec version but we don't really recommend using it for production.
Here's a fun Matrix-based project for you, sponsored by yours truly and my employer FUTO.
The "Golden Tiger" senior capstone project team at Portland State University just delivered the results of two quarters's design and implementation work on a secure/private, self-hostable, end-to-end encrypted cloud security camera using Raspberry Pi's and Matrix. The idea of this prototype project was to provide similar functionality to commercial services like Ring or Nest, but without letting any nosy third party see inside (or around) your home.
The students' code is available on Github in two repos:
Hey all, it's been a holiday for much of the team this week, so from the plane of Maple trees I present to you the spec update.
The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals
In terms of Spec Core Team MSC focus for this week, we've been working on getting Matrix 1.3 out the door and generally working towards a better process for handling releases alongside critical proposals for the protocol. The goal is to be able to ship these major features while reducing the risk of a release falling behind. Watch this space for more details as this should result in MSCs getting through the process a little bit quicker.
For Matrix 1.3, we're targeting Thursday, June 16th, 2022 as our release date, though this comes with a small asterisk: we're looking to land quite a lot of stuff so might have to adjust the date once more as needed. We do feel reasonably confident in the date though - watch for blog posts titled Matrix 1.3 in the coming weeks.
Random MSC of the week
The script has chosen MSC2846 - Decentralizing media through CIDs as your random MSC this week. The MSC raises an interesting question about how to make media more akin to events in Matrix. If you're interested in this area, take a read of it and the related MSCs.
Hey there, I'm Aditya Rajput (aka BURG3R5), a sophomore at IIT Roorkee, India. This summer I'll be working on implementing encrypted Search in Matrix rooms. More details about this project can be found here. Once a week, I'll be blogging about my progress (plus some neat stuff I find during research) in this blog. Technical discussion and more frequent updates can be found in the public room #encrypted-search:matrix.org.
While writing my GSoC proposal, I'd already made some progress w.r.t. the actual code required for this project, and my preliminary implementation can be found here.
Looking forward to working in this ecosystem with you all!
Hello everyone, my name is nannanko. From today, I will officially participate in GSoC to contribute code to the Kazv Project. I hope I can get along well with you all.
Hey there!
I'm Binesh Munukurthi, a Computer Science student from India. I'll be working on the 3rd Party Authorised Room Membership project for this year's GSoC.
This project aims to develop an application that has the ability to delegate membership of a room based on a user’s interaction with other third party services.
For more details on what I'll be implementing, please refer to the link
I'll be posting my weekly updates here
If you are interested in knowing more about the project's progress, feel free to join the public room #matrix-cerberus:cadair.com.
I look forward to spending a wonderful summer with you all!
Thanks!
Hi! I'm Snehit Sah. I will be contributing to KDE's Matrix client, NeoChat, during GSoC. I will add Spaces support to the application. My progress can be followed at my blog.
Hoping for a fun stay!
This week was a short one, but work continues on fast room joins, testing Synapse with workers on Complement,
and increasing the efficiency of Synapse with regard to database i/o. In addition, we released v1.60.0 (https://github.com/matrix-org/synapse/releases), which includes
work to reduce the possibility of database corruption
a fix to a bug introduced in Synapse 1.60.0rc1 that would break some imports from synapse.module_api
This week we released Dendrite 0.8.7! This is a highly recommended update since it fixes some fairly large bugs:
Support added for room version 10
A number of state handling bugs have been fixed, which previously resulted in missing state events, unexpected state deletions, reverted memberships and unexpectedly rejected/soft-failed events in some specific cases
Fixed destination queue performance issues as a result of missing indexes, which speeds up outbound federation considerably
A bug which could cause the /register endpoint to return HTTP 500 has been fixed
As always, please feel free to join us in the #dendrite:matrix.org room for more discussion!
Quadrix is a single-codebase, multi-platform project, using the meanwhile deprecated ReactXP framework (Microsoft's answer to Flutter), which compiles to iOS, Android, and web/Electron. The Quadrix apps are available in the main app stores for mobiles and desktops (including Snapcraft and Flathub). Repo is at https://github.com/alariej/quadrix. Would be great to have a few people test the apps and leave feedback in the repo or in the (still empty) #quadrix:matrix.org room!
Important: Quadrix doesn't support E2EE yet, but it's on the TODO.
Our prototypes for a new app layout are going well. Just a few people have early access in order to give us feedback to ensure we’re building a new layout that works for everyone. Watch this space!
We’ve been working on improving our onboarding flow so that users signing up for Matrix and Element find it much easier. This work is making great progress.
Live location sharing is also making great progress and is quickly approaching the beta phase
On the ElementX front we’ve been busy adopting async/await, introducing state machines, a new design kit component and hooking up crash reporting and sentry.io
We are a bit late on the release candidate. We want to fix a performance regression when switching spaces. It will be available early next week.
We have merged the UnifiedPush PR from P1gP1g. We are doing some adjustments but users will be able to choose their push distributor soon.
The prototypes that are with a small group of early access tester are going down a storm - we’re getting lots of great feedback that will help to build a new app layout that works for all.
Our improvements to the first time user experience are nearly ready for testing and shipping! We’re hoping to make our onboarding experience a lot simpler for folks that are creating an account.
We've had breakthroughs this week on implementing a native Matrix SFU (selective forwarding unit) which speaks MSC3401, thanks to Sean DuBois - all round WebRTC superstar, project lead for the Pion WebRTC implementation for Go, and author of WebRTC for the curious.
Sean generously contributed an initial proof of concept to show how you'd build an MSC3401 SFU using Pion at https://github.com/matrix-org/sfu-to-sfu - which (I think?) is the first time that the first implementation of a major core MSC has been contributed from outside the core team. Huge thanks to Sean to setting the ball going on this - the current PoC demonstrates not just SFU capability but also the decentralised cascading architecture which makes MSC3401 unique. The initial PoC speaks a 'test jig' version of MSC3401 hooked up to a simple web client for experimentation, but Matthew's now experimenting with adding genuine Matrix support to it via mautrix-go, and hooking up Element Call to speak to it.
Adding SFU support to Element Call will mean that we can support more than ~7 simultaneous calls - and with MSC3401-style decentralised cascading, we should be able to support hundreds or even thousands. There's lots of work remaining here, but the ball is now rolling. For more info about SFUs and MSC3401, check out Matthew's CommCon 2021 talk (which was what prompted Sean's implementation work here!)
Separately, we've been running Element Call on staging with E2EE enabled for the last few weeks, and should be releasing the first major Element Call update next week. And once SFUs land, then Element Call can exit beta - watch out Zoom!
libolm 3.2.12 has been released. The main update in this release is that the olm_sas_calculate_mac_fixed_base64 is now exposed in all the official bindings, so that MSC3783 can be implemented. Aside from that, there have been some minor fixes and improvements. See the changelog for more information.
While most of the work in the background continues (Sliding-Sync PoC, Wasm+NodeJS support, UniFFI macros), the first few parts surface through the cracks and show themselves in new PRs: basic wasm web-js and nodejs support has landed with more APIs, tests and documentation on the way; the Sliding Sync PoC has been upgraded to the latest JSON layout and now provides a first set of reactive API via FFI, too. Furthermore this week has seen a bunch of cleanups, simplifications and clarifications around the OlmMachine and crypto types, and we're fixing a bug in the state store, where not all data has been properly encrypted in the past.
This week, the team released version 0.9.9. In this new version, we added a search function to allow searching for an event in database and on server through several requests to the /messages endpoint.
There was also some work on updating the image size when generating thumbnail, which allow saving a bit of calculation when sending events. If using a custom resizer make sure to update the resize response as here.
We also did some refactor with the event sender getter. The getter result is now a future to allow requesting the member event to the server if we cannot find it in memory, to help fix the issue when the user was not found in memory.
Finally, a quick helper function (client.waitForSync) was added. It allows you to wait for a room to appear in (left, join, invited) section of the sync response.
Only a small update this week. One new feature and some much needed Ansible improvements.
Pipes move messages from one room to an other. Useful if you are in important rooms but only want to monitor a single room.
The Ansible role can now manage more aspects of MCM. Mainly in the post setup phase. The major one being if you run into encryption errors for any reason you can run the playbook with the fix tag and it will generate you a new device ID and encryption store.
Pipes move all messages from one room to an other.
Adding a filter now uses the room the command was sent from by default if the roomid is not specified.
Ansible role now has tags for start, stop, configs, fix and all.
Ansible role backs up the configuration.toml file to prevent the possibility of data loss.
We just wanted to take a moment to welcome Rocket.Chat to Matrix, given the recent announcement that they are switching to using Matrix for standards-based interoperable federation! This is incredible news: Rocket.Chat is one of the leading open source collaboration platforms with over 12 million users, and they will all shortly have the option to natively interoperate with the wider Matrix network: the feature has already landed (in alpha) in Rocket.Chat 4.7.0!
We’d like to thank the whole Rocket.Chat team for putting their faith in Matrix and joining the network: the whole idea of Matrix is that by banding together, different independent organisations can build an open decentralised network which is far stronger and more vibrant than any closed communication platform. The more organisations that join Matrix, the more useful and valuable the network becomes for everyone, and the more momentum there is to further refine and improve the protocol. Our intention is that Matrix will grow into a massive open ecosystem and industry, akin to the open Web itself… and that every organisation participating, be that Rocket.Chat, Element, Gitter, Beeper, Famedly or anyone else will benefit from being part of it. We are stronger together!
Rocket.Chat’s implementation follows the “How do you make an existing chat system talk Matrix?” approach we published based on our experiences of linking Gitter into Matrix. Looking at the initial pull request, the implementation lets Rocket.Chat act as a Matrix Application Service, effectively acting as a bridge to talk to an appropriate Matrix homeserver. From chatting with the team, it sounds like next steps will involve adding in encryption via our upcoming matrix-sdk-crypto node bindings - and then looking at ways to transparently embed a homeserver like Dendrite, sharing data as much as possible between RC and Matrix, so Rocket.Chat deployments can transparently sprout Matrix interoperability without having to run a separate homeserver. Super exciting!
You can see a quick preview of a Rocket.Chat user chatting away with an Element user on matrix.org via Matrix here:
So, exciting times ahead - needless to say we’ll be doing everything we can to support Rocket.Chat and ensure their Matrix integration is a success. And at this rate, they might be distinctly ahead of the curve if they start shipping Dendrite! Meanwhile, we have to wonder who will be next? Nextcloud? Mattermost? Place your bets… ;)
--Matthew
Update
Aaron from Rocket.Chat just published an excellent guide & video tour for how to actually set up your Rocket.Chat instance with Dendrite to get talking Matrix!
