Hey everyone, it's time for a new Synapse release! Synapse
1.69 is out, fresh
out of the oven. But before we take a look at it, here's a quick announcement:
We have recently disclosed a moderate severity security vulnerability, which
was fixed in Synapse
1.62 (released on
July 5th 2022). This issue affects all homeservers running a version of
Synapse older than 1.62 with open federation. If this is the case for your
deployment, please update to a more recent version of Synapse at your earliest
convenience.
Following up on last week's tutorial about using Docker Compose to install Synapse, this week Thib explains how to use Ansible to deploy your own Matrix homeserver.
Earlier in the year, t2bot.io passed 1 Million known rooms and now it's passed 10 Million bridged users (10,039,915 users to be exact, at time of writing). Most of these users will be people who have participated in a channel/chat on Discord or Telegram that was bridged to Matrix through t2bot.io's free service, with about 500 thousand being active each month.
Approximately 8 Million of the users are from Telegram, covering about 11% of all Telegram users (previously 15% based on information available at the time). The remaining 2 Million are Discord users, roughly 0.5% of Discord's user base. For perspective, t2bot.io has just over 683 Million events in the database and is bridging between 30 and 40 thousand people a day.
Like last time, this is just a milestone update, though it's also a good reminder to host your own server if you can. Element's own hosting platform is a great option if you'd like to have a server without running it yourself, and Beeper offers a richer bridging experience than t2bot.io can feasibly provide. If you'd like to go down the self-hosting route, check out Thib's video guide on hosting synapse or last week's Matrix Live for a better understanding of what hosting Synapse actually means.
As for an interesting statistic: despite not having much functionality that deals with Spaces, t2bot.io can see 2687 Spaces from the wider world. The plan in the coming months is to support a way to bridge a whole Discord server to a Matrix Space, making this statistic hopefully more interesting as time goes on.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
The Spec Core Team has been continuing to push forward on the spec. Several new MSCs have been opened recently. The Spec Core Team is available in #sct-office:matrix.org when MSC authors think that they are ready for primetime.
Bridges sometimes are unable to relay messages to the remote service for one reason or another. This MSC proposes a way to allow bridges to indicate that a message failed to be delivered, and allow users to tell the bridge to retry.
Usman's internship, working on Favourite Messages, is coming to an end! Check out Usman's blog post and Andy's blog post! To follow progress on Favourite Messages (which is still very much a prototype), check out the tracking issue: Tracking issue for Favourite Messages. Thanks to Usman for being an awesome mentee!
This week we've released Synapse 1.66.0rc1! This upcoming release deprecates delegating email validation to an identity server (more info here) and includes improved validation around user-interactive authentication, support for a couple of experimental features, as well as the usual batch of bug fixes and performance improvements 🙂
As always, any help with testing and feedback on this RC is appreciated! Feel free to drop any feedback or bug report in #synapse:matrix.org and the Synapse repo respectively.
This week we released Dendrite 0.9.5 which includes a number of fixes, particularly for federation:
The roomserver will now correctly unreject previously rejected events if necessary when reprocessing
The handling of event soft-failure has been improved on the roomserver input by no longer applying rejection rules and still calculating state before the event if possible
The federation /state and /state_ids endpoints should now return the correct error code when the state isn't known instead of returning a HTTP 500
The federation /event should now return outlier events correctly instead of returning a HTTP 500
A bug in the federation backoff allowing zero intervals has been corrected
The create-account utility will no longer error if the homeserver URL ends in a trailing slash
A regression in /sync introduced in 0.9.4 should be fixed
As always, please feel free to join us in #dendrite:matrix.org for more Dendrite-related discussion.
Thanks to Aine of etke.cc, matrix-docker-ansible-deploy can now set up the new Postmoogle email bridge/bot. Postmoogle is like the email2matrix bridge (also already supported by the playbook), but more capable and with the intention to soon support sending emails, not just receiving.
Quadrix v1.2.5 has been released! The update is already available for Linux, MacOS and iOS. The Windows and Android updates are awaiting approval from the respective stores. This release has mostly "under the hood" improvements (upgrade to React Native 0.69, React 18 and other key dependencies), but also fixes a few bugs and brings minor UI improvements.
Great news: Quadrix finally made it to https://matrix.org/clients/ :-) Many thanks to @madlittlemods:matrix.org!!!
Please leave feedback/comments at #quadrix:matrix.org or in the issues at https://github.com/alariej/quadrix (stars welcome :-))
In labs (you can enable labs features in settings on develop.element.io or on Nightly):
We’re working hard on updating Threads, squashing bugs and improving performance. We have several MSCs open introducing new functionality to read receipts so that notifications work better than ever.
We’re working hard on making the new layout ready for general use, squashing bugs and taking names until everything is in tip top shape. We have a test flight build out: we’ve delayed the release to next week while we iron out the last creases.
In ElementX land we have started on adding analytics and Xcode Cloud support and have updated our logging strategy. We will also start adopting sliding sync and using the new Rust Timeline providers
Element Call v0.2.7 and v0.2.8 have been released this past week, adding local volume control, full screen mode, audio in screen sharing and, ahem, fixing an embarrassing bug where we broke walkie-talkie mode... 🐑 Oh, and it's also all in TypeScript now. 🚀 https://github.com/vector-im/element-call/releases/tag/v0.2.7
With a few people out of office, this weeks has been one of the more quiet ones, but progress has been made non-the-less. Again a lot happens in draft PRs and the background, like with the upcoming Timeline API but also the path forward for integrating the crypto bindings into the js-sdk. There are a few notable PRs merged this week still improving the API (#972 and #973, #961), upgrading to latest ruma, removing dependencies (parking_lot) to improve compile times as well as merging the release infrastructure for crypto-js.
👉 Wanna hack on matrix rust? Go check out our help wanted tagged issues and join our matrix channel at Matrix Rust SDK.
This new bot allows users to use webhooks to forward monitoring alerts (e.g from prometheus) to matrix rooms. This means that you no longer have to use E-Mail or Slack to receive alerts. To set it up visit Github Alertmanager or
join #alertbot:hyteck.de
Matrix AI that generates messages based off other users' messages using a neural network. The bot trains its GPT-2 model using the CPU and is written in JavaScript (Node.JS) and Python. The project's code can be found here.
Hi all,
quite a lot happened since the last twim post a few months ago.
In a nutshell, we refactored the feed page and user page for a better viewing experience. We also now allow displaying and commenting post images in a dedicated view.
Also, you can now send follow request using knocking, thanks to profile as space support. (Yes, MSC is coming)
Finally, we have now multi account support, better stories display and refactored login and settings page.
Well... we almost modified everything :D
See more at https://minestrix.henri2h.fr/posts/
Stay tuned, event organization is coming soon (you can see the first implementation in the blog post.
PS: For those at the Matrix summit, I will be presenting it tomorrow
I recently made a blog post / video walk through of Matrix, hopefully it will be helpful to someone:
https://www.covingtoncreations.com/blog/what-can-matrix-do-for-your-organization
Have you ever felt lost in the Matrix world? Too many rooms and spaces to manage? Well, back by popular demand (with Timo's blessing), I present, The Room of the Week! Every week we strive to highlight a room or a space that we believe deserves attention for discussing interesting going on across the Matrix Network.
This week on room of the week:
We Are All Tech enthusiasts on The Matrix Network, but do you ever experience Tech burnout? Do you ever wish you could find discussions in The Matrix Universe about things other than Tech? Well, this week we bring you a very technical solution!
A space where you will find information about everything except technology. Groups are helpfully categorized by Subspace, and feature discussions about everything from musical instruments to beverages. If it isn't about computing, it's there.
If you have a room you wish to see highlighted, join us at:
https://matrix.to/#/!bIyiUUnriVoHtYzuPS:fachschaften.org?via=chat.shawnsorbom.net&via=matrix.org&via=fachschaften.org
To get your favorite room of the week highlighted.
Matthew shared with us the Matrix Summer Special 2022! Come read all about what's happened in Matrix-land so far this year, and what's coming up next, right here: https://matrix.org/blog/2022/08/15/the-matrix-summer-special-2022
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
This week the Spec Core Team focused on improvements to the spec source itself. richvdh opened a PR for edit events, and yours truly did a small PR to clarify the required state of the response to /_matrix/client/v3/login/.
There's a lot more open issues available for people to tackle however, so feel free to get involved and help out if you have some spare time!
Finally other than the usual rounds of review by the team, I've been working on a spec process document that aims to explain the practical portions of the text found at https://spec.matrix.org/proposals/, but in a easily scannable manner. Look out for a PR with that in the near future.
Historically, widgets have laid outside of the spec and have only been implemented in a small subset of clients - mainly Element. As of recent weeks though, there's now a team at Element backing the feature. So exciting times ahead!
Regardless, let's highlight this MSC! It solves a crucial problem with a simple solution. Widgets can ask the client they're embedded in to do certain things (if granted certain capabilities), and the client, potentially after asking the user for permission, can allow or deny those actions. This MSC adds the machinery for a further step: the client will tell the widget what capabilities they requested were allowed, and which were denied.
I believe this spec is non-contentious, but is blocked on widgets entering the spec as a whole. Regardless, if this particular piece of the puzzle interests you, or you'd like to read all about widgets in general, the see either the MSC above or this widget spec tracking PR: https://github.com/matrix-org/matrix.org/pull/825.
It's the last days of summer here and we are toiling away at making Synapse faster and leaner! In addition to continued work on faster joins, we released Synapse v1.65.0, with new features such as support for stable prefixes for private read receipts and a new module API method for creating a room (plus some other features!), as well as a host of bugfixes and internal changes to make Synapse faster and more stable.
Make sure to check it out!
This week we released Dendrite 0.9.4, containing primarily bug fixes:
A bug in the roomserver around handling rejected outliers has been fixed
Backfilled events will now use the correct history visibility where possible
The device list updater backoff has been fixed, which should reduce the number of outbound HTTP requests and Failed to query device keys for some users log entries for dead servers
The /sync endpoint will no longer incorrectly return room entries for retired invites which could cause some rooms to show up in the client "Historical" section
The /createRoom endpoint will now correctly populate is_direct in invite membership events, which may help clients to classify direct messages correctly
The create-account tool will now log an error if the shared secret is not set in the Dendrite config
A couple of minor bugs have been fixed in the membership lazy-loading
Queued EDUs in the federation API are now cached properly
As always, please feel free to join us in #dendrite:matrix.org for more Dendrite-related discussion.
This week has seen a quite a few Helm Chart updates; element-web got updated to 1.11.3, matrix-media-repo got a Redis usage fix, and matrix-synapse got updated to 1.65.0
Nheko now has some very dirty hack to render spoilers on desktop platforms. This does not show the reason and not work in mobile mode, doesn't hide it from notifications or from the sidebar. But it is at least something. Similarly we tightened what tags we allow when validating the incoming html again. Also, as a small fix, DMs should now also properly start with encryption enabled when started from a profile and there were a few crash fixes when searching for direct chat partners and closing the window too quickly or when a user uploads a device with invalid keys.
Bug fixes and final polishing has been taking place for our “Start DM on first message” project. This is where the user receiving a new room invite as a DM will not get the notification until you’ve sent a message.
The team is testing embedding Element Call in Element Web, as well as working on other improvements to Video rooms.
The new user’s checklist is live in product. It’s our first version so let us know what you think!
In labs (you can enable labs features in settings on develop.element.io or on Nightly):
Notifications improvements to Threads are underway. The team has been testing the new MSC and related Proof of Concept (POC) which we think will solve most of the issues with Threads right now.
We did the AppStore review dance and version 1.8.27 is now available. We even got better usage strings out of it.
We now have UI integration and performance tests in ElementX. Even more, they’re joined by some really nice Screenshot UI tests. Test all the things!
We’ve fixed some small bugs and some not so small ones, coming to an Element close to you early next week. The way things are laid out, you might even see a new feature land 😉
The new app layout testing session went well and we are looking for more iOS testers for future sessions. If you’d like to help out in future sessions, please join #element-community-testing:matrix.org
We had a very successful testing session with the all new app layout. If you’d like to take part in a future session, join us over at #element-community-testing:matrix.org
This week we’ve been working on fixing some FTUE crashes and covering some edge cases. Thanks to the community members submitting bugs and more info - keep it coming!
We’re investigating reports of missing messages, as well as a bug with the Threads beta where not all 'threaded messages' are showing up in the right place…
Just pushed another version (2.8.0) of the Ruby Matrix SDK, which drops support for the EoL Ruby 2.6 (and drops a bunch of workarounds for it) in order to support much improved caching of room state data, along with more helper methods and a fix for a floating accessor that didn't get hooked up correctly.
The Matrix Rust SDK team has been busy this week, too. Progress has been made on Siding Sync in particular, the types have been finalised and merged into mainline ruma, and more API has been made accessible via FFI. The new reactive Timeline API has been progressed, it, too, has some FFI definitions now, allowing mobile client to start playing with it. Crypto-JS, too, has progressed, adding support for de/encrypting attachments in an memory efficient fashion. On the crypto-side, the longer on-going refactor and improvements have yielded another few PRs, too, that have successfully merged, while others are still pending reviews. Other than that, we've seen quite a few smaller fixes and improvements, around logging, docs and the examples.
👉️ Wanna hack on matrix rust? Go check out our help wanted tagged issues and join our matrix channel at Matrix Rust SDK.
This week in the dart SDK we mostly fixed bugs. Many of those are related to call negotiations where streams were closed in the wrong order, not closed at all or in group calls the call never learned about new members or the call_ids would not match. There were also a few fixes to the background thumbnailing support added in 0.11.1, helper methods were added to easily send the right message corresponding to the mimetype of some media and fetching a timeline for some event id should work properly again.
We have some support to mark a room as either a dm or a group using slashcommands now, you have more flexibility when implementing the SSSS Bootstrap now (using the extra Bootstrap parameter in onUpdate() and to round it all off, we now have nice coverage numbers as well as coverage display on merge request diffs.
For more info, check the release notes for 0.11.2, 0.12.0, 0.12.1 and 0.12.2: https://pub.dev/packages/matrix/changelog ;-)
🔗Six days until Matrix Community Summit Berlin 2022
In less than a week the Matrix Community Summit Berlin is taking place at c-base.
Join us early on Thursday (25th August) for a Barcamp where we will brainstorm, draft and prototype new ideas.
The main conference days are Friday and Saturday (26th and 27th August). We have a schedule all about Matrix hosting, clients and development.
With three simultaneous tracks there sure is something for you to listen to. It's also the perfect place to get to know other community members.
Look forward to talks, workshops, a signing party, a Matrix P2P live test, dinner and barbecue!
We're not planing to stream or record the event. Our focus lies on providing a great in-person activities.
If attendees want to blog or toot/tweet about it, please use the hashtag #MatrixCommunitySummit.
Also, German-speaking folks can look forward to more coverage from the event on my podcast.
https://chrpaul.de/podcast/
Because we've heard about some confusion:
The event is NOT organised by the Matrix Foundation. To minimize the misconception, we've renamed it to the Matrix Community Summit Berlin.
This is an event initiated by Yan, jaller94 and other Matrix enthusiasts. We also organise the Matrix Meetup Berlin.
Have you ever felt lost in the Matrix world? Too many rooms and spaces to manage? Well, back by popular demand (with Timo's blessing), I present, The Room of the Week! Every week we strive to highlight a room or a space that we believe deserves attention for discussing interesting going on across the Matrix Network.
Last week, we were serving coffee, this week, it's tea! Specifically, tea at
Do you prefer sheng or shu puerh? Maybe bit more into lapsang, liuan or perhaps cliff oolongs? Greens or whites? Sencha? Having mood to discuss impact of varios clays and pot shapes? Is it better warm up water in bofura, tetsubin or electric kettle? Simply everything about the tea.
Come join us while the kettle is whistling.
If you have a room you wish to see highlighted, join us at:
https://matrix.to/#/!bIyiUUnriVoHtYzuPS:fachschaften.org?via=chat.shawnsorbom.net&via=matrix.org&via=fachschaften.org
To get your favorite room of the week highlighted.
A feature that the more privacy-focused users of Matrix have been missing was
the ability to hide read receipts from other users. Read receipts in rooms can
tell a user which messages another user has read in a room. However, they can
also be an unwelcome indicator that a user is currently reading a certain room,
thus giving away the user's activity on Matrix at a given time.
Hiding one's read receipts from other Matrix users is unfortunately not as
straightforward as simply preventing a client from sharing read receipts with
the server. This is because read receipts are also used by Matrix homeservers to
calculate how much of a room a user has read, and generate notification counts
for rooms accordingly.
Synapse 1.65 introduces stable support for private read receipts. This feature,
described by
MSC2285, allows
clients to send a different type of read receipt to the server. This then tells
the homeserver to use this piece of information to update the user's
notification counts, but not to share it with other users.
This version of Synapse includes two new module API methods to help Synapse
modules interact and manage rooms. The first one,
lookup_room_alias,
allows modules to retrieve the room ID corresponding to a given room alias. This
works both for local and remote aliases. The second one,
create_room,
allows modules to create new rooms on behalf of an existing user.
The
update_room_membership
method has also been updated in this release of Synapse to allow modules to join
a room the server is not already in via federation. This can be done by using
the new remote_room_hosts argument, which takes a list of homeservers to try
to join via.
Synapse 1.65 stabilises the implementation of
MSC3827, which
allows filtering public room searches on room types. This means it is now
possible to search specifically for public spaces. For more information on this
feature, see the Synapse 1.63
announcement.
Additionally, Synapse 1.65 implements the new experimental error codes
documented by
MSC3848. Once
stabilised, these error codes will allow clients to show more specific errors to
their users about why an event could not be sent.
See the full
changelog for a
complete list of changes in this release.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper,
andrewdoh, Julian-Samuel
Gebühr and Dirk
Klimpel, as well as anyone helping us make Synapse
better by sharing their feedback and reporting issues.
Synapse 1.4.0
introduced a configuration option (account_threepid_delegates.email) to allow
homeservers to delegate validating the ownership of email addresses to an
external identity server. This validation is used by Synapse when adding an
email address to a Matrix account, or before performing a password reset.
As of Synapse 1.64, this option is deprecated, and Synapse will print a warning
if it is used. This is because this option relies on old API endpoints that have
since been
removed from the
Matrix specification.
Synapse can do this validation internally provided it is configured with details
of an SMTP server. Administrators currently relying on
account_threepid_delegates.email should therefore ensure that an SMTP server
is correctly configured, and remove the account_threepid_delegates.email
option. See the configuration
guide
for more information.
We plan to fully remove this configuration option in Synapse 1.66, which is
expected to be released on August 30th.
Note that the equivalent option to validate the ownership of phone numbers
(account_threepid_delegates.msisdn) can still be used, though we expect to
deprecate it in a future release of Synapse.
When configuring an SMTP server to use to send out emails to users, server
administrators can configure Synapse to use TLS to communicate to that server.
Until now, only STARTTLS was
supported in this case.
Synapse 1.64 introduces a new force_tls configuration option in the email
section of the configuration file. If this new setting is set to true Synapse
will use TLS for the initial connection rather than upgrading via STARTTLS.
A couple of weeks ago, we
identified a
memory leak within frozendict, which is
a library that Synapse relies on. This would in turn cause Synapse instances to
slowly leak memory when processing /sync requests.
We highly encourage server administrators who installed Synapse via pip to
upgrade their local version of frozendict to version 2.3.3 or later, which
includes a fix to this issue. The Docker image matrixdotorg/synapse and the
Debian packages from packages.matrix.org already include the updated library.
This version of Synapse introduces support for room version 10! This new room
version enables support for the new knock_restricted join rule, to allow
knocking into rooms which are otherwise restricted to members of a specific room
(or space). See the Matrix specification about room version
10 for more information.
Additionally, Synapse 1.64 features a new rate limiter to limit the rate of joins to the same
room. It is intended as a mitigation against abuse scenarios involving joining a
lot of users from different homeservers to a room to then send spam across it.
See the configuration
guide
for more information.
This release of Synapse also extends the List
Rooms
and Room
Details
admin APIs to include the type of a room in responses, allowing server
administrators to differentiate spaces from other rooms.
See the full
changelog for a
complete list of changes in this release. Also please have a look at the
upgrade
notes
for this version.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper,
andrewdoh, Thomas
Weston, jejo86,
villepeh, Jörg
Behrmann and Jacek
Kuśnierz, as well as anyone helping us make Synapse
better by sharing their feedback and reporting issues.
Synapse has the ability to report usage statistics to the Matrix.org Foundation
(or to another location, if configured to do so). These statistics, such as
number of users, number of rooms joined by the server, etc. (they don't feature
any identifiable information about users and rooms) help us monitor the health
of the public federation.
In this release of Synapse, we have updated our public documentation about this
feature to clarify how it works and what exactly is being reported. This
documentation can be found right
here.
Note that previous documentation and prompts surrounding this feature called it
"anonymised" server statistics. This could easily be misinterpreted, as while
per-user statistics are not reported, homeserver server names are. We have
therefore changed said documentation and prompts to be clearer about
what is actually reported.
Note that your homeserver will never report any statistics if the report_stats
configuration option is set to false. Server admins who are curious about
which software is used by the Matrix.org Foundation to record server statistics
can check out panopticon.
This version of Synapse ships with an experimental implementation of
MSC3827 which
allows filtering public room search results by room type. This feature will
enable better discoverability for Matrix Spaces (which are rooms with a specific
type, under the hood), as it will enable clients to search specifically for
public spaces.
This feature is still experimental as its
MSC hasn't
completed the MSC process yet, though it is in its final comment period at the
time this post is being written. This means a stable implementation will be
coming to Synapse very soon, so watch this space!
Synapse 1.63 also includes a new rate limiter to limit invites per issuer. This
rate limiter can be configured using the new rc_invites.per_issuer
configuration setting, see the
documentation
for more information.
See the full
changelog for a
complete list of changes in this release.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper,
villepeh and Petr
Vaněk, as well as anyone helping us make Synapse
better by sharing their feedback and reporting issues.
In the past few weeks, the Synapse team has been working with the Matrix.org
Trust & Safety team to help module developers build more efficient protections
against spam. As a consequence of this work, Synapse 1.62 introduces new ways
for modules to communicate the result of actions taken against a specific
message or operation through the spam checker module
callbacks.
Previously, most spam checker callbacks would be expected to return a boolean
indicating whether a specific operation should be qualified as spam. Starting
from Synapse 1.62, modules are now expected to return either
synapse.module_api.NOT_SPAM (to indicate an action is not spammy), or an error
code that is part of synapse.module_api.errors.Codes.
Note that the previous behaviour is still supported but is now deprecated, and
will be removed in a future version of Synapse.
See the upgrade
notes
for a list of the affected callbacks and an example of this change. Note that on
top of the list described there, the check_event_for_spam callback was also
updated with a similar
change
in Synapse 1.61.
This release of Synapse includes important performance improvements around
syncing, specifically around handling device lists and notifications.
Synapse 1.62 also introduce a changes of its optional dependency on the LDAP3
authentication provider
module to
v0.2.1
in order to fix an issue with usernames that include uppercase characters.
See the full
changelog for a
complete list of changes in this release. Also please have a look at the
upgrade
notes
for this version.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper, Sami
Olmari, Daniel
Aloni,
Thumbscrew and Hannes
Lerchl.
