Dan Callahan

29 posts tagged with "Dan Callahan" (See all Author)

Synapse 1.46.0 released

02.11.2021 00:00 — Releases Dan Callahan

Synapse 1.46.0 is out now and includes support for Ubuntu 21.10: Impish Indri.

This release includes a fix for a performance regression that surfaced in Synapse 1.44.

Progress on Threading

While still disabled by default, we're beginning to land code in Synapse to support threaded discussions in Matrix. In particular, the m.thread event relationship defined in MSC3440 is now implemented behind a configuration flag.

There are still many open questions here, and MSC3440 has not yet been approved to merge into the Matrix spec, but it's a start. Threading is essential to Matrix's continued growth and adoption, and we're excited to provide server-side support to this effort.

Bug Fixes and Refactors

This release is mostly comprised of bug fixes and improvements to static typing. Of note:

  • The export-data admin command now works and is tested in CI. This command helps server administrators respond to GDPR Subject Access Requests.
  • A weeks-long effort to refactor how Synapse validates the auth_events field of incoming PDUs has concluded, resolving a few corner cases which could incorrectly allow events into the room state which should instead be rejected.
  • Further static type hints have been added to Synapse, improving our precise type coverage to 85% of all lines. Currently, 13% of the Python files in synapse/ are skipped during mypy runs. We'd like to cut this in half by the end of the year, as well as increasing overall precise coverage in the codebase.

Sydent 2.5.0

This week also saw the release of Sydent 2.5.0, the reference implementation of a Matrix Identity Server. In addition to fixing bugs, Sydent 2.5.0 passes mypy --strict, uses Jinja2 for templates, and supports the room_type field from MSC3288 to better differentiate between invitations to rooms and invitations to Spaces.

Everything Else

  • The Admin API to create or modify accounts now accepts a user_type field, allowing for accounts to be set as belonging to bots or support staff.
  • Password auth providers can now be implemented using the new pluggable extension module APIs.

Please see the Synapse Release Notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including aaronraimist, AndrewFerr, dklimpel, and Legogris.

Synapse 1.45.1 released

20.10.2021 00:00 — Releases Dan Callahan

Synapse 1.45.1 is out now! Python 3.10 and PostgreSQL 14 are now tested and supported by Synapse. Support for Python 3.6 and PostgreSQL 9.6 will be removed by the end of the year.

Note: This release may require changes to how media storage providers access your homeserver's configuration. See the Upgrade Notes for more information.

Note: Synapse 1.45.0 was released yesterday and changed how Synapse's monthly active user limits were calculated. Today's release of 1.45.1 reverts that change, but is otherwise identical to 1.45.0.

Support for oEmbed Autodiscovery

Synapse can now automatically discover rich metadata when generating previews of links to sites which support oEmbed.





Note that URL previews are generated server-side, and thus generally disabled in encrypted rooms to avoid leaking information about message content to your homeserver. You may need to adjust the room's settings to see the new oEmbed previews.

Fixing Stuck Messages

This release of Synapse fixes a race condition which would occasionally prevent your sent events from syncing back down to all of your clients. This caused messages to look like they were stuck at the bottom of the room, waiting to finish sending, even though other users would receive and see them normally.

Improved Privacy for Per-Room Nicknames

Matrix allows users to set their display names to be different things in different rooms. For example, you might use an alias in public rooms, but your real name in rooms shared with friends and family.

To make it easy to initiate conversations with people, each homeserver maintains a user directory with the Matrix ID, display name, and avatar of the users it sees. Previously, this directory would be updated with the most recent profile metadata that Synapse had seen for a user, even if it was only changed in a single room.

As of 1.45, Synapse only uses includes the default display name of local users in its user directory, ignoring room-specific nicknames or avatars. (#5677).


This release includes numerous fixes and improvements to Synapse's internals.

  • We've added countless static type annotations to Synapse (and related projects like Sydent), giving us greater confidence in its correctness and reducing maintenance costs. Several modules newly have all of their definitions typed, allowing us to require and enforce complete type coverage for all future edits therein.

  • This release includes meaningful fixes and improvements to our OpenTracing and logging machinery, helping us better catch and eliminate bugs in Synapse. This work ultimately reduced matrix.org's Sentry event volume by an order of magnitude.

  • Magic accessor methods have been removed from Synapse's Config class. Previously, Synapse would interpret references like config.send_federation by attempting to guess a reasonable full path, like config.worker.send_federation. As of Synapse 1.45, the full path must be specified directly. This prevents errors where values could be drawn from unexpected or incorrect sections of the server's configuration.

Everything Else

We'd like to extend a special thanks to Fizzadar from Beeper for improving Synapse's update_synapse_database script (#10954) to allow schema changes to occur while Synapse is running. This is a great step toward reducing the downtime associated with upgrades.

These are just the highlights; please see the Upgrade Notes and Release Notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including AndrewFerr, dklimpel, Fizzadar, lukaslihotzki, and maxkratz.

Synapse 1.44.0 released

05.10.2021 23:44 — Releases Dan Callahan

Synapse 1.44.0 is out now!

Loads of Bug Fixes

This release primarily includes fixes for over a dozen long-standing bugs. For example:

  • Messages containing null bytes are now properly indexed for search.
  • The Room Search Admin API now allows searching for rooms with non-ascii titles.
  • URL preview caches, which are meant to be ephemeral, are no longer mirrored from the local media store into other media providers.

Faster JSON Responses

We now stay within C code while generating large JSON objects for responses, which should be substantially faster than the previous technique, which fell back to Python for encoding.

New Extension Module APIs

Spam checker modules can now use a user_may_create_room_with_invites callback to inspect room creation events which include invitations to users via Matrix or other media (email, etc.).

Additionally, the ModuleApi can now inspect IP and User Agent data, as well as checking whether Rooms and MXIDs are local to the current homeserver.

Everything Else

We've also been busy refactoring. For example:

  • Prometheus stats now distinguish between cache evictions due to entries expiring, and cache evictions due to the cache being full.
  • The synapse.handlers, synapse.rest, and synapse.storage.databases.state modules are now fully type annotated and checked by mypy, among others.
  • The user directory search code was cleaned up in preparation for future fixes.
  • Another half dozen pull requests were merged with an aim toward clarifying federated event authentication code (and associated logging).

Lastly, this release has seen considerable work toward supporting MSC2716: Incrementally importing history into existing rooms, along with an update to match the current state of MSC3231: Token authenticated registration.

These are just the highlights; please see the Upgrade Notes and Release Notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including aaronraimist, cvwright, govynnus, Kokokokoka, and tulir.

Synapse 1.43.0 released

21.09.2021 22:03 — Releases Dan Callahan

Synapse 1.43.0 is out now!

Preparing for Spaces

Quite a lot of work has gone into preparing for Spaces to graduate from beta testing. For example, Synapse now:

  • Asks clients to prefer room version 9 when creating restricted rooms (#10772), via the API defined in MSC3244: room version capabilities.
  • Allows the Spaces Summary APIs to be handled by worker processes.
  • Omits rooms with unknown room versions from the Spaces Summary.
  • Properly handles upgrades of Spaces to different room versions.

Everything Else

This release of Synapse also:

  • Includes initial work toward fully supporting oEmbed for embedding URL previews (#10714, #10759).
  • Slightly speeds up room joins over federation (#10754, #10755, #10756, #10780, #10784).
  • Somewhat improves service restart times for large Synapse deployments.
  • Significantly refactors federation event authentication code for greater clarity (#10744, #10745, #10746, #10771, #10773, #10781).
  • Adds further static type hints to various modules.

We've also spent quite a lot of time on SyTest, our integration test suite. In particular, many of the tests made assumptions about event processing which were not correct when targeting a multi-worker Synapse deployment. These flakey tests have plagued our continuous integration pipelines, and are finally being fixed.

These are just the highlights; please see the Release Notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including AndrewFerr, BramvdnHeuvel, and cuttingedge1109.

Disclosing CVE-2021-40823 and CVE-2021-40824: E2EE vulnerability in multiple Matrix clients

13.09.2021 17:29 — Security Denis Kasak

Today we are disclosing a critical security issue affecting multiple Matrix clients and libraries including Element (Web/Desktop/Android), FluffyChat, Nheko, Cinny, and SchildiChat. Element on iOS is not affected.

Specifically, in certain circumstances it may be possible to trick vulnerable clients into disclosing encryption keys for messages previously sent by that client to user accounts later compromised by an attacker.

Exploiting this vulnerability to read encrypted messages requires gaining control over the recipient’s account. This requires either compromising their credentials directly or compromising their homeserver.

Thus, the greatest risk is to users who are in encrypted rooms containing malicious servers. Admins of malicious servers could attempt to impersonate their users' devices in order to spy on messages sent by vulnerable clients in that room.

This is not a vulnerability in the Matrix or Olm/Megolm protocols, nor the libolm implementation. It is an implementation bug in certain Matrix clients and SDKs which support end-to-end encryption (“E2EE”).

We have no evidence of the vulnerability being exploited in the wild.

This issue was discovered during an internal audit by Denis Kasak, a security researcher at Element.

Remediation and Detection

Patched versions of affected clients are available now; please upgrade as soon as possible — we apologise sincerely for the inconvenience. If you are unable to upgrade, consider keeping vulnerable clients offline until you can. If vulnerable clients are offline, they cannot be tricked into disclosing keys. They may safely return online once updated.

Unfortunately, it is difficult or impossible to retroactively identify instances of this attack with standard logging levels present on both clients and servers. However, as the attack requires account compromise, homeserver administrators may wish to review their authentication logs for any indications of inappropriate access.

Similarly, users should review the list of devices connected to their account with an eye toward missing, untrusted, or non-functioning devices. Because an attacker must impersonate an existing or historical device, exploiting this vulnerability would either break an existing login on the user’s account, or a historical device would be re-added and flagged as untrusted.

Lastly, if you have previously verified the users / devices in a room, you would witness the safety shield on the room turn red during the attack, indicating the presence of an untrusted and potentially malicious device.

Affected Software

Given the severity of this issue, Element attempted to review all known encryption-capable Matrix clients and libraries so that patches could be prepared prior to public disclosure.

Known vulnerable software:

We believe the following software is not vulnerable:

We believe the following are not vulnerable due to not implementing key sharing:


Matrix supports the concept of “key sharing”, letting a Matrix client which lacks the keys to decrypt a message request those keys from that user's other devices or the original sender's device.

This was a feature added in 2016 in order to address edge cases where a newly logged-in device might not have the necessary keys to decrypt historical messages. Specifically, if other devices in the room are unaware of the new device due to a network partition, they have no way to encrypt for it—meaning that the only way the new device will be able to decrypt history is if the recipient's other devices share the necessary keys with it.

Other situations where key sharing is desirable include when the recipient hasn't backed up their keys (either online or offline) and needs them to decrypt history on a new login, or when facing implementation bugs which prevent clients from sending keys correctly. Requesting keys from a user's other devices sidesteps these issues.

Key sharing is described here in the Matrix E2EE Implementation Guide, which contains the following paragraph:

In order to securely implement key sharing, clients must not reply to every key request they receive. The recommended strategy is to share the keys automatically only to verified devices of the same user.

This is the approach taken in the original implementation in matrix-js-sdk, as used in Element Web and others, with the extension of also letting the sending device service keyshare requests from recipient devices. Unfortunately, the implementation did not sufficiently verify the identity of the device requesting the keyshare, meaning that a compromised account can impersonate the device requesting the keys, creating this vulnerability.

This is not a protocol or specification bug, but an implementation bug which was then unfortunately replicated in other independent implementations.

While we believe we have identified and contacted all affected E2EE client implementations: if your client implements key sharing requests, we strongly recommend you check that you cryptographically verify the identity of the device which originated the key sharing request.

Next Steps

The fact that this vulnerability was independently introduced so many times is a clear signal that the current wording in the Matrix Spec and the E2EE Implementation Guide is insufficient. We will thoroughly review the related documentation and revise it with clear guidelines on safely implementing key sharing.

Going further, we will also consider whether key sharing is still a necessary part of the Matrix protocol. If it is not, we will remove it. As discussed above, key sharing was originally introduced to make E2EE more reliable while we were ironing out its many edge cases and failure modes. Meanwhile, implementations have become much more robust, to the point that we may be able to go without key sharing completely. We will also consider changing how we present situations in which you cannot decrypt messages because the original sender was not aware of your presence. For example, undecryptable messages could be filed in a separate conversation thread, or those messages could require that keys are shared manually, effectively turning a bug into a feature.

We will also accelerate our work on matrix-rust-sdk as a portable reference implementation of the Matrix protocol, avoiding the implicit requirement that each independent library must necessarily reimplement this logic on its own. This will have the effect of reducing attack surface and simplifying audits for software which chooses to use matrix-rust-sdk.

Finally, we apologise to the wider Matrix community for the inconvenience and disruption of this issue. While Element discovered this vulnerability during an internal audit of E2EE implementations, we will be funding an independent end-to-end audit of the reference Matrix E2EE implementations (not just Olm + libolm) in the near future to help mitigate the risk from any future vulnerabilities. The results of this audit will be made publicly available.


Ultimately, Element took two weeks from initial discovery to completing an audit of all known, public E2EE implementations. It took a further week to coordinate disclosure, culminating in today's announcement.

  • Monday, 23rd August — Discovery that Element Web is exploitable.
  • Thursday, 26th August — Determination that Element Android is exploitable with a modified attack.
  • Wednesday, 1 September — Determination that Element iOS fails safe in the presence of device changes.
  • Friday, 3 September — Determination that FluffyChat and Nheko are exploitable.
  • Tuesday, 7th September — Audit of Matrix clients and libraries complete.
  • Wednesday, 8th September — Affected software authors contacted, disclosure timelines agreed.
  • Friday, 10th September — Public pre-disclosure notification. Downstream packagers (e.g., Linux distributions) notified via Matrix and e-mail.
  • Monday, 13th September — Coordinated releases of all affected software, public disclosure.

Synapse 1.42.0 released

07.09.2021 00:00 — Releases Dan Callahan

Synapse 1.42.0 is out now!

This release includes changes that you may need to be aware of before upgrading, such as the removal of two deprecated Admin APIs or a retroactive fix to ensure that email notifications are only sent to addresses which are presently associated with an account. Please see the Upgrade Notes for details.

Room Version 9: A Bugfix for Restricted Rooms

Synapse 1.42 includes support for Room Version 9, which fixes an oversight in the list of event fields which were protected from redaction in Room Version 8's restricted rooms. This makes it possible, in certain circumstances, for a restricted room to degrade into a state where participating servers will disagree about the room's membership.

Because changing a room version's redaction algorithm also changes the way that event IDs are calculated, properly fixing this issue required the creation of a new room version.

To ensure compatibility with existing servers, Synapse 1.42's MSC3244: Room version capabilities hints will continue to ask clients to prefer Room Version 8 when creating restricted rooms and Room Version 6 otherwise. A future release of Synapse will ask clients to prefer Room Version 9 for restricted rooms.

Handling Unknown Rooms

Very rarely, users find themselves in rooms created with unstable or experimental room versions. Then, when Synapse removes support for these versions, bad things happen. The server no longer understands how to interact with that room version, which means you can't interact with that room. And if you can't interact with that room, you can't leave.

In Synapse 1.42, rooms with unknown room versions are no longer returned down /sync. This prevents them from appearing in your client, though you may need to empty your client's cache and re-sync to see any effect.

New MSCs

In addition to Room Version 9 (MSC3375), this release includes:

  • An initial implementation of MSC3231: Token authenticated registration, which makes it possible for homeservers to disable user registration while still allowing new accounts to be made by people who know a pre-shared secret.

    This MSC and its implementations were produced as part of a Google Summer of Code (GSoC) project by Callum Brown.

  • An updated implementation of MSC2946: Spaces Summary following recent changes to the proposal.

  • Support for MSC3283: Expose capabilities for profile actions, which hints to clients whether or not a server allows users to change their display name, avatar, or email address.

Everything Else

In addition to the usual array of improvements to performance, type hints, error messages, and documentation:

  • Custom Presence Router modules can now be built using Synapse's new, unified module interface which debuted in Synapse 1.37.
  • Code around federation event handling and authentication has been significantly refactored to improve reliability and maintainability, including extracting nearly 1,800 lines of code from the FederationHandler class into a separate FederationEventHandler class.
  • Backfilling history and fetching missing events now use the same code paths, reducing the potential for bugs.
  • Concurrently fetching the same large set of events (#10703) is now much more efficient, preventing the process hangs which were possible in prior, extreme cases.

These are just the highlights; please see the Upgrade Notes and Release Notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including aaronraimist, dklimpel, govynnus, and HugoDelval.

Synapse 1.41.1 released

31.08.2021 00:00 — Releases Dan Callahan

Today we are releasing Synapse 1.41.1, a security update based on last week's release of Synapse 1.41.0. This release patches two moderate severity issues which could reveal metadata about private rooms:

  • GHSA-3x4c-pq33-4w3q / CVE-2021-39164: Enumerating a private room's list of members and their display names.

    If an unauthorized user both knows the Room ID of a private room and that room's history visibility is set to shared, then they may be able to enumerate the room's members, including their display names.

    The unauthorized user must be on the same homeserver as a user who is a member of the target room.

  • GHSA-jj53-8fmw-f2w2 / CVE-2021-39163: Disclosing a private room's name, avatar, topic, and number of members.

    If an unauthorized user knows the Room ID of a private room, then its name, avatar, topic, and number of members may be disclosed through Group / Community features.

    The unauthorized user must be on the same homeserver as a user who is a member of the target room, and their homeserver must allow non-administrators to create groups (enable_group_creation in the Synapse configuration; off by default).

Note that in both cases:

  • The private room's Room ID must be known to the attacker.
  • Another user on the attacker's homeserver must be a legitimate member of the target room.
  • The information disclosed is already present in the database and thus legitimately known to the administrators of homeservers participating in the target room.

We'd like to credit 0xkasper for discovering and responsibly disclosing these issues.

This release also fixes a small regression in 1.41.0 (#10709) which broke compatibility with older Twisted versions when Synapse was a configured to send email.

Please update at your earliest convenience.

Synapse 1.41.0 released

24.08.2021 00:00 — Releases Dan Callahan

Synapse 1.41.0 is out now!

NOTE: We anticipate publishing a security release next Tuesday, the 31st of August.

Synapse 1.41.1 will contain fixes for two moderate severity issues.

Synapse 1.41 includes changes to forward proxies, template settings, and media workers which may require your attention. Please see the Upgrade Notes for details.

Also, this release removes support for Ubuntu 20.10 (Groovy Gorilla), which reached End of Life last month. Support for Ubuntu 18.04 LTS (Bionic Beaver) and Ubuntu 21.04 (Hirsute Hippo) will be withdrawn near the end of this year under our platform deprecation policy.

Promoting Restricted Rooms

Synapse 1.41 includes experimental support for MSC3244: Room version capabilities, which indicates to clients that they should use Room Version 8 when creating restricted rooms. For example, Element's clients will use the MSC3244 metadata, which is on by default in Synapse 1.41, to determine whether to show end users the UI for creating restricted rooms.

Room Version 6 will remain the default for newly created rooms which do not explicitly request the restricted room capabilities at creation time. This strikes a balance between the broad compatibility of an older default room version, while still making newer features available upon request.

New MSCs

This version of Synapse implements experimental support for:

We've also added pagination to the Spaces Summary API based on updates to MSC2946.

Work towards implementing MSC2716: Incrementally importing history into existing rooms continues, with several related pull requests landing in this release.

More Admin APIs

Synapse now has an Admin APIs to:

Everything Else

In addition to the usual boosts to performance and reliability like #10119 or #10513, this release includes several notable improvements:

  • The /createRoom endpoint can now be handled by workers.
  • Synapse can now route outbound federation requests, remote media downloads, and public key fetches through a forward proxy.
  • Custom template configuration has now been centralized into a single custom_templates_directory configuration setting.
  • Matrix clients which allow users to set a status_msg with their presence will find that Synapse no longer arbitrarily unsets the message when a user goes offline. Instead, each user's status_msg will persist until it is deliberately cleared by their client.
  • The extension module API now exposes a function, get_userinfo_by_id, which accepts an MXID and returns a UserInfo object. This should simplify writing extension modules like spam checkers.

These are just the highlights; please see the Upgrade Notes and Release Notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including Bubu, dklimpel, H-Shay, ilmari, Kentokamoto, SimonBrandner, and warricksothr.

Synapse 1.40.0 released

10.08.2021 00:00 — Releases Dan Callahan

Synapse 1.40.0 is out now!

Support for Room Version 8

This release of Synapse includes stable support for Room Version 8! Version 8 codifies a concept of "restricted rooms" which are private rooms joinable only by direct invitation or by being a member of other, designated Spaces / rooms.

This means you can create a Matrix Space for your company, team, or guild, and instead of inviting people to the Space and directly to a bunch of private rooms, you can simply configure your private rooms such that membership in the Space automatically confers access to the private rooms.

This is a major improvement to access control in Matrix, and we can't wait to use it ourselves.

Note that client support for managing restricted rooms is still forthcoming, and you'll want to ensure that your users are on homeservers which support v8 before upgrading existing rooms. To ensure compatibility across federation, the default for newly created rooms remains at v6 for the time being.

Everything Else

A few other items worth calling out:

  • Synapse can now optionally recycle PostgreSQL connections after a specified number of transactions (#10440), which may mitigate slow memory leaks in the database. Thanks to GitHub user hifi.
  • We have a whole new documentation page explaining Room DAG Concepts like depth, stream ordering, extremities, outliers, and state groups.
  • The Admin API for listing accounts now returns a creation_ts property indicating when an account was created. Thanks to work by dklimpel.
  • Synapse should better detect character encodings in URL previews thanks to a contribution by srividyut.

In addition to further work towards implementing MSC2716: Incrementally Importing History into Existing Rooms, we've landed support for:

  • MSC2033: The /account/whoami endpoint now includes the device_id associated with the access token used to make the request.
  • (Experimental) MSC3244: The /_matrix/client/r0/capabilities endpoint may include metadata about which room versions support which capabilities (like knocking on restricted joins).
  • (Experimental) MSC2285: Clients may mark messages as read on the server without updating their externally visible read receipts, implemented by SimonBrandner.

These are just the highlights; please see the Release Notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including dklimpel, hifi, SimonBrandner, and srividyut.

Synapse 1.39.0 released

29.07.2021 00:00 — Releases Dan Callahan

Synapse 1.39.0 is now available!

We've done quite a bit of work this release on Synapse's new interface for extension modules:

  • Synapse's account_validity option has been extracted into a built-in extension module. If your configuration previously enabled account_validity, it will continue working as before, but it is now easier to override and customize.
  • The third party event rules callbacks have also been experimentally ported to the new module interface.

We've also improved the Space Summary API to list all rooms which a user could join (#10298). This is especially important with the pending standardization of Room Version 8, which includes MSC3083: Restricting room membership based on membership in other rooms.

MadLittleMods landed several pull requests on the path to implementing MSC2716: Incrementally importing history into existing rooms (#10250, #10276).

More than anything else, this release contains an absolutely enormous load of internal cleanup. For example:

  • ShadowJonathan ran pyupgrade on the entire codebase, bringing it up to modern Py3.6+ conventions, then separately used com2ann to move us from legacy # type: foo type hints to more modern inline annotations.
  • We finished converting several integer columns in our database to bigint as a precautionary measure.
  • We've made innumerable improvements to our CI configuration; including moving completely to GitHub Actions.
  • Countless small improvements to performance, reliability, and error logging.

But that's not all! Synapse 1.39 also allows for setting credentials for HTTP proxy connections thanks to work by dklimpel; previously, Synapse was only able to provide authentication for HTTPS proxies.

Please see the Upgrade Notes and Release Notes for a complete list of changes in this rele ase.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including andir, dklimpel, ilmari, LukeWalsh, moritzdietz, ShadowJonathan, and xmunoz.