Hello people, it’s Synapse 1.7.0 time.

This release includes some long requested features, most notably the ability to automatically delete message data after a predefined period. For more details take a look at the config here ─ it should be pretty self explanatory.

Another significant change this release is to explicitly set room directories to be private by default. Previously it was possible to inadvertently configure the directory to be visible to arbitrary Matrix servers and the internet in general.

This means that for those admins who want their room directories to be publicly searchable (matrix.org for instance) they need to explicitly say so in the config. For more details see the upgrade notes and our blog post explaining the situation in greater detail.

We also have early support for ephemeral messages, as well as the ability to specify a reason when rejecting an invite (amongst other actions).

Aside from all of that, we want to let you know about some changes on the horizon. Currently Synapse runs Sqlite by default. This is great in that it gets new admins going quickly without needing to install and configure Postgres. The downside of using Sqlite is that it offers very poor performance, especially once a server tries to join the federation. In truth Sqlite is only really there to demonstrate the service, but for anything other than the most trivial cases it is essential to migrate to Postgres.

Over the past few months we’ve been working to improve the migration path to Postgres such that finally we feel confident to actively encourage admins to migrate. What’s more, in a future release we will forcibly prevent SQLite-backed servers federating unless the admin explicitly sets a config flag to show that they understand the trade-off they are making.

Overall we see these changes as something that will improve everyone’s experience of the matrix federation. We’ll talk more about this closer to the time, but please expect a change in the coming months and if you are running SQLite, consider this a nudge to get yourself migrated.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page.

The changelog since 1.6.1 follows:

Synapse 1.7.0 (2019-12-13)

This release changes the default settings so that only local authenticated users can query the server's room directory. See the upgrade notes for details.

Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11.

Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available here. A future release of synapse will, by default, disable federation for servers using SQLite.

No significant changes since 1.7.0rc2.

Synapse 1.7.0rc2 (2019-12-11)

Bugfixes

• Fix incorrect error message for invalid requests when setting user's avatar URL. (#6497)
• Fix support for SQLite 3.7. (#6499)
• Fix regression where sending email push would not work when using a pusher worker. (#6507, #6509)

Synapse 1.7.0rc1 (2019-12-09)

Features

• Implement per-room message retention policies. (#5815, #6436)
• Add etag and count fields to key backup endpoints to help clients guess if there are new keys. (#5858)
• Add /admin/v2/users endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. (#5925)
• Require User-Interactive Authentication for /account/3pid/add, meaning the user's password will be required to add a third-party ID to their account. (#6119)
• Implement the /_matrix/federation/unstable/net.atleastfornow/state/<context> API as drafted in MSC2314. (#6176)
• Configure privacy-preserving settings by default for the room directory. (#6355)
• Add ephemeral messages support by partially implementing MSC2228. (#6409)
• Add support for MSC 2367, which allows specifying a reason on all membership events. (#6434)

Bugfixes

• Transfer non-standard power levels on room upgrade. (#6237)
• Fix error from the Pillow library when uploading RGBA images. (#6241)
• Correctly apply the event filter to the state, events_before and events_after fields in the response to /context requests. (#6329)
• Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. (#6332)
• Prevent account data syncs getting lost across TCP replication. (#6333)
• Fix bug: TypeError in register_user() while using LDAP auth module. (#6406)
• Fix an intermittent exception when handling read-receipts. (#6408)
• Fix broken guest registration when there are existing blocks of numeric user IDs. (#6420)
• Fix startup error when http proxy is defined. (#6421)
• Fix error when using synapse_port_db on a vanilla synapse db. (#6449)
• Fix uploading multiple cross signing signatures for the same user. (#6451)
• Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. (#6462)
• Fix synapse_port_db not exiting with a 0 code if something went wrong during the port process. (#6470)
• Improve sanity-checking when receiving events over federation. (#6472)
• Fix inaccurate per-block Prometheus metrics. (#6491)
• Fix small performance regression for sending invites. (#6493)
• Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. (#6494)

Improved Documentation

• Update documentation and variables in user contributed systemd reference file. (#6369, #6490)
• Fix link in the user directory documentation. (#6388)
• Add build instructions to the docker readme. (#6390)
• Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. (#6443)
• Write some docs for the quarantine_media api. (#6458)
• Convert CONTRIBUTING.rst to markdown (among other small fixes). (#6461)

Deprecations and Removals

• Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. (#5925)
• Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. (#6488)

Internal Changes

• Add benchmarks for structured logging and improve output performance. (#6266)
• Improve the performance of outputting structured logging. (#6322)
• Refactor some code in the event authentication path for clarity. (#6343, #6468, #6480)
• Clean up some unnecessary quotation marks around the codebase. (#6362)
• Complain on startup instead of 500'ing during runtime when public_baseurl isn't set when necessary. (#6379)
• Add a test scenario to make sure room history purges don't break /messages in the future. (#6392)
• Clarifications for the email configuration settings. (#6423)
• Add more tests to the blacklist when running in worker mode. (#6429)
• Refactor data store layer to support multiple databases in the future. (#6454, #6464, #6469, #6487)
• Port synapse.rest.client.v1 to async/await. (#6482)
• Port synapse.rest.client.v2_alpha to async/await. (#6483)
• Port SyncHandler to async/await. (#6484)

Synapse 1.6.0 has landed and is here to brighten your day!

1.6.0's most notable feature is that of label based filtering. It allows for messages to be tagged with a given label so that clients can filter on the label, this means that clients can subscribe to specific topics in a room, such as #lunch.

Completely separately, from here on in new rooms will be version 5 by default, all this means in practice is that servers will respect server signing key validity periods. This won't make a lot of difference in day to day operation, but it is an important security consideration and we now have sufficient penetration across the federation to make version 5 the default.

Aside from that there are a bunch of bug fixes and improvements, including fixing a bug that in some cases prevented messages being decrypted shortly after a restart (#6363) and generally improving the room upgrade experience (#6232, #6235).

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page

The changelog since 1.5.1 follows:

Synapse 1.6.0 (2019-11-26)

Bugfixes

• Fix phone home stats reporting. (#6418)

Synapse 1.6.0rc2 (2019-11-25)

Bugfixes

• Fix a bug which could cause the background database update handler for event labels to get stuck in a loop raising exceptions. (#6407)

Synapse 1.6.0rc1 (2019-11-20)

Features

• Add federation support for cross-signing. (#5727)
• Increase default room version from 4 to 5, thereby enforcing server key validity period checks. (#6220)
• Add support for outbound http proxying via http_proxy/HTTPS_PROXY env vars. (#6238)
• Implement label-based filtering on /sync and /messages (MSC2326). (#6301, #6310, #6340)

Bugfixes

• Fix LruCache callback deduplication for Python 3.8. Contributed by @V02460. (#6213)
• Remove a room from a server's public rooms list on room upgrade. (#6232, #6235)
• Delete keys from key backup when deleting backup versions. (#6253)
• Make notification of cross-signing signatures work with workers. (#6254)
• Fix exception when remote servers attempt to join a room that they're not allowed to join. (#6278)
• Prevent errors from appearing on Synapse startup if git is not installed. (#6284)
• Appservice requests will no longer contain a double slash prefix when the appservice url provided ends in a slash. (#6306)
• Fix /purge_room admin API. (#6307)
• Fix the hidden field in the devices table for SQLite versions prior to 3.23.0. (#6313)
• Fix bug which caused rejected events to be persisted with the wrong room state. (#6320)
• Fix bug where rc_login ratelimiting would prematurely kick in. (#6335)
• Prevent the server taking a long time to start up when guest registration is enabled. (#6338)
• Fix bug where upgrading a guest account to a full user would fail when account validity is enabled. (#6359)
• Fix to_device stream ID getting reset every time Synapse restarts, which had the potential to cause unable to decrypt errors. (#6363)
• Fix permission denied error when trying to generate a config file with the docker image. (#6389)

Improved Documentation

• Contributor documentation now mentions script to run linters. (#6164)
• Modify CAPTCHA_SETUP.md to update the terms private key and public key to secret key and site key respectively. Contributed by Yash Jipkate. (#6257)
• Update INSTALL.md Email section to talk about account_threepid_delegates. (#6272)
• Fix a small typo in account_threepid_delegates configuration option. (#6273)

Internal Changes

• Add a CI job to test the synapse_port_db script. (#6140, #6276)
• Convert EventContext to an attrs. (#6218)
• Move persist_events out from main data store. (#6240, #6300)
• Reduce verbosity of user/room stats. (#6250)
• Reduce impact of debug logging. (#6251)
• Expose some homeserver functionality to spam checkers. (#6259)
• Change cache descriptors to always return deferreds. (#6263, #6291)
• Fix incorrect comment regarding the functionality of an if statement. (#6269)
• Update CI to run isort over the scripts and scripts-dev directories. (#6270)
• Replace every instance of logger.warn method with logger.warning as the former is deprecated. (#6271, #6314)
• Port replication http server endpoints to async/await. (#6274)
• Port room rest handlers to async/await. (#6275)
• Remove redundant CLI parameters on CI's flake8 step. (#6277)
• Port federation_server.py to async/await. (#6279)
• Port receipt and read markers to async/wait. (#6280)
• Split out state storage into separate data store. (#6294, #6295)
• Refactor EventContext for clarity. (#6298)
• Update the version of black used to 19.10b0. (#6304)
• Add some documentation about worker replication. (#6305)
• Move admin endpoints into separate files. Contributed by Awesome Technologies Innovationslabor GmbH. (#6308)
• Document the use of lint.sh for code style enforcement & extend it to run on specified paths only. (#6312)
• Add optional python dependencies and dependant binary libraries to snapcraft packaging. (#6317)
• Remove the dependency on psutil and replace functionality with the stdlib resource module. (#6318, #6336)
• Improve documentation for EventContext fields. (#6319)
• Add some checks that we aren't using state from rejected events. (#6330)
• Add continuous integration for python 3.8. (#6341)
• Correct spacing/case of various instances of the word "homeserver". (#6357)
• Temporarily blacklist the failing unit test PurgeRoomTestCase.test_purge_room. (#6361)

Matrix Live 🎙

Sorry folks, no matrix live this week, your regular programming will continue next week.

Dept of Spec

A big week in spec land.

Client-server r0.6.0 and Identity r0.3.0 spec releases

TravisR offered:

If you've been waiting all this time to start implementing some of the privacy improvements the team has been making over the last few months, now's the best time to do it. Clients interacting with identity servers or 3rd party identifiers (3PIDs) have some changes to make, and identity servers themselves have a whole new authed API so they can expose terms of service requirements to users.

Check out the specs at https://matrix.org/docs/spec/client_server/r0.6.0 and https://matrix.org/docs/spec/identity_service/r0.3.0

You can read all about it here

anoa announced:

Approved MSCs

• MSC2334 - Change default room version to v5
  • Our fastest FCP ever - less than 24hrs!

Final Comment Period

• MSC2244: Mass redactions
• MSC1946: Secure Secret Storage and Sharing

New MSCs

• [WIP] MSC2346: Bridge information state event

The Spec Core Team have chosen MSC1756 (cross-signing), MSC2324 (new spec process), and MSC2313 (ban lists) to focus on this week :)

Dept of Clients 📱

Riot-iOS

Manu offered:

• 0.10.1 has been released on the app store. It includes minor improvements and bug fixes like the call issue. Full release descriptions can be found on respective repos: Riot, matrix-ios-kit and matrix-ios-sdk.
• This release includes better logging to track app kills in background but it seems that iOS13.2.2 released by Apple yesterday fixes the issue. We are looking for more feedbacks on that topic.
• On develop, the app can now use the integrations manager advertised by the homeserver.

RiotX

benoit reported:

We have finished implementing long click on a Room item, to configure notification settings of the room and to be able to leave the room. We can now ignore user (after a report of content only for the moment). The list of ignored users is displayed in the setting. Users can be un-ignored. We are also working on improving performance and improving code structure. As usual, we have also fixed some bugs. A release will be done at the beginning of next week, then we will try to work on the login flow and account creation flow.

Riot Android

benoit offered:

We are working on the Integration Manager management.

Riot Web

Neil had this to say:

The Riot gang landed 1.5.1 which contained the emoji picker (thanks Tulir!).

Additionally they have been working furiously to make some progress against e2ee device cross signing and have just merged the ability to authenticate via DMs. It's behind a labs flag and will only work if both parties are enabled, but this is big step towards our cross signing dreams. Watch this space for more cross signing features over the coming weeks.

Fractal

Alexandre Franke offered:

We did a couple bugfixes including one for a long standing crasher and another more recent crasher.

Seshat

@poljar:matrix.org said:

Seshat gained the ability to encrypt the event database and index. A PR has landed and is awaiting review. Now your Matrix client can securely store and index your E2EE messages.

Quarternion

kitsune said:

I took time to port Quaternion from Qt Quick Controls 1 (deprecated upstream) to Qt Quick Controls 2, the lighter UI widgets kit that Spectral also uses. Most of regressions are fixed and the result is likely to land in the master branch sometime next week. The overall looks will remain the same, just a minor refresh of visuals. Aside from improving performance the porting should help to solve widget scaling issues on multi-monitor configurations.

Matrix Notepad

@kb1rd:kb1rd.net reported:

• Released version 0.1.4, which fixes a number of bugs and cleans up a lot of the code
• I'm considering converting the algorithm code only to TypeScript (it's a mess rn)
• I said this last week, but I think I finally got all the bugs out of the internal algorithm 🤞
• I'm also working on a proper explanation of how the algorithm works (An explanation of just the events is here)

Dept of Servers 🏢

Synapse

Neil said:

This week we shipped 1.5.1 which contains a small bug fix to prevent a potential DoS vector.

Separately we merged support for label based filtering into develop and are now working on allowing admins to configure message retention policies.

Aside from that we’ve continued to work on sharding out the database which we’ll put live once we have migrated matrix.org onto new hardware (woo!) and finally we’ve been fixing some bugs affecting event auth rules.

Coming up on the horizon are ephemeral messages (the ability to send messages with a specific ttl), more io perf work and a bit further down the line we’ll dust off our attempts to shard out room processing from the master process, meaning Synapses running in worker mode will have much more CPU headroom.

Rust Workers

Black Hat told us:

I'm working on writing Synapse workers in Rust. Only a few things are working, but the performance seems good.

Department of Ops

Kubernetes

Ananace announced:

Just pushed the K8s-optimized Synapse 1.5.1 image

Docker - improved LDAP

JCG reported:

another Synapse container image, but this one is new: If you had problems with LDAP in the official Synapse image, try this image: https://gitlab.com/famedly/container/synapse-ldap/container_registry. It's based on the official images, but updates the LDAP auth provider to the latest commit of the master branch. The official image comes with the latest version released to pypi.org, which is a bit older. Aside of that change, it's exactly the same, so you can use it as a drop in replacement.

Docker - multi-arch

Black Hat told us:

my multi-arch synapse docker image is updated to synapse 1.5.1

Docker

Mathijs said:

The images for synapse v1.5.1 were pushed to docker hub for avhost and my personal repo.

Server Tombola

@mj:nltrix.net offered:

It gets a tombola of versions at the list of Public Synapse Homeservers https://www.anchel.nl/matrix-publiclist/

Dept of Bridges 🌉

Half-Shot announced:

Hey folks, I've released matrix-appservice-irc 0.13.1 which fixes a critical bug in 0.13.0 where messages from matrix would crash the bridge. Users brave enough to be running develop do not need to do anything. https://github.com/matrix-org/matrix-appservice-irc/releases/tag/0.13.1. This would only have affected you if you tried to install or update the 0.13 bridge in the last two weeks.

Dept of Ping 🏓

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

That's right folks Synapse 1.5.0 is here and ready to make your life just a little bit better.

First things first, this release includes a security fix (#6262, below). Administrators are encouraged to upgrade as soon as possible.

Aside from that, the main thing you'll notice in 1.5.0 is a massive performance improvement to the room directory, which means that servers with large directories to scan will return much more quickly. This is especially true for matrix.org but all servers will benefit.

Another key win is finally fixing some bugs in the sqlite -> postgres migrator script. Sqlite mode is there strictly for testing purposes and should never be used in a production setting let alone a federating homeserver. So if you are currently using Sqlite now is the time to migrate. What's more the script is now in CI so it can't easily break in the future (with apologies to anyone bitten by the old script...).

A final final point, we have some preparatory work for e2ee cross signing, the overall feature is not ready for release just yet but we are getting really close. Watch this space.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page

The changelog since 1.4.1 follows:

Synapse 1.5.0 (2019-10-29)

Security updates

This release includes a security fix (#6262, below). Administrators are encouraged to upgrade as soon as possible.

Bugfixes

• Fix bug where room directory search was case sensitive. (#6268)

Synapse 1.5.0rc2 (2019-10-28)

Bugfixes

• Update list of boolean columns in synapse_port_db. (#6247)
• Fix /keys/query API on workers. (#6256)
• Improve signature checking on some federation APIs. (#6262)

Internal Changes

• Move schema delta files to the correct data store. (#6248)
• Small performance improvement by removing repeated config lookups in room stats calculation. (#6255)

Synapse 1.5.0rc1 (2019-10-24)

Features

• Improve quality of thumbnails for 1-bit/8-bit color palette images. (#2142)
• Add ability to upload cross-signing signatures. (#5726)
• Allow uploading of cross-signing keys. (#5769)
• CAS login now provides a default display name for users if a displayname_attribute is set in the configuration file. (#6114)
• Reject all pending invites for a user during deactivation. (#6125)
• Add config option to suppress client side resource limit alerting. (#6173)

Bugfixes

• Return an HTTP 404 instead of 400 when requesting a filter by ID that is unknown to the server. Thanks to @krombel for contributing this! (#2380)
• Fix a bug where users could be invited twice to the same group. (#3436)
• Fix /createRoom failing with badly-formatted MXIDs in the invitee list. Thanks to @wener291! (#4088)
• Make the synapse_port_db script create the right indexes on a new PostgreSQL database. (#6102, #6178, #6243)
• Fix bug when uploading a large file: Synapse responds with M_UNKNOWN while it should be M_TOO_LARGE according to spec. Contributed by Anshul Angaria. (#6109)
• Fix user push rules being deleted from a room when it is upgraded. (#6144)
• Don't 500 when trying to exchange a revoked 3PID invite. (#6147)
• Fix transferring notifications and tags when joining an upgraded room that is new to your server. (#6155)
• Fix bug where guest account registration can wedge after restart. (#6161)
• Fix monthly active user reaping when reserved users are specified. (#6168)
• Fix /federation/v1/state endpoint not supporting newer room versions. (#6170)
• Fix bug where we were updating censored events as bytes rather than text, occasionally causing invalid JSON being inserted breaking APIs that attempted to fetch such events. (#6186)
• Fix occasional missed updates in the room and user directories. (#6187)
• Fix tracing of non-JSON APIs, /media, /key etc. (#6195)
• Fix bug where presence would not get timed out correctly if a synchrotron worker is used and restarted. (#6212)
• synapse_port_db: Add 2 additional BOOLEAN_COLUMNS to be able to convert from database schema v56. (#6216)
• Fix a bug where the Synapse demo script blacklisted ::1 (ipv6 localhost) from receiving federation traffic. (#6229)

Updates to the Docker image

• Fix logging getting lost for the docker image. (#6197)

Internal Changes

• Update user_filters table to have a unique index, and non-null columns. Thanks to @pik for contributing this. (#1172, #6175, #6184)
• Allow devices to be marked as hidden, for use by features such as cross-signing. This adds a new field with a default value to the devices field in the database, and so the database upgrade may take a long time depending on how many devices are in the database. (#5759)
• Move lookup-related functions from RoomMemberHandler to IdentityHandler. (#5978)
• Improve performance of the public room list directory. (#6019, #6152, #6153, #6154)
• Edit header dicts docstrings in SimpleHttpClient to note that str or bytes can be passed as header keys. (#6077)
• Add snapcraft packaging information. Contributed by @devec0. (#6084, #6191)
• Kill off half-implemented password-reset via sms. (#6101)
• Remove get_user_by_req opentracing span and add some tags. (#6108)
• Drop some unused database tables. (#6115)
• Add env var to turn on tracking of log context changes. (#6127)
• Refactor configuration loading to allow better typechecking. (#6137)
• Log responder when responding to media request. (#6139)
• Improve performance of find_next_generated_user_id DB query. (#6148)
• Expand type-checking on modules imported by synapse.config. (#6150)
• Use Postgres ANY for selecting many values. (#6156)
• Add more caching to _get_joined_users_from_context DB query. (#6159)
• Add some metrics on the federation sender. (#6160)
• Add some logging to the rooms stats updates, to try to track down a flaky test. (#6167)
• Remove unused timeout parameter from _get_public_room_list. (#6179)
• Reject (accidental) attempts to insert bytes into postgres tables. (#6186)
• Make version optional in body of PUT /room_keys/version/{version}, since it's redundant. (#6189)
• Make storage layer responsible for adding device names to key, rather than the handler. (#6193)
• Port synapse.rest.admin module to use async/await. (#6196)
• Enforce that all boolean configuration values are lowercase in CI. (#6203)
• Remove some unused event-auth code. (#6214)
• Remove Auth.check method. (#6217)
• Remove format_tap.py script in favour of a perl reimplementation in Sytest's repo. (#6219)
• Refactor storage layer in preparation to support having multiple databases. (#6231)
• Remove some extra quotation marks across the codebase. (#6236)

Wheyhey as I live and breathe 1.4.0 is here!

1.4.0 should be considered ‘The Privacy Release’ and is the cumulation of a huge body of work spanning multiple projects to improve data privacy in the matrix ecosystem. You can read more about the full project here.

While we consider 1.4.0 to be a huge leap forward in terms of data privacy, it is really important to note that it contains breaking changes.

• If you currently rely on email or SMS delivery via an identity server you must modify your Synapse configuration.
• If you have configured custom templates, then eight new templates must be added to your templates directory.

If either apply to you failure to act will break your installation. Full details can be found in the upgrade notes.

So what has changed?

It is possible for a user to associate an email address or phone number with their account, for a number of reasons:

• For use when logging in, as an alternative to the user id.
• In the case of email, as an alternative contact to help with account recovery.
• In the case of email, to receive notifications of missed messages.

Before an email address or phone number can be added to a user's account, or before such an address is used to carry out a password-reset, Synapse must confirm the operation with the owner of the email address or phone number. It does this by sending an email or text giving the user a link or token to confirm receipt. This process is known as '3pid verification'. ('3pid', or 'threepid', stands for third-party identifier, and we use it to refer to external identifiers such as email addresses and phone numbers.)

Previous versions of Synapse delegated the task of 3pid verification to an identity server by default. In most cases this server is vector.im or matrix.org.

In Synapse 1.4.0, for security and privacy reasons, the homeserver will no longer delegate this task to an identity server by default. Instead, the server administrator will need to explicitly decide how they would like the verification messages to be sent.

In the medium term, the vector.im and matrix.org identity servers will disable support for delegated 3pid verification entirely. However, in order to ease the transition, they will retain the capability for a limited period. Delegated email verification will be disabled on Monday 2nd December 2019 (giving roughly 2 months notice). Disabling delegated SMS verification will follow some time after that once SMS verification support lands in Synapse.

Once delegated 3pid verification support has been disabled in the vector.im and matrix.org identity servers, all Synapse versions that depend on those instances will be unable to verify email and phone numbers through them. There are no imminent plans to remove delegated 3pid verification from Sydent generally. (Sydent is the identity server project that backs the vector.im and matrix.org instances).

Why is this necessary?

Prior to 1.4.0, the identity server was providing two related-but-separate functions:

a directory for users to publish their contact details and to look up their contacts by their email addresses and phone numbers. a "trusted third party communications network delegate", providing SMS- and email-sending functionality to all homeservers for registration and password reset.

The intention behind the identity server's providing 2. was one of convenience: to save homeserver admins the burden of configuring their homeservers to send email, as well as sparing us the effort of writing generic SMS-gateway integration that would allow any homeserver admin to configure their homeserver to send SMS via their chosen SMS aggregator.

By exposing this 'trusted email/sms delegate' functionality, however, and by including references to the New Vector identity servers in the default configuration for both Synapse and Riot, we created a situation in which users on non-New Vector run homeservers (who had never seen our privacy notice) could easily end up sharing their email addresses and phone numbers with a New Vector identity server.

Using identity servers for registration and password reset introduced yet further complexity - since password reset is a sensitive operation, it was important that the homeserver only use 'trusted' identity servers for this purpose (with the trust being configured by the homeserver admin in the homeserver config). But this created ambiguity over who was ultimately responsible for the choice of identity server and when they could make that choice - the client could present a default identity server at login/registration/password reset time, which the user could choose to override before logging in, but unless the identity server ultimately selected were on the homeserver admin's 'trusted identity servers' list, any identity server operations that were proxied via the homeserver would have been blocked by the homeserver. However, not all identity server operations initiated by the client were proxied via the homeserver - some were sent directly from the client to the identity server, and would not have been filtered.

The best way to solve this problem is to have individual homeservers take ownership for their own email and SMS needs. In the case of email this means configuring details of an appropriate SMTP server or continuing to delegate through an identity server that will allow it to do so. If a homeserver admin makes the active choice to use New Vector's identity servers for delegation (up until 1st December), they should make their own users aware through their privacy notice.

This delegation is entirely separate from the user's choice of identity server for user directory services. As of right now the user is free to choose and trust whichever identity server they wish, or to choose not to use an identity server at all.

Are there any other data privacy features?

Yes, 1.4.0 now automatically garbage collects redacted messages (defaults to 7 days) and removes unused IP and user agent information stored in the user_ips table (defaults to 30 days). Finally, Synapse now warns in its logs if you are using matrix.org as a trusted key server, in case you wish to use a different server to help discover other servers’ keys.

Anything else?

Aside from privacy, we’ve expanded our OpenTracing support and fixed a host of bugs. However the thing that is most exciting is switching on our solution for mitigating forward extremities build up’ by default.

In some cases rooms can accumulate ‘forward extremities’, which are simply an artefact of attempting to resolve the room state over multiple servers. Forward extremities are necessary to ensure that each server can independently arrive at the same view of the room eventually, however processing these extremities can be computationally expensive and degrade server performance overall.

Originally it was an experimental config option but we now feel confident to turn it on by default for all instances - it should make a big difference for the CPU of servers in fragmented rooms.

So that’s it folks, thanks for making it this far. As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page

The changelog since 1.3.1 follows:

Synapse 1.4.0 (2019-10-03)

Bugfixes

• Redact client_secret in server logs. (#6158)

Synapse 1.4.0rc2 (2019-10-02)

Bugfixes

• Fix bug in background update that adds last seen information to the devices table, and improve its performance on Postgres. (#6135)
• Fix bad performance of censoring redactions background task. (#6141)
• Fix fetching censored redactions from DB, which caused APIs like initial sync to fail if it tried to include the censored redaction. (#6145)
• Fix exceptions when storing large retry intervals for down remote servers. (#6146)

Internal Changes

• Fix up sample config entry for redaction_retention_period option. (#6117)

Synapse 1.4.0rc1 (2019-09-26)

Note that this release includes significant changes around 3pid verification. Administrators are reminded to review the upgrade notes.

Features

• Changes to 3pid verification:
  • Add the ability to send registration emails from the homeserver rather than delegating to an identity server. (#5835, #5940, #5993, #5994, #5868)
  • Replace trust_identity_server_for_password_resets config option with account_threepid_delegates, and make the id_server parameteter optional on */requestToken endpoints, as per MSC2263. (#5876, #5969, #6028)
  • Switch to using the v2 Identity Service /lookup API where available, with fallback to v1. (Implements MSC2134 plus id_access_token authentication for v2 Identity Service APIs from MSC2140). (#5897)
  • Remove bind_email and bind_msisdn parameters from /register ala MSC2140. (#5964)
  • Add m.id_access_token to unstable_features in /versions as per MSC2264. (#5974)
  • Use the v2 Identity Service API for 3PID invites. (#5979)
  • Add POST /_matrix/client/unstable/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account. (#5980, #6062)
  • Use account_threepid_delegate.email and account_threepid_delegate.msisdn for validating threepid sessions. (#6011)
  • Allow homeserver to handle or delegate email validation when adding an email to a user's account. (#6042)
  • Implement new Client Server API endpoints /account/3pid/add and /account/3pid/bind as per MSC2290. (#6043)
  • Add an unstable feature flag for separate add/bind 3pid APIs. (#6044)
  • Remove bind parameter from Client Server POST /account endpoint as per MSC2290. (#6067)
  • Add POST /add_threepid/msisdn/submit_token endpoint for proxying submitToken on an account_threepid_handler. (#6078)
  • Add submit_url response parameter to */msisdn/requestToken endpoints. (#6079)
  • Add m.require_identity_server flag to /version's unstable_features. (#5972)
• Enhancements to OpenTracing support:
  • Make OpenTracing work in worker mode. (#5771)
  • Pass OpenTracing contexts between servers when transmitting EDUs. (#5852)
  • OpenTracing for device list updates. (#5853)
  • Add a tag recording a request's authenticated entity and corresponding servlet in OpenTracing. (#5856)
  • Add minimum OpenTracing for client servlets. (#5983)
  • Check at setup that OpenTracing is installed if it's enabled in the config. (#5985)
  • Trace replication send times. (#5986)
  • Include missing OpenTracing contexts in outbout replication requests. (#5982)
  • Fix sending of EDUs when OpenTracing is enabled with an empty whitelist. (#5984)
  • Fix invalid references to None while OpenTracing if the log context slips. (#5988, #5991)
  • OpenTracing for room and e2e keys. (#5855)
  • Add OpenTracing span over HTTP push processing. (#6003)
• Add an admin API to purge old rooms from the database. (#5845)
• Retry well-known lookups if we have recently seen a valid well-known record for the server. (#5850)
• Add support for filtered room-directory search requests over federation (MSC2197, in order to allow upcoming room directory query performance improvements. (#5859)
• Correctly retry all hosts returned from SRV when we fail to connect. (#5864)
• Add admin API endpoint for setting whether or not a user is a server administrator. (#5878)
• Enable cleaning up extremities with dummy events by default to prevent undue build up of forward extremities. (#5884)
• Add config option to sign remote key query responses with a separate key. (#5895)
• Add support for config templating. (#5900)
• Users with the type of "support" or "bot" are no longer required to consent. (#5902)
• Let synctl accept a directory of config files. (#5904)
• Increase max display name size to 256. (#5906)
• Add admin API endpoint for getting whether or not a user is a server administrator. (#5914)
• Redact events in the database that have been redacted for a week. (#5934)
• New prometheus metrics:
  • synapse_federation_known_servers: represents the total number of servers your server knows about (i.e. is in rooms with), including itself. Enable by setting metrics_flags.known_servers to True in the configuration.(#5981)
  • synapse_build_info: exposes the Python version, OS version, and Synapse version of the running server. (#6005)
• Give appropriate exit codes when synctl fails. (#5992)
• Apply the federation blacklist to requests to identity servers. (#6000)
• Add report_stats_endpoint option to configure where stats are reported to, if enabled. Contributed by @Sorunome. (#6012)
• Add config option to increase ratelimits for room admins redacting messages. (#6015)
• Stop sending federation transactions to servers which have been down for a long time. (#6026)
• Make the process for mapping SAML2 users to matrix IDs more flexible. (#6037)
• Return a clearer error message when a timeout occurs when attempting to contact an identity server. (#6073)
• Prevent password reset's submit_token endpoint from accepting trailing slashes. (#6074)
• Return 403 on /register/available if registration has been disabled. (#6082)
• Explicitly log when a homeserver does not have the trusted_key_servers config field configured. (#6090)
• Add support for pruning old rows in user_ips table. (#6098)

Bugfixes

• Don't create broken room when power_level_content_override.users does not contain creator_id. (#5633)
• Fix database index so that different backup versions can have the same sessions. (#5857)
• Fix Synapse looking for config options password_reset_failure_template and password_reset_success_template, when they are actually password_reset_template_failure_html, password_reset_template_success_html. (#5863)
• Fix stack overflow when recovering an appservice which had an outage. (#5885)
• Fix error message which referred to public_base_url instead of public_baseurl. Thanks to @aaronraimist for the fix! (#5909)
• Fix 404 for thumbnail download when dynamic_thumbnails is false and the thumbnail was dynamically generated. Fix reported by rkfg. (#5915)
• Fix a cache-invalidation bug for worker-based deployments. (#5920)
• Fix admin API for listing media in a room not being available with an external media repo. (#5966)
• Fix list media admin API always returning an error. (#5967)
• Fix room and user stats tracking. (#5971, #5998, #6029)
• Return a M_MISSING_PARAM if sid is not provided to /account/3pid. (#5995)
• federation_certificate_verification_whitelist now will not cause TypeErrors to be raised (a regression in 1.3). Additionally, it now supports internationalised domain names in their non-canonical representation. (#5996)
• Only count real users when checking for auto-creation of auto-join room. (#6004)
• Ensure support users can be registered even if MAU limit is reached. (#6020)
• Fix bug where login error was shown incorrectly on SSO fallback login. (#6024)
• Fix bug in calculating the federation retry backoff period. (#6025)
• Prevent exceptions being logged when extremity-cleanup events fail due to lack of user consent to the terms of service. (#6053)
• Remove POST method from password-reset submit_token endpoint until we implement submit_url functionality. (#6056)
• Fix logcontext spam on non-Linux platforms. (#6059)
• Ensure query parameters in email validation links are URL-encoded. (#6063)
• Fix a bug which caused SAML attribute maps to be overridden by defaults. (#6069)
• Fix the logged number of updated items for the users_set_deactivated_flag background update. (#6092)
• Add sid to next_link for email validation. (#6097)
• Threepid validity checks on msisdns should not be dependent on threepid_behaviour_email. (#6104)
• Ensure that servers which are not configured to support email address verification do not offer it in the registration flows. (#6107)

Updates to the Docker image

• Avoid changing UID/GID if they are already correct. (#5970)
• Provide SYNAPSE_WORKER envvar to specify python module. (#6058)

Improved Documentation

• Convert documentation to markdown (from rst) (#5849)
• Update INSTALL.md to say that Python 2 is no longer supported. (#5953)
• Add developer documentation for using SAML2. (#6032)
• Add some notes on rolling back to v1.3.1. (#6049)
• Update the upgrade notes. (#6050)

Deprecations and Removals

• Remove shared-secret registration from /_matrix/client/r0/register endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. (#5877)
• Deprecate the trusted_third_party_id_servers option. (#5875)

Internal Changes

• Lay the groundwork for structured logging output. (#5680)
• Retry well-known lookup before the cache expires, giving a grace period where the remote well-known can be down but we still use the old result. (#5844)
• Remove log line for debugging issue #5407. (#5860)
• Refactor the Appservice scheduler code. (#5886)
• Compatibility with v2 Identity Service APIs other than /lookup. (#5892, #6013)
• Stop populating some unused tables. (#5893, #6047)
• Add missing index on users_in_public_rooms to improve the performance of directory queries. (#5894)
• Improve the logging when we have an error when fetching signing keys. (#5896)
• Add support for database engine-specific schema deltas, based on file extension. (#5911)
• Update Buildkite pipeline to use plugins instead of buildkite-agent commands. (#5922)
• Add link in sample config to the logging config schema. (#5926)
• Remove unnecessary parentheses in return statements. (#5931)
• Remove unused jenkins/prepare_sytest.sh file. (#5938)
• Move Buildkite pipeline config to the pipelines repo. (#5943)
• Remove unnecessary return statements in the codebase which were the result of a regex run. (#5962)
• Remove left-over methods from v1 registration API. (#5963)
• Cleanup event auth type initialisation. (#5975)
• Clean up dependency checking at setup. (#5989)
• Update OpenTracing docs to use the unified trace method. (#5776)
• Small refactor of function arguments and docstrings in RoomMemberHandler. (#6009)
• Remove unused origin argument on FederationHandler.add_display_name_to_third_party_invite. (#6010)
• Add a failure_ts column to the destinations database table. (#6016, #6072)
• Clean up some code in the retry logic. (#6017)
• Fix the structured logging tests stomping on the global log configuration for subsequent tests. (#6023)
• Clean up the sample config for SAML authentication. (#6064)
• Change mailer logging to reflect Synapse doesn't just do chat notifications by email now. (#6075)
• Move last-seen info into devices table. (#6089)
• Remove unused parameter to get_user_id_by_threepid. (#6099)
• Refactor the user-interactive auth handling. (#6105)
• Refactor code for calculating registration flows. (#6106)

Some of you will have been bitten by a bug that prevented Synapse 1.3.0 from starting up correctly. If this is you, please upgrade.

Additionally we have taken the opportunity to fix a dependency bug for our intrepid packagers.

Apologies for the inconvenience.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page

The changelog since 1.3.0 follows:

Synapse 1.3.1 (2019-08-17)

Features

• Drop hard dependency on sdnotify python package. (#5871)

Bugfixes

• Fix startup issue (hang on ACME provisioning) due to ordering of Twisted reactor startup. Thanks to @chrismoos for supplying the fix. (#5867)

Well now, Synapse 1.3.0 is here.

The main thing to know about 1.3.0 is that is contains performance improvements to reduce disk I/O and reduce RAM usage. We’ve been running it on matrix.org for a week or so and are really pleased with the results.

Check out our message send heat map.

Other than that there are a bunch of bug fixes and tweaks to generally make things run more smoothly.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page

The changelog since 1.2.1 follows:

Synapse 1.3.0 (2019-08-15)

Bugfixes

• Fix 500 Internal Server Error on publicRooms when the public room list was cached. (#5851)

Synapse 1.3.0rc1 (2019-08-13)

Features

• Use M_USER_DEACTIVATED instead of M_UNKNOWN for errcode when a deactivated user attempts to login. (#5686)
• Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify. (#5732)
• Synapse will no longer serve any media repo admin endpoints when enable_media_repo is set to False in the configuration. If a media repo worker is used, the admin APIs relating to the media repo will be served from it instead. (#5754, #5848)
• Synapse can now be configured to not join remote rooms of a given "complexity" (currently, state events) over federation. This option can be used to prevent adverse performance on resource-constrained homeservers. (#5783)
• Allow defining HTML templates to serve the user on account renewal attempt when using the account validity feature. (#5807)

Bugfixes

• Fix UISIs during homeserver outage. (#5693, #5789)
• Fix stack overflow in server key lookup code. (#5724)
• start.sh no longer uses deprecated cli option. (#5725)
• Log when we receive an event receipt from an unexpected origin. (#5743)
• Fix debian packaging scripts to correctly build sid packages. (#5775)
• Correctly handle redactions of redactions. (#5788)
• Return 404 instead of 403 when accessing /rooms/{roomId}/event/{eventId} for an event without the appropriate permissions. (#5798)
• Fix check that tombstone is a state event in push rules. (#5804)
• Fix error when trying to login as a deactivated user when using a worker to handle login. (#5806)
• Fix bug where user /sync stream could get wedged in rare circumstances. (#5825)
• The purge_remote_media.sh script was fixed. (#5839)

Deprecations and Removals

• Synapse now no longer accepts the -v/--verbose, -f/--log-file, or --log-config command line flags, and removes the deprecated verbose and log_file configuration file options. Users of these options should migrate their options into the dedicated log configuration. (#5678, #5729)
• Remove non-functional 'expire_access_token' setting. (#5782)

Internal Changes

• Make Jaeger fully configurable. (#5694)
• Add precautionary measures to prevent future abuse of window.opener in default welcome page. (#5695)
• Reduce database IO usage by optimising queries for current membership. (#5706, #5738, #5746, #5752, #5770, #5774, #5792, #5793)
• Improve caching when fetching get_filtered_current_state_ids. (#5713)
• Don't accept opentracing data from clients. (#5715)
• Speed up PostgreSQL unit tests in CI. (#5717)
• Update the coding style document. (#5719)
• Improve database query performance when recording retry intervals for remote hosts. (#5720)
• Add a set of opentracing utils. (#5722)
• Cache result of get_version_string to reduce overhead of /version federation requests. (#5730)
• Return 'user_type' in admin API user endpoints results. (#5731)
• Don't package the sytest test blacklist file. (#5733)
• Replace uses of returnValue with plain return, as returnValue is not needed on Python 3. (#5736)
• Blacklist some flakey tests in worker mode. (#5740)
• Fix some error cases in the caching layer. (#5749)
• Add a prometheus metric for pending cache lookups. (#5750)
• Stop trying to fetch events with event_id=None. (#5753)
• Convert RedactionTestCase to modern test style. (#5768)
• Allow looping calls to be given arguments. (#5780)
• Set the logs emitted when checking typing and presence timeouts to DEBUG level, not INFO. (#5785)
• Remove DelayedCall debugging from the test suite, as it is no longer required in the vast majority of Synapse's tests. (#5787)
• Remove some spurious exceptions from the logs where we failed to talk to a remote server. (#5790)
• Improve performance when making .well-known requests by sharing the SSL options between requests. (#5794)
• Disable codecov GitHub comments on PRs. (#5796)
• Don't allow clients to send tombstone events that reference the room it's sent in. (#5801)
• Deny redactions of events sent in a different room. (#5802)
• Deny sending well known state types as non-state events. (#5805)
• Handle incorrectly encoded query params correctly by returning a 400. (#5808)
• Handle pusher being deleted during processing rather than logging an exception. (#5809)
• Return 502 not 500 when failing to reach any remote server. (#5810)
• Reduce global pauses in the events stream caused by expensive state resolution during persistence. (#5826)
• Add a lower bound to well-known lookup cache time to avoid repeated lookups. (#5836)
• Whitelist history visibility sytests in worker mode tests. (#5843)

Today we release Synapse 1.2.1 as a critical security update. It contains patches relating to redactions and event federation. The patches address long standing bugs, and are not regressions specific to the previous version (1.2). All admins, regardless of current version, should upgrade asap.

This release includes four security fixes:

• Prevent an attack where a federated server could send redactions for arbitrary events in v1 and v2 rooms. (#5767)
• Prevent a denial-of-service attack where cycles of redaction events would make Synapse spin infinitely. Thanks to @lrizika:matrix.org for identifying and responsibly disclosing this issue. (0f2ecb961)
• Prevent an attack where users could be joined or parted from public rooms without their consent. Thanks to Dylanger for identifying and responsibly disclosing this issue. (#5744)
• Fix a vulnerability where a federated server could spoof read-receipts from users on other servers. Thanks to Dylanger for identifying this issue too. (#5743)

Additionally, the following fix was in Synapse 1.2.0, but was not correctly identified during the original release:

• It was possible for a room moderator to send a redaction for an m.room.create event, which would downgrade the room to version 1. Thanks to @/dev/ponies:ponies.im for identifying and responsibly disclosing this issue! (#5701)

You can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Alternatively check out our Synapse installation guide page

Thanks for bearing with us.

Hey hey, Synapse 1.2.0 is here. It contains aggregations support, better error handling for deactivated accounts and some important bug fixes for redacting messages. Special thanks to community members skalarproduktraum and Lrizika for submissions to improve our documentation.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. Also, check out our Synapse installation guide page

The changelog since 1.1.0 follows:

Synapse 1.2.0 (2019-07-25)

No significant changes.

Synapse 1.2.0rc2 (2019-07-24)

Bugfixes

• Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. (#5734)

Synapse 1.2.0rc1 (2019-07-22)

Features

• Add support for opentracing. (#5544, #5712)
• Add ability to pull all locally stored events out of synapse that a particular user can see. (#5589)
• Add a basic admin command app to allow server operators to run Synapse admin commands separately from the main production instance. (#5597)
• Add sender and origin_server_ts fields to m.replace. (#5613)
• Add default push rule to ignore reactions. (#5623)
• Include the original event when asking for its relations. (#5626)
• Implement session_lifetime configuration option, after which access tokens will expire. (#5660)
• Return "This account has been deactivated" when a deactivated user tries to login. (#5674)
• Enable aggregations support by default (#5714)

Bugfixes

• Fix 'utime went backwards' errors on daemonization. (#5609)
• Various minor fixes to the federation request rate limiter. (#5621)
• Forbid viewing relations on an event once it has been redacted. (#5629)
• Fix requests to the /store_invite endpoint of identity servers being sent in the wrong format. (#5638)
• Fix newly-registered users not being able to lookup their own profile without joining a room. (#5644)
• Fix bug in #5626 that prevented the original_event field from actually having the contents of the original event in a call to /relations. (#5654)
• Fix 3PID bind requests being sent to identity servers as application/x-form-www-urlencoded data, which is deprecated. (#5658)
• Fix some problems with authenticating redactions in recent room versions. (#5699, #5700, #5707)
• Ignore redactions of m.room.create events. (#5701)

Updates to the Docker image

• Base Docker image on a newer Alpine Linux version (3.8 -> 3.10). (#5619)
• Add missing space in default logging file format generated by the Docker image. (#5620)

Improved Documentation

• Add information about nginx normalisation to reverse_proxy.rst. Contributed by @skalarproduktraum - thanks! (#5397)
• --no-pep517 should be --no-use-pep517 in the documentation to setup the development environment. (#5651)
• Improvements to Postgres setup instructions. Contributed by @Lrizika - thanks! (#5661)
• Minor tweaks to postgres documentation. (#5675)

Deprecations and Removals

• Remove support for the invite_3pid_guest configuration setting. (#5625)

Internal Changes

• Move logging code out of synapse.util and into synapse.logging. (#5606, #5617)
• Add a blacklist file to the repo to blacklist certain sytests from failing CI. (#5611)
• Make runtime errors surrounding password reset emails much clearer. (#5616)
• Remove dead code for persiting outgoing federation transactions. (#5622)
• Add lint.sh to the scripts-dev folder which will run all linting steps required by CI. (#5627)
• Move RegistrationHandler.get_or_create_user to test code. (#5628)
• Add some more common python virtual-environment paths to the black exclusion list. (#5630)
• Some counter metrics exposed over Prometheus have been renamed, with the old names preserved for backwards compatibility and deprecated. See docs/metrics-howto.rst for details. (#5636)
• Unblacklist some user_directory sytests. (#5637)
• Factor out some redundant code in the login implementation. (#5639)
• Update ModuleApi to avoid register(generate_token=True). (#5640)
• Remove access-token support from RegistrationHandler.register, and rename it. (#5641)
• Remove access-token support from RegistrationStore.register, and rename it. (#5642)
• Improve logging for auto-join when a new user is created. (#5643)
• Remove unused and unnecessary check for FederationDeniedError in _exception_to_failure. (#5645)
• Fix a small typo in a code comment. (#5655)
• Clean up exception handling around client access tokens. (#5656)
• Add a mechanism for per-test homeserver configuration in the unit tests. (#5657)
• Inline issue_access_token. (#5659)
• Update the sytest BuildKite configuration to checkout Synapse in /src. (#5664)
• Add a docker type to the towncrier configuration. (#5673)
• Convert synapse.federation.transport.server to async. Might improve some stack traces. (#5689)
• Documentation for opentracing. (#5703)

Right folks, this is our first post 1.0 release, which means that we have now officially dropped support for Python 2 and Postgres 9.4. This means that we can start making use of Python 3 specific features and you should expect lots of associated performance wins over the coming months. See the upgrade notes for more.

Synapse 1.1.0 also contains a reworked approach to the Docker image, as well lots of performance improvements with special focus on DB IO - expect more to come in this area.

Special thanks to community member Alexander Trost for rounding out our SAML support and also to Daniel Hoffend for contributing the ability to disable local password authentication.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. Also, check out our Synapse installation guide page

The changelog since 1.0.0 follows:

Synapse 1.1.0 (2019-07-04)

As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4. See the upgrade notes for more details.

This release also deprecates the use of environment variables to configure the docker image. See the docker README for more details.

No changes since 1.1.0rc2.

Synapse 1.1.0rc2 (2019-07-03)

Bugfixes

• Fix regression in 1.1rc1 where OPTIONS requests to the media repo would fail. (#5593)
• Removed the SYNAPSE_SMTP_* docker container environment variables. Using these environment variables prevented the docker container from starting in Synapse v1.0, even though they didn't actually allow any functionality anyway. (#5596)
• Fix a number of "Starting txn from sentinel context" warnings. (#5605)

Internal Changes

• Update github templates. (#5552)

Synapse 1.1.0rc1 (2019-07-02)

As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4. See the upgrade notes for more details.

Features

• Added possibility to disable local password authentication. Contributed by Daniel Hoffend. (#5092)
• Add monthly active users to phonehome stats. (#5252)
• Allow expired user to trigger renewal email sending manually. (#5363)
• Statistics on forward extremities per room are now exposed via Prometheus. (#5384, #5458, #5461)
• Add --no-daemonize option to run synapse in the foreground, per issue #4130. Contributed by Soham Gumaste. (#5412, #5587)
• Fully support SAML2 authentication. Contributed by Alexander Trost - thank you! (#5422)
• Allow server admins to define implementations of extra rules for allowing or denying incoming events. (#5440, #5474, #5477)
• Add support for handling pagination APIs on client reader worker. (#5505, #5513, #5531)
• Improve help and cmdline option names for --generate-config options. (#5512)
• Allow configuration of the path used for ACME account keys. (#5516, #5521, #5522)
• Add --data-dir and --open-private-ports options. (#5524)
• Split public rooms directory auth config in two settings, in order to manage client auth independently from the federation part of it. Obsoletes the "restrict_public_rooms_to_local_users" configuration setting. If "restrict_public_rooms_to_local_users" is set in the config, Synapse will act as if both new options are enabled, i.e. require authentication through the client API and deny federation requests. (#5534)
• The minimum TLS version used for outgoing federation requests can now be set with federation_client_minimum_tls_version. (#5550)
• Optimise devices changed query to not pull unnecessary rows from the database, reducing database load. (#5559)
• Add new metrics for number of forward extremities being persisted and number of state groups involved in resolution. (#5476)

Bugfixes

• Fix bug processing incoming events over federation if call to /get_missing_events fails. (#5042)
• Prevent more than one room upgrade happening simultaneously on the same room. (#5051)
• Fix a bug where running synapse_port_db would cause the account validity feature to fail because it didn't set the type of the email_sent column to boolean. (#5325)
• Warn about disabling email-based password resets when a reset occurs, and remove warning when someone attempts a phone-based reset. (#5387)
• Fix email notifications for unnamed rooms with multiple people. (#5388)
• Fix exceptions in federation reader worker caused by attempting to renew attestations, which should only happen on master worker. (#5389)
• Fix handling of failures fetching remote content to not log failures as exceptions. (#5390)
• Fix a bug where deactivated users could receive renewal emails if the account validity feature is on. (#5394)
• Fix missing invite state after exchanging 3PID invites over federaton. (#5464)
• Fix intermittent exceptions on Apple hardware. Also fix bug that caused database activity times to be under-reported in log lines. (#5498)
• Fix logging error when a tampered event is detected. (#5500)
• Fix bug where clients could tight loop calling /sync for a period. (#5507)
• Fix bug with jinja2 preventing Synapse from starting. Users who had this problem should now simply need to run pip install matrix-synapse. (#5514)
• Fix a regression where homeservers on private IP addresses were incorrectly blacklisted. (#5523)
• Fixed m.login.jwt using unregistered user_id and added pyjwt>=1.6.4 as jwt conditional dependencies. Contributed by Pau Rodriguez-Estivill. (#5555, #5586)
• Fix a bug that would cause invited users to receive several emails for a single 3PID invite in case the inviter is rate limited. (#5576)

Updates to the Docker image

• Add ability to change Docker containers timezone with the TZ variable. (#5383)
• Update docker image to use Python 3.7. (#5546)
• Deprecate the use of environment variables for configuration, and make the use of a static configuration the default. (#5561, #5562, #5566, #5567)
• Increase default log level for docker image to INFO. It can still be changed by editing the generated log.config file. (#5547)
• Send synapse logs to the docker logging system, by default. (#5565)
• Open the non-TLS port by default. (#5568)
• Fix failure to start under docker with SAML support enabled. (#5490)
• Use a sensible location for data files when generating a config file. (#5563)

Deprecations and Removals

• Python 2.7 is no longer a supported platform. Synapse now requires Python 3.5+ to run. (#5425)
• PostgreSQL 9.4 is no longer supported. Synapse requires Postgres 9.5+ or above for Postgres support. (#5448)
• Remove support for cpu_affinity setting. (#5525)

Improved Documentation

• Improve README section on performance troubleshooting. (#4276)
• Add information about how to install and run black on the codebase to code_style.rst. (#5537)
• Improve install docs on choosing server_name. (#5558)

Internal Changes

• Add logging to 3pid invite signature verification. (#5015)
• Update example haproxy config to a more compatible setup. (#5313)
• Track deactivated accounts in the database. (#5378, #5465, #5493)
• Clean up code for sending federation EDUs. (#5381)
• Add a sponsor button to the repo. (#5382, #5386)
• Don't log non-200 responses from federation queries as exceptions. (#5383)
• Update Python syntax in contrib/ to Python 3. (#5446)
• Update federation_client dev script to support .well-known and work with python3. (#5447)
• SyTest has been moved to Buildkite. (#5459)
• Demo script now uses python3. (#5460)
• Synapse can now handle RestServlets that return coroutines. (#5475, #5585)
• The demo servers talk to each other again. (#5478)
• Add an EXPERIMENTAL config option to try and periodically clean up extremities by sending dummy events. (#5480)
• Synapse's codebase is now formatted by black. (#5482)
• Some cleanups and sanity-checking in the CPU and database metrics. (#5499)
• Improve email notification logging. (#5502)
• Fix "Unexpected entry in 'full_schemas'" log warning. (#5509)
• Improve logging when generating config files. (#5510)
• Refactor and clean up Config parser for maintainability. (#5511)
• Make the config clearer in that email.template_dir is relative to the Synapse's root directory, not the synapse/ folder within it. (#5543)
• Update v1.0.0 release changelog to include more information about changes to password resets. (#5545)
• Remove non-functioning check_event_hash.py dev script. (#5548)
• Synapse will now only allow TLS v1.2 connections when serving federation, if it terminates TLS. As Synapse's allowed ciphers were only able to be used in TLSv1.2 before, this does not change behaviour. (#5550)
• Logging when running GC collection on generation 0 is now at the DEBUG level, not INFO. (#5557)
• Reduce the amount of stuff we send in the docker context. (#5564)
• Point the reverse links in the Purge History contrib scripts at the intended location. (#5570)