Synapse 0.99.3 released!

Hey hey, a Synapse release for you today.

The big news in 0.99.3 is that the user directory has been completely re-written and should now be much more performant - this will benefit all installations, but especially those housing larger servers.

Aside from that we continue our 1.0 preparations and relatedly we've improved our docs, in particular to explain how .well-known works. On the perf side we've added rate limiting to login and register endpoints as well as now batching up read receipts to send over federation.

I've said it before, and I'll say it again:-

The most important thing that admins should know is that prior to 1.0 landing later this month, it is essential that the federation API has a valid TLS certificate - self signed certificates will no longer be accepted. For more details see our handy guide. Failure to do this will result in being unable to federate with other 1.0 servers.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. Also, check out our new Synapse installation guide page.

🔗Synapse 0.99.3 changelog

🔗Features

• The user directory has been rewritten to make it faster, with less chance of falling behind on a large server. (#4537, #4846, #4864, #4887, #4900, #4944)
• Add configurable rate limiting to the /register endpoint. (#4735, #4804)
• Move server key queries to federation reader. (#4757)
• Add support for /account/3pid REST endpoint to client_reader worker. (#4759)
• Add an endpoint to the admin API for querying the server version. Contributed by Joseph Weston. (#4772)
• Include a default configuration file in the 'docs' directory. (#4791, #4801)
• Synapse is now permissive about trailing slashes on some of its federation endpoints, allowing zero or more to be present. (#4793)
• Add support for /keys/query and /keys/changes REST endpoints to client_reader worker. (#4796)
• Add checks to incoming events over federation for events evading auth (aka "soft fail"). (#4814)
• Add configurable rate limiting to the /login endpoint. (#4821, #4865)
• Remove trailing slashes from certain outbound federation requests. Retry if receiving a 404. Context: #3622. (#4840)
• Allow passing --daemonize flags to workers in the same way as with master. (#4853)
• Batch up outgoing read-receipts to reduce federation traffic. (#4890, #4927)
• Add option to disable searching the user directory. (#4895)
• Add option to disable searching of local and remote public room lists. (#4896)
• Add ability for password providers to login/register a user via 3PID (email, phone). (#4931)

🔗Bugfixes

• Fix a bug where media with spaces in the name would get a corrupted name. (#2090)
• Fix attempting to paginate in rooms where server cannot see any events, to avoid unnecessarily pulling in lots of redacted events. (#4699)
• 'event_id' is now a required parameter in federated state requests, as per the matrix spec. (#4740)
• Fix tightloop over connecting to replication server. (#4749)
• Fix parsing of Content-Disposition headers on remote media requests and URL previews. (#4763)
• Fix incorrect log about not persisting duplicate state event. (#4776)
• Fix v4v6 option in HAProxy example config. Contributed by Flakebi. (#4790)
• Handle batch updates in worker replication protocol. (#4792)
• Fix bug where we didn't correctly throttle sending of USER_IP commands over replication. (#4818)
• Fix potential race in handling missing updates in device list updates. (#4829)
• Fix bug where synapse expected an un-specced prev_state field on state events. (#4837)
• Transfer a user's notification settings (push rules) on room upgrade. (#4838)
• fix test_auto_create_auto_join_where_no_consent. (#4886)
• Fix a bug where hs_disabled_message was sometimes not correctly enforced. (#4888)
• Fix bug in shutdown room admin API where it would fail if a user in the room hadn't consented to the privacy policy. (#4904)
• Fix bug where blocked world-readable rooms were still peekable. (#4908)

🔗Internal Changes

• Add a systemd setup that supports synapse workers. Contributed by Luca Corbatto. (#4662)
• Change from TravisCI to Buildkite for CI. (#4752)
• When presence is disabled don't send over replication. (#4757)
• Minor docstring fixes for MatrixFederationAgent. (#4765)
• Optimise EDU transmission for the federation_sender worker. (#4770)
• Update test_typing to use HomeserverTestCase. (#4771)
• Update URLs for riot.im icons and logos in the default notification templates. (#4779)
• Removed unnecessary $ from some federation endpoint path regexes. (#4794)
• Remove link to deleted title in README. (#4795)
• Clean up read-receipt handling. (#4797)
• Add some debug about processing read receipts. (#4798)
• Clean up some replication code. (#4799)
• Add some docstrings. (#4815)
• Add debug logger to try and track down #4422. (#4816)
• Make shutdown API send explanation message to room after users have been forced joined. (#4817)
• Update example_log_config.yaml. (#4820)
• Document the generate option for the docker image. (#4824)
• Fix check-newsfragment for debian-only changes. (#4825)
• Add some debug logging for device list updates to help with #4828. (#4828)
• Improve federation documentation, specifically .well-known support. Many thanks to @vaab. (#4832)
• Disable captcha registration by default in unit tests. (#4839)
• Add stuff back to the .gitignore. (#4843)
• Clarify what registration_shared_secret allows for. (#4844)
• Correctly log expected errors when fetching server keys. (#4847)
• Update install docs to explicitly state a full-chain (not just the top-level) TLS certificate must be provided to Synapse. This caused some people's Synapse ports to appear correct in a browser but still (rightfully so) upset the federation tester. (#4849)
• Move client read-receipt processing to federation sender worker. (#4852)
• Refactor federation TransactionQueue. (#4855)
• Comment out most options in the generated config. (#4863)
• Fix yaml library warnings by using safe_load. (#4869)
• Update Apache setup to remove location syntax. Thanks to @cwmke! (#4870)
• Reinstate test case that runs unit tests against oldest supported dependencies. (#4879)
• Update link to federation docs. (#4881)
• fix test_auto_create_auto_join_where_no_consent. (#4886)
• Use a regular HomeServerConfig object for unit tests rater than a Mock. (#4889)
• Add some notes about tuning postgres for larger deployments. (#4895)
• Add a config option for torture-testing worker replication. (#4902)
• Log requests which are simulated by the unit tests. (#4905)
• Allow newsfragments to end with exclamation marks. Exciting! (#4912)
• Refactor some more tests to use HomeserverTestCase. (#4913)
• Refactor out the state deltas portion of the user directory store and handler. (#4917)
• Fix nginx example in ACME doc. (#4923)
• Use an explicit dbname for postgres connections in the tests. (#4928)
• Fix ClientReplicationStreamProtocol.__str__(). (#4929)