Category – Releases
74 posts tagged with "Releases" (See all categories)

Dendrite 0.2.0 released

2020-10-20 — Releases — Matthew Hodgson

Hi all,

It's been over a week since our next-generation homeserver Dendrite entered beta, and it's been a wild rollercoaster ride as the team has been frantically zapping all the initial teething issues that came up - mostly around room federation getting 'stuck' due to needing to fix bugs in how room state is managed. Huge huge thanks to everyone who has spun up a Dendrite to experiment and report bugs!

We're now in an impressively better place, and it's feeling way more stable now (but please don't trust it with your data yet). So we've skipped 0.1.x and jumped straight to 0.2.0.

Now would be a great time for more intrepid explorers to try spinning up a server from https://github.com/matrix-org/dendrite and see how it feels - the more feedback the better. And if you got scared off by weird bugs in 0.1.0, now's the right time to try it again!

Full changelog follows:

Dendrite 0.2.0 (2020-10-20)

Important

  • This release makes breaking changes for polylith deployments, since they now use the multi-personality binary rather than separate binary files
    • Users of polylith deployments should revise their setups to use the new binary - see the Features section below
  • This release also makes breaking changes for Docker deployments, as are now publishing images to Docker Hub in separate repositories for monolith and polylith
    • New repositories are as follows: matrixdotorg/dendrite-monolith and matrixdotorg/dendrite-polylith
    • The new latest tag will be updated with the latest release, and new versioned tags, e.g. v0.2.0, will preserve specific release versions
    • Sample Compose configs have been updated - if you are running a Docker deployment, please review the changes
    • Images for the client API proxy and federation API proxy are no longer provided as they are unsupported - please use nginx (or another reverse proxy) instead

Features

  • Dendrite polylith deployments now use a special multi-personality binary, rather than separate binaries
    • This is cleaner, builds faster and simplifies deployment
    • The first command line argument states the component to run, e.g. ./dendrite-polylith-multi roomserver
  • Database migrations are now run at startup
  • Invalid UTF-8 in requests is now rejected (contributed by Pestdoktor)
  • Fully read markers are now implemented in the client API (contributed by Lesterpig)
  • Missing auth events are now retrieved from other servers in the room, rather than just the event origin
  • m.room.create events are now validated properly when processing a /send_join response
  • The roomserver now implements KindOld for handling historic events without them becoming forward extremity candidates, i.e. for backfilled or missing events

Fixes

  • State resolution v2 performance has been improved dramatically when dealing with large state sets
  • The roomserver no longer processes outlier events if they are already known
  • A SQLite locking issue in the previous events updater has been fixed
  • The client API /state endpoint now correctly returns state after the leave event, if the user has left the room
  • The client API /createRoom endpoint now sends cumulative state to the roomserver for the initial room events
  • The federation API /send endpoint now correctly requests the entire room state from the roomserver when needed
  • Some internal HTTP API paths have been fixed in the user API (contributed by S7evinK)
  • A race condition in the rate limiting code resulting in concurrent map writes has been fixed
  • Each component now correctly starts a consumer/producer connection in monolith mode (when using Kafka)
  • State resolution is no longer run for single trusted state snapshots that have been verified before
  • A crash when rolling back the transaction in the latest events updater has been fixed
  • Typing events are now ignored when the sender domain does not match the origin server
  • Duplicate redaction entries no longer result in database errors
  • Recursion has been removed from the code path for retrieving missing events
  • QueryMissingAuthPrevEvents now returns events that have no associated state as if they are missing
  • Signing key fetchers no longer ignore keys for the local domain, if retrieving a key that is not known in the local config
  • Federation timeouts have been adjusted so we don't give up on remote requests so quickly
  • create-account no longer relies on the device database (contributed by ThatNerdyPikachu)

Known issues

  • Old events can incorrectly appear in /sync as if they are new when retrieving missing events from federated servers, causing them to appear at the bottom of the timeline in clients
  • Memory can explode when catching up after a federation outage.

Synapse 1.21.2 released, and security advisory.

2020-10-15 — Releases, Security — Richard van der Hoff

Hi folks,

Today we have released Synapse 1.21.2, which fixes a couple of minor bugs that crept into the previous release. Full details are below.

Separately, we are advising any administrators who have not yet upgraded to Synapse 1.21.0 or later to do so as soon as possible. Previous versions of Synapse were vulnerable to a cross-site-scripting (XSS) attack; the bug was fixed in Synapse 1.21.0 with PR #8444.

The changelog for 1.21.2 is as follows:

Synapse 1.21.2 (2020-10-15)

Debian packages and Docker images have been rebuilt using the latest versions of dependency libraries, including authlib 0.15.1. Please see bugfixes below.

Bugfixes

  • Fix rare bug where sending an event would fail due to a racey assertion. (#8530)
  • An updated version of the authlib dependency is included in the Docker and Debian images to fix an issue using OpenID Connect. See #8534 for details.

Synapse 1.21.1 released

2020-10-13 — Releases — Neil Johnson

Synapse 1.21.1 has landed!

Highlights of 1.21.1 include:-

  • Add experimental support for sharding event persister. (#8294, #8387, #8396, #8419)

  • Add experimental prometheus metric to track numbers of "large" rooms for state resolutiom. (#8425)

  • Add prometheus metrics to track federation delays. (#8430)

  • Fix messages not being sent over federation until an event is sent into the same room. (#8230, #8247, #8258, #8272, #8322)

We've also made some improvements to SSO and added new admin APIs.

Get the new releases from any of the usual sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md. 1.21.1 is on github here

The changelog for 1.21.1 is as follows:

Synapse 1.21.1 (2020-10-13)

This release fixes a regression in v1.21.0 that prevented debian packages from being built.

It is otherwise identical to v1.21.0.

Synapse 1.21.0 (2020-10-12)

No significant changes since v1.21.0rc3.

As noted in v1.20.0, a future release will drop support for accessing Synapse's Admin API under the /_matrix/client/* endpoint prefixes. At that point, the Admin API will only be accessible under /_synapse/admin.

Synapse 1.21.0rc3 (2020-10-08)

Bugfixes

  • Fix duplication of events on high traffic servers, caused by PostgreSQL could not serialize access due to concurrent update errors. (#8456)

Internal Changes

  • Add Groovy Gorilla to the list of distributions we build .debs for. (#8475)

Synapse 1.21.0rc2 (2020-10-02)

Features

  • Convert additional templates from inline HTML to Jinja2 templates. (#8444)

Bugfixes

  • Fix a regression in v1.21.0rc1 which broke thumbnails of remote media. (#8438)
  • Do not expose the experimental uk.half-shot.msc2778.login.application_service flow in the login API, which caused a compatibility problem with Element iOS. (#8440)
  • Fix malformed log line in new federation "catch up" logic. (#8442)
  • Fix DB query on startup for negative streams which caused long start up times. Introduced in #8374. (#8447)

Synapse 1.21.0rc1 (2020-10-01)

Features

  • Require the user to confirm that their password should be reset after clicking the email confirmation link. (#8004)
  • Add an admin API GET /_synapse/admin/v1/event_reports to read entries of table event_reports. Contributed by @dklimpel. (#8217)
  • Consolidate the SSO error template across all configuration. (#8248, #8405)
  • Add a configuration option to specify a whitelist of domains that a user can be redirected to after validating their email or phone number. (#8275, #8417)
  • Add experimental support for sharding event persister. (#8294, #8387, #8396, #8419)
  • Add the room topic and avatar to the room details admin API. (#8305)
  • Add an admin API for querying rooms where a user is a member. Contributed by @dklimpel. (#8306)
  • Add uk.half-shot.msc2778.login.application_service login type to allow appservices to login. (#8320)
  • Add a configuration option that allows existing users to log in with OpenID Connect. Contributed by @BBBSnowball and @OmmyZhang. (#8345)
  • Add prometheus metrics for replication requests. (#8406)
  • Support passing additional single sign-on parameters to the client. (#8413)
  • Add experimental reporting of metrics on expensive rooms for state-resolution. (#8420)
  • Add experimental prometheus metric to track numbers of "large" rooms for state resolutiom. (#8425)
  • Add prometheus metrics to track federation delays. (#8430)

Bugfixes

  • Fix a bug in the media repository where remote thumbnails with the same size but different crop methods would overwrite each other. Contributed by @deepbluev7. (#7124)
  • Fix inconsistent handling of non-existent push rules, and stop tracking the enabled state of removed push rules. (#7796)
  • Fix a longstanding bug when storing a media file with an empty upload_name. (#7905)
  • Fix messages not being sent over federation until an event is sent into the same room. (#8230, #8247, #8258, #8272, #8322)
  • Fix a longstanding bug where files that could not be thumbnailed would result in an Internal Server Error. (#8236, #8435)
  • Upgrade minimum version of canonicaljson to version 1.4.0, to fix an unicode encoding issue. (#8262)
  • Fix longstanding bug which could lead to incomplete database upgrades on SQLite. (#8265)
  • Fix stack overflow when stderr is redirected to the logging system, and the logging system encounters an error. (#8268)
  • Fix a bug which cause the logging system to report errors, if DEBUG was enabled and no context filter was applied. (#8278)
  • Fix edge case where push could get delayed for a user until a later event was pushed. (#8287)
  • Fix fetching malformed events from remote servers. (#8324)
  • Fix UnboundLocalError from occuring when appservices send a malformed register request. (#8329)
  • Don't send push notifications to expired user accounts. (#8353)
  • Fix a regression in v1.19.0 with reactivating users through the admin API. (#8362)
  • Fix a bug where during device registration the length of the device name wasn't limited. (#8364)
  • Include guest_access in the fields that are checked for null bytes when updating room_stats_state. Broke in v1.7.2. (#8373)
  • Fix theoretical race condition where events are not sent down /sync if the synchrotron worker is restarted without restarting other workers. (#8374)
  • Fix a bug which could cause errors in rooms with malformed membership events, on servers using sqlite. (#8385)
  • Fix "Re-starting finished log context" warning when receiving an event we already had over federation. (#8398)
  • Fix incorrect handling of timeouts on outgoing HTTP requests. (#8400)
  • Fix a regression in v1.20.0 in the synapse_port_db script regarding the ui_auth_sessions_ips table. (#8410)
  • Remove unnecessary 3PID registration check when resetting password via an email address. Bug introduced in v0.34.0rc2. (#8414)

Improved Documentation

  • Add /_synapse/client to the reverse proxy documentation. (#8227)
  • Add note to the reverse proxy settings documentation about disabling Apache's mod_security2. Contributed by Julian Fietkau (@jfietkau). (#8375)
  • Improve description of server_name config option in homserver.yaml. (#8415)

Deprecations and Removals

  • Drop support for prometheus_client older than 0.4.0. (#8426)

Internal Changes

  • Fix tests on distros which disable TLSv1.0. Contributed by @danc86. (#8208)
  • Simplify the distributor code to avoid unnecessary work. (#8216)
  • Remove the populate_stats_process_rooms_2 background job and restore functionality to populate_stats_process_rooms. (#8243)
  • Clean up type hints for PaginationConfig. (#8250, #8282)
  • Track the latest event for every destination and room for catch-up after federation outage. (#8256)
  • Fix non-user visible bug in implementation of MultiWriterIdGenerator.get_current_token_for_writer. (#8257)
  • Switch to the JSON implementation from the standard library. (#8259)
  • Add type hints to synapse.util.async_helpers. (#8260)
  • Simplify tests that mock asynchronous functions. (#8261)
  • Add type hints to StreamToken and RoomStreamToken classes. (#8279)
  • Change StreamToken.room_key to be a RoomStreamToken instance. (#8281)
  • Refactor notifier code to correctly use the max event stream position. (#8288)
  • Use slotted classes where possible. (#8296)
  • Support testing the local Synapse checkout against the Complement homeserver test suite. (#8317)
  • Update outdated usages of metaclass to python 3 syntax. (#8326)
  • Move lint-related dependencies to package-extra field, update CONTRIBUTING.md to utilise this. (#8330, #8377)
  • Use the admin_patterns helper in additional locations. (#8331)
  • Fix test logging to allow braces in log output. (#8335)
  • Remove __future__ imports related to Python 2 compatibility. (#8337)
  • Simplify super() calls to Python 3 syntax. (#8344)
  • Fix bad merge from release-v1.20.0 branch to develop. (#8354)
  • Factor out a _send_dummy_event_for_room method. (#8370)
  • Improve logging of state resolution. (#8371)
  • Add type annotations to SimpleHttpClient. (#8372)
  • Refactor ID generators to use async with syntax. (#8383)
  • Add EventStreamPosition type. (#8388)
  • Create a mechanism for marking tests "logcontext clean". (#8399)
  • A pair of tiny cleanups in the federation request code. (#8401)
  • Add checks on startup that PostgreSQL sequences are consistent with their associated tables. (#8402)
  • Do not include appservice users when calculating the total MAU for a server. (#8404)
  • Typing fixes for synapse.handlers.federation. (#8422)
  • Various refactors to simplify stream token handling. (#8423)
  • Make stream token serializing/deserializing async. (#8427)

Dendrite is entering Beta!

2020-10-08 — Releases — Matthew Hodgson

Hi all,

We’re very excited to announce that Dendrite, the next-generation Matrix homeserver from the core Matrix team, is at last exiting alpha development and entering beta testing!

The path we’ve taken to get here has been quite a curious one, and it’s worth recapping to give context on why it’s taken reality a little while to catch up with the dream. :)

The Dendrite project has its roots in 2016 as Dendron: an attempt to write a next-generation homeserver in Golang rather than Python, in order to benefit from Go’s stronger typing, ease of profiling (no twisted stack-shredding via deferredInlineCallbacks), multithreading and faster GC performance. The idea for Dendron was to do a strangler pattern rewrite of Synapse - where we’d insert Dendron in front of Synapse as a load balancer, and incrementally replace Synapse’s API endpoints with ones implemented by Dendron.

However, as the project started to progress, it became clear that this was going to end up with many of Synapse’s architectural choices being baked into the project - particularly the DB schema and data flow architecture, such that the new endpoints could interoperate with the existing Python ones. We got as far as putting Dendron live on matrix.org and moving some of the login/registration APIs over to it… but then work fizzled out due to Synapse demanding more urgent attention as traffic grew on Matrix.org, combined with concerns about whether Dendron was the right approach in general.

So, towards the end of 2016 (after the rush to launch Vector Riot Element that summer), we went back to the drawing board to devise Dendrite—“Dendron done right!”—as opposed to Dendron, which in retrospect was Dendrite done wrong. ;) The new vision was:

  • Build a massively horizontally scalable architecture, such that large Matrix deployments like matrix.org and big government deployments could run smoothly without the constant scalability headaches we were seeing at the time with Synapse
  • Do so by splitting the server into well-defined microservice components, each of which could independently horizontally scale, each with its own DB (if desired)
  • Connect the components together with a set of append-only logs via Kafka or similar, easily letting components shard and maintain their databases from the logs, allowing rolling upgrades, possibly schema upgrades, and all sorts of other niceties. The logs effectively become a primary source of truth rather than putting all the onus on a massive monolithic ever-growing database

Rather than Dendron’s top-down approach, instead Dendrite started bottom-up with the very hardest bit: gomatrixserverlib, a standalone Go library implementing the state resolution algorithms and performing federation requests (such that it might also someday be used as a general purpose way to add Matrix federation support to an existing Go codebase).

Then we started building out the various components to implement the various services, starting with the roomserver (the service which models the history and state of one or more rooms in the server), then the syncserver (the service which implements the /sync API to let clients receive messages), etc. We even implemented a simplified in-memory version of Kafka named naffka—useful for glueing together the microservice components when running them all within a single binary.

Things were looking pretty positive by the summer of 2017: we had the server sending/receiving messages, federating with Synapse, and looking tantalisingly close:

However, we then hit three fairly major obstacles:

  • Matrix lost its funding
  • In the ensuing uncertainty, the two lead developers (Mjark & Kegan) went to work elsewhere
  • Meanwhile, Matrix uptake was starting to explode and Synapse was failing to scale to handle the traffic on matrix.org (and elsewhere)

At first, having formed what would become New Vector (now Element) to keep the rest of the core team hired, we pushed to see if we could get Dendrite finished fast enough to replace Synapse, with Erik & richvdh jumping over from Synapse to pick up the remaining work. However, it became clear that we urgently needed a quicker solution to address all the overloaded Synapses out there, and so they swung back to focus on improving Synapse (taking inspiration from some of the design of Dendrite - e.g. offloading endpoints onto worker processes connected via replication streams, and using OpenTracing to debug traffic as it flows over the various services).

At this point, Dendrite maintenance was in effect valiantly taken over by the community, with Brendan and later Anoa keeping the ball going in 2017, joined by APWhitehat in GSoC 2018 and cnly in GSoC 2019. The fact that Dendrite is now here today is thanks in no small part to their work to keep the project alive in its “wilderness years” between Sept 2017 and Dec 2019.

Meanwhile, it became clear that we were overdue getting Matrix itself out of beta - and the last thing we wanted to do was to split and dilute the implementation work of Matrix 1.0 over both Synapse and Dendrite - so we consciously made the decision to focus all our effort on Synapse for solving the remaining bugs and challenges.

Then, in July 2019, Matrix and Synapse exited beta, and we finally started to see light at the end of the tunnel. In October we started dusting off Dendrite again - looking to use it as a relatively simple and flexible codebase for experimenting with Peer-to-Peer Matrix, not least because being Go it can compile to WebAssembly and run clientside, and because even though Dendrite was originally built with massive deployments in mind, it turns out the elastic scaling means it can also scale down pretty small too—as a part of the iOS P2P demo, we’ve even ran full Dendrite homeservers on iPhones embedded into Element iOS! :)

In Dec 2019, we finally got to the point where Element could fund full-time dedicated development on Dendrite once again, with Neil Alexander joining the project and focusing fulltime on getting Dendrite out of alpha and getting it working for P2P and embedded usage (adding libp2p as a federation transport, and adding SQLite support) - and in Jan 2020 we got Dendrite successfully running clientside in a WASM service worker (just in time for FOSDEM!). Then, in Feb 2020, Kegan returned to the project to work fulltime on Dendrite - and the race began in earnest to get Dendrite ready for beta!

Here’s a pretty picture courtesy of GitHub to visualise the progress:

020-10-08-dendrite-contributors.png

Throughout 2020 there’s been a huge amount of stabilisation work and polish:

  • Refactoring much of Dendrite’s foundation to make the codebase more maintainable
  • Created all-new user server, key server, signing key server microservices
  • Moving some work from existing microservices (ultimately superseding the former currentstateserver, publicroomsapi and typingserver microservices altogether)
  • Developing new testing infrastructure:
    • Complement - our brand new Golang Matrix integration test harness
    • Are We Synapse Yet - an aggregator which parses sytest/complement output to compare how close Dendrite is to passing
  • All the Matrix 1.0 work - particularly state res v2 & room version support
  • Making it work with more P2P transports for all the exciting P2P experiments
  • Supporting backfill and fetching missing events
  • Fixing up SQLite support to make it work as a first class citizen (with shared storage code where we can!)
  • Supporting both sending and rejecting invites (even over federation)
  • E2E encryption support (one-time keys, device lists, send-to-device support)
  • Improved federation sender logic (resend retries, backoffs, blacklisting, metrics, resetting backoffs when receiving transactions)
  • Handling both inbound and outbound redactions
  • User interactive authentication (and implemented on various ‘sudo’ endpoints e.g. deleting devices and changing passwords)
  • Respecting server ACLs
  • Rejecting / soft-failing events properly
  • Support for database schema upgrades

... which brings us at last to the present day (Oct 2020), as we declare Dendrite sufficiently stable that we consider it ready for beta testing!

In practice, this means Dendrite is now ready for experimentation by adventurous Matrix sysadmins. It is NOT ready for production usage yet, but we need folks to test it and help us iron out the remaining bugs! Please do not trust it with sensitive data yet, and we don’t recommend trying to run it at scale yet as we haven’t done any serious optimisation work yet.

That said, we do provide the following guarantees:

  • We’re providing versioned releases from here on in, beginning with 0.1.0
  • We don’t expect any major breaking changes to the config or architecture before 1.0
  • Ready for early adopters to try running Dendrite without experiencing ~daily breaking churn
  • The database schema is now stable and will upgrade itself going forwards - your database should now be here to stay! (assuming we don’t hit any nasty data loss bugs during beta)

In terms of comparison with Synapse, the main things you should get excited about are:

  • Dendrite aims to provide an efficient, reliable and scalable alternative to Synapse:
    • Efficient: A small memory footprint with better baseline performance than an out-of-the-box Synapse
    • Reliable: Implements the Matrix specification as written, using the same test suite as Synapse as well as a brand new Go test suite
    • Scalable: can run on multiple machines and eventually scale to massive homeserver deployments
  • This means significantly less memory usage than Synapse (depends on joined rooms, often between 50MB - 400MB resident memory) - although we haven’t tuned this at all yet!
  • All-new database model, where every microservice instance has its own database tables, letting them scale arbitrarily wide
  • The ability to efficiently use all your available CPU cores without needing to split into separate processes, thanks to Go and our extensive use of goroutines. No more Python global interpreter lock! :)
  • Future experimental MSCs are likely to land in Dendrite before Synapse (e.g MSC2753 Peeking via /sync and MSC2444 Peeking over Federation are already being prototyped (#1370 and #1391) in Dendrite rather than Synapse!)

The provisos you should know about however are:

  • We’re not feature complete yet: sytest reports 56% CS API coverage and 77% Federation coverage. NB: these are always going to be underestimates of how much Dendrite actually performs due to how the tests are spread out, in actuality it’s likely more 70% CS, 95% Fed.
  • No read receipts, membership lazy-loading, presence, push notifications, search, event context, key backups, cross-signing. See changelog for full limitations.
  • Not battle-tested in the wild by many people (there are probably only ~10 dendrites on the open network today!) - so there’s likely to be a broad spectrum of bugs at first.
  • Clients that require more exotic features, like lazy loading, may not behave properly yet
  • Please use Postgres rather than SQLite wherever possible—it’s faster and has fewer issues regarding concurrency (some requests on SQLite Dendrites may 500 with ‘database is locked’ - though we’ve worked hard to eliminate most of these)
  • Dendrite can run in either “monolith” or “polylith” mode. In monolith, all the microservices are linked into a single binary - and we recommend running in this configuration wherever possible for now. Monolith mode is extremely capable as it is and has fewer moving parts for things to go wrong and will be the right choice for the majority of beta deployments!
  • Whilst Dendrite is nearly 100% federation compatible, there may still be situations where it will split-brain and disagree with the current room state that Synapse has calculated. We expect these issues to resolve as we get more user feedback.

Architecture-wise, this is what Dendrite looks like under the hood today:

2020-10-08-dendrite-arch.svg

To get up and running, please install Go and head on over to the Get Started guide at https://github.com/matrix-org/dendrite#get-started to join the fun :)

In terms of where we’re going next:

  • Read receipts. It’s a major missing feature and impacts UX significantly.
  • 100% Federation coverage (according to sytest). It’s crucial that Dendrite instances play nicely with other servers. This will be the best metric we have for asserting that we are just as capable as Synapse at the fed level.
  • Optimisation—Dendrite has not been optimised yet for speed or resource utilisation!
    • We plan to add benchmarks which will stress test different microservices in the presence of many different scaling factors (number of users, number of rooms, size of room, number of devices per user, number of sync requests, etc). This will hopefully allow us to identify early on bottlenecks and slow algorithms
    • Good old fashioned pprof with known slow scenarios to see what’s consuming CPU/memory and fixing issues ad-hoc (which we’ve already done a bit of pre-beta). This may involve adding additional in-memory caches, with a healthy respect for the complexities it may introduce (which Synapse has been bitten by)
  • We plan to add first class feature flag support for experimental MSCs—experimentation is one thing which makes Dendrite notably different from Synapse, and supporting it more thoroughly going forwards will be important. This may mean adding additional hooks; potentially a dedicated microservice to cleanly separate experiments, we don’t know yet
  • P2P work will continue with vigour now we have a working, featureful, and relatively stable HS to embed and play with

Longer term, it’s pretty hard to say right now when we expect to exit beta (it took Synapse 5 years to exit beta, after all ;) - but obviously we’ll need Dendrite to have parity with Synapse and have no known serious bugs.

Finally: you’re probably wondering what this means for Synapse. Synapse is here to stay - with tens of thousands of deployments around the world serving tens of millions of users. The majority of the core team is still focused on improving and optimising Synapse, and we’ll be keeping improving it for the foreseeable.

However, we’ll certainly be experimenting with new stuff on Dendrite first - whether that’s P2P, portable accounts, new-style communities, peeking etc. We expect Synapse to be the stable long-term-supported solution, while Dendrite (particularly while in beta) will be the more unstable and experimental platform. In the longer term we’ll provide ways of migrating from Synapse to Dendrite however (probably via portable accounts), and perhaps in future new deployments may choose to use Dendrite - a bit like you might choose to use nginx rather than Apache for a new web server these days. But this will be a long transition—meanwhile we expect to see more and more next-generation homeservers like Conduit, Mascarene or Construct coming of age too.

So, there you have it. If you’re an intrepid sysadmin please spin up a Dendrite and start filing bugs! :)

— Matthew, Neil Alexander, Kegan and the whole Matrix team.

Here’s the official changelog:

Client-Server API Features

Account registration and management

  • Registration: By password only.
  • Login: By password only. No fallback.
  • Logout: Yes.
  • Change password: Yes.
  • Link email/msisdn to account: No.
  • Deactivate account: Yes.
  • Check if username is available: Yes.
  • Account data: Yes.
  • OpenID: No.

Rooms

  • Room creation: Yes, including presets.
  • Joining rooms: Yes, including by alias or ?server_name=.
  • Event sending: Yes, including transaction IDs.
  • Aliases: Yes.
  • Published room directory: Yes.
  • Kicking users: Yes.
  • Banning users: Yes.
  • Inviting users: Yes, but not third-party invites.
  • Forgetting rooms: No.
  • Room versions: All (v1 - v6)
  • Tagging: Yes.

User management

  • User directory: Basic support.
  • Ignoring users: No.
  • Groups/Communities: No.

Device management

  • Creating devices: Yes.
  • Deleting devices: Yes.
  • Send-to-device messaging: Yes.

Sync

  • Filters: Timeline limit only. Rest unimplemented.
  • Deprecated /events and /initialSync: No.

Room events

  • Typing: Yes.
  • Receipts: No.
  • Read Markers: No.
  • Presence: No.
  • Content repository (attachments): Yes.
  • History visibility: No, defaults to joined.
  • Push notifications: No.
  • Event context: No.
  • Reporting content: No.

End-to-End Encryption

  • Uploading device keys: Yes.
  • Downloading device keys: Yes.
  • Claiming one-time keys: Yes.
  • Querying key changes: Yes.
  • Cross-Signing: No.

Misc

  • Server-side search: No.
  • Guest access: Partial.
  • Room previews: No, partial support for Peeking via MSC2753.
  • Third-Party networks: No.
  • Server notices: No.
  • Policy lists: No.

Federation Features

  • Querying keys (incl. notary): Yes.
  • Server ACLs: Yes.
  • Sending transactions: Yes.
  • Joining rooms: Yes.
  • Inviting to rooms: Yes, but not third-party invites.
  • Leaving rooms: Yes.
  • Content repository: Yes.
  • Backfilling / get_missing_events: Yes.
  • Retrieving state of the room (/state and /state_ids): Yes.
  • Public rooms: Yes.
  • Querying profile data: Yes.
  • Device management: Yes.
  • Send-to-Device messaging: Yes.
  • Querying/Claiming E2E Keys: Yes.
  • Typing: Yes.
  • Presence: No.
  • Receipts: No.
  • OpenID: No.

Synapse 1.20.0 released

2020-09-22 — Releases — Neil Johnson

Synapse 1.20.0 is here!

Highlights of 1.20.0 include:-

  • Shadow ban support.
  • Unread message counts in the sync response to help our client developers, this is a precursor to improving notification support.
  • No less than 28 async/await PRs, so we can finally share all the hard work.

Also take note that in a future release, we will be dropping support for accessing Synapse's Admin API using the /_matrix/client/* prefixes. More details follow in the changelog.

Get the new releases from any of the usual sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md. 1.20.0 is on github here, and 1.20.0rc4 is here.

The changelog for 1.20.0 is as follows:

Synapse 1.20.0 (2020-09-22)

No significant changes since v1.20.0rc5.

Removal warning

Historically, the Synapse Admin API has been accessible under the /_matrix/client/api/v1/admin, /_matrix/client/unstable/admin, /_matrix/client/r0/admin and /_synapse/admin prefixes. In a future release, we will be dropping support for accessing Synapse's Admin API using the /_matrix/client/* prefixes. This makes it easier for homeserver admins to lock down external access to the Admin API endpoints.

Synapse 1.20.0rc5 (2020-09-18)

In addition to the below, Synapse 1.20.0rc5 also includes the bug fix that was included in 1.19.3.

Features

  • Add flags to the /versions endpoint for whether new rooms default to using E2EE. (#8343)

Bugfixes

  • Fix rate limiting of federation /send requests. (#8342)
  • Fix a longstanding bug where back pagination over federation could get stuck if it failed to handle a received event. (#8349)

Internal Changes

Synapse 1.20.0rc4 (2020-09-16)

Synapse 1.20.0rc4 is identical to 1.20.0rc3, with the addition of the security fix that was included in 1.19.2.

Synapse 1.20.0rc3 (2020-09-11)

Bugfixes

  • Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. (#8291)

Synapse 1.20.0rc2 (2020-09-09)

Bugfixes

  • Fix a bug introduced in v1.20.0rc1 causing some features related to notifications to misbehave following the implementation of unread counts. (#8280)

Synapse 1.20.0rc1 (2020-09-08)

Removal warning

Some older clients used a disallowed character (:) in the client_secret parameter of various endpoints. The incorrect behaviour was allowed for backwards compatibility, but is now being removed from Synapse as most users have updated their client. Further context can be found at #6766.

Features

  • Add an endpoint to query your shared rooms with another user as an implementation of MSC2666. (#7785)
  • Iteratively encode JSON to avoid blocking the reactor. (#8013, #8116)
  • Add support for shadow-banning users (ignoring any message send requests). (#8034, #8092, #8095, #8142, #8152, #8157, #8158, #8176)
  • Use the default template file when its equivalent is not found in a custom template directory. (#8037, #8107, #8252)
  • Add unread messages count to sync responses, as specified in MSC2654. (#8059, #8254, #8270, #8274)
  • Optimise /federation/v1/user/devices/ API by only returning devices with encryption keys. (#8198)

Bugfixes

  • Fix a memory leak by limiting the length of time that messages will be queued for a remote server that has been unreachable. (#7864)
  • Fix Re-starting finished log context PUT-nnnn warning when event persistence failed. (#8081)
  • Synapse now correctly enforces the valid characters in the client_secret parameter used in various endpoints. (#8101)
  • Fix a bug introduced in v1.7.2 impacting message retention policies that would allow federated homeservers to dictate a retention period that's lower than the configured minimum allowed duration in the configuration file. (#8104)
  • Fix a long-standing bug where invalid JSON would be accepted by Synapse. (#8106)
  • Fix a bug introduced in Synapse v1.12.0 which could cause /sync requests to fail with a 404 if you had a very old outstanding room invite. (#8110)
  • Return a proper error code when the rooms of an invalid group are requested. (#8129)
  • Fix a bug which could cause a leaked postgres connection if synapse was set to daemonize. (#8131)
  • Clarify the error code if a user tries to register with a numeric ID. This bug was introduced in v1.15.0. (#8135)
  • Fix a bug where appservices with ratelimiting disabled would still be ratelimited when joining rooms. This bug was introduced in v1.19.0. (#8139)
  • Fix logging in via OpenID Connect with a provider that uses integer user IDs. (#8190)
  • Fix a longstanding bug where user directory updates could break when unexpected profile data was included in events. (#8223)
  • Fix a longstanding bug where stats updates could break when unexpected profile data was included in events. (#8226)
  • Fix slow start times for large servers by removing a table scan of the users table from startup code. (#8271)

Updates to the Docker image

  • Fix builds of the Docker image on non-x86 platforms. (#8144)
  • Added curl for healthcheck support and readme updates for the change. Contributed by @maquis196. (#8147)

Improved Documentation

  • Link to matrix-synapse-rest-password-provider in the password provider documentation. (#8111)
  • Updated documentation to note that Synapse does not follow HTTP 308 redirects due to an upstream library not supporting them. Contributed by Ryan Cole. (#8120)
  • Explain better what GDPR-erased means when deactivating a user. (#8189)

Internal Changes

  • Add filter name to the /users admin API, which filters by user ID or displayname. Contributed by Awesome Technologies Innovationslabor GmbH. (#7377, #8163)
  • Reduce run times of some unit tests by advancing the reactor a fewer number of times. (#7757)
  • Don't fail /submit_token requests on incorrect session ID if request_token_inhibit_3pid_errors is turned on. (#7991)
  • Convert various parts of the codebase to async/await. (#8071, #8072, #8074, #8075, #8076, #8087, #8100, #8119, #8121, #8133, #8156, #8162, #8166, #8168, #8173, #8191, #8192, #8193, #8194, #8195, #8197, #8199, #8200, #8201, #8202, #8207, #8213, #8214)
  • Remove some unused database functions. (#8085)
  • Add type hints to various parts of the codebase. (#8090, #8127, #8187, #8241, #8140, #8183, #8232, #8235, #8237, #8244)
  • Return the previous stream token if a non-member event is a duplicate. (#8093, #8112)
  • Separate get_current_token into two since there are two different use cases for it. (#8113)
  • Remove ChainedIdGenerator. (#8123)
  • Reduce the amount of whitespace in JSON stored and sent in responses. (#8124)
  • Update the test federation client to handle streaming responses. (#8130)
  • Micro-optimisations to get_auth_chain_ids. (#8132)
  • Refactor StreamIdGenerator and MultiWriterIdGenerator to have the same interface. (#8161)
  • Add functions to MultiWriterIdGen used by events stream. (#8164, #8179)
  • Fix tests that were broken due to the merge of 1.19.1. (#8167)
  • Make SlavedIdTracker.advance have the same interface as MultiWriterIDGenerator. (#8171)
  • Remove unused is_guest parameter from, and add safeguard to, MessageHandler.get_room_data. (#8174, #8181)
  • Standardize the mypy configuration. (#8175)
  • Refactor some of LoginRestServlet's helper methods, and move them to AuthHandler for easier reuse. (#8182)
  • Fix wait_for_stream_position to allow multiple waiters on same stream ID. (#8196)
  • Make MultiWriterIDGenerator work for streams that use negative values. (#8203)
  • Refactor queries for device keys and cross-signatures. (#8204, #8205, #8222, #8224, #8225, #8231, #8233, #8234)
  • Fix type hints for functions decorated with @cached. (#8240)
  • Remove obsolete order field from federation send queues. (#8245)
  • Stop sub-classing from object. (#8249)
  • Add more logging to debug slow startup. (#8264)
  • Do not attempt to upgrade database schema on worker processes. (#8266, #8276)

Synapse 1.19.2 released

2020-09-16 — Releases, Security — Neil Johnson

Synapse 1.19.2 is a security patch. All federating instances should upgrade immediately.

Today we are releasing Synapse 1.19.2, which is a security patch release containing a fix to encountering invalid events over federation. We are also putting out a fourth release candidate for the upcoming Synapse 1.20.0 release with the same fix.

The bug prevents affected Synapse instances from joining rooms with invalid events. Server administrators running federating instances are strongly encouraged to update as soon as possible.

Those on Synapse 1.19.1 or earlier should upgrade to Synapse 1.19.2, while those who are running a release candidate of Synapse 1.20.0 should upgrade to 1.20.0rc4.

Get the new releases from any of the usual sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md. 1.19.2 is on github here, and 1.20.0rc4 is here.

The changelog for 1.19.2 is as follows:

Synapse 1.19.2 (2020-09-16)

Due to the issue below server admins are encouraged to upgrade as soon as possible. Bugfixes

  • Fix joining rooms over federation that include malformed events. (#8324)
PreviousPage 1
NextPage 3