Good people, it's release time.
With the core team focusing on upcoming performance work and GDPR management tooling, v0.31.0 is most notable for improvements to system stats. Additionally, work continues on our py3 port and a host of small bug fixes and perf improvements.
Get it now from https://github.com/matrix-org/synapse/releases/tag/v0.31.0
Changes in synapse v0.31.0 (2018-06-06)
Most notable change from v0.30.0 is to switch to python prometheus library to improve system stats reporting. WARNING this changes a number of prometheus metrics in a backwards-incompatible manner. For more details, seedocs/metrics-howto.rst
Bug Fixes:
- Fix metric documentation tables (PR #3341)
- Fix LaterGauge error handling (694968f)
- Fix replication metrics (b7e7fd2)
Changes in synapse v0.31.0-rc1 (2018-06-04)
Features:
- Switch to the Python Prometheus library (PR #3256, #3274)
- Let users leave the server notice room after joining (PR #3287)
Changes:
- daily user type phone home stats (PR #3264)
- Use iter* methods for _filter_events_for_server (PR #3267)
- Docs on consent bits (PR #3268)
- Remove users from user directory on deactivate (PR #3277)
- Avoid sending consent notice to guest users (PR #3288)
- disable CPUMetrics if no /proc/self/stat (PR #3299)
- Add local and loopback IPv6 addresses to url_preview_ip_range_blacklist (PR #3312) Thanks to @thegcat!
- Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy (PR #3307)
- Add private IPv6 addresses to example config for url preview blacklist (PR #3317) Thanks to @thegcat!
- Reduce stuck read-receipts: ignore depth when updating (PR #3318)
- Put python's logs into Trial when running unit tests (PR #3319)
Changes, python 3 migration:
Bugs:
- Fix federation backfill bugs (PR #3261)
- federation: fix LaterGauge usage (PR #3328) Thanks to @intelfx!
It's release o'clock - GDPR time!!!!
v0.30.0 sees the introduction of Server Notices, which provides a channel whereby server administrators can send messages to users on the server, as well as Consent Management for tracking whether users have agreed to the terms and conditions set by the administrator of a server - and blocking access to the server until they have.
In conjunction these features support GDPR compliance in the form of providing a client agnostic means to contact users and ask for consent/agreement to a Privacy Notice.
For more information about our approach to GDPR compliance take a look here (although be aware that our position has evolved a bit; see the upcoming new privacy policy for the Matrix.org homeserver for details).
Additionally there are a host of bug fixes and refactors as well as an enhancement to our Dockerfile.
Get it now from https://github.com/matrix-org/synapse/releases/tag/v0.30.0
Changes in synapse v0.30.0 (2018-05-24)
'Server Notices' are a new feature introduced in Synapse 0.30. They provide a
channel whereby server administrators can send messages to users on the server.
They are used as part of communication of the server policies (see Consent Tracking),
however the intention is that they may also find a use for features such
as "Message of the day".
This feature is specific to Synapse, but uses standard Matrix communication mechanisms,
so should work with any Matrix client. For more details see here.
Further Server Notices/Consent Tracking Support:
- Allow overriding the server_notices user's avatar (PR #3273)
- Use the localpart in the consent uri (PR #3272)
- Support for putting %(consent_uri)s in messages (PR #3271)
- Block attempts to send server notices to remote users (PR #3270)
- Docs on consent bits (PR #3268)
Changes in synapse v0.30.0-rc1 (2018-05-23)
GDPR Support:
- ConsentResource to gather policy consent from users (PR #3213)
- Move RoomCreationHandler out of synapse.handlers.Handlers (PR #3225)
- Infrastructure for a server notices room (PR #3232)
- Send users a server notice about consent (PR #3236)
- Reject attempts to send event before privacy consent is given (PR #3257)
- Add a 'has_consented' template var to consent forms (PR #3262)
- Fix dependency on jinja2 (PR #3263)
Features:
Changes:
- Remove unused
update_external_syncs (PR #3233)
- Use stream rather depth ordering for push actions (PR #3212)
- Make purge_history operate on tokens (PR #3221)
- Don't support limitless pagination (PR #3265)
Bug Fixes:
- Fix logcontext resource usage tracking (PR #3258)
- Fix error in handling receipts (PR #3235)
- Stop the transaction cache caching failures (PR #3255)
It's release time people, not to be outdone by our friends on the Riot web team, Synapse v0.29.1 lands today.
v0.29.1 contains an officially supported docker image (many thanks to the contribution from @kaiyou), continued progress towards Python 3 (thanks to @NotAFile) - as well as a heap of refactorings and bug fixes.
Something worth noting is a potentially breaking change in the error code that /login returns in the Client Server API. Details follow, but the change closes a gap between Synapse behaviour and the spec.
We'd like to give huge thanks to Silvio Fricke and Andreas Peters for writing and maintaining Synapse's first Dockerfile, as well as allmende, jcgruenhage, ptman, and ilianaw for theirs! The new Dockerfile from kaiyou has ended up being merged into the main synapse tree and we're going to try to maintain it going forwards, but folks should use whichever one they prefer.
You can pick it up from https://github.com/matrix-org/synapse/releases/tag/v0.29.1 and thanks to everyone who tested the release candidate.
Changes in synapse v0.29.1 (2018-05-17)
Changes:
- Update docker documentation (PR #3222)
Changes in synapse v0.29.0 (2018-05-16)
No changes since v0.29.0-rc1
Changes in synapse v0.29.0-rc1 (2018-05-14)
Potentially breaking change:
- Make Client-Server API return 401 for invalid token (PR #3161).This changes the Client-server spec to return a 401 error code instead of 403 when the access token is unrecognised. This is the behaviour required by the specification, but some clients may be relying on the old, incorrect behaviour.Thanks to @NotAFile for fixing this.
Features:
Changes - General:
- nuke-room-from-db.sh: added postgresql option and help (PR #2337) Thanks to @rubo77!
- Part user from rooms on account deactivate (PR #3201)
- Make 'unexpected logging context' into warnings (PR #3007)
- Set Server header in SynapseRequest (PR #3208)
- remove duplicates from groups tables (PR #3129)
- Improve exception handling for background processes (PR #3138)
- Add missing consumeErrors to improve exception handling (PR #3139)
- reraise exceptions more carefully (PR #3142)
- Remove redundant call to preserve_fn (PR #3143)
- Trap exceptions thrown within run_in_background (PR #3144)
Changes - Refactors:
- Refactor /context to reuse pagination storage functions (PR #3193)
- Refactor recent events func to use pagination func (PR #3195)
- Refactor pagination DB API to return concrete type (PR #3196)
- Refactor get_recent_events_for_room return type (PR #3198)
- Refactor sync APIs to reuse pagination API (PR #3199)
- Remove unused code path from member change DB func (PR #3200)
- Refactor request handling wrappers (PR #3203)
- transaction_id, destination defined twice (PR #3209) Thanks to @damir-manapov!
- Refactor event storage to prepare for changes in state calculations (PR #3141)
- Set Server header in SynapseRequest (PR #3208)
- Use deferred.addTimeout instead of time_bound_deferred (PR #3127, #3178)
- Use run_in_background in preference to preserve_fn (PR #3140)
Changes - Python 3 migration:
Bug Fixes:
- synapse fails to start under Twisted >= 18.4 (PR #3157) Thanks to @Half-Shot!
- Fix a class of logcontext leaks (PR #3170)
- Fix a couple of logcontext leaks in unit tests (PR #3172)
- Fix logcontext leak in media repo (PR #3174)
- Escape label values in prometheus metrics (PR #3175, #3186)
- Fix 'Unhandled Error' logs with Twisted 18.4 (PR #3182) Thanks to @Half-Shot!
- Fix logcontext leaks in rate limiter (PR #3183)
- notifications: Convert next_token to string according to the spec (PR #3190) Thanks to @mujx!
- nuke-room-from-db.sh: fix deletion from search table (PR #3194) Thanks to @rubo77!
- add guard for None on purge_history api (PR #3160) Thanks to @krombel!
Well now, today sees the release of Synapse 0.28.0!
This release is particularly exciting as it's a major bump mainly thanks to lots and lots of contributions from the wider community - including support for running Synapse on PyPy (thanks Valodim) and lots of progress towards official Python3 support (thanks notafile)!! However, almost all the changes are under the hood (and some are quite major), so this is more a performance, bugfix and synapse internals release rather than adding many new APIs or features
As always, you can get it from https://github.com/matrix-org/synapse/releases/tag/v0.28.0 and thanks to everyone who tested the release candidates.
Changes in synapse v0.28.0 (2018-04-26)
Bug Fixes:
- Fix quarantine media admin API and search reindex (PR #3130)
- Fix media admin APIs (PR #3134)
Changes in synapse v0.28.0-rc1 (2018-04-24)
Minor performance improvement to federation sending and bug fixes.
(Note: This release does not include state resolutions discussed in matrix live)
Features:
- Add metrics for event processing lag (PR #3090)
- Add metrics for ResponseCache (PR #3092)
Changes:
Bug Fixes:
- Return 401 for invalid access_token on logout (PR #2938) Thanks to @dklug!
- Return a 404 rather than a 500 on rejoining empty rooms (PR #3080)
- fix federation_domain_whitelist (PR #3099)
- Avoid creating events with huge numbers of prev_events (PR #3113)
- Reject events which have lots of prev_events (PR #3118)
Today we release Synapse 0.27.3!
Hot on the heels of 0.27.2, notable changes include API support for joining groups/communities as well as a major bug fix (#3082), that is particularly important for those upgrading for the first time in a while. Also new metrics and phone home stats. Phone home stats include better visibility of system usage so we can have a better idea of how synapse's RAM and CPU is behaving in the wild. Also, recording the number of users on the server who have been using it for at least 30 days.
Phone home stats are entirely optional and can be enabled/disabled by setting "report_
stat
s" in homeserver.yaml.
Please consider enabling phone home stats if you currently have not done so - this data is really important to us in improving Matrix as a whole (and justifying future funding for Matrix.org).
As always, you can get it from https://github.com/matrix-org/synapse/releases/tag/v0.27.3 and thanks to everyone who tested the release candidates.
Changes in synapse v0.27.3 (2018-04-11)
Bug fixes:
- URL quote path segments over federation (#3082)
Changes in synapse v0.27.3-rc2 (2018-04-09)
v0.27.3-rc1 used a stale version of the develop branch so the changelog overstates
the functionality. v0.27.3-rc2 is up to date, rc1 should be ignored.
Changes in synapse v0.27.3-rc1 (2018-04-09)
Notable changes include API support for joinability of groups. Also new metrics
and phone home stats. Phone home stats include better visibility of system usage
so we can tweak synpase to work better for all users rather than our own experience
with matrix.org. Also, recording 'r30' stat which is the measure we use to track
overall growth of the Matrix ecosystem. It is defined as:-
Counts the number of native 30 day retained users, defined as:-
- Users who have created their accounts more than 30 days
- Where last seen at most 30 days ago
- Where account creation and last_seen are > 30 days"
Features:
- Add joinability for groups (PR #3045)
- Implement group join API (PR #3046)
- Add counter metrics for calculating state delta (PR #3033)
- R30 stats (PR #3041)
- Measure time it takes to calculate state group ID (PR #3043)
- Add basic performance statistics to phone home (PR #3044)
- Add response size metrics (PR #3071)
- phone home cache size configurations (PR #3063)
Changes:
- Add a blurb explaining the main synapse worker (PR #2886) Thanks to @turt2live!
- Replace old style error catching with 'as' keyword (PR #3000) Thanks to @NotAFile!
- Use .iter* to avoid copies in StateHandler (PR #3006)
- Linearize calls to _generate_user_id (PR #3029)
- Remove last usage of ujson (PR #3030)
- Use simplejson throughout (PR #3048)
- Use static JSONEncoders (PR #3049)
- Remove uses of events.content (PR #3060)
- Improve database cache performance (PR #3068)
Bug fixes:
- Add room_id to the response of
rooms/{'{'}roomId{'}'}/join (PR #2986) Thanks to @jplatte!
- Fix replication after switch to simplejson (PR #3015)
- Fix replication after switch to simplejson (PR #3015)
- 404 correctly on missing paths via NoResource (PR #3022)
- Fix error when claiming e2e keys from offline servers (PR #3034)
- fix tests/storage/test_user_directory.py (PR #3042)
- use PUT instead of POST for federating groups/m.join_policy (PR #3070) Thanks to @krombel!
- postgres port script: fix state_groups_pkey error (PR #3072)
We released Synapse v0.27.2 today (the first stable release in the 0.27.x series) - it contains loads of work since Synapse v0.26 back in January. The main highlights are:
- All the perf improvements which we've been landing as we race to keep the matrix.org homeserver in the face of ever-expanding traffic levels over the last few months
- Support for custom storage providers for media repository.
- Ability to limit the email addresses allowed to register on your HS, and ability to limit the homeservers your homeserver is allowed to federate with
- All new purge API - letting you purge history by date as well as by event (and having a nice new async way of doing it)
- Make search work again!!! (by switching from GIST to GIN indexes)
And a few release notes worth calling out:
- The common case for running Synapse is not to run separate workers, but for those that do, be aware that synctl no longer starts the main synapse when using
-a option with workers. A new worker file should be added with worker_app: synapse.app.homeserver.
- This release also begins the process of renaming a number of the metrics reported to prometheus. See
docs/metrics-howto.rst
Note that the v0.28.0 release will remove the deprecated metric names.
As always, you can get it from https://github.com/matrix-org/synapse/releases/tag/v0.27.2
thanks for flying Matrix!
Changes in synapse v0.27.2 (2018-03-26)
Bug fixes:
- Fix bug which broke TCP replication between workers (PR #3015)
Changes in synapse v0.27.1 (2018-03-26)
Meta release as v0.27.0 temporarily pointed to the wrong commit
Changes in synapse v0.27.0 (2018-03-26)
No changes since v0.27.0-rc2
Changes in synapse v0.27.0-rc2 (2018-03-19)
Pulls in v0.26.1
Bug fixes:
- Fix bug introduced in v0.27.0-rc1 that causes much increased memory usage in state cache (PR #3005)
Changes in synapse v0.27.0-rc1 (2018-03-14)
The common case for running Synapse is not to run separate workers, but for those that do, be aware that synctl no longer starts the main synapse when using -a option with workers. A new worker file should be added with worker_app: synapse.app.homeserver.
This release also begins the process of renaming a number of the metrics
reported to prometheus. See docs/metrics-howto.rst <docs/metrics-howto.rst#block-and-response-metrics-renamed-for-0-27-0>_.
Note that the v0.28.0 release will remove the deprecated metric names.
Features:
- Add ability for ASes to override message send time (PR #2754)
- Add support for custom storage providers for media repository (PR #2867, #2777, #2783, #2789, #2791, #2804, #2812, #2814, #2857, #2868, #2767)
- Add purge API features, see
docs/admin_api/purge_history_api.rst <docs/admin_api/purge_history_api.rst>_ for full details (PR #2858, #2867, #2882, #2946, #2962, #2943)
- Add support for whitelisting 3PIDs that users can register. (PR #2813)
- Add
/room/{'{'}id{'}'}/event/{'{'}id{'}'} API (PR #2766)
- Add an admin API to get all the media in a room (PR #2818) Thanks to @turt2live!
- Add
federation_domain_whitelist option (PR #2820, #2821)
Changes:
- Continue to factor out processing from main process and into worker processes. See updated
docs/workers.rst <docs/metrics-howto.rst>_ (PR #2892 - #2904, #2913, #2920 - #2926, #2947, #2847, #2854, #2872, #2873, #2874, #2928, #2929, #2934, #2856, #2976 - #2984, #2987 - #2989, #2991 - #2993, #2995, #2784)
- Ensure state cache is used when persisting events (PR #2864, #2871, #2802, #2835, #2836, #2841, #2842, #2849)
- Change the default config to bind on both IPv4 and IPv6 on all platforms (PR #2435) Thanks to @silkeh!
- No longer require a specific version of saml2 (PR #2695) Thanks to @okurz!
- Remove
verbosity/log_file from generated config (PR #2755)
- Add and improve metrics and logging (PR #2770, #2778, #2785, #2786, #2787, #2793, #2794, #2795, #2809, #2810, #2833, #2834, #2844, #2965, #2927, #2975, #2790, #2796, #2838)
- When using synctl with workers, don't start the main synapse automatically (PR #2774)
- Minor performance improvements (PR #2773, #2792)
- Use a connection pool for non-federation outbound connections (PR #2817)
- Make it possible to run unit tests against postgres (PR #2829)
- Update pynacl dependency to 1.2.1 or higher (PR #2888) Thanks to @bachp!
- Remove ability for AS users to call /events and /sync (PR #2948)
- Use bcrypt.checkpw (PR #2949) Thanks to @krombel!
Bug fixes:
- Fix broken
ldap_config config option (PR #2683) Thanks to @seckrv!
- Fix error message when user is not allowed to unban (PR #2761) Thanks to @turt2live!
- Fix publicised groups GET API (singular) over federation (PR #2772)
- Fix user directory when using
user_directory_search_all_users config option (PR #2803, #2831)
- Fix error on
/publicRooms when no rooms exist (PR #2827)
- Fix bug in quarantine_media (PR #2837)
- Fix url_previews when no Content-Type is returned from URL (PR #2845)
- Fix rare race in sync API when joining room (PR #2944)
- Fix slow event search, switch back from GIST to GIN indexes (PR #2769, #2848)
Hi folks,
Synapse 0.24 is out (currently at 0.24.1)! This is a pretty big release as it includes initial support for Groups, also known as Communities (UI for which is landing currently on Riot/Web and later Riot/Mobile). Groups let you associate together a set of users and rooms, letting you define a community - e.g. +matrix:matrix.org is the community of the core Matrix project itself (whose users are the core Matrix.org team, and whose public rooms are the rooms we officially manage/moderate as Matrix.org). We'll yell more about Groups once the UI is ready for action in the near future, but the good news is that Synapse should be ready to go (although the API is still fairly experimental and very much evolving).
Other stuff worth calling out in this release includes: massive performance improvements on receiving federation traffic (we now process federation traffic for different rooms in parallel); fixing a major cause of performance issues (caused when processing spurious events for rooms you've actually left); modularising and improving the the spamchecker; @room notification support; backup media repository support; and finally the ability to autojoin new users to a set of rooms on the server!
You can get the latest release from Github as usual; have fun - and thanks for flying Matrix :)
Changes in synapse v0.24.1 (2017-10-24)
Bug fixes:
- Fix updating group profiles over federation (PR #2567)
Changes in synapse v0.24.0 (2017-10-23)
No changes since v0.24.0-rc1
Changes in synapse v0.24.0-rc1 (2017-10-19)
Features:
- Add Group Server (PR #2352, #2363, #2374, #2377, #2378, #2382, #2410, #2426,
#2430, #2454, #2471, #2472, #2544)
- Add support for channel notifications (PR #2501)
- Add basic implementation of backup media store (PR #2538)
- Add config option to auto-join new users to rooms (PR #2545)
Changes:
- Make the spam checker a module (PR #2474)
- Delete expired url cache data (PR #2478)
- Ignore incoming events for rooms that we have left (PR #2490)
- Allow spam checker to reject invites too (PR #2492)
- Add room creation checks to spam checker (PR #2495)
- Spam checking: add the invitee to user_may_invite (PR #2502)
- Process events from federation for different rooms in parallel (PR #2520)
- Allow error strings from spam checker (PR #2531)
- Improve error handling for missing files in config (PR #2551)
Bug fixes:
- Fix handling SERVFAILs when doing AAAA lookups for federation (PR #2477)
- Fix incompatibility with newer versions of ujson (PR #2483) Thanks to
@jeremycline!
- Fix notification keywords that start/end with non-word chars (PR #2500)
- Fix stack overflow and logcontexts from linearizer (PR #2532)
- Fix 500 error when fields missing from power_levels event (PR #2552)
- Fix 500 error when we get an error handling a PDU (PR #2553)
