Synapse 0.25 is out... as is Matrix Specification 0.3(!!!)

Hi all,

Today is a crazy release day here - not only do we have Synapse 0.25, but we've also made a formal release of the Matrix Specification (CS API) for the first time in 16 months!

Matrix CS API 0.3

Talking first about the spec update: the workflow of the Matrix spec is that new experimental features get added to an /unstable API prefix, and then whenever we release the Matrix spec, these get moved over to being part of the /r0 prefix (or whatever version we happen to be on).  We've been very constrained on manpower to work on the spec over the last ~18 months, but we've been keeping it up-to-date on a best effort basis, with a bit of help from the wider community.   As such, this latest release does not contain all the latest APIs (and certainly not experimental ones like Groups/Communities which are still evolving), but it does release all of the unstable ones which we've managed to document and which are considered stable enough to become part of the 'r0' prefix.  Going forwards, we're hoping that the wider community will help us fill in the remaining gaps (i.e. propose PRs against the matrix-org/matrix-doc repository to formalise the various spec drafts flying around the place) - and we're also hoping (if/when funding crisis is abated) to locate full-time folk to work on the spec.

The full changelog for 0.3 of the spec is:

• Breaking changes:
  • Change the rule kind of .m.rule.contains_display_name from underride to override. This works with all known clients which support push rules, but any other clients implementing the push rules API should be aware of this change. This makes it simple to mute rooms correctly in the API (#373).
  • Remove /tokenrefresh from the API (#395).
  • Remove requirement that tokens used in token-based login be macaroons (#395).
• Changes to the API which will be backwards-compatible for clients:
  • Add filename parameter to POST /_matrix/media/r0/upload (#364).
  • Document CAS-based client login and the use of m.login.token in /login (#367).
  • Make origin_server_ts a mandatory field of room events (#379).
  • Add top-level account_data key to the responses to GET /sync and GET /initialSync (#380).
  • Add is_direct flag to POST /createRoom and invite member event. Add 'Direct Messaging' module (#389).
  • Add contains_url option to RoomEventFilter (#390).
  • Add filter optional query param to /messages (#390).
  • Add 'Send-to-Device messaging' module (#386).
  • Add 'Device management' module (#402).
  • Require that User-Interactive auth fallback pages call window.postMessage to notify apps of completion (#398).
  • Add pagination and filter support to /publicRooms. Change response to omit fields rather than return null. Add estimate of total number of rooms in list. (#388).
  • Allow guest accounts to use a number of endpoints which are required for end-to-end encryption. (#751).
  • Add key distribution APIs, for use with end-to-end encryption. (#894).
  • Add m.room.pinned_events state event for rooms. (#1007).
  • Add mention of ability to send Access Token via an Authorization Header.
  • New endpoints:
    • GET /joined_rooms (#999).
    • GET /rooms/{'{'}roomId{'}'}/joined_members  (#999).
    • GET /account/whoami (#1063).
    • GET /media/{'{'}version{'}'}/preview_url  (#1064).
• Spec clarifications:
  • Add endpoints and logic for invites and third-party invites to the federation spec and update the JSON of the request sent by the identity server upon 3PID binding (#997)
  • Fix "membership" property on third-party invite upgrade example (#995)
  • Fix response format and 404 example for room alias lookup (#960)
  • Fix examples of m.room.member event and room state change, and added a clarification on the membership event sent upon profile update (#950).
  • Spell out the way that state is handled by POST /createRoom (#362).
  • Clarify the fields which are applicable to different types of push rule (#365).
  • A number of clarifications to authentication (#371).
  • Correct references to user_id which should have been sender (#376).
  • Correct inconsistent specification of redacted_because fields and their values (#378).
  • Mark required fields in response objects as such (#394).
  • Make m.notice description a bit harder in its phrasing to try to dissuade the same issues that occurred with IRC (#750).
  • GET /user/{'{'}userId{'}'}/filter/{'{'}filterId{'}'}  requires authentication (#1003).
  • Add some clarifying notes on the behaviour of rooms with no m.room.power_levels event (#1026).
  • Clarify the relationship between username and user_id in the /register API (#1032).
  • Clarify rate limiting and security for content repository. (#1064).

...and you can read the spec itself of course over at https://matrix.org/docs/spec.  It's worth noting that we have slightly bent the rules by including three very minor 'breaking changes' in 0.3, but all for features which to our knowledge nobody is depending on in the wild.  Technically this should mean bumping the major version prefix (i.e. moving to r1), but given how minor and nonimpacting these are we're turning a blind eye this time.

Meanwhile, Synapse 0.25 is out!

This is a medium-sized release; the main thing being to support configurable room visibility within groups (so that whenever you add a room to a group, you're not forced into sharing their existence with the general public, but can choose to just tell group members about them).  There's also a bunch of useful bug fixes and some performance improvements, including lots of contributions from the community this release (thank you!).  Full release notes are:

Changes in synapse v0.25.0 (2017-11-15)

Bug fixes:

• Fix port script (PR #2673)

Changes in synapse v0.25.0-rc1 (2017-11-14)

Features:

• Add is_public to groups table to allow for private groups (PR #2582)
• Add a route for determining who you are (PR #2668) Thanks to @turt2live!
• Add more features to the password providers (PR #2608, #2610, #2620, #2622, #2623, #2624, #2626, #2628, #2629)
• Add a hook for custom rest endpoints (PR #2627)
• Add API to update group room visibility (PR #2651)

Changes:

• Ignore tags when generating URL preview descriptions (PR #2576) Thanks to @maximevaillancourt!
• Register some /unstable endpoints in /r0 as well (PR #2579) Thanks to @krombel!
• Support /keys/upload on /r0 as well as /unstable (PR #2585)
• Front-end proxy: pass through auth header (PR #2586)
• Allow ASes to deactivate their own users (PR #2589)
• Remove refresh tokens (PR #2613)
• Automatically set default displayname on register (PR #2617)
• Log login requests (PR #2618)
• Always return is_public in the /groups/:group_id/rooms API (PR #2630)
• Avoid no-op media deletes (PR #2637) Thanks to @spantaleev!
• Fix various embarrassing typos around user_directory and add some doc. (PR #2643)
• Return whether a user is an admin within a group (PR #2647)
• Namespace visibility options for groups (PR #2657)
• Downcase UserIDs on registration (PR #2662)
• Cache failures when fetching URL previews (PR #2669)

Bug fixes:

• Fix port script (PR #2577)
• Fix error when running synapse with no logfile (PR #2581)
• Fix UI auth when deleting devices (PR #2591)
• Fix typo when checking if user is invited to group (PR #2599)
• Fix the port script to drop NUL values in all tables (PR #2611)
• Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services (PR #2631) Thanks to @xyzz!
• Fix updating rooms avatar/display name when modified by admin (PR #2636) Thanks to @farialima!
• Fix bug in state group storage (PR #2649)
• Fix 500 on invalid utf-8 in request (PR #2663)

Finally...

If you haven't noticed already, Riot/Web 0.13 is out today, as is Riot/iOS 0.6.2 and Riot/Android 0.7.4.  These contain massive improvements across the board - particularly mainstream Communities support at last on Riot/Web; CallKit/PushKit on Riot/iOS thanks to Denis Morozov (GSoC 2017 student for Matrix) and Share Extension on iOS thanks to Aram Sargsyan (also GSoC 2017 student!); and End-to-end Key Sharing on Riot/Android and a full rewrite of the VoIP calling subsystem on Android.

Rather than going on about it here, though, there's a full write-up over on the Riot Blog.

And so there you go - new releases for eeeeeeeeveryone!  Enjoy! :)

--Matthew, Amandine & the team.