Synapse v0.17.0 is finally here, which includes a couple of security fixes so please upgrade. Other notable new things are:

• A bunch of new admin APIs, including purging locally cached data (which has been long requested to help free up disk space). See the docs folder for more details.
• Device management APIs in preparation for end to end encryption.
• Better support for LDAP authentication, thanks to Martin Weinelt! (This may break existing LDAP configuration, see PR #843 for more details.)
• Lots and lots of bug fixes and various bits of performance work.

For a full list of everything that has changed see below or the release page.

I'd also like to thank Will Hunt, Martin Weinelt and Kent Shikama for their contributions!

🔗Changes in synapse v0.17.0 (2016-08-08)

This release contains significant security bug fixes regarding authenticating events received over federation. PLEASE UPGRADE.

This release changes the LDAP configuration format in a backwards incompatible way, see PR #843 for details.

Changes:

• Add federation /version API (PR #990)
• Make psutil dependency optional (PR #992)

Bug fixes:

• Fix URL preview API to exclude HTML comments in description (PR #988)
• Fix error handling of remote joins (PR #991)

🔗Changes in synapse v0.17.0-rc4 (2016-08-05)

Changes:

• Change the way we summarize URLs when previewing (PR #973)
• Add new /state_ids/ federation API (PR #979)
• Speed up processing of /state/ response (PR #986)

Bug fixes:

• Fix event persistence when event has already been partially persisted (PR #975, #983, #985)
• Fix port script to also copy across backfilled events (PR #982)

🔗Changes in synapse v0.17.0-rc3 (2016-08-02)

Changes:

• Forbid non-ASes from registering users whose names begin with '_' (PR #958)
• Add some basic admin API docs (PR #963)

Bug fixes:

• Send the correct host header when fetching keys (PR #941)
• Fix joining a room that has missing auth events (PR #964)
• Fix various push bugs (PR #966, #970)
• Fix adding emails on registration (PR #968)

🔗Changes in synapse v0.17.0-rc1 (2016-07-28)

This release changes the LDAP configuration format in a backwards incompatible way, see PR #843 for details.

Features:

• Add purge_media_cache admin API (PR #902)
• Add deactivate account admin API (PR #903)
• Add optional pepper to password hashing (PR #907, #910 by @KentShikama)
• Add an admin option to shared secret registration (breaks backwards compat) (PR #909)
• Add purge local room history API (PR #911, #923, #924)
• Add requestToken endpoints (PR #915)
• Add an /account/deactivate endpoint (PR #921)
• Add filter param to /messages. Add 'contains_url' to filter. (PR #922)
• Add device_id support to /login (PR #929)
• Add device_id support to /v2/register flow. (PR #937, #942)
• Add GET /devices endpoint (PR #939, #944)
• Add GET /device/{'{'}deviceId{'{'} (PR #943)
• Add update and delete APIs for devices (PR #949)

Changes:

• Rewrite LDAP Authentication against ldap3 (PR #843 by @mweinelt)
• Linearize some federation endpoints based on (origin, room_id) (PR #879)
• Remove the legacy v0 content upload API. (PR #888)
• Use similar naming we use in email notifs for push (PR #894)
• Optionally include password hash in createUser endpoint (PR #905 by @KentShikama)
• Use a query that postgresql optimises better for get_events_around (PR #906)
• Fall back to 'username' if 'user' is not given for appservice registration. (PR #927 by @Half-Shot)
• Add metrics for psutil derived memory usage (PR #936)
• Record device_id in client_ips (PR #938)
• Send the correct host header when fetching keys (PR #941)
• Log the hostname the reCAPTCHA was completed on (PR #946)
• Make the device id on e2e key upload optional (PR #956)
• Add r0.2.0 to the "supported versions" list (PR #960)
• Don't include name of room for invites in push (PR #961)

Bug fixes:

• Fix substitution failure in mail template (PR #887)
• Put most recent 20 messages in email notif (PR #892)
• Ensure that the guest user is in the database when upgrading accounts (PR #914)
• Fix various edge cases in auth handling (PR #919)
• Fix 500 ISE when sending alias event without a state_key (PR #925)
• Fix bug where we stored rejections in the state_group, persist all rejections (PR #948)
• Fix lack of check of if the user is banned when handling 3pid invites (PR #952)
• Fix a couple of bugs in the transaction and keyring code (PR #954, #955)

We've just released r0.2.0 of the Client-Server API specification. This release bundles up a number of clarifications and incremental improvements, as well as removing some outdated text relating to the pre-r0 event syncing APIs.

We've also taken the opportunity to make the license on the specifications explicit (we're using the Apache license), and have finally settled a long-running argument on what a user ID should look like.

As ever, the evolution of the spec has been helped tremendously by contributions and bug reports from the members of community - thanks to all those who have helped it on its way!

Vector Android has been added to the F-Droid catalogue. F-Droid is an installable catalogue of FOSS (Free and Open Source Software) applications for the Android platform. Many people have asked or requested Vector to be added to F-Droid, so we are happy to be able to announce its inclusion.

In order to meet the requirements for F-Droid, the build is not using GCM (Google Cloud Messaging) for notifications - instead it will keep syncing in the background. If you find that the ongoing background sync is using too much battery, you can add a delay or change the timeout of the sync - or even disable background sync completely - in the settings page.

Finally, if you have feedback on any of the Vector clients, there is #vector-feedback:matrix.org.

We've been made aware of a critical security issue in Synapse present in versions 0.12 through 0.16.1 inclusive which can allow users' accounts to be accessed by other unauthorized users on the same server. The issue was reported at 14:40 UTC on 2016-07-07 by Patrik Oldsberg at Ericsson (many thanks Patrik for discovering the issue and swiftly informing us). The source of the issue was identified, and a patch was created and distributed to package maintainers at roughly 16:30 UTC the same day.

We are not aware of any exploit in the wild, but it is critical for all synapse homeservers later than v0.12 to be upgraded immediately.

The github repository, as well as major 3rd party packages, have been updated with patched versions.

If an update is not available for your system you should manually apply the security patch that is included below. (This can be done by running patch -p1 sec.patch in the synapse source directory.)

The git commit SHA of the fix is: 067596d341a661e008195f7f3a6887ade7cafa32. This is included in release v0.16.1-r1.

Whilst Synapse (and Matrix) is still in beta, we nonetheless take such security issues seriously. In the coming days we will be reviewing how this vulnerability was introduced, and any steps that could have been taken to prevent the issue. We will also be auditing the remaining access control system to ensure there are no other existing issues. The full findings will be published when completed.

We apologise for the inconvenience of this emergency upgrade.

Thank you for your continued support, The Matrix Team

---

Various upgrade instructions:

• If you installed via git: git pull.
• If you installed via pip: pip install https://github.com/matrix-org/synapse/tarball/master
• If you installed via debian package: apt-get update; apt-get install matrix-synapse

After upgrade you will need to restart synapse.

Links to 3rd party packages: Arch: https://aur.archlinux.org/packages/matrix-synapse Fedora: https://obs.infoserver.lv/project/show/matrix-synapse

The patch against v0.16.x is: sec-0.16.patch, sec-0.16.patch.signed

The patch against v0.14.x is: sec-0.14.patch, sec-0.14.patch.signed

Signed announcement: fulldisclosure.signed

We have recently been made aware of a critical security issue in Synapse. Full disclosure of the issue and patch will be made at 2016-07-08 13:00 UTC. We are coordinating with package maintainers to ensure that patched versions of the packages will be available at that time.

If you run your own Synapse please be prepared to upgrade as soon as the patched versions are released.

Thank you for your time, patience and understanding while we resolve this issue, The Matrix Team

Signed pre-disclosure notice

Hi folks - another few months have gone by and once again the core Matrix team has ended up too busy hacking away on the final missing pieces of the Matrix jigsaw puzzle to have been properly updating the blog; sorry about this. The end is in sight for the current crunch however, and we expect to return to regular blog updates shortly! Meanwhile, rather than letting news stack up any further, here's a quick(?) attempt to summarise all the things which have been going on!

Continue reading…

The original promise of the Internet was to be an interoperable platform for distributing data. However, we have since increasingly seen our data fragmented and trapped in a number of proprietary silos. Matrix hopes to fix this by being a federated, open standard for data exchange that any service can use.

The Decentralized Web Summit is a meetup for anyone interested in building the Decentralized Web, which aims to make the Web open, secure and free of censorship by distributing data, processing, and hosting across millions of computers around the world, with no centralized control. It takes place at the Internet Archive, San Francisco, CA on Wednesday June 8 and Thursday June 9, 2016.

Matrix will be represented at the event, and we hope to also host a workshop or a talk about Matrix.

The meetup has a Slack room set up for pre-meetup conversations - you can also access this room via Matrix: #decentralizedweb-general:matrix.org

We are looking forward to interesting people and interesting conversations at the first Decentralized Web Summit!

Last week I went to Kamailio World 2016 in Berlin to meet fellow VoIP-developers and tell them all about Matrix. It's a fairly small conference, which is actually quite nice as it means you get to talk to almost everyone. A lot of people were interested in Matrix - both new and familiar faces - in fact, some of them heard about Matrix a year ago at Kamailio World 2015 and were interested in hearing what progress we've made since.

As always, Matrix participated in James Body's dangerous demos session - and I also gave a 30min talk on Matrix and recent updates to a full room on the first day of the conference.

Several people mentioned that Matrix could be interesting to their project, either as a glue between services, or for adding text-based chat to VoIP apps. I hope to see some of you in Matrix at some point - please join us in #matrix:matrix.org and say hi! It's also a good place to ask questions and discuss how Matrix can work with your project. Auf Wiedersehen!

Congratulations to Aviral Dasgupta and Will "Half-Shot" Hunt who will be working with Matrix for their Google Summer of Code projects!

As mentioned, picking two projects out of all our proposals was no easy task. However, we now look forward to getting started, and we are sure Aviral and Half-Shot will help make Matrix even better over the next few months!

Aviral will be developing a flexible plugin system to facilitate integrating various services such as github/trello/duckduckgo with Matrix. Meanwhile, Half-Shot will be looking at adding features on top of Matrix - infact, he's already built a MPD DJ bot and started working on a .NET SDK. Aviral too, has been committing various enhancements already.

According to Google's GSoC timeline we are currently in the "Community Bonding" phase, which lasts till May 22, 2016 - which is when the projects formally kick off.

We're looking forward to seeing what awesome things Aviral and Half-Shot come up with!

As previously announced, Matrix is participating in Google Summer of Code (GSoC) 2016. We have had a lot of interest: lots of people joining Matrix to talk to us about their project ideas and a total of 38 project proposals. We have even had some code contributions to our various projects from people who discovered Matrix via GSoC!

It's our first year as a GSoC mentoring organisation and we were only allocated two project slots. This means that we had the tough decision of choosing between some really good projects - and that means a lot of you who applied will unfortunately be left feeling disappointed. Selecting our two projects was very difficult, and we talked it over until we all agreed. Please remember that not being picked does not mean that your proposal was bad.

If you missed out on a GSoC slot this year, that doesn't have to stop you from contributing, either by hacking on your own project or contributing to an existing Matrix project. It's a great way to hone your programming skills and we'll be more than happy to help out and support you - find us in #matrix:matrix.org and #matrix-dev:matrix.org.

All the best from the Matrix team and good luck to everyone in their summer projects, whether GSoC or not!