Matthew Hodgson

160 posts tagged with "Matthew Hodgson" (See all Author)

Synapse 1.7.2 released

20.12.2019 00:00 — Releases Matthew Hodgson

Hi all,

We've just released Synapse 1.7.2 - a minor point release to address two regressions which snuck into 1.7.0 and 1.7.1. Sorry for the upgrade faff; hopefully we will be back to a saner release cadence shortly. Reminder that if you are on 1.7.0 or earlier you should upgrade asap as 1.7.1 contained security fixes.

Get the new release from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.

The changelog since 1.7.1 is:

Synapse 1.7.2 (2019-12-20)

This release fixes some regressions introduced in Synapse 1.7.0 and 1.7.1.

Bugfixes

  • Fix a regression introduced in Synapse 1.7.1 which caused errors when attempting to backfill rooms over federation. (#6576)
  • Fix a bug introduced in Synapse 1.7.0 which caused an error on startup when upgrading from versions before 1.3.0. (#6578)

Welcoming Mozilla to Matrix!

19.12.2019 00:00 — In the News Matthew Hodgson

Hi all,

We’re incredibly excited that Mozilla just announced that they’ve selected Matrix as the successor to IRC as the communication platform for the public Mozilla community!! This comes off the back of a formal 1-month trial in September to evaluate various options side by side, and now New Vector will be helping Mozilla get their homeserver up and running on the Modular.im hosting platform over the coming weeks - and federating openly with the rest of the open global Matrix network! :)

We have always been massive fans of Mozilla: they have been an excellent role model as champions of the open web, open standards, not to mention open source - and it’s fair to say that Mozilla has been a major inspiration to how Matrix has evolved (Riot aspires to be to Matrix what Firefox is to the Web: a flagship open source app which provides an accessible friendly interface into an open standard network). It’s very reassuring to see that Mozillians from the trial recognise the alignment and have converged on Matrix as the way forward - it’s a massive win for the open web and standards-based communication in general.

It’s worth noting that we’ve also always been massive fans of IRC, and Matrix is unashamedly derivative of IRC in capabilities and culture, while broadening the scope to decentralised synchronisation and relaying of any kind of data. For context, the genesis of the team which eventually spawned Matrix was on a student IRC server ~20 years ago - and subsequently everything we’ve worked on (up to Matrix) was coordinated exclusively through IRC. We even used to give conference talks on how to run your project/company off IRC. I can’t really overstate how fundamental IRC is to our history - and we still keep our private IRC network online for old time’s sake (albeit bridged to Matrix). The very first protocol bridge we built for Matrix back in 2015 was for IRC - and Moznet and Freenode were the first public bridges we turned on. As of right now, /stats u on Moznet says that there are 4950 connected users, of which 1724 (so 35%) are actually Matrix users connected via the Moznet bridge - effectively using Matrix as a big decentralised IRC bouncer in the sky.

All of this is to say is that we deeply understand how dependent Mozilla has been on IRC over the years, and that we built Matrix to be a worthy successor which tries to capture all the best bits of IRC while providing much richer primitives (E2E encryption, openly federated decentralised chatrooms, arbitrary data sync, HTTP API, VoIP, etc). It’s also worth noting that even though Moznet is being turned off, matrix-ircd exists as a very promising project that exposes any Matrix homeserver as an ircd - so for all you IRC die-hards, Moznet can absolutely live on in the afterlife! (matrix-ircd is still alpha right now, but it’s a relatively modest amount of Rust and PRs are very welcome - if you grok IRC it should be a really really fun project to contribute to).

In other news, the trial in September was an amazing opportunity to gather feedback first-hand from a wide range of Mozillians as they gave Riot and Matrix a spin, often for the first time - and it was a lot of fun to take that feedback and rapidly act on it to improve the app. For instance, having direct expert feedback on our screenreader support meant that we were able to radically improve our accessibility, and we’ve kept up the momentum on this since the trial (regardless of the outcome) with Mozilla & Riot devs hacking together with the aim of making Riot the most accessible communication app out there without exception. Huge thanks to Marco Zehe for all his guidance (and PRs), as well as the rest of #a11y:matrix.org!

Meanwhile, Riot’s UX continues to mature in general. One of our two primary projects right now is to improve First Time User Experience (FTUE) - i.e. making our UX as smooth and polished and predictable as possible, especially as seen by new users. This project had just kicked off in September as the Mozilla trial began, and some of the major improvements to the Room Directory and Room Creation flow which subsequently landed in Riot/Web 1.5 were prioritised directly based on Mozillian feedback. Since the trial we’ve been focusing more on our other primary project (getting E2E Encryption enabled by default), but we will be back on FTUE asap - particularly to incorporate all the feedback we anticipate as Mozilla goes live! We are absolutely determined for Riot to have as good if not better UX than the likes of Slack or Discord. New Vector is also actively hiring more designers to come work fulltime on Riot’s UI and UX as we shift Riot’s focus from being developer-led to design-led - if this sounds interesting, please get in touch! And finally, everything is of course open source and PRs are genuinely appreciated to keep Riot heading in the right direction (please just check first if they change the UI/UX).

Finally, in case you’re dreading having to use a graphical chat client like Riot, the Mozilla instance will of course be accessible to any Matrix client that floats your boat - for instance, weechat-matrix also got a spurt of development to support Mozilla IAM single-sign-on so that commandline junkies can get their fix too. (It’s worth noting that weechat-matrix really is an incredibly fully featured and usable client - complete with full end-to-end encryption support. If you haven’t tried it, you’re missing out).

So, to conclude: it has been indescribably valuable to have the expertise and enthusiasm of the Mozilla community in contributing feedback and fixes to Riot (and even building new Matrix bots!). Huge thanks to everyone who invested their time and energy participating in the trial and for their trust in concluding that Matrix was the way forward. We see this as a massive responsibility and honour to help power the wider Mozilla community, and we will do everything we can to make it as successful as conceivably possible :)

To the future of an open web, with even more open communications!

Matthew, Amandine & the whole Matrix & Riot team :)

P.S. we’ve come a long way since Matrix was first proposed for Mozilla :D

Avoiding unwelcome visitors on private Matrix servers

09.11.2019 00:00 — Privacy Matthew Hodgson

Hi all,

Over the course of today we've been made aware of folks port-scanning the general internet to discover private Matrix servers, looking for publicly visible room directories, and then trying to join rooms listed in them.

If you are running a Matrix server that is intended to be private, you must correctly configure your server to not expose its public room list to the general public - and also ensure that any sensitive rooms are invite-only (especially if the server is federated with the public Matrix network).

In Synapse, this means ensuring that the following options are set correctly in your homeserver.yaml:

# If set to 'false', requires authentication to access the server's public rooms
# directory through the client API. Defaults to 'true'.
#
#allow_public_rooms_without_auth: false

# If set to 'false', forbids any other homeserver to fetch the server's public
# rooms directory via federation. Defaults to 'true'.
#
#allow_public_rooms_over_federation: false

For private servers, you will almost certainly want to explicitly set these to false, meaning that the server's "public" room directory is hidden from the general internet and wider Matrix network.

You can test whether your room directory is visible to arbitrary Matrix clients on the general internet by viewing a URL like https://sandbox.modular.im/_matrix/client/r0/publicRooms (but for your server). If it gives a "Missing access token" error, you are okay.

You can test whether your room directory is visible to arbitrary Matrix servers on the general internet by loading Riot (or similar) on another server, and entering the target server's domain name into the room directory's server selection box. If you can't see any rooms, then are okay.

Relatedly, please ensure that any sensitive rooms are set to be "invite only" and room history is not world visible - particularly if your server is federated, or if it has public registration enabled. This stops random members of the public peeking into them (let alone joining them).

Relying on security-by-obscurity is a very bad idea: all it takes is for someone to scan the whole internet for Matrix servers, and then trying to join (say) #finance on each discovered domain (either by signing up on that server or by trying to join over federation) to cause problems.

Finally, if you don't want the general public reading your room directory, please also remember to turn off public registration on your homeserver. Otherwise even with the changes above, if randoms can sign up on your server to view & join rooms then all bets are off.

We'll be rethinking the security model of room directories in future (e.g. whether to default them to being only visible to registered users on the local server, or whether to replace per-server directories with per-community directories with finer grained access control, etc) - but until this is sorted, please heed this advice.

If you have concerns about randoms having managed to discover or join rooms which should have been private, please contact [email protected].

Fun and games with certificate transparency logs

06.11.2019 00:00 — Security Matthew Hodgson

Hi all,

This morning (06:11 UTC) it became apparent through mails to [email protected] that a security researcher was working through the TLS Certificate Transparency logs for *.matrix.org,*.riot.im and *.modular.im to identify and try to access non-public services run by New Vector (the company formed by the original Matrix team, which hosts *.matrix.org on behalf of the Matrix.org Foundation, and develops Riot and runs the https://modular.im hosting service).

Certificate Transparency (CT) is a feature of the TLS ecosystem which lets you see which public certificates have been created and signed by given authorities - intended to help identify and mitigate against malicious certificates. This means that the DNS name of any host with a dedicated public TLS certificate (i.e. not using a wildcard certificate) is visible to the general public.

In practice, this revealed a handful of internal-facing services using dedicated public TLS certificates which were accessible to the general internet - some of which should have been locked to be accessible only from our internal network.

Specifically:

  • kibana.ap-southeast-1.k8s.i.modular.im - a Kibana deployment for a new experimental Modular cluster which is being set up in SE Asia. The Kibana is in the middle of being deployed, and was exposed without authentication during deployment due to a firewall & config error. However, it is not a production system and carries no production traffic or user data (it was just being used for experimentation for hypothetical geography-specific Modular deployments). We firewalled this off at 07:53 UTC, and are doing analysis to confirm there was no further compromise, and will then rebuild the cluster (having fixed the firewall config error from repeating).
  • AWX deployments used by our internal Modular platform, which were behind authentication but should not be exposed to the public net.
  • Various semi-internal dev and testing services which should be IP-locked to our internal network (but are all locked behind authentication too).

Additionally, certain historical Modular homeservers & Riots (from before we switched to using wildcard certs, or where we’ve created a custom LetsEncrypt certificate for the server) are named in the CT logs - thus leaking the server’s name (which is typically public anyway in that server’s matrix IDs if the server is federated).

We’re working through the services whose names were exposed checking for any other issues, but other than the non-production SE Asia Kibana instance we are not aware of problems resulting from this activity.

Meanwhile, we’ll be ensuring that semi-internal services are only exposed on our internal network in future, and that Modular server names are not exposed by CT logs where possible.

TL;DR: You can list all the public non-wildcard TLS certs for a given domain by looking somewhere like https://crt.sh/?q=%25.matrix.org. This lets you find internal-sounding services to try to attack. In practice no production services were compromised, and most of our internal services are correctly firewalled from the public internet. However, we’re reviewing the IP locking for ones in the grey zone (and preventing the bug which caused an experimental Kibana to be exposed without auth).

We’d like to thank Linda Lapinlampi for notifying us about this. We’d also like to remind everyone that we operate a Security Disclosure Policy (SDP) and Hall of Fame at https://matrix.org/security-disclosure-policy/ which is designed to protect innocent users from being hurt by security issues - everyone: please consider disclosing issues responsibly to us as per the SDP.

Synapse 1.4.1 released

18.10.2019 00:00 — Releases Matthew Hodgson

Hi all,

We've released Synapse 1.4.1 as a small but important bugfix to 1.4.0.

This fixes a regression which crept in with our newly implemented "erase redacted data after N days" feature where some APIs would fail when hitting erased redactions - anyone on Synapse 1.4.0 will want to upgrade asap.

As always, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page

The changelog since 1.4.0 follows:

Synapse 1.4.1 (2019-10-18)

No changes since 1.4.1rc1.

Synapse 1.4.1rc1 (2019-10-17)

Bugfixes

  • Fix bug where redacted events were sometimes incorrectly censored in the database, breaking APIs that attempted to fetch such events. (#6185, 5b0e9948)

New Vector raises $8.5M to accelerate Matrix/Riot/Modular

10.10.2019 00:00 — In the News Matthew Hodgson

Hi all,

Massive news for the Matrix ecosystem today: New Vector (the startup which the Matrix core team formed to fund development in 2017) has raised an additional $8.5M of funding in order to speed up Riot/Matrix development and expand Matrix hosting via Modular.im!

The new funding comes in the form of a Series-A equity investment in New Vector from three of the top venture capital funds in London. The round is led by Notion - a fund set up by the founders of MessageLabs, who many will know as one of the leaders in secure hosted email services. Notion's long history with email means they immediately clocked the potential of Matrix's mission to build a new open global communication network - after all, Matrix aims to provide a worthy replacement to email (and the phone network, for that matter!). Joining Notion in the round is First Minute - a fund set up by the founders of Lastminute.com (arguably the UK's most famous original dotcom), and Dawn - one of the largest SaaS tech specialist funds in Europe (famous for backing iZettle, Mimecast, Neo4J and many more).

The last funding round in Jan 2017 from Status was instrumental in stabilising the big 1.0 release of Matrix and exiting beta in June; creating the Matrix.org Foundation as a neutral custodian for the standard; stabilising and optimising Synapse; redesigning Riot’s user interface; bringing in a full-time professional UI/UX designer to the team; supporting the huge amount of encryption work required to turn on E2EE by default (cross-signing, key backups, device verification, e2e search, the pantalaimon e2e daemon etc); creating RiotX/Android; and launching the Modular.im hosting platform.

With today’s new funding, the priorities for Matrix will be:

  • Turning on end-to-end encryption by default for DMs

  • Much better support for grouping rooms into Communities

  • More anti-abuse/anti-spam mechanisms

  • Shrinking Synapse (and/or finishing Dendrite)

  • Canonical DMs (having one DM per user, and have them feel clearly distinct from ‘rooms’)

  • Extensible Profiles

  • Decentralised accounts

  • Threading

  • ...and furthering development on P2P Matrix, so users can have full control of their communications without having to run or trust a server.

On the New Vector side, this funding will support:

  • A whole new wave of UX improvements to Riot (particularly around onboarding and first time user experience).

  • Making Modular hosting as polished and powerful as possible.

  • Creating a whole new set of next-generation Modular integrations.

While New Vector’s contributions to the Matrix ecosystem can’t be ignored, it’s important to remember that the Matrix protocol and specification itself is governed and controlled by the independent and neutral Matrix.org Foundation and its extensive governance processes. We set up the Foundation very deliberately to enforce the protocol's neutrality, formalise the project's mission, goals and values and hold true to them no matter what - specifically to protect the project from conflicts of interest with commercial Matrix endeavours, including New Vector.

That said, New Vector would not be taking money from any investors if they did not believe their goals are aligned with Matrix's. To clarify:

  • Matrix exists to create an open secure decentralised communication network and protocol for the benefit of all.
  • New Vector exists to help grow Matrix and be one of many successful companies in the Matrix ecosystem.
  • Tech VCs exist to invest their money in growing companies in order to get a return when the company IPOs or gets bought.

It turns out that these goals are not incompatible if one understands that the potential of the Matrix ecosystem is directly linked to its openness and size (hint: funding sources who didn’t understand this self-selected out ;). By funding Matrix development and helping the open ecosystem and public network grow, New Vector can go provide more Matrix hosting via Modular.im and more Government & Enterprise deployments via Vector.im. Critically, other companies can and do build on top of Matrix too - and frankly the more players there are, the more valuable the network, and the more value to be shared for everyone (including New Vector). This model worked relatively well for the Web, and we believe it'll work for Matrix too.

Update: the best way to gauge the investment in New Vector is to hear it first hand from the investors. Jos from Notion is leading the round, and has a fascinating blog post (written with zero input from Matrix or New Vector folks) to explain where the investors are coming from.

Update 2: More excellend first-hand analysis from Dan at Dawn Capital, who does a really deep dive into how they see Matrix and New Vector. Another must read.

In this case, all of New Vector's new investors have a background as respected tech entrepreneurs, and everyone involved categorically understands that Matrix itself is a neutral open source project, and the mission is to help build up the whole network to be as successful as possible rather than sabotage it by constraining it in any way.

All in all, it’s great news for the ecosystem: Matrix is 5 years old now, and while the project is growing faster than ever (over 300% more active users in the last year!) - it's fair to say that we haven't moved as fast as our mainstream competition - for instance, Slack is only a year older, and Discord is a year younger(!) Obviously much of this is due to Matrix being a completely different proposition: we've been creating an open spec; multiple client codebases; multiple server codebases; the bridges; a fault tolerant decentralised network - not to mention the complexities of decentralised E2E encryption. Based on comparing with our endeavours prior to Matrix, we estimate building this stuff in an open and decentralised manner takes roughly 6 times longer.

But the project is now in a position where the foundations are solid: the protocol is out of beta, reference servers and clients are production ready, and it’s more than time to make all of this mainstream. We have to redouble our focus on user experience and ensure that we compare favourably to today’s established alternatives while staying true to Matrix’s principles. Making sure there are Matrix apps out there which provide a credible alternative to with the likes of Slack and WhatsApp (until they eventually join Matrix, of course) is what will make the difference between Matrix being a cliquey FOSS curiosity versus really being the natural successor to today’s instant messaging, email and phone networks.

In the end; Matrix needs full-time contributors in order to continue to grow, and keeping New Vector funded is a very good way to achieve that (New Vector is hiring!). (That said, if any philanthropic billionaires are reading this, the Matrix.org Foundation is actively soliciting donations to improve Matrix independently of New Vector's efforts - particularly around the areas of countering online abuse and disinformation).

In the meantime, huge thanks to Jos at Notion for believing in Matrix and leading this funding round in New Vector - and huge thanks to the other investors who saw the potential! And most of all, thank you to all those supporting Matrix, whether by donating to the Foundation, promoting and using the protocol, or contributing code to the ecosystem. You are the ones keeping the dream alive :)

You can read things from the NV angle over at https://blog.vector.im/8-5m-to-accelerate-matrix/. We hope you’re as excited as we are to open a whole new chapter as Matrix picks up yet more momentum :D

-- Matthew, Amandine, and the whole Matrix team

Privacy improvements in Synapse 1.4 and Riot 1.4

27.09.2019 00:00 — Privacy Matthew Hodgson

Hi all,

Back in June we wrote about our plans to tighten up data privacy in Matrix after some areas for improvement were brought to our attention. To quickly recap: the primary concern was that the default config for Riot specifies identity servers and integration managers run by New Vector (the company which the original Matrix team set up to build Riot and fund Matrix dev) - and so folks using a standalone homeserver may end up using external services without realising it. There were some other legitimate issues raised too (e.g. contact information should be obfuscated when checking if your contacts are on Matrix; Riot defaulted to using Google for STUN (firewall detection) if no TURN server had been set up on their server; Synapse defaults to using matrix.org as a key notary server).

We’ve been working away at this fairly solidly over the last few months. Some of the simpler items shipped quickly (e.g. Riot/Web had a stupid bug where it kept incorrectly loading the integration manager; Riot/Android wasn’t clear enough about when contact discovery was happening; Riot/Web wasn’t clear enough about the fact device names are publicly visible; etc) - but other bits have turned out to be incredibly time-consuming to get right.

However, we’re in the process today of releasing Synapse 1.4.0 and Riot/Web 1.4.0 (it’s coincidence the version numbers have lined up!) which resolve the majority of the remaining issues. The main changes are as follows:

  1. Riot no longer automatically uses identity servers by default. Identity servers are only useful when inviting users by email address, or when discovering whether your contacts are on Matrix. Therefore, we now wait until the user tries to perform one of these operations before explaining that they need an identity server to do so, and we prompt them to select one if they want to proceed. This makes it abundantly clear that the user is connecting to an independent service, and why.

  2. Integration Managers and identity servers now have the ability to force users to accept terms of use before using them. This means they can explicitly spell out the data privacy & usage policy of the server as required by GDPR, and it should now be impossible for a user to use these services without realising it. This was particularly fun in the case of identity servers, which previously had no concept of users and so couldn’t track whether users had agreed to their terms & conditions or not… and because homeservers sometimes talk to the identity server on behalf of users rather than the user talking direct, the privacy policy flow gets even hairier. But it’s solved now, and a nice side-effect of this is that users can now explicitly select their Integration Manager in Riot, in case they want to use Dimension or similar rather than the default provided by Modular.

  3. Synapse no longer uses identity servers for verifying registrations or verifying password reset. Originally, Synapse made use of the fact that the Identity Service contains email/msisdn verification logic to handle identity verification in general on behalf of the homeserver. However, in retrospect this was a mistake: why should the entity running your identity server have the right to verify password resets or registration details on your homeserver? So, we have moved this logic into Synapse. This means Synapse 1.4.0 requires new configuration for email/msisdn verification to work - please see the upgrade notes for full details.

  4. Sydent now supports discovering contacts based on hashed identifiers. MSC2134 specifies entirely new IS APIs for discovering contacts using a hash of their identifier rather than directly exposing the raw identifiers being searched for. This is implemented in Riot/iOS and Riot/Android and should be in the next major release; Riot/Web 1.4.0 has it already.

  5. Synapse now warns in its logs if you are using matrix.org as a default trusted key server, in case you wish to use a different server to help discover other servers’ keys.

  6. Synapse now garbage collects redacted messages after N days (7 days by default). (It doesn’t yet garbage collect attachments referenced from redacted messages; we’re still working on that).

  7. Synapse now deletes account access data (IP addresses and User Agent) after N days (28 days by default) of a device being deleted.

  8. Riot warns before falling back to using STUN (and defaults to turn.matrix.org rather than stun.google.com) for firewall discovery (STUN) when placing VoIP calls, and makes it clear that this is an emergency fallback for misconfigured servers which are missing TURN support. (We originally deleted the fallback entirely, but this broke things for too many people, so we’ve kept it but warn instead).

All of this is implemented in Riot/Web 1.4.0 and Synapse 1.4.0. Riot/Web 1.4.0 shipped today (Fri Sept 27th) and we have a release candidate for Synapse 1.4 (1.4.0rc1) today which fully ship on Monday.

For full details please go check out the Riot 1.4.0 and Synapse 1.4.0 blog posts.

Riot/Mobile is following fast behind - most of the above has been implemented and everything should land in the next release. RiotX/Android doesn’t really have any changes to make given it hadn’t yet implemented Identity Service or Integration Manager APIs.

This has involved a surprisingly large amount of spec work; no fewer than 9 new Matrix Spec Changes (MSC) have been required as part of the project. In particular, this results in a massive update to the Identity Service API, which will be released very shortly with the new MSCs. You can see the upcoming changes on the unstable branch and compare with the previous 0.2.1 stable release, as well as checking the detailed MSCs as follows:

This said, there is still some work remaining for us to do here. The main things which haven’t made it into this release are:

  • Preferring to get server keys from the source server rather than the notary server by default (https://github.com/matrix-org/synapse/pull/6110). This almost made it in, but we need to test it more first - until then, your specified notary server will see roughly what servers your servers are trying to talk to. In future this will be mitigated properly by MSC1228 (removing mxids from events).
  • Configurable data retention periods for rooms. We are tantalisingly close with this - https://github.com/matrix-org/synapse/pull/5815 is an implementation that the French Govt deployment is using; we need to port it into mainline Synapse.
  • Authenticating access to the media repository - for now, we still rely on media IDs being almost impossible to guess to protect the data rather than authenticating the user.
  • Deleting items from the media repository - we still need to hook up deletion APIs.
  • Garbage collecting forgotten rooms. If everyone leaves & forgets a room, we should delete it from the DB.
  • Communicating erasure requests over federation

We’ll continue to work on these as part of our ongoing maintenance backlog.

Separately to the data privacy concerns, we’ve had a separate wave of feedback regarding how we handle GDPR Data Subject Access Requests (DSARs). Particularly: whether DSAR responses should contain solely the info your have directly keyed by the requesting Matrix ID - or if we should provide all the data “visible” to that ID (i.e. the history of the conversations they’ve been part of). We went and got professional legal advice on this one, and the conclusion is that we should keep our responses to DSARs as tightly scoped as possible. We updated Matrix.org’s privacy policy and DSAR tools to reflect the new legal input.

Finally, it’s really worth calling out the amount of effort that went into this project. Huge huge thanks to everyone involved (given it’s cut across pretty much every project & subteam we have working on the core of Matrix) who have soldiered through the backlog. We’ve been tracking progress using our feature-dashboard tool which summarises Github issues based on labels & issue lifecycle, and for better or worse it’s ended up being the biggest project board we’ve ever had. You can see the live data here (warning, it takes tens of seconds to spider Github to gather the data) - or, for posterity and ease of reference, I’ve included the current issue list below. The issues which are completed have “done” after them; the ones still in progress say “in progress”, and ones which haven’t started yet have nothing. We split the project into 3 phases - phases 1 and 2 represent the items needed to fully solve the privacy concerns, phase 3 is right now a mix of "nice to have" polish and some more speculative items. At this point we’ve effectively finished phase 1 on Synapse & Riot/Web, and Riot/Mobile is following close behind. We're continuing to work on phase 2, and we’ll work through phase 3 (where appropriate) as part of our general maintenance backlog.

I hope this gives suitable visibility on how we’re considering privacy; after all, Matrix is useless as an open communication protocol if the openness comes at the expense of user privacy. We’ll give another update once the remaining straggling issues are closed out; and meanwhile, now the bulk of the privacy work is out of the way on Riot/Web, we can finally get back to implementing the UI E2E cross-signing verification and improving first time user experience.

Thanks for your patience and understanding while we’ve sorted this stuff out; and thanks once again for flying Matrix :)

In the absence of comments on the current blog, please feel free to discuss over at HN, or alternatively come ask stuff in our AMA over at /r/privacy (starting ~5pm GMT+1 (UK) on Friday Sept 27th).

The Privacy Project Dashboard Of Doom

Tightening up privacy in Matrix

30.06.2019 00:00 — General Matthew Hodgson

Hi all,

A few weeks ago there was some discussion around the privacy of typical Matrix configurations, particularly how Riot's default config uses vector.im as an Identity Server (for discovering users on Matrix by their email address or phone number) and scalar.vector.im as an Integration Manager (i.e. the mechanism for adding hosted bots/bridges/widgets into rooms). This means that Riot, even if using a custom homeserver and running from a custom Riot deployment, will try to talk to *.vector.im (run by New Vector; the company formed by the core Matrix team in 2017) for some operations unless an alternative IS or IM has been specified in the config.

We haven't done as good a job at explaining this as we could have, and this blog post is a progress update on how we're fixing that and improving other privacy considerations in general.

Firstly, the reason Riot is configured like this is for the user's convenience: in general, we believe most users just want to discover other people on Matrix as easily as possible, and a logically-centralised server for looking up user matrix IDs by email/phone number (called third party IDs, or 3PIDs) is the only comprehensive way of doing so. Decentralising this data while protecting the privacy of the 3PIDs and their matrix IDs is a Hard Problem which we're unaware of anyone having solved yet. Alternatively, you could run a local identity server, but it will end up having to delegate to a centralised identity server anyway for IDs it has no other way to know about. Similarly, providing a default integration server that just works out of the box (rather than mandating the user configures their own) is a matter of trying to keep Riot's UX simple, especially when onboarding users, and especially given Riot's reputation for complexity at the best of times.

That said, the discussion highlighted some areas for improvement. Specifically:

  1. When doing work on making Matrix GDPR compliant back in May 2018, we set up a single privacy policy for the services we run, and got users to agree to it by locking them out of the matrix.org homeserver until they did. However, we missed that users not on the matrix.org homeserver might still be using our Identity Service (IS) & Integration Manager (IM) without accepting the privacy policy. Over the last few weeks we've been working on addressing this - it turns out that it's a pain to fix, given the Identity Service doesn't have the concept of users, so tracking which users have agreed to the policy policy or not means some fairly major changes. The current proposal is to change the Identity Service to use a form of OpenID to authenticate users against their homeserver in order to check they've accepted the IS's terms of use - see MSC2140 for the gory details.

Meanwhile, Riot is being updated to prompt the user to accept the IS & IM terms of use (if different to the HS's), and thus make it crystal clear to the user that they are using an IS & IM and that they have the option not to if desired - see https://github.com/vector-im/riot-web/issues/10167 and associated issues. This includes also explicitly prompting the user as to whether they want 3PIDs they provide at registration to be discoverable, as per https://github.com/vector-im/riot-web/issues/10091.

  1. Riot on iOS & Android gives the option of scanning your local addressbook to discover which of your contacts are on Matrix. The wording explaining this wasn't clear enough on Android - which we promptly fixed. Separately, the contact details sent to the server are currently not obfuscated. This is partially because we hadn't got to it, and partially because obfuscating them doesn't actually help much with privacy, given an attacker can just scan through possible obfuscated phone numbers and email addresses to deobfuscate them. However, we've been working through obfuscating the contact details anyway by hashing as per MSC2134, which has all the details. We're also adding an explicit lookup warning in Riot/Web, as per https://github.com/vector-im/riot-web/issues/10093.

  2. There was a bug where Riot/Web was querying the Integration Manager every time you opened a room, even if that room had no integrations (actually, it did it 3 times in a row). This got fixed and released in Riot/Web 1.2.2 back on June 19th.

  3. Matrix needs to authenticate whether events were actually sent by the server that claimed to send them. We do this by having servers sign their events when they create them, and publishing the public half of their signing keys for anyone to query. However, this poses problems if you receive an event which is signed by a server which isn't currently online. To solve this, we have the concept of trusted_key_servers (aka notary servers), which your server can query to see if they know about the missing server's keys. By default, matrix.org is configured as Synapse's trusted notary, but you can of course change this. If you choose an unreliable server as the notary (e.g. by not setting one at all) then there's a risk that you won't be able to look up signing keys, and a splitbrain will result where your server can't receive certain events, but other servers in the room can. This can then result in your server being unable to participate in the room entirely, if it's missing key events in the room's lifetime.

    Our plan here is to get rid of notaries entirely by changing how event signing works as per MSC1228, but this is going to take a while. Meanwhile we're going to check Synapse's code to ensure it doesn't talk to the notary server unnecessarily. (E.g. it should be caching the signing keys locally, and it should only use the notary server if the remote server is down.)

  4. When doing VoIP in Matrix, clients need to use a TURN server to discover their network conditions and perform firewall traversal. The TURN server should be specified by your homeserver (and each homeserver deployment should ideally include a TURN server). However, for users who have not configured a TURN server, Riot (on all 3 platforms) defaulted to use Google's public STUN service (stun.l.google.com). STUN is a subset of TURN which provides firewall discovery, but not traffic relaying. This slightly increased the chances of calls working for users without a proper TURN server, but not by much - and rather than fall back to Google, we've decided to simply remove it from Riot (e.g. https://github.com/matrix-org/matrix-ios-sdk/commit/24832a2b14fb72ae6f051d5aba40262d11eef65d). This means that VoIP might get less reliable for users who were relying on this fallback, but you really should be running your own TURN server anyway if you want VoIP to work reliably on your homeserver.

  5. We should make it clearer in Riot that device names are world-readable, and not just for the user's own personal reference. This is https://github.com/vector-im/riot-web/issues/10216

As you can see, much of the work on improving these issues is still in full swing, although some has already shipped. As should also be obvious, these issues are categorically not malicious: Matrix (and Riot) literally exists to give users full control and autonomy over their communication, and privacy is a key part of that. These are avoidable issues which can and will be solved. It's worth noting that we have to prioritise privacy issues alongside all the other development in Matrix however: there's no point in having excellent privacy if there are other bugs stopping the platform from being usable.

We'll do another blog post to confirm once most of the fixes here have landed - meanwhile, hopefully this post provides some useful visibility on how we're going about improving things.

Introducing Matrix 1.0 and the Matrix.org Foundation

11.06.2019 00:00 — General Matthew Hodgson

Matrix 1.0

Hi all,

We are very excited to announce the first fully stable release of the Matrix protocol and specification across all APIs - as well as the Synapse 1.0 reference implementation which implements the full Matrix 1.0 API surface.

This means that after just over 5 years since the initial work on Matrix began, we are proud to have finally exited beta!! This is the conclusion of the work which we announced at FOSDEM 2019 when we cut the first stable release of the Server-Server API and began the Synapse 0.99 release series in anticipation of releasing a 1.0.

Now, before you get too excited, it’s critical to understand that Matrix 1.0 is all about providing a stable, self-consistent, self-contained and secure version of the standard which anyone should be able to use to independently implement production-grade Matrix clients, servers, bots and bridges etc. It does not mean that all planned or possible features in Matrix are now specified and implemented, but that the most important core of the protocol is a well-defined stable platform for everyone to build on.

On the Synapse side, our focus has been exclusively on ensuring that Synapse correctly implements Matrix 1.0, to provide a stable and secure basis for participating in Matrix without risk of room corruption or other nastinesses. However, we have deliberately not focused on performance or features in the 1.0 release - so I’m afraid that synapse’s RAM footprint will not have got significantly better, and your favourite long-awaited features (automatically defragmenting rooms with lots of forward extremities, configurable message retention, admin management web-interface etc) have not yet landed. In other words, this is the opposite of the Riot 1.0 release (where the entire app was redesigned and radically improved its performance and UX) - instead, we have adopted the mantra to make it work, make it work right, and then (finally) make it fast. You can read the full release notes here. It’s also worth looking at the full changelog through the Synapse 0.99 release series to see the massive amount of polishing that’s been going on here.

All this means that the main headline features which land in Matrix 1.0 are vitally important but relatively dry:

  • Using X.509 certificates to trust servers rather than perspective notaries, to simplify and improve server-side trust. This is a breaking change across Matrix, and we’ve given the community several months now to ensure their homeservers run a valid TLS certificate. See MSC1711 for full details, and the 2 week warning we gave. As of ~9am UTC today, the matrix.org homeserver is running Synapse 1.0 and enforcing valid TLS certificates - the transition has begun (and so far we haven’t spotted any major breakage :). Thank you to everyone who got ready in advance!
  • Using .well-known URIs to discover servers, in case you can’t get a valid TLS certificate for your server’s domain.
  • Switching to room version 4 by default for creating new rooms. This fixes the most important defects that the core room algorithm has historically encountered, particularly:
  • Specifying the ability to upgrade between room versions
  • Full specification of lazy loading room members
  • Short Authentication String (Emoji!) interactive verification of E2EE devices
  • ...and lots and lots and lots of bugfixes and spec omission fixes.

That said, there is a lot of really exciting stuff in flight right now which sadly didn’t stabilise in time for Matrix 1.0, but will be landing as fast as we can finalise it now that 1.0 is at last out the door. This includes:

  • Editable messages! (These are in Synapse 1.0 and Riot already, but still stabilising so not enabled by default)
  • Reactions! (Similarly these are in develop)
  • Threading!! (We’ve planted the seeds for this in the new ‘aggregations’ support which powers edits & reactions - but full thread support is still a bit further out).
  • Cross-signed verification for end-to-end encryption (This is on a branch, but due to land any day now). We’ve also held off merging E2E backups into the Matrix 1.0 spec until cross-signing lands, given it may change the backup behaviour a bit. Once this is done, we can seriously talk about turning on E2E by default everywhere.
  • Live-tracking of room statistics and state in Synapse! (This is in Synapse 1.0 already if you check out the new room_stats and room_state tables, but we need to provide a nice admin interface for it).
  • Support for smaller footprint homeservers by reducing memory usage and stopping them from joining overly complex rooms.

Then stuff which we haven’t yet started, but is now unlocked by the 1.0 release:

  • Fixing extremities build-up (and so massively improving performance)
  • Rewriting Communities. Groups/Communities deliberately didn’t land in Matrix 1.0 as the current implementation has issues we want to fix first. MSC1772 has the details.
  • Rewritten room directory using the new room stats/state tables to be super-speedy.
  • Super-speedy incremental state resolution
  • Removing MXIDs from events (MSC1228)

Just to give a quick taster of the shape of things to come, here’s RiotX/Android, the all-new Riot client for Android, showing off Edits & Reactions in the wild…

...and here’s a screenshot of the final test jig for cross-signing devices in end-to-end encryption, so you will never have to manually verify new devices for a trusted user ever again! We demoed a very early version of this at FOSDEM, but this here is the testing harness for real deal, after several iterations of the spec and implementation to nail down the model. + means the device/user's cross-signing key is trusted, T means it's TOFU:

So, there you have it - welcome to Matrix 1.0, and we look forward to our backlog of feature work now landing!

Massive massive thanks to everyone who has stuck with the project over the years and helped support and grow Matrix - little did we think back in May 2014 that it’d take us this long to exit beta, but hopefully you’ll agree that it’s been worth it :)

Talking of which, we were looking through the photos we took from the first ever session hacking on Matrix back in May 2014…

Whiteboard 1

...suffice it to say that of the architectural options, we went with #3 in the end...

Whiteboard 2

...and that nowadays we actually know how power levels work, in excruciating and (hopefully) well-specified detail :)

There has been an absolutely enormous amount of work to pull Matrix 1.0 together - both on the spec side (thanks to the Spec Core Team for corralling proposals, and everyone who's contributed proposals, and particularly to Travis for editing it all) and the implementation side (thanks to the whole Synapse team for the tedious task of cleaning up everything that was needed for 1.0). And of course, huge thanks go to everyone who has been helping test and debug the Synapse 1.0 release candidates, or just supporting the project to get to this point :)

The Matrix.org Foundation

Finally, as promised, alongside Matrix 1.0, we are very happy to announce the official launch of the finalised Matrix.org Foundation!

This has been a long-running project to ensure that Matrix’s future is governed by a neutral non-profit custodian for the benefit of everyone in the Matrix ecosystem. We started the process nearly a year ago back with the initial proposal Towards Open Governance of Matrix.org, and then legally incorporated the Foundation in October, and published the final governance proposal in January.

As of today the Foundation is finalised and operational, and all the assets for Matrix.org have been transferred from New Vector (the startup we formed in 2017 to hire the core Matrix team). In fact you may already have seen Matrix.org Foundation notices popping up all over the Matrix codebase (as all of New Vector’s work on the public Matrix codebase for the foreseeable is being assigned to the Matrix.org Foundation).

Most importantly, we’re excited to introduce the Guardians of the Matrix.org Foundation. The Guardians are the legal directors of the non-profit Foundation, and are responsible for ensuring that the Foundation (and by extension the Spec Core Team) keeps on mission and neutrally protects the development of Matrix. Guardians are typically independent of the commercial Matrix ecosystem and may even not be members of today’s Matrix community, but are deeply aligned with the mission of the project. Guardians are selected to be respected and trusted by the wider community to uphold the guiding principles of the Foundation and keep the other Guardians honest.

We have started the Foundation with five Guardians - two being the original founders of the Matrix project (Matthew and Amandine) and three being entirely independent, thus ensuring the original Matrix team forms a minority which can be kept in check by the rest of the Guardians. The new Guardians are:

  • Prof. Jon Crowcroft - Marconi Professor of Communications Systems in the Computer Lab at the University of Cambridge and the Turing Institute. Jon is a pioneer in the field of decentralised communication, and a fellow of the Royal Society, the ACM, the British Computer Society, the Institution of Engineering and Technology, the Royal Academy of Engineering and the Institute of Electrical and Electronics Engineers.

    Jon is a global expert in decentralisation and data privacy, and is excellently placed to help ensure Matrix stays true to its ideals.

  • Ross Schulman - Ross is a senior counsel and senior policy technologist at New America’s Open Technology Institute, where he focuses on internet measurement, emerging technologies, surveillance, and decentralization. Prior to joining OTI, Ross worked for Google.

    Ross brings a unique perspective as a tech- and decentralisation-savvy lawyer to the Foundation, as well as being one of the first non-developers in the Matrix community to run his own homeserver. Ross has been known to walk around Mozfest clutching a battery-powered Synapse in a box, promoting decentralised communication for all.

  • Dr. Jutta Steiner - As co-founder and CEO of Parity Technologies, Jutta is dedicated to building a better internet - Web 3.0 - where users’ privacy & control come first. Parity Technologies is a leader in the blockchain space – known to many as the creator of one of the most popular Ethereum clients, it is also the creator of two ambitious new blockchain technlogies, Polkadot and Substrate, that make it easier to experiment and innovate on scalability, encryption and governance.

    Parity has been pioneering Matrix enterprise use since the moment they decided to rely on Matrix for their internal and external communication back in 2016, and now run their own high-volume deployment, with end-to-end encryption enabled by default. Jutta represents organisations who are professionally dependent on Matrix day-to-day, as well as bringing her unique experiences around decentralisation and ensuring that Web 3.0 will be a fair web for all.

We’d like to offer a very warm welcome to the new Guardians, and thank them profusely for giving up their time to join the Foundation and help ensure Matrix stays on course for the years to come.

For the full update on the Foundation, please check out the new website content at https://matrix.org/foundation which should tell you everything you could possibly want to know about the Foundation, the Guardians, the Foundation’s legal Articles of Association, and the day-to-day Rules which define the Open Governance process.

And finally…

Matrix 1.0 has been a bit of an epic to release, but puts us on a much stronger footing for the future.

However, it’s very unlikely that we’d have made it this far if most of the core dev team wasn’t able to work on Matrix as their day job. Right now we are actively looking for large-scale donations to the Matrix.org Foundation (and/or investment in New Vector) to ensure that the team can maintain as tight a focus on core Matrix work as possible, and to ensure the project realises its full potential. While Matrix is growing faster than ever, this perversely means we have more and more distractions - whether that’s keeping the Matrix.org server safe and operational, or handling support requests from the community, or helping new members of the ecosystem get up and running. If you would like Matrix to succeed, please get in touch if you’d like to sponsor work, prioritise features, get support contracts, or otherwise support the project. We’re particularly interested in sponsorship around decentralised reputation work (e.g. publishing a global room directory which users can filter based on their preferences).

Finally, huge thanks to everyone who has continued to support us through thick and thin on Patreon, Liberapay or other platforms. Every little helps here, both in terms of practically keeping the lights on, and also inspiring larger donations & financial support.

So: thank you once again for flying Matrix. We hope you enjoy 1.0, and we look forward to everything else landing on the horizon!

- Matthew, Amandine & the whole Matrix.org Team.

Final countdown to 1.0

24.05.2019 00:00 — General Matthew Hodgson

Hi all,

After lots of refinements, polishing and a few distractions we’re finally at the point of announcing the final timeline for both Matrix 1.0 and Synapse 1.0! We are targeting Monday 10th June as our release date - please consider this your two week warning!

This is the end game of the process we began back in February when we released the first stable release of the Server-Server API at FOSDEM, and started the Synapse 0.99 release series to prepare for 1.0.

Matrix 1.0 refers to the upcoming set of API releases which provides a matched set of stable and secure APIs across all of Matrix - at which point the project (at last) exits beta! In practice, this will be Client-Server API 0.5 (including final membership lazy loading, E2E backups and interactive verification and lots more), SS API 0.2 (including server key validity period fixes and associated v5 room protocol) and any other spec updates. The next 2 weeks will see a flurry of spec activity as we get everything together - you can see the full list and track the progress for the CS 0.5 spec release at https://github.com/matrix-org/matrix-doc/projects/2.

Meanwhile, Synapse 1.0 will be the reference implementation of Matrix 1.0, and so makes the changes required to implement Matrix 1.0 and close all currently known security and stability issues and thus exit beta. This means changing the default room protocol version used for new rooms to be v4, which includes the new state resolution algorithm, as well as collision-resistant event IDs, which are now formatted to be URL safe. Support for v4 rooms shipped in Synapse 0.99.5.1, so please upgrade asap to 0.99.5.1 before 1.0 is released to ease the transition.. Synapse 1.0 will also ship with support for the upcoming v5 room protocol (which enforces honouring server key validity periods), but this will not used as the default for new rooms until sufficient servers are speaking Matrix 1.0.

As part of the security work, Matrix 1.0 and Synapse 1.0 also contains a breaking change that requires a valid TLS certificate on the federation API endpoint. Servers that do not configure their certificate will no longer be able to federate post 1.0

You can check that your server has been correctly configured here and see here for more info on what you need to do. If in doubt head to #synapse:matrix.org.

We've been tracking readiness for the certificate change at https://arewereadyyet.com, at the time of writing 68% of active servers on the federation have valid certificates. We obviously would want that number to be higher, however since the largest installations have upgraded the total number of users who are ready for 1.0 stands at 96%, which we consider to be high enough to release 1.0.

This is not a drill, from here until 10th June we need everyone to not only ensure that their own server is ready, but also to encourage their fellow admins to update as well. With your help we can get everyone over the line!

Thanks everyone for your help to date, especially those providing support in #synapse:matrix.org.

Onwards!