Category – General
128 posts tagged with "General" (See all categories)

Welcoming Rocket.Chat to Matrix!

2022-05-30 — General — Matthew Hodgson
Rocket.Chat ♥️ Matrix

Hi folks,

We just wanted to take a moment to welcome Rocket.Chat to Matrix, given the recent announcement that they are switching to using Matrix for standards-based interoperable federation! This is incredible news: Rocket.Chat is one of the leading open source collaboration platforms with over 12 million users, and they will all shortly have the option to natively interoperate with the wider Matrix network: the feature has already landed (in alpha) in Rocket.Chat 4.7.0!

We’d like to thank the whole Rocket.Chat team for putting their faith in Matrix and joining the network: the whole idea of Matrix is that by banding together, different independent organisations can build an open decentralised network which is far stronger and more vibrant than any closed communication platform. The more organisations that join Matrix, the more useful and valuable the network becomes for everyone, and the more momentum there is to further refine and improve the protocol. Our intention is that Matrix will grow into a massive open ecosystem and industry, akin to the open Web itself… and that every organisation participating, be that Rocket.Chat, Element, Gitter, Beeper, Famedly or anyone else will benefit from being part of it. We are stronger together!

Rocket.Chat’s implementation follows the “How do you make an existing chat system talk Matrix?” approach we published based on our experiences of linking Gitter into Matrix. Looking at the initial pull request, the implementation lets Rocket.Chat act as a Matrix Application Service, effectively acting as a bridge to talk to an appropriate Matrix homeserver. From chatting with the team, it sounds like next steps will involve adding in encryption via our upcoming matrix-sdk-crypto node bindings - and then looking at ways to transparently embed a homeserver like Dendrite, sharing data as much as possible between RC and Matrix, so Rocket.Chat deployments can transparently sprout Matrix interoperability without having to run a separate homeserver. Super exciting!

You can see a quick preview of a Rocket.Chat user chatting away with an Element user on matrix.org via Matrix here:



So, exciting times ahead - needless to say we’ll be doing everything we can to support Rocket.Chat and ensure their Matrix integration is a success. And at this rate, they might be distinctly ahead of the curve if they start shipping Dendrite! Meanwhile, we have to wonder who will be next? Nextcloud? Mattermost? Place your bets… ;)

--Matthew

Update

Aaron from Rocket.Chat just published an excellent guide & video tour for how to actually set up your Rocket.Chat instance with Dendrite to get talking Matrix!

Independent public audit of Vodozemac, a native Rust reference implementation of Matrix end-to-end encryption

2022-05-16 — General — Matthew Hodgson

Hi all,

It’s been quite a while since our last independent E2EE review, and so we’re incredibly proud to present an entirely new independent security audit of vodozemac, our next generation native Rust implementation of Matrix’s Olm and Megolm E2EE protocols. The audit has been conducted by Least Authority, who specialise in comprehensive audits of security-sensitive decentralized technologies - and we are very grateful to gematik, who kindly shared the costs of funding the audit as part of their commitment to Matrix for healthcare in Germany.

This audit was a bit of a whirlwind, as while we were clearly overdue an audit of Matrix’s E2EE implementations, we decided quite late in the day to focus on bringing vodozemac to auditable production quality rather than simply doing a refresh of the original libolm audit. However, we got there in time, thanks to a monumental sprint from Damir and Denis over Christmas. The reason we went this route is that vodozemac is an enormous step change forwards in quality over libolm, and vodozemac is now the reference Matrix E2EE implementation going forward. Just as libolm went live with NCC’s security review back in 2016, similarly we’re kicking off the first stable release of vodozemac today with Least Authority’s audit. In fact, vodozemac just shipped as the default E2EE library in matrix-rust-sdk 0.5, released at the end of last week!

The main advantages of vodozemac over libolm include:

  • Native Rust - type, memory and thread safety is preserved both internally, and within the context of larger Rust programs (such as those building on matrix-rust-sdk). This is particularly important given the memory bugs which libolm sprouted, despite our best efforts to the contrary.
  • Performance - vodozemac benchmarks roughly 5-6x faster than libolm on typical hardware
  • Better primitives - vodozemac is built on the best practice cryptographic primitives from the Rust community, rather than the generic Brad Conte primitives used by libolm.

Also, we’ve finally fixed one of the biggest problems with libolm - which was that the hardest bit of implementing E2EE in Matrix wasn’t necessarily the encryption protocol implementation itself, but how you glue it into your Matrix client’s SDK. It turns out ~80% of the code and complexity needed to securely implement encryption ends up being in the SDK rather than the Olm implementation - and each client SDK ended up implementing its own independent state machine and glue for E2EE, leading to a needlessly large attack & bug surface.

To address this problem, vodozemac is designed to plug into matrix-sdk-crypto - an SDK-independent Rust library which abstracts away the complexities and risks of implementing E2EE, designed to plug into existing SDKs in any language. For instance, Element Android already supports delegating its encryption to matrix-sdk-crypto; Element iOS got this working too last week, and we’re hard at work adding it to Element Web too. (This set of projects is codenamed Element R). Meanwhile, Element X (the project to switch Element iOS and Element Android to use matrix-rust-sdk entirely) obviously benefits from it too, as matrix-rust-sdk now leans on matrix-sdk-crypto for its encryption.

Therefore we highly recommend that developers using libolm migrate over to vodozemac - ideally by switching to matrix-sdk-crypto as a way of managing the interface between Matrix and the E2EE implementation. Vodozemac does also provides a similar API to libolm with bindings for JS and Python (and C++ in progress) if you want to link directly against it - e.g. if you’re using libolm for something other than Matrix, for example XMPP, ActivityPub or Jitsi. We’ll continue to support and maintain libolm for now though, at least until the majority of folks have switched to vodozemac.

In terms of the audit itself - we recommend reading it yourself, but the main takeaway is that Least Authority identified 10 valid concerns, of which we addressed 8 during the audit process. The remaining two are valid but lower priority, and we’ll fix them as part of our maintenance backlog. All the issues identified are excellent valid points, and we’re very glad that Least Authority have added huge value here by highlighting some subtle gotchas which we’d missed. (If you write Rust, you’ll particularly want to check out their zeroisation comments).

So: exciting times! Vodozemac should be landing in a Matrix client near you in the near future - we’ll yell about it loudly once Element switches over. In the meantime, if you have any questions, please head over to #e2ee:matrix.org.

Thanks again to gematik for helping fund the audit, and to Least Authority for doing an excellent job - and being patient and accommodating beyond the call of duty when we suddenly switched the scope from libolm to vodozemac at the last minute ;)

Next up: we’re going to get the Rust matrix-sdk-crypto independently audited (once this burndown is complete) so that everyone using the matrix-sdk-crypto state machine for Matrix E2EE can have some independent reassurance too - a huge step forward from the wild west of E2EE SDK implementations today!

– Matthew

Technical FAQ on the Digital Markets Act

2022-03-30 — General — Matthew Hodgson

Hi all,

We've been flooded with questions about the DMA since it was announced last week, and have spotted some of the gatekeepers jumping to the wrong conclusions about what it might entail. Just in case you don't want to wade through yesterday's sprawling blog post, we've put together a quick FAQ to cover the most important points based on our understanding.

What does DMA mean for the gatekeepers?

The gatekeepers will have to open and document their existing APIs, so that alternative clients and/or bridges can connect to their service. The DMA requires that the APIs must expose the same level of privacy for remote users as for local users. So, if their service is end-to-end-encrypted (E2EE), the APIs must expose the same E2EE semantics (e.g. so that an alternative client connecting would have to implement E2EE too). For E2EE-capable APIs to work, the gatekeeper will likely have to model remote users as if they were local in their system. In the short term (one year horizon) this applies only to 1:1 chats and file transfers. In the long term (three year horizon) this applies to group chats and voip calls/conferences too.

Who counts as a “gatekeeper”?

The DMA defines any tech company worth over €75B or with over €7.5B of turnover as a gatekeeper, who must open their communication APIs. This means only the tech giants are in scope (e.g. as of today that includes Meta, Apple, Google, Microsoft, Salesforce/Slack - not Signal, Telegram, Discord, Twitter).

Does this mean the gatekeepers are being forced to implement an open standard such as Matrix or XMPP?

No. They can keep their existing implementations and APIs. For interoperability with other service providers, they will need to use a bridge (which could bridge via a common language such as Matrix or XMPP).

Are bridges secure?

If the service lacks end-to-end-encryption (Slack, Teams, Google Chat, non-secret chats on Facebook Messenger, Instagram, Google Messages etc) then the bridge does not reduce security or privacy, beyond the fact that bridged conversations by definition will be visible to the bridge and to the service you are interoperating with.

If the service has E2EE (WhatsApp, iMessage, secret chats on Messenger) then the bridge will necessarily have to decrypt (and reencrypt, where possible) the data when passing it to the other service. This means the conversation is no longer E2EE, and thus less secure (the bridge could be compromised and inspect or reroute their messages) - and so gatekeepers must warn the user that their conversation is leaving their platform and is no longer E2EE with something like this:


Why is the DMA good?

The upside is that the user has the freedom to use an infinite number of services (bots, virtual assistants, CRMs, translation services, etc) as well as speak to any other user in the world, regardless what platform they use.

It also puts much-needed pressure on the gatekeepers to innovate and differentiate rather than rely on their network effects to attract new users - creating a much more vibrant, open, competitive marketplace for users.

See "What's driven the DMA?" for more details.

If the DMA requires that remote users have the same security as local users, how can bridges work?

The DMA requires that the APIs expose the same level of security as for local users - ie E2EE must be exposed. If the users in a conversation choose to use a bridge and thus reencrypt the messages, then it is their choice to tradeoff encryption in favour of interoperability for a given conversation.

Does this undermine the gatekeepers’ current encryption?

Absolutely not. Users talking to other users within the same E2EE-capable gatekeeper will still be E2EE (assuming the gatekeeper doesn’t pull that rug from under its users) - and in fact it gives the gatekeepers an excellent way to advertise the selling point that E2EE is only guaranteed when you speak to other users on the same platform.

But why do we need bridges? If everyone spoke a common protocol, you wouldn’t ever have to decrypt messages to convert them between protocols.

Practically speaking, we don’t expect the gatekeepers to throw away their existing stacks (or implement multihead messengers that also speak open protocols). It’s true that if they natively spoke Matrix or XMPP then the reencryption problem would go away, but it’s more realistic to focus on opening the existing APIs than interpret the legislation as a mandate to speak Matrix. Perhaps in future players will adopt Matrix of their own volition.

Where do these bridges come from?

There is already a vibrant community of developers who build unofficial bridges to the gatekeepers - eg Element, Beeper and hundreds of open source developers in the Matrix and XMPP communities. Historically these bridges have been hampered by having to use unofficial and private APIs, making them a second class citizen - but with open documented APIs guaranteed by the DMA we eagerly anticipate an explosion of high quality transparent bridges which will be invisible to the end user.

Can you run E2EE bridges clientside to make them safer?

Maybe. For instance, current iMessage bridges work by running iMessage on a local iPhone or Mac and then reencrypting the messages there for interoperability. Given the messages are already exposed on the client anyway, this means that E2EE is not broken - and avoids decrypting them on a server. There is lots of development in this space currently, and with open APIs guaranteed by DMA the pace should speed up significantly.

How can you tell what service you should use to talk to a given remote user?

For 1:1 chats this is easy: you can simply ask the user which service they want to use to talk to a given user, if that user is not already on that service.

For group chats it is harder, and this is why the deadline for group chats is years away. The problem is that you need a way to verify the identity of arbitrary numbers of remote users on different platforms - effectively looking up their identity in a secure manner which stops services from maliciously spoofing identities.

One possible way to solve this would be for users to explicitly link their identity on one service with that on the gatekeeper’s service - eg “Alice on AliceChat is talking in the same room as Bob on BobChat; Bob will be asked to prove to AliceChat that he is the real Bob” - and so if AliceChat has already validated Bob’s identity, then this can be used to spot him popping up on other services. It also gives Bob a way to block themselves from ever being unwittingly bridged to AliceChat.

There are many other approaches too - and the onus is on the industry to figure out the best solution for decentralised identity in the next 3-4 years in order to realise the most exciting benefits of the DMA.


For a much deeper dive into the above, please check out our post at "How do you implement interoperability in a DMA world?"

How do you implement interoperability in a DMA world?

2022-03-29 — General — Matthew Hodgson

With last week’s revelation that the EU Digital Markets Act will require tech gatekeepers (companies valued at over $75B or with over $7.5B of turnover) to open their communication APIs for the purposes of interoperability, there’s been a lot of speculation on what this could mean in practice, To try to ground the conversation a bit, we’ve had a go at outlining some concrete proposals for how it could work.

However, before we jump in, we should review how the DMA has come to pass.

What’s driven the DMA?

Today’s gatekeepers all began with a great product, which got more and more popular until it grew to such a size that one of the biggest reasons to use the service is not necessarily the product any more, but the benefits of being able to talk to a large network of users. This rapidly becomes anti-competitive, however: the user becomes locked into the network and can’t switch even if they want to. Even when people have a really good reason to move provider (e.g. WhatsApp’s terms of use changing to share user data with Facebook, Apple doing a 180 on end-to-end encrypting iCloud backups, or Telegram not actually being end-to-end encrypted), in practice hardly anything changes - because the users are socially obligated to keep using the service in order to talk to the rest of the users on it.

As a result, it’s literally harmful to the users. Even if a new service launches with a shiny new feature, there is enormous inertia that must be overcome for users to switch, thanks to the pull of their existing network - and even if they do, they’ll just end up with their conversations haphazardly fragmented over multiple apps. This isn’t accepted for email; it isn’t accepted for the phone network; it isn’t accepted on the Internet itself - and it shouldn’t be accepted for messaging apps either.

Similarly: the closed networks of today’s gatekeepers put a completely arbitrary limit on how users can extend and enrich their own conversations. On email, if you want to use a fancy new client like Superhuman - you can. If you want to hook up a digital assistant or translation service to help you wrangle your email - you can. If you want to hook up your emails to a CRM to help you run your business - you can. But with today’s gatekeepers, you have literally no control: you’re completely at the mercy of the service provider - and for something like WhatsApp or iMessage the options are limited at best.

Finally - all the users’ conversation metadata for that service (who talks to who, and when) ends up centralised in the gatekeepers’ databases, which then become an incredibly valuable and sensitive treasure trove, at risk of abuse. And if the service provider identifies users by phone number, the user is forced to disclose their phone number (a deeply sensitive personal identifier) to participate, whether they want to or not. Meanwhile the user is massively incentivised not to move away: effectively they are held hostage by the pull of the service’s network of users.

So, the DMA exists as a strategy to improve the situation for users and service providers alike by building a healthier dynamic ecosystem for communication apps; encouraging products to win by producing the best quality product rather than the biggest network. To quote Cédric O (Secretary of State for the Digital Sector of France), the strategy of the legislation came from Washington advice to address the anticompetitive behaviour of the gatekeepers “not by breaking them up… but by breaking them open.” By requiring the gatekeepers to open their APIs, the door has at last been opened to give users the option to pick whatever service they prefer to use, to choose who they trust with their data and control their conversations as they wish - without losing the ability to talk to their wider audience.

However, something as groundbreaking as this is never going to be completely straightforward. Of course while some basic use cases (i.e. non-E2EE chat) are easy to implement, they initially may not have a UX as smooth as a closed network which has ingested all your address book; and other use cases(eg E2EE support) may require some compromises at first. It’s up to the industry to figure out how to make the most of that challenge, and how to do it in a way which minimises the impact on privacy - especially for end-to-end encrypted services.

What problems need to be solved?

We’ve already written about this from a Matrix perspective, but to recap - the main challenge is the trade-off between interoperability and privacy for gatekeepers who provide end-to-end encryption, which at a rough estimate means: WhatsApp, iMessage, secret chats in Facebook Messenger, and Google Messages. The problem is that even with open APIs which correctly expose the end-to-end encrypted semantics (as DMA requires), the point where you interoperate with a different system inevitably means that you’ll have to re-encrypt the messages for that system, unless they speak precisely the same protocol - and by definition you end up trusting the different system to keep the messages safe. Therefore this increases the attack surface on the conversations, putting the end-to-end encryption at risk.

Alex Stamos (ex-CISO at Facebook) said that “WhatsApp rolling out mandatory end-to-end encryption was the largest improvement in communications privacy in human history” – and we agree. Guaranteed end-to-end encrypted conversations on WhatsApp is amazing, and should be protected at all costs. If users are talking to other users on WhatsApp (or any set of users communicating within the same E2EE messenger), E2EE should and must be maintained - and there is nothing in the DMA which says otherwise.

But what if the user consciously wants to prioritise interoperability over encryption? What if the user wants to hook their WhatsApp messages into a CRM, or run them through a machine translation service, or try to start a migration to an alternative service because they don’t trust Meta? Should privacy really come so spectacularly at the expense of user freedom?

We also have the problem of figuring out how to reference users on other platforms. Say that I want to talk to a user with a given phone number, but they’re not on my platform - how do I locate them? What if my platform only knows about phone numbers, but you’re trying to talk to a user on a platform which uses a different format for identifiers?

Finally, we have the problem of mitigating abuse: opening up APIs makes it easier for bad actors to try to spam or phish or otherwise abuse users within the gatekeepers. There are going to have to be changes in anti-abuse services/software, and some signals that the gatekeeper platforms currently use are going to go away or be less useful, but that doesn't mean the whole thing is intractable. It will require changes and innovative thinking, but we’ve been making steady progress (e.g. the work done by Element’s trust and safety team). Meanwhile, the gatekeepers already have massive anti-abuse systems in place to handle the billions of users within their walled gardens, and unofficial APIs are already widespread: adding official APIs does not change the landscape significantly (assuming interoperability is implemented in such a way that the existing anti-abuse mechanisms still apply).

In the past, gatekeepers dismissed the effort of interop as not being worthwhile - after all, the default course of action is to build a walled garden, and having built one, the temptation is to try to trap as many users as possible. It was also not always clear that there were services worth interoperating with (thanks to the chilling effects of the gatekeepers themselves, in terms of stifling innovation for communication startups). Nowadays this situation has fundamentally changed however: there is a vibrant ecosystem of open communication startups out there, and a huge appetite to build a vibrant open ecosystem for interoperable communication, but like the open web itself.

What are the requirements?

Before going further in considering solutions, we need to review the actual requirements of the DMA. Our best understanding at this point is that the DMA will mandate that:

  • Gatekeepers will have to provide open and documented APIs to their services, on request, in order to facilitate interoperability (i.e. so that other services can communicate with their users).
  • These APIs must preserve the same level of end-to-end encryption (if any) to remote users as is available to local users.
  • This applies to 1:1 messaging and file transfer in the short term, and group messaging, file-transfer, 1:1 VoIP and group VoIP in the longer term.

So, what could this actually look like?

The DMA legislation deliberately doesn’t focus on implementation, instead letting the industry figure out how this could actually work in practice. There are many different possible approaches, and so from our point of view as Matrix we’ve tried to sketch out some options to make the discussion more concrete. Please note these are preliminary thoughts, and are far from perfect - but hopefully useful as a starting point for discussion.

Finding Bob

Imagine that you have a user Alice on an existing gatekeeper, which we’ll call AliceChat, who runs an E2EE messaging service which identifies users using phone numbers. Say that they want to start a 1-to-1 conversation with Bob, who doesn’t use AliceChat, but Alice knows he is a keen user of BobChat. Today, you’d have no choice but to send them an SMS and nag them to join AliceChat (sucks to be them if they don’t want to use that service, or if they’re unable to for whatever reason - e.g. their platform isn’t supported, or their government has blocked access, etc), or join BobChat yourself.

 

However, imagine if instead the gatekeeper app had a user experience where the app prompted you to talk to the user via a different platform instead. It’d be no different to your operating system prompting you to pick which app to use to open a given file extension (rather than the OS vendor hardcoding it to one of their own apps - another win for user rights led by the EU!).


Now, the simplest approach in the short term would be for each gatekeeper to pre-provision a set of options of possible alternative networks. (The DMA says that, on request, other service providers can ask to have access to the gatekeeper’s APIs for the purposes of interoperability, so the gatekeeper knows who the alternative networks may be). “Bob is not on AliceChat - do you want to try to reach him instead on BobChat, CharlieChat, DaveChat (etc)”.

Much like users can configure their preferred applications for file extensions in an operating system today, users would also be able to add their own preferred service providers - simply specifying their domain name.

Connecting to Bob

Now, AliceChat itself needs to figure out how to query the remote service provider to see if Bob actually exists there. Given the DMA requires that gatekeepers provide open APIs with the same level of security to remote users as their local ones using today’s private APIs - and very deliberately doesn’t mandate specific protocols for interoperability - they will need to locate a bridge which can connect to the other system.

In this thought experiment, the bridge used would be up to the destination provider. For instance, bobchat.com could announce that AliceChat users should connect to it via alicechat-bridge.bobchat.com using the AliceChat protocol(or matrix-bridge.bobchat.com via Matrix or xmpp-bridge.bobchat.com via XMPP) by a simple HTTP API or even a .well-known URL. Users might also be able to override the bridge used to connect to them (e.g. to point instead at a client-side bridge), and could sign the advertisement to prove that it hadn’t been tampered with.

AliceChat would then connect to the discovered bridge using AliceChat’s vendor-specific newly opened API, and would then effectively treat Bob as if they were a real AliceChat user and client to all intents and purposes. In other words, Bob would effectively be a “ghost user” on AliceChat, and subject to all their existing anti-abuse mechanisms.

Meanwhile, the other side of the bridge converts through to whatever the target system is - be that XMPP, Matrix, a different proprietary API, etc. For Matrix, it’d be chatting away to a homeserver via the Application Service API (using End-to-Bridge Encryption via MSC3202). It’s also worth noting that the target might not even be a bridge - it could be a system which already natively speaks AliceChat’s end-to-end encrypted API, thus preserving end-to-end encryption without any need to re-encrypt. It’s also worth noting that while historically bridges have had a bad reputation as being a second class (often a second class afterthought), Matrix has shown that by considering them as a first class citizen and really focusing on mapping the highest common denominator between services rather than lowest common denominator, it’s possible for them to work transparently in practice. Beeper is a great example of Matrix bridging being used for real in the wild (rather amusingly they just shipped emoji reactions for WhatsApp on iOS via their WhatsApp<->Matrix bridge before WhatsApp themselves did…)

Architecturally, it could look like this:

Or, more likely (given a dedicated bridge between two proprietary services would be a bit of a special case, and you’d have to solve the dilemma of who hosts the bridge), both services could run a bridge to a common open standard protocol like Matrix or XMPP instead (thus immediately enabling interoperability with everyone else connected to that network):

Please note that while these examples show server-side bridges, in practice it would be infinitely preferable to use client-side bridges when connecting to E2EE services - meaning that decrypted message data would only ever be exposed on the client (which obviously has access to the decrypted data already). Client-side bridges are currently complicated by OS limits on background tasks and push notification semantics (on mobile, at least), but one could envisage a scenario where you install a little stub AliceChat client on your phone which auths you with AliceChat and then sits in the background receiving messages and bridging them through to Matrix or XMPP, like this:

Another possible architecture could be for the E2EE gatekeeper to expose their open APIs on the clients, rather than the server. DMA allows this, to the best of our knowledge - and would allow other apps on the device to access the message data locally (with appropriate authorisation, of course) - effectively doing a form of realtime data liberation from the closed service to an open system, looking something like this:

Finally, it's worth noting that when peer-to-peer decentralised protocols like P2P Matrix enter production, clientside bridges could bridge directly into a local communication server running on the handset - thus avoiding metadata being exposed on Matrix or XMPP servers providing a common language between the service providers.

Locating users

Now, the above describes the simplest and most naive directory lookup system imaginable - the problem of deciding which provider to use to connect to each user is shouldered by the users. This isn’t that unreasonable - after all, users may have strong feelings about what providers to use to talk to a given user. Alice might be quite happy to talk to Bob via BobChat, but might be very deliberately avoiding talking to him on DaveChat, for whatever ominous reasons.

However, it’s likely in future we will also see other directory services appear in order to map phone numbers (or other identities) to providers - whether these piggyback on top of existing identity providers (gatekeepers, DNS, telcos, SSO providers, governments) or are decentralised through some other mechanism. For instance, Bob could send AliceChat a blinded proof that he authorises them to automatically route traffic to him over at BobChat, with BobChat maintaining a matching proof that Bob is who he claims to be (having gone through BobChat’s auth process) - and the proofs could be linked using a temporary key such that Bob doesn’t even need to maintain a long-term one. (Thanks to James Monaghan for suggesting this one!)

Another alternative to having the user decide where to find each other could be to use a decentralised Keybase-style identity system to track verified mappings of identities (phone numbers, email addresses etc) through to service providers - perhaps something like IDX might fit the bill? While this decentralised identity lookups have historically been a hard problem, there is a lot of promising work happening in this space currently and the future looks promising.

Talking to Bob

Meanwhile, Alice still needs to talk to Bob. As already discussed, unless everyone speaks the same end-to-end encrypted protocol (be it Matrix, WhatsApp or anything else), we inevitably have a trade-off here between interoperability and privacy if Bob is not on the same system as Alice (assuming AliceChat is end-to-end encrypted) - and we will need to clearly warn Alice that the conversation is no longer end-to-end encrypted:


To be clear: right now, today, if Bob were on AliceChat, he could be copy-pasting all your messages into (say) Google Translate in a frantic effort to workaround the fact that his closed E2EE chat platform has no way to do machine translation. However, in a DMA world, Bob could legitimately loop a translation bot into the conversation… and Alice would be warned that the conversation was no longer secure (given the data is now being bridged over to Google).

This is a clear improvement in user experience and transparency. Likewise, if I’m talking to a bridged user today on one of these platforms, I have no way of telling that they have chosen to prioritise interop over E2EE - which is frankly terrifying. If I’m talking to someone on WhatsApp today I blindly assume that they are E2EE as they are on the same platform - and if they’re using an unofficial app or bridge, I have no way to tell. Whereas in a DMA world, you would expect the gatekeeper to transparently expose it.

If anything, this is good news for the gatekeeper in that it consciously advertises a big selling point for them: that for full E2EE, users need to talk to other users in the same walled garden (unless of course the platform speaks the same protocol). No more need for bus shelter adverts to remind everywhere that WhatsApp is E2EE - instead they can remind the user every time they talk to someone outside the walled garden!

Just to spell it out: the DMA does not require or encourage any reduction in end-to-end encryption for WhatsApp or similar: full end-to-end encryption will still be there for users in the same platform, including through to users on custom clients (assuming the gatekeeper doesn’t flex and turn it off for other reasons).

Obviously, this flow only considers the simple case of Alice inviting Bob. The flow is of course symmetrical for Bob inviting Alice; AliceChat will need to advertise bridges which can be used to connect to its users. As Bob pops up from BobChat, the bridge would use AliceChat’s newly open APIs to provision a user for him, authing him as per any other user (thus ensuring that AliceChat doesn’t need to have trusted BobChat to have authenticated the user). The bridge then sends/receives messages on Bob’s behalf within AliceChat.

Group communication

This is all very well for 1:1 chats - which are the initial scope of the DMA. However, over the coming years, we expect group chats to also be in scope. The good news is that the same general architecture works for group chats too. We need a better source of identity though: AliceChat can’t possibly independently authenticate all the new users which might be joining via group conversations on other servers (especially if they join indirectly via another server). This means adopting one of the decentralised identity lookup approaches outlined earlier to determine whether Charlie on CharlieChat is the real Charlie or an imposter.

Another problem which emerges with group chats which span multiple service providers is that of indirect routing, especially if the links between the providers use different protocols. What if AliceChat has a direct bridge to BobChat (a bit like AIM and ICQ both spoke OSCAR), BobChat and CharlieChat are connected by Matrix bridges, and AliceChat and CharlieChat are connected via XMPP bridges? We need a way for the bridges to decide who forwards traffic for each network, and who bridges the users for which network. If they were all on Matrix or XMPP this would happen automatically, but with mixed protocols we’d probably have to extend the lookup protocol to establish a spanning tree for each conversation to prevent forwarding loops.

Here’s a deliberately twisty example to illustrate the above thought experiment:

There is also a risk of bridge proliferation here - in the worst case, every service would have to source bridges to directly connect to every other service who came along, creating a nightmarish n-by-m problem. But in practice, we expect direct proprietary-to-proprietary bridges to be rare: instead, we already have open standard communication protocols like Matrix and XMPP which provide a common language between bridges - so in practice, you could just end up in a world where each service has to find a them-to-Matrix or them-to-XMPP bridge (which could be run by them, or whatever trusted party they delegate to).

Conclusion

A mesh of bridges which connect together the open APIs of proprietary vendors by converting them into open standards may seem unwieldy at first - but it’s precisely the sort of ductwork which links both phone networks and the Internet together in practice. As long as the bridging provides for highest common denominator fidelity at the best impedance ratio, then it’s conceptually no different to converting circuit switched phone calls to VoIP, or wired to wireless Ethernet, or any of the other bridges which we take entirely for granted in our lives thanks to their transparency.

Meanwhile, while this means a bit more user interface in the communication apps in order to select networks and warn about trustedness, the benefits to users are enormous as they put the user squarely back in control of their conversations. And the UX will improve as the tech evolves.

The bottom line is, we should not be scared of interoperability, just because we’ve grown used to a broken world where nothing can interconnect. There are tractable ways to solve it in a way that empowers and informs the user - and the DMA has now given the industry the opportunity to demonstrate that it can work.

Interoperability without sacrificing privacy: Matrix and the DMA

2022-03-25 — General — Matthew Hodgson

Yesterday the EU Parliament & Council agreed on the contents of the Digital Markets Act - new legislation from the EU intended to limit anticompetitive behaviour from tech “gatekeepers”, i.e. big tech companies (those with market share larger than €75B or with more than €7.5B a year of revenue).

This is absolutely landmark legislation, where the EU has decided not to break the gatekeepers up in order to create a more competitive marketplace - but instead to “break them open”. This is unbelievably good news for the open Internet, as it is obligating the gatekeepers to provide open APIs for their communication services. In other words: no longer will the tech giants be able to arbitrarily lock their users inside their walled gardens - there will be a legal requirement for them to expose APIs to other services.

While the formal outcomes of yesterday’s agreement haven’t been published yet (beyond this press release), our understanding is that the DMA will mandate:

  • Gatekeepers will have to provide open and documented APIs to their services, on request, in order to facilitate interoperability (i.e. so that other services can communicate with their users).
  • These APIs must preserve the same level of end-to-end encryption (if any) to remote users as is available to local users.
  • This applies to 1:1 messaging and file transfer in the short term, and group messaging, file-transfer, 1:1 VoIP and group VoIP in the longer term.

This is the best possible outcome imaginable for the open internet. Never again will a big tech company be able to hold their users hostage in a walled garden, or arbitrarily close down or sabotage their APIs.

So, what’s the catch?

Since the DMA announcement on Thursday, there’s been quite a lot of yelling from some very experienced voices that mandating interoperability via open APIs is going to irrevocably undermine end-to-end encrypted messengers like WhatsApp. This seems to mainly be born out of a concern that the DMA is somehow trying to subvert end-to-end encryption, despite the fact that the DMA explicitly mandates that the APIs must expose the same level of security, including end-to-end encryption, that local users are using. (N.B. Signal doesn’t qualify as a gatekeeper, so none of this is relevant to Signal).

So, for WhatsApp, it means that the API would expose both the message-passing interface as well as the key management APIs required to interoperate with WhatsApp using your own end-to-end-encrypted WhatsApp client - E2EE would be preserved.

However, this does mean that if you were to actively interoperate between providers (e.g. if Matrix turned up and asked WhatsApp, post DMA, to expose an API we could use to write bridges against), then that bridge would need to convert between WhatsApp’s E2EE’d payloads and Matrix’s E2EE’d payloads. (Even though both WhatsApp and Matrix use the Double Ratchet, the actual payloads within the encryption are completely different and would need to be converted). Therefore such a bridge has to re-encrypt the traffic - which means that the plaintext is exposed on the bridge, putting it at risk and breaking the end-to-end encryption guarantee.

There are solutions to this, however:

  • We could run the bridge somewhere relatively safe - e.g. the user’s client. There’s a bunch of work going on already in Matrix to run clientside bridges, so that your laptop or phone effectively maintains a connection over to iMessage or WhatsApp or whatever as if it were logged in… but then relays the messages into Matrix once re-encrypted. By decentralising the bridges and spreading them around the internet, you avoid them becoming a single honeypot that bad actors might look to attack: instead it becomes more a question of endpoint compromise (which is already a risk today).

  • The gatekeeper could switch to a decentralised end-to-end encrypted protocol like Matrix to preserve end-to-end encryption throughout. This is obviously significant work on the gatekeeper’s side, but we shouldn’t rule it out. For instance, making the transition for a non-encrypted service is impressively little work, as we proved with Gitter. (We’d ideally need to figure out decentralised/federated identity-lookup first though, to avoid switching from one centralised identity database to another).

  • Worst case, we could flag to the user that their conversation is insecure (the chat equivalent of a scary TLS certificate warning). Honestly, this is something communication apps (including Matrix-based ones!) should be doing anyway: as a user you should be able to tell what 3rd parties (bots, integrations etc) have been added to a given conversation. Adding this sort of semantic actually opens up a much richer set of communication interactions, by giving the user the flexibility over who to trust with their data, even if it breaks the platonic ideal of pure E2E encryption.

On balance, we think that the benefits of mandating open APIs outweigh the risks that someone is going to run a vulnerable large-scale bridge and undermine everyone’s E2EE. It’s better to have the option to be able to get at your data in the first place than be held hostage in a walled garden.

Other considerations

One other complaint which has come up a bunch is around speed of innovation: the idea that WhatsApp or similar would be seriously slowed down by having to (effectively) maintain stable documented federation APIs, and figure out how to do backwards compatibility for new features. It’s true that this will take a bit more effort (similar to how adding GDPR compliance takes some effort), but the ends make it more than worth it. Plus, if the rag-tag Matrix ecosystem can do it, it doesn’t seem unreasonable to think that a $600B company like Meta can figure it out too...

Another consideration is that it might make it too easy to build malicious 3rd party clients - e.g. building your own "special" version of Signal which connects to the official service, but deliberately or otherwise has security flaws. The fact is that we're already in this position though: there are illicit alternative clients flying around all over the place, and the onus is on the app stores to protect their users from installing malware. This isn't reason to throw the baby of interoperability out with the bathwater of bootleg clients.

The final complaint is about moderation and abuse: while open APIs are good news for consumer choice, they can also be used by spammers, phishers and other miscreants to cause problems for the users within the gatekeeper. Much like a mediaeval citadel; opening up your walled garden means that both good and bad people can turn up. And much like real life, this is a solvable problem, even if it’s unfortunate: the benefits of free trade massively outweigh the downsides of having to police strangers more effectively. Frankly, moderation and anti-abuse approaches on the Internet today are infamously broken, with centralised moderation by gatekeepers producing increasingly erratic results. By opening the walled gardens, we are forcing a much-needed opportunity to review how to empower users and admins to filter unwanted content on their own terms. There’s a recent write-up of the proposed approach for Matrix at https://element.io/blog/moderation-needs-a-radical-change/, which outlines one strategy - but there are many others. Honestly, having to improve moderation tooling is a worthwhile price to pay for the benefits of open APIs.

So, there you have it. Hopefully you’ll agree that the benefits here outweigh the risks: without open APIs we wouldn't even have the option to talk about interoperability. We should be celebrating a new dawn for open access, rather than fearing that the sky is falling and this is nefarious attempt to undermine end-to-end encryption.

Hosting FOSDEM 2022 on Matrix

2022-02-07 — General — Thib

Last year was the first time FOSDEM was hosted on Matrix, and it was generally a huge success - and so the FOSDEM team trusted us again this year and we’re happy to say that it seems to have gone really well! This year’s FOSDEM was massive once again, featuring 654 speakers, 731 events, and 103 tracks.

This year hosting the event went smoother than last year, the only significant issue was some of the Q&A Jitsis not being broadcast to the devrooms on Saturday before 10:15 UTC, for which we offer our apologies to the speakers impacted. This turned out to be a problem with the Matrix<->Jitsi access control sync system which hadn’t showed up during earlier testing, but we patched around it rapidly on the day.

The most notable difference between this year and the previous year has been the usage of a “attendees.fosdem.org” instance in addition to the original “fosdem.org” one, specifically for attendees. The graphs speak for themselves: Synapse could handle the load of the 23K users (13K joined users and 10K lurkers) spread across a total number of 941 rooms. The real eye-opener however is that of the 13K joined users, only 4K came came from the FOSDEM attendee server, and 1K from Libera Chat, meaning that ~70% of the Matrix participants were already on Matrix and came in from existing servers! 🤯 That means the vast majority of people attended over federation. Decentralisation at work, people! It works! We didn’t host the conference… you did!!

FOSDEM attendees heatmap

But not only did the backend handle the load smoothly: the general user experience felt tightly integrated. People were welcomed by a tailor-made home page in Element to help them navigate through all the tracks and stands:.

Homescreen of chat.fosdem.org

One of the great things is it doesn’t require heavy modifications to Element: anyone who installs their own instance of Element can use a simple html file to display relevant information to their audience.

New this year, we also generated a space hierarchy for the whole conference at #fosdem2022:fosdem.org to help navigate the maze of rooms, making it even easier for users on their own servers to jump in:

FOSDEM Space listing all the FOSDEM rooms

Another greatly appreciated feature was the famous “maximised widgets” I (Thib) keep telling you about in Matrix Live episodes. Attendees and speakers could give the conference the central attention it deserved while simultaneously keeping an eye on what was happening in the chat.

FOSDEM stand with maximised widget view

From the speaker's perspective, we tried to streamline the user journey as much as possible: a bot invited them to a backstage room, in which they joined a Jitsi widget while their talk was being played in the track or devroom. They could see the most upvoted questions by the audience in a dedicated widget. A few minutes before their pre-recorded talk was over, a countdown (new this year!) could be displayed to tell them and the host they were about to go live. At the end of the countdown, the backstage Jitsi was broadcasted to the track so the speaker could answer the questions.

Backstage room where the speaker and host could talk and see the questions

If you want to have an in-depth look at the backend’s architecture, it didn’t change much from last year. You can have a look at last year’s blog post for the details on the setup. Most of the heavy lifting was around the conference bot used to set rooms up, create the spaces, populate them with widgets, arrange layouts and trigger countdowns before going live…

Finally we are immensely proud that we had our first ever Matrix Devroom! Not only does it mean the staff of a large international open source conference thinks it’s worth having a track of our own, it also means our community has grown to the point where we could have 8 hours of back to back Matrix content! Here’s a link to the YouTube playlists to which we are going to add the videos of FOSDEM as they’re being published.

Huge thanks to the FOSDEM team for trusting us, massive shout-out to Element Matrix Services and Element’s Ops and infrastructure team for their fantastic job in setting everything up and making sure everything was ready in time, a sincere thank you to all the fantastic speakers who shared awesome content, and finally to all the attendees. What a weekend!

NextPage 2