A few years ago, back when Matrix was originally implementing end-to-end encryption, we asked Moxie (the project lead for Signal) whether he’d ever consider connecting Signal (then TextSecure) to Matrix. After all, one of Matrix’s goals is to be an interoperability layer between other communication silos, and one of the reasons for us using Signal’s Double Ratchet Algorithm for Matrix’s encryption was to increase our chances of one day connecting with other apps using the same algorithm (Signal, WhatsApp, Google Allo, Skype, etc). Moxie politely declined, and then a few months later wrote “The ecosystem is moving” to elaborate his thoughts on why he feels he “no longer believes that it is possible to build a competitive federated messenger at all.”
At the time we didn’t respond via a blog post; instead we ended up talking it through a few times in person to see how misaligned we really were. The conclusion was that we agreed to disagree and Moxie said he’d be happy to be proved wrong, and wished us good luck. However, the subject has come up again thanks to Moxie’s talk on the same subject at 36C3 last week, and we keep getting asked to write a formal response on the Matrix side. So, here’s an attempt to do so. (Moxie didn’t want the 36C3 talk recorded, and I haven’t watched it, so this is responding to the original blog post).
From my perspective, the main points proposed in ‘The ecosystem is moving’ boil down to:
Decentralised systems are harder to design and build than centralised ones, as coordination is harder if you don’t have a single authority to trust.
Decentralised systems are harder and slower to evolve than centralised ones, as you can’t force participants to rapidly roll out (or even agree on) new features.
Users in federated systems tend to coalesce around the best/biggest server that the bulk of people use - which means that server typically gets to see a disproportionate amount of communication metadata (who’s talking to who, and when), and has disproportionate power over the network, which could bully others away from running their own deployments.
If users don’t trust their app provider, they can always go switch apps, which gives them freedom.
Open systems are less secure because you have no control over the quality of the implementations - if anyone can bring their own client or server to the table, all it takes is one bad implementation to compromise everyone in the vicinity.
Now, all of these points are valid to some extent.
It’s absolutely true that decentralised systems are harder than centralised ones. Prior to Matrix we built centralised comms systems - we literally can do a side-by-side comparison for the same team to see how easily and fast we built our centralised comms system relative to Matrix. Empirically It took us around 6 times longer to get to the same feature-set with Matrix.
It’s also true that decentralised systems are harder to evolve than centralised ones - you can’t just push out a given feature with a single app update, but you have to agree and publish a public spec, support incremental migration, and build governance processes and community dynamics which encourage everyone to implement and upgrade. This is hard, but not impossible: we’ve spent loads of time and money on Matrix’s governance model and spec process to get it right. It’s still not perfect, but we haven’t seen much fragmentation so far, and when we’re pushing out a feature empirically we can and do go just as fast as the centralised alternatives. (E2E by default is a bit of a special case because we’ve had to go and reimplement many features users take for granted today in an E2E-capable manner, but we’re sprinting to get it done in the coming weeks). A bigger problem is that there are hundreds of spec change proposals which folks would like to see in the protocol, and finding a way to manage expectations and parallelise spec progress is hard - something we’re looking to improve in 2020 (although still figuring out how!)
It’s also fair that in a multi-server federated model, users naturally tend to sign up on the most prominent server(s) (e.g. the matrix.org homeserver in the case of Matrix). In practice, the matrix.org homeserver currently makes up about 35% of the visible Matrix network by active users. It’s also true that Matrix servers currently store metadata about who’s talking to who, and when, as a side-effect of storing and relaying messages on behalf of their users. And without an adequate protocol governance system in place, a large server could start pushing around smaller ones in terms of protocol behaviour. In practice, we’re looking into solving metadata protection in Matrix by experimenting with hybrid P2P / Client Server models - letting users store their metadata purely clientside if they so desire, and potentially obfuscating who’s talking to who via mixnets of blinded store & forward servers (more about this coming up at FOSDEM). Combined with nomadic accounts, this would let us eventually turn off the matrix.org server entirely and eliminate the pseudo-centralisation effect - the default ‘server’ would be the one running on your client.
It’s true that if a user doesn’t trust (say) Telegram, they are free to go switch to Signal or WhatsApp or whatever instead… at the massive expense of having to persuade all their friends to install yet another app, and fragmenting their conversation history across multiple apps.
Finally, it’s also true that because anyone can develop a Matrix client or server and connect to the global network, there’s a risk of bad quality implementations in the wild. There are many forks of Riot on the app stores - we simply can’t vouch for whether they are secure. Similarly there are Matrix clients whose E2E encryption is partial, missing, or unreviewed. And there are a wide range of different Matrix servers run by different people with different agendas in different locations, which may be more or less trustworthy.
HOWEVER: all of this completely ignores one critical thing - the value of freedom. Freedom to select which server to use. Freedom to run your own server (perhaps invisibly in your app, in a P2P world). Freedom to pick which country your server runs in. Freedom to select how much metadata and history to keep. Freedom to choose which apps to use - while still having the freedom to talk to anyone you like (without them necessarily installing yet another app). Freedom to connect your own functionality - bots, bridges, integrations etc. Freedom to select which identifiers (if any) to use to register your account. Freedom to extend the protocol. Freedom to write your own client, or build whole new as-yet-unimagined systems on top.
It’s true that if you’re writing a messaging app optimised for privacy at any cost, Moxie’s approach is one way to do it. However, this ends up being a perversely closed world - a closed network, where unofficial clients are banned, with no platform to build on, no open standards, and you end up thoroughly putting all your eggs in one basket, trusting past, present & future Signal to retain its values, stay up and somehow dodge compromise & censorship… despite probably being the single highest value attack target on the ‘net.
Quite simply, that isn’t a world I want to live in.
We owe the entire success of the Internet (let alone the Web) to openness, interoperability and decentralisation. To declare that openness, interoperability and decentralisation is ‘too hard’ and not worth the effort when building a messaging solution is to throw away all the potential of the vibrancy, creativity and innovation that comes from an open network. Sure, you may end up with a super-private messaging app - but one that starts to smell alarmingly like a walled garden like Facebook’s Internet.org initiative, or an AOL keyword, or Google’s AMP.
So, we continue to gladly take up Moxie’s challenge to prove him wrong - to show that it’s both possible and imperative to create an open decentralised messaging platform which (if you use reputable apps and servers) can be as secure and metadata-protecting as Signal… and indeed more so, given you can run your server off the grid, and don’t need to register with a phone number, and in future may not even need a server at all.
(Comments over at HN)
Hi all, Ben here.
Since joining the core team as Developer Advocate last year it's been quite a ride. One of the best things about the job is getting the chance to talk to so many people about their projects and what they would like to see happen in the matrix ecosystem. With so much going on, I just want to say thanks to everyone who has been so welcoming to me and share some of my personal highlights, as I recall them, from 2018!
Matrique became Spectral, and is generally awesome. Apparently the name "Matrique" was chosen because it sounds French, but those who speak the language well revealed that this name was not ideal! The project was re-named "Spectral", and is going strong. I really appreciate the multi-user facility! It's a great looking client, and runs great on macOS too (protip: get more attention from
/me by providing a macOS build…)
Ubuntu Touch has the most Matrix clients per-user of any platform. UT epitomises the resilience and collaborative spirit of Open Source. It's a true community maintenance effort, and is as friendly a community as you might meet. uMatriks came first, but it's FluffyChat that prompted me to install it on my battered old OnePlus One. FluffyChat is now extremely full-featured, with E2EE support being actively discussed.
In the command line, gomuks appeared and quickly became a competent client, but in terms of sheer enthusiasm and momentum, I must give commendation to matrix-client.el, a newly revived mode for Emacs which turns your editor/OS into a great Matrix Client. I enjoyed using it enough that it began to change my mind about using emacs. Laptops have more than 8mb memory these days anyway.
Around May, I started to notice another obsession brewing in the community. Bridging is a core part of the Matrix mission, but it was around this time I started seeing it in the wild.
Summer 2018 Half-Shot began working in the Matrix core team, and was hugely productive in maintaining and developing the bridge infrastructure for matrix.org. IRC bridging is far more stable and reliable now than it was a year ago. And yet there are still more bridges - too many to list, so I'm picking the ones I've used and enjoyed.tulir's suite of bridges including mautrix-telegram and mautrix-whatsapp are extremely stable and useful - big thank you to TravisR for maintaining t2bot.io and hosting the Telegram bridge too.SMSMatrix, a phone-hosted bridge is simple and works great for SMS bridging.
In April, kitsune announced v0.2 of libqmatrixclient describing it as “the first one more or less functional and stable" - confidence! This library now powers both Quaternion and Spectral. QMatrixClient has continued to get updates, plus features including lazy loading and VoIP signalling.
There are a few libs I want to pay more attention to this year, starting with tulir's maubot now that it has been rewritten in Python. I'm also excited to see jmsdk, part of ma1uta's broader ecosystem of Matrix tooling - a Java-based SDK.
DSN Traveller tries to get a rough overview of how the Matrix network is structured today. It records how many rooms it finds, how many users and servers take part in those rooms, and how they relate to each other, meaning how many users a server has and of how many rooms it is part of.Florian's thesis was handed in last August. Source code is available.
All details at https://dsn-traveller.dsn.scc.kit.edu/, room at #dsn-traveller:dsn-traveller.dsn.scc.kit.edu.
I want to use Matrix, and I want to host my own homeserver. As such, matrix-docker-ansible-deploy is a project I absolutely love. It uses Synapse docker images from the Matrix core team, and combines them with Ansible playbooks written and organised by Slavi. It lets you quickly deploy everything needed for a Synapse homeserver, and it's simple enough that even I can use it.Construct, a homeserver implementation in C++ began successfully federating with Matrix, work progressed from around April/May.
Having a Matrix-native mode for shields.io (those counter/indicator images you often see at the top of repos) seems like something petty at first, but it's actually a great indicator of the importance of Matrix from the outside. Plus, I love seeing the images at the top of different repos. Thanks Brendan for helping this along.
Two students worked on Matrix-related projects during GSOC 2018.
If you think I've missed something, or if there's a project I should have included rather than another, or even if you just disagree with my choices, let's discuss it in #twim:matrix.org. See you there, and let's all parade ahead to a productive, open, interoperable 2019!
Thank you to Half-Shot for all your work on Bridges over the last months and beyond. Today is your last day, but I'm sure we'll see you again before long. Text below is from Half-Shot.
Today marks my last day of my 3 month internship at New Vector (the startup which hires many of the core Matrix team). For those of you who haven't been reading Ben's fabulous blog posts, I've been working exclusively on bridges; in particular the IRC bridge.
Tasked with the goal of making it crash less and run faster, I hope that the evidence is visible and people are generally having a better experience on it!
Some stats pulled from the matrix-appservice-irc repo:
But aside from showing off some stats, I wanted to mention all the new features:
There are future plans to make bridging more visible to Matrix Clients as a first class citizen. Ideas like speccing a state event (MSC1410) so that bridges can interact with each other properly and clients can create full bridge management views which are still decentralised from an integration manager.
I'd like to give a shoutout to Travis who has reviewed nearly all my changes that have made their way into the bridge, on top of all the other tasks he has on his plate. And of course a thank you to all of the Matrix team who have been very supportive during my time here.
Over the last few weeks there's been a huge movement in the cryptocurrency communities over needing to find a better communication medium than Slack. Some of the biggest communities for projects like Status, Aragon, TenX, Tezos, OmiseGo, Polkadot and many others are getting overrun by phishing attacks where malicious users have set up bots which auto-DM users joining the room in order to try to extract private keys to steal funds. Slack has very limited support for avoiding this sort of abuse (especially at the free service tiers), so the search is on for an alternative solution. There seems to be some confusion over what Matrix & Riot can and can't do to help the situation, so we thought we'd write a blog post about it (especially after we had so much fun at the ETHLDN meetup last week!).
To be clear: we see Ethereum, Bitcoin, Ripple, Stellar and all the other decentralised currencies as being very closely related to Matrix. Just as distributed ledgers disrupt the fragmented oligopoly old-school banking industry, we want Matrix to disrupt the relatively old-school communications systems of today. And so we'd really rather like that Matrix and Riot rocked when it comes to supporting cryptocurrency communities, and this is something we intend to dedicate resources to long term: we've got some big plans.
Decentralisation. Rather than each community having its own silo, with users having to juggle accounts over all of them, Matrix decentralises rooms over all the different servers. Users can have a single account and still jump into all the other communities (as well as the rest of the Matrix universe). However, each community can run its own server instance (if they want to) and have complete control over its behaviour.
Encryption. Matrix has first-class end-to-end encryption (although the UX in Riot needs refinement and is technically still beta). This is great for encrypting rooms which need privacy - although it does come at the expense of being able to do server-side content filtering, which is desirable for fixing phishing attacks. So you probably don't want to turn on encryption for rooms which need phish filtering (or you could use a bot to decrypt and autoremove malicious content).
A standard real-time API. One bit of feedback we've heard recently is that “Riot has no realtime API”. This is spectacularly untrue; Riot is a client for the Matrix protocol, which is in and of itself an open standard realtime API for messaging, which you can use for writing whatever bots and extensions your heart desires.
Finely grained permissions per room. Likewise there seems to be some confusion over Matrix's access control model. In Matrix, each user in a room has a ‘power level' - typically a number between 0 and 100. By convention, normal users who have just joined the room have 0; the room creator and ‘admins' have 100; and ‘moderators' have 50. Pretty much every access you can do in a room then has a threshold which defines how much power a user needs to perform the action. It doesn't get much more finely grained than this!
Ability to disable DMs and room invites. Architecturally Matrix lets you prevent users who use a given server from receiving invites (the homeserver can just autoreject the invites, based on some set of rules).
We're currently putting together a quick demo to show this off in the Synapse server implementation, but it boils down to having an option to cancel invites here (federated) and here (local).
Check out the demo below!
Ability to filter content. Similarly, Matrix architecturally lets a given server filter out messages based on content or some other pattern from being received by its users.
We're also putting together a demo of this too in Synapse, which boils down to redacting inappropriate events here (federated) and here (local). The demo isn't quite ready yet but we'll update this & yell when it is.
Check out the demo below!
the DM/invite disabling and spam/phish filtering code has now landed on the develop branch of Synapse, and we've deployed an demo example of it at https://phishfree.riot.im. Messages containing the word 'SPAM' are filtered, and invites are disabled (unless you are the local server admin).
Other stuff. Matrix and Riot give loads of other fun stuff too:
E2E UX (and Riot UX in general): While the underlying encryption of Matrix is solid, the UX exposed by Riot needs considerable work - specifically to improve the device verification flow and automatically share keys between trusted devices. We're continuing to work on this over the next few months. Likewise there are many areas for possible improvement in Riot's overall UX and design that we're working through as urgently as we can.
Active Application Services: The per-server filtering described above is good if you just want to protect users on a given server (e.g. the server you point your community at). However, if you want to filter all the messages for a given room which may be federated over multiple servers, you need a way to define a centralised chokepoint to define the filtering rules. Architecturally this is meant to be performed by an ‘Active Application Service' in Matrix, but we've not yet defined or implemented this API. The idea for the room to define a list of services that messages are filtered through by all servers before they may be accepted for the room. This would be the ideal solution to the phishing-filtering problem, but in practice filtering just local users (and perhaps disabling federation for particularly sensitive rooms or servers) is probably good enough for the immediate problem here.
Hope this provides some much-needed clarity to the debate! If there are other features cryptocurrency communities need to thrive please let us know, as we'd like to actively help to support decentralized communities.#matrix-dev:matrix.org is probably the best place for further questions :)
Finally: one thing that has come up a few times in this discussion has been “Matrix's funding crisis means they may not be here to stay”. All I can say is that Matrix is here to stay. Even if the core team ended up just being Matthew hacking away by himself funded by Patreon/Liberapay, we have a large and passionate wider dev community who aren't going anywhere. But more importantly (and not wishing to jinx it), in the last few weeks we have received offers of significant funding which may hopefully resolve the funding crisis for the foreseeable. Nothing is signed yet, but watch this space, and meanwhile I strongly suggest betting on Matrix being here to stay!
Something that has kept coming up since we ran into funding problems in July is the idea that Matrix could launch a cryptocurrency - a token for use when exchanging items of value within Matrix. This isn't such a far-fetched idea: folks are already starting to look at how to sell content/services within Matrix, and the idea of using a Matrix-specific currency rather than credit card, PayPal, or an existing cryptocurrency could have some major advantages. Specifically:
Anyway, we thought it was worth giving an update on our thoughts about cryptocurrencies - and to publicly clarify that anyone claiming that they are running a Matrix.org ICO is lying.
We'd genuinely be very interested to hear feedback from the community on whether an ICO for Matrix would be a good idea or not - #matrix:matrix.org is probably the best place to discuss it. It's important to understand that our core focus will always be on Matrix itself, where we still have a lot of work to get through - and if we do an ICO it'll be in partnership with specialist cryptocurrency experts, and hopefully minimise the impact to the core Matrix project itself. But right now, we would be foolish not to be seriously considering the option.
Matthew, Amandine & the team.
We're currently looking into different ways that Matrix is being used in the wild, and an important question that has come up is whether anyone is using Matrix yet for decentralised communication in parts of the world where centralised communication poses a problem - due to bad connectivity or privacy concerns. Similarly we'd love to hear from anyone who is seriously trialling Matrix's end-to-end encryption for use in geographies where privacy is a particularly big issue for human rights.
So, if anyone has stories (anecdotal or otherwise) about how they're using or planning to use Matrix to make the world a better place, in a location where that's particularly critical, please can you let us know as soon as possible (@matthew:matrix.org or @Amandine:matrix.org). This is fairly urgent because we're currently looking at various options for how to prioritise effort and funding for Matrix, and if there are people out there who are depending on Matrix in this manner it would significantly help us support them!
Matthew, Amandine & the team.