As part of our semi-regular series of having Matrix core team members write about how they see the overall project, here’s Oddvar Lovaas’ view. Oddvar helps out with project management and promoting Matrix.
According to the homepage, Matrix is “a new open standard for interoperable Instant Messaging and VoIP, providing pragmatic HTTP APIs and open source reference implementations for creating and running your own real-time communication infrastructure. ”
That’s all good – but you might be asking yourself “sure, but what can it do?” And more importantly, “what can it do for me?”
The original inspiration for Matrix was to fix the problem of fragmented IP communications, by creating a standard for creating and running your own real-time communication infrastructure. This means that if you want your app or program or website to be able to communicate user to user for example, you can use Matrix. Matrix is the protocol through which your communication packets are sent and received, and we provide HTTP APIs to make it easy to make use of this protocol in your code.
The nice thing here is that the user can to talk to any other user anywhere in the Matrix ecosystem, much like email or the web. For example, let’s imagine I have an app whose goal is to keep the user updated on anything happening in the football world. Whenever any news drops in, the app is notified and thousands of users check the app for the news. This app could have a communication element where the users can talk in rooms (maybe a #general room and rooms for each football club) – or even between themselves or in groups of friends. Today, a lot of people would use an IM-client to do this, but with Matrix it wouldn’t matter if you use a dedicated IM-app or talk inside the football app – since you are using the same Matrix account, you will get the same conversations in both clients!
In fact, imagine that later on you are chatting with some (non-football) friends on your Matrix-supporting chat-application. You can then easily check the previous conversation to see if anyone’s appreciated the great joke you made earlier – without having to go back into the football app.
But Matrix’s real potential and ultimate mission is to be a generic messaging and data synchronisation system for the web – allowing people, services and devices to easily communicate with each other with full history. It’s easier to visualise the chat-application because we are used to chat-messages going back and to, but there’s nothing stopping you from putting other data instead of chat-messages. For example, you could use the Matrix protocol to exchange moves – encrypted and secured, of course – in your Chess-game. In fact, your Chess-game could use Matrix both for chatting and exchanging payloads of data.
Imagine if you open up your favourite chat-application, and your contacts there include other users of the same app and also other Matrix-users (so the app has exposed itself to Matrix). Your friend, however, much prefers a different app, but he can still talk to you over the Matrix protocol. And if he ever moves to the other app (or any other Matrix-supporting app) – he would still have all the backlog and history of the conversation!
Obviously the problem here is that we can’t instantly make the various chat-applications support Matrix. We believe if we can encourage and grow a truly open communication ecosystem, users will get used to the availability and benefits of interoperable services and they will demand it everywhere.
I’m Matthew, and I’m responsible for the techie side of what we’re up to with Matrix.
Matrix is the result of a lot of work my team’s done over the last 10 years or so (first at MX Telecom, then OpenMarket, and then Amdocs) in developing next-generation IP communication solutions. First we started with an Asterisk-based platform running basic PSTN IVR services, and then shifted to an in-house IAX-based IVR platform built in Java, and then added circuit-switched (3G-324M) video calling, then switched to SIP/RTP, C++ and a massively-distributed softswitch architecture affectionately called ‘The Next Generation’. Then the iPhone and Android came along, and we realised we didn’t have to be constrained by built-in phone capabilities and ported our whole C++ SIP/RTP VoIP stack over to iOS/Android and got writing Video/VoIP calling apps. This evolved to developing full-blown unified communication apps (e.g. Blah), using XMPP at first for messaging before switching to our own HTTP-based messaging APIs.
Somewhere along the way it became painfully obvious that VoIP and IM hasn’t really evolved as well as the rest of the internet. Back when SIP/RTP first emerged, it simply wasn’t mature enough to work on the open internet as well as HTTP or SMTP or even FTP – it needed STUN, ICE, TURN, Opus and many other refinements to be really usable in the wild. And similarly XMPP hasn’t taken over the world quite as much as we once hoped. Meanwhile, many folks went and built their own proprietary walled-garden solutions (be it Skype, FaceTime, Viber, WhatsApp, or even our own efforts) and we’ve ended up in the current horrible situation where our online communication is fragmented across hundreds of isolated apps and websites. It’s like a world where email was never unified, and half the world is still stuck on Compuserve. And it’s counter to the whole ethos of the internet as an open platform for collaboration and communication.
We decided that we want to fix this and so we have built and published a new open standard, together with open source (ASLv2) reference server (Python/Twisted) and client (JS/Angular, Python, Perl) codebases, and so provide new building blocks that can be used to build truly interoperable federated IM and VoIP functionality. We consider this effectively an investment in the industry: by creating a strictly non-profit initiative like Matrix, we both make the world a better place for end users – as well as creating new business opportunities (both opensource and commercial) for the telecoms industry as a whole.
The standard and code are all brand new and very much still in creation at this point, but we’re releasing it early to get as much feedback and input from the community as early on as we possibly can. Right now our focus is on fully decentralised federated group messaging, but VoIP and identity management is coming together well too. You can think of it as “making VoIP/IM as interoperable and flexible as email”, or perhaps “the missing signalling layer for WebRTC”, “XMPP for an HTTP world”, or “what would happen if IRC, XMPP, SIP, SMTP, IMAP and NNTP had kids?” Here are some reasons we think that you should use Matrix:
- Simple pragmatic RESTful HTTP/JSON APIs. No more XMPP or SIP stacks and wrestling XML streams or torture-testing SIP parsers.
- No single points of control for channels of communication (unless you really want it for moderation or similar). Room state for a room is synchronised with eventual consistency over all participating Matrix servers – no single server controls the room.
- No more netsplits – history re-heals itself if the matrix fractures
- All communication is group by default: 1:1 chat is just a subset of group chat.
- Multi-device aware: all state is stored and synchronised in realtime across all devices, and away-state and notifications are aware of multiple devices.
- Uses arbitrary 3rd party identifiers – doesn’t rely on JIDs or SIP URIs for identity.
- Share the same simple HTTP signalling channel for messaging and VoIP
- Support more efficient transports if you want (e.g. low-bandwidth/low-roundtrip sync on mobile)
- Built for mobile – e.g. support push notification and low-bandwidth/low-latency client-server transports if needed (in progress)
- TLS (HTTPS) by default, either with self-signed certs with published public keys or proper SSL CA signed certs (in progress)
- End-to-end PKI encryption (in progress)
- Trusted federation of public identity servers available for publishing your PKI public keys and tracking your validated 3rd party IDs
If this sounds good to you, then please take a look at the spec, or our tutorials, or jump straight into playing with the APIs, or try running your own Matrix homeserver, or sign up to our mailing lists – and whatever else, come swing by #matrix:matrix.org and say hi!
Hi, I’m Amandine and I look after the businessy bits of Matrix.org. I have a technical background and have always had the need to see the bigger picture. My motivation in starting Matrix has mainly been to make my dream of ubiquitous real-time communications come true and fix what felt broken in the industry. Here is the story.
When studying telecoms I was fascinated by converging networks, the “Next Generation Network” as we called it. Wow, imagine! Having fixed lines, mobiles and computers able to communicate with each other? Call a number and have the ability to answer seamlessly on any device? And the capability to do video calls and share files? That was definitely Next Generation at the time! Especially given the best we could do with a mobile was bad GSM, MMS and WAP if you were lucky…. A decade later video calling is possible on any device; I can share anything I want with my contacts (pictures, videos, files, random stickers) thanks to hundreds of different apps; sometimes my history is even synced across several devices! And if I choose my mobile provider carefully I can even have a “One Number” service! :)
But still, we’re far away from the ubiquitous dream of 10 years ago. None of my apps are talking to each other. One number is more often “one login per app”. My conversation history is scattered across all the apps (who never experienced the “Did you tell me on Facebook, Whatsapp, Skype or SMS???” question followed by 10 minutes of intense fiddling with your phone which of course is running out of battery?). My address book and profile data is stored everywhere by companies like Facebook, Google and random startups, but who knows how much they really value your privacy…
Is the tech not good enough yet? Only partially: most of the features, typically IM, video, VoIP, are already available and widely used. However it is a reality that no tech has really imposed itself as an interoperability standard. What about economic blockers then? A better bet: what are the incentives for big companies and small startups to share their communities? Most of these corporations choose a business model that locks their users into walled gardens, directly linking their valuation to this user base. But ultimately this is bad for competition and bad for the users.
What about phone networks then? Why don’t they do anything? Well, they have tried. It’s called RCS (Rich Communication Services), or joyn. It’s a good initiative designed by committee and implemented as a functionality deeply embedded in the network rather than a light over-the-top deployment and consequently very expensive and time-consuming to put in place. Similarly as per telco’s historical positioning, RCS is more focused on quality of service than quality of user experience. But RCS is facing a few challenges: communications over IP must be free for the user to compete with the Skype & WhatsApp of this world, which limits a lot the return on investment of the deployment of an expensive network update, limiting the adoption of RCS by mobile networks. Besides quality of service is not necessarily what will trigger a success when competing with over the top apps which are 100% focused on providing the best user experience… And despite the GSMA’s best efforts implementing interoperability between networks is proving very painful, limiting the growth of the ecosystem.
But ultimately we all know that today success is driven by users. This multitude of minds which decides the exact split of fun and usefulness that will define the value of a product and make it crazily successful overnight. So why are users not pushing for this ubiquity? I see two reasons: first they often don’t realize how useful it could be and second: there is nothing to push! Indeed it can easily be considered as handy to have one app per group of people I want to communicate with: Facebook messenger with my school friends, Whatsapp with international friends, Skype with my family, Snapchat with my boyfriend, Voxer with my best friend… But why can’t I have only one app, the one I prefer, to get in touch with everyone, whatever they are using? And if I want a fancy Tango-like video experience then I couldstill launch Tango for this scenario and have the whole chat history there. I truly believe that once fragmentation will be over everyone will wonder how we could ever live with it.
And what about privacy? Isn’t it bad enough that the second I send an email to my friend asking if he’s booked his flights to Madrid I get a flurry of airline adverts for flights to Spain? Even without going into the debate of whether governments should file every citizen, it’s a question of principles – I’d rather have my communication history and my contacts stored by someone I trust and avoid useless adverts. Some of you will say “Oh but you’re French. ‘Vive la Révolution!’ is your default motto…” – but think about it for a minute:
Imagine: only install the apps you want on your phone for communication. Use the ones you prefer because the UX is great or you love their smileys. Your whole address book is there, correctly merged. All the conversation history with your friends and family is there. And you don’t care what app they’re using. Or how they logged in: your app discovers them based on any ID (email, phone number, Facebook): you just need to have them in your address book. You know where your data is – perhaps stored by a new startup using renewable energy for their data center, just because you want to save the planet. Or by your geeky brother running his own server under his bed (using end-to-end encryption if you think he might be nosy!). You can still have many apps but each allows you to do different fun stuff: fancy video, crazy pictures, drawings, imaginative stickers. You control your communication. You decide what to you want to use, who you want to trust. Welcome to Matrix.