Synapse 0.3.4 released!

25.09.2014 00:00 — TechMatthew Hodgson

Hi all,

Synapse 0.3.4 is out! The main changes here are a huge fleet of bugfixes to the webclient, and the addition of TURN support for VoIP relaying. This makes VoIP/Video call setup much more reliable (although the UI still needs some love).

Meanwhile, we set an IRC bridge (https://github.com/tm604/Matrix-IRCBridge) running linking #matrix:matrix.org and #matrix-dev:matrix.org to their Freenode counterparts (#matrix & #matrix-dev respectively) - much fun to see innocent IRC users assimilated up into Matrix, and for us to now be using Matrix for /all/ our development chat and interaction (whether it's through a Matrix client or an IRC client).

To upgrade to the latest homeserver, just git pull && ./synctl restart. If you want to start playing with VoIP in earnest on your own homeserver, you'll need to install a TURN server - see docs/turn-howto.rst for details on how to set one up.

http://matrix.org/alpha has already been upgraded to 0.3.4, so feel free to play with all the new stuff there.

thanks,

Matthew

🔗Changes in synapse 0.3.4 (2014-09-25)

This version adds support for using a TURN server. See docs/turn-howto.rst on how to set one up.

Homeserver:

  • Add support for redaction of messages.
  • Fix bug where inviting a user on a remote home server could take up to 20-30s.
  • Implement a get current room state API.
  • Add support specifying and retrieving turn server configuration.

Webclient:

  • Add button to send messages to users from the home page.
  • Add support for using TURN for VoIP calls.
  • Show display name change messages.
  • Fix bug where the client didn't get the state of a newly joined room until after it has been refreshed.
  • Fix bugs with tab complete.
  • Fix bug where holding down the down arrow caused chrome to chew 100% CPU.
  • Fix bug where desktop notifications occasionally used "Undefined" as the display name.
  • Fix more places where we sometimes saw rooms as IDs rather than aliases.
  • Fix bug which caused lag when entering text in the text box.

Synapse 0.3.3 released

22.09.2014 00:00 — TechMatthew Hodgson

Hello world,

Synapse 0.3.3 was released today, concentrating on both feature & stability improvements on the webclient. The big news is that 1:1 video calling via WebRTC is implemented (although we're still working on some stability issues, and Synapse doesn't support TURN yet. This will be coming in the next few days however!).

Play with the demo server at http://matrix.org/alpha, or update your own homeserver with a git pull && ./synctl restart or by cloning a new checkout with git clone https://github.com/matrix-org/synapse.git

--Matthew

🔗Changes in synapse 0.3.3 (2014-09-22)

Homeserver:

  • Fix bug where you continued to get events for rooms you had left.

Webclient:

  • Add support for video calls with basic UI.
  • Fix bug where one to one chats were named after your display name rather than the other person's.
  • Fix bug which caused lag when typing in the textarea.
  • Refuse to run on browsers we know won't work.
  • Trigger pagination when joining new rooms.
  • Fix bug where we sometimes didn't display invitations in recents.
  • Automatically join room when accepting a VoIP call.
  • Disable outgoing and reject incoming calls on browsers we don't support VoIP in.
  • Don't display desktop notifications for messages in the room you are non-idle and speaking in.

Synapse 0.3.2 released

18.09.2014 00:00 — TechMatthew Hodgson

We just pushed out a major new release (0.3.0) of Synapse: the current reference Matrix homeserver, swiftly followed by some hotfixes (0.3.1, 0.3.2).

The big changes are:

  • Change the VoIP setup API to be more efficient (bundling together media stream candidates into a single setup object). API breaking change!
  • Change the Registration API to be more consistent. API breaking change!
  • Lots of nice usability refinements to webclient - e.g. per-message notifications; cursor-key history navigation; faster room loading
  • More bugfixes and stability fixes on the homeserver

Please reload the page if you're lurking on http://matrix.org/alpha to pull in the new webclient.

If you're running a homeserver - please upgrade with a quick git pull && ./synctl restart.

If you're not running a homeserver... why not? ;D

--Matthew

🔗Changes in synapse 0.3.2 (2014-09-18)

Webclient:

  • Fix regression where an empty "bing words" list in old accounts didn't send notifications when it should have done.

🔗Changes in synapse 0.3.1 (2014-09-18)

This is a release to hotfix v0.3.0 to fix two regressions.

Webclient:

  • Fix a regression where we sometimes displayed duplicate events.
  • Fix a regression where we didn't immediately remove rooms you were banned in from the recents list.

🔗Changes in synapse 0.3.0 (2014-09-18)

See UPGRADE for information about changes to the client server API, including breaking backwards compatibility with VoIP calls and registration API.

Homeserver:

  • When a user changes their displayname or avatar the server will now update all their join states to reflect this.
  • The server now adds "age" key to events to indicate how old they are. This is clock independent, so at no point does any server or webclient have to assume their clock is in sync with everyone else.
  • Fix bug where we didn't correctly pull in missing PDUs.
  • Fix bug where prev_content key wasn't always returned.
  • Add support for password resets.

Webclient:

  • Improve page content loading.
  • Join/parts now trigger desktop notifications.
  • Always show room aliases in the UI if one is present.
  • No longer show user-count in the recents side panel.
  • Add up & down arrow support to the text box for message sending to step through your sent history.
  • Don't display notifications for our own messages.
  • Emotes are now formatted correctly in desktop notifications.
  • The recents list now differentiates between public & private rooms.
  • Fix bug where when switching between rooms the pagination flickered before the view jumped to the bottom of the screen.
  • Add support for password resets.
  • Add bing word support.
  • Update VoIP implementation to use new API

Registration API:

  • The registration API has been overhauled to function like the login API. In practice, this means registration requests must now include the following: 'type':'m.login.password'. See UPGRADE for more information on this.
  • The 'user_id' key has been renamed to 'user' to better match the login API.
  • There is an additional login type: 'm.login.email.identity'.
  • The command client and web client have been updated to reflect these changes.

Synapse v0.2.3 released

13.09.2014 00:00 — TechMatthew Hodgson

We're back home from TechCrunch where we had a great time arguing the pros and cons of Matrix versus XMPP with anyone and everyone - thanks to all who came to talk to us!

Meanwhile, Synapse v0.2.3 was released yesterday: the emphasis this week has been on improving stability issues on the server, and lots of cosmetic and functionality fixes on the webclient.

If you're running a homeserver, please upgrade asap - upgrading has never been easier - assuming you've checked out the master branch, simply:

git pull
./synctl restart

...and you will be up and running on 0.2.3. We keep the master branch tracking the latest release.  http://matrix.org/alpha is already on the latest release should you wish to experiment first!

Alternatively, if you're installing from scratch, try:

git clone https://github.com/matrix-org/synapse.git
python setup.py develop --user
./synctl start # to generate a default config
./synctl start # to actually set it running

and then head over to http://localhost:8008 to play with your very own Matrix homeserver!

🔗Changes in synapse 0.2.3 (2014-09-12)

Homeserver:

  • Fix bug where we stopped sending events to remote home servers if a user from that home server left, even if there were some still in the room.
  • Fix bugs in the state conflict resolution where it was incorrectly rejecting events.
  • Many stability fixes

Webclient:

  • Display room names and topics.
  • Allow setting/editing of room names and topics.
  • Display information about rooms on the main page.
  • Handle ban and kick events in real time.
  • VoIP UI and reliability improvements.
  • Add glare support for VoIP.
  • Improvements to initial startup speed.
  • Don't display duplicate join events.
  • Show message sent/sending state via opacity
  • Multiline message sending support
  • Many other bug & stability fixes

Techcrunch Update!

09.09.2014 00:00 — EventsMatthew Hodgson

The Matrix team is at TechCrunch Disrupt SF 2014 this week - we had a great time at the hackathon on Saturday/Sunday where we were really excited to see several teams building their hacks on Matrix APIs - we stayed the night to support whilst building out our own hack (experimenting with sending collaborative 3D animations over Matrix as JSON)!  Huge thanks to everyone who built on top of Matrix - hope you had as much fun as we did :)

Please come talk to us in person in Startup Alley (next to the Brazilian Pavilion) if you're attending Disrupt and find out all about Matrix first hand!

6am Hackathon...

(Photo shows us somehow surviving in the 6am slot.  Mental note to never try to do raycasting vector maths after being up for 24 hours again...)

High Level Overview

06.09.2014 00:00 — GeneralAmandine Le Pape

Hey,

We realized we're missing a high level overview highlighting the key points of the specifications and architecture of Matrix, which could be useful for those who don't feel like going straight into the specs. So here is a quick presentation to fill in some of the gaps!

Please fire away with any questions :)

Synapse v0.2.2 released!

06.09.2014 00:00 — GeneralMatthew Hodgson

Hi all,

We just pushed our first major update since matrix.org launched for Synapse, the reference Matrix homeserver:

🔗Changes in synapse 0.2.2 (2014-09-06)

Webclient:

  • Fix desktop notifications
  • Add support for captchas on registration
  • Handle m.room.aliases events.
  • Implement local echo when sending message
  • Inform the UI when a message failed to send.
  • Only autoscroll on receiving a new message if the user was already at the bottom of the screen.
  • Add support for ban/kick reasons.
  • Add /join support for IRC acolytes
  • Make IRC-style commands a little more forgiving

Homeserver:

  • Validate m.room.power_level events.
  • When the server returns state events it now also includes the previous content if any to aid pagination
  • Add support for inviting people when creating a new room.
  • Make the homeserver inform the room via m.room.aliases when a new alias is added for a room.

The matrix.org homeserver & webclient has already been updated to the latest version - if you're running your own homeserver, please update. v0.2.2 retains backwards compatibility with previous homeserver releases.

Get the code from http://github.com/matrix-org/synapse!

Hello world

03.09.2014 00:00 — GeneralMatthew Hodgson

I'm Matthew, and I'm responsible for the techie side of what we're up to with Matrix.

Matrix is the result of a lot of work my team's done over the last 10 years or so (first at MX Telecom, then OpenMarket, and then Amdocs) in developing next-generation IP communication solutions.  First we started with an Asterisk-based platform running basic PSTN IVR services, and then shifted to an in-house IAX-based IVR platform built in Java, and then added circuit-switched (3G-324M) video calling, then switched to SIP/RTP, C++ and a massively-distributed softswitch architecture affectionately called 'The Next Generation'.  Then the iPhone and Android came along, and we realised we didn't have to be constrained by built-in phone capabilities and ported our whole C++ SIP/RTP VoIP stack over to iOS/Android and got writing Video/VoIP calling apps.  This evolved to developing full-blown unified communication apps (e.g. Blah), using XMPP at first for messaging before switching to our own HTTP-based messaging APIs.

Somewhere along the way it became painfully obvious that VoIP and IM hasn't really evolved as well as the rest of the internet.  Back when SIP/RTP first emerged, it simply wasn't mature enough to work on the open internet as well as HTTP or SMTP or even FTP - it needed STUN, ICE, TURN, Opus and many other refinements to be really usable in the wild.  And similarly XMPP hasn't taken over the world quite as much as we once hoped.  Meanwhile, many folks went and built their own proprietary walled-garden solutions (be it Skype, FaceTime, Viber, WhatsApp, or even our own efforts) and we've ended up in the current horrible situation where our online communication is fragmented across hundreds of isolated apps and websites.  It's like a world where email was never unified, and half the world is still stuck on Compuserve.  And it's counter to the whole ethos of the internet as an open platform for collaboration and communication.

We decided that we want to fix this and so we have built and published a new open standard, together with open source (ASLv2) reference server (Python/Twisted) and client (JS/Angular, Python, Perl) codebases, and so provide new building blocks that can be used to build truly interoperable federated IM and VoIP functionality. We consider this effectively an investment in the industry: by creating a strictly non-profit initiative like Matrix, we both make the world a better place for end users - as well as creating new business opportunities (both opensource and commercial) for the telecoms industry as a whole.

The standard and code are all brand new and very much still in creation at this point, but we're releasing it early to get as much feedback and input from the community as early on as we possibly can. Right now our focus is on fully decentralised federated group messaging, but VoIP and identity management is coming together well too.  You can think of it as "making VoIP/IM as interoperable and flexible as email", or perhaps "the missing signalling layer for WebRTC", "XMPP for an HTTP world", or “what would happen if IRC, XMPP, SIP, SMTP, IMAP and NNTP had kids?” Here are some reasons we think that you should use Matrix:

  • Simple pragmatic RESTful HTTP/JSON APIs.  No more XMPP or SIP stacks and wrestling XML streams or torture-testing SIP parsers.
  • No single points of control for channels of communication (unless you really want it for moderation or similar). Room state for a room is synchronised with eventual consistency over all participating Matrix servers - no single server controls the room.
  • No more netsplits - history re-heals itself if the matrix fractures
  • All communication is group by default: 1:1 chat is just a subset of group chat.
  • Multi-device aware: all state is stored and synchronised in realtime across all devices, and away-state and notifications are aware of multiple devices.
  • Uses arbitrary 3rd party identifiers - doesn't rely on JIDs or SIP URIs for identity.
  • Share the same simple HTTP signalling channel for messaging and VoIP
  • Support more efficient transports if you want (e.g. low-bandwidth/low-roundtrip sync on mobile)
  • Built for mobile - e.g. support push notification and low-bandwidth/low-latency client-server transports if needed (in progress)
  • TLS (HTTPS) by default, either with self-signed certs with published public keys or proper SSL CA signed certs (in progress)
  • End-to-end PKI encryption (in progress)
  • Trusted federation of public identity servers available for publishing your PKI public keys and tracking your validated 3rd party IDs
If this sounds good to you, then please take a look at the spec, or our tutorials, or jump straight into playing with the APIs, or try running your own Matrix homeserver, or sign up to our mailing lists - and whatever else, come swing by #matrix:matrix.org and say hi!

Welcome to the Matrix blog!

03.09.2014 00:00 — Generaladmin

Today is a big day for us as we are launching Matrix.org, an open initiative which has the goal of making real-time communication over IP as seamless and interoperable as email.

How do we want to achieve this? Very simply by creating a new federated, open-source ecosystem for VoIP and instant messaging on the internet! Ambitious right? :)

Actually not that much! In practice it's fairly simple: we are publishing the specification of a pragmatic and lightweight open standard, some opensource reference implementations of the servers and clients, and pragmatic RESTful HTTP JSON APIs, all of which are available right now on Github!

What does this mean for the world?

For techies, it gives developers a new way of building and running their own communications functionality, or integrating existing services into the Matrix ecosystem.

For consumers, it means that eventually they may be able to choose to use their favourite app from their trusted app provider, whilst reaching anyone they like given the entire Matrix ecosystem is interoperable.

So this is the very high level description of Matrix, but it misses the reasons why we've started this. We thought that rather than summarizing the main reasons for launching this project in a faceless post it would be interesting to present it from different perspectives, so we've asked some of the core team to give their take on it:

- Read Matthew's post and get an insight of Matrix' technical history and rationale.

- Check out Amandine's view on why the users need Matrix.

The Matrix Team

PS: Matrix is hoping to talk about the future of communication in general at SXSW Interactive  - if you like the idea of Matrix, please vote for us by clicking here before September 5th!

Why Matrix?

03.09.2014 00:00 — GeneralAmandine Le Pape

Hi, I'm Amandine and I look after the businessy bits of Matrix.org. I have a technical background and have always had the need to see the bigger picture. My motivation in starting Matrix has mainly been to make my dream of ubiquitous real-time communications come true and fix what felt broken in the industry. Here is the story.

When studying telecoms I was fascinated by converging networks, the “Next Generation Network” as we called it. Wow, imagine! Having fixed lines, mobiles and computers able to communicate with each other? Call a number and have the ability to answer seamlessly on any device? And the capability to do video calls and share files? That was definitely Next Generation at the time! Especially given the best we could do with a mobile was bad GSM, MMS and WAP if you were lucky.... A decade later video calling is possible on any device; I can share anything I want with my contacts (pictures, videos, files, random stickers) thanks to hundreds of different apps; sometimes my history is even synced across several devices! And if I choose my mobile provider carefully I can even have a “One Number” service! :)

But still, we're far away from the ubiquitous dream of 10 years ago. None of my apps are talking to each other. One number is more often “one login per app”. My conversation history is scattered across all the apps (who never experienced the “Did you tell me on Facebook, Whatsapp, Skype or SMS???” question followed by 10 minutes of intense fiddling with your phone which of course is running out of battery?). My address book and profile data is stored everywhere by companies like Facebook, Google and random startups, but who knows how much they really value your privacy...

Is the tech not good enough yet? Only partially: most of the features, typically IM, video, VoIP,  are already available and widely used. However it is a reality that no tech has really imposed itself as an interoperability standard. What about economic blockers then? A better bet: what are the incentives for big companies and small startups to share their communities? Most of these corporations choose a business model that locks their users into walled gardens, directly linking their valuation to this user base. But ultimately this is bad for competition and bad for the users.

What about phone networks then? Why don't they do anything? Well, they have tried. It's called RCS (Rich Communication Services), or joyn. It's a good initiative designed by committee and implemented as a functionality deeply embedded in the network rather than a light over-the-top deployment and consequently very expensive and time-consuming to put in place. Similarly as per telco's historical positioning, RCS is more focused on quality of service than quality of user experience. But RCS is facing a few challenges: communications over IP must be free for the user to compete with the Skype & WhatsApp of this world, which limits a lot the return on investment of the deployment of an expensive network update, limiting the adoption of RCS by mobile networks. Besides quality of service is not necessarily what will trigger a success when competing with over the top apps which are 100% focused on providing the best user experience... And despite the GSMA's best efforts implementing interoperability between networks is proving very painful, limiting the growth of the ecosystem.

But ultimately we all know that today success is driven by users. This multitude of minds which decides the exact split of fun and usefulness that will define the value of a product and make it crazily successful overnight. So why are users not pushing for this ubiquity? I see two reasons: first they often don't realize how useful it could be and second: there is nothing to push! Indeed it can easily be considered as handy to have one app per group of people I want to communicate with: Facebook messenger with my school friends, Whatsapp with international friends, Skype with my family, Snapchat with my boyfriend, Voxer with my best friend... But why can't I have only one app, the one I prefer, to get in touch with everyone, whatever they are using? And if I want a fancy Tango-like video experience then I couldstill launch Tango for this scenario and have the whole chat history there. I truly believe that once fragmentation will be over everyone will wonder how we could ever live with it.

And what about privacy? Isn't it bad enough that the second I send an email to my friend asking if he's booked his flights to Madrid I get a flurry of airline adverts for flights to Spain? Even without going into the debate of whether governments should file every citizen, it's a question of principles - I'd rather have my communication history and my contacts stored by someone I trust and avoid useless adverts.  Some of you will say “Oh but you're French. ‘Vive la Révolution!' is your default motto...” - but think about it for a minute:

Imagine: only install the apps you want on your phone for communication. Use the ones you prefer because the UX is great or you love their smileys. Your whole address book is there, correctly merged. All the conversation history with your friends and family is there. And you don't care what app they're using. Or how they logged in: your app discovers them based on any ID (email, phone number, Facebook): you just need to have them in your address book. You know where your data is - perhaps stored by a new startup using renewable energy for their data center, just because you want to save the planet. Or by your geeky brother running his own server under his bed (using end-to-end encryption if you think he might be nosy!). You can still have many apps but each allows you to do different fun stuff: fancy video, crazy pictures, drawings, imaginative stickers. You control your communication.  You decide what to you want to use, who you want to trust. Welcome to Matrix.