It's been 3 months since the Matrix Holiday Special and once again we've all been too busy writing code to put anything that detailed on the blog. So without further a do here's a quick overview of how things have progressed so far in 2016!

🔗Home servers

🔗Synapse

Work on Synapse (our reference homeserver) has been primarily focused on improving performance. This may sound boring, but there's been a huge amount of improvement here since synapse 0.12 was released on Jan 4. Synapse 0.13 on Feb 10 brought huge CPU savings thanks to a whole fleet of caching and other optimisation work - the best way of seeing the difference here is to look at the load graph of the server that hosts matrix.org's synapse+postgres over the last few months:

Ignoring the unrelated blip during March, you can see an enormous step change in system load (which had a matching decrease in actual CPU usage) at the beginning of Feb when the 0.13 optimisations landed on matrix.org :)

Meanwhile, Synapse 0.14 is due any day now with 0.14.0-rc2 released on Wednesday. Here, the focus has been all about memory optimisation - anyone who's run a Synapse seriously will be aware that it can be a memory hog thanks to aggressively caching as much state and history in RAM as possible to avoid hitting the database and keeping everything responsive. 0.14 should improve memory usage just as dramatically as 0.13 improved CPU utilisation - introducing a quick-and-dirty SYNAPSE_CACHE_FACTOR environment variable that lets admins dial down the aggressiveness of the caching (at the expense of performance), but more interestingly implementing string interning and ensuring that events are cached by ID rather than duplicated across multiple caches in order to make memory usage more efficient. It's too early to have impressive looking graphs, and there are still a few memory spikes being tracked down before we release 0.14, but we're hoping for at least a 50% reduction in memory footprint.

Featurewise the highlights include: server-generated unread notification & highlight counts and push badge support, lots of support and refinements for guest access and 3rd party ID invites. Meanwhile we've finally fixed some of the most embarrassing long-standing missing features such as letting folks logout serverside(!), delete aliases and determine whether rooms should be published in the room directory or not.

Finally, Synapse is now part of FreeBSD Ports thanks to Brendan Molloy, and NixOS thanks to Robin Lambertz! Huge thanks to them for contributing the packages to the respective OSes and to all the other synapse package maintainers out there!

It's incredibly exciting to see Synapse's maturity improving and hitting the optimisation stage of its life; huge kudos to Erik for spearheading the optimisation work. We strongly recommend folks upgrade to 0.14 when it's available; it's never been a better time to run a homeserver! :D

🔗Dendron

Meanwhile, Dendron (our next generation homeserver) development has been progressing interestingly: we finished an initial spike to get a Golang skeleton server in place, albeit one that delegates most of the endpoints through to Synapse. In fact, matrix.org itself has been running via Dendron since February!

The whole point of Dendron is to provide an architecture where we can split apart the various endpoints that Synapse provides today, re-implementing them where appropriate in Golang, and critically letting the endpoints scale horizontally with clusters of backend servers abstracted by the single Dendron API facade. As a result, most of the Dendron work has actually ended up going into restructuring Synapse such that multiple Synapses can be run in a cluster behind a single Dendron, allowing us to horizontally scale API endpoints at last. This takes the form of adding cluster replication support to Synapse. This is still work-in-progress as we go through fixing up more and more state to be replicable (replicatable?) between synapses - hopefully it should land in the Synapse 0.15 timeframe. And then we enter a very very interesting new world of horizontally scalable homeservers...

🔗Ruma

Ruma has also seen some progress over the last few months - Ruma is an independent Rust language homeserver project led by Jimmy Cuadra, and whilst in early development still (currently focusing on the user login and registration system) shows a lot of promise. Lots of work has ended up going into the required Rust dependencies rather than the Matrix code itself, but if you're interested in Rust then please drop by #ruma:matrix.org or #ruma on Freenode and say hi!

🔗Clients

Whilst homeserver development is mainly all about performance and scaling work currently, the client side of the Matrix ecosystem is the polar opposite - with lots of rapid progress on exciting new clients happening from all over the community.

🔗Perpetually Talking Online (PTO)

PTO has evolved enormously since Torrie Fischer first revealed it at the end of 2015. PTO is an independent project that acts as a Matrix client that exposes an IRC server interface - effectively turning any Matrix homeserver into an ircd; letting folks hook their favourite IRC clients directly into Matrix and use it as an enormous decentralised IRC network. (N.B. this is not to be confused with matrix-appservice-irc, which acts as a server-side bridge between Matrix rooms and IRC channels.) Obviously you lose some of the Matrix specific features (read receipts, typing notifs, VoIP, etc) but there's clearly a huge benefit for the IRC community to be able to use Matrix as if it were an IRC network.

There have been three releases so far, with the v0.3.0 ("Carburetor") release in March being tantalisingly close to being usable for everyday purposes. We actually have pto.matrix.org all set up and ready to go as an IRC frontend for the matrix.org homeserver and once issue #60 is resolved we'll be turning it on :)

There's one catch though - XChat was never quite built to handle the hundreds of rooms that we've got used to Matrix supporting... :D

Come hang out in #pto:oob.systems if you're interested in PTO!

🔗Quaternion

Quaternion is a new Qt/QML/C++ desktop client created by Felix Rohrbach. It's a fairly early alpha but still quite usable and in very active development. #quaternion:matrix.org is the place to talk all things Quaternion :)

🔗matrix-glib-sdk

Meanwhile, over on the GTK side of the world, Gergely Polonkai has been been making great progress on his matrix-glib-sdk Glib client SDK for Matrix. The end goal here is to implement a full Telepathy plugin for Matrix on top of the SDK. Originally written in C, but now shifted to Vala, the SDK is in very active development and now implements all(?) of the Matrix client-server API - a snapshot of the work-in-progress SDK API docs can be found at http://gergely.polonkai.eu/matrix-glib-sdk. Next up is a formal release and building out clients on top!

🔗matrix-react-sdk, matrix-ios-sdk, matrix-android-sdk and Vector

Finally, huge amounts of time and effort have continued to be pumped into the official matrix-react-sdk, matrix-ios-sdk and matrix-android-sdk - driven substantially by requirements for Vector, the FOSS Matrix-powered collaboration app that we've been helping with:

The best way of seeing what's been going on here is probably by considering Vector itself, which is currently in formal beta (0.4.1 for web, 0.1.2 for iOS and #116 on Android). The big news includes:

• Beta iOS and Android apps. These are early beta but feedback is very much appreciated - the Android beta can be downloaded from Jenkins; if you want to help beta iOS via TestFlight, come ask on #ios:matrix.org.
• Guest access. Anyone can jump straight into Matrix by going to http://vector.im without even having to sign up for an account. Guests are quite restricted on what they can do (and can only join rooms which explicitly have guest access enabled), but this is a *huge* improvement in getting folks using Matrix.
• Ability to jump to any message ever - e.g. when clicking through search results or when permalinking a message... using precisely the same UI that you use when chatting. Permalinks are awesome. If you want to randomly jump back in time to the first weeks of #matrix:matrix.org, now you can...
• Read Markers, scrolling that remembers the scroll offset per-room, and the ability to jump to unread messages
• Synchronised missed notification and missed highlighted notification information per-room
• Badge counts for unread notifications
• Entirely reworked Room Settings
• Entirely reworked User Settings, including push notification configuration
• Entirely reworked Room Directory
• Lots of performance improvements
• Much improved inviting by email
• Much improved reliability on video conferencing
• Closing literally hundreds and hundreds of bugs...

All that remains right now is yet more bugfixing and incorporating feedback from the current betas! Please give as much feedback as possible in #vector:matrix.org :)

🔗Bridges & Bots

Bridges, bots, and other integrations and application services have inevitably taken slightly lower priority whilst we've been focusing on the core server and client bits of the ecosystem. However, as of March we've started a major new project to get these moving again, starting with a big update to the IRC Bridge. This is due to be released next week, but you can get a sneak peek at what's going into the release at the commit log. Highlights include the ability to persist nicks; connect via IPv6; improve formatted message handling; actually feed error messages from IRC back to Matrix; and much much more.

matrix-appservice-verto also got some love, which means that multiway video conferencing powered by FreeSWITCH now works reliably. The quality still could be improved, but the unreliable call setup that plagued earlier versions is now fixed.

In the next few months we're expecting to see a lot more activity on bridges & bots... watch this space :)

Update Sat March 26:

Totally forgot to mention a few of the key new bridges which have been contributed by the community this year - particularly interesting are the Rocket.Chat<->Matrix bridge written by Sing-Li over at Rocket.Chat which provides basic bridging between the awesome Rocket.Chat collaboration app and the wider Matrix ecosystem. It's early days, but this is incredibly promising for 'hardcoded' bridging between specific rooms - it just needs Rocket.Chat to support 'virtual' users and will then be seamless federation.

Similarly, matrix-appservice-gitter is a Gitter<->Matrix bridge built by Leonerd on top of the matrix-appservice-bridge Node library. Again, it's early days but is working well for 'hardcoded' bridging - supporting dynamic rooms and users is next on the todo list :)

🔗The Spec

We started our formal release process for the spec just before Christmas with r0.0.0 - and released r0.0.1 in January with minor clarifications and updates. In practice the spec feels quite stable right now, although things have moved on a bit since January and r0.0.2 is definitely overdue at this point.

In the meantime, you can always get the very latest bleeding edge copy of the spec via the speculator. We've also added an initial cut at a spec for the Identity Service at last.

🔗Events

We've been focusing on writing code than evangelising Matrix recently, although we did get out to FOSDEM 2016 and TADHack Mini Japan and WebRTC Conference and Enterprise Connect 2016 where we showed off Matrix & Vector in the WebRTC Real World Innovation showcase.

🔗GSoC

We are incredibly grateful to have been accepted as an organisation into Google Summer of Code 2016! The last two weeks have been the window for students to propose projects to us that they could work on over the course of the summer, and it's been fascinating to meet the GSoCers and see a whole new community pop up on Matrix and advise and mentor applicants through their proposals. At the last count we've received 35 proposals, many inspired by our list of ideas, including some really impressive candidates - many thanks to all the students who have applied to us. We don't know yet how many slots Google will allocate to us, but one way or another we're really looking forward to helping the GSoCers make the most out of their summer of Matrix! All GSoC discussion is happening in #gsoc:matrix.org.

🔗What's next?

In no particular order, the urgent stuff that still remains includes:

• Continuing to polish synapse and build out dendron-based clustering
• Building as many bridges, bots and other integrations as possible
• The matrix.to URL-handler service: having client-agnostic https://matrix.to/#matrix:matrix.org URLs to help with sharing matrix room aliases etc
• End-to-end crypto. No progress since December; we need to get back to it asap.
• Exiting Vector from beta
• Finishing the server-to-server API specification
• Improving the security model for access_tokens
• Editable messages
• Pinned, tagged, and 'liked' messages
• Threading
• Decentralised accounts
• Decentralised reputation

In practice, Bridging and E2E crypto is likely to get the most attention first (beyond the standard ongoing polishing). There's obviously a significant amount of work there, but we expect to see benefits pretty quickly throughout Matrix - especially from bridging. Hopefully it's true to say that the next few months should be quite transformational :D

Anyway, thanks for reading this sprawling update and for supporting Matrix. And please come say hi in #matrix:matrix.org if you have any questions :)

• Matthew, Amandine & the Matrix.org team.

We are very happy to be one of the companies selected for Google Summer of Code (GSoC) 2016!

GSoC is a great, global opportunity for students to work on open source projects during their university summer break. The idea is for students to propose a project for any of the open source organisations picked by Google, and - if accepted - receive a stipend for working on it. We are very eager to see what projects students will propose - we have written up some ideas here, but students are expected to do some research and come up with projects themselves.

If you are a student wanting to participate in GSoC for Matrix, please come talk to us in #gsoc:matrix.org - we are happy to discuss project ideas and review application drafts. We have also added some general tips on what to include in the application here.

Applications can be submitted starting next Monday, so there's still plenty of time to have a play with Matrix and come up with a cool project idea.

Good luck!

The try-matrix-now page is now being generated by jekyll and all the project pages have been moved to the matrix-doc project on github.

The idea is to make it very easy for anyone to add or update a project entry. All you need to do is to submit a PR with the project details; feel free to start with the template, and you can also add images (thumbnail and/or a main picture for the project page) to the images subfolder (just use the same relative URL that is in the template). Any kind of project using Matrix is welcome; if you are unsure which category to use, just use "other".

Jekyll requires a date in the project filename; we use the date to sort the various project lists (newest projects first). It might be best to submit new entries with a date like 2015-01-01.

Any questions or comments? Come talk to us in #matrix:matrix.org!

We have put an updated version of the Android Matrix Console app (v0.5.3) on the Play store!

This release mainly includes performance improvements, such as using the new "V2" sync API, and other optimisations which should make your user experience a lot nicer. There's also a few new features in the SDK (e.g. tags support) - these will be added to the app hopefully soon.

For the full list of changes, look at the CHANGES files in the android console and SDK projects

Get it from the Google play store!

Enjoy! And please do let us know your feedback in #matrix:matrix.org or #android:matrix.org!

So, you've installed an configured synapse and started chatting from your very own Matrix home server? What's the next step? Well, right now you're probably accessing your new home server over plaintext HTTP, which is bad, particularly because you'll be sending your password over this connection when you log in. You could connect to Synapse's secure HTTP port, but your browser won't trust it by default because you'd normally need to pay for a certificate that your browser would recognise. That is, until recently!

Let's Encrypt is a new initiative that issues SSL certificates free of charge, in an effort to make SSL universal on the Internet. In this blog post, we'll be walking through an example of how we can use this service to get ourselves a securely hosted Synapse.

We're going to assume you have a Synapse installed and listening on the standard ports, 8008 and 8448. If not, follow the Synapse README and come back here when you're done. Everybody ready? Okay, good.

So, in order to get a certificate from Let's Encrypt, we need to prove that we own our domain. The simplest way to do this is to host some special files on our web server. Now, Synapse won't do this. We could use a separate web server, but then we'd have to stop Synapse and start the other web server every time we renewed our certificate, and that means downtime. Instead, let's put our Synapse behind a proper web server and let that serve the files. This has added advantages, including that we can host our Matrix home server on the standard port 443 without having to run Synapse as root.

For this example, we're going to use NGINX, so start by installing NGINX in whatever way your Linux distribution of choice recommends.

Now, you should have a webroot for your new web server somewhere. Hopefully your helpful Linux distribution has started you off with a config file - let's see:

# nano /etc/nginx/nginx.conf

We're looking for the 'server' section of that file. We need to make it look something like this:

    server {'{'}
        # Make sure this is 0.0.0.0: no use listening on 127.0.0.1 or we'll only be
        # serving to ourselves! There's no port here, which means we'll listen on
        # port 80
        listen 0.0.0.0;

        server_name example.com www.example.com;

        access_log /var/log/nginx/example.com.access_log main;
        error_log /var/log/nginx/example.com info;

        # This is where we put the files we want on our site
        root /var/www/examplecom/htdocs;

        # Here's where it gets interesting: This will send any path that starts
        # with /_matrix to our Synapse!

        location /_matrix {'{'}
            proxy_pass http://localhost:8008;
        {'}'}
    {'}'}

When you're happy with the look of that file, let's restart the server:

# nginx -s reload

Before we go any further, let's test our new configuration:

$ curl http://example.com/_matrix/key/v2/server/auto
{'{'}"old_verify_keys":{'{'}{'}'},"server_name":"example.com","signatures":{'{'}"example.com":{'{'}"ed25519:auto":"RWb+w6vHUUokoDgElwG6Cg50ezZvBrzXtJmJIH8jEwI5x0JQ7prn3FwjhbgKTH5jE7J8Ily3HEc4COn4JCCvCA"{'}'}{'}'},"tls_fingerprints":[{'{'}"sha256":"DMbzSZ5Uj7/6p/RT/UtQYJLHm5o0TwBSVYXsqpDdVDs"{'}'}],"valid_until_ts":1455203001035,"verify_keys":{'{'}"ed25519:auto":{'{'}"key":"1YiTDjmE86AlmrbIYE2lyqauV9wPo8jw2kxZAZFfl/Q"{'}'}{'}'}{'}'}

Those are your server's public keys! Now we have a web server running, we can get our SSL certificate. Let's Encrypt have their own client which will automate everything including rewriting your NGINX config file, however that means it has a large number of dependencies and needs to be run as root. For this example, we're going to use the much simpler acme_tiny.py. I'm going to assume you have a user called, 'letsencrypt', so, as root, let's set up the place for it to write its challenge files:

# mkdir /var/www/examplecom/htdocs/.well-known/acme-challenge
# chown letsencrypt:users /var/www/examplecom/htdocs/.well-known/acme-challenge

Now let's switch to our letsencrypt user:

$ ssh [email protected]

We'll start by getting ourselves a copy of acme_tiny.py:

$ git clone https://github.com/diafygi/acme-tiny.git

Now let's set up a directory structure (let's say we might want to manage more than one domain someday):

$ mkdir examplecom
$ cd examplecom
$ ln -s /var/www/examplecom/htdocs/.well-known/acme-challenge challenges

Now, we'll need to generate two keys for Let's Encrypt, and account key and a domain key. The former is what we use to identify ourselves to Let's Encrypt and the latter is the key we use to do the actual SSL.

$ openssl genrsa 4096 > letsencrypt_examplecom_account.key
Generating RSA private key, 4096 bit long modulus
..++
.................................................................................................................................................................................................................................................................................................................................................................................................................++
e is 65537 (0x10001)
$ chmod 600 letsencrypt_examplecom_account.key
$ openssl genrsa 4096 > letsencrypt_examplecom_domain.key
Generating RSA private key, 4096 bit long modulus
.............++
.............................................................................................................................................................................................++
e is 65537 (0x10001)
$ chmod 600 letsencrypt_examplecom_domain.key

Now, store those keys somewhere safe! After you've done that, let's generate a certificate request for our domain. Note that we're requesting one for both example.com and www.example.com: this isn't strictly necessary for Matrix but could be useful if we want to host a website too.

$ openssl req -new -sha256 -key letsencrypt_examplecom_domain.key -subj "/" -reqexts SAN -config <(cat /etc/ssl/openssl.cnf <(printf "[SAN]\\nsubjectAltName=DNS:example.com,DNS:www.example.com")) > examplecom.csr

Okay, we have our keys, our certificate request, and somewhere to host our challenge files, so we're ready to request a certificate! Be careful about this part and make sure you've got everything right, because Let's Encrypt enforce strict rate limits on the number of certificates you can request for one domain. Here we go:

$ python ~/acme-tiny/acme_tiny.py --account letsencrypt_examplecom_account.key --csr examplecom.csr --acme-dir challenges/ > examplecom.crt
Parsing account key...
Parsing CSR...
Registering account...
Registered!
Verifying example.com...
example.com verified!
Verifying www.example.com...
www.example.com verified!
Signing certificate...
Certificate signed!

Is that it, did it work? Well, let's see:

$ openssl x509 -in examplecom.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            01:02:22:77:02:1b:eb:d5:3d:c3:14:6d:87:43:22:3d:fc:0f
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3
        Validity
            Not Before: Feb  6 21:37:00 2016 GMT
            Not After : May  6 21:37:00 2016 GMT
        Subject: CN=example.com
        Subject Public Key Info:
[etc]

Congratulations, you have an official, signed certificate for your domain! Now, before we can use it, we need to add the Let's Encrypt certificate to it, because our web server needs to send both:

$ wget https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem
--2016-02-06 23:38:55--  https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem
Resolving letsencrypt.org... 23.66.17.98, 2a02:26f0:60:489::2a1f, 2a02:26f0:60:481::2a1f
Connecting to letsencrypt.org|23.66.17.98|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1675 (1.6K) [application/x-x509-ca-cert]
Saving to: ‘lets-encrypt-x3-cross-signed.pem'
lets-encrypt-x3-cross-signed.pe 100%[======================================================>]   1.64K  --.-KB/s   in 0s
2016-02-06 23:38:55 (61.5 MB/s) - ‘lets-encrypt-x3-cross-signed.pem' saved [1675/1675]
$ cat examplecom/examplecom.crt lets-encrypt-x3-cross-signed.pem >examplecom/examplecom_cert_chain.crt

Now's let's symlink it in place, along with the domain key, so we can renew it easily later. We'll need to be root again for this:

$ ssh [email protected]
# ln -s /home/letsencrypt/examplecom/examplecom_cert_chain.crt /etc/ssl/nginx/examplecom_cert.pem
# ln -s /home/letsencrypt/examplecom/letsencrypt_examplecom_domain.key /etc/ssl/nginx/examplecom_key.pem

Now, one more crucial thing we have to do before using our SSL is to give NGINX some Diffie Hellman parameters. This is a good thing to do for any SSL configuration (it will increase your score on SSL Labs) but it's absolutely crucial for us because Synapse will only negotiate forward secret connections, so otherwise other Matrix home servers will refuse to talk to us! (Technically, Synapse also support elliptic curve Diffie Hellman, which doesn't need DH parameters, but not all Synapses will support this.) You'll already have some Diffie Hellman parameters from you existing Synapse, so you could use them:

# cp /home/synapse/synapse/matrix.example.com.tls.dh /etc/ssl/nginx/examplecom_dhparams.pem

...or you can generate your own. You'll probably want to do this on your desktop or laptop if you have OpenSSL installed, it will be much faster:

$ openssl dhparam -out examplecom_dhparams.pem 2048
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
........................................................+................[etc, etc]
$ scp examplecom_dhparams.pem [email protected]:/etc/ssl/nginx/examplecom_dhparams.pem

Now, let's get our new certificate in action! Open up your NGINX config file again, and add another server block that look like this:

    server {'{'}
        listen 0.0.0.0:443;
        server_name example.com www.example.com;
        ssl on;
        ssl_certificate /etc/ssl/nginx/examplecom_crt.pem;
        ssl_certificate_key /etc/ssl/nginx/examplecom_key.pem;
        ssl_dhparam /etc/ssl/nginx/examplecom_dhparams.pem;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_prefer_server_ciphers on;
        # mozilla intermediate list, jan 2016
        ssl_ciphers "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA";
        ssl_session_cache shared:SSL:50m;
        access_log /var/log/nginx/examplecom.ssl_access_log main;
        error_log /var/log/nginx/examplecom.ssl_error_log info;
        root /var/www/examplecom/htdocs;
        location /_matrix {'{'}
            proxy_pass http://localhost:8008;
        {'}'}
    {'}'}

It looks pretty similar to our previous server block, except for all that stuff about SSL in the middle. We're pointing NGINX at our certificate, key and Diffie Hellman parameter files and specifying what protocols and ciphers we want our server to talk. The long list here is taken from Mozilla's Server Side TLS guidelines and is their 'Intermediate' list. See that page for more information on what that means, and choose a different list of ciphers if you prefer: just remember we must support at least the ephemeral Diffie Hellman ciphers, or other home servers won't talk to us!

Now let's restart our NGINX and see if it works:

# nginx -s reload

...and that command again, this time with https:

$ curl https://example.com/_matrix/key/v2/server/auto
{'{'}"old_verify_keys":{'{'}{'}'},"server_name":"example.com","signatures":{'{'}"example.com":{'{'}"ed25519:auto":"RWb+w6vHUUokoDgElwG6Cg50ezZvBrzXtJmJIH8jEwI5x0JQ7prn3FwjhbgKTH5jE7J8Ily3HEc4COn4JCCvCA"{'}'}{'}'},"tls_fingerprints":[{'{'}"sha256":"DMbzSZ5Uj7/6p/RT/UtQYJLHm5o0TwBSVYXsqpDdVDs"{'}'}],"valid_until_ts":1455203001035,"verify_keys":{'{'}"ed25519:auto":{'{'}"key":"1YiTDjmE86AlmrbIYE2lyqauV9wPo8jw2kxZAZFfl/Q"{'}'}{'}'}{'}'}

Hooray! You should now be able to open a browser to https://example.com/matrix/ and log in securely over SSL!

🔗Renewing Your Certificate

Now, there's one important step left, and that's to set up renewal for the certificate, otherwise we'll find our shiny new SSL will stop working in three months time. We can use the same acme_tiny command to do this:

$ python ~/acme-tiny/acme_tiny.py --account letsencrypt_examplecom_account.key --csr examplecom.csr --acme-dir challenges/ > examplecom.crt
Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying example.com...
example.com verified!
Verifying www.example.com...
www.example.com verified!
Signing certificate...
Certificate signed!
$ wget https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem
--2016-02-06 23:38:55--  https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem
Resolving letsencrypt.org... 23.66.17.98, 2a02:26f0:60:489::2a1f, 2a02:26f0:60:481::2a1f
Connecting to letsencrypt.org|23.66.17.98|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1675 (1.6K) [application/x-x509-ca-cert]
Saving to: ‘lets-encrypt-x3-cross-signed.pem'
lets-encrypt-x3-cross-signed.pe 100%[======================================================>]   1.64K  --.-KB/s   in 0s
2016-02-06 23:38:55 (61.5 MB/s) - ‘lets-encrypt-x3-cross-signed.pem' saved [1675/1675]
$ cat examplecom/examplecom.crt lets-encrypt-x3-cross-signed.pem >examplecom/examplecom_cert_chain.crt

Synapse will automatically pick up the new certificate, but we'll need to tell NGINX to reload:

# nginx -s reload

Setting up a cronjob to automate this is left as an exercise to the reader!

🔗Federation behind the HTTP Proxy

If you like, you can stop reading now: our clients can access our home server securely but other home server are still talking to our Synapse directly on port 8448. This is fine, and if you're happy with this, you can stop reading now. But remember how we made sure other Synapses could talk to our NGINX? Well, why not put federation behind our new web server too?

Now, we need to do a couple of things to make this work: were you looking carefully at the JSON those curl commands returned? If you were, you might have noticed a key called, 'tls_fingerprints'. Our home server serves up a fingerprint of the TLS certificate its using from this API, and we've just given our web server a different certificate, so we need to give Synapse our new certificate.

How are we going to tell other home servers to talk to our NGINX instead? Well, ultimately we're going to change our DNS SRV record to point at port 443 instead of port 8448, but that change could take a while to propagate through caches, so let's test it by having our NGINX listen on port 8448 temporarily. We can do this by copying that same block from above, but with a different port:

    server {'{'}
        listen 0.0.0.0:8448;
        server_name example.com www.example.com;
    [etc]

Don't restart NGINX just yet: we need to tell our Synapse to stop listening on that port first, so lets do that and give it our new certificate:

$ nano /home/synapse/synapse/homeserver.yaml

Now we'll want to find and edit the following lines:

tls_certificate_path: "/etc/ssl/nginx/examplecom_crt.pem"
# We can comment this out, as long as we set no_tls to true below
# tls_private_key_path: "/whatever/path/synapse/generated"
# PEM dh parameters for ephemeral keys
tls_dh_params_path: "/whatever/path/synapse/generated"
# Turn off TLS everywhere (this overrides the listeners section below)
no_tls: True
  - port: 8008
    tls: false
    # We can bind to only localhost since only the local nginx needs to hit this
    bind_address: '127.0.0.1'
    type: http
    # Set this so that Synapse obeys nginx's X-Forwarded-For headers, then IP addresses will be correct in Synapse's logs
    x_forwarded: true
    resources:
      - names: [client, webclient]
        compress: true
      - names: [federation]
        compress: false

Note: if you have an old enough config file that you have 'bind_host' and 'bind_port' directives, now is the time to remove them.

Now let's restart Synapse and our web server to swap over what's listening on our port 8448:

$ synctl restart
# nginx -s reload

Now let's try that test again on port 8448:

$ curl https://example.com:8448/_matrix/key/v2/server/auto
{'{'}"old_verify_keys":{'{'}{'}'},"server_name":"example.com","signatures":{'{'}"example.com":{'{'}"ed25519:auto":"bdca31805e4209f6ff4d644251a29d0cb1dc828a4d6131c57cf8305288f337c0"{'}'}{'}'},"tls_fingerprints":[{'{'}"sha256":"1d9ec66599e229654a79f28e26675fdeb585027553af6d581926e821a6b6527c"{'}'}],"valid_until_ts":1455203001035,"verify_keys":{'{'}"ed25519:auto":{'{'}"key":"1YiTDjmE86AlmrbIYE2lyqauV9wPo8jw2kxZAZFfl/Q"{'}'}{'}'}{'}'}

Notice anything different? The tls_fingerprints part has changed because we now have a different certificate. The signatures/example.com/ed25519:auto value has changed too: that's because that part is a signature of the rest of JSON object, so changing the tls_fingerprints has caused this to change too.

And that's it! If you're happy everything is working, you can then change your DNS SRV record to point at port 443 instead of 8448, then after leaving a few days for the change to propagate through caches, remove the extra server block from your nginx.conf and restart to stop your nginx listening on port 8448.

Hi all,

Synapse 0.13 was released this afternoon, bringing a new wave of features, bug fixes and performance fixes. The main headlines include: huge performance increases (big catchup /syncs that were taking 20s now take 0.3s!), support for server-side per-room unread message and notification badge counts, ability for guest accounts to upgrade into fully-fledged accounts, change default push rules back to notifying for group chats, and loads of bug fixes. This release incorporates what-was 0.12.1-rc1.

Please note that on first launch after upgrading a pre-0.13 server to 0.13 or later, synapse will add a large database index which may take several minutes to complete. Whilst the index is added the service will be unresponsive.

Please get the new release from https://github.com/matrix-org/synapse and have fun!

Matthew

Full release notes:

こんにちは

Matrix is on its way to Japan where Kegan is attending the TADHack-mini (Feb 13th and 14th) and WebRTC Conference (Feb 16th and 17th).

Kegan will help hackers with their projects during the TADHack, but first, he will give a talk on Matrix and how it can be used. We are again awarding a trossen robot to the best hack using Matrix, and we are as always curious to see what kind of cool and crazy ideas people will come up with!

A couple of days later, Kegan will be giving a talk during the WebRTC Conference: "The missing signalling layer for WebRTC".

Both of the talks will be live-translated, and there will also be a translator available during the events, so please come and say hello to Kegan-san! As always, we are also available in the Matrix HQ room, via a client like Vector or any other client!

The Matrix team had a very successful trip to FOSDEM '16 last weekend. Many, many words were exchanged, and at times there was a queue of people just waiting to have a chat! We spoke to a whole lot of interested and interesting people, some of which had heard of Matrix already, and some who hadn't. The nice thing with the crowd at FOSDEM is that they very quickly "get" what we're trying to do with Matrix - and then start thinking aloud about how they might want to use it or extend it - which means we have many great conversations!

At the beginning of FOSDEM, I think we had around 2000 people "currently" in the Matrix HQ room - the next day, that number had increased by a hundred (and now it's even bigger due to a hackernews post where we mentioned Matrix)!

Our talk in the IoT dev room was very popular; unfortunately a lot of people that were queueing to get in never made it due to the limited space. However, the talk was recorded, and it's already been made available:

You can also look at the slides from the talk. Note that due to technical problems, the first half of the talk was not recorded.

We also did a talk in the Real Time dev room, which was equally popular. Hopefully the video from this talk will also be available shortly (you can keep an eye on the video repository for this dev room) - once it is, I will add it to this post.

For now, you can look at the slides from the talk.

Another interesting talk where Matrix ended up being represented, was Daniel Pocock's Improving Telepathy talk. I'll add the video from that too, once it's available.

Finally, thanks to everyone who came to say hello and have a chat - and to everyone who has since set up their own homeserver. Please do join the Matrix HQ room (using any of these clients) and let us know how your Matrix experience is going!

Matrix will again be represented at FOSDEM (Free and Open Source Software Developers' European Meeting) in Brussels, Belgium, January 30th and 31st. This year, we have two talks scheduled: one in the IoT devroom on Saturday at noon, and one in the Real Time devroom at 3.10pm the same day.

We also have a stand on the 2nd floor of the K building - next to the Real Time lounge (potentially same spot as last year). If you're going to FOSDEM, please come and say hi to us at the stand or at the talks!

Our trip to FOSDEM '15 was very enjoyable; we met a lot of smart people and learnt about a lot of interesting technologies and projects. Hopefully we made a few people enthusiastic about Matrix as well!

As always, for any questions or comments, please come talk to us in the official Matrix HQ room - you can join via any of these clients (or write your own!).

Happy 2016 everyone!

To greet the new year, we bring you all new Synapse 0.12. The focus here has been a wide range of polishing, bugfixes, performance improvements and feature tweaks. The biggest news are that the 'v2' sync APIs are now production ready; the search APIs now work much better; 3rd party ID invites now work; and we now mount the whole client-server API under the /_matrix/client/r0 URI prefix, as per the r0.0.0 release of the Client Server API from a few weeks ago. The r0 release unifies what were previously the somewhat confusing mix of 'v1' and 'v2' APIs as a single set of endpoints which play nice together.

We highly recommend all homeservers upgrading to v0.12.0 as soon as possible. Get it now from https://github.com/matrix-org/synapse/ or via our shiny new Debian packages at https://matrix.org/packages/debian/.

Full changelog follows:

🔗Changes in synapse v0.12.0 (2016-01-04)

• Expose /login under r0 (PR #459)

🔗Changes in synapse v0.12.0-rc3 (2015-12-23)

• Allow guest accounts access to /sync (PR #455)
• Allow filters to include/exclude rooms at the room level rather than just from the components of the sync for each room. (PR #454)
• Include urls for room avatars in the response to /publicRooms (PR #453)
• Don't set a identicon as the avatar for a user when they register (PR #450)
• Add a display_name to third-party invites (PR #449)
• Send more information to the identity server for third-party invites so that it can send richer messages to the invitee (PR #446)
• Cache the responses to /initialSync for 5 minutes. If a client retries a request to /initialSync before the a response was computed to the first request then the same response is used for both requests (PR #457)
• Fix a bug where synapse would always request the signing keys of remote servers even when the key was cached locally (PR #452)
• Fix 500 when pagination search results (PR #447)
• Fix a bug where synapse was leaking raw email address in third-party invites (PR #448)

🔗Changes in synapse v0.12.0-rc2 (2015-12-14)

• Add caches for whether rooms have been forgotten by a user (PR #434)
• Remove instructions to use --process-dependency-link since all of the dependencies of synapse are on PyPI (PR #436)
• Parallelise the processing of /sync requests (PR #437)
• Fix race updating presence in /events (PR #444)
• Fix bug back-populating search results (PR #441)
• Fix bug calculating state in /sync requests (PR #442)

🔗Changes in synapse v0.12.0-rc1 (2015-12-10)

• Host the client APIs released as r0 by https://matrix.org/docs/spec/r0.0.0/client_server.html on paths prefixed by/_matrix/client/r0. (PR #430, PR #415, PR #400)
• Updates the client APIs to match r0 of the matrix specification.
  • All APIs return events in the new event format, old APIs also include the fields needed to parse the event using the old format for compatibility. (PR #402)
  • Search results are now given as a JSON array rather than a JSON object (PR #405)
  • Miscellaneous changes to search (PR #403, PR #406, PR #412)
  • Filter JSON objects may now be passed as query parameters to /sync (PR #431)
  • Fix implementation of /admin/whois (PR #418)
  • Only include the rooms that user has left in /sync if the client requests them in the filter (PR #423)
  • Don't push for m.room.message by default (PR #411)
  • Add API for setting per account user data (PR #392)
  • Allow users to forget rooms (PR #385)
• Performance improvements and monitoring:
  • Add per-request counters for CPU time spent on the main python thread. (PR #421, PR #420)
  • Add per-request counters for time spent in the database (PR #429)
  • Make state updates in the C+S API idempotent (PR #416)
  • Only fire user_joined_room if the user has actually joined. (PR #410)
  • Reuse a single http client, rather than creating new ones (PR #413)
• Fixed a bug upgrading from older versions of synapse on postgresql (PR #417)