by Matthew Hodgson | Jul 4, 2016 | Uncategorized |
Hi folks – another few months have gone by and once again the core Matrix team has ended up too busy hacking away on the final missing pieces of the Matrix jigsaw puzzle to have been properly updating the blog; sorry about this. The end is in sight for the current crunch however, and we expect to return to regular blog updates shortly! Meanwhile, rather than letting news stack up any further, here’s a quick(?) attempt to summarise all the things which have been going on!
Synapse 0.16.1 released!
This one’s a biggy: in the mad rush during June to support the public debut for Vector, we made a series of major Synapse releases which apparently we forgot to tell anyone about (sorry!). The full changelog is at the bottom of the post as it’s huge, but the big features are:
- Huge performance improvements, including adding write-thru event caches and improving caching throughout, and massive improvements to the speed of the room directory API.
- Add support for inline URL previewing!
- Add email notifications!
- Add support for LDAP authentication (thanks to Christoph Witzany)
- Add support for JWT authentication (thanks to Niklas Riekenbrauck)
- Add basic server-side ignore functionality and abuse reporting API
- Add ability to delegate /publicRooms API requests to a list of secondary homeservers
- Lots and lots and lots of bug fixes.
If you haven’t upgraded, please do asap from https://github.com/matrix-org/synapse!
There’s also been a huge amount of work going on behind the scenes on horizontal scalability for Synapse. We haven’t drawn much attention to this yet (or documented it) as it’s still quite experimental and in flux, but the main change is to add the concept of application-layer replication to Synapse – letting you split the codebase into multiple endpoints which can then be run in parallel, each replicating their state off the master synapse process. For instance, right now the Matrix.org homeserver is actually running off three different processes: the main synapse; another specific to calculating push notifications and another specific to serving up the /sync endpoint. These three are then abstracted behind the dendron layer (which also implements the /login endpoint). The idea is that one can then run multiple instances of the /sync and pusher (and other future) endpoints to horizontally scale. For now, they share a single database writer, but in practice this has improved our scalability and performance on the Matrix.org HS radically.
In future we’ll actually document how to run these, as well as making it easy to spin up multiple concurrent instances – in the interim if you find you’re hitting performance limits running high-traffic synapses come talk to us about it on #matrix-dev:matrix.org. And the longer term plan continues to be to switch out these python endpoint implementations in future for more efficient implementations. For instance, there’s a golang implementation of the media repository currently in development which could run as another endpoint cluster.
Vector released!
Much has been written about this elsewhere, but Web, iOS and Android versions of the Vector clients were finally released to the general public on June 9th at the Decentralised Web Summit in San Francisco. Vector is a relatively thin layer on top of the matrix-react-sdk, matrix-ios-sdk and matrix-android-sdk Matrix.org client SDKs which showcases Matrix’s collaboration and messaging capabilities in a mass-market usable app. There’s been huge amounts of work here across the SDKs for the 3 platforms, with literally thousands of issues resolved. You can find the full SDK changelogs on github for React, iOS and Android. Some of the more interesting recent additions to Vector include improved room notifications, URL previews, configurable email notifications, and huge amounts of performance stability work.
Future work on Vector is focused on showcasing end-to-end encryption, providing a one-click interface for adding bots/integrations & bridges to a room, and generally enormously improving the UX and polish. Meanwhile, there’s an F-Droid release for Android landing any day now.
If you haven’t checked it out recently, it’s really worth a look :)
Matrix Spec 0.1.0
In case you didn’t notice, we also released v0.1.0 of the Matrix spec itself in May – this is a fairly minor update which improves the layout of the document somewhat (thanks to a PR from Jimmy Cuadra) and a some bugfixes. You can see the full changelog here. We’re overdue a new release since then (albeit again with relatively minor changes).
Google Summer of Code
We’re in the middle of the second half of GSoC right now, with our GSoC students Aviral and Half-Shot hacking away on Vector and Microblogging projects respectively. There’s a lot of exciting stuff coming out of this – Aviral contributing Rich Text Editing, Emoji autocompletion, DuckDuckGo and other features into Vector (currently on branches, but will be released soon) and Half-Shot building a Twitter bridge as part of his Matrix-powered microblogging system. Watch this space for updates!
Ruma
Lots of exciting stuff has been happening recently over at Ruma.io – an independent Matrix homeserver implementation written in Rust. Over the last few weeks Jimmy and friends have got into the real meat of implementing events and the core of the Matrix CS API, and as of the time of writing they’re the topmost link on HackerNews! There’s a lot of work involved in writing a homeserver, but Ruma is looking incredibly promising and the feedback from their team has been incredibly helpful in keeping us honest on the Matrix spec and ensuring that it’s fit for purpose for 3rd party server implementers.
Also, Ruma just released some truly excellent documentation as a high-level introduction to Matrix (thanks to Leah and Jimmy) – much better than anything we have on the official Matrix.org site. Go check it out if you haven’t already!
End to End Encryption
There has been *LOADS* of work happening on End to End encryption: finalising the core 1:1 “Olm” cryptographic ratchet; implementing the group “Megolm” ratchet (which shares a single ratchet over all the participants of a room for scalability); fully hooking Olm into matrix-js-sdk and Vector-web at last, and preparing for a formal and published-to-the-public 3rd party security audit on Olm which will be happening during July.
This deserves a post in its own right, but the key thing to know is that Olm is almost ready – and indeed the work-in-progress E2E UX is even available on the develop branch of vector if you enable E2E in the new ‘Labs’ section in User Settings. Olm itself is usable only for ‘burn after reading’ strictly PFS messages, but Megolm integration with Vector & Synapse will follow shortly afterwards which will finally provide the E2E nirvana we’ve all been waiting for :)
Decentralised Web Summit
Matrix had a major presence as a sponsor at the first ever Decentralised Web Summit hosted by the Internet Archive in San Francisco back in June. This was a truly incredible event – with folks gathering from across the world to discuss, collaborate and debate on ensure that the web is not fragmented or trapped into proprietary silos – with the likes of Tim Berners-Lee, Vint Cerf and Brewster Kahle in attendance. We ran a long 2 hour workshop on Matrix and showed off Vector to anyone and everyone – and meanwhile the organisers were kind enough to promote Matrix as the main decentralised chat interface for the conference itself (bridged with their Slack). A full writeup of the conference really merits a blog post in its own right, but the punchline is that you could genuinely tell that this is the beginning of a new era of the internet – whether it’s using Merkle DAGs (like Matrix) or Blockchain or similar technologies: we are about to see a major shift in the balance of power on the internet back towards its users.
We strongly recommend checking out the videos which have all been published at Decentralised Web Summit, including lightning talks introducing both Matrix and Vector, and digging into as many of the projects advertised as possible. It was particularly interesting for us to get to know Tim Berners-Lee’s latest project at MIT: Solid – which shares quite a lot of the same goals as Matrix, and subsequently seeing Tim pop up on Matrix via Vector. We’re really looking forward to working out how Matrix & Solid can complement each other in future.
Matrix.to
Not the most exciting thing ever, but heads up that there’s a simple site up at https://matrix.to to provide a way of doing client-agnostic links to content in Matrix. For instance, rather than linking specifically into an app like Vector, you can now say https://matrix.to/#/#matrix:matrix.org to go there via whatever app you choose. This is basically a bootstrapping process towards having proper mx:// URLs in circulation, but given mx:// doesn’t exist yet, https://matrix.to hopefully provides a useful step in the right direction :)
PRs very welcome at https://github.com/matrix-org/matrix.to.
Bridges and Bots
Much of the promise of Matrix is the ability to bridge through to other silos, and we’ve been gradually adding more and more bridging capabilities in.
For instance, the IRC bridge has had a complete overhaul to add in huge numbers of new features and finally deployed for Freenode a few weeks ago:
New Features:
- Nicks set via
!nick will now be preserved across bridge restarts.
- EXPERIMENTAL: IRC clients created by the bridge can be assigned their own IPv6 address.
- The bridge will now send connection status information to real Matrix users via the admin room (the same room
!nickcommands are issued).
- Added
!help.
- The bridge will now fallback to
body if the HTML content contains any unrecognised tags. This makes passing Markdown from Matrix to IRC much nicer.
- The bridge will now send more metrics to the statsd server, including the join/part rate to and from IRC.
- The config option
matrixClients.displayName is now implemented.
Bug fixes:
- Escape HTML entities when sending from IRC to Matrix. This prevents munging occurring between IRC formatting and textual < element > references, whereby if you sent a tag and some colour codes from IRC it would not escape the tag and therefore send invalid HTML to Matrix.
- User IDs starting with
- are temporarily filtered out from being bridged.
- Deterministically generate the configuration file.
- Recognise more IRC error codes as non-fatal to avoid IRC clients reconnecting unecessarily.
- Add a 10 second timeout to join events injected via the
MemberListSyncer to avoid HOL blocking.
- ‘Frontier’ Matrix users will be forcibly joined to IRC channels even if membership list syncing I->M is disabled. This ensures that there is always a Matrix user in the channel being bridged to avoid losing traffic.
- Cache the
/initialSync request to avoid hitting this endpoint more than once, as it may be very slow.
- Indexes have been added to the NeDB .db files to improve lookup times.
- Do not recheck if the bridge bot should part the channel if a virtual user leaves the channel: we know it shouldn’t.
- Refine what counts as a “request” for metrics, reducing the amount of double-counting as requests echo back from the remote side.
- Fixed a bug which caused users to be provisioned off their
user_id even if they had a display name set.
Meanwhile, a Gitter bridge is in active development (and in testing with the Neovim community on Gitter/Matrix/Freenode), although lacking documentation so far.
Finally, NEB – the Matrix.org bot framework is currently being ported from Python to Golang to act as a general Go SDK for rapidly implementing new bot capabilities.
There’s little point in all of the effort going into bridges and bots if it’s too hard for normal users to deploy them, so on the Vector side of things there’s an ongoing project to build a commercial-grade bot/bridge hosted service offering for Matrix which should make it *much* easier for non-sysadmins to quickly add their own bots and bridges into their rooms. There’s nothing to see yet, but we’ll be yelling about it when it’s ready!
Conclusion
I’m sure there’s a lot of stuff missing from the quick summary above – suffice it to say that the Matrix ecosystem is growing so fast and so large that it’s pretty hard to keep track of everything that’s going on. The big remaining blockers we see at this point are:
- End-to-end Encryption roll-out
- Polishing UX on Vector – showing that it’s possible to build better-than-Slack quality UX on top of Matrix
- Bots, Integrations and Bridges – making them absolutely trivial to build and deploy, and encouraging everyone to write as many as they can!
- Improving VoIP, especially for conferencing, especially on Mobile
- Threading
- Editable messages
- Synapse scaling and stability – this is massively improved, but there’s still work to be done. Meanwhile projects like Ruma give us hope for light at the end of the Synapse tunnel!
- Spec refinements – there are still a lot of open spec bugs which we need to resolve so we can declare the spec (and thus Matrix!) out of beta.
- More clients – especially desktop ones; helping out with Quaternion, Tensor, PTO, etc.
…and then all the pieces of the jigsaw will finally be in place, and Matrix should hopefully fulfil its potential as an invaluable, open and decentralised data fabric for the web.
Thanks, once again, to everyone who’s been supporting and using Matrix – whether it’s by hanging out in the public chatrooms, running your own server, writing your own clients, bots, or servers, or just telling your friends about the project. The end of the beginning is in sight: thanks for believing in us, and thank you for flying Matrix.
Matthew, Amandine & the Matrix Team.
Appendix: The Missing Synapse Changelogs
Changes in synapse v0.16.1 (2016-06-20)
Bug fixes:
- Fix assorted bugs in
/preview_url (PR #872)
- Fix TypeError when setting unicode passwords (PR #873)
Performance improvements:
- Turn
use_frozen_events off by default (PR #877)
- Disable responding with canonical json for federation (PR #878)
Changes in synapse v0.16.1-rc1 (2016-06-15)
Features: None
Changes:
- Log requester for
/publicRoom endpoints when possible (PR #856)
- 502 on
/thumbnail when can’t connect to remote server (PR #862)
- Linearize fetching of gaps on incoming events (PR #871)
Bugs fixes:
- Fix bug where rooms where marked as published by default (PR #857)
- Fix bug where joining room with an event with invalid sender (PR #868)
- Fix bug where backfilled events were sent down sync streams (PR #869)
- Fix bug where outgoing connections could wedge indefinitely, causing push notifications to be unreliable (PR #870)
Performance improvements:
- Improve
/publicRooms performance (PR #859)
Changes in synapse v0.16.0 (2016-06-09)
NB: As of v0.14 all AS config files must have an ID field.
Bug fixes:
- Don’t make rooms published by default (PR #857)
Changes in synapse v0.16.0-rc2 (2016-06-08)
Features:
- Add configuration option for tuning GC via
gc.set_threshold (PR #849)
Changes:
- Record metrics about GC (PR #771, #847, #852)
- Add metric counter for number of persisted events (PR #841)
Bug fixes:
- Fix ‘From’ header in email notifications (PR #843)
- Fix presence where timeouts were not being fired for the first 8h after restarts (PR #842)
- Fix bug where synapse sent malformed transactions to AS’s when retrying transactions (Commits310197b, 8437906)
Performance Improvements:
- Remove event fetching from DB threads (PR #835)
- Change the way we cache events (PR #836)
- Add events to cache when we persist them (PR #840)
Changes in synapse v0.16.0-rc1 (2016-06-03)
Version 0.15 was not released. See v0.15.0-rc1 below for additional changes.
Features:
- Add email notifications for missed messages (PR #759, #786, #799, #810, #815, #821)
- Add a
url_preview_ip_range_whitelist config param (PR #760)
- Add /report endpoint (PR #762)
- Add basic ignore user API (PR #763)
- Add an openidish mechanism for proving that you own a given user_id (PR #765)
- Allow clients to specify a server_name to avoid ‘No known servers’ (PR #794)
- Add secondary_directory_servers option to fetch room list from other servers (PR #808, #813)
Changes:
- Report per request metrics for all of the things using request_handler (PR #756)
- Correctly handle
NULL password hashes from the database (PR #775)
- Allow receipts for events we haven’t seen in the db (PR #784)
- Make synctl read a cache factor from config file (PR #785)
- Increment badge count per missed convo, not per msg (PR #793)
- Special case m.room.third_party_invite event auth to match invites (PR #814)
Bug fixes:
- Fix typo in event_auth servlet path (PR #757)
- Fix password reset (PR #758)
Performance improvements:
- Reduce database inserts when sending transactions (PR #767)
- Queue events by room for persistence (PR #768)
- Add cache to
get_user_by_id (PR #772)
- Add and use
get_domain_from_id (PR #773)
- Use tree cache for
get_linearized_receipts_for_room (PR #779)
- Remove unused indices (PR #782)
- Add caches to
bulk_get_push_rules* (PR #804)
- Cache
get_event_reference_hashes (PR #806)
- Add
get_users_with_read_receipts_in_room cache (PR #809)
- Use state to calculate
get_users_in_room (PR #811)
- Load push rules in storage layer so that they get cached (PR #825)
- Make
get_joined_hosts_for_room use get_users_in_room (PR #828)
- Poke notifier on next reactor tick (PR #829)
- Change CacheMetrics to be quicker (PR #830)
Changes in synapse v0.15.0-rc1 (2016-04-26)
Features:
- Add login support for Javascript Web Tokens, thanks to Niklas Riekenbrauck (PR #671,#687)
- Add URL previewing support (PR #688)
- Add login support for LDAP, thanks to Christoph Witzany (PR #701)
- Add GET endpoint for pushers (PR #716)
Changes:
- Never notify for member events (PR #667)
- Deduplicate identical
/sync requests (PR #668)
- Require user to have left room to forget room (PR #673)
- Use DNS cache if within TTL (PR #677)
- Let users see their own leave events (PR #699)
- Deduplicate membership changes (PR #700)
- Increase performance of pusher code (PR #705)
- Respond with error status 504 if failed to talk to remote server (PR #731)
- Increase search performance on postgres (PR #745)
Bug fixes:
- Fix bug where disabling all notifications still resulted in push (PR #678)
- Fix bug where users couldn’t reject remote invites if remote refused (PR #691)
- Fix bug where synapse attempted to backfill from itself (PR #693)
- Fix bug where profile information was not correctly added when joining remote rooms (PR #703)
- Fix bug where register API required incorrect key name for AS registration (PR #727)
by David Baker | Feb 10, 2016 | Tech, Tutorials, Uncategorized |
So, you’ve installed an configured synapse and started chatting from your very own Matrix home server? What’s the next step? Well, right now you’re probably accessing your new home server over plaintext HTTP, which is bad, particularly because you’ll be sending your password over this connection when you log in. You could connect to Synapse’s secure HTTP port, but your browser won’t trust it by default because you’d normally need to pay for a certificate that your browser would recognise. That is, until recently!
Let’s Encrypt is a new initiative that issues SSL certificates free of charge, in an effort to make SSL universal on the Internet. In this blog post, we’ll be walking through an example of how we can use this service to get ourselves a securely hosted Synapse.
We’re going to assume you have a Synapse installed and listening on the standard ports, 8008 and 8448. If not, follow the Synapse README and come back here when you’re done. Everybody ready? Okay, good.
So, in order to get a certificate from Let’s Encrypt, we need to prove that we own our domain. The simplest way to do this is to host some special files on our web server. Now, Synapse won’t do this. We could use a separate web server, but then we’d have to stop Synapse and start the other web server every time we renewed our certificate, and that means downtime. Instead, let’s put our Synapse behind a proper web server and let that serve the files. This has added advantages, including that we can host our Matrix home server on the standard port 443 without having to run Synapse as root.
For this example, we’re going to use NGINX, so start by installing NGINX in whatever way your Linux distribution of choice recommends.
Now, you should have a webroot for your new web server somewhere. Hopefully your helpful Linux distribution has started you off with a config file – let’s see:
# nano /etc/nginx/nginx.conf
We’re looking for the ‘server’ section of that file. We need to make it look something like this:
server {
# Make sure this is 0.0.0.0: no use listening on 127.0.0.1 or we'll only be
# serving to ourselves! There's no port here, which means we'll listen on
# port 80
listen 0.0.0.0;
server_name example.com www.example.com;
access_log /var/log/nginx/example.com.access_log main;
error_log /var/log/nginx/example.com info;
# This is where we put the files we want on our site
root /var/www/examplecom/htdocs;
# Here's where it gets interesting: This will send any path that starts
# with /_matrix to our Synapse!
location /_matrix {
proxy_pass http://localhost:8008;
}
}
When you’re happy with the look of that file, let’s restart the server:
# nginx -s reload
Before we go any further, let’s test our new configuration:
$ curl http://example.com/_matrix/key/v2/server/auto
{"old_verify_keys":{},"server_name":"example.com","signatures":{"example.com":{"ed25519:auto":"RWb+w6vHUUokoDgElwG6Cg50ezZvBrzXtJmJIH8jEwI5x0JQ7prn3FwjhbgKTH5jE7J8Ily3HEc4COn4JCCvCA"}},"tls_fingerprints":[{"sha256":"DMbzSZ5Uj7/6p/RT/UtQYJLHm5o0TwBSVYXsqpDdVDs"}],"valid_until_ts":1455203001035,"verify_keys":{"ed25519:auto":{"key":"1YiTDjmE86AlmrbIYE2lyqauV9wPo8jw2kxZAZFfl/Q"}}}
Those are your server’s public keys! Now we have a web server running, we can get our SSL certificate. Let’s Encrypt have their own client which will automate everything including rewriting your NGINX config file, however that means it has a large number of dependencies and needs to be run as root. For this example, we’re going to use the much simpler acme_tiny.py. I’m going to assume you have a user called, ‘letsencrypt’, so, as root, let’s set up the place for it to write its challenge files:
# mkdir /var/www/examplecom/htdocs/.well-known/acme-challenge
# chown letsencrypt:users /var/www/examplecom/htdocs/.well-known/acme-challenge
Now let’s switch to our letsencrypt user:
$ ssh letsencrypt@example.com
We’ll start by getting ourselves a copy of acme_tiny.py:
$ git clone https://github.com/diafygi/acme-tiny.git
Now let’s set up a directory structure (let’s say we might want to manage more than one domain someday):
$ mkdir examplecom
$ cd examplecom
$ ln -s /var/www/examplecom/htdocs/.well-known/acme-challenge challenges
Now, we’ll need to generate two keys for Let’s Encrypt, and account key and a domain key. The former is what we use to identify ourselves to Let’s Encrypt and the latter is the key we use to do the actual SSL.
$ openssl genrsa 4096 > letsencrypt_examplecom_account.key
Generating RSA private key, 4096 bit long modulus
..++
.................................................................................................................................................................................................................................................................................................................................................................................................................++
e is 65537 (0x10001)
$ chmod 600 letsencrypt_examplecom_account.key
$ openssl genrsa 4096 > letsencrypt_examplecom_domain.key
Generating RSA private key, 4096 bit long modulus
.............++
.............................................................................................................................................................................................++
e is 65537 (0x10001)
$ chmod 600 letsencrypt_examplecom_domain.key
Now, store those keys somewhere safe! After you’ve done that, let’s generate a certificate request for our domain. Note that we’re requesting one for both example.com and www.example.com: this isn’t strictly necessary for Matrix but could be useful if we want to host a website too.
$ openssl req -new -sha256 -key letsencrypt_examplecom_domain.key -subj "/" -reqexts SAN -config <(cat /etc/ssl/openssl.cnf <(printf "[SAN]\nsubjectAltName=DNS:example.com,DNS:www.example.com")) > examplecom.csr
Okay, we have our keys, our certificate request, and somewhere to host our challenge files, so we’re ready to request a certificate! Be careful about this part and make sure you’ve got everything right, because Let’s Encrypt enforce strict rate limits on the number of certificates you can request for one domain. Here we go:
$ python ~/acme-tiny/acme_tiny.py --account letsencrypt_examplecom_account.key --csr examplecom.csr --acme-dir challenges/ > examplecom.crt
Parsing account key...
Parsing CSR...
Registering account...
Registered!
Verifying example.com...
example.com verified!
Verifying www.example.com...
www.example.com verified!
Signing certificate...
Certificate signed!
Is that it, did it work? Well, let’s see:
$ openssl x509 -in examplecom.crt -noout -text
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
01:02:22:77:02:1b:eb:d5:3d:c3:14:6d:87:43:22:3d:fc:0f
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3
Validity
Not Before: Feb 6 21:37:00 2016 GMT
Not After : May 6 21:37:00 2016 GMT
Subject: CN=example.com
Subject Public Key Info:
[etc]
Congratulations, you have an official, signed certificate for your domain! Now, before we can use it, we need to add the Let’s Encrypt certificate to it, because our web server needs to send both:
$ wget https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem
--2016-02-06 23:38:55-- https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem
Resolving letsencrypt.org... 23.66.17.98, 2a02:26f0:60:489::2a1f, 2a02:26f0:60:481::2a1f
Connecting to letsencrypt.org|23.66.17.98|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1675 (1.6K) [application/x-x509-ca-cert]
Saving to: ‘lets-encrypt-x3-cross-signed.pem’
lets-encrypt-x3-cross-signed.pe 100%[======================================================>] 1.64K --.-KB/s in 0s
2016-02-06 23:38:55 (61.5 MB/s) - ‘lets-encrypt-x3-cross-signed.pem’ saved [1675/1675]
$ cat examplecom/examplecom.crt lets-encrypt-x3-cross-signed.pem >examplecom/examplecom_cert_chain.crt
Now’s let’s symlink it in place, along with the domain key, so we can renew it easily later. We’ll need to be root again for this:
$ ssh root@example.com
# ln -s /home/letsencrypt/examplecom/examplecom_cert_chain.crt /etc/ssl/nginx/examplecom_cert.pem
# ln -s /home/letsencrypt/examplecom/letsencrypt_examplecom_domain.key /etc/ssl/nginx/examplecom_key.pem
Now, one more crucial thing we have to do before using our SSL is to give NGINX some Diffie Hellman parameters. This is a good thing to do for any SSL configuration (it will increase your score on SSL Labs) but it’s absolutely crucial for us because Synapse will only negotiate forward secret connections, so otherwise other Matrix home servers will refuse to talk to us! (Technically, Synapse also support elliptic curve Diffie Hellman, which doesn’t need DH parameters, but not all Synapses will support this.) You’ll already have some Diffie Hellman parameters from you existing Synapse, so you could use them:
# cp /home/synapse/synapse/matrix.example.com.tls.dh /etc/ssl/nginx/examplecom_dhparams.pem
…or you can generate your own. You’ll probably want to do this on your desktop or laptop if you have OpenSSL installed, it will be much faster:
$ openssl dhparam -out examplecom_dhparams.pem 2048
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
........................................................+................[etc, etc]
$ scp examplecom_dhparams.pem root@example.com:/etc/ssl/nginx/examplecom_dhparams.pem
Now, let’s get our new certificate in action! Open up your NGINX config file again, and add another server block that look like this:
server {
listen 0.0.0.0:443;
server_name example.com www.example.com;
ssl on;
ssl_certificate /etc/ssl/nginx/examplecom_crt.pem;
ssl_certificate_key /etc/ssl/nginx/examplecom_key.pem;
ssl_dhparam /etc/ssl/nginx/examplecom_dhparams.pem;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
# mozilla intermediate list, jan 2016
ssl_ciphers "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA";
ssl_session_cache shared:SSL:50m;
access_log /var/log/nginx/examplecom.ssl_access_log main;
error_log /var/log/nginx/examplecom.ssl_error_log info;
root /var/www/examplecom/htdocs;
location /_matrix {
proxy_pass http://localhost:8008;
}
}
It looks pretty similar to our previous server block, except for all that stuff about SSL in the middle. We’re pointing NGINX at our certificate, key and Diffie Hellman parameter files and specifying what protocols and ciphers we want our server to talk. The long list here is taken from Mozilla’s Server Side TLS guidelines and is their ‘Intermediate’ list. See that page for more information on what that means, and choose a different list of ciphers if you prefer: just remember we must support at least the ephemeral Diffie Hellman ciphers, or other home servers won’t talk to us!
Now let’s restart our NGINX and see if it works:
# nginx -s reload
…and that command again, this time with https:
$ curl https://example.com/_matrix/key/v2/server/auto
{"old_verify_keys":{},"server_name":"example.com","signatures":{"example.com":{"ed25519:auto":"RWb+w6vHUUokoDgElwG6Cg50ezZvBrzXtJmJIH8jEwI5x0JQ7prn3FwjhbgKTH5jE7J8Ily3HEc4COn4JCCvCA"}},"tls_fingerprints":[{"sha256":"DMbzSZ5Uj7/6p/RT/UtQYJLHm5o0TwBSVYXsqpDdVDs"}],"valid_until_ts":1455203001035,"verify_keys":{"ed25519:auto":{"key":"1YiTDjmE86AlmrbIYE2lyqauV9wPo8jw2kxZAZFfl/Q"}}}
Hooray! You should now be able to open a browser to https://example.com/matrix/ and log in securely over SSL!
Renewing Your Certificate
Now, there’s one important step left, and that’s to set up renewal for the certificate, otherwise we’ll find our shiny new SSL will stop working in three months time. We can use the same acme_tiny command to do this:
$ python ~/acme-tiny/acme_tiny.py --account letsencrypt_examplecom_account.key --csr examplecom.csr --acme-dir challenges/ > examplecom.crt
Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying example.com...
example.com verified!
Verifying www.example.com...
www.example.com verified!
Signing certificate...
Certificate signed!
$ wget https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem
--2016-02-06 23:38:55-- https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem
Resolving letsencrypt.org... 23.66.17.98, 2a02:26f0:60:489::2a1f, 2a02:26f0:60:481::2a1f
Connecting to letsencrypt.org|23.66.17.98|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1675 (1.6K) [application/x-x509-ca-cert]
Saving to: ‘lets-encrypt-x3-cross-signed.pem’
lets-encrypt-x3-cross-signed.pe 100%[======================================================>] 1.64K --.-KB/s in 0s
2016-02-06 23:38:55 (61.5 MB/s) - ‘lets-encrypt-x3-cross-signed.pem’ saved [1675/1675]
$ cat examplecom/examplecom.crt lets-encrypt-x3-cross-signed.pem >examplecom/examplecom_cert_chain.crt
Synapse will automatically pick up the new certificate, but we’ll need to tell NGINX to reload:
# nginx -s reload
Setting up a cronjob to automate this is left as an exercise to the reader!
Federation behind the HTTP Proxy
If you like, you can stop reading now: our clients can access our home server securely but other home server are still talking to our Synapse directly on port 8448. This is fine, and if you’re happy with this, you can stop reading now. But remember how we made sure other Synapses could talk to our NGINX? Well, why not put federation behind our new web server too?
Now, we need to do a couple of things to make this work: were you looking carefully at the JSON those curl commands returned? If you were, you might have noticed a key called, ‘tls_fingerprints’. Our home server serves up a fingerprint of the TLS certificate its using from this API, and we’ve just given our web server a different certificate, so we need to give Synapse our new certificate.
How are we going to tell other home servers to talk to our NGINX instead? Well, ultimately we’re going to change our DNS SRV record to point at port 443 instead of port 8448, but that change could take a while to propagate through caches, so let’s test it by having our NGINX listen on port 8448 temporarily. We can do this by copying that same block from above, but with a different port:
server {
listen 0.0.0.0:8448;
server_name example.com www.example.com;
[etc]
Don’t restart NGINX just yet: we need to tell our Synapse to stop listening on that port first, so lets do that and give it our new certificate:
$ nano /home/synapse/synapse/homeserver.yaml
Now we’ll want to find and edit the following lines:
tls_certificate_path: "/etc/ssl/nginx/examplecom_crt.pem"
# We can comment this out, as long as we set no_tls to true below
# tls_private_key_path: "/whatever/path/synapse/generated"
# PEM dh parameters for ephemeral keys
tls_dh_params_path: "/whatever/path/synapse/generated"
# Turn off TLS everywhere (this overrides the listeners section below)
no_tls: True
- port: 8008
tls: false
# We can bind to only localhost since only the local nginx needs to hit this
bind_address: '127.0.0.1'
type: http
# Set this so that Synapse obeys nginx's X-Forwarded-For headers, then IP addresess will be correct in Synapse's logs
x_forwarded: true
resources:
- names: [client, webclient]
compress: true
- names: [federation]
compress: false
Note: if you have an old enough config file that you have ‘bind_host’ and ‘bind_port’ directives, now is the time to remove them.
Now let’s restart Synapse and our web server to swap over what’s listening on our port 8448:
$ synctl restart
# nginx -s reload
Now let’s try that test again on port 8448:
$ curl https://example.com:8448/_matrix/key/v2/server/auto
{"old_verify_keys":{},"server_name":"example.com","signatures":{"example.com":{"ed25519:auto":"bdca31805e4209f6ff4d644251a29d0cb1dc828a4d6131c57cf8305288f337c0"}},"tls_fingerprints":[{"sha256":"1d9ec66599e229654a79f28e26675fdeb585027553af6d581926e821a6b6527c"}],"valid_until_ts":1455203001035,"verify_keys":{"ed25519:auto":{"key":"1YiTDjmE86AlmrbIYE2lyqauV9wPo8jw2kxZAZFfl/Q"}}}
Notice anything different? The tls_fingerprints part has changed because we now have a different certificate. The signatures/example.com/ed25519:auto value has changed too: that’s because that part is a signature of the rest of JSON object, so changing the tls_fingerprints has caused this to change too.
And that’s it! If you’re happy everything is working, you can then change your DNS SRV record to point at port 443 instead of 8448, then after leaving a few days for the change to propagate through caches, remove the extra server block from your nginx.conf and restart to stop your nginx listening on port 8448.
Recent Comments