On April 14th, the Spec Core Team conducted a long-overdue retrospective
about the things that were working in the
Matrix Spec Proposal process,
and those that were not.
The most glaring item on the list was the sluggish pace that many Matrix
Spec Changes (MSCs) take throughout the proposal process, as well as the
general lack of activity from the Spec Core Team members on proposals that
have not yet started a Final Comment Period.
We deeply apologize for the frustration this has likely caused many MSC
authors, and want to shed some light on the reasoning behind it, and what we
plan to do to prevent leaving authors in the dark about why there may be no
Spec Core Team activity on their proposal.
Proposal Triaging
There are currently 136 open MSCs that have yet to undergo Final Comment
Period (FCP), 75 of which are marked as proposal-in-review, and 20 that have
a FCP proposed. Relative to the 65 MSCs that have ever been closed, this is a
lot of outstanding ideas, features and maintenance changes.
The Spec Core Team itself is made up of 8 members, each of which have
separate full-time jobs. All team members are well-placed to be on the team
given their wide breadth of knowledge across the Matrix ecosystem,
however the majority are some of the most busy pushing forward Matrix's
reference implementations - without which, Matrix will unquestionably fail.
This limits the amount of MSCs that the team can effectively work on at a
given time.
But there is also a large backlog of MSCs that provide even more fundamental
fixes and additions to the protocol that the team needs to prioritise. These
include things like
cross-signing devices,
the communities rewrite and
finally merging
reactions and edits into the spec.
While we announce what MSCs we're focusing on during a given week during
TWIM, it's not as clear which items we're looking
to pull from the backlog next. To help tackle this, and to help keep us
honest, we've begun putting each MSC into either "feature", "maintenance", or
"core" buckets. This materialises in the form of github tags, which can be
used to filter the list of MSCs like so:
feature,
maintenance,
core.
For a given timespan, we’ll pick a track and pull MSCs out of that category
when possible. More information about MSC categories are now detailed on
the proposals page.
As for the next 6 to 12 months, we plan to work on items from the “core”
category. We need to get Matrix to a point where it can compete with other,
proprietary chat protocols and items in "core" are decidedly the proposals
that will take us the furthest in that direction. This doesn't mean we won't
occasionally look at an MSC in a different category, but it will heavily
influence our prioritisation.
Future
We'll try this approach out over the next few months and see how it goes. The
next Spec Core Team retro will occur in the middle of May, where we will
review the process once again.
For now, if you have any feedback please come and chat with us in
#matrix-spec:matrix.org :)
Some snapshots of what the team has been working on. Look out for a clear explanation from Erik about Synapse performance (and what will be done about it!), and lots of other goodies
Rescue18 (a French non-governmental news website for emergency services) echoed this announcement earlier this month, praising the security aspect of the platform and the possibility it offers to improve social and professional exchanges throughout the hierarchy as well as the rest of the government and public services.
Article is all in some other language - thanks Brendan for providing this summary.
We've switched up our focus to the following MSCs: MSC2366 (verification flows), MSC2454 (SSO UI Auth), and MSC2472 (Symmetric SSSS). Expect a blog post next week describing the new MSC categories.
Shipped 1.12.4 a patch release to fix some cross signing related bugs.
Redis support to replace our existing cross replication system, merged to develop. This is a precursor to the sharding project. We have a PR to support cache invalidation across workers which, once merged should make it relatively easy to pull off event persistence from the master process. Just removing event persistence could give some decent perf wins, but the prize is then splitting persistence to run across across multiple workers.
A final PR to support SSO for user interactive auth will merge imminently, thereby freeing up full cross signing support for worker installs.
Aside from that we’ve been working on fixing a weird bug where rooms can end up with zero forward extremities and generally showing Sygnal some love.
Upcoming
Put Redis into production
Ship 1.13.0 which will contain Redis support as well as a host of SSO bug fixes.
Merge our new caching config logic to give finer granularity on cache settings.
Merge cache invalidation and try to get events persistence running in its own worker.
Ben added:
Cross signing, oh cross signing! Can you believe it‽ Cross signing!!!
Matrix Corporal
matrix-corporal manages your Matrix server according to a configuration policy
matrix-corporal v1.9.0 has been released. It adds support for preventing certain users from creating rooms (that is, from using the /createRoom API). I've been thinking about developing this feature for a while now, as people have been requesting it frequently.
matrix-media-repo
matrix-media-repo is a highly customizable multi-domain media repository for Matrix
This week I worked a lot on getting the first bit of federation working. My goal was to query the public room list of another server and forward it to a client.
This involved setting up a keypair to sign the json I send to the server and putting authorization headers into my requests. Most of the work is already done and https://federationtester.matrix.org/ reports it's all good.
Sadly there seems to be a problem with the signing, because other homeservers reject requests: Unable to verify signature: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt. I'll continue to work on this in the next week(s).
Dendrite now supports perspective key fetching as a configurable option, and will automatically attempt to use the perspective key servers for keys that couldn't be fetched directly
Dendrite now uses a newer version of the Sarama library
Some caching has been added to the roomserver API for room versions, which should help quite a bit with performance over federation and handling transactions
Auth checks for /send_join are now quite a bit more robust
A bug has been fixed where the default state filter limited the amount of room state that was visible after a join
gomatrixserverlib has quite a bit of new support for backfilling now, which will be wired into Dendrite soon
gomatrixserverlib has new primitives for v2 invite responses and /get_missing_events requests
Invite support is being worked on currently, which should also enable support for creating DMs in Riot
Matrix native integration for the Sailfish OS, KDE and other linux systems called telepathy-tank got attertion from the community this week and @aa13q updated it a bit thanks to @Kaffeine. Writing the post for the TWIM has grown into the small blog post, so @aa13q shared it sepatetely: https://hackmd.io/@aa13q/telepathy-tank
Nheko
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
Nheko released two new version, 0.7.0 and 0.7.1. The second one mostly fixes some minor issues in the 0.7.0 release, which we got from all the people trying it out. Thanks for reporting them!
We hope you will try it out and have a good experience! If you don't, you can complain to us in #nheko-reborn:matrix.org or open an issue in our bugtracker.
gomuks got a fancy initial sync loading bar so it no longer looks like it's broken after you log in. Implementing end-to-end encryption in mautrix-go has also been slowly progressing, might have something working in a few weeks.
Riot Web
A glossy Matrix collaboration client for the web. https://riot.im
This week we released a fourth RC for Riot Web 1.6, the upcoming release with cross-signing, addressing a few issues found through everyone's testing and feedback. Please do continue to test https://riot.im/develop, the latest RiotX build, and also E2EE search in Riot Desktop Nightly.
I created an client-sdk to interact with matrix in Spring Boot. It is written in kotlin and should also work with Java. You can find the repo here: https://github.com/benkuly/matrix-spring-boot-sdk
It contains also a bot-sdk to write bots by adding a dependency and a few lines of code only.
It is tested with unit-tests, but integration-tests against a real matrix-server are currently missing. A simple ping bot works and is also integrated in the repo as example.
Currently I'm working on getting this to Maven Central.
Dept of Ops 🛠
matrix-docker-ansible-deploy
This Ansible playbook is meant to easily let you run your own Matrix homeserver.
As per the previous edition of TWIM, we had a blast on Sunday watching and reacting to Doctor Who episodes The Stolen Earth and Journey's End.
The next watchalong will be this Friday (Apr 24th), and we'll be watching another two-parter together, Human Nature & The Family of Blood, starting from 7PM BST! Guests such as Paul Cornell (writer), Charles Palmer (director) & Lor Wilson (Lucy Cartwright aka Daughter of Mine) will also be discussing the episode along for even more fun and behind the scenes!
This means that if you're reading this when this edition of TWIM goes out, and you fancy watching some Doctor Who this evening, join the fun at #doctor-who-watchalong:abolivier.bzh 😀
And if not, feel free to join the room anyway to find out when the next watchalong is happening 🙂
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
fairydust.space
321
2
envs.net
363
3
maunium.net
420
4
nerdsin.space
421
5
maescool.be
470
6
lyseo.edu.ouka.fi
773
7
mjdsystems.ca
820
8
fab.network
832.5
9
eisfunke.com
1048
10
matrix.nilux.be
1275
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
We had our first Spec Core Team retro in a very long time this week, with a new regular schedule of every month. It ran about an hour overtime - there was a lot to talk about. We'll post the conclusion in a separate blog post soon.
Dept of Servers 🏢
Dendrite / gomatrixserverlib
Dendrite is a next-generation homeserver written in Go
This week I spent some time improving Riot support. Now there should be a lot less warnings and some new features: typing events, read receipts, better room directory, invites, user search.
The only bigger bug that exists currently is that the message history is not automatically loaded when you join a room. Sometimes reloading the page helps, but not always. The reason for that is that Riot calls the deprecated /initialSync endpoint, which I do not support (I hope someone works on Riot to fix that).
In the next few weeks I plan to improve the single-homeserver experience further and add permission management, while also working on federation.
I also want to thank my supporters on Liberapay for their kindness 😊.
I tested the performance of Conduit yesterday by running a ruma client that sends a /send request in a for loop 10k times, which took ~46.8 seconds (all locally on my pc). This means the server can handle ~210 /sends per second under these circumstances.
The flamegraph shows what the cpu is spending most of its time doing. Plateaus are things that take time. You can see most of the time is used for a function in sled::io (I spoke with one of the devs and they said this is expected, but will be improved in the next release)
We continue to focus on performance and spent a good chunk of time improving worker replication test coverage to build confidence that our upcoming changes will work as expected. Erik has been working on how to how to factor out cache invalidation which is now starting to take good shape. Once we have that, factoring out things like event persistence are next on the horizon.
Aside from that we’ve been working with the Riot client teams as they near release for cross signing, fixing bugs and improving performance on key actions.
SSO support continues to improve, and more fixes for user inactive auth for worker'd installs will land next week. Poor old Rich has been banging his head against a sync bug blocking develop, but hopefully he has broken its back now, and we'll get that merged asap.
Finally we experienced some operational issues with push over the weekend, so we spent some time cleaning up and figuring out how to make things more resilient.
The Construct server made significant progress this week with end-to-end encryption, client compatibility, application services support, and build-related enhancements. I'd like to thank Yan Minari for contributing spec-compliant code paths for application services in the user registration system. Special thanks this week to Tony O. for continued work on the NixOS (#nix:matrix.org) derivation for reproducible builds, in addition to contributing fixes to Nio for optimal compatibility with Construct. I'd also like to thank Wessel for contributing a much needed reorganization of the Dockerfiles in preparation for Construct operating on many more platforms. Many thanks to Nico for working with the team this week in #test:zemos.net on various compatibility fixes to mtxclient and Nheko; the responsiveness when using Nheko with Construct is mind-blowing. I'd also like to thank David Vo (a.k.a. auscompgeek) for also contributing to Nio toward Construct compatibility.
Finally, I owe much appreciation to tulir, jochen, qy and others for setting up brand new installations this week and providing feedback in #test:zemos.net. Construct is the first and only federating alternative to Synapse. It is written in C++ for maximum performance, the lowest possible TCO, and designed for horizontal scaling with Construct Cluster. If you are interested in starting a low-cost Matrix hosting service, Construct may be your ideal choice. To stay informed and show your support please visit us in #test:zemos.net today!
Which includes a bunch of small fixes, including a performance improvement to StateLookup hits. Apparently it's bad to hit the homeserver with 1000s of concurrent /state requests on startup. Most importantly it also includes a critical fix to the addAppServicePath function, which previously did not authenticate requests, see the CHANGELOG for more information on that.
We're still working on the RCs for this release, having discovered and fixed a bunch of failures that can happen around getting connected to IRC. Be aware that the above security fix is applied in this release, which means provisioning requests will now be authenticated.
This is another big load of bug fixes, with many thanks to our new bridge crew member, Christian!. This release includes the above security fixes, as well as some metrics improvements to track activity on the bridge.
A clickable command-line client Matrix written in Rust. Although, still in the early stages, enough has been implemented to show a working demo. rumatui uses tui to power the UI and matrix-rust-sdk for all communication with the server. So far the message formatting has been the most visually satisfying feature, although implementing it was not much fun. The goal of the project is to have a UI driven command-line client, useable from either keyboard and/or mouse.
Big thanks to @poljar for allowing me to work on matrix-rust-sdk, @jplatte for the opportunity to work with the ruma crates and everyone in the Matrix community for being so helpful!
Check it out https://github.com/DevinR528/RumaTui! Help is welcome, although as matrix-rust-sdk develops the design and structure of rumatui will evolve quickly.
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
0.7.0 will contain a lot of work, including replies, e2ee attachments and a completely rewritten timeline (the full changelog is too big for TWIM, but you can find it in the github releases). It also fixes a lot of issues. We should probably released it a lot earlier, but we always had one more feature we wanted to finish... For the future we plan to do smaller, more regular releases!
If you are a packager, it would be awesome if you can check, that you don't have any build issues. If you need support, just message us in #nheko-reborn:matrix.org !
Cross-signing and E2EE by default for private rooms is nearly here! 🎉 It would be great to have more people testing this, as we're very close to release, and extra testing will help reveal those last issues. Please help test cross-signing on https://riot.im/develop (suggesting develop since there's already some fixes there beyond the RC on staging) and the latest RiotX build. It would also be great to test E2EE search in Riot Desktop Nightly.
We are finalizing cross-signing, QuadS and bootstrap. We will release a new version next week. We are still working on migration of the database from Realm to sqldelight solution.
Let me copy Riot-Web and RiotX-Android: Cross-signing and E2EE by default for private rooms is nearly here! 🎉.
A TestFlight will be available over the weekend. Here is the link if you want to join the program. This release candidate still needs some polish but a new release will be definitely available on the AppStore next week.
In parallel of that, the maintenance work is going well. We are erasing our technical debt as much as possible. The new Apple Push API raises some privacy concerns. We may no longer be able to offer CallKit as it requires events to go in clear between servers.
Dept of SDKs and Frameworks 🧰
Ruma
Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.
Many of you will probably have heard already, but Ruma is now officially dead. The homeserver, that is. Outside of that, the API definitions in ruma-client-api have seen some work and our first two endpoints in ruma-federation-api have been written! ruma-api and ruma-identifiers are also being worked on, with the following new releases being published in the past week:
ruma-identifiers 0.15.0 was released with loads of changes; this might be the last 0.x release!
identifiers are now stored as strings (with minimal additional data)
before, we would parse the server name into a url::Host
this means that parsing now doesn't allocate at all under some circumstances, and obtaining the string representation of one of our identifier types is essentially a no-op
there no longer is a single non-optional dependency, everything that requires them is now behind a feature flag
some new helper functions have been made available, mostly around user IDs
We've been doing Matrix editions of Emily Cook's Doctor Who watchalongs since confinement started around Europe. The concept is pretty simple: on a given day, at a given time, everyone presses play on the same episode. Folks are then free to then watch it on their own, or discuss the episode as they watch it in the Matrix room and/or in the Jitsi call. We're also bridging tweets sent from directors, writers, comedians, etc. watching the episode at the same time to Matrix.
The next watchalong is planned for next Sunday (Apr 19th) at 7PM BST; and we'll be watching the epic two-episode long season finale "The Stolen Earth" & "Journey's End", with Russell T Davies (writer), Graeme Harper (director), David Tennant (The Doctor) and Catherine Tate (Donna Noble) tweeting along (and possibly more to come) 😀
To relay the tweets from guests during Doctor Who watchalongs, I've written a simple bot that reads the timeline of a Twitter list, optionally filters it by a hashtag, and posts the tweet to a Matrix room.
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
envs.net
394
2
maunium.net
437
3
fairydust.space
472
4
maescool.be
493
5
dodsorf.as
495
6
nerdsin.space
540
7
c.mau.dev
569
8
matrix.vgorcum.com
571
9
digi.co.hu
623.5
10
elcyb.org
716
Final Thoughts 💭
While discussing the use of QML in Nheko, aa13q became extremely energised, so I'm including this piece:
Qt/QML fanboy here, feel free to skip, just a love letter to the framework :D
Qt/QML is super-awesome, it's hard to describe how pretty declarative syntax is and how beautiful MVC approach with native Qt C++ side integration %) It is very hard to contribute to HTML/CSS/JS projects when you have seen QML once.
Talking about resources, there are even projects for very low-level embedded boards with arm cortex m7 core [1,2]
I'm actually thinking Qt is the bright future of very wide development areas. I'm both frightened and intrigued looking at relatively low popularity of the project. Since I want it to live and prosper even more but afraid of the possible mistakes for even harder tasks. The better popularity of the project will be harder to coordinate and the future tasks will be even harder. Similar to other cool projects (like matrix! :) I guess it's very challenging to balance between running as hard as you can and not to "die" from exhaustion %)
I don't even know is there something similar comparing to the Qt project, that's also why I'm worried about the future of the project %)
So-called software crisis (the late 60s – mid-80s [3,4]) was successfully overcome by introducing Object-Oriented Programming (it's always funny to see some people who are more into functional programming hates OOP in that context [5]) And notably Alan Kay is not only OOP pioneer (Smalltalk language) but also a first GUI applications developer. While the current OOP languages like Java and C++ are quite imperative, OOP is not only about the objects but also about the messages between them (Smalltalk is probably a nice example here too).
Qt in that context restores interesting balance to quite imperative C++ introducing the so-called meta-object system (for the messaging between objects). After that in ~Qt5, the declarative QML approach was introduced to develop these ideas even more. Not only GUI subject matter could be expressed with QML (like Qt Quick), but the others too (for example, QBS build system).
Sorry for typos and poor style, not a native speaker :) And I could be wrong in my views.
This week we held the first in a series of virtual meetup events, Open Tech Will Save Us.
We were really excited to make this event feel like a real meetup - with a live chat, and interactivity from the chance to ask questions of the speakers. There was a great response, with participants in a 100+ user room chatting about the talks and generally adding to the atmosphere.
Lineup included:
Saúl Ibarra Corretgé from Jitsi on the rapid growth they've experienced since the start of the 2020 crisis
This week the Spec Core Team will be focusing on MSC2457 (password invalidation), MSC2454 (SSO UI Auth), and MSC2472 (Symmetric SSSS), which each have proposed FCPs in flight.
Before next TWIM we'll be carrying out a long-overdue retro to go over how we've been doing in the last few months. We'll then continue to do those regularly once a month and post conclusions from those meetings here.
This week was really successful for the homeserver project. Not only does event sending and syncing work properly, but creating/joining rooms, state events and the public room directory works as well. But the biggest archievement I made this week is Riot support.
Big thanks to MTRNord for fixing bugs in Riot web and deploying an instance of Riot with all patches applied and to jplatte and @iinuwa for improving the ruma libraries.
Today we did the first actual test of the new homeserver. We had ~5 people sending and syncing messages in a room and all of them were able to send messages almost instantly. We'll see how well that scales when we implement federation in the future.
The main focus continues to be performance and we are starting to make good progress in figuring out how to split out and shard the event streams. This is a project where most of the bang will come all at once, rather than there being multiple small incremental improvements.
Aside from that we continue to improve the SSO experience, landing UIA support for CAS providers. We also made a small change to the default behaviour so that only room admins can enable e2ee (applies to new rooms only).
Next week, is more of the same. Expect a few short term performance improvements (specifically to help with cross signing UX) and more SSO support.
As of this PR Gitea has the ability to directly send webhooks to Matrix. Currently it's only in the master branch, but should be in Gitea 1.12.0 which is due by May.
script for forwarding unread Wilma messages to Matrix
I wrote a script for forwarding unread Wilma messages to Matrix. Wilma is a website and mobile app for teacher-parent communication. Download the script from https://k2c42.dy.fi/git/wilmatrix.git and give feedback at #wilmatrix:ellipsis.fi
mijutu seems to lament:
Recently Wilma usage increased from occasional messages to everyday messages, so I had to do something.
Wilma has been widely used and well-known in Finland for over a decade, but not sure how well-known it is elsewhere?
There was just an article (not in English) about the origins of Wilma, that was interesting
So someone got an Amiga in the 80s and decided to make a scheduling program for their local school, and add a bit of time and now there is a web interface, login for students, teachers, and parents with messaging, homework, grades and everything
As promised last week, mautrix-facebook and mautrix-hangouts now support end-to-bridge encryption. It works the same way in all bridges, so the instructions from the mautrix-telegram wiki work for all of them.
Hello everyone. The IRC bridge has hit 0.17.0-rc1. This release is lighter than past releases, but fixes a few nasty bugs. Please test and report back :)
Highlights include:
Disconnect a PM room from IRC when another user is invited, and disallow invites to PM rooms.
On name change, inform Matrix users, if their preferred IRC name is taken.
Add ability to deactivate users permanently via the DebugAPI.
Two bugfixes on the !storepass feature, both bugs would cause the user to be unable to use the bridge 😟.
mx-puppet-bridge
mx-puppet-bridge is a general bridging library that supports (double)bridging and relays. The goal is to make it as easy as possible for others to bridge new third-party protocols to matrix. Support roomDonate
Soo.....mx-puppet-bridge also supports plumbed rooms now. That means that this one little library supports all types of bridges mentioned in https://matrix.org/bridges/ Yay!
Somehow it seems we never previously featured untidylamp's ofono SMS/MMS bridge. This is a Python3 project to bridge Matrix and SMS/MMS messages via ofono.
They say:
I'm running this with ubuntu touch on my nexus 5. The goal is to have this hosted on my PinePhone.
We continued to improve the implementation of cross-signing. One of added features is the gossip of the private key of the key backup: when you complete the security on a new sign-in, this new device automatically retrieves all message keys from your key backup. All your e2ee history is available with no additional effort.
Endpoints are continually being updated, and instead of just repeating this vague statement again and again, here is a number: at least 60% of our endpoint definitions are up-to-date with r0.6.0.
We're also getting lots of feedback from conduit's development. Some of things we're now planning to change based on that will require large-scale refactorings that haven't yet started, but will hopefully happen in the coming months.
The matrix-rust-sdk is steadily progressing, improvements have been made on many fronts thanks to devinr528 (better event emitting, easier tests, room name calculation...).
The encryption side of things has been going on as well and a sneak peek can be found here https://streamable.com/xjb83d#
matrix-docker-ansible-deploy's Jitsi setup has seen lots of improvements around authentication, thanks to teutat3s. Refer to our Jitsi docs page for getting started or to learn how to rebuild your existing Jitsi installation in a more secure manner.
If you want your Sentry issues to a Matrix room, there is now a new bot for that: https://github.com/matrix-org/matrix-sentry-webhooks . Currently it supports a simple "projects to rooms" mapping and should work with both legacy webhook integrations and integration platform webhooks.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
gottliebtfreitag.de
403
2
maescool.be
469
3
nerdsin.space
503
4
envs.net
512.5
5
lyseo.edu.ouka.fi
544
6
maunium.net
576
7
imninja.net
577
8
matrix.vgorcum.com
697
9
kapsi.fi
751.5
10
chat.matrix4me.de
913
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
Over the last few weeks there's been huge increase in interest from folks
wanting the security and autonomy of running their remote collaboration
services, rather than being at the mercy of traditional proprietary
centralised apps. Meanwhile, the Matrix.org homeserver has been very
overloaded (although we're at last making excellent progress in radically
improving Synapse's performance) - so it's particularly important right now to
help folks run their own servers.
Therefore we're very happy to announce that it's easier than ever before now to
self-host your own video conferencing alongside Riot & Synapse: as of Riot/Web
1.5.15 (released last week), it's now a single config option to point Riot at
a specific Jitsi rather than needing to hook up to an integration manager!
Meanwhile, over the last 18 months, it's got easier and easier to run your
own Matrix deployments: the Debian packages are unrecognisably better now, and
with .well-known URL support it's trivial to set up federation without
needing to worry about complicated DNS, TLS or load balancer configurations.
So, to try to show off just how smooth this has become, we thought we'd do a
run-through video showing installing Synapse, Riot & Jitsi on a completely
fresh Debian install. It's (almost) filmed in a single shot, and takes about
20 minutes from beginning to end.
Please note that this does assume you're pretty familiar with Linux system
administration. If you're not, then we'd recommend using a Matrix hosting
provider such as Modular.im (which directly supports development of the core team),
Ungleich.ch, or StartupStack.
Finally, while the video shows how to install on Debian via Debian packages,
there are many many other environments and architectures (e.g. installing
under Docker) - this is just one relatively easy way to skin the cat. Perhaps
there will be other 'speed-run' videos in future :)
If you want to follow along at home without listening to the video (and I can't blame you if you do ;) the high level steps are as follows:
Debian & DNS
Take one fresh Debian 10 install.
Point the DNS for your domain to it. You should use separate subdomains for the various services as a hygiene measure to make cross-site scripting attacks less effective. In this example, we set up DNS for:
dangerousdemos.net (general website, and for hosting a .well-known path to advertise the Matrix service)
matrix.dangerousdemos.net (Synapse)
riot.dangerousdemos.net (Riot/Web)
jitsi.dangerousdemos.net (Jitsi video conferencing)
In practice, we used a *.dangerousdemos.net wildcard DNS record for the three subdomains in this instance.
Nginx and LetsEncrypt
Install nginx as a webserver: apt-get update && apt -y install nginx
Go to /etc/nginx/sites-enabled and copy the vhost configuration block from the bottom of default to new files called dangerousdemos.net, matrix.dangerousdemos.net, and riot.dangerousdemos.net. We don't set up jitsi.dangerousdemos.net at this point as the jitsi installer handles it for us.
Rename the server_name field in the new files to match the hostname of each host, and point root to an appropriate location per domain (e.g. /var/www/dangerousdemos.net for the main domain, or /var/www/riot.dangerousdemos.net/riot for riot)
For the Synapse domain (matrix.dangerousdemos.net here), you should replace the contents of the location block with proxy_pass http://localhost:8008; - telling nginx to pass the traffic through to synapse, which listens by default for plaintext HTTP traffic on port 8008. (N.B. do not put a trailing slash on the URL here, otherwise nginx will mangle the forwarded URLs.)
Enable TLS via LetsEncrypt on nginx, by: apt install -y python3-certbot-nginx && certbot --nginx -d dangerousdemos.net -d riot.dangerousdemos.net -d matrix.dangerousdemos.net (or whatever your domains are).
You should be able to go to https://dangerousdemos.net at this point and see a page with valid HTTPS.
Synapse
Then, install Synapse via Debian packages using the instructions at https://github.com/matrix-org/synapse/blob/master/INSTALL.md#debianubuntu (see below). If you're not on Debian, keep an eye out for all the other OSes we support too!
You should specify the server name to be the domain you want in your matrix IDs - i.e. dangerousdemos.net in this example.
Please report anonymous aggregate stats to us so we can gauge uptake and help justify funding for Matrix!
You should now be able to go to https://matrix.dangerousdemos.net and see a valid "It works! Synapse is running" page.
Then, you should enable registration on your synapse by switching enable_registration: true in /etc/matrix-synapse/homeserver.yaml and restarting synapse via systemctl restart matrix-synapse.
Now you need to tell the rest of Matrix how to find your server. The easiest way to do this is to publish a file at https://dangerousdemos.net/.well-known/matrix/server which tells everyone the hostname and port where they can find the synapse for dangerousdemos.net - in this instance, it's matrix.dangerousdemos.net:443:
mkdir -p /var/www/dangerousdemos.net/.well-known/matrix
cd /var/www/dangerousdemos.net/.well-known/matrix
echo '{ "m.server": "matrix.dangerousdemos.net:443" }' > server
Alternatively, you could advertise the server via DNS, if you don't have write access to /.well-known on your main domain. However, to prove you are allowed to host the Matrix traffic for dangerousdemos.net, you would have to configure nginx to use the dangerousdemos.net TLS certificate for the matrix.dangerousdemos.net vhost (i.e. the "wrong" one), and in general we think that /.well-known is much easier to reason about. In this case you would advertise the server with an SRV record like this:
_matrix._tcp.dangerousdemos.net. 300 IN SRV 10 5 443 matrix.dangerousdemos.net.
Riot/Web
Then, install Riot/Web. Grab the latest .tgz release from https://github.com/vector-im/riot-web/releases. You should check its GnuPG signature too:
mkdir /var/www/riot.dangerousdemos.net
cd /var/www/riot.dangerousdemos.net
wget https://github.com/vector-im/riot-web/releases/download/v1.5.15/riot-v1.5.15.tar.gz
# check its GnuPG signature (particularly advisable, given Riot is what stores
# your end-to-end encryption keys)
apt install -y gnupg
wget https://github.com/vector-im/riot-web/releases/download/v1.5.15/riot-v1.5.15.tar.gz.asc
# grab the signing key for the riot releases repository, ideally from a keyserver...
gpg --keyserver keyserver.ubuntu.com --search-keys [email protected]# ...and/or you can grab or cross-check the signing key from packages.riot.im
wget https://packages.riot.im/riot-release-key.asc
gpg --import riot-release-key.asc
gpg --verify riot-v1.5.15.tar.gz.asc
# hopefully this will report "Good signature", even though it won't know to trust the riot release key.
# you could also choose to explicitly trust the key by editing it, entering 'trust' and then '5' for ultimate trust.
gpg --edit-key 74692659bda3d940
tar -xzvf riot-v1.5.15.tar.gz
ln -s riot-v1.5.15 riot
chown www-data:www-data -R riot
cd riot
cp config.sample.json config.json
You then tweak the config.json to change the base_url of the homeserver to be https://matrix.dangerousdemos.net (i.e. where to find the Client Server API for your server), and change the server_name to be dangerousdemos.net (i.e. the name of your server).
You should then be able to go to https://riot.dangerousdemos.net, register for an account, sign in, and talk to the rest of Matrix!
Jitsi
Finally, we install Jitsi so you can run your own video conferencing. We take the instructions from Jitsi's quick install guide:
We give the installer the hostname jitsi.dangerousdemos.net. Make sure this DNS is already set up, otherwise the installer will fail!
The installer magically detects you have nginx installed and adds in an appropriate vhost!
We select a self-signed certificate for now, and then upgrade it to LetsEncrypt after the fact with /usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh.
Alternatively, you could have specified manual certificates, and then used certbot alongside the rest of nginx to create a certificate for jitsi.dangerousdemos.net - both work.
You should now be able to go to https://jitsi.dangerousdemos.net and use the Jitsi directly.
Finally, and this is the cool new bit: you can now point Riot to use the new Jitsi by going to its config.json at /var/www/riot.dangerousdemos.net/riot/config.json and changing the preferredDomain of the jitsi block from https://jitsi.riot.im to your own self-hosted https://jitsi.dangerousdemos.net.
You then refresh your Riot/Web, and you should be all set to use Jitsi from within your new Riot - as Riot/Web 1.5.15 and later has the ability to natively embed Jitsi straight into the app without needing to use an integration manager.
Conclusion
Matrix nowadays provides an excellent alternative to the centralised solutions. It gives:
Full autonomy over how to host and store your own conversations
Full freedom to talk to anyone else on the wider global Matrix network (or indeed anyone else bridged into Matrix)
Full privacy via full end-to-end-encryption for chats, file transfer and 1:1 voice/video calls (when enabled)
Full transparency by being 100% open source (as well as benefiting from the overall open source community)
Hopefully this gives some confidence that it's pretty easy to run your own fully functional Matrix instance these days.
If not, then hopefully someone will do a similar one to show off Docker!
And if that's still too scary, please take a look at a hosting services like Modular.im.
Featuring p2p, E2EE, FTUE, Open Tech Will Save Us and Mirage
Dept of Status of Matrix 🌡
Open Tech Will Save Us
Open Tech Will Save Us is a virtual meetup, taking the form of a monthly live video stream broadcasting on the second Wednesday of every month at 5pm UTC.
The first event will have speakers from Jitsi, IPFS and Matrix.
We'll cover the importance of preserving privacy and ways to keep your communications under control. Read more at https://matrix.org/open-tech-meetup/
GSoC Student Applications closed
Google closed the door to applications on Tuesday, and we have an absolute bounty of proposals. More information will be shared when the announcements are made in a few weeks.
Not much changing state this week as most people are focusing on implementation and existing MSCs.
Spec Core Team
Next week the Spec Core Team is focusing on the same as last week: MSC2457 (password invalidation), MSC2454 (SSO UI Auth), and MSC2472 (Symmetric SSSS).
Dendrite now has support for sending v2 invites, and partial support for receiving them
The typing server in Dendrite has now been renamed to a more general-purpose EDU server and support for sending/receiving typing notifications has been fixed
Room version v3 and v4 support in Dendrite is mostly finished, hopefully will be merged soon
A new version of the P2P demo was released yesterday with some fixes (more information and discussion in #p2p:matrix.org)
gomatrixserverlib now has new types and support for the new invite format, as used in the v2 endpoint
gomatrixserverlib has received some bugfixes, including with event ID generation and avoiding null in marshalled JSON for auth_events and prev_events
Construct
Update from Jason:
End-to-end encryption support has landed in Construct. The server now supports cross-signing, secret storage, and key backups for clients. I'm pleased with how this all came together on schedule for the upcoming transition to e2ee by default for Matrix. If you haven't been paying attention in #construct:zemos.net last week full support for push-rules including efficient highlight-counting also landed. Next week Application Service support is scheduled.
Construct is quickly nearing complete coverage over the full breadth of the Matrix protocol, and the transition into release packaging is fast approaching. Construct is the C++ homeserver built for maximum performance with the lowest possible cost of ownership. If you're in need of a faster homeserver please show your support in #construct:zemos.net and tell all your friends to lend a hand today!
Hello! This week I worked on a Matrix homeserver written in Rust. Registering and logging in works already and I am currently adding support for sending events and inserting them into the event graph. I spent most of my time reading the Matrix specification to understand how this process works, because with federation (which I plan to add to my homeserver in the future) servers might disagree on which event was created first and there are strictly defined rules to resolve this. The Matrix homeserver uses the Ruma libraries as it's base and handles requests and responses using the Rocket crate. I use sled to store information as (key, value) pairs in a database. Most of these libraries are still experimental and I take this attitude myself by straying off from the reference implementations and changing algorithms in hopes of better performance. If you want to help, check out #rustmatrix:koesters.xyz.
mautrix-asmux wasn't made this week, but I haven't mentioned it on TWIM before. Basically, it acts as a proxy between appservices and the homeserver. The primary point is to make it possible to dynamically provision appservices.
The dynamically provisioned appservices connect to mautrix-asmux with individual access tokens, which asmux checks and then proxies the requests to Synapse with its own global access token. Synapse is only aware of one appservice, which will have a large user ID and alias namespace (e.g. everything starting with _). In the other direction, mautrix-asmux maintains a room ID -> appservice mapping, which it uses to send incoming events to the correct appservice. As a side effect, mautrix-asmux implements MSC2190, which is needed for end-to-bridge encryption.
We continue on our performance drive. We’ll get Redis into production early next week replacing our home grown TCP based replication system. We also landed some SSO fixes for user interactive auth. Additionally we shipped a bug fix release in 1.12.3.
mautrix-telegram now has experimental support for end-to-bridge encryption. It's intended for cases where you don't want the homeserver to have access to messages, e.g. user-hostable appservices. It's currently on the e2be branch and should be merged to master soon. You can find setup instructions on the wiki: https://github.com/tulir/mautrix-telegram/wiki/End‐to‐bridge-encryption
The same feature will make its way to mautrix-facebook and mautrix-hangouts in the near future. After that, I'm going to start working on porting matrix-nio's crypto code to Go to bring end-to-bridge encryption to mautrix-whatsapp and possibly also general e2ee support for gomuks.
I got annoyed by how difficult plumbing IRC rooms is and wrote a simple Python script to do it more easily from command line. Also cleaning old rooms and chats in Riot is quite tedious so I added support for easily leaving rooms to the script. I decided to creatively call it matrixtools and created a github repo for it for others to enjoy. It's still in infancy but more features are already planned such as more IRC management stuff and creating tombstone events to point rooms to other rooms. The tool uses Python and matrix-nio library. Contributions welcome! https://github.com/vranki/matrixtools
mx-puppet-bridge
mx-puppet-bridge is a general bridging library that supports (double)bridging and relays. The goal is to make it as easy as possible for others to bridge new third-party protocols to matrix. Support roomDonate
The bridge bot now sends read indicators for messages successfully delivered to the remote network - only supported on protocol implementations with event synchronising. 🦊
mx-puppet-slack
Advanced relay support! Similar to the discord bridge, you can now make an advanced relay, where, if in relay mode, the messages from matrix users appear correctly with username+avatar on the slack side. This works for both classic slack apps and new slack apps, via the events API.
For that the underlying slack client connection had to be significantly re-written, which became its own repository.
mx-puppet-skype
The skype puppet received a couple of bugfixes, so that it shouldn't need to be manually restarted every day anymore.
Hey folks! I've not posted a proper bridge update in a while, so let's go into another project that isn't a bridge. I've started working on a new project called node-jitsi, which will allow people to connect their services to Jitsi Meet conferences. It's designed to allow bridges to start bridging across actual calls into Matrix via Jitsi, but could really be used by any project which is Node.JS based and wants Jitsi call functionality. The goal is to keep the interface clean and simple, and do all of the heavy lifting (WebRTC negotiations, XMPP handling) itself.
Currently the project has just started, although progress should hopefully be swift! Come check us out in #node-jitsi:half-shot.uk and get chatting
Nio for iOS has been getting some attention this week. Message display is now much better (visually grouped by sender) and I added some preliminary support for contextual event actions. For the time being emoji reactions and redactions have been implemented. (The context menu does trigger the best result of the visual flipping workaround in SwiftUI though 😅)
Loading dialog when sending files is displayed too long
Fixed device settings list
Fix a lazy loading bug
Improve app icon
The current version in the PlayStore is outdated. I need to write a privacy policy first and this can take some time. A TestFlight version for iOS is planned too but not yet ready. Here is a screenshot of the new voice messages feature:
The Quotient project has made two "sustaining" releases this week, and then three^Wfour more to fix foolish mistakes - right on the Fools' Day! Long story short - if you're on Quaternion 0.0.9.4* (with any trailing letter) and, respectively, libQMatrixClient 0.5.x, make sure to upgrade to Quaternion 0.0.9.4e and libQMatrixClient 0.5.3.2 because, as of this writing, these are the latest and fixedest in the breed. Aside from many backend bugfixes (mostly described in release notes from 0.5.3), this refreshed pair can handle SSO and no more interferes with Pantalaimon in encrypted rooms. Further plans include scrapping the whole Quotient thing and switch to Python because it's so much easier. Just kidding :) the plans didn't change, Quotient 0.6 is the next milestone, and the backend for the next Quaternion release. And you won't need to wait another year for them, I promise.
Riot Web 1.5.14 and 1.5.15 were released this week with a simpler Jitsi integration, new keyboard shortcuts (along with shortcut help via Cmd / Ctrl+/), and layout performance fixes. 1.5.14 has a security issue with the Jitsi widget wrapper, so please remove any copies of 1.5.14 if you installed it. Lots of cross-signing polish work continues as we get closer to release.
We made a hot fix release (0.10.5) this week. Cross-signing work is still progressing well. We will run much more tests next week with other Riots to finish the feature.
Ismail, who joined the team mid-week 🥳🎉, has started to do some maintenance work required by iOS 13 SDK.
Through a steady increase in demand thanks to GSoC, matrix-rust-sdk and timokoesters' homeserver, our foundational crates are receiving more attention than ever before, resulting in the following releases last week:
Hello! I'm working on a PHP library (GPL v3.0+) that allows to communicate with a Matrix instance. Currently the library allows to create users, rooms, login to the Matrix server and send messages, although more work needs to be done: https://github.com/artyom-poptsov/matrix-php
I created the library due to my own needs, but probably it will be of some interest for others.
Apparently the author has a need for Matrix-Moodle integration, sounds interesting! An update! Late breaking:
I've updated Matrix-PHP, now there's documentation in README.md on how to use the library. If anyone wants to use it/contribute, this should make the things easier.
There's no release yet though. But for now, the library is capable of:
creating users with on the servers where registration is disabled;
changing users passwords (by users themselves or by admins);
logging in to the server using m.login.password method;
sending text messages to rooms;
checking if a username is available for registration on a server (only for admins.)
I've just released v1.0 of my Matrix helm chart, to help deploy a complete homeserver stack in Kubernetes. Partially inspired by spantaleev's wonderful ansible playbook, dacruz21/matrix-chart aims to be an all-in-one installation of Synapse, Riot, bridges, and other services. It includes a number of Kubernetes security and scalability features, and is ideal for anyone running a large homeserver, or a homeserver in a corporate environment where security is paramount.
Lives in the cloud now so should be more reliable.
Fairly sure Peter is referring to his tracking bot being on new hosting. He could mean that he has ascended to the clouds, and is therefore more reliable, but perhaps he wouldn't have posted to TWIM in that case.
In addition to some performance fixes, the chat bot integration in Rust for openSUSEs Open Build Service can now listen also to openQA events.
openQA is an operating system level integration testing framework that makes it possible to test the installation process, GUIs and TUIs by simulating user interaction on different hardware or virtualization technologies.
OBS is used for development of the openSUSE distribution and can build packages from the same sources for Fedora, Debian, Ubuntu, SUSE Linux Enterprise and other distributions.
Mozilla is hosting a Fix-the-Internet Lab from April 15th - June 15th that specifically calls out decentralization and messaging.
Mozilla’s Fix-The-Internet MVP Lab is an 8 week-long incubator-style program this Spring to mobilize & fund around products and technologies that enable everyone to connect and build a better society.
Sounds like it could be a good fit for a Matrix community project. Each participant would get a $2500 stipend and access to mentorship. There are also significant prizes.
Applications are due Monday April 6 at 11:59PM ET.
Maybe more tangentically relevant, but the Swedish government is launching a three-day remote hackathon starting tomorrow (3rd-6th of April) focusing on saving lives, communities, and businesses. A Matrix solution may very well be interesting to develop for this, perhaps something taking another look at the CoAP/CBOR work.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
envs.net
357
2
gottliebtfreitag.de
429
3
maunium.net
456.5
4
kif.rocks
520
5
imninja.net
632.5
6
encom.eu.org
657
7
fairydust.space
676
8
xethos.net
804
9
chat.matrix4me.de
913
10
synod.im
1096
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
A small patch release to fix regressions introduced in v1.11.0 and v1.12.0.
Update 1.12.3 is a fast follow release from 1.12.2
Get 1.12.3 (not 1.12.2) from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.
Changelog since v1.12.0
Synapse 1.12.3 (2020-04-03)
Remove the the pin to Pillow 7.0 which was introduced in Synapse 1.12.2, and
correctly fix the issue with building the Debian packages. (#7212)
Synapse 1.12.2 (2020-04-02)
This release fixes an issue with building the debian packages.
Synapse 1.12.1 (2020-04-02)
No significant changes since 1.12.1rc1.
Synapse 1.12.1rc1 (2020-03-31)
Bugfixes
Fix starting workers when federation sending not split out. (#7133). Introduced in v1.12.0.
Avoid importing sqlite3 when using the postgres backend. Contributed by David Vo. (#7155). Introduced in v1.12.0rc1.
Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature. (#7177). Introduced in v1.11.0.
We released 1.12.0 this week which contains some decent performance wins. Our theme for the moment is to continue working on performance, and the sharding project is starting to make some real progress. Hopefully we get onto breaking out event persistence next week.
Aside from that you'll see a lot of work to improve SSO support, in particular the ability for SSO and User Interactive Auth to work together is just around the corner.
Sorry for all the matrix.org slowness, we're doing everything we can!
Updated the Kubernetes-optimized images of Synapse to 1.12.0.
Both the alpine and debian-based - with jemalloc support - tags use the latest twisted package.
NB; If you're using both workers and application services, make sure that your application service config files are mounted into the worker pods as well.
Debian backports for Buster will see updated synapse and Twisted around the weekend when packages migrate to testing
Debian packages for synapse have been updated and now pull the patched version of Twisted with CVE-2020-10108, CVE-2020-10109 and other security vulnerabilities fixed
the patched version of Twisted in Debian is 18.9.0-8, Ubuntu ships security fixes in 18.9.0-6ubuntu1; both include all necessary security fixes
20.3.0 is currently in Debian experimental and hopefully will be uploaded to unstable by the end of the week
Submitted a PR that adds support for running the Synapse state compression tool to the synapse-purge utility written by djmaze. So that it's able to both launch purges for all rooms, as well as compress their state afterwards.
Dept of Bridges 🌉
mx-puppet-skype
mx-puppet-bridge is a general bridging library that supports (double)bridging and relays. The goal is to make it as easy as possible for others to bridge new third-party protocols to matrix. Support roomDonate
There is a new skype bridge! mx-puppet-skype, also based on mx-puppet-bridge.
It already features:
Sending normal and rich formatted text in both directions
Sending images, audio and other media in both directions
Some skype emoji are parsed to normal emoji
Skype->Matrix typing and read indicators
Skype->Matrix user updates
Please note that skype seems to be having some issues lately where sometimes messages aren't received (also in the official client), and that sadly also affects the bridge.
Mirage is a new fancy Matrix client written in Qt/QML + Python, currently in alpha and available on Linux.
Some of its main features are:
A fluid interface that adapts to any size
Keyboard shortcuts for (almost) everything, including filtering/switching rooms, scrolling, sending files, selecting and copying messages...
A dynamic, programmable theming system
Multiple accounts in one client
Support for end-to-end encryption
Nheko
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
The image overlay you get, when you click on an image, should now stay out of your way, when downloading an image (adasauce)
Nheko now shouldn't show a console on launch on Windows anymore (abma)
Typing notifications now won't sanitize users twice. This shouldn't increase Corona infections, just make special characters like < and & show up correctly, so you can put more <3 into your names. (Emi)
A lot of edge cases, where avatars didn't show up correctly were fixed. One issue with dynamic thumbnailing and synapse is still waiting for a synapse pr to be merged.
If you restart your server, Nheko should now use less CPU in the time the server is down. It may take a bit longer to reconnect though.
I'd like to thank all the contributors, that contribute seemingly random, small patches. It really helps make Nheko better in the long term and I really appreciate your help! <3
Quotient 0.5.3 RC goes out - this is a backwards-compatible release that you can safely use with your Quaternion 0.0.9.4. Packagers are welcome to do all the preparations necessary - 0.5.3 release will happen right after the weekend! Despite back-compatibility, this version sports quite a few recent improvements backported from the master branch, including SSO, support of reactions and message editing, .well-known and, of course, quite a few bug fixes. Check it out at https://github.com/quotient-im/libQuotient/releases/tag/0.5.3-rc.
P.S. For those who want more, Quotient 0.6 (and Quaternion 0.0.9.5) are coming real-soon-now, too!
We are still working on cross-signing but we see the end :). Gossiping of cross-signing private keys is now automatically done in background. We have a new modal that pops up on unverified sign-ins. We started to make cross-platform tests and fixed issues. Next week will be all about polishing this feature.
You need to tell the playbook to use the :develop riot-web though. The documentation page instructs people to do that (for now).
I did a 4-person video conference the other day and all Jitsi components combined seemed to take some ~500MB of memory and quite a lot of CPU, while the call was ongoing.
Now comes with a !risk command that takes an age and calculates some outcome probabilities for that age group. Thanks to @dctremblay:matrix.org for adding this in!
v0.2.0 of the reminder plugin for maubot was released recently. The primary change was a new custom date parser, which should have less weird results than before.
It also has a locale system. You can set locales with !reminder locale <list of locales>. The system is fairly simple: it just tries to parse the input with each locale in the order specified and uses the first match. It doesn't have many locales so far, so contributions are welcome if you're not afraid of regexes (contributions for reducing the number of regexes are also welcome :D)
for those who quite understandably couldn’t endure listening to 90 minutes of waffle about Matrix on The Changelog, the transcript is now up at https://changelog.com/podcast/384!
To help people stay in contact in times of coronavirus, there are now coronavirus rooms in 4 languages (German: #covid-19-de:ungleich.ch , English: #covid-19:ungleich.ch, French: #covid-19-fr:ungleich.ch and Korean #covid-19-kr:ungleich.ch) They are all listed on https://ungleich.ch/u/blog/coronavirus-chats/ to give new matrix users an easy entrance
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
servicedesk-ttc.com
343
2
ccc.ac
419
3
gottliebtfreitag.de
421.5
4
envs.net
476
5
maunium.net
478
6
c-base.org
484
7
mailstation.de
506
8
maescool.be
576
9
encom.eu.org
585.5
10
matrix.vgorcum.com
648
Final Thoughts 💭
Next Tuesday is the student submission deadline for GSoC! Check out our list of projects and come and chat to us.
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
The most important reason to upgrade to 1.12.0 is that it contains performance improvements to boost the efficiency of state resolution in room versions greater than v1 (#7095).
It also contains an implementation of (MSC2432) designed to limit room alias abuse. The MSC contains the full details, but in short admins now have more control over aliases as they appear on their local server.
There is also plenty of ground work for our master process sharding project. Nothing that can be used today unfortunately, but expect big improvements for worker based deployments over the coming weeks.
Please pay special attention to the security advisory in the changelog, TL;DR make sure your installation is using Twisted 20.3.0.
Also, note that once Synapse 1.13.0 is released, the default branch will change to being develop rather than master. Again more details follow in the changelog.
Get the new release from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.
Changelog since Synapse 1.11.1
Synapse 1.12.0 (2020-03-23)
No significant changes since 1.12.0rc1.
Debian packages and Docker images are rebuilt using the latest versions of
dependency libraries, including Twisted 20.3.0. Please see security advisory
below.
Security advisory
Synapse may be vulnerable to request-smuggling attacks when it is used with a
reverse-proxy. The vulnerabilities are fixed in Twisted 20.3.0, and are
described in
CVE-2020-10108
and
CVE-2020-10109.
For a good introduction to this class of request-smuggling attacks, see
https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn.
We are not aware of these vulnerabilities being exploited in the wild, and
do not believe that they are exploitable with current versions of any reverse
proxies. Nevertheless, we recommend that all Synapse administrators ensure that
they have the latest versions of the Twisted library to ensure that their
installation remains secure.
Administrators who have installed Synapse from distribution packages should
consult the information from their distributions.
The matrix.org Synapse instance was not vulnerable to these vulnerabilities.
Advance notice of change to the default git branch for Synapse
Currently, the default git branch for Synapse is master, which tracks the
latest release.
After the release of Synapse 1.13.0, we intend to change this default to
develop, which is the development tip. This is more consistent with common
practice and modern git usage.
Although we try to keep develop in a stable state, there may be occasions
where regressions creep in. Developers and distributors who have scripts which
run builds using the default branch of Synapse should therefore consider
pinning their scripts to master.
Synapse 1.12.0rc1 (2020-03-19)
Features
Changes related to room alias management (MSC2432):
Publishing/removing a room from the room directory now requires the user to have a power level capable of modifying the canonical alias, instead of the room aliases. (#6965)
Validate the alt_aliases property of canonical alias events. (#6971)
Users with a power level sufficient to modify the canonical alias of a room can now delete room aliases. (#6986)
Implement updated authorization rules and redaction rules for aliases events, from MSC2261 and MSC2432. (#7037)
Stop sending m.room.aliases events during room creation and upgrade. (#6941)
Synapse no longer uses room alias events to calculate room names for push notifications. (#6966)
The room list endpoint no longer returns a list of aliases. (#6970)
Remove special handling of aliases events from MSC2260 added in v1.10.0rc1. (#7034)
Expose the synctl, hash_password and generate_config commands in the snapcraft package. Contributed by @devec0. (#6315)
Check that server_name is correctly set before running database updates. (#6982)
Break down monthly active users by appservice_id and emit via Prometheus. (#7030)
Render a configurable and comprehensible error page if something goes wrong during the SAML2 authentication process. (#7058, #7067)
Add an optional parameter to control whether other sessions are logged out when a user's password is modified. (#7085)
Add prometheus metrics for the number of active pushers. (#7103, #7106)
Improve performance when making HTTPS requests to sygnal, sydent, etc, by sharing the SSL context object between connections. (#7094)
Bugfixes
When a user's profile is updated via the admin API, also generate a displayname/avatar update for that user in each room. (#6572)
Fix a couple of bugs in email configuration handling. (#6962)
Fix an issue affecting worker-based deployments where replication would stop working, necessitating a full restart, after joining a large room. (#6967)
Fix duplicate key error which was logged when rejoining a room over federation. (#6968)
Prevent user from setting 'deactivated' to anything other than a bool on the v2 PUT /users Admin API. (#6990)
Fix py35-old CI by using native tox package. (#7018)
Fix a bug causing org.matrix.dummy_event to be included in responses from /sync. (#7035)
Fix a bug that renders UTF-8 text files incorrectly when loaded from media. Contributed by @TheStranjer. (#7044)
Fix a bug that would cause Synapse to respond with an error about event visibility if a client tried to request the state of a room at a given token. (#7066)
Repair a data-corruption issue which was introduced in Synapse 1.10, and fixed in Synapse 1.11, and which could cause /sync to return with 404 errors about missing events and unknown rooms. (#7070)
Fix a bug causing account validity renewal emails to be sent even if the feature is turned off in some cases. (#7074)
Improved Documentation
Updated CentOS8 install instructions. Contributed by Richard Kellner. (#6925)
Fix POSTGRES_INITDB_ARGS in the contrib/docker/docker-compose.yml example docker-compose configuration. (#6984)
