🔗Open Tech Will Save Us #3 🎙

If you somehow missed this, please take a look!

• 1:00 Rabble talking about Planetary and SSB
• 31:19 Annie on Ditto
• 1:10:20 Erik on Synapse Perf

🔗Dept of Spec 📜

🔗Spec

anoa announced:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

🔗MSC Status

Merged MSCs:

• MSC2265: Proposal for mandating case folding when processing e-mail address localparts

MSCs in Final Comment Period:

• MSC2399: Reporting that decryption keys are withheld (merge)

New MSCs:

• MSC2631: Add fallback_content to PusherData

• MSC2630: checking public keys in SAS verification

• MSC2625: Add mark_unread push rule action

• MSC2618: Helping others with mandatory implementation guides

🔗Spec Core Team

In terms of Spec Core Team MSC focus for this week, we're sticking with the same three from last week: MSC2366 (verification flows), MSC2403 (knocking), and MSC2630 (SAS security).

In possibly more exciting news, the Spec Core Team has decided to start separating the scattered casual implementation tips and notes in the spec out and putting them in a series of implementation guides! These will take the form of gitbook-style things, and will initially feature front-to-back implementation guides for a generic Matrix client and homeserver. For more details to provide feedback, check out the associated MSC2618!

🔗Dept of GSoC 🎓

This is the second week of the coding phase of GSoC 2020. Find updates from the students below:

🔗GSOC/Opsdroid

tyagdit told us:

• The port from matrix-python-sdk to matrix-nio is almost completed. Sending and receiving most events works properly. Changes are expected be merged in a few days. This PR itself doesn't implement E2EE, that will be a different PR

• Alongside that I've been working on implementing E2EE with nio which is partially done, sending and receiving messages works so far. some changes to the MatrixStore interaction and testing is left.

🔗Nheko

Chethan told us:

Device verification for Nheko

• Done
  • Adding Device List ( Although it already exists it is re-written in qml )
  • Detecting to_device device_verification messages
  • Part of the flow is done
• Yet to Complete
  • Complete the rest of the flow

🔗go-neb

nikofil offered:

• Implementing end-to-end encryption for go-neb turns out to be a lot easier than expected thanks to the https://github.com/tulir/mautrix library, which contains out of the box e2ee support!

• The first week of GSoC was spent switching go-neb's usage of gomatrix for mautrix, which was easy as the latter is a fork of the former. This caused a pretty big PR that touched most of go-neb's files, although the changes were simple and easy to review. The PR was merged shortly after into go-neb.

• During the second week, it was time to add some basic e2ee support to go-neb. This was complicated slightly due to the bot's ability to create multiple clients and its treatment of /sync responses, however in the end e2ee is now working.

• The code is still of course a mess and session keys are often lost in the void. The code will be cleaned up and tested further next week, bugs will be (hopefully) fixed and another PR will be submitted to go-neb then to officially add e2ee support!

• Code can be found at https://github.com/nikofil/go-neb/tree/e2ee until it can be cleaned up

🔗HTML-Embeddable rooms

From arnav-t:

Matrix has enormous potential to be used in many different use cases be it embedded chat rooms, comments section, chat boxes, etc. This project aims to fill that niche by providing an easy to use and easy to deploy client for users to embed on their webpages.
The client would mesh well wherever it is embedded by providing a postMessage interface to allow the parent webpage to interact with the client and dynamically modify it whenever needed (For example:- A comments section that is always the same theme as its parent webpage).

🔗Features

As of now, the following features have been added (with more on the way)

• List of joined rooms that can be selected
• Live room timeline events
• Support for image-based messages
• Message composer and ability to send messages to a room
• Dark and light themes for the client
• Changeable highlight colors
• Toggleable room header, room timeline, and message composer components
• postMessage interface for sending commands from the parent window

Currently, I'm working on adding more options to the postMessage interface as well as adding more interactivity to the client itself.

Find out more on the project page: https://github.com/arnav-t/riot-embedded.

🔗Ruma/Matrix GSoC update

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

devinr528 announced:

This week in ruma-events' Google Summer of Code project, I was able to finish the macros needed to generate the event content enums and trait implementations required for events. I started by defining the generic event structs (state, message, etc.) and manually writing the Serialize and Deserialize implementations. Over the next few days, this was moved into a custom derive macro called Event. The derive now implements all necessary traits with appropriate bounds, so a StateEvent<C> can not contain any ephemeral event content and so forth. I have removed the raw mod and related FromRaw and TryFromRaw traits, moving the validation into the deserialization and constructor for the few types that needed it. On the event content side of things, a function like procedural macro was used to allow declaring the enum using Matrix event type identifiers.

event_content_enum! {

    /// Any message event.
    name: AnyMessageEventContent,

    events: [
        "m.call.answer",

        "m.room.message",
        // ...

    ]
}

// Produces
pub enum AnyMessageEventContent {

    CallAnswer(CallAnswerEventContent),
    RoomMessage(MessageEventContent),

}

🔗Dept of Servers 🏢

🔗Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander offered:

Much of this week's work has been maintenance-related with refactoring, cleaning up etc.

• Key fetching/expiry behaviour has been fixed, improving the reliability of federation.

• The monolith and component setup has now been refactored.

• A number of unused internal APIs have now been removed.

• Other internal roomserver APIs have been streamlined.

We have also started building an Yggdrasil-powered P2P demo (as opposed to the previous demos built using libp2p) and even includes support for embedding Riot Web in the single binary. It's not very stable/usable yet but there will be more news on that to follow within the next couple of weeks.

We have also started building and releasing Docker images for Dendrite on Docker Hub for those who want to use Docker to test Dendrite deployment.

Spec compliance is unchanged from last week:

• Client-Server APIs: 36%

• Server-Server APIs: 35%

🔗Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo reported:

Welcome back! This week I focused my efforts on better error handling. My goal was that every problem would be forwarded to the client (e.g. InternalServerError response) and also be logged. Yesterday I merged the giant PR this resulted in. Hopefully this will make Conduit easier to debug in the future.

Here are some other things I finished this week:

• User-interactive authentication (e.g. you need to type your password again if you want to delete devices)

• Config option to disable registration

I also found the problem with Riot not showing notifications for new messages and created a bug report. With a few tweaks, I managed to solve this on my account and now mentions, DMs and more play the notification sound (it also works on Riot X!).

@gnieto fixed a bug that prevented Riot from loading history when you join a room and @PublicByte implemented the /whoami endpoint.

Thanks to my supporters on Liberapay!

🔗Synapse

Neil said:

This week we shipped 1.15.0, edited highlights include:

Features

• Add admin APIs to allow server admins to manage users' devices. Contributed by @dklimpel. (#7481)

• Add an option to disable autojoining rooms for guest accounts. (#6637)

• For SAML authentication, add the ability to pass email addresses to be added to new users' accounts via SAML attributes. Contributed by Christopher Cooper. (#7385)

• Add support for generating thumbnails for WebP images. Previously, users would see an empty box instead of preview image. Contributed by @WGH-. (#7586)

• Support the standardized m.login.sso user-interactive authentication flow. (#7630)

Bugfixes

• Allow new users to be registered via the admin API even if the monthly active user limit has been reached. Contributed by @dklimpel. (#7263)

• Fix a bug in automatic user creation during first time login with m.login.jwt. Regression in v1.6.0. Contributed by @olof. (#7585)

• Fix a bug causing the cross-signing keys to be ignored when resyncing a device list. (#7594)

• Fix bug where returning rooms for a group would fail if it included a room that the server was not in. (#7599)

• Fix duplicate key violation when persisting read markers. (#7607)

• Prevent an entire iteration of the device list resync loop from failing if one server responds with a malformed result. (#7609)

• Pass device information through to the login endpoint when using the login fallback. (#7629)

• Advertise the m.login.token login flow when OpenID Connect is enabled. (#7631)

• Fix bug in account data replication stream. (#7656)

Internal Changes

• Improve query performance for fetching state from a PostgreSQL database. Contributed by @ilmari. (#7567)

• Speed up processing of federation stream RDATA rows. (#7584)

• Refactor Ratelimiter to limit the amount of expensive config value accesses. (#7595)

• Clean up exception handling in SAML2ResponseResource. (#7614)

• Check if the localpart of a Matrix ID is reserved for guest users earlier in the registration flow, as well as when responding to requests to /register/available. (#7625)

Aside from that we continue out push towards improving performance both in terms of sharding workers but also improving tools to manage disk space usage. Specifically this week and next we are looking at db tuning and sharding the media repo worker. We are also noticing that the typing handler is chewing more CPU than would be expected. It could be a good candidate for moving away from the master process.

More broadly we are changing the defaults for notifications and making changes to support a more general overhaul of notifications.

Finally we are looking at how to not only delete messages after a certain time period, but also how to delete the associated media (which becomes difficult when considering e2e content along side long lived content such as avatars and sticker packs).

For more detail on what is coming next, take a look at our public task board.

🔗Synapse Deployment 📥

🔗Kubernetes

Ananace told us:

Just pushed version 1.15.0 of the K8s-optimized Synapse images

🔗Dept of Bridges 🌉

🔗mautrix-whatsapp

Tulir told us:

The WhatsApp bridge can now bridge location messages from WhatsApp to m.location on Matrix and contact messages as vCard files (until Matrix gets a native contact message type).

It also now encrypts media properly when bridging messages in encrypted rooms. I had apparently missed that when adding end-to-bridge encryption earlier.

🔗mautrix-facebook

Tulir said:

mautrix-facebook got a few improvements and bugfixes:

• Added timestamp massaging

• Added support for bridging video and file messages to Matrix

• Fixed backfilling with SQLite (it didn't work due to the python timezone mess, fix contributed by erdnaxeli)

• Improved options for automatic reconnecting. The bridge should now work fairly reliably when using all the auto-reconnect options and backfilling, it only breaks if facebook decides you've been hacked

🔗mautrix bridges in general

Tulir said:

Since MSC2190 and the related Synapse PR aren't moving much, all my bridges now generate appservice registration files with the workaround. The workaround is needed to avoid patching synapse or using other hacks, and I decided to enable it by default as many people had problems related to having to apply it manually.

The WhatsApp, Facebook and Hangouts bridges now have an option to use double puppeting to disable notifications when backfilling. It can be useful if you want to copy chat history to Matrix, but don't want to be spammed with hundreds of notifications while it's happening.

🔗Nova-slack

eric offered:

To get things ready for NovaChat launch, we added a bunch of new features to a fork of the mx-puppet-slack client that Sorunome created. You can try it out here. We'll be pushing our changes upstream soon.

New features:

• message backfilling at room creation and reconnect

• syncs read status to/from Slack.com

• adds all users to bridged room during initial creation

• automatically bridges all channels and DMs that you are part of during initial setup, so you don't have to wait for a message to arrive for it to appear in Matrix

• tons of little reliability fixes

🔗matrix-sms-bridge

Benedict announced:

My matrix-sms-bridge got an large update, which introduced an automated way to create and write messages to telephone numbers via commands in a bot room. See the docs for more information, how to use it.

🔗Dept of Clients 📱

🔗NovaChat v0.1.1

eric reported:

Introducing NovaChat - a desktop chat client that supports 9+ different chat networks, built on top of Matrix. Think of it as a modern Adium/Trillian.

How it works

NovaChat is a system with three parts designed to work together:

• a new desktop client, based on Riot-web

• a federating homeserver (nova.chat)

• a set of Matrix bridges configured to run on a private server called a bridgebox

Matrix bridges are awesome but generally require you to run own homeserver. NovaChat removes that restriction, without sacrificing security and privacy. Your remote chat network credentials never leave your bridgebox, and messages are encrypted* on the bridgebox before being sent to the NovaChat homeserver.

*Encryption is currently enabled on Telegram/FB/Hangouts/Whatsapp bridges, mx-puppet-bridges coming shortly)

Very rough system diagram:

> NovaChat desktop client <> NovaChat homeserver <> bridgebox <> remote chat networks

Features

• Fancy new UI, heavily inspired by Telegram. See photo below or see it in action

• Easy graphical bridge set up (no more fiddling with config files)

• Reliable bridging, with delivery confirmations and bridge disconnect warnings

• Includes 8 pre-configured bridges: Facebook/Whatsapp/Hangouts/Telegram/Slack/Instagram/Twitter/Skype. Planning to add more soon, or submit a PR!

• Works on Mac OS, Linux and Windows

You don't need to be technical to use NovaChat, it's designed to Just WorkTM. If you need help setting up a bridgebox, we are offering hosting and maintenance for USD $5 per month.

The vast majority of this software has been created by the talented tulir and dm0141. I personally am a terrible coder (you can tell which parts I did), but I do have experience building products and teams, and I'm excited to begin working in the Matrix universe. I wrote recently about how I believe Matrix is the Universal Communication Bus! Thanks goes to Sorunome for creating the mx-puppet-bridges!

We're developing the whole project in the open (though not all components are open source), check the issue backlog :)

Want to help? We're hiring Matrix developer to work on the project with us. Send me a message if you're interested! Particularly looking for a React dev to help with frontend on nova-web.

NovaChat chat room: #novachat:nova.chat

🔗Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

red_sky announced:

nheko 0.7.2 is ready for release. Here's the summary of changes:

🔗Highlights

• Reactions

  • React to a message with an emoji! 🎉

  • Reactions are shown below a message in a small bubble with a counter.

  • By clicking on that, others can add to the reaction count.

  • It may help you celebrating a new Nheko Release or react with a 👎 to a failed build to express your frustration.

  • This uses a new emoji picker. The picker will be improved in the near future (better scrolling, sections, favorites, recently used or similar) and then probably replace the current picker.

• Support for tagging rooms [tag]

  • Assign custom tags to rooms from the context menu in the room list.

  • This allows filtering rooms via the group list. This puts you in a focus mode showing only the selected tags.

  • You can assign multiple tags to group rooms however you like.

• SSO Login

  • With this you can now login on servers, that only provide SSO.

  • Just enter any mxid on the server. Nheko will figure out that you need to use SSO and redirect your browser to the login page.

  • Complete the login in your browser and Nheko should automatically log you in.

🔗Features

• Support for showing users presence and status message as well as setting your own status message"

• Respect exif rotation of images

• An italian translation (contributed by Lorenzo Ancora)

• Optional alerts in your taskbar (contributed by z33ky)

• Optional bigger emoji only messages in the timeline (contributed by lkito)

• Optional hover feedback on messages (contributed by lkito)

• /roomnick to change your displayname in a single room.

• Preliminary support for showing inline images.

• Warn about unencrypted messages in encrypted rooms.

🔗Improvements

• perf: Use less CPU to sort the room list.

• Limit size of replies. This currently looks a bit rough, but should improve in the future with a gradient or at some other transition.

• perf: Only clean out old messages from the database every 500 syncs. (There is usually more than one sync every second)

• Improve the login and register masks a bit with hints and validation.

• Descriptions for settings (contributed by lkito)

• A visual indicator, that nheko is fetching messages and improved scrolling (contributed by Lasath Fernando)

🔗Bugfixes

• Fix not being able to join rooms

• Fix scale factor setting

• Buildfixes against gcc10 and Qt5.15 (missing includes)

• Settings now apply immediately again after changing them (only exception should be the scale factor)

• Join messages should never have empty texts now

• Timeline should now fail to render less often on platforms with native sibling windows.

• Don't rescale images on every frame on highdpi screens.

🔗Matrix-Static release v0.3.0

Michael (t3chguy) reported:

New Matrix-Static release v0.3.0

With improved support for media and tombstone events and a bunch of bug fixes! https://github.com/matrix-org/matrix-static/releases

Should be deployed to view.matrix.org shortly

🔗Arch Linux package of radical-native

Peter Roberts told us:

I've made an Arch Linux package of radical-native.

🔗Riot X

valere announced:

• Several improvements on SSO login flow 🔒

• Support of power levels ⚡️

• Add remove favorites 🌟

• Multiple bugs and glitches fixes 🐛

The team is currently actively working on:

• Voip 1:1 Calls 📞

• Cross Signing bootstrap simplification 🔑

• Performances Improvement 🔥

• More room settings ⚙️

🔗Riot Web

Ryan told us:

There's been lots of work many fronts this week, including:

• Continued work on cross-signing bootstrapping robustness

• New room list implementation in progress

• Upgraded fonts for new emoji

• A refreshed look for matrix.to is underway

• The appearance tab is being tweaked and expanded

• Read marker consistency improvements

Most of this work is still in progress or has only landed on develop so far. Our next RC will happen next week.

🔗Riot-iOS

Manu offered:

This week, we continued to iterate on the implementation of the new push notification service thanks to user feedbacks. Those feedbacks also show that the app uses less battery.

On login, this is now possible to recover your cross-signing and your key backup from your recovery passphrase or key.

🔗Dept of SDKs and Frameworks 🧰

🔗libolm 3.1.5

uhoreg told us:

libolm 3.1.5 has been released. It mainly collects the various build system and documentation fixes that have accumulated since the last release. It also includes a new wasm build target (which apparently is used for running matrix-rust-sdk in the browser), as well as TypeScript definitions for the JavaScript bindings.

🔗Ruma

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

jplatte told us:

Most of the work this past week was on ruma-events, for more on that see the GSoC section! Devin got far enough with that so we now know it all works out and can replace the previous ruma-events API. When that was clear, ruma-events was also merged into our new monorepo; now the only remaining crate to be merged is ruma-client (where I'm also waiting for some pending changes).

🔗Dept of Ops 🛠

🔗Matrix Helm chart

Kubernetes Helm chart for easily deploying a Matrix homeserver stack

Typo Kign told us:

Version 2 of my Matrix Helm chart is out! It now allows for running a federating homeserver without needing an external load balancer for the federation service, potentially lowering costs (depending on your cloud provider), and definitely reducing the headache of managing TLS certificates!

dacruz21/matrix-chart is a Kubernetes Helm chart for easily deploying a Matrix homeserver stack, complete with Synapse, Riot, VoIP calls, bridges, and other backing services. It's ideal for anyone looking to run a large homeserver, or for corporate environments where security is paramount!

🔗Dept of Internet of Things 💡

🔗Tuple Weather Station Example

Alexander Eisele announced:

This week I've programmed a small POC for a Matrix weather station.

It's build using a ESP8266 Microcontroller and a BME280 Sensor for temperature, humidity and barometric pressure.

The data is send over the Tuple Matrix <-> MQTT bridge.

Code: https://github.com/derEisele/tuple-weather-example

You can join this project on #tuple:eiselecloud.de.

I'm personally really excited about this project!

🔗Dept of Bots 🤖

🔗Matrix bot for Redmine

progserega told us:

https://github.com/progserega/matrix_redmine_bot

1. Bot can create, accept (to user) new issues in Redmine from matrix

2. Bot can add comment or/and files to issue by it id

3. Bot can show messages (for user) from Redmine in matrix room

🔗Dept of Guides 🧭

Levans said:

Anyone running a federating instance of Synapse will likely have seen this: Synapse is database-hungry. It tends to take a lot of space. In this post, I'm documenting how I shrunk my homeserver database from 100GB to a little under 8GB, during a long maintenance cleanup. https://levans.fr/shrink-synapse-database.html

Very pleased to see this, I have a feeling I'm going to be running it in the near future...

🔗New Public Rooms 🏟

🔗New room "Matrix Outreach"

JulianF reported:

There's a new room "Matrix Outreach" #matrix-outreach:matrix.org for folks interested in "Helping each other explain and promote Matrix to the public: creating slogans, introduction leaflets, info-graphics, elevator pitch, Wikipedia articles, and so on."

🔗Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	fairydust.space	391
2	envs.net	443.5
3	maunium.net	506
4	tum.de	603.5
5	neko.dev	822
6	maclemon.at	887
7	cyllos.me	916
8	shortestpath.dev	1072.5
9	settgast.org	1079
10	maescool.be	1324

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.15.0 release day folks.

We continue our push to improve performance across the board. Factoring out event persistence into a separate worker pointed the way to a host of small but collectively important improvements. Expect further changes in coming releases.

Aside from that the admin API continues to grow (this time device management) and we have improved device list syncing to aid e2ee reliability.

We are receiving an increasing number of high quality PRs from the community, please keep them coming. Special thanks to cg505, dklimpel, WGH, olof and ilmari

Get 1.15.0 from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.

Changelog since v1.14.0

🔗Synapse 1.15.0 (2020-06-11)

No significant changes.

🔗Synapse 1.15.0rc1 (2020-06-09)

🔗Features

• Advertise support for Client-Server API r0.6.0 and remove related unstable feature flags. (#6585)
• Add an option to disable autojoining rooms for guest accounts. (#6637)
• For SAML authentication, add the ability to pass email addresses to be added to new users' accounts via SAML attributes. Contributed by Christopher Cooper. (#7385)
• Add admin APIs to allow server admins to manage users' devices. Contributed by @dklimpel. (#7481)
• Add support for generating thumbnails for WebP images. Previously, users would see an empty box instead of preview image. Contributed by @WGH-. (#7586)
• Support the standardized m.login.sso user-interactive authentication flow. (#7630)

🔗Bugfixes

• Allow new users to be registered via the admin API even if the monthly active user limit has been reached. Contributed by @dklimpel. (#7263)
• Fix email notifications not being enabled for new users when created via the Admin API. (#7267)
• Fix str placeholders in an instance of PrepareDatabaseException. Introduced in Synapse v1.8.0. (#7575)
• Fix a bug in automatic user creation during first time login with m.login.jwt. Regression in v1.6.0. Contributed by @olof. (#7585)
• Fix a bug causing the cross-signing keys to be ignored when resyncing a device list. (#7594)
• Fix metrics failing when there is a large number of active background processes. (#7597)
• Fix bug where returning rooms for a group would fail if it included a room that the server was not in. (#7599)
• Fix duplicate key violation when persisting read markers. (#7607)
• Prevent an entire iteration of the device list resync loop from failing if one server responds with a malformed result. (#7609)
• Fix exceptions when fetching events from a remote host fails. (#7622)
• Make synctl restart start synapse if it wasn't running. (#7624)
• Pass device information through to the login endpoint when using the login fallback. (#7629)
• Advertise the m.login.token login flow when OpenID Connect is enabled. (#7631)
• Fix bug in account data replication stream. (#7656)

🔗Improved Documentation

• Update the OpenBSD installation instructions. (#7587)
• Advertise Python 3.8 support in setup.py. (#7602)
• Add a link to #synapse:matrix.org in the troubleshooting section of the README. (#7603)
• Clarifications to the admin api documentation. (#7647)

🔗Internal Changes

• Convert the identity handler to async/await. (#7561)
• Improve query performance for fetching state from a PostgreSQL database. Contributed by @ilmari. (#7567)
• Speed up processing of federation stream RDATA rows. (#7584)
• Add comment to systemd example to show postgresql dependency. (#7591)
• Refactor Ratelimiter to limit the amount of expensive config value accesses. (#7595)
• Convert groups handlers to async/await. (#7600)
• Clean up exception handling in SAML2ResponseResource. (#7614)
• Check that all asynchronous tasks succeed and general cleanup of MonthlyActiveUsersTestCase and TestMauLimit. (#7619)
• Convert get_user_id_by_threepid to async/await. (#7620)
• Switch to upstream dh-virtualenv rather than our fork for Debian package builds. (#7621)
• Update CI scripts to check the number in the newsfile fragment. (#7623)
• Check if the localpart of a Matrix ID is reserved for guest users earlier in the registration flow, as well as when responding to requests to /register/available. (#7625)
• Minor cleanups to OpenID Connect integration. (#7628)
• Attempt to fix flaky test: PhoneHomeStatsTestCase.test_performance_100. (#7634)
• Fix typos of m.olm.curve25519-aes-sha2 and m.megolm.v1.aes-sha2 in comments, test files. (#7637)
• Convert user directory, state deltas, and stats handlers to async/await. (#7640)
• Remove some unused constants. (#7644)
• Fix type information on assert_*_is_admin methods. (#7645)
• Convert registration handler to async/await. (#7649)

🔗Firstly... 🏃‍♀️

Make an entry in your calendar, don't just write it on your hand: Open Tech Will Save Us is back for a third edition!
Next Wednesday, 10th June 2020!

🔗Matrix Live 🎙

🔗Dept of Spec 📜

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

🔗MSC Status

Merged MSCs:

• Proposal for storing megolm keys serverside

MSCs in Final Comment Period:

• MSC2265: Proposal for mandating case folding when processing e-mail address localparts (merge)

New MSCs:

• MSC2611: Remove m.login.token User-Interactive Authentication type from the specification

• MSC2610: Remove m.login.oauth2 User-Interactive Authentication type from the specification

• MSC2604: Accept device information for the login fallback endpoint

• MSC2596: Proposal to always allow rescinding invites

• MSC2589: Improve replies

🔗Spec Core Team

In terms of Spec Core Team MSC focus for this week, we've still got MSC2366 (verification flows) on our plate, but we're removing MSC2399 (UISI messaging) as it just needs one more final tick. We're adding MSC2403 (knocking) in its place.

🔗Dept of Servers 🏢

🔗Mascarene

Nico said:

Mascarene is a quite new homeserver implementation project started a few months ago. Now it's time to introduce it.

Mascarene is written in Scala, runs on JVM and relies on Akka actor model. Out of the box it provides features like efficient streaming I/O, clustering and live data distribution. Data are backed in a PostgreSQL database.

The project is in early stage but already provides a few endpoints which make Mascarene able to talk to Riot.

You can test it at https://snapshot.mascarene.org. You should be able to register, login, create a room and talk to yourself.

Current work is focused on client API implementation; federation or e2ee will come later.

Contributions are welcome, join us on #mascarene:beerfactory.org

🔗Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo told us:

Welcome back! This week I went through the whole Client-Server API and wrote down everything that is still missing from Conduit. You can find issues for all of them in the milestone for Conduit 1.0. This should make it more easy for contributors to find something to work on and gives a lot more structure to the repo (maybe I also did it, because it's so satisfying to close issues and see the milestone percentage go up 😛).

Some of the issues I could already close this week include:

• Implement redaction

• Implement loading the message someone replied to (/context)

• Fix bugs with notifications

• Don't send typing events when nothing happened

• Bundle typing events into one EDU

• Don't send notification counts every /sync

• Implement heroes

As you can tell, a lot of work is starting to go into smaller improvements, which is a good thing, because it means most of the core work is already done!

Thanks to my supporters on Liberapay (Take a look, I improved it 🙂).

🔗Synapse

Neil told us:

This week in Synapse land we have been focusing on bug fixes especially SSO and soft logout.

We also spent some time thinking about how to further improve performance for large scale deployments (like matrix.org). In short this will mean sharding contested workers such as the media repo worker, the federation sender and the federation reader. We also want to remove some more functionality from the master process because it is still sitting at about 80% and we’d like some more head room. We'd also like to fix up the state compressor so it can easily be run as a background progress to save on disk space, this will benefit all installations big and small. Finally we want to speed up joins over federation. Exciting times.

Next week, more of the same and we should start on some of the perf items. Also look out for changes to notification defaults.

🔗Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander offered:

Updates:

• Room version 6 support has been added

• Server signing keys which have passed their validity period are now re-requested and updated properly, increasing reliability of federation substantially

• Federated room joins have been significantly sped up by requesting missing auth events inline when verifying room state, rather than repeating the entire verification for each missing event

• Local send-to-device support has been added (federated send-to-device coming soon)

• A lot of code clean-up has taken place (with more to come), including cleaning up the internal APIs and the component bases

• Decoding of room version 3 event IDs in request URLs has been improved

• Inbound federation requests now interrupt backoff intervals on outbound sends

• A new internal API exists now for interrupting backoff intervals on the federation sender

• The federated /devices endpoint now returns in the correct format

• Public rooms are now sorted by their member count

• Only our own aliases are stored in the public rooms directory now

• gomatrixserverlib is now responsible for marking room versions as supported/stable

• Spec-compliant transactions are now sent to the appservice component

• File URIs for SQLite databases are now set up using common functions

• Handling registration for a user that already exists has been fixed (thanks S7evinK!)

• Dendrite can now be configured to use a HTTP proxy server for outbound HTTP (thanks dr-bonez!)

Sytest compliance:

• Client-server APIs: 36%

• Server-server APIs: 35%

In other news, Kegan wrote a post on how p2p.riot.im works, including Dendrite's journey to SQLite and WebAssembly!

Additionally! You may have already read about v0.1.1 of the P2P Matrix demo - this is a big update, with local storage, room alias handling, federation improvements and more!

🔗Dept of Bridges 🌉

🔗Matrix Appservice Minecraft bridge now in beta

Dylan reported:

My Matrix Appservice Minecraft bridge is now in beta

• https://github.com/dhghf/matrix-appservice-minecraft

Here is our Matrix room #minecraft:dhdf.dev

More screenshots: https://github.com/dhghf/matrix-appservice-minecraft#screenshots

🔗Tuple

Alexander Eisele reported:

Tuple is a very early simple matrix <--> mqtt bridge for IOT and other embedded devices.

It's my very first golang project, but it works(TM).

• Repo: https://gitlab.eiselecloud.de/matrix/tuple

• Discussion/Testing: #tuple:eiselecloud.de

This is exciting! I would love to see more IoT-related projects!

🔗mautrix-hangouts

Tulir reported:

mautrix-hangouts can also do backfilling now, both when creating portal and missed messages when connecting

🔗mautrix-facebook

Tulir told us:

mautrix-facebook got support for backfilling history when creating portals and backfilling missed messages when connecting. There's also an option to periodically reconnect to potentially fix the bug where Facebook decides that you don't get messages anymore, but doesn't actually disconnect you.

🔗mautrix-whatsapp

Tulir said:

I had apparently forgotten to implement bridge information in the whatsapp bridge when I implemented it in my other bridges, but that mistake is now corrected. I also fixed a bug with encryption where it would generate too many one-time keys and cause messages to be undecryptable. Finally, mautrix-whatsapp had its first release this week since it's working reasonably well now.

🔗Dept of Clients 📱

🔗Radical Native

stoic reported:

Eager to test the keytar feature (pickle key stored in OS secret storage) with Riot Web? Support landed in radical-native v0.1beta13, which uses Rust keytar bindings under the hood. Testing requires the latest and greatest of Riot Web, so riot.im/develop might be the place to see it in action

Keytar support is a very very new feature, uhoreg, who has been working on it in the shadows and wasn't expecting to see it land yet, commented:

😲

🔗Pattle: version 0.17.0

Pattle is a client written for Flutter using Dart

Wilko offered:

A new version has been made available and will become available on Google Play and TestFlight!

⚠️ If you're not logged out, it's recommend that you logout and additionally clear your data after installing the new version! ⚠️

🔗Changes

• Added the ability to reply to messages!

• You can now accept (and reject) invites!

• You can now join and search channels!

• Chat input has been redesigned!

• Chat details has been somewhat redesigned

• The group icon in the chat's details is now blurred and can be shown unblurred by scrolling to the top

• Replies are now a maximum of 3 lines long

• The current version is now shown in settings

• Wait a bit after typing before checking the homeserver when logging in

• Chat name change events are now shown

• Chat icon change events are now shown

• Errors are now reported to Sentry again!

• You won't enter the app with empty chats if the initial load failed

• Show date above chat creation event

• Fix time sometimes being dark and not readable on image messages

• Fix sender's name going outside of bubble

• Sender is now in bold for chat description change event

• More messages are now loaded if initially not enough are displayed

• Show ripple effect on chat app bar when tapping

• And more!

🔗Get Pattle

• Android

  • F-Droid
  • Google Play

• iOS (TestFlight)

🔗Get involved

• Follow development: #app:pattle.im

• Issues or suggestions: Post them here!

• Donate: If you want to support Pattle, you can donate!

🔗RiotX v0.21.0

benoit announced:

RiotX v0.21.0 has been released on May 28th. It adds support to identity server configuration and e-mail/phone binding and unbinding. It's also now possible to switch between languages inside the app and to display the list of attachments (media and file) of a room. A new setting has been also added to hide redacted (deleted) events in the timeline. As Riot-Android SAS v2 is also included in this release. Also formatted_body for m.notice and m.emote are now taken into account.

This week we were still working on implementing VoIP (audio and video call). Support for integration manager, widgets and sending stickers has been merged to develop. Many new features are coming soon, stay tuned!

🔗Riot-Android classic

benoit offered:

Riot-Android: Version 0.9.12 has been released on May 20th. I forgot to mention it the TWIM of last week. It contains a fix for favorite rooms vanishing and also the SAS v2 implementation (new key agreement method).

🔗Mirage

miruka said:

Mirage 0.5.1 has been released today:

• Persistent, cross-client, push rules-aware unread message counters

• Read receipts are now sent to mark rooms as read

• Basic room settings can now be edited and saved

• More control over window alerts, e.g. only enabling them for messages mentioning you

• Lots of bugfixes

Full changelog

🔗Riot-iOS

Manu said:

This week, we finalised our work around the iOS13 SDK. The coming public TestFlight (0.12.2) is the result of this work. It comes with a lot of changes including a full rewrite of the push handling codebase, trying to find solutions to make matrix e2ee compatible with iOS Notification Service Extension. Most of these changes are unfortunately invisible for end users. Please report any issues. Note this is safe to switch between this TF and 0.11.5.

In parallel, we started to implement the cross-signing bootstrap and the recovery management. The UX of this is going to be refined on Riot-Web and RiotX-Android. Riot-iOS will implement the refined UX directly.

🔗Riot Web

A glossy Matrix collaboration client for the web. https://riot.im

Ryan told us:

We had several Riot Web releases this week, first with 1.6.3:

• Security fix for auth handling in SSO deployments, thanks Quentin Gliech for responsible disclosure

• Added more shortcuts to jump to read marker and upload a file

• Fixed notification tray icon stability for desktop on Windows

• Added an IRC style layout (currently available in labs only)

• Added font scaling settings (currently available in labs only)

• Fixed several bugs in the cross-signing bootstrap process

• Converted older banner-style UI for updates etc. to newer toasts

This was quickly followed by 1.6.4 to fix a few regressions with dismissing toasts, compact timeline layout, key backup, and tooltips on scroll.

🔗Dept of SDKs and Frameworks 🧰

🔗Matrix Dart SDK: version 0.32.0

Wilko announced:

• Databases are now deleted as a migration strategy

This will be in effect until 1.0 (or a bit before). If needed, make a backup between every minor version bump.

• All members necessary to render the timeline are now loaded from the store

• Room.myMembership has been removed, use Room.me.membership

• Add Rooms.enter and Room.join to join rooms

• Add Room.leave method

• Add PublicRooms

• Add useful getters to Member like isInvited or hasJoined

• Remove AliasesChangeEvent

• Errors in the separate isolate are now added to the updates stream where possible, or thrown in the calling isolate

• Fix profile API not working on servers where authorization is required

• Add MyUser.hasSynced

• Fix count not being nullable in MemberTimeline.load

• More fixes and small changes!

Get it on pub!

🔗Ruma

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

jplatte offered:

This week,

• Devin, our GSoC student, started blogging on ruma.io

• We released ruma-identifiers 0.16.2 and ruma-federation-api 0.0.2

• We started updating ruma-events and ruma-client-api with the changes from version r0.6.1 of the client-server specification

• We started moving all of our libraries into one monorepo!

  • As part of this, we introduced a kind of meta-library so you no longer have to keep the versions of the individual libraries in sync manually

  • We're hoping that this makes experiments across multiple libraries and the overall contribution process easier 🙂

🔗Dept of Interesting Projects 🛰

🔗Safe Support Chat

Brendan said:

Safe Support Chat, previously known as OCRCC Chatbox, now has a website with guides on how to set it up for organisations: https://www.safesupport.chat/

For context, Safe Support Chat is an embeddable chatbox built on Matrix with cool features such as end-to-end encryption support to enable individuals to seek support with organisations that provide it. It was initially built for the OCRCC (Ontario Coalition of Rape Crisis Centres) in order to allow survivors of sexual violence to communicate and seek help safely with OCRCC facilitators.

Kim Allen of Primal Glow Communications, one of the parties involved in Safe Support Chat's development, also recorded a podcast with Nomadic Labs about this project: https://podcast.nomadiclabs.ca/episodes/the-adventure-of-building-an-open-source-chatbox

🔗Dept of Ops 🛠

🔗matrix-docker-ansible-deploy

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi reported:

Thanks to benkuly's efforts, matrix-docker-ansible-deploy now supports bridging to SMS (with one telephone number only) via matrix-sms-bridge.

To learn more, see our Setting up Matrix SMS bridging docs page.

🔗CLI Federation Tester

Kai reported:

A new version of the terminal-based federation tester has been released. The current version 1.11 should now cover the entire server discovery process including scenarios with a .well-known configuration without a port, but with a DNS SRV entry.

The script relies on bash, curl, jq and openssl. It has been tested on linux but should also work on *bsd and macos.

The script can be downloaded here:

#cli_federation-tester_de:matrix.chat-secure.de

(The message about the latest version will always be pinned)

🔗FLAP

Louis Chmn announced:

Hi, I've build FLAP to help other self-host multiple free software. FLAP can currently help you to easily setup Synapse, Riot, Nextcloud, SOGo and Jitsi all with SSO and low maintenance needs. Feel free to give it a try !

• Sources are here: https://gitlab.com/flap-box/flap

• If you need help setting things up, Matrix room is here: #flap-software:matrix.org

For French companies, I am also offering managed paid hosting: https://www.flap.cloud

🔗Dept of Bots 🤖

🔗sparky's Matrix Bot

sparky told us:

Announcing Matrix Bot (name is a work in progress)

The first "official" release of matrix bot has been made and you can find the latest release source at https://github.com/sparky8251/matrix-bot/tree/v1.0.0

It's flagship features are currently

• an imperial <--> metric unit converter for common conversational units

• a configurable search and link for issues/pulls that responds to any message containing something in the format of jf#123 or jf #1234

• a configurable general purpose URL linker that responds to messages containing things such as docs@hwa or link @troubleshooting (both sides of the @ are configurable)

There are plans to enable group pings, get a help command that can be limited to function only in a specific room, and more in the works so be sure to check back soon!

🔗matrix-reminder-bot

anoa reported:

matrix-reminder-bot bot to remind you about things. Supports reminding an individual in a room or the whole room itself. Also supports setting alarms, which will continue to sound periodically until silenced.

Built with nio-template and the excellent matrix-nio library.

🔗Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	fairydust.space	354
2	aragon.sh	419
3	envs.net	426
4	maunium.net	456.5
5	maclemon.at	559
6	matrix.vgorcum.com	667.5
7	threatmodel.io	695
8	gottliebtfreitag.de	734
9	neko.dev	818
10	nzbr.de	892

🔗Final Thoughts 💭

There are bounties to be had! AU$500 to support Emoji Reactions in Fractal, for reference this is tracked on the GNOME gitlab.

We didn't hear from Spectral too recently, but there are signs of code being written.

This week I've been chatting with conference organisers making really interesting virtual event plans, and I'm bursting to share some of the Matrix integration plans, hopefully more details coming soon.

elmussol said:

For those who Hubzilla or Zap or ActivityPub or Friendica or Diaspora, there is now a Hubzilla forum for all things Matrix: https://elsmussols.net/channel/matrix

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

TL;DR: we shipped a major update (v0.1.1) to https://p2p.riot.im - fire up a desktop Chrome or Firefox in not-private-browsing mode and give it a go!

Hi folks,

As many know by now, a few of us have been working away since mid-December on experimenting with running Matrix in a peer-to-peer architecture - one where every user has absolute total autonomy and ownership of their conversations, because the only place their conversations exist is on the devices they own.

In some ways this is the logical end goal of Matrix: our aim has always been to empower users to have full control over their communication rather than being beholden to any given service provider, and in a P2P world we completely return power over secure communication to the people.

🔗Why P2P?

P2P Matrix is about more than just letting users store their own conversations: it can also avoid dependencies on the Internet itself by working over local networks, mesh networks, or situations where the Internet has been cut off. Even more interestingly, without homeservers, there is nowhere for metadata to accumulate about who is talking to who, and when - which is a legitimate complaint about today’s Matrix network, given the homeservers of all users in a given conversation necessarily have to store that conversation’s metadata. P2P also lets us radically simplify signup for new users if they don’t have to pick a server to get going - and we avoid the unintentional centralisation of users piling onto public servers.

P2P also forces us to solve many of the hardest remaining problems in Matrix: e.g multi-homed accounts, given multi-device P2P requires your account to exist in multiple places. This in turn unlocks high availability and geo-redundancy for accounts on today’s Matrix network (imagine having a primary and backup homeserver that magically did the right thing!), as well as account portability, and thus also vhosting and load-balancing accounts between servers, and even improved GDPR compliance (for if your user IDs are ephemeral they are no longer personally identifying information baked into your Matrix rooms). We’ll also need better safety mechanisms to avoid folks exploiting the anonymous nature of the network for abuse, accelerating the work we’re already doing for today’s Matrix network.

The way we’ve been approaching P2P is the “hamfisted but genius” approach of taking homeservers and running them on the client, alongside or within your Matrix client - meaning that there are literally no changes required for any Matrix client to talk P2P Matrix, and so P2P Matrix can instantly benefit from all the work which has gone into Riot and other apps. As a result, P2P is also a huge motivator towards developing much smaller homeservers which can run efficiently clientside (e.g. Dendrite!) - which is of course great news for Matrix as a whole. It also forces us to develop more scalable routing algorithms (as you don’t want your client to have to talk to every other device in a room every time it sends a message!) and also spurs development of low bandwidth Matrix transports (as you don’t want the additional chatter of talking to multiple peers to consume all your bandwidth). Finally, it forces us to really ruggedize federation, given nodes are constantly appearing and disappearing, giving the federation much more of a stress test than we see with today’s relatively static homeservers.

🔗P2P in Practice

So, P2P has been acting as fuel for a lot of our longer term Matrix work over the last few months. There have been three main experiments so far: at FOSDEM we showed off running our next-gen Dendrite homeserver running clientside using HTTP over libp2p as the transport. We also highlighted Timothée Floure’s project at EPFL experimenting with Synapse talking P2P CoAP over yggdrasil as the transport via a proxy.

Most recently, however, we’ve been experimenting with compiling Dendrite down to Web Assembly and running it embedded in Riot Web as a Service Worker, using HTTP over libp2p’s websocket transport (coordinated via a websocket rendezvous server). Architecturally, it looks like this:

Today, we’re shipping a major new alpha (v0.1.1) of this P2P demo up at https://p2p.riot.im (requires desktop Chrome or Firefox in non-private-browsing mode) - which hopefully should give a really usable and concrete taste of the shape of things to come.

The main features are:

• Your conversations are now persisted in your browser storage (via IndexedDB), meaning that as long as all the browsers participating in a given conversation don’t clear their local storage, rooms on the P2P network are here to stay!
• Your room directory lists all the aliases for all the rooms published by active nodes on the network. Moreover, we now automatically publish a local room alias whenever you join a public room, so that others will be able to discover that room via you, even if the server who originally created the alias has disappeared.
• Lots and lots of federation improvements between the nodes - for instance, when a node comes online, others should now automatically detect and send scrollback to it. Invites should work, and there should no longer be any unexpectedly redacted messages.

Needless to say, all the code for this is open source under the Apache license, and if you’re feeling particularly adventurous you can embed your very own P2P Dendrite into Riot Web by using the Dockerfile at https://github.com/matrix-org/dendrite/blob/master/build/docker/DendriteJS.Dockerfile or following the instructions at https://github.com/matrix-org/dendrite/blob/master/docs/p2p.md.

Please report bugs to https://github.com/matrix-org/dendrite/issues!

Finally, please understand that the demo is very likely not what the final version of P2P Matrix will look like - this is just one step in a series of experiments as we investigate the best paths forward :)

🔗What’s next?

For the current demo, there’s still lots of stuff remaining, including:

• More federation debugging (and hooking in tardis and writing up everything we’ve learned about implementing federation in Dendrite!)
• Making the content repository work in-browser (gotta fill up those IndexedDBs with some GIFs!)
• Hooking up E2E Encryption APIs in Dendrite (not that it buys us much in a pure P2P world)
• WebRTC transports. Turns out that service workers aren’t allowed to speak WebRTC, so we’ll have to shim through to Riot to speak true peer-to-peer WebRTC data channels rather than relaying all the traffic through the websocket rendezvous server.
• Decentralised accounts for multidevice support - reviewing MSC1228 and getting Dendrite supporting multihoming accounts!
• Finishing all of Dendrite’s other remaining APIs.

Beyond this, there are some bigger picture questions left to be answered in future experiments.

Firstly: we do not yet have a solution for “store and forward” nodes which can relay messages on behalf of a room if all the participating devices are offline. A first cut will be to run a P2P-capable homeserver server-side for this, but then metadata will start to accumulate server-side for the conversations it hosts. A more interesting approach would be to use a store and forward system which obfuscates who is talking to who, such as a mixnet, and could even provide resistance to network traffic pattern analysis. This is very much an open area of research, but one we are getting into :D

Secondly: we want to experiment more with other transports, and find out which works best for Matrix. Libp2p has some really exciting new stuff in the form of Gossipsub v1.1 - a much smarter routing algorithm for pubsub traffic in libp2p, which David Dias gave us a VIP tour of at the first Open Tech Will Save Us meetup. So we’ll need to restructure our libp2p transport as pubsub to see how it works in practice. Separately, we also want to play with hooking up Yggdrasil (the encrypted overlay network) as a transport as a totally different approach - Yggdrasil will easily let us span different underlying network transports, but comes with different tradeoffs (e.g. no browser support yet). We also want to take a look at the DAT / hypercore / hyperswarm / Cabal ecosystem to see if there’s a match :)

Thirdly and finally: we obviously want to unify the new P2P Matrix network with today’s federated one. The ideal outcome here would be to have a hybrid model, where teams who want their users to have a dedicated homeserver (for availability, IT policies, etc) can continue to have one as they do today - but newbies who have just installed Riot would float around on P2P unless they decided to consciously put down roots on a server or two. Best of all, it would let us turn off the matrix.org homeserver: the best public homeserver is one you run yourself on your own phone ;) The approach we take for linking P2P and today’s Matrix will depend very much on the transport we select for P2P in the long run, but the likelihood is that today’s homeservers will sprout P2P gateways to link the networks.

🔗Conclusion

So, there you have it. P2P Matrix exists, and is developing at an alarming speed - and pushing Dendrite development along with it. Most excitingly, there have been no changes yet to the Matrix spec for P2P at all; we’ve just swapped https for http-over-libp2p as the transport. So all of the work we’ve been doing making Dendrite work in a P2P world has directly translated into making Dendrite work on today’s Matrix too You can now stand up a Dendrite and have it federate pretty reliably with the wider Matrix network, although we’re still rushing through implementing APIs (we’re up to 35% passing sytest coverage - although that 35% does contain most of the important tests :)

Finally, in case you’re worried about why the Matrix core team is off chasing P2P dreams rather than improving Riot’s UX, or implementing Communities, or Extensible Profiles, or working through the MSC backlog etc... in practice only two people (ignoring Matthew) have been working on P2P - Neil Alexander (author of the original FOSDEM demo, Dendrite wrangler and Yggdrasil co-maintainer) and Kegan Dougal (of the original Matrix dev team, one of the original authors of Dendrite, and now wrangling the WASM P2P work too). Huge thanks to Kegan & Neil for pushing P2P forwards - and huge thanks to everyone else on the core team and the wider community for keeping today’s Matrix advancing too!

Hope this has given a tempting glimpse of the shape of things to come. Honestly we never thought we’d get as far as P2P when we started Matrix back in 2014, but it’s really fun to be finally catching up with the future :D

-- Matthew

P.S. You can read more about this from Neil Alexander’s point of view over at his blog (including more thoughts on the potential Yggdrasil demo!)

P.P.S You can read the gory details of the P2P and WASM implementation from Kegan's point of view over at the Dendrite wiki.

P.P.P.S Comments over at HN

🔗Matrix Live 🎙 Demos!

This week we have demos from our core team remote sync!

🔗Dept of Spec 📜

anoa announced:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

🔗MSC Status

Merged MSCs:

• MSC2540: Stricter event validation: JSON compliance

• MSC2240: Room version 6

• MSC2209: Alter auth rules to check notifications in m.room.power_levels

• MSC1466: Soft Logout

MSCs in Final Comment Period:

• No MSCs are in FCP.

New MSCs:

• MSC2582: Remove mimetype from EncryptedFile object
• MSC2579: Improved tagging support

🔗Spec Core Team

In terms of Spec Core Team MSC focus for this week, we've still got MSC2366 (verification flows) and MSC2399 (UISI messaging), however they are now joined by a third, secret MSC that we'll be working on internally for now :)

🔗Spec releases: Room Version 6, Client-Server r0.6.1, and Federation r0.1.4

Says TravisR:

It's been a little while since we've done a round of spec releases, so we have a bunch of spec releases to share. If you're implementing something from the spec, please have a look at the news: https://matrix.org/blog/2020/05/27/room-version-6-client-server-r-0-6-1-and-federation-r-0-1-4-spec-releases/

As always, any questions about the spec itself can be answered in #matrix-spec:matrix.org, #matrix-dev:matrix.org, or in the various area-specific rooms floating around the federation.

🔗Dept of Servers 🏢

🔗Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo said:

Hello again! This week I worked on access control, which means that private rooms are private and invite-only rooms are invite-only. The public room directory only shows rooms marked as such now.

I also made a milestone to track progress on Conduit 1.0. This release marks the point where Conduit is fully usable as a non-federating homeserver. All features of the client server specification should be implemented, unless they can be seen as optional extensions (e.g. alternative registrations methods like email). Conduit should also be free of all known bugs with those features.

Adding federation on top of the 1.0 release will very likely require breaking changes. This means that the release which brings federation will be 2.0 or later and will require a partial or full database deletion.

@TravisR wrote a fix for Riot Web, so you should be able to create accounts on Conduit from <https://riot.im/app> soon.

Thanks to my supporters on Liberapay.

🔗Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander said:

Updates:

• The monolith now has a "hybrid" mode that uses the full HTTP APIs instead of short-circuiting

• A number of the internal HTTP APIs are now tested fully and have been fixed as a result, which will also improve running a full polylith component deployment

• Server keys are now managed by a separate server key API component

• Batched sends in the federation sender are now working properly, improving send performance significantly

• Error handling in the federation /send endpoint has been tweaked

• /get_missing_events now returns events that don't fail signature checks

• Federated room joins by room alias now attempt the server specified in the alias much more reliably

• The correct room ID is now returned from the PerformJoin API

• Much of the room server storage code has been deduplicated

• Fixed some bugs in the device database storage code

• Sync streams are now managed per-user-per-device, rather than just per-user

• We no longer incorrectly default to federated joins in response to local invites

Spec compliance:

• Client-Server APIs: 34% (228/672 tests)

• Federation APIs: 34% (35/103 tests)

🔗Synapse

Says Neil:

The main news this week is that we switched event persistence away from the matrix.org master process which has made a big difference to overall performance. The impact is best demonstrated by graphing message send times.

We also shipped 1.14.0 which includes OpenId Connect support and improved cache control granularity. It also contains rooms v6.

Next week will be having a think about what to do next for matrix.org performance. Watch this space.

🔗TARDIS (Time Agnostic Room DAG Inspection Service)

Matthew offered:

I made a TARDIS (Time Agnostic Room DAG Inspection Service) - a simple time-travelling debugger for Room DAGs which plugs into Dendrite's internal APIs to visualise room DAGs. The intention is to provide it as a widget which Riot can use to visualise DAGs for debugging, particularly for P2P Matrix. The time-travel bit is simply that it could let you filter out the newer events to show how the DAG has evolved over time, although that isn't hooked up yet. https://github.com/matrix-org/tardis

🔗Synapse Deployment 📥

🔗Kubernetes

Ananace said:

Just pushed the K8s-optimized images for Synapse 1.14.0 - slight changes have been done to the image to further streamline it a bit.

🔗Docker-matrix

Mathijs said:

With the release of synapse 1.14.0 the avhost image, including jemalloc, mjolnir and coturn has been updated as well, avhost/docker-matrix:v1.14.0, as always the release candidates were uploaded at mvgorcum/docker-matrix:v1.14.0rc1 and rc2

🔗Dept of SDKs and Frameworks 🧰

🔗Ruma

jplatte offered:

This week,

• iinuwa added some new endpoints to ruma-federation-api

• devinr528 wrote lots of macro code for the next major revision of ruma-events as part of his GSoC project

• Timo (author of the Conduit homeserver) reported lots of small issues with ruma-client-api of which some are now fixed already

Additionally, we released two new bugfix library versions:

• ruma-events 0.21.3 fixes an event serialization issue

• ruma-serde 0.2.2 (an internal dependency) fixed an incomplete part of querystring deserialization, so Conduit (and other future homeservers building on ruma-*-api) can properly handle querystrings in which parameters are repeated (like the server_name in /join/{roomIdOrAlias})

🔗buildbot-matrix v0.0.5

Thanks haywo for this update:

buildbot-matrix v0.0.5 has been released:

• Option to only get notifications on finished builds was added.

buildbot-matrix is a small plugin for the Buildbot CI framework which sends notifications to matrix rooms. It's available an GitHub and PyPi, take a look at https://github.com/HayWo/buildbot-matrix for more information.

🔗Dept of Bridges 🌉

🔗Let's start with the big news of the week

Half-Shot announced:

I, Half-Shot, changed the status of the IRC Bridge on https://matrix.org/docs/projects/bridge/matrix-org-irc-bridge from "Early beta" to "Stable". This does not mean it's feature complete, or even all the way there, but it sure doesn't crash nearly as often as it did when I started, so that's good enough for me.

🔗matrix-appservice-slack 1.4.0-rc1

Half-Shot offered:

Hey folks, we've released the first RC for 1.4.0 of the slack bridge. This change includes a number of bug fixes around puppeting, but includes a few nice to haves like a /ready endpoint for kube/docker deployments and setting profile information on startup. Please test, and feedback in #matrix_appservice_slack:cadair.com :)

🔗Mau-bridge multi-update

Tulir told us:

I've been working on improving the reliability of my bridges, primarily by making sure that the user knows if something went wrong. All my bridges now support delivery receipts (implemented as bridge bot read receipts) and can send delivery error messages if there was an error bridging a message. The Facebook and Hangouts bridges also now send connection status messages to a bridge notice room like WhatsApp already did before.

🔗Minetrix

Certainly not from this week, but we haven't featured it before: Minetrix allows you to bridge a Minecraft server with a Matrix room.

I read recently that Minecraft still has 126 million MAU!

🔗Dept of Clients 📱

🔗Daydream

mtrnord told us:

A Matrix webclient written in (mostly) Rust using the official Matrix-Rust-SDK and Rusts WASM Support.

Daydream is the Idea of a Rust Matrix Client. It is still in a very early state and at the time of writing this supports basic displaying of text, image and bot messages as well as sending plain text and markdown.

Try it at https://daydream.nordgedanken.dev/ or check out the Source Code at https://github.com/MTRNord/Daydream

Feel free to join the Matrix Room: #daydream:nordgedanken.dev

🔗RumaTui

Thanks devinr528 for sharing this WIP Command line Matrix client using matrix-rust-sdk:

• Display read receipts for the last few messages
• Display membership status when updated
• Join a room you have been invited to
• Client sends read receipts to server
• Display when messages have been read
• Leave a room by pressing Delete key (this should probably be a Ctrl-some key deal...)
• Specify homeserver to join on start up (before the login screen)
  • Simply run rumatui [HOMESERVER], defaults to "http://matrix.org"
• Displays errors, albeit not very helpful or specific
• Receive and display messages
  • formatted messages display as rendered markdown
• Send messages
  • local echo is removed
  • Send textbox grows as more lines of text are added
• Selectable rooms list
  • change rooms using the arrow keys, making this clickable may be difficult
• Login widget is click/arrow key navigable
  • hides password

A new notification bar under the message window displays notifications such as membership changes, typing, and read notices. I am hoping to have an actual release on crates.io by the end of the day. If you have any thoughts join #rumatui:matrix.org or check it out on github.

🔗Riot-web

Bruno said:

This week on Riot-web, we've been:

• starting work on shifting the UX for the encryption upgrade towards using recovery keys in favor of passphrases.

• starting early work on a redesigned matrix.to website

• starting looking to integrate riot desktop with the native password manager of your platform of choice through keytar

• fixing some issues with the (still experimental) IRC layout

• iterating on the (also still experimental) new room list

• looking into some issues that arose from the cross-signing release

🔗Riot-iOS

Says Manu:

Riot-iOS (and MatrixSDK and MatrixKit) is now fully compatible with iOS13 SDK. A TestFlight 0.12.0 will be available soon.
The work is still on its branches (xcode11). It comes with the new notification service mechanism and we want to run a lot of tests next week before going to production.

🔗Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) offered:

• Ikito added options to show larger emoji in the timeline and added tooltips to all settings, that provide additional explanations.

• You can now limit the width of the timeline to make long messages more readable, if you maximise Nheko on a widescreen monitor.

• Lorenzo added a pretty much complete italian translation!

• Settings that apply to the qml parts in Nheko now apply immediately without a restart.

🔗Dept of Bots 🤖

🔗midori jabbot 0.5

Torrey Searle reported:

midori jabbot 0.5 has now been released.

Jabbot is a multi-protocol chat bot that can be taught new commands in-chat

This is the first release of jabbot with support of matrix in it. This release also includes support for DISCORD.

Additionally this release is the first release that can be installed via APT

echo "deb [trusted=yes] <https://dl.bintray.com/midoricorp/deb> unstable main" | sudo tee -a /etc/apt/sources.list
apt-get update

apt-get install jabbot-full

See https://github.com/midoricorp/jabbot/blob/master/README.md for details

or goto #midori-dev:hackerspaces.be for support/live-demo

🔗Dept of Guides 🧭

🔗Cross Signing explanation

This is a great resource for helping explain how cross-signing works!

JCG said:

After explaining cross signing to others multiple times in the past and seeing the question again on mastodon, I decided to write a short blog post on the topic, explaining how the keys interact with each other and how Alice's devices then end up trusting Bob's devices. You can read it over on https://jcg.re/blog/quick-overview-matrix-cross-signing/

🔗Cross-signing implementors guide

This one isn't quite merged yet as we're going to give a thorough review, but big thank you to sorunome for taking the time to put this together:

While implementing cross-signing for fluffychat soru decided to write a guide that would help other client developers to implement that, too. So far it is still WIP and doesn't cover too much, yet, but soru might have typed up more once this TWIM goes live. The PR (and thus the markdown of the guide) can be found here: https://github.com/matrix-org/matrix.org/pull/740

🔗Dept of Event Videos 📹

Matthew spoke at Rust London meetup earlier in the week, check out the video!

(Matrix at around 29:30)

🔗New Public Rooms 🏟

Ludwig announced:

May I advertise #matrix-design:matrix.org to you.

It's a bunch of people who are passively learning design for their matrix projects. You might pick up a few things if you join :p

🔗Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	fairydust.space	350.5
2	jonas-schuermann.name	437.5
3	fachschaften.org	446
4	revreso.de	576.5
5	heitkoetter.net	598
6	matrix.vgorcum.com	634.5
7	c-base.org	694
8	fab.network	828
9	sumnerevans.com	1023
10	nltrix.net	1032

🔗Final Thoughts

You may have seen that Mastodon is preparing to implement Olm! Read some news in German or why not watch Gargon on Open Tech Will Save Us #2 where the idea was first discussed? It's like you're in the room where it happens... 🎶

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.14.0 has landed.

This release contains OpenId Connect support, so that admins can configure a whole range of SSO options. We're really excited about what this means for helping users sign up and are considering it for matrix.org.

1.14.0 also contains finer grained cache configuration support, so that admins can tune caches in a more granular way. Our experience is that cache tuning can make a big difference to Synapse performance and over time we hope to continue improving cache tooling. Potentially we could even have caches auto-tune.

You'll also find support for rooms v6. v6 contains some small but important changes to improve the security of the room model. As with all room version upgrades we can't switch to v6 as the default version immediately since we need to build up a critical mass of servers that can support it. So upgrade and watch this space for when we switch over properly.

As an aside we continue to be very happy with using Redis for worker inter-communication and will likely recommend it for production use in the coming weeks.

Get 1.14.0 from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.

Changelog since v1.13.0

🔗Synapse 1.14.0 (2020-05-28)

No significant changes.

🔗Synapse 1.14.0rc2 (2020-05-27)

🔗Bugfixes

• Fix cache config to not apply cache factor to event cache. Regression in v1.14.0rc1. (#7578)
• Fix bug where ReplicationStreamer was not always started when replication was enabled. Bug introduced in v1.14.0rc1. (#7579)
• Fix specifying individual cache factors for caches with special characters in their name. Regression in v1.14.0rc1. (#7580)

🔗Improved Documentation

• Fix the OIDC client_auth_method value in the sample config. (#7581)

🔗Synapse 1.14.0rc1 (2020-05-26)

🔗Features

• Synapse's cache factor can now be configured in homeserver.yaml by the caches.global_factor setting. Additionally, caches.per_cache_factors controls the cache factors for individual caches. (#6391)
• Add OpenID Connect login/registration support. Contributed by Quentin Gliech, on behalf of les Connecteurs. (#7256, #7457)
• Add room details admin endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. (#7317)
• Allow for using more than one spam checker module at once. (#7435)
• Add additional authentication checks for m.room.power_levels event per MSC2209. (#7502)
• Implement room version 6 per MSC2240. (#7506)
• Add highly experimental option to move event persistence off master. (#7281, #7374, #7436, #7440, #7475, #7490, #7491, #7492, #7493, #7495, #7515, #7516, #7517, #7542)

🔗Bugfixes

• Fix a bug where event updates might not be sent over replication to worker processes after the stream falls behind. (#7384)
• Allow expired user accounts to log out their device sessions. (#7443)
• Fix a bug that would cause Synapse not to resync out-of-sync device lists. (#7453)
• Prevent rooms with 0 members or with invalid version strings from breaking group queries. (#7465)
• Workaround for an upstream Twisted bug that caused Synapse to become unresponsive after startup. (#7473)
• Fix Redis reconnection logic that can result in missed updates over replication if master reconnects to Redis without restarting. (#7482)
• When sending m.room.member events, omit displayname and avatar_url if they aren't set instead of setting them to null. Contributed by Aaron Raimist. (#7497)
• Fix incorrect method label on synapse_http_matrixfederationclient_{requests,responses} prometheus metrics. (#7503)
• Ignore incoming presence events from other homeservers if presence is disabled locally. (#7508)
• Fix a long-standing bug that broke the update remote profile background process. (#7511)
• Hash passwords as early as possible during password reset. (#7538)
• Fix bug where a local user leaving a room could fail under rare circumstances. (#7548)
• Fix "Missing RelayState parameter" error when using user interactive authentication with SAML for some SAML providers. (#7552)
• Fix exception 'GenericWorkerReplicationHandler' object has no attribute 'send_federation_ack', introduced in v1.13.0. (#7564)
• synctl now warns if it was unable to stop Synapse and will not attempt to start Synapse if nothing was stopped. Contributed by Romain Bouyé. (#6590)

🔗Updates to the Docker image

• Update docker runtime image to Alpine v3.11. Contributed by @Starbix. (#7398)

🔗Improved Documentation

• Update information about mapping providers for SAML and OpenID. (#7458)
• Add additional reverse proxy example for Caddy v2. Contributed by Jeff Peeler. (#7463)
• Fix copy-paste error in ServerNoticesConfig docstring. Contributed by @ptman. (#7477)
• Improve the formatting of reverse_proxy.md. (#7514)
• Change the systemd worker service to check that the worker config file exists instead of silently failing. Contributed by David Vo. (#7528)
• Minor clarifications to the TURN docs. (#7533)

🔗Internal Changes

• Add typing annotations in synapse.federation. (#7382)
• Convert the room handler to async/await. (#7396)
• Improve performance of get_e2e_cross_signing_key. (#7428)
• Improve performance of mark_as_sent_devices_by_remote. (#7429, #7562)
• Add type hints to the SAML handler. (#7445)
• Remove storage method get_hosts_in_room that is no longer called anywhere. (#7448)
• Fix some typos in the notice_expiry templates. (#7449)
• Convert the federation handler to async/await. (#7459)
• Convert the search handler to async/await. (#7460)
• Add type hints to synapse.event_auth. (#7505)
• Convert the room member handler to async/await. (#7507)
• Add type hints to room member handler. (#7513)
• Fix typing annotations in tests.replication. (#7518)
• Remove some redundant Python 2 support code. (#7519)
• All endpoints now respond with a 200 OK for OPTIONS requests. (#7534, #7560)
• Synapse now exports detailed allocator statistics and basic GC timings as Prometheus metrics (pypy_gc_time_seconds_total and pypy_memory_bytes) when run under PyPy. Contributed by Ivan Shapovalov. (#7536)
• Remove Ubuntu Cosmic and Disco from the list of distributions which we provide .debs for, due to end-of-life. (#7539)
• Make worker processes return a stubbed-out response to GET /presence requests. (#7545)
• Optimise some references to hs.config. (#7546)
• On upgrade room only send canonical alias once. (#7547)
• Fix some indentation inconsistencies in the sample config. (#7550)
• Include synapse.http.site in type checking. (#7553)
• Fix some test code to not mangle stacktraces, to make it easier to debug errors. (#7554)
• Refresh apt cache when building dh_virtualenv docker image. (#7555)
• Stop logging some expected HTTP request errors as exceptions. (#7556, #7563)
• Convert sending mail to async/await. (#7557)
• Simplify reap_monthly_active_users. (#7558)

🔗Room Version 6, Client-Server r0.6.1, and Federation r0.1.4 spec releases

Hey all,

It's been a little while since we've done a spec release, so here we are with Room Version 6, Client-Server r0.6.1, and Federation r0.1.4.

Room Version 6 (and the associated Federation r0.1.4 release) is largely something for implementations to worry about. It contains new event authorisation rules, changes to the redaction algorithm, and stricter compliance for JSON.

Client-Server r0.6.1 contains a number of clarifications as well as SSO support for authorisation, "soft logout" to avoid needlessly destroying e2e history, and new ways to publish aliases within rooms.

If you're wondering where all the E2E-by-default related MSCs are - we're doing final iterations based on the real-world feedback from the E2E-by-default launch a few weeks ago, and they are then expected to land in the upcoming Client-Server r0.7.

Here's all the MSCs that got merged since the last release:

• MSC2240: Room version 6
• MSC2457: Invalidating devices during password modification
• MSC2454: Support UI auth for SSO
• MSC2451: Remove query_auth federation endpoint
• MSC2432: Updated semantics for publishing room aliases
• MSC2324: Facilitating early releases of software dependent on spec
• MSC2313: Moderation policies as rooms
• MSC1466: Soft Logout
• MSC2540: Stricter event validation: JSON compliance
• MSC2209: Alter auth rules to check notifications in m.room.power_levels
• MSC1802: Remove the '200' value from some federation responses

🔗Client-Server r0.6.1 changelog

🔗New Endpoints

• Added /rooms/{roomId}/aliases for retrieving local aliases for a room. (#2562)

🔗Backwards Compatible Changes

• Added data structures for defining moderation policies in rooms per MSC2313. (#2434)
• Optionally invalidate other access tokens during password modification per MSC2457. (#2523)
• Add User-Interactive Authentication for SSO-backed homeserver per MSC2454. (#2532)
• Add soft-logout support per MSC1466. (#2546)
• Replaced legacy room alias handling with a more sustainable solution per MSC2432. (#2562)

🔗Spec Clarifications

• List available enum values for the room versions capability. (#2245)
• Fix various spelling errors throughout the specification. (#2351, #2415, #2453, #2524, #2553, #2569)
• Minor clarifications to token-based User-Interactive Authentication. (#2369)
• Minor clarification for what the user directory searches. (#2381)
• Fix key export format example to match the specification. (#2430)
• Clarify the IV data type for encrypted files. (#2492)
• Fix the .m.rule.contains_user_name default push rule to set the highlight tweak. (#2519)
• Clarify that an event_id is returned when sending events. (#2525)
• Fix some numbers in the specification to match their explanation text. (#2554)
• Move redaction algorithm into the room version specifications. (#2563)
• Clarify signature object structures for encryption. (#2566)
• Clarify which events are created as part of /createRoom. (#2571)
• Remove claims that the homeserver is exclusively responsible for profile information in membership events. (#2574)

🔗Server-Server (Federation) r0.1.4 changelog

🔗New Endpoints

• Add new POST /publicRooms endpoint for filtering the room directory. (#2305)
• Add new v2 /send_join and /send_leave endpoints per MSC1802. (#2547)

🔗Removed Endpoints

• Remove the unused query_auth API per MSC2451. (#2470)

🔗Spec Clarifications

• Move auth event selection to a more obvious location. (#2392)
• Fix typo in Request Authentication python example. (#2510)
• Clarify which fields are required on the key server endpoints. (#2527)
• Clarify the limits of prev_events and auth_events for PDUs. (#2538)
• Clarify which events are targeted by backfill. (#2559)
• Fix the response format of the /send endpoint. (#2560)
• Clarify signature object structures for encryption. (#2566)
• Clarify the server names to use when signing requests. (#2570)
• Clarify the state/auth chain requirements for /send_join. (#2575)
• Fix various spelling errors throughout the specification. (#2577)

🔗Matrix Live 🎙

🔗Dept of Status of Matrix 🌡

🔗Automattic invests in New Vector

Huge news, read more detail here! Check out Matrix Live with Matt Mullenweg above, too.

Matthew announced:

Automattic invested in New Vector in order to support Matrix, and are also hiring Matrix Integration Engineers! https://automattic.com/work-with-us/matrix-integrations-engineer/

Do check out the job description - this is a really fascinating role! If only I didn't have a job already...

🔗Dept of Spec 📜

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

🔗MSC Status

Merged MSCs:

• MSC2526: Add ability to delete key backups
• MSC2472: Symmetric SSSS

MSCs in Final Comment Period:

• MSC2540: Stricter event validation: JSON compliance (merge)

• MSC2240: Room version 6 (merge)

• MSC2209: Alter auth rules to check notifications in m.room.power_levels (merge)

New MSCs:

• MSC2557: Proposal to clarify spoilers

🔗Spec Core Team

In terms of Spec Core Team MSC focus for this week, the room v6 MSCs from last week's focus have all reached FCP 🎉 So we're continuing on with the E2E MSCs; MSC2366 (verification flows) and MSC2399 (UISI messaging).

We'll soon be releasing Federation API r0.1.3 and Client-Server API r0.6.1, which brings room v6.

Interestingly, this was produced with some new automation.

As if this wasn't enough, sorunome reported:

heya twimers who were excited about the emotes msc thing, soru added a few more things and it'd be cool if someone could cross-check it https://github.com/matrix-org/matrix-doc/pull/2545

🔗Dept of Servers 🏢

🔗Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo offered:

This week I finished support for end to end encryption and also implemented media uploading, downloading and thumbnail generation! See it in action here:

If you want to see it with your own eyes, create an account via https://riot.conduit.rs. We hope to push our changes to Riot upstream soon, so you can use your own Riot to connect.

Finally, I want to thank my kind supporters on Liberapay.

🔗P2P/Dendrite

Dendrite is a next-generation homeserver written in Go

kegan told us:

Dendrite has seen a number of improvements this week, including:

• Backfill will now backfill more messages than before, resulting in a snappier experience when scrolling back in a room.

• Dendrite is now compliant with Synapse on backfill behaviour.

• Inline CS API filters will now honour the timeline limit value.

• Making more Sytests pass.

• Modifications to how Dendrite tracks memberships.

• Addition of an -api flag to expose internal HTTP APIs.

• Clearer separation of internal/external HTTP APIs.

This has culminated in the following values for Are We Synapse Yet:

Client-Server APIs: 34% (227/672 tests) - up from 33%
Federation APIs: 34% (35/103 tests) - up from 27%

As for P2P....

🔗p2p.riot.im v0.1.0 released!

This is a significant milestone because we now persist Dendrite databases to IndexedDB, greatly improving UX and giving users a consistent identity on the P2P network. In addition to this, the following changes have been made:

• We now bake in the Dendrite commit SHA into the logs on startup.

• Dendrite will now restart itself in case of panics/segfaults.

Known issues with this release:

• Databases are only flushed to IndexedDB every 30 seconds, making it possible to lose the most recent messages. This has a variety of effects on the Dendrite instance, but in testing it has been able to mostly gracefully recover. Messages that you have sent will be backfilled but will be redacted. Rooms you have joined will be forgotten, and you need to be kicked/re-invited/re-join for communication to be re-established. We're looking into a better flushing mechanism going forwards.

• If you have previously visited p2p.riot.im, you MUST manually clear your service worker via chrome:serviceworker-internals or about:debugging#/runtime/this-firefox or else you might get errors such as "LinkError: WebAssembly.instantiate(): memory import 37 has no maximum limit, expected at most 4294967295".

Please do not rely on p2p.riot.im to have any form of stability. Patch version bumps (0.1.0 -> 0.1.1) will not remove databases but minor version bumps will (0.1.1 -> 0.2.0) so we can continue to progress with Dendrite development at speed.

🔗Synapse

anoa announced:

Synapse v1.13.0 has landed with SSO fixes and support in User-Interactive Authentication sessions, the ability to run in worker mode using redis as an experimental backend (which will soon be considered the default transport for Synapse workers), the ability for server admins to define password policies, some alias bugfixes, some cross-signing fixes as well as many more continuous improvements.

v1.14.0 in now in development, which will feature room v6 support.

Be sure to read the upgrade notes before upgrading.

And btw, upgrade notes != release notes

🔗Synapse Deployment 📥

🔗Kubernetes

Ananace offered:

1.13.0 tags for the K8s-optimized docker image have been pushed

🔗Docker-matrix

Mathijs announced:

The synapse 1.13.0 image including coturn and jemalloc is available as avhost/docker-matrix:v1.13.0, as always the RCs were (and are) available under mvgorcum/docker-matrix:v1.13.0rcX

🔗Dept of Bridges 🌉

🔗matrix-sms-bridge

Benedict told us:

I released a new matrix bridge called matrix-sms-bridge, which allows you to bridge matrix rooms to SMS with one telephone number only. It is build on top of matrix-spring-boot-sdk and written in kotlin. Currently, only modems via Gammu are supported, but new Providers (e.g. some REST API) can be added very easily.

I'm going to integrate it into matrix-docker-ansible-deploy to use it in my corporate-environment.

🔗mautrix-telegram

Tulir said:

mautrix-telegram v0.8.0-rc2 was released with some bugfixes and a basic implementation of Half-Shot's bridge information state event MSC. There will probably be at least one more RC with some more bugfixes in a week or two.

🔗mautrix-facebook

Tulir said:

Last week I updated the messenger library mautrix-facebook uses to the latest upstream version, which might have made things work a bit better. This week mautrix-facebook got some more bugfixes and also a new feature to bridge animated stickers on Messenger into gifs on Matrix.

🔗Dept of Clients 📱

🔗Fluffychat

sorunome reported:

Fluffychat Version 0.14.0 is out now and this is new:

• Implement image viewer

• Implement room pills

• New chat appBar showing presences and room avatars

• Implement well-known support

• Minor fixes, refactoring and performance improvements

F-Droid is already published here: https://christianpauly.gitlab.io/fluffychat-website/en/fdroid.html

PlayStore and TestFlight will follow in the next few days.

🔗Radical

stoic said:

Riot Web as Firefox Add-on? Yep, it's (still) a thing, now available as version 1.6.1 and supports search in encrypted rooms (via Radical Native)

🔗Mirage

miruka offered:

Mirage 0.5.0 has been released today:

• Major performance improvements. Although there is still work to be done, such has persisting client state between restarts to avoid initial syncs everytime, accounts will hundreds of rooms should now be able to finish loading and no longer take your entire RAM and CPU until everything freezes

• Rooms with unread messages or mentions for your user (currently only those received while the client is running) will be highlighted and sorted first

• New UI component allowing you to quickly jump between accounts in the room list, and always keep an eye how many total unread messages or mentions you have

• Messages can now be replied to

• Room members can now be kicked and banned

• New keybinds for jumping between accounts, cycling through unread rooms, replying to messages and opening links in a message

• Kinetic scrolling can now be disabled, this should fix scrolling for some trackpads incorrectly handled by Qt

• The order in which accounts are shown in the side pane can now be customized

• And a lot more, see full changelog

🔗Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) announced:

• lkito has been contributing this week to Nheko as part of his homework for university or so? Anyway, he added an option to highlight whatever message you are currently hovering over, so that you have an easier time locating the proper timestamps and buttons for that specific message. He's now working through a few other issues to improve his coding skills, like optionally having big emojis in the timeline or showing tooltips to explain the settings Nheko has.

• You can now set a /roomnick

• You can now tag rooms using the room context menu. You can assign multiple tags and create new tags as you like. This helps organizing your rooms, since Nheko supported filtering you room list using tags for a few years already, but until now you couldn't assign them.

• red_sky has been tying up the loose ends for reaction support. Once that is done, we'll release 0.7.2

🔗RiotX Android

valere offered:

The team is working on getting feature parity with old-riot

• Identity server support has been merged to develop
• Switch language

1:1 Voip calls, Widgets, and room attachment list screen (media, files) are work in progress, and coming next

🔗Matrix Android SDK

v0.9.35 has been released

• Fixes Favs bug (no more favs after update)

=> Riot android release coming soon with SDK update

🔗Riot-iOS

From Manu:

This week, we released Riot-iOS 0.11.5 with the support of the new version of SAS (emoji) verification. This release contains also a fix for share and Siri extensions to make them compatible with cross-signing.
We continued to make progress on the full support of the iOS13 SDK. The new push extension is almost here but it is tricky to make it work as VoIP pushes we abused for years.

🔗Dept of SDKs and Frameworks 🧰

🔗Ruby

Ananace told us:

Just released version 2.1.0 of the Ruby SDK, headlines changes are the addition of request IDs (and timing) to the debug output to make request/response tracking easier, as well as finalized support for extending both the low-level and higher-level clients with MSCs - along with a much improved implementation for MSC2108 (Sync over SSE). Also included are some bug fixes, as usual, as well as many improvements to the provided example bot.

You can still find it on rubygems as matrix_sdk, and feel free to drop by #ruby-matrix-sdk:kittenface.studio for questions, comments, or just general discussion.

🔗Ruma

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

jplatte told us:

🔗Rust bits

This week, we worked on support for endpoints with multiple values for one query parameter (in a livestream), federation endpoints for joining rooms and the next revision of ruma-events [1, 2] (that will, once finished, allow us to finally support redacted events properly, amongst other things). We also fixed two small bugs in ruma-client-api. Just today, we released ruma-client-api 0.9.0 and ruma-client 0.4.0 (sorry, no change log ☹︎).

🔗www.ruma.io

Our website has been revamped! It has been ported from Middleman to Zola and has received some style improvements. RSS support has temporarily been removed, but you can expect it to be restored along with some content updates soon.

🔗Dept of Guides 🧭

🔗New documentation from Nataraj

Nataraj offered:

I have written a guide about configuring bridge to FreeNode: https://github.com/matrix-org/matrix-appservice-irc/wiki/Setting-Up-Bridge-To-FreeNode

And wrote a HowTo for installation matrix-appservice-irc on Debian: https://wiki.debian.org/matrix-appservice-irc

🔗Dept of Tulir 🇫🇮

A rather infrequent section.

Tulir offered:

Since multiple people have asked me about it, I now have a GitHub sponsor thing where you can get a fancy flair in my rooms and access to a highly exclusive sponsor room.

I didn't have any public donation things earlier due to the Finnish money collection act, but I realized that I can bypass that by selling things (like access to a Matrix room) instead of it only being a donation.

Completely unrelated to that: Following the bridge info state event implementation in my bridges, I've made a PR to Riot web to fix some bugs related to showing the bridge info there: https://github.com/matrix-org/matrix-react-sdk/pull/4621

🔗Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	fairydust.space	329
2	envs.net	429
3	maunium.net	473.5
4	danielvinci.com	540
5	maescool.be	565
6	matrix.vgorcum.com	614
7	aragon.sh	774
8	dmnd.sh	1448
9	utzutzutz.net	1689.5
10	kittenface.studio	2009.5

🔗Final thoughts 💭

Last week we featured some Sytest output and compared the results between homeservers, which was fun! We'll bring this back as an occasional feature.

I didn't feature this last month: GNU Radio are now using Matrix.

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Hi all,

We’re very excited indeed to announce that Automattic, the creators of WordPress.com, are jumping head first into the Matrix ecosystem with a strategic investment of almost $5M into New Vector (the company which makes Riot and Modular.im, founded by the core Matrix team in 2017). More importantly, Matt Mullenweg (co-founder of WordPress and founder of Automattic) and the Automattic gang are committing to make the most of Matrix in their work going forwards!

This is huge news, not least because WordPress literally runs over 36% of the websites on today’s web - and the potential of bringing Matrix to all those users is incredible. Imagine if every WP site automatically came with its own Matrix room or community? Imagine if all content in WP automatically was published into Matrix as well as the Web? (This isn’t so far fetched an idea - turns out that Automattic already runs a XMPP bridge for wordpress.com over at im.wordpress.com!). Imagine there was an excellent Matrix client available as a WordPress plugin for embedding realtime chat into your site? Imagine if Tumblr (which is part of Automattic these days) became decentralised!?

In fact, if you’re a developer in either the Matrix or WordPress communities, now might be a good time to think about how to cross the streams.... not least because Automattic just opened up a role for a Matrix.org/WordPress Integrations Engineer! Quite aside from the investment, this shows Automattic is serious about Matrix - and we’d like to thank them for opening up jobs in these challenging times to further accelerate Matrix. Perhaps some day Matrix Engineer will be as common a career choice as Web Developer ;)

That said, it’s super early days for integration work, and there isn’t a concrete project to announce yet beyond the investment in New Vector (which is effectively an extension of the funding NV raised in October) and Automattic’s Job opening - but these are the sort of ideas we’ve been kicking around. And at the very least, we should expect to see Automattic’s communities migrating over to Matrix in the coming months.

It’s been loads of fun working with Matt and the team on this: we see a huge overlap in terms of a genuine love for the open web, open source and open standards. It’s also no coincidence that Matt (independently of Automattic) donated substantially to Matrix via Patreon back in 2017 when we needed it the most. We’re also looking forward to benefiting from Automattic’s experience in sustainably and responsibly funding and growing open source projects in general - WordPress.com is an excellent example of how one can support development of a project like WordPress without compromising its open source nature.

So, we’d like to formally welcome WordPress and the rest of the Automattic family into Matrix. It’s incredibly exciting times, and we can’t wait to see what will come of the partnership! And meanwhile, if any other massive open source organisations want to join Automattic and Mozilla in leaping into Matrix, you know where to find us… :D

Huge thanks go to Matt for believing in Matrix - watch this space for updates.

• Matthew, Amandine & the Matrix Team.

Synapse 1.13.0 is here and it's a whopper!

Highlights include new support for User Interactive Authentication (UIA) for Single Sign-on (SSO) installations. This means that for the first time features that require the user to re-authenticate are available for servers that authenticate by SSO. Notably this means that these servers now support cross signing!

SSO admins should take a look at the SSO notes in the changelog.

We have been working hard on performance for large scale installations. Anyone supporting more than a few thousand users is probably running Synapse in worker mode. This means splitting out functionality from the master process and making use of multiple cores (or machines) to spread the load. Cross process communication was previously handled by a home grown TCP based replication protocol. As part of our ongoing efforts to improve performance we have replaced this replication system with Redis and have been running Redis in production on matrix.org for the past 2 weeks.

Redis itself does not provide a significant performance win directly but it means that it is much less expensive to add new workers to a cluster. The topology of the old system meant that every additional worker carried a small but not insignificant overhead to the master process. Since moving to Redis we have doubled the number of workers backing matrix.org.

We still consider Redis to be experimental and admins should not feel obliged to upgrade. However it looks very promising and we are likely to deprecate the old replication system in future. So watch this space - Redis is the future.

Finally we've seen lots of improvements to our documentation and many thanks for those in the community making contributions in this area.

It is also worth noting for those of you contributing to Synapse that the develop branch is now the default. More details in the changelog, so take a look.

Get 1.13.0 from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.

Changelog since v1.12.4

🔗Synapse 1.13.0 (2020-05-19)

This release brings some potential changes necessary for certain configurations of Synapse:

• If your Synapse is configured to use SSO and have a custom sso_redirect_confirm_template_dir configuration option set, you will need to duplicate the new sso_auth_confirm.html, sso_auth_success.html and sso_account_deactivated.html templates into that directory.
• Synapse plugins using the complete_sso_login method of synapse.module_api.ModuleApi should instead switch to the async/await version, complete_sso_login_async, which includes additional checks. The former version is now deprecated.
• A bug was introduced in Synapse 1.4.0 which could cause the room directory to be incomplete or empty if Synapse was upgraded directly from v1.2.1 or earlier, to versions between v1.4.0 and v1.12.x.

Please review UPGRADE.rst for more details on these changes and for general upgrade guidance.

🔗Notice of change to the default git branch for Synapse

With the release of Synapse 1.13.0, the default git branch for Synapse has changed to develop, which is the development tip. This is more consistent with common practice and modern git usage.

The master branch, which tracks the latest release, is still available. It is recommended that developers and distributors who have scripts which run builds using the default branch of Synapse should therefore consider pinning their scripts to master.

🔗Features

• Extend the web_client_location option to accept an absolute URL to use as a redirect. Adds a warning when running the web client on the same hostname as homeserver. Contributed by Martin Milata. (#7006)
• Set Referrer-Policy header to no-referrer on media downloads. (#7009)
• Add support for running replication over Redis when using workers. (#7040, #7325, #7352, #7401, #7427, #7439, #7446, #7450, #7454)
• Admin API POST /_synapse/admin/v1/join/<roomIdOrAlias> to join users to a room like auto_join_rooms for creation of users. (#7051)
• Add options to prevent users from changing their profile or associated 3PIDs. (#7096)
• Support SSO in the user interactive authentication workflow. (#7102, #7186, #7279, #7343)
• Allow server admins to define and enforce a password policy (MSC2000). (#7118)
• Improve the support for SSO authentication on the login fallback page. (#7152, #7235)
• Always whitelist the login fallback in the SSO configuration if public_baseurl is set. (#7153)
• Admin users are no longer required to be in a room to create an alias for it. (#7191)
• Require admin privileges to enable room encryption by default. This does not affect existing rooms. (#7230)
• Add a config option for specifying the value of the Accept-Language HTTP header when generating URL previews. (#7265)
• Allow /requestToken endpoints to hide the existence (or lack thereof) of 3PID associations on the homeserver. (#7315)
• Add a configuration setting to tweak the threshold for dummy events. (#7422)

🔗Bugfixes

• Don't attempt to use an invalid sqlite config if no database configuration is provided. Contributed by @nekatak. (#6573)
• Fix single-sign on with CAS systems: pass the same service URL when requesting the CAS ticket and when calling the proxyValidate URL. Contributed by @Naugrimm. (#6634)
• Fix missing field default when fetching user-defined push rules. (#6639)
• Improve error responses when accessing remote public room lists. (#6899, #7368)
• Transfer alias mappings on room upgrade. (#6946)
• Ensure that a user interactive authentication session is tied to a single request. (#7068, #7455)
• Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors. (#7089)
• Return the proper error (M_BAD_ALIAS) when a non-existent canonical alias is provided. (#7109)
• Fix a bug which meant that groups updates were not correctly replicated between workers. (#7117)
• Fix starting workers when federation sending not split out. (#7133)
• Ensure is_verified is a boolean in responses to GET /_matrix/client/r0/room_keys/keys. Also warn the user if they forgot the version query param. (#7150)
• Fix error page being shown when a custom SAML handler attempted to redirect when processing an auth response. (#7151)
• Avoid importing sqlite3 when using the postgres backend. Contributed by David Vo. (#7155)
• Fix excessive CPU usage by prune_old_outbound_device_pokes job. (#7159)
• Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature. (#7177)
• Fix a bug which could cause incorrect 'cyclic dependency' error. (#7178)
• Fix a bug that could cause a user to be invited to a server notices (aka System Alerts) room without any notice being sent. (#7199)
• Fix some worker-mode replication handling not being correctly recorded in CPU usage stats. (#7203)
• Do not allow a deactivated user to login via SSO. (#7240, #7259)
• Fix --help command-line argument. (#7249)
• Fix room publish permissions not being checked on room creation. (#7260)
• Reject unknown session IDs during user interactive authentication instead of silently creating a new session. (#7268)
• Fix a SQL query introduced in Synapse 1.12.0 which could cause large amounts of logging to the postgres slow-query log. (#7274)
• Persist user interactive authentication sessions across workers and Synapse restarts. (#7302)
• Fixed backwards compatibility logic of the first value of trusted_third_party_id_servers being used for account_threepid_delegates.email, which occurs when the former, deprecated option is set and the latter is not. (#7316)
• Fix a bug where event updates might not be sent over replication to worker processes after the stream falls behind. (#7337, #7358)
• Fix bad error handling that would cause Synapse to crash if it's provided with a YAML configuration file that's either empty or doesn't parse into a key-value map. (#7341)
• Fix incorrect metrics reporting for renew_attestations background task. (#7344)
• Prevent non-federating rooms from appearing in responses to federated POST /publicRoom requests when a filter was included. (#7367)
• Fix a bug which would cause the room directory to be incorrectly populated if Synapse was upgraded directly from v1.2.1 or earlier to v1.4.0 or later. Note that this fix does not apply retrospectively; see the upgrade notes for more information. (#7387)
• Fix bug in EventContext.deserialize. (#7393)
• Fix a long-standing bug which could cause messages not to be sent over federation, when state events with state keys matching user IDs (such as custom user statuses) were received. (#7376)
• Restore compatibility with non-compliant clients during the user interactive authentication process, fixing a problem introduced in v1.13.0rc1. (#7483)
• Hash passwords as early as possible during registration. (#7523)

🔗Improved Documentation

• Update Debian installation instructions to recommend installing the virtualenv package instead of python3-virtualenv. (#6892)
• Improve the documentation for database configuration. (#6988)
• Improve the documentation of application service configuration files. (#7091)
• Update pre-built package name for FreeBSD. (#7107)
• Update postgres docs with login troubleshooting information. (#7119)
• Clean up INSTALL.md a bit. (#7141)
• Add documentation for running a local CAS server for testing. (#7147)
• Improve README.md by being explicit about public IP recommendation for TURN relaying. (#7167)
• Fix a small typo in the metrics_flags config option. (#7171)
• Update the contributed documentation on managing synapse workers with systemd, and bring it into the core distribution. (#7234)
• Add documentation to the password_providers config option. Add known password provider implementations to docs. (#7238, #7248)
• Modify suggested nginx reverse proxy configuration to match Synapse's default file upload size. Contributed by @ProCycleDev. (#7251)
• Documentation of media_storage_providers options updated to avoid misunderstandings. Contributed by Tristan Lins. (#7272)
• Add documentation on monitoring workers with Prometheus. (#7357)
• Clarify endpoint usage in the users admin api documentation. (#7361)

🔗Deprecations and Removals

• Remove nonfunctional captcha_bypass_secret option from homeserver.yaml. (#7137)

🔗Internal Changes

• Add benchmarks for LruCache. (#6446)
• Return total number of users and profile attributes in admin users endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. (#6881)
• Change device list streams to have one row per ID. (#7010)
• Remove concept of a non-limited stream. (#7011)
• Move catchup of replication streams logic to worker. (#7024, #7195, #7226, #7239, #7286, #7290, #7318, #7326, #7378, #7421)
• Convert some of synapse.rest.media to async/await. (#7110, #7184, #7241)
• De-duplicate / remove unused REST code for login and auth. (#7115)
• Convert *StreamRow classes to inner classes. (#7116)
• Clean up some LoggingContext code. (#7120, #7181, #7183, #7408, #7426)
• Add explicit instance_id for USER_SYNC commands and remove implicit conn_id usage. (#7128)
• Refactored the CAS authentication logic to a separate class. (#7136)
• Run replication streamers on workers. (#7146)
• Add tests for outbound device pokes. (#7157)
• Fix device list update stream ids going backward. (#7158)
• Use stream.current_token() and remove stream_positions(). (#7172)
• Move client command handling out of TCP protocol. (#7185)
• Move server command handling out of TCP protocol. (#7187)
• Fix consistency of HTTP status codes reported in log lines. (#7188)
• Only run one background database update at a time. (#7190)
• Remove sent outbound device list pokes from the database. (#7192)
• Add a background database update job to clear out duplicate device_lists_outbound_pokes. (#7193)
• Remove some extraneous debugging log lines. (#7207)
• Add explicit Python build tooling as dependencies for the snapcraft build. (#7213)
• Add typing information to federation server code. (#7219)
• Extend room admin api (GET /_synapse/admin/v1/rooms) with additional attributes. (#7225)
• Unblacklist '/upgrade creates a new room' sytest for workers. (#7228)
• Remove redundant checks on daemonize from synctl. (#7233)
• Upgrade jQuery to v3.4.1 on fallback login/registration pages. (#7236)
• Change log line that told user to implement onLogin/onRegister fallback js functions to a warning, instead of an info, so it's more visible. (#7237)
• Correct the parameters of a test fixture. Contributed by Isaiah Singletary. (#7243)
• Convert auth handler to async/await. (#7261)
• Add some unit tests for replication. (#7278)
• Improve typing annotations in synapse.replication.tcp.streams.Stream. (#7291)
• Reduce log verbosity of url cache cleanup tasks. (#7295)
• Fix sample SAML Service Provider configuration. Contributed by @frcl. (#7300)
• Fix StreamChangeCache to work with multiple entities changing on the same stream id. (#7303)
• Fix an incorrect import in IdentityHandler. (#7319)
• Reduce logging verbosity for successful federation requests. (#7321)
• Convert some federation handler code to async/await. (#7338)
• Fix collation for postgres for unit tests. (#7359)
• Convert RegistrationWorkerStore.is_server_admin and dependent code to async/await. (#7363)
• Add an instance_name to RDATA and POSITION replication commands. (#7364)
• Thread through instance name to replication client. (#7369)
• Convert synapse.server_notices to async/await. (#7394)
• Convert synapse.notifier to async/await. (#7395)
• Fix issues with the Python package manifest. (#7404)
• Prevent methods in synapse.handlers.auth from polling the homeserver config every request. (#7420)
• Speed up fetching device lists changes when handling /sync requests. (#7423)
• Run group attestation renewal in series rather than parallel for performance. (#7442)
• Fix linting errors in new version of Flake8. (#7470)
• Update the version of dh-virtualenv we use to build debs, and add focal to the list of target distributions. (#7526)