This Week in Matrix 2020-07-17

Matrix Live 🎙

Let's hear from Christian about working on his dream projects, his thoughts on bridging, hosting, and the importance and nature of chatbots and assistants.

Dept of Status of Matrix 🌡️

"Element"

This week was the announcement of the much-awaited rebrand: Riot is now Element. In fact, three brands are coalescing into one: Riot and New Vector will be referred to as Element, while the SaaS platform known as Modular.im is now Element Matrix Services.

Note that Matrix is not involved in this change. Matrix is still Matrix, don't worry about that!

CommCon Virtual 2020, with added Matrix

CommCon is an event dedicated to Real-Time Communications. In 2020 they made the difficult decision to go online-only, but had a fear of missing out on the "hallway-track" that is so important to industry events. Their solution was to include a live chat to run alongside their streamed talks!

Matrix was a natural choice for the crowd, but they wanted a way to encourage viewers to join the correct room from their own Matrix clients. To do this, they used the GSOC project from arnav-t - an HTML Embedded client - to present a live scrolling-view of the chat. You can see the result at https://2020.commcon.xyz/live/.

Dept of Spec 📜

uhoreg said:

anoa is away this week, so no pretty graph, I'm afraid. (this is intolerable - BP)

MSC status

Merged MSCs

• No MSCs were merged this week.

MSCs in Final Comment Period:

• No MSCs in Final Comment Period this week.

New MSCs:

• MSC2689: Fix E2EE for guests

Spec Core Team

This week, the Spec Core Team will be focusing on MSC2610 (Removing m.login.oauth2 from User-Interactive Authentication), MSC2611 (Removing m.login.token from User-Interactive Authentication), MSC2663 (Errors for dealing with non-existent push rules), MSC2674 (Event Relationships), and MSC2689 (Fix E2EE for guests).

Dept of GSoC 🎓️

Ruma/Matrix GSoC update

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

devinr528 reported:

The End is Nigh

This week in the ruma/matrix Google Summer of Code project, I worked on refactoring both ruma-api and ruma-events. After moving some of the larger chunks of the ruma_api_macro::api::Api::to_tokens method to helper functions, I spent time removing repetition from the Request/Response code generated by the ruma_api! macro. For ruma-events, the input parsing was changed to only allow valid names for the Any*Event enums. Altering the input parsing had the added benefit of replacing all of the string comparison and manipulation with strongly typed comparison and manipulation.

The final few issues to be resolved before the next crates.io release for ruma-events can happen are related to redacted events. Support for redacted events was added to the Any*Event enums, they now have redacted variants of each event kind. A few follow-up PR's have been merged to fully integrate redacted events into ruma-events, fixing specific event deserialization issues and splitting the UnsignedData struct into Unsigned and RedactedUnsigned.

HTML Embeddable Matrix

arnav-t reported:

This week for the GSoC project of HTML Embeddable Matrix Client:

• Added a read receipts menu for messages.

• Minor bug fixes

• Automatic hyperlinking of URLs in pipeline

See above for details of this project being used in the wild!

mautrix-go

nikofil announced:

• Added the capabilities to request room keys from other devices, as well as share room keys with other devices that requested them

• Currently working on device verification using SAS, will then work on key export / import

neb

• Updated to latest mautrix version with some minor fixes, most notably using m.encryption event parameters

Opsdroid(GSoC)

tyagdit said:

End to End Encryption with matrix

• The new connector has been fully implemented!

• Device verification has been put on hold for the time being

• No extra configuration is required

• Check out all the changes here

Matrix database

• Work has begun on porting the matrix database module to matrix-nio and integrating it into opsdroid core

• It uses matrix rooms as databases

• Users can store key value pairs as state events in a room

• Also works with encrypted rooms to allow for encrypted storage

matrix-ircd GSOC

karlik announced:

for matrix-ircd:

• Merge in futures 0.3 support for matrix and http modules

• Opened pull request for updating tests, irc module, and bridge module to futures 0.3 & remove several futures dependencies

• updating several packages TLS packages to their more modern counterparts

Dept of Servers 🏢

matrix-media-repo

matrix-media-repo is a highly customizable multi-domain media repository for Matrix

TravisR said:

matrix-media-repo has just released v1.1.3 with several improvements and bug fixes - please give it a go, and get support in #media-repo:t2bot.io

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo told us:

• @the0 finished /event

• I rebased Jamie's WIP appservice branch

• I wrote some documentation for our database: https://git.koesters.xyz/timo/conduit/wiki/Database

I also investigated why so many sytests fail and created this issue: https://github.com/matrix-org/sytest/issues/913 Apparently Sytest did not expect a server to optimize their /sync responses as much as Conduit does :P.

Note: The official Conduit server is now reachable at "https://conduit.koesters.xyz". Thanks to everyone who supports me on Liberapay or Bitcoin!

Synapse

Neil said:

A big week for matrix.org performance.

Hot on the heels of shipping shardable federation readers last week, this week we shipped shardable federation senders. There is still plenty of work to do, but together these changes has made a massive difference to federation lag overall and hopefully those of you not on matrix.org are noticing the difference when you talk to matrix.org users.

This graph shows the impact to outbound federation lag.

Aside from that we also shipped shardable push and frontend proxy workers as both were starting to max out on CPU as well as a shardable client reader, allowing us to shard registration which was especially important this week :)

Finally we moved typing notifications from the master process and optimised incoming replication queuing to buy us a little more head room.

Next steps are to revisit where all the remaining cycles are going on the master process. To help us profile we are migrating to async/await semantics and Patrick produced this natty graph to track progress.

Outside of performance we shipped a bug fix to prevent large initial syncs taking out the synchrotrons. The admin api sprouted an end point to list room members (thanks awesome-michaeland the ability to reactivate previously deactivated users.

Coming up we’ll dust off the notifications project which has been put on hold while Riot transmogrified into Element and we’ll continue with chipping away at the master process.

Synapse Deployment 📥️

Kubernetes

Ananace said:

And another bump of the K8s-optimized Synapse images, this time to 1.17.0

Synapse adoption graphs

We only featured this a month ago, but I want to call attention again to this awesome Synapse version adoption tracking project from Chris . I wanted to know how quickly Synapse 1.17.0 would be the most deployed version (~36 hours), and the answer is right there!

Dept of Bridges 🌉

mautrix-twitter

Tulir announced:

I started working on a Twitter DM bridge that uses the internal API instead of the complicated official one. It should be easier to set up than bridges using the official API, since there won't be a need to get API keys and forward webhooks. The main risk is that Twitter doesn't like people using the internal API and starts blocking users like Facebook does, but hopefully that doesn't happen.

So far I've only made a simple library for the API, but the bridge should be usable by the next TWIM. The bridge and API library are at https://github.com/tulir/mautrix-twitter and there's a discussion room at #twitter:maunium.net.

Dept of Clients 📱

Element Android 1.0.0

benoit announced:

Element Android 1.0.0 is out! Download or upgrade it from the PlayStore at https://play.google.com/store/apps/details?id=im.vector.app or download it from the F-Droid store at https://f-droid.org/packages/im.vector.app/

• PlayStore users: Element should come as a regular upgrade of the application Riot-Android, and the upgrade will update your previous account and data, without the need to log in again.

• F-Droid users: Element is a new app, you have to install it and log in again.

• RiotX: RiotX will disappear from the PlayStore (it was only a beta application). We will provide a very last update to inform users to install Element Android

Please report any issues at https://github.com/vector-im/riotX-android (which will probably be renamed soon), because Element Android is actually RiotX code! Thanks for all the contributors of RiotX, we still have lots to do to make the app even better and full featured.

Radical

Davo announced:

After seeing all the Element room avatars, I thought to myself, "you know what, #radical-webext:matrix.org needs a new icon"… so I sat down and combined a couple of icons. The result:

Radical also quickly (as always) got updates when Element 1.7.0 and 1.7.1 were released. Huge thank you to stoic for making Radical in the first place.

Element F-Droid repos

krombel announced:

I updated my F-Droid repos which are containing the dev builds to reflect the latest naming change to Element.

The people who formerly used my repos for getting the dev repo of Riot-Android should update to the new repo. It wont be updated to get Element

As always you can grab the F-Droid or GPlay flavor and you can pick the repo which reflects your flavor on https://fdroid.krombel.de

Note: As "the new repo" is internally the same repo as the one of RiotX (and Riot.imX) the people who already used that don't need to update their packet source. They will get the updates to Element via the old repo urls.

Feel free to use it 🙂

Dept of SDKs and Frameworks 🧰

maubot

Tulir announced:

As promised last week, maubot now supports end-to-end encryption. It's a bit bare at the moment, so you need to manually insert the device ID into the database for it to start working, but other than that it works. Plugins don't need to be changed at all, the framework will handle everything.

While I was adding e2ee to maubot, I also improved mautrix-python's crypto stuff so that it's easier to use it directly as a client library with e2ee. Some day I might even add docs :D

matrix-appservice-cr

Chris Vincent reported:

Related to my previous update here, I've just published an initial version of a Matrix Client-Server client library for the Crystal programming language: https://github.com/cvincent/matrix-client-cr

Is this the first time we've seen tooling for Crystal?

Dept of Ops 🛠

matrix-docker-ansible-deploy

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi announced:

matrix-docker-ansible-deploy is now ready to help you with the Riot -> Element transition.

We have a few ways to handle the migration depending on how much change and breakage you're willing to tolerate. See our Migrating to Element documentation page.

Incredibly thorough migration paths here!

Dept of Bots 🤖

Matrix-Alertmanager bot v0.3.0

jaywink offered:

Matrix-Alertmanager bot has a new release of v0.3.0. Highlights are compatibility with AWS Lambda, better error handling if message fails to send and the ability to do a @room mention on firing alerts. Find it here.

cody 0.6.0

carl reported:

REPL for your matrix rooms

This week, we got support for javascript (nodejs) with !js. The code

complexity was considerably reduced, which should make adding new languages much easier in the future. All Python sub-dependencies are now pinned, making

cody builds even more reproducible.

Chat with cody: @cody:bordum.dk

Read the source: https://gitlab.com/carlbordum/matrix-cody

Dept of Interesting Projects 🛰️

Noteworthy (Matrix over Wireguard overlay networks)

balaa said:

Noteworthy team (patrick and myself) have open sourced both components (spoke & hub) of Noteworthy (Matrix over Wireguard overlay networks -- https://github.com/decentralabs/noteworthy) we are actively welcoming testers/contributors and working with a handful of projects on incorporating our deployment model over in #noteworthy:tincan.community -- it is also the fastest way (less than a minute) to deploy your own home server! Also, working on incorporating Noteworthy into the popular ansible deployment repo to enable users who don't have access to a publicly accessible to deploy their own home servers.

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	fairydust.space	390
2	exp.farm	555.5
3	matrix.vgorcum.com	768
4	tchncs.de	810.5
5	settgast.org	1026
6	moritzdietz.com	1046
7	ragon.xyz	1288
8	aragon.sh	1383
9	elcyb.org	1755.5
10	kapsi.fi	2034

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!