This Week in Matrix 2020-04-17

🔗Matrix Live 🎙

Matthew & Amandine update on cross-signing

🔗Dept of Spec 📜

anoa offered:

Here's your weekly spec update!

🔗MSC Status

Merged MSCs:

• No MSCs were merged this week

MSCs in Final Comment Period:

• No MSCs are currently in FCP

New MSCs:

• MSC2499: Fixes for Client Well-known URI

🔗Spec Core Team

We had our first Spec Core Team retro in a very long time this week, with a new regular schedule of every month. It ran about an hour overtime - there was a lot to talk about. We'll post the conclusion in a separate blog post soon.

🔗Dept of Servers 🏢

🔗Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander reported:

• Rooms now default to version 4

• Backfill messages are now topologically ordered

• Searching for backward extremities when backfilling is now improved by preventing duplicate prev events

• HTTP tracing and SQL tracing environment variables have now been added

• Direct key requests are now parallelised (which can help to speed up room joins substantially)

• Events in /messages responses are now topologically ordered

• gomatrixserverlib can now perform signature checks with strict validity checking as optional

• We now return far less 500 errors over federation which helps federation with Synapse

🔗p2p.riot.im

kegan announced:

New version released which supports v4 rooms as well as improvements in how we publish rooms into directories of local nodes.

🔗Conduit (Rust Homeserver)

A Matrix homeserver written in Rust https://conduit.rs

timo announced:

This week I spent some time improving Riot support. Now there should be a lot less warnings and some new features: typing events, read receipts, better room directory, invites, user search.

The only bigger bug that exists currently is that the message history is not automatically loaded when you join a room. Sometimes reloading the page helps, but not always. The reason for that is that Riot calls the deprecated /initialSync endpoint, which I do not support (I hope someone works on Riot to fix that).

In the next few weeks I plan to improve the single-homeserver experience further and add permission management, while also working on federation.

Check out the repository on https://git.koesters.xyz/timo/conduit and our Matrix room on #conduit:koesters.xyz. If you want to try out the server yourself, you can try to connect to it using https://riot.nordgedanken.de/#/register (though it might be offline when you read this).

I also want to thank my supporters on Liberapay for their kindness 😊.

I tested the performance of Conduit yesterday by running a ruma client that sends a /send request in a for loop 10k times, which took ~46.8 seconds (all locally on my pc). This means the server can handle ~210 /sends per second under these circumstances.

The flamegraph shows what the cpu is spending most of its time doing. Plateaus are things that take time. You can see most of the time is used for a function in sled::io (I spoke with one of the devs and they said this is expected, but will be improved in the next release)

🔗Synapse

Neil told us:

We continue to focus on performance and spent a good chunk of time improving worker replication test coverage to build confidence that our upcoming changes will work as expected. Erik has been working on how to how to factor out cache invalidation which is now starting to take good shape. Once we have that, factoring out things like event persistence are next on the horizon.

Aside from that we’ve been working with the Riot client teams as they near release for cross signing, fixing bugs and improving performance on key actions.

SSO support continues to improve, and more fixes for user inactive auth for worker'd installs will land next week. Poor old Rich has been banging his head against a sync bug blocking develop, but hopefully he has broken its back now, and we'll get that merged asap.

Finally we experienced some operational issues with push over the weekend, so we spent some time cleaning up and figuring out how to make things more resilient.

Coming up, performance, cross signing support, SSO improvements.

🔗The Construct

Construct is a performance-focused homeserver written in C++

Tulir reported:

Update from Jason:

The Construct server made significant progress this week with end-to-end encryption, client compatibility, application services support, and build-related enhancements. I'd like to thank Yan Minari for contributing spec-compliant code paths for application services in the user registration system. Special thanks this week to Tony O. for continued work on the NixOS (#nix:matrix.org) derivation for reproducible builds, in addition to contributing fixes to Nio for optimal compatibility with Construct. I'd also like to thank Wessel for contributing a much needed reorganization of the Dockerfiles in preparation for Construct operating on many more platforms. Many thanks to Nico for working with the team this week in #test:zemos.net on various compatibility fixes to mtxclient and Nheko; the responsiveness when using Nheko with Construct is mind-blowing. I'd also like to thank David Vo (a.k.a. auscompgeek) for also contributing to Nio toward Construct compatibility.

Finally, I owe much appreciation to tulir, jochen, qy and others for setting up brand new installations this week and providing feedback in #test:zemos.net. Construct is the first and only federating alternative to Synapse. It is written in C++ for maximum performance, the lowest possible TCO, and designed for horizontal scaling with Construct Cluster. If you are interested in starting a low-cost Matrix hosting service, Construct may be your ideal choice. To stay informed and show your support please visit us in #test:zemos.net today!

🔗Dept of Bridges 🌉

🔗Matrix.org bridges bumper pack edition

Half-Shot announced:

Hello, this week has been busy for us over in bridge corner. We've released:

🔗matrix-appservice-bridge 1.12.2

Which includes a bunch of small fixes, including a performance improvement to StateLookup hits. Apparently it's bad to hit the homeserver with 1000s of concurrent /state requests on startup. Most importantly it also includes a critical fix to the addAppServicePath function, which previously did not authenticate requests, see the CHANGELOG for more information on that.

🔗matrix-appservice-irc 0.17.0-rc3

We're still working on the RCs for this release, having discovered and fixed a bunch of failures that can happen around getting connected to IRC. Be aware that the above security fix is applied in this release, which means provisioning requests will now be authenticated.

🔗matrix-appservice-slack 1.2.0-rc1

This is another big load of bug fixes, with many thanks to our new bridge crew member, Christian!. This release includes the above security fixes, as well as some metrics improvements to track activity on the bridge.

🔗matrix-appservice-gitter 0.1.1

The gitter bridge has been updated to include the changes from the matrix-appservice-bridge releases, as well as updating some dependencies.

🔗Dept of Clients 📱

🔗rumatui

WIP Command line Matrix client using matrix-rust-sdk

devinr528 announced:

A clickable command-line client Matrix written in Rust. Although, still in the early stages, enough has been implemented to show a working demo. rumatui uses tui to power the UI and matrix-rust-sdk for all communication with the server. So far the message formatting has been the most visually satisfying feature, although implementing it was not much fun. The goal of the project is to have a UI driven command-line client, useable from either keyboard and/or mouse.

Big thanks to @poljar for allowing me to work on matrix-rust-sdk, @jplatte for the opportunity to work with the ruma crates and everyone in the Matrix community for being so helpful!

Check it out https://github.com/DevinR528/RumaTui! Help is welcome, although as matrix-rust-sdk develops the design and structure of rumatui will evolve quickly.

#rumatui:matrix.org is the room to join.

🔗Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) reported:

Nheko is preparing for the 0.7.0 release! You can find the first release candidate here: https://github.com/Nheko-Reborn/nheko/releases/tag/v0.7.0-rc1

0.7.0 will contain a lot of work, including replies, e2ee attachments and a completely rewritten timeline (the full changelog is too big for TWIM, but you can find it in the github releases). It also fixes a lot of issues. We should probably released it a lot earlier, but we always had one more feature we wanted to finish... For the future we plan to do smaller, more regular releases!

If you want to see Nheko translated to your language, you can translate it here: https://weblate.nheko.im/engage/nheko/

If you are a packager, it would be awesome if you can check, that you don't have any build issues. If you need support, just message us in #nheko-reborn:matrix.org !

🔗Riot Web

Ryan reported:

Cross-signing and E2EE by default for private rooms is nearly here! 🎉 It would be great to have more people testing this, as we're very close to release, and extra testing will help reveal those last issues. Please help test cross-signing on https://riot.im/develop (suggesting develop since there's already some fixes there beyond the RC on staging) and the latest RiotX build. It would also be great to test E2EE search in Riot Desktop Nightly.

Matthew added:

e2ee by default and cross signing and encrypted search is almost here - please test the release candidates! (urls at https://twitter.com/RiotChat/status/1250881462066446336)

🔗RiotX

benoit announced:

We are finalizing cross-signing, QuadS and bootstrap. We will release a new version next week. We are still working on migration of the database from Realm to sqldelight solution.

🔗Riot-iOS

Manu reported:

Let me copy Riot-Web and RiotX-Android: Cross-signing and E2EE by default for private rooms is nearly here! 🎉.

A TestFlight will be available over the weekend. Here is the link if you want to join the program. This release candidate still needs some polish but a new release will be definitely available on the AppStore next week.

In parallel of that, the maintenance work is going well. We are erasing our technical debt as much as possible. The new Apple Push API raises some privacy concerns. We may no longer be able to offer CallKit as it requires events to go in clear between servers.

🔗Dept of SDKs and Frameworks 🧰

🔗Ruma

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

jplatte said:

Many of you will probably have heard already, but Ruma is now officially dead. The homeserver, that is. Outside of that, the API definitions in ruma-client-api have seen some work and our first two endpoints in ruma-federation-api have been written! ruma-api and ruma-identifiers are also being worked on, with the following new releases being published in the past week:

• ruma-api 0.15.1 was released with a small bugfix

• ruma-identifiers 0.15.0 was released with loads of changes; this might be the last 0.x release!

• identifiers are now stored as strings (with minimal additional data)

• before, we would parse the server name into a url::Host

• this means that parsing now doesn't allocate at all under some circumstances, and obtaining the string representation of one of our identifier types is essentially a no-op

• there no longer is a single non-optional dependency, everything that requires them is now behind a feature flag

• some new helper functions have been made available, mostly around user IDs

🔗Dept of Bots 🤖

🔗CovBot v0.1.4

Peter Roberts reported:

• When there are a lot of matches ask the user to be more specific rather than sending a huge list.

• Let user know if we can't make sense of the age given to !risk.

• Fix formatting of !compare tables on old Riot for android.

• Refactor into separate files for bot and data sourcing.

GitHub

🔗Dept of Interesting Projects 🛰

🔗Doctor Who confinement watchalongs

Brendan offered:

We've been doing Matrix editions of Emily Cook's Doctor Who watchalongs since confinement started around Europe. The concept is pretty simple: on a given day, at a given time, everyone presses play on the same episode. Folks are then free to then watch it on their own, or discuss the episode as they watch it in the Matrix room and/or in the Jitsi call. We're also bridging tweets sent from directors, writers, comedians, etc. watching the episode at the same time to Matrix.

If folks are interested in joining in, head over to #doctor-who-watchalong:abolivier.bzh!

The next watchalong is planned for next Sunday (Apr 19th) at 7PM BST; and we'll be watching the epic two-episode long season finale "The Stolen Earth" & "Journey's End", with Russell T Davies (writer), Graeme Harper (director), David Tennant (The Doctor) and Catherine Tate (Donna Noble) tweeting along (and possibly more to come) 😀

🔗Tweetalong bot

Brendan reported:

To relay the tweets from guests during Doctor Who watchalongs, I've written a simple bot that reads the timeline of a Twitter list, optionally filters it by a hashtag, and posts the tweet to a Matrix room.

The code is available at https://github.com/babolivier/matrix-tweetalong-bot, and there's already an instance running in #doctor-who-watchalong:abolivier.bzh if folks want to see it live.

🔗Dept of Guides 🧭

Lots of new guides this week! I particularly encourage you to read neilalexander's guide to state resolution - I found it very approachable.

Matthew said:

neilalexander wrote a kickass guide to how state resolution works and should be implemented: https://matrix.org/docs/guides/implementing-stateres

and then

Kaushik Chakraborty wrote an awesome NixOS tutorial for setting up Synapse+Riot+Jitsi as a response to my Debian one! https://kaushikc.org/posts/matrix-jitsi-nixos.html

suddenly

...and Pablo Fredrikson recorded a fantastic video run-through in Spanish of setting up Synapse on Docker! https://www.youtube.com/watch?v=ZiVQalt5xhk

🔗Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	envs.net	394
2	maunium.net	437
3	fairydust.space	472
4	maescool.be	493
5	dodsorf.as	495
6	nerdsin.space	540
7	c.mau.dev	569
8	matrix.vgorcum.com	571
9	digi.co.hu	623.5
10	elcyb.org	716

🔗Final Thoughts 💭

While discussing the use of QML in Nheko, aa13q became extremely energised, so I'm including this piece:

Qt/QML fanboy here, feel free to skip, just a love letter to the framework :D

Qt/QML is super-awesome, it's hard to describe how pretty declarative syntax is and how beautiful MVC approach with native Qt C++ side integration %) It is very hard to contribute to HTML/CSS/JS projects when you have seen QML once.

Talking about resources, there are even projects for very low-level embedded boards with arm cortex m7 core [1,2]

I'm actually thinking Qt is the bright future of very wide development areas. I'm both frightened and intrigued looking at relatively low popularity of the project. Since I want it to live and prosper even more but afraid of the possible mistakes for even harder tasks. The better popularity of the project will be harder to coordinate and the future tasks will be even harder. Similar to other cool projects (like matrix! :) I guess it's very challenging to balance between running as hard as you can and not to "die" from exhaustion %)

I don't even know is there something similar comparing to the Qt project, that's also why I'm worried about the future of the project %)

So-called software crisis (the late 60s – mid-80s [3,4]) was successfully overcome by introducing Object-Oriented Programming (it's always funny to see some people who are more into functional programming hates OOP in that context [5]) And notably Alan Kay is not only OOP pioneer (Smalltalk language) but also a first GUI applications developer. While the current OOP languages like Java and C++ are quite imperative, OOP is not only about the objects but also about the messages between them (Smalltalk is probably a nice example here too).

Qt in that context restores interesting balance to quite imperative C++ introducing the so-called meta-object system (for the messaging between objects). After that in ~Qt5, the declarative QML approach was introduced to develop these ideas even more. Not only GUI subject matter could be expressed with QML (like Qt Quick), but the others too (for example, QBS build system).

Sorry for typos and poor style, not a native speaker :) And I could be wrong in my views.

[1]: https://www.youtube.com/watch?v=p9_Qy3kw1wc
[2]: https://github.com/siili-auto/respirator-qt-for-mcu
[3]: Cox, Brad J. There Is a Silver Bullet // BYTE Magazine, October 1990.
[4]: Dahl, Ole-Johan. The Birth of Object Orientation: the Simula Languages / Owe, Olaf; Krogdahl, Stein; Lyche, Tom (eds.). From Object-Orientation to Formal Methods. Essays in Memory of Ole-Johan Dahl. New York: Springer, 2004.
[5]: https://i.imgur.com/49Vajzh.png

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!