This Week in Matrix 2020-06-19

Matrix Live 🎙

Dept of Status of Matrix 🌡

Matrix Merch Store is back

Are you emerging from lockdown? Dazed and wondering what to wear? Need some apparel which says "I'm cool and I have great taste in decentralised communications protocols"? Then you should visit The Matrix Merch Store!

T-Shirts, Hoodies, Zipped Hoodies and Stickers available now!

Matrix Decomposition: Analysis of an Access Control Approach on Transaction-based DAGs without Finality

Florian told us:

We published another scientific paper on Matrix, this time it is called Matrix Decomposition: Analysis of an Access Control Approach on Transaction-based DAGs without Finality! Its main topic is the question of how access control can work based on a structure that only provides a partial order on events and no consensus on the current state, i.e. the Matrix Event Graph. We found some concrete, non-critical security issues related to both incorrect specification as well as divergent homeserver behaviour. While the last remaining mitigations found their way into Synapse 1.14.0, we provide ideas on structural solutions to avoid both problem classes in the future using formal verification.

The paper was published at the ACM Symposium on Access Control Models and Technologies(SACMAT) last week. My talk went well (🎉), it was recording and is yet to be published by the conference.

The paper was previously mentioned on the Matrix blog.

Dept of Spec 📜

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

• MSC2399: Reporting that decryption keys are withheld
• MSC1703: encrypting recovery keys for online megolm backups

MSCs in Final Comment Period:

• No MSCs are in FCP.

New MSCs:

• MSC2644: matrix.to URI syntax v2
• MSC2638: Ability for clients to request homeservers to resync device lists

Spec Core Team

In terms of Spec Core Team MSC focus for this week, the team has been spread a bit thin with other work. As such, we're still on the same three MSCs: MSC2366 (verification flows), MSC2403 (knocking), and MSC2630 (SAS security). We're hoping this week will be more productive in terms of spec.

Dept of Servers 🏢

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander announced:

This week has seen a lot of work put into fixing broken tests and refactoring parts of the codebase.

• The media API has had quite a bit of attention this week and now media over federation works properly, as well as much better filename handling

• A new user API component is now responsible for user accounts, devices, access tokens and account data

• The server key API has been updated with more reliable validity checking and new tests

• Logging into Dendrite using Riot iOS is now supported thanks to some minor tweaks/fixes

• URL handling for version 3 rooms has been improved

• A bug in the format of /v1/send_join has been fixed

• The /joined_members response has now been fixed

In addition to that, we've made good progress on embedding the Dendrite-powered Yggdrasil demo into Riot iOS, which we'll hopefully be demoing in a couple of weeks!

Spec compliance has improved:

• Client-Server APIs: 40%, up from 36% last week

• Server-Server APIs: 38%, up from 35% last week

Synapse

Neil offered:

Highlights this week

• We released 1.15.1, this is a bug fix release, sorry if you were bitten.

Changelog in full

Bugfixes

• Fix a bug introduced in v1.15.0 that would crash Synapse on start when using certain password auth providers.

• Fix a bug introduced in v1.15.0 which meant that some 3PID management endpoints were not accessible on the correct URL.

Outside of that

• Implemented unread message counts (MSC 2625) - This is part of the a more general notifications improvement project in conjunction with the Riot teams. The aim being to make Notifications easier to configure. We’ve also been working on push rule definitions for the default behaviour. More on that next week.

• Have the ability to shard the media repository worker. This means we can now run an arbitrary number in parallel, thereby improving upload reliability and performance, we are running this on Matrix.org.

• Re-enabled Jaeger on matrix.org - previously we needed to switch it off due to the performance overhead, we needed a few tweaks but now it is working well and helping us to determine the cause of slow requests. It seems like the HTTP requests between the event creator and event persister workers can sometimes be slow - we are not sure of the cause yet, we are working on it but it will mean further performance improvements to message sending.

• Next up performance wise is working on sharding the federation sender to improve federation lag. We’ll also try replacing simplejson with stdlib json which seems to be much faster at dumps nowadays.

• Finally, we’ve fixed a bug that meant that quarantining media would from time to time include quarantining sticker packs

Other interesting bugs

We fixed a spec compliance bug in fetching remote media raised by our pals in Dendrite-land. We fixed a bug causing federation .well-known requests to fail due to not including the user-agent header. We removed references to six (thanks @ilmari) and finally, fixed a bug that caused stream id to go backwards on the replication stream.

Next week we’ll continue to work on notifications and performance. You can track our progress here https://github.com/orgs/matrix-org/projects/8

Synapse adoption graphs

Chris said:

Wondering how quick this weeks synapse release was adopted by homeserver admins? Head over to https://graph.settgast.org/ and see yourself that it only took a little over 2 days for 1.15.1 to become the most used synapse version. I plot hourly distribution of synapse versions there. (Disclaimer: Only collects stats for homeservers that my homeserver sees in any of its rooms, which are around 2000 at the moment)

This is super cool! I love that this has been created, and also how well it reflects on homeserver admins! /me scurries to update to 1.15.1...

Asked to share the process, Chris sent:

Its a horribly hacky python script, but I can at least share it here: https://gist.github.com/csett86/96190592390b2c3d8c12c020ce312723

What version are you running?

Synapse on a ROCKPro64

How is Synapse performance these days? PINE64 are running their instance on their own ROCKPro64 hardware.

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo announced:

This week I worked on server-side key backups and cross-signing - the second most requested features (federation being the first one). Key backups are already working! Take a look:

Cross-signing should be working, but there is a bug where emoji verification gets stuck. I hope we can find the mistakes and finish the PR next week.

Thanks to everyone who supports me on Liberapay or (new!) Bitcoin!

Synapse Deployment 📥

Kubernetes

Ananace said:

Just pushed 1.15.1 tags for the K8s-optimized Synapse image.

Docker-matrix

Mathijs announced:

The image for synapse v1.15.1 including jemalloc and mjolnir anti-spam has been released to avhost/docker-matrix:v1.15.1.

Dept of Bridges 🌉

matrix-ircd, GSOC project

karlik offered:

For matrix-ircd google summer of code:

• Began moving the codebase to the more modern futures 0.3 (previously futures 0.1)

• Introduced more modern async / await syntax for improved code clarity

• Wrote some additional unit tests, doubling the overall test coverage to 32%

• Updating the 2016 code to use rust 2018 idioms PR # 64

• Swapped repository to use stable rust (previously nightly) PR #65

mautrix bridges

Tulir told us:

To make the bridge info state events a bit more useful, all my bridges now include the room name and avatar URL in them. The events are also updated whenever the name or avatar changes, and there's a new option to re-send the event to all existing portals so old portals would get it too.

Dept of Clients 📱

RiotX: Version 0.22.0

benoit said:

RiotX: Version 0.22.0 has been released on Tuesday, it includes integration manager support, sending stickers, modifying power levels, and lots of other things! See https://github.com/vector-im/riotX-android/releases/tag/v0.22.0 for a complete list of the changes.

This week we've been working on audio and video calls (The PR is in review). We are also trying to improve the performance of the application. We are adding room settings, and we have started to work on the migration Riot-Android -> RiotX.

PS check out Valere trying the new features:

Arch Linux (AUR) package for Revolution - a Riot fork by ponies.im

Sophie still alive said:

I made an Arch Linux (AUR) package for Revolution - a Riot fork by ponies.im:

• https://aur.archlinux.org/packages/revolution-desktop-git/

• https://aur.archlinux.org/packages/revolution-web-git/

• https://gitlab.eastcoast.hosting/ponies

Daydream

MTRNord told us:

Daydream is getting a new Design and the first page is ready at https://app.daydream.im/

Additional changes are:

• Removed client side js (except what's needed to load wasm stuff)

• Replaced loading animation with css only version

• Replace icons with css only version

• Add e2e tests

• Add Basic Notifications

Currently on top of the TODO list of Daydream is:

• Adding a wasm compatible (indexed db based) state store

• Notifications

Rumatui is basically usable

WIP Command line Matrix client using matrix-rust-sdk

devinr528 told us:

After some slow progress, rumatui, the rust command line chat client, is usable by anyone. You no longer need an account or have previously joined rooms. Registration and room search have been implemented! Using left/right arrow keys from the login screen will toggle to registration. Once registered hitting the left arrow will bring up the room search window. After typing your search term hit Enter and select the room to join by pressing Ctrl-d.

For the brave who would like to give it a try cargo install --git <https://github.com/DevinR528/rumatui> --branch room-search.

Added since last update

• Room search is now available

  • Public rooms can be joined from the room search window

• A user can register from the new register window

  • This features complete User Interactive Authentication by opening a web browser

• Message edits are shown

  • When markdown is part of the message they are properly formatted

• Reactions display under the respective message

• Redaction events are handled for reactions (emoji) and messages

"Basically usable" is a standard we can all aspire to for our projects and endeavours.

Riot-iOS

Manu told us:

This week, we worked on the new UX for cross-signing and secure backup. In parallel, we continued to improve the new push implementation. We are very closed to complete the two features.

gomuks

gomuks is a terminal based Matrix client written in Go. Source on GitHub

Tulir announced:

I added a command to add and remove local aliases for rooms. Changes to alt_aliases are also now rendered properly.

Riot Web

Ryan offered:

This week we had an off-cycle 1.6.5 release to fix registration on some homeservers when email validation is required. After that, we've made a 1.6.6 RC including:

• Added support for new emoji in Unicode 13

• Emoji picker now supports searching by emoji characters (to find related ones) as well as text

• Added a homeserver admin .well-known option to control default E2EE behaviour

• Many more bug fixes and smaller improvements

Riot Web Themes

jo offered:

made a couple riot themes! Dracula and Dracula Flat

https://raw.githubusercontent.com/aaronraimist/riot-web-themes/master/Dracula/Non-flat/Dracula.json https://raw.githubusercontent.com/aaronraimist/riot-web-themes/master/Dracula/Flat/DraculaFlat.json

Find images here and here

Go to https://github.com/aaronraimist/riot-web-themes to find out how to use these.

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) offered:

• We released 0.7.2, which was a bit more messy than we would have liked...

• As with every release we are currently working through the new bug reports. :D

• I started working on SSSS and online key backup support. For this I wrote my own base64 and base58 encoders. As a result of this we dropped libsodium as a dependency, which should make Nheko a bit easier to package or build. It's also fun to write base conversions, although 58 is a horrible base!

From the changelog:

• Reactions
  • React to a message with an emoji! 🎉
  • Reactions are shown below a message in a small bubble with a counter.
  • By clicking on that, others can add to the reaction count.
  • It may help you celebrating a new Nheko Release or react with a 👎 to a failed build to express your frustration.
  • This uses a new emoji picker. The picker will be improved in the near future (better scrolling, sections, favorites, recently used or similar) and then probably replace the current picker.
• Support for tagging rooms [tag]
  • Assign custom tags to rooms from the context menu in the room list.
  • This allows filtering rooms via the group list. This puts you in a focus mode showing only the selected tags.
  • You can assign multiple tags to group rooms however you like.
• SSO Login
  • With this you can now login on servers, that only provide SSO.
  • Just enter any mxid on the server. Nheko will figure out that you need to use SSO and redirect your browser to the login page.
  • Complete the login in your browser and Nheko should automatically log you in.
• Presence
  • Shows online status of the people you are talking to.
  • You can define a custom status message to tell others what you are currently up to.
  • The status message appears next to the usernames in the timeline.
  • Your server needs to have presence enabled for this to work.

Wowow that's a lot this time!

Dept of SDKs and Frameworks 🧰

quotient

kitsune said:

after a couple of setbacks that didn't let us to make the releases earlier, the Quotient project has finally made two new releases:

• GTAD (the piece of code magically producing readable C++ code from a Swagger API description) has achieved version 0.7 (https://github.com/KitsuneRal/gtad/releases/tag/v0.7.0) adding a few tricks in order to make...

• ...libQuotient 0.6 beta2 (https://github.com/quotient-im/libQuotient/releases/tag/0.6-beta2) rely entirely on the upstream matrix-doc specification, rather than a soft fork closely following the main sources! From now on it's "upstream first", in a sense that the original matrix-doc will be used to build Quotient codebase. Let's see how often it breaks ;) In other news from the last few weeks:

• some housekeeping and deprecation work in the API has been done in preparation to getting User Interactive Authentication along the next (post-0.6) release cycle of libQuotient.

• also thanks to the updated code generator, the CS API code has been optimised, consolidating more code in the header files and making data deserialisation lazy; this helped significantly reduce compilation times, and runtime performance also improved a bit.

• the number of configurations tested by CI has been extended, allowing to chase down a few more bugs that managed to go under the radar before.

• when used with new enough Qt, CBOR is used to cache data locally - entirely transparently for clients.

Expect more news in the coming weeks, including continued work on matrix: URI proposal and its implementation in Quotient.

ruma-events' Google Summer of Code project

devinr528 sent us this late entry!

This week in ruma-events' Google Summer of Code project, after trying out the new Any*Event enums matrix-rust-sdk, we found a few big flaws. There was no easy way to go from StateEvent<AnyStateEventContent> to StateEvent<SpecificEventContent>, the other issue was one could create a StateEvent with differing content and prev_content fields using the AnyStateEventContent enum. The 0.22 ruma-events will be similar to the existing API; each event type has a corresponding event enum variant.

pub enum AnyStateEvent {
    RoomMember(StateEvent<MemberEventContent>),
    RoomAliases(StateEvent<RoomAliases>),
    // ...
}

There were a few minor fixes during the week also. Unknown field deserialization is fixed, allowing deserialization of a JSON blob that has extra fields which are ignored. Custom events are now present in the Any*EventContent enums, although now they have to be moved up to be included in Any*Event enums. Benchmarks for deserialization have been added and used to increase performance.

🧙 Polyjuice Client

Polyjuice Client is a Matrix library for Elixir

uhoreg announced:

Polyjuice Client, a Matrix library for Elixir, had a new release. This release fixes syncs, which apparently were completely broken in the last release (whoops). The wizarding community also welcomes a new contributor, Pierre de Lacroix.

Dept of Ops 🛠

Gitea

s7evink reported:

Gitea 1.12.0 was finally released. It's now possible to send notifications (commits, pull requests, etc.) directly to Matrix using webhooks.

Dept of Internet of Things 💡

Mozilla WebThings Matrix Adapter

Christian offered:

I heard that you people like IoT?! With the Mozilla WebThings Matrix Adapter your Raspberry Pi sends you messages about your home. Want a log of when the front door has been opened? Need a low battery alert for your IoT devices? With the Mozilla IoT gateway and this adapter, you can send these events to a Matrix room of your choice!

https://gitlab.com/webthings/matrix-adapter

Dept of Bots 🤖

Alt alias maubot plugin

Tulir announced:

Ever since m.room.aliases was replaced with alt_aliases, there hasn't really been any way to find aliases on other servers, since most room admins don't bother finding and publishing alternate aliases. To help solve that problem, I made a maubot plugin to let users publish alternate aliases in rooms: https://github.com/maubot/altalias

By default it lets users publish aliases with the same localpart as any already published alias. If that behavior is sufficient, you can simply invite it to a room and give it permissions to send m.room.canonical_alias events. Alternatively, it can be configured to allow aliases that match specific regexes.

Once it's set up, users can create a local alias and use !altalias publish <their alias> to publish it. The bot will make sure the alias points to the right room, check that the localpart is allowed and then add it to alt_aliases.

I have it running at @alias:maunium.net and I've added it to all my rooms already. Other rooms are also free to use that instance.

go-neb on NixOS

maralorn offered:

As of https://github.com/NixOS/nixpkgs/pull/89327 a go-neb module was added to nixos and will soon be available on nixos-unstable and in the 20.09 release. You can use it to declaratively install and configure go-neb matrix bots on nixos. I use it for prometheus alertmanager alerts and it works really cool! Thx to hexa- who did most of the work on it.

Opsdroid

Cadair offered:

opsdroid 0.19 has been released, not many matrix specific features in this release, just making it slightly easier to get a connector or database instance by name. However, this release paves the way for the merge of the pull request transitioning opsdroid to use the matrix-nio Python library, so the next release should be packed full of matrix stuff 😀

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	fairydust.space	384.5
2	maescool.be	390
3	gottliebtfreitag.de	453
4	construct.grin.hu	610
5	aragon.sh	639.5
6	privacytools.io	670
7	matrix.vgorcum.com	734
8	envs.net	756
9	settgast.org	865
10	intothecyber.space	1005

That's all I know 🏁

Thanks to Alexander and Nico for their help editing this edition.

See you next week, and be sure to stop by #twim:matrix.org with your updates!