Matrix Live 🎙
Let's hear from Christian about working on his dream projects, his thoughts on bridging, hosting, and the importance and nature of chatbots and assistants.
Dept of Status of Matrix 🌡️
This week was the announcement of the much-awaited rebrand: Riot is now Element. In fact, three brands are coalescing into one: Riot and New Vector will be referred to as Element, while the SaaS platform known as Modular.im is now Element Matrix Services.
Note that Matrix is not involved in this change. Matrix is still Matrix, don't worry about that!
CommCon Virtual 2020, with added Matrix
CommCon is an event dedicated to Real-Time Communications. In 2020 they made the difficult decision to go online-only, but had a fear of missing out on the "hallway-track" that is so important to industry events. Their solution was to include a live chat to run alongside their streamed talks!
Matrix was a natural choice for the crowd, but they wanted a way to encourage viewers to join the correct room from their own Matrix clients. To do this, they used the GSOC project from arnav-t - an HTML Embedded client - to present a live scrolling-view of the chat. You can see the result at https://2020.commcon.xyz/live/.
Dept of Spec 📜
anoa is away this week, so no pretty graph, I'm afraid. (this is intolerable - BP)
- No MSCs were merged this week.
MSCs in Final Comment Period:
- No MSCs in Final Comment Period this week.
Spec Core Team
This week, the Spec Core Team will be focusing on MSC2610 (Removing
m.login.oauth2from User-Interactive Authentication), MSC2611 (Removing
m.login.tokenfrom User-Interactive Authentication), MSC2663 (Errors for dealing with non-existent push rules), MSC2674 (Event Relationships), and MSC2689 (Fix E2EE for guests).
Dept of GSoC 🎓️
Ruma/Matrix GSoC update
Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.
The End is Nigh
This week in the ruma/matrix Google Summer of Code project, I worked on refactoring both
ruma-events. After moving some of the larger chunks of the
ruma_api_macro::api::Api::to_tokensmethod to helper functions, I spent time removing repetition from the Request/Response code generated by the
ruma-events, the input parsing was changed to only allow valid names for the
Any*Eventenums. Altering the input parsing had the added benefit of replacing all of the string comparison and manipulation with strongly typed comparison and manipulation.
The final few issues to be resolved before the next crates.io release for
ruma-eventscan happen are related to redacted events. Support for redacted events was added to the
Any*Eventenums, they now have redacted variants of each event kind. A few follow-up PR's have been merged to fully integrate redacted events into
ruma-events, fixing specific event deserialization issues and splitting the UnsignedData struct into
HTML Embeddable Matrix
This week for the GSoC project of HTML Embeddable Matrix Client:
Added a read receipts menu for messages.
Minor bug fixes
Automatic hyperlinking of URLs in pipeline
See above for details of this project being used in the wild!
Added the capabilities to request room keys from other devices, as well as share room keys with other devices that requested them
Currently working on device verification using SAS, will then work on key export / import
- Updated to latest mautrix version with some minor fixes, most notably using m.encryption event parameters
End to End Encryption with matrix
The new connector has been fully implemented!
Device verification has been put on hold for the time being
No extra configuration is required
Check out all the changes here
Work has begun on porting the matrix database module to matrix-nio and integrating it into opsdroid core
It uses matrix rooms as databases
Users can store key value pairs as state events in a room
Also works with encrypted rooms to allow for encrypted storage
Merge in futures 0.3 support for matrix and http modules
Opened pull request for updating tests, irc module, and bridge module to futures 0.3 & remove several futures dependencies
updating several packages TLS packages to their more modern counterparts
Dept of Servers 🏢
matrix-media-repo is a highly customizable multi-domain media repository for Matrix
matrix-media-repo has just released v1.1.3 with several improvements and bug fixes - please give it a go, and get support in #media-repo:t2bot.io
Conduit is a Matrix homeserver written in Rust https://conduit.rs
timo told us:
@the0 finished /event
I rebased Jamie's WIP appservice branch
I wrote some documentation for our database: https://git.koesters.xyz/timo/conduit/wiki/Database
I also investigated why so many sytests fail and created this issue: https://github.com/matrix-org/sytest/issues/913 Apparently Sytest did not expect a server to optimize their /sync responses as much as Conduit does :P.
A big week for matrix.org performance.
Hot on the heels of shipping shardable federation readers last week, this week we shipped shardable federation senders. There is still plenty of work to do, but together these changes has made a massive difference to federation lag overall and hopefully those of you not on matrix.org are noticing the difference when you talk to matrix.org users.
This graph shows the impact to outbound federation lag.
Aside from that we also shipped shardable push and frontend proxy workers as both were starting to max out on CPU as well as a shardable client reader, allowing us to shard registration which was especially important this week :)
Finally we moved typing notifications from the master process and optimised incoming replication queuing to buy us a little more head room.
Next steps are to revisit where all the remaining cycles are going on the master process. To help us profile we are migrating to async/await semantics and Patrick produced this natty graph to track progress.
Outside of performance we shipped a bug fix to prevent large initial syncs taking out the synchrotrons. The admin api sprouted an end point to list room members (thanks awesome-michaeland the ability to reactivate previously deactivated users.
Coming up we’ll dust off the notifications project which has been put on hold while Riot transmogrified into Element and we’ll continue with chipping away at the master process.
Synapse Deployment 📥️
And another bump of the K8s-optimized Synapse images, this time to 1.17.0
Synapse adoption graphs
We only featured this a month ago, but I want to call attention again to this awesome Synapse version adoption tracking project from Chris . I wanted to know how quickly Synapse 1.17.0 would be the most deployed version (~36 hours), and the answer is right there!
Dept of Bridges 🌉
I started working on a Twitter DM bridge that uses the internal API instead of the complicated official one. It should be easier to set up than bridges using the official API, since there won't be a need to get API keys and forward webhooks. The main risk is that Twitter doesn't like people using the internal API and starts blocking users like Facebook does, but hopefully that doesn't happen.
So far I've only made a simple library for the API, but the bridge should be usable by the next TWIM. The bridge and API library are at https://github.com/tulir/mautrix-twitter and there's a discussion room at #twitter:maunium.net.
Dept of Clients 📱
Element Android 1.0.0
Element Android 1.0.0 is out! Download or upgrade it from the PlayStore at https://play.google.com/store/apps/details?id=im.vector.app or download it from the F-Droid store at https://f-droid.org/packages/im.vector.app/
PlayStore users: Element should come as a regular upgrade of the application Riot-Android, and the upgrade will update your previous account and data, without the need to log in again.
F-Droid users: Element is a new app, you have to install it and log in again.
RiotX: RiotX will disappear from the PlayStore (it was only a beta application). We will provide a very last update to inform users to install Element Android
Please report any issues at https://github.com/vector-im/riotX-android (which will probably be renamed soon), because Element Android is actually RiotX code! Thanks for all the contributors of RiotX, we still have lots to do to make the app even better and full featured.
After seeing all the Element room avatars, I thought to myself, "you know what, #radical-webext:matrix.org needs a new icon"… so I sat down and combined a couple of icons. The result:
Element F-Droid repos
I updated my F-Droid repos which are containing the dev builds to reflect the latest naming change to Element.
The people who formerly used my repos for getting the dev repo of Riot-Android should update to the new repo. It wont be updated to get Element
As always you can grab the F-Droid or GPlay flavor and you can pick the repo which reflects your flavor on https://fdroid.krombel.de
Note: As "the new repo" is internally the same repo as the one of RiotX (and Riot.imX) the people who already used that don't need to update their packet source. They will get the updates to Element via the old repo urls.
Feel free to use it 🙂
Dept of SDKs and Frameworks 🧰
As promised last week, maubot now supports end-to-end encryption. It's a bit bare at the moment, so you need to manually insert the device ID into the database for it to start working, but other than that it works. Plugins don't need to be changed at all, the framework will handle everything.
While I was adding e2ee to maubot, I also improved mautrix-python's crypto stuff so that it's easier to use it directly as a client library with e2ee. Some day I might even add docs :D
Chris Vincent reported:
Related to my previous update here, I've just published an initial version of a Matrix Client-Server client library for the Crystal programming language: https://github.com/cvincent/matrix-client-cr
Is this the first time we've seen tooling for Crystal?
Dept of Ops 🛠
This Ansible playbook is meant to easily let you run your own Matrix homeserver.
We have a few ways to handle the migration depending on how much change and breakage you're willing to tolerate. See our Migrating to Element documentation page.
Incredibly thorough migration paths here!
Dept of Bots 🤖
Matrix-Alertmanager bot v0.3.0
Matrix-Alertmanager bot has a new release of v0.3.0. Highlights are compatibility with AWS Lambda, better error handling if message fails to send and the ability to do a
@roommention on firing alerts. Find it here.
REPL for your matrix rooms
!js. The code
complexity was considerably reduced, which should make adding new languages much easier in the future. All Python sub-dependencies are now pinned, making
cody builds even more reproducible.
Chat with cody: @cody:bordum.dk
Read the source: https://gitlab.com/carlbordum/matrix-cody
Dept of Interesting Projects 🛰️
Noteworthy (Matrix over Wireguard overlay networks)
Noteworthy team (patrick and myself) have open sourced both components (spoke & hub) of Noteworthy (Matrix over Wireguard overlay networks -- https://github.com/decentralabs/noteworthy) we are actively welcoming testers/contributors and working with a handful of projects on incorporating our deployment model over in #noteworthy:tincan.community -- it is also the fastest way (less than a minute) to deploy your own home server! Also, working on incorporating Noteworthy into the popular ansible deployment repo to enable users who don't have access to a publicly accessible to deploy their own home servers.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!