Matrix Live 🎙

In which I chat with Kitsune about the work done to get a Matrix URI schema agreed, and the state of the work.

See also, Open Tech Will Save Us #7 took place this week! Go watch.

Dept of Status of Matrix 🌡️

Meta-counting

As a crude measure of growth, Matthew commented about #twim:matrix.org:

I love that this room has ~700 people in it, spread over ~350 servers :D

That is something to love. Come join us in the room to share your news and see what's new from others.

Elokapina (Extinction Rebellion Finland) migrates to Matrix

kapina-jaywink told us:

In recent weeks the XR Finland community has been moving over from Wire to our own Matrix homeserver for encrypted secure chat. This was something that had been planned for a while but kicked off in recent weeks due to Wire suffering from serious encryption key delivery issues, causing messages for many to be unreadable in large groups. Currently we've migrated almost 300 rebels with more to come. Feedback has mostly been very positive, people generally like the Element clients 🤩 One of the interesting changes has been a huge uptick in the amount of discussion, which can be taken as a good sign. The plan is to next start bridging to some of the international XR chapters, for example those on Mattermost, Telegram and Slack. And maybe get them over to Matrix too eventually ;)

To aid in community management, we've started creating a bot called Bubo. Right now it mostly helps with maintaining rooms and allowing mass invites, but more features to help the community cooperate are coming. We were planning to utilize (actual) communities so it has some functionality for those, but decided then to wait for the communities rewrite. It doesn't yet have any releases, will update in coming weeks as features are added and releases made.

kapina-jaywink also

wants to clarify that XR is a decentralized movement and this does not mean other chapters will adapt Matrix - but we can hope and for sure here in Finland we'll be spreading the good experiences to other chapters ;)

Dept of Spec 📜

New Spec Website!

wbamberg told us:

We're working on a new platform for the specification docs, aimed initially at improving navigation and general usability.

The initial demo site is at https://adoring-einstein-5ea514.netlify.app/spec/ and the main tracking bug is at https://github.com/matrix-org/matrix-doc/issues/2822

Spec

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

• No MSCs were merged this week.

MSCs in Final Comment Period:

• No MSCs are in FCP.

New MSCs:

• MSC2821: Testing Pushers

• MSC2815: Proposal to allow room moderators to view redacted event content

• MSC2813: Widget API error handling and validation

Spec Core Team

In terms of Spec Core Team MSC focus for this week, widgets are still the main focus: MSC2774 (widget URL template param), MSC2765 (widget avatars), and MSC2790 (modal widgets).

Dept of Servers 🏢

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander reported:

We released our first beta last week and we've been busy taking feedback from the community and fixing problems as they have been reported to us. We will be cutting a v0.2.0 release candidate later today, which contains a significant number of important fixes and performance improvements.

We didn't publish a full changelog last week due to the beta announcement, therefore the following list includes changes from the last two weeks:

• Database migrations are now ran automatically at startup

• State resolution v2 performance has been improved dramatically on large state sets, seen to be running up to 20x faster in some rooms

• Dendrite no longer runs state resolution over single trusted state snapshots

• Monolith deployments with Kafka now work again (each component sets up connections independently to avoid duplicate consumers)

• Dendrite now correctly rejects invalid UTF-8 - thanks to Lesterpig

• Fully read markers are now handled - thanks to Lesterpig

• The completed field is now returned properly for user-interactive auth - thanks to Lesterpig

• The devices table now tracks last seen timestamps, IPs and user agents - thanks to S7evinK

• A bug has been fixed in the reverse topological ordering algorithm which resulted in us giving up on inbound references after the first prev/auth event

• A bug with concurrent map writes in the rate limiting in the client API has been fixed

• Forward extremities and their previous events are now checked fully against the database

• Typing events are now ignored if the sender domain doesn't match the origin

• Duplicate redaction entries no longer result in database errors

• Some bugs in /send have been fixed where the full room state wasn't requested properly before sending a new snapshot to the roomserver

• The membership updaters now use database writers properly, which fixes some SQLite locking issues

• The sync API no longer burns CPU processing unnecessary device key change notifications

• QueryStateAfterEvents now resolves state from multiple snapshots properly

• Cumulative room state is now sent to the roomserver when creating rooms locally

• Missing auth events will now be retrieved from multiple servers in the room if necessary

• Federation requests now have variable timeouts, allowing us to wait much longer for a remote server to process certain tasks

• The /send endpoint now returns 500 errors far less often, reducing the frequency that other servers back off from sending to Dendrite

• Backfill no longer uses the request context for persisting events, which was resulting in us failing to store those events sometimes

• Invite stripped state from the sync API now includes the stripped invite itself, so that Element Web can display who sent the invite properly

• The signing key fetch mechanism no longer gives up if it is unable to fetch specific keys

• Handling of invalid display name or avatar URLs in membership events has been improved

Spec compliance has improved a little:

• Client-server APIs: 56%, same as last week

• Server-server APIs: 80%, up from 79% last week

As always, please feel free to join us in #dendrite:matrix.org for general Dendrite chat, and #dendrite-dev:matrix.org if you are interested in contributing!

Good grief that's a big update! For a video discussion of the status and future of Dendrite, check out Open Tech Will Save Us #7.

Matthew added:

my personal dendrite is now in roughly the same set of rooms as my personal synapse. Dendrite is idling at 180MB of RAM, Synapse is idling at 1.8GB of RAM :)

Conduit

Timo:

Hello everyone, this week we merged the federation branch into master. It's not ready to be used properly yet, but we're merging it as it seems stable enough for now. We also improved performance of the federation branch a lot by turning off debug logs.

Other news:

• I opened two issues on element-ios which currently break register and login support on Conduit, making it completely unusable. Hopefully they can be resolved soon (https://github.com/vector-im/element-ios/issues/3736, https://github.com/vector-im/element-ios/issues/3737)
• I'm working on an MSC for threading. It's still WIP, but you can take a look here: https://demo.codimd.org/s/SykbuAUwP

Thanks to everyone who supports me on Liberapay or Bitcoin!

Synapse

Neil offered:

Stonking week for Synapse as we landed sharded event persisters and deployed to matrix.org. This is the last significant component other than the main process to go through the sharding process and a major hurdle in horizontal scalability of Synapse.

Initial results look good with event persistence apdex improving, however we think there are still some significant performance improvements available through configuration and will continue to experiment.

We also moved off background processes from the main process. This is significant because it means that while the main process is not shardable it really doesn’t do anything anymore other than orchestration.

Again the initial impact looks very promising and we will continue to tune. Having moved the background processes away it also makes profiling the main process that much easier.

Aside from all of that we continue to progress room knocking put out a 1.21.2 - a bug fix release though please please ensure you are running at least Synapse >= 1.21.0 since 1.21.0 contains a XSS security fix.

Next week we will carrying on tuning matrix.org and start to look at improving state resolution performance.

Synapse Deployment 📥️

Kubernetes

Ananace told us:

Just pushed the updated image tags and chart version for my K8s-optimized Synapse image for version 1.21.0, as well as a chart update for element-web 1.7.9

then later

And to expand on my previous update, got Synapse 1.21.2 up on the chart and K8s-optimized image.

YunoHost

Pierre said:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.20.1 (1.21.0 available in branch testing)

Element Web integration had been updated to 1.7.8 (1.7.9 available in branch testing)

Dept of Bridges 🌉

Bridges increase their presence

Half-Shot told us:

Small bit of news I wanted to talk about from Bridge Island. My implementation for MSC2409 has been merged which means appservices / bridges can now listen in for incoming presence, typing and read receipt events! This means that the Slack bridge can now reliably send your typing status to Slack, and Bifrost can reliably bridge your everything to XMPP. The MSC is still in flux and could change, but for now this could really improve the native feeling of bridges :)

(Oh and I should mention anyone using matrix-appservice-bridge v2.2.0+ can use this behaviour for free)

mautrix-python implements Half-Shot's new features

Hot on his heels, Tulir announced:

I've been adding support for the MSCs Half-Shot implemented in Synapse to my bridges:

• Enabling end-to-bridge encryption now uses appservice login (MSC2778), which means setting up the shared secret login module is no longer required for e2be.

• mautrix-python has support for receiving ephemeral events via MSC2409 in a branch, which will be merged once Synapse v1.22 is released. After it's merged, /syncing with double puppets will no longer be necessary to bridge ephemeral events.

Both of these will also be implemented in mautrix-go/whatsapp soon.

Now I just need Half-Shot to make synapse send to-device events to appservices, after which bridges won't need any hacky /syncing at all.

Dept of Clients 📱

Hydrogen

Bruno offered:

Several releases this week (0.1.11 to 0.1.15) with lots of changes:

• url-based navigation has landed! All navigation in the app is now done through urls, meaning you can also bookmark any UI state (e.g. grid configurations).

• fixed 2 memory leaks (exposed now because you can unload your session without refreshing the page)

• fixed an issue with libolm running out of memory if you send a message to more than 44 devices (see issue #150).

• some logical additions now we have url navigation: restoring the last url when opening the app with the default route, and a button to close your session and go back to the picker.

• the app now blocks concurrent access to the same session from different tabs (it just closes the session in the non-active tab). This will prevent multiple syncs tripping over each other writing to indexeddb (e.g. ConstraintErrors and friends).

• updates are announced in the app (for now through a confirm dialog, but will use an in app notification once we have it)

• fixes updates not installing on iOS, by having an update prompt. To get this update on iOS though, you'll need to unpin the app, and pinning it again. You'll need to login again after this. All future updates should be installable through the update prompt once you have 0.1.15 though, you won't have to do this again normally.

• uses the hydrogen icon when pinning on iOS

I really recommend hitting https://hydrogen.element.io/ - what great progress!

Element Android

benoit reported:

We are currently preparing the release of the version 1.0.9 of Element Android, which contain searching messages in clear rooms and a lots of other improvements and new features.

The SDK 1.0.9 will also be released, with an updated readme, and a brand new sample app, written by Ganfra. It will help developers to start using the new SDK and can be found here: https://github.com/matrix-org/matrix-android-sdk2-sample. This sample app is able to let the user connects to an existing account on any homeserver with password login, display the room list, display a room timeline and send message to a room. a brand new sample app

YES! This is the best documentation

Element-iOS

Manu offered:

This week, we released 1.0.16 on the App Store (and TestFlight).

Konheko (Sailfish client)

Nico (@deepbluev7:neko.dev) offered:

I published the first preview of my Sailfish client called Konheko. While you can run Android applications on Sailfish, they usually are a subpar experience, since they really don't fit the platforms design and style and also usually don't properly send notifications.

So about a year ago I started working on a Matrix client for SailfishOS, but I never really made much progress. Well, last weekend I did, and so it can now send plain text messages as well as various forms of media messages, I made a basic application icon and I've been using it this week already (for unencrypted rooms).

It is still missing a lot of features, but if you want you can install it from OpenRepos. Sources are available here. Just be aware, that it currently stores all messages in RAM, so every restart will take forever to load your rooms and it may run out of RAM at some point. Storing messages in some database will come at some point. Also, a lot of menus may lead nowhere, since those are just placeholders for me atm.

SchildiChat Web/Desktop

SpiritCroc announced:

Recently, I tweaked Element-web to feature a few changes similar to SchildiChat for Android.

For now, it's probably best seen rather as a proof-of-concept than a finished product, as there are still some layout bugs, and no settings available for the added features (I know some people prefer separate lists for direct and group chats). I consider it usable though.

Particular changes compared to upstream Element-web include:

• A common section for groups and direct chats in the overview

• Message bubbles

• Bigger items in the room overview

• A different dark theme, similar to SchildiChat for Android

I don't know how much I will work on this in the future, but I figured it might be interesting to share either way. Maybe even someone with more web-development skills than me might want to help improving it :)

For further discussion, I have created #schildichat-web:matrix.org .

The current version of SchildiChat-Deskop is available for Desktop here, and I host the web variant here. If you want to build it yourself, check out this repo.

Element Web

Neil offered:

Element 1.7.9 has landed, highlights include

• Many small fixes with edits and replies

• Fixed a race during cross-signing key upload at registration time

• Clarified when you have unsaved changes in profile settings

Aside from that has been pushing on with the widgets project. A picture says a thousand words, so here you go.

As you can see T3chguy was really pleased to have me interrupt him to take this picture. Expect the new design to merge next week.

Finally we will most likely ship a new release next week to fix some Jitsi bugs.

Dept of Ops 🛠

Wake-on-LAN bot

JCG announced:

I wrote a Wake-on-LAN bot to wake up hosts by sending a matrix message. It is configurable with multiple hosts and has a list of users per host who are allowed to wake it up. It's using the matrix-rust-sdk, source is available over at https://git.jcg.re/jcgruenhage/matrix-wol, and if anyone has questions, feel free to join #matrix-wol:jcg.re.

When I asked what this is used for:

I have stuff on my workstation that I need access to most of the time, but keeping it running uses too much power (but I did it anyway so far), this is so that I can suspend it when I leave but can still power it on when I need something from there on the got

Dept of Events and Talks 🗣️

Talking about Bridges and Bots in Matrix (German)

Oleg announced:

I was invited to the German Podcast MacMittwoch (no, it's not only about Macs) to talk about Bridges and Bots in Matrix. It was a very interesting and funny round.

Here is the recording:

• https://macmittwoch.de/podcast/matrix_mit_bridges_und_bots

Dept of Interesting Projects 🛰️

matrix-emoji-upload

mewmew offered:

This is a script I've created for use with MSC2545. It allows easy uploading of emoji packs to Matrix rooms. Feel free to check it out on Gitea, or join the project room #matrix-emoji-upload:blob.cat if you have any questions/comments/issues.

n8n.io support

Matthew announced:

n8n.io (FOSS extendable workflow automation) just added Matrix support! https://n8n.io/integrations/n8n-nodes-base.matrix and https://github.com/n8n-io/n8n/pull/1046

Exciting! Yet the saga that followed only adds to the excitement!

First Oleg noticed a problem:

Could it be that only matrix.org as HS currently is supported?

I'm getting Matrix credentials are not valid! and I see the request to matrix.org on a traffic capture.

jaywink discovered the alarming truth:

the original PR mentions mantrixorg indeed: https://github.com/n8n-io/n8n/pull/1024 .. sounds like PR time for someone :)

Tulir, like a coiled spring, provided a PR:

well now it's a pull request https://github.com/n8n-io/n8n/pull/1065

...

oh nice it got merged already

Faelar noticed:

For those not following on github, n8n released a new version including the fix for homeserver

Oleg, who brings us back to the start said:

Just tested it.

It works! 🥳

Open Source!

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Matrix Live 🎙

Dept of Status of Matrix 🌡️

Matrix in use at IETF and TeamSpeak

Matthew said:

IETF is trialling Matrix as a replacement to Slack and a complement to XMPP: https://mailarchive.ietf.org/arch/msg/tools-discuss/cUd9P35cj-nGsaioZ9HmMLmf1G4/

ALSO

Matthew commented:

looks like teamspeak is using Matrix (a Matrix endpoint was previously noticed at https://tschat-1.teamspeak.com/)

Nico (@deepbluev7:neko.dev) offered:

More info here: https://community.teamspeak.com/t/teamspeak-development-status-update/11419/36

Dept of Spec 📜

anoa told us:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals

MSC Status

Merged MSCs:

• MSC2788: Room version 6 as the default room version
• MSC2414: Make reason and score parameters optional for reporting content

MSCs in Final Comment Period:

• No MSCs are in FCP.

New MSCs:

• [WIP] MSC2812: Role-based power/permissions

• MSC2810: Consistent globs specification

• [WIP] MSC2802: Flip The Table for Full Room Abstraction

Spec Core Team

In terms of Spec Core Team MSC focus for this week, we're going to be keeping the same three MSCs as last week, as their review is still a work in progress. As a reminder, those are MSC2774 (widget URL template parameters), MSC2765 (widget avatars), and MSC2790 (modal widgets).

Dept of Servers 🏢

Dendrite goes beta

Dendrite is a next-generation homeserver written in Go

kegan told us:

Dendrite has officially entered beta!

It aims to be an efficient, reliable and scalable alternative to Synapse.

We encourage early adopters to try it out starting with v0.1.0 in Monolith/Postgres mode.

Dendrite supports a large amount of the Matrix spec including:

• All room versions

• E2E encryption (but not cross-signing)

• Federation

• and many others!

We'll be hard at work in the coming weeks ironing out any issues that crop up as Dendrite enters the global

federation of Matrix servers. At present, Dendrite has not been optimised for memory/CPU usage, nor does it fully support sharding, so there is still much scope to improve the already impressive resource usage whilst

heading towards 100% spec compliance.

Spec compliance:

• Client-Server APIs: 56%, same as last week

• Server-Server APIs: 77%, same as last week

stefan announced:

I created an AUR package for dendrite. So it is easy to get it running on Arch Linux.

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo told us:

This week I worked on a new feature: the Admin Room. This is a room that is automatically created for the first user of a new homeserver. This room will be used to send warnings to server admins in the future and can be used to control Conduit (like querying information, running cleanup, banning servers, hiding public rooms or restarting Conduit). Power levels can be used to control who in the room has access to which commands. The admin room will work over federation, so it's easy to help someone fix his server.

Other news:

• Federation is now disabled by default and can be enabled in the config

• Bug with typing notifications is fixed

Thanks to everyone who supports me on Liberapay or Bitcoin!

Synapse

Neil told us:

This week we put out v1.21.0rc3, which fixes a duplicate messages bug. We expect the full release to go out early next week.

After a long and sometimes painful journey we plan to run sharded event persisters on matrix.org from Monday. All being well we will start running background processes on their own worker to free up the main process. These two projects combined should noticeably improve performance of matrix.org and any large scale deployments.

Aside from that we are continuing to work on room knocking as a feature and have been considering ways to reduce load attributable to state resolution.

We will also dust off the notifications project.

Dept of Bridges 🌉

Gitter

Eric Eastwood told us:

The native Gitter <-> Matrix bridge is underway and we're working on virtual users which is the first piece to making the bridge feel native. You can see what this will look like in this ominous and exciting tweet: https://twitter.com/gitchat/status/1314252991705292800

There is also a GitLab epic you can track which breaks down some of the tasks and holds more details of what we're thinking of for the bridge.

🌈 Once more into the Bifrost

Half-Shot announced:

Asgardians rejoice, for Bifrost development resumed this week! We've worked extremely hard this week to improve general gateway performance, edge case bugs and basically plowing on with the project. A 0.2.0 release will shortly be due with the new changes, and matrix.org is already running the latest and greatest. You can checkout the project over at https://github.com/matrix-org/matrix-bifrost

Dept of Clients 📱

Fractal

Alexandre Franke reported:

Former Fractal GSoC intern Alejandro Dominguez 😎️ got his Integrate matrix-sdk (I) merge request in “Ready for review” state. That’s a +3910/-5439, 38 commits diff‼️ poljar, who is the main contributor of matrix-rust-sdk, already had a look and provided some insights, but given 😱️ the sheer size of the changeset, the moar 👀️ we can get on it the better.

fluffychat

krille offered:

FluffyChat now runs on native Linux desktop (x86).

You can download Linux tarballs from the CI and run them on any system which has sqlite3 installed (which all distros may have). If you have libolm3 installed, you can also use e2ee. Oh and by the way. Its NOT electron or any html5 stuff! Its a native Linux app. We are working on packaging solutions like snap and Flatpak. 😊

Vtx

CraftedCart offered:

I've started to make a Matrix client over the past few weeks, just as a hobby project for now, using Qt and a 'lil C++20! :D

There's not a whole lot to show right now - currently we've got password login, and a simple timeline that can show plain text or m.image thumbnails.

Source code's up at https://gitlab.com/CraftedCart/vtx

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) reported:

This week we finally cleaned up the remaining parts of Chethans GSoC work and merged it. This means Nheko now has partial support for cross-signing. You can either start the verification from another client or when viewing the profile of a user. There are a few things still missing though:

• Nheko does not use the verification status at all at the moment. So while you can verify your master key and let others verify your master key, from Nhekos perspective there is not much benefit yet. You will however show up with a green checkmark in other clients.

• While nheko supports verifying your own master key and after that will also let others verify your master key, it does not yet download the self and user signing keys. This means you won't be able to upload cross-signing signatures for your own devices and other users master keys yet. I'll see if I can implement in the next weeks or so (since literally only the private key download or sharing is missing).

• Nheko can't bootstrap cross-signing yet. This means you need to bootstrap it from Element or another client once. That will probably take a bit until this is complete.

• The UX is still WIP. Currently it shows a lot of unnecessary buttons, that are a bit confusing as well as not enough green checkmarks, so that needs some work.

Other things that happened in the mean time or are work in progress:

• Decryption errors should be reduced by a bit now. This is still work in progress though. With device trust landed now, we should be able to do automatic key requests, sharing and key backup soon.

• Lurkki is currently cleaning up the design of the built in video player.

• manu_kamath is improving the design when showing images at the same time.

• LorenDB has been updating our screenshots and working on an Esperanto translation.

• trilene has been ever busy improving edge cases around the Voip support in Nheko.

So far Hacktoberfest seems to be going well with quite a few people trying to clean up some of the rougher edges in Nheko. If you are using Nheko and think some things could be better, why not try to fix it yourself? October is the perfect time for that!

Featurework of this cycle is also slowly nearing its end. Once cross-signing, voip and the new event store are stable, we plan to improve the UX a bit more as well as some other annoyances and maybe we can do a new release then! A nice and spooky Hacktober everyone!

David Mehren told us:

I just built nheko-git 0.7.2.r2021.517a126a-1 from the AUR and cross-signing works

Element Web

Neil offered:

This week we put out 1.7.9-rc.1 which is available at https://staging.element.io

Highlights include

• Many small fixes with edits and replies

• Fixed a race during cross-signing key upload at registration time

• Clarified when you have unsaved changes in profile settings

Finally, the improved widget support is looking very exciting. Here is a design grab to give you an idea of what to expect. Note this is not a real Element screen shot, the base functionality is there, but the look and feel of the final version is subject to change.

Michael (t3chguy) added:

Element Web is currently transitioning the codebase over to TypeScript - track our progress at http://arewetsyet.bit.ovh/

Hydrogen

Bruno told us:

This week, Hydrogen gained better caching (and a little over-eager on Safari), room list filtering, a grid layout to show up to 6 rooms simultaneously (see screenshot) and work is well on its way for url-based navigation within the app.

also adding the screenshot on iOS that I didn't get in last week, showing the full range of display formats supported 🙂

The app also received some visual polish, thanks to Nad, our designer.

Element-iOS

Manu offered:

We blocked the last week release (1.0.14) because of 2 issues: Unable to decrypt errors and timeline not updating when coming from a notification.

1.0.15 fixes those issues and more. The release should be available in TF soon. Hopefully, we will be able to publish it on the App Store on Monday

Dept of VoIP 🤙

Matrix VoIP Tester

reivilibre said:

https://github.com/matrix-org/voip-tester/pull/18 (updated demo: https://test.voip.librepush.net)

This week, I have improved the scoring rubric, so that the verdict given to you by the tester more accurately reflects the likelihood of connection establishment.

I have also added more hints on what the verdicts mean.

Finally, I have added workarounds to support Chromium browsers that decided to break their WebRTC API so that they could pedantically rename ip to address. In any case, it now seems to be on par with Firefox.

The VoIP tester is now roughly 'usable' but please note that IPv6 support is totally untested (I don't have IPv6) — it is also suspected that some browsers won't try IPv6 anyway.

There are also some more features on the wish list, such as screenshot safety and extra help when things go wrong.

For discussion, feel free to join #voip-tester:librepush.net .

Dept of Encryption 🔐

Olm 🦎

uhoreg reported:

Versions 3.2.0 and 3.2.1 of libolm have been released. 3.2.0 supports fallback keys (MSC2732) in the C library and the JavaScript binding, and includes several JavaScript/TypeScript fixes and improvements. 3.2.1 was just a fix to the TypeScript definition file.

Dept of SDKs and Frameworks 🧰

Ruma

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

jplatte said:

This week, q-b implemented moderation policy events, bringing us to full compatibility with r0.6.1 of the client-server API. iinuwa added the only endpoint we were missing from the appservice API. Additionally, I improved Ruma's JSON canonicalization, making it both simpler and more efficient.

Due to two issues regarding future-compatibility, we're still a while away from our next set of releases, but we're working on it!

New library: ObjMatrix (Objective-C)

js introduced it with:

ObjMatrix (GitHub) is a new Matrix client library written in modern Objective-C for ObjFW. Because it uses ObjFW, it is not limited to Apple platforms, and instead is extremely portable and should run on basically everything.

This is currently in its very early stages and is intended to be suitable to develop bots and clients with very soon. Being suitable to develop bridges with it is another goal for later.

Since ObjFW also works on platforms that are usually hard to support or port software to (e.g. MorphOS), this will hopefully also bring clients for these less mainstream operating systems. I'm especially hopeful about MorphOS here, since ObjFW already runs quite well there (parts of it are included with the OS even!), and hardware on which MorphOS runs on is usually not powerful enough to run Element Web. And eventually, I will want to have a Matrix client on my Amiga, too 😉 (which this should enable).

This currently does not have a decicated Matrix room yet, so all discussions about it currently happen in #objfw:nil.im. So please feel free to join there and follow along development. And contributions are of course very welcome!

Dept of Ops 🛠

famedly.matrix ansible collection

JCG told us:

the famedly.matrix ansible collection has seen a release v0.2.0. Highlights since the last TWIM mention:

• synapse_register: a new module for registering users using synapse's admin API.

• matrix_member: another new module, for inviting/kicking/banning people to/from rooms.

• updates to synapse and element have been merged into their roles

• support deploying release candidates of synapse and element

Dept of Guides 🧭

Noteworthy Hub Guide

balaa told us:

We’ve put together a write up on how to deploy a Noteworthy Hub (transparent TLS proxy) in order to make it easy to run a federation capable matrix home server behind a firewall or NAT: https://noteworthy.tech/hub

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

That's all I know 🏁

Thanks anoa for TCB* the last two weeks! Was fun to be the one to read TWIM for myself. :D

See you next week, and be sure to stop by #twim:matrix.org with your updates!

* Taking Care of Business

Matrix Live 🎙

Dept of Status of Matrix 🌡️

Gitter Enters the Matrix

Half-Shot reported:

Gitter is joining the Matrix ecosystem!

It's true! Element has acquired Gitter from GitLab and will be implementing Gitter's current chat features in Matrix, before rebuilding Gitter as a Matrix client! Before all that however, the first step is to build a proper, modern bridge between the two networks, replacing the old one that's been around for years.

The full spectrum of news is:

• The Matrix.org blog post
• The Element blog post
• The Gitter blog post
• Techcrunch article
• ChangeLog Podcast episode w/ Matthew Hodgson from Matrix and Sytse "Sid" Sijbrandij from GitLab

And if you'd like to chat with your fellow Matrix and Gitter users, there's a bridged room already set up at #gitter:matrix.org!

Welcome to the community, Gitter! 🎉

Dept of Spec 📜

anoa reported:

Spec

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

• No MSCs were merged this week.

MSCs in Final Comment Period:

• MSC2788: Room version 6 as the default room version (merge)
• MSC2414: Make reason and score parameters optional for reporting content (merge)

New MSCs:

• MSC2801: Make it explicit that event bodies are untrusted data

Spec Core Team

In terms of Spec Core Team MSC focus for this week, after dropping off MSC2414 in the FCP bucket, we're heading back for another big swing at widgets. MSC2774 (widget ID URL parameter), MSC2765 (widget avatars) and MSC2790 (modal widgets) are the focus for this week 🙂

Matrix URIs

kitsune told us:

being totally shook up about the Gitter announcement for a good part of the week, I completed the work on MSC2312 (it's about Matrix URIs, ICYMI) and it hopefully won't be long before it becomes the actual standard to share Matrix coordinates in popular web browsers. Ok, who am I kidding - before Matrix clients get on with adoption.

This will allow people to post links to Matrix rooms/messages/users and when clicked will open right in your favourite Matrix client. Super convenient and great for adoption!

Dept of P2P 👥

iOS P2P Demo

Dendrite is a next-generation homeserver written in Go. It is currently serving as the basis for peer-to-peer Matrix experiments

Neil Alexander announced:

Build 38 of the iOS P2P Demo, as built using Element iOS and Dendrite, has been submitted to TestFlight and will hopefully be available for testers shortly (pending Apple approval)! It features lots of updates in the Dendrite backend which should hopefully make it more reliable.

If you have an iPhone or iPad and enjoy things that sometimes work, join the TestFlight here!

Dept of Servers 🏢

Synapse

Neil reported:

This week we put out a new release candidate -1.21.0rc2

Highlights include

• Add experimental support for sharding event persister. (#8294, #8387, #8396, #8419)

• Add experimental prometheus metric to track numbers of "large" rooms for state resolutiom. (#8425)

• Add prometheus metrics to track federation delays. (#8430)

• Fix messages not being sent over federation until an event is sent into the same room. (#8230, #8247, #8258, #8272, #8322)

• Fix a regression in v1.21.0rc1 which broke thumbnails of remote media. #8438

Aside from that we are working on moving background processes away from the main process, actually getting the event persister sharding onto matrix.org and trying to improve Synapse stability generally.

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander said:

Dendrite is nearing beta! Today we will be cutting a candidate 0.1.0rc1 version after a week of hunting and fixing bugs. We are on track to release version 0.1.0 next week, at which point we will be inviting people to try installing and using Dendrite!

Changes this week include:

• Initial sync is fixed after a bug caused us to fall back on incremental sync

• The Content-Type HTTP header is now handled properly when MIME-formatted

• Internal API calls over HTTP in polylith mode now use their own HTTP client with higher timeouts

• Dendrite now tries harder to find missing auth events, using fetcher workers

• Dendrite no longer falls back on /state unnecessarily, which was the cause of a major memory leak and high CPU usage

• Federation HTTP calls now include the User-Agent header

• The event depth field is now ignored in the federation API

• We now report the password change capability properly (thanks bn4t!)

• Registration flows now include the completed field after a failure (thanks Lesterpig!)

• A bug in the previous event updater in the roomserver has been fixed

• A bug where we didn't check our own old verify keys when verifying event signatures is fixed

• A bug where we didn't handle event ID domains being different to the event origin in earlier room versions has been fixed

• TLS fingerprints have been removed

Spec compliance is unchanged:

• Client-Server APIs: 56%, same as last week

• Server-Server APIs: 77%, same as last week

Recently, we updated some of our documentation and created a whole new set of easy first issues and medium to hard issues for contributors.

If contributing to Dendrite sounds like something you would be interested in, please take a look at these issues and join us in #dendrite-dev:matrix.org! There's also #dendrite:matrix.org for general Dendrite chat and updates and #dendrite-alerts:matrix.org for release notifications and important alerts.

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo reported:

• Fix bug leading to many requests at the same time due to error in waiting code

• Implement /query/profile over federation

• Start work on /invite over federation

Thanks to everyone who supports me on Liberapay or Bitcoin!

The Construct

Jason reported:

This week in Matrix Construct supports aarch64 (ARMv8) architectures. The experience has been phenomenal, with performance exceeding all expectations. Last week I wrote about all new vector code inside Construct using SSE through AVX-512 hardware acceleration in servers; that same code is now accelerated by SVE (Scalable Vector Extensions) on ARM architectures when you compile with clang.

This result is important: ARM virtual machines are offered at a significantly lower price compared to x86 from the same vendor. The systems are cheaper, require less power, but generally perform worse. The trick is to optimize the software for the weaker hardware. The benefit allows, for example, Matrix server hosts running Construct to make th eir profit margin from the lower TCO, getting more out of every computing cycle.

Construct is available at https://github.com/matrix-construct/construct and don't forget to idle and idle #construct:zemos.net.

Synapse Deployment 📥️

YunoHost

Pierre announced:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.19.3 (1.20.1 available in branch testing)

Element Web integration had been updated to 1.7.7 (1.7.8 available in branch testing)

Dept of Bridges 🌉

mautrix-signal

Tulir said:

I made a Signal bridge using signald.

Currently it supports bridging messages, reactions and signal read receipts. End-to-bridge encryption also exists (mautrix-python does all the work there). The bridge can be linked as a secondary device and possibly even registered as the main device, but I didn't actually test registering yet. Setup instructions are currently somewhat non-existent, but it's mostly the same as my other bridges plus signald as a separate daemon.

The repo is https://github.com/tulir/mautrix-signal and the room is #signal:maunium.net

He then popped up again a few days later to say:

After the initial announcement earlier this week, I implemented a few of the missing features like media bridging, and even added setup instructions.

The speed at which Tulir writes bridges scares me sometimes.

mx-puppet-discord

mx-puppet-discord is a (double)puppeting and relay bridge for discord, based on mx-puppet-bridge

sorunome said:

mx-puppet-discord got updated to the newest discord.js version, meaning you have to update if you want to continue to operate it, due to discord having changed their gateway url!

mx-puppet-discord now also supports the intent stuff, so be sure to update by 7th oct

If you run mx-puppet-discord (like I do), make sure to update by October 7th or it will stop working!

Bridges do Hackertoberfest

Half-Shot reported:

Hey folks, I wanted to give another shoutout to say that we are still accepting PRs as part of hacktoberfest. Contributing 4 PRs will get you a T-Shirt (sadly not a Matrix one). Obviously, please ensure your PRs are meaningful (no copyright adjustments, typo fixes).

You can work on any issue, but we've highlighted some issues that would be perfect for newcomers over at https://github.com/matrix-org/matrix-appservice-bridge/issues?q=is%3Aopen+is%3Aissue+label%3Ahacktoberfest and https://github.com/matrix-org/matrix-appservice-slack/issues?q=is%3Aopen+is%3Aissue+label%3Ahacktoberfest

Cadair reported:

In the hope of expanding the number of people contributing to the matrix-appservice-slack repo I have spent a chunk of my morning improving the issue descriptions and labelling up issues. If you are interested in fixing a little annoyance with the slack bridge or just fancy writing some typescript see the good first issue label on the repo.

matrix-appservice-slack 1.6.0 rocks the block

Half-Shot reported:

The matrix.org team are delighted to bring you the latest in Slack bridging technology. Do not let the

minor version bump fool you, this release is packed with the good stuff. The headline feature is that our phase 1 encryption feature has landed and is free for users to experiment with. Head over to the

docs to see how to set this up.

There have been other notable changes, such as:

• New configuration options to allow or deny some channels from being bridged.

• Support removing reactions from Slack and Matrix messages.

• Add onboarding message for new users when puppeting is enabled, to encourage them to puppet.

• Improved feature documentation

(and many many bugfixes)

You can read the release information over at https://github.com/matrix-org/matrix-appservice-slack/releases

Looking forward to seeing all the new Hacktoberfest contributors! 🎃

Dept of Clients 📱

fluffychat

sorunome offered:

Fluffychat 0.19.1 has been released!

Features

• Implemented ignore list

• Jump to events in timeline: When tapping on a reply and when tapping a matrix.to link

• Display messages with up to 10 emotes or emoji bigger

• New design for the chat list and message bubbles

• Implement reactions

• Implement password change

• Implement deactivate user account

Fixes

• Timeline randomly resorting while more history is being fetched

• Automatically request history if the "load more" button is on the screen

Hydrogen

Bruno said:

More browser compatibility work this week, making Hydrogen run IE11 on Windows 7, and on Safari on macOS and iOS (still with some caveats). Also fixed several bugs:

• fix for unable to open session after a synapse bug manifested itself

• prevent the app locking up when you start the app with previously unsent messages

• fix sync errors being reported as "null" in the banner

• handle timeout during initial sync (important for large accounts) (although I have seen 1 report that this still isn't fixed, please report if you can't login with a large account)

SchildiChat for Android

SpiritCroc said:

SchildiChat's codebase has been updated to Element 1.0.8!

Furthermore, there have been a few design updates:

• Media items (pictures, videos, stickers) are no longer displayed in message bubbles

• Bigger stickers

• Avatars are now hidden in direct chats (when using dual-side message bubbles)

Finally, Schildi doesn't crash anymore if somebody sends an empty message.

SpiritCroc also mentioned some relavent links!

• website: https://spiritcroc.de/schildichat-android

• repo: https://github.com/SpiritCroc/SchildiChat-android

• room: #schildichat-android:matrix.org

Element-iOS

Manu announced:

1.0.14 is in on the release path. It has:

• Room: Differentiate wordings for DMs

• Room: New room details screen

• Add Estonian support

• Polishment in several areas and many bug fixes

Full changelog:

https://github.com/vector-im/element-ios/releases/tag/v1.0.14 https://github.com/vector-im/element-ios/releases/tag/v1.0.13

Element Web

Neil announced:

This week released v1.7.8

Highlights include

• Secure Backup has been moved out of the registration flow to a toast when you first encounter an E2EE room, which simplifies the new user experience

• Added options to hide various UI features when hosting a custom Element

Aside from that we continue to improve on widget support for resizable widgets, modal widgets and generally making widgets better.

We are also continuing to work on instrumenting the app, improving mobile support for matrix.to and making jitsi calling more reliable.

Element Android

benoit told us:

Element Android: Version 1.0.8 is now available on the stores, it fixes issues with verification and PIN code among other issues (see https://github.com/vector-im/element-android/releases/tag/v1.0.8 for more details). Now we are working on improving performance when sending messages to rooms, and also improving global UX, especially of the home (rooms list). Search messages (in clear rooms for the moment) is coming soon, and it will be also possible to filter the room members list.

We will also spend some time on the new Android SDK, https://github.com/matrix-org/matrix-android-sdk2, which is for the moment a quick extract of what we have in Element Android. We have to take care of it as a real product now: document it properly, set up CI, export Javadoc, develop a sample app, etc.

Element for Nextcloud

Gary Kim reported:

Element for Nextcloud v0.6.11 has been released this week. The new version comes with various bug fixes, dependency upgrades, and an upgrade to Element Web v1.7.8. The version is also compatible with Nextcloud 20 which is being released soon.

Dept of SDKs and Frameworks 🧰

Ruma

iinuwa told us:

Over the past couple of weeks, we've received PRs for all of the remaining federation endpoints, and all but one have already been merged!

Now that the end is in sight, we're turning our focus elsewhere. We're working on cleaning up and fixing a few bugs in our event signing code and soon will create tracking issues for filling out the Identity Service API.

Dept of Services 🚀

t2bot.io

TravisR reported:

t2bot.io has crossed 1M monthly active users

All of these users are Telegram/Discord users that have been brought into Matrix over the last 30 days. This doesn't appear to be a temporary spike either: over the last 8 weeks t2bot.io has been hovering at 900-950 thousand monthly active users, up from 600-700 thousand. Record-setting traffic levels have also been achieved, with matrix.org being able to keep up for the first time in a long while.

Overall it's a good sign to see so many communities making the jump to Matrix and sticking around ❤️

Dept of Bots 🤖

zabbix-matrix

progserega told us:

I added zabbix-bot to my zabbix-matrix repo, which can get information about current problems from zabbix-server and send it to matrix user. https://github.com/progserega/matrix_zabbix

In our company we use it for get current situation and https://github.com/progserega/im_sender_service for sending events from zabbix.

I've found that having your systems reporting in a room while you chat around it can be really productive. Props to supporting yet another monitoring platform!

Dept of Interesting Projects 🛰️

Jitsi E2EE Calls using Olm

While 1-1 calls benefit from end-to-end encryption due to WebRTC, Jitsi group calls have always only benefitted from transport encryption.

A while ago Jitsi announced that they were adding E2EE to Jitsi. But did you know that it's using Matrix's Olm encryption under the hood? It's currently available as an experimental feature on https://meet.jit.si!

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

We also now have a room where the ping bots are only hosted on non-Synapse servers! See the scoreboard below.

Final Thoughts 💭

XKCD 2365

Alexandre Franke told us:

XKCD made another strip about messaging systems and I’m outraged to see that Matrix has been left out of it.

Not to worry though, we've fixed it up!

TWIM authorship

I hope you enjoyed this week and last's TWIM editions! Benpa will return next week for more of your regularly scheduled programming. Ciao!

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Hello all, and welcome to this week's addition of This Week in Matrix! Matrix is an open network protocol for secure, decentralized communication on the web.

My name is Andrew (aka anoa), and I'm a Synapse developer at Element. Thanks to Ben for letting me take over the reins of TWIM for this week! I'll actually be doing the same for next week as well, so please adjust your clocks to Anoa Nonstandard Time accordingly.

With that out of the way, let's jump right in!

Matrix Live 🎙

It's demos week again. This time around we've got the following lineup:

• Michael shows off all the new widget goodies coming to Element Web!
• Bruno shows off Hydrogen's new encrypted session backup support!
• Ismail details the new room creation flow on Element iOS!
• Jorik presents his work on revamping the UX of https://matrix.to that he completed for his second summer internship at Element!
• Hubert fixes another class of failure-to-decrypt messages edge case with Device Dehydration!
• Half-Shot closes off with the tale of Encrypted Bridges!

Don't forget that Matrix Live is also available in podcast form, if you're into that sort of thing. Search for "Matrix Live" wherever you get your podcasts.

Dept of Spec 📜

anoa (hey that's me!) told us:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

• MSC1960: OpenID information exchange with widgets

MSCs in Final Comment Period:

• No MSCs are in FCP.

New MSCs:

• MSC2790: Modal widgets (acquiring user input from a widget)
• MSC2788: Room version 6 as the default room version
• MSC2787: Portable Identities
• MSC2785: Event notification attributes and actions
• [WIP] MSC2783: Homeserver Migration Data Format
• MSC2782: Pushers with the full event content
• MSC2781: Deprecate the (reply) fallbacks in the Matrix specification

Spec Core Team

In terms of Spec Core Team MSC focus for this week, we've been rather busy with implementation, so we'll be continuing on with the same focus as last week. As a reminder, that's MSC2414 (making reason and score optional on reports).

Dept of Servers 🏢

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go.

Neil Alexander announced:

This week has mostly been spent trying to improve stability and to fix bugs ahead of the beta release, which is so far still planned to go ahead in the next two weeks.

Changes this week include:

• Room version 6 is now the default for newly created rooms

• Soft-fail of events that aren't allowed by the current room state is now implemented

• Support for configuring old_verify_keys has been added to the Dendrite config

• Correct formatting for signing key IDs is now enforced in the configuration

• Initial support for peeking over federation (MSC2444) is in progress and it "even works!" (thanks Matthew!)

• Federated joins will now continue being processed even if the client gives up on the join due to a HTTP timeout

• Backoff code has been refactored a bit more, and now correctly affects device list syncing

• /make_join now errors correctly if a federated user tries to join a room which all members have left

• A bug where a single user could start multiple simultaneous federated joins to the same room has been fixed

• Some initial (but unfinished) support for the /key/v2/query notary endpoint has been added

• Signature verification has been updated to not fail if the event origin field is missing (although it still requires a signature from the domain of the sender field)

• A number of places where we use SQL transactions have been updated with safe wrappers (thanks samcday!)

• A couple of error codes on invite endpoints for room version 6 JSON violations have been fixed

Spec compliance has improved slightly for federation:

• Client-Server APIs: 56%, same as last week

• Server-Server APIs: 77%, up from 74% last week

As always, if contributing to Dendrite sounds like something you would be interested in, please feel free to join us in #dendrite-dev:matrix.org ! There's also #dendrite:matrix.org

Synapse

Neil offered:

This week we released 1.20.0 (and 1.20.1) highlights include shadow banning support and including unread message counts in the sync response. This will help client developers and is a precursor to improving notification support.

We’ve also been looking at adding monitoring to get a better sense of which rooms are most expensive from a state resolution perspective, we also want a better way to track federation lag.

Up next we’ll move all background tasks away from the main process and try it out on matrix.org, we are hoping for a 10-15% saving in CPU. Event persistence sharding, specifically the new stream token format is on hold slightly while we work on a nasty race condition on start up of the event persister. We hope to get back to the main sharding project next week.

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo announced:

• Respect SRV record when sending requests over federation
• Don't send new requests to servers if we are already waiting
• Implement get_missing_events
• Bug fixes and code cleanup

We also started work on a system that retries failed or blocked requests after some time.

Thanks to everyone who supports me on "Liberapay" (https://liberapay.com/timokoesters) or Bitcoin!

The Construct

Jason reported:

This week in Matrix concludes a summer of Construct where several transformative rounds of optimization took place. I'd like to talk about these achievements and why they are important for future directions.

Over the summer, Construct introduced new vector extensions to the project. As server software, our primary target is the datacenter which (for now) is dominated by x86 hardware. The latest features of x86 chips include AVX-2, AVX-512, and on-die GPU. These advancements are important because they help mitigate current limitations of hardware, such as the cubic relationship between a processor's frequency and its power consumption. These limitations mean that CPU cores aren't completing more cycles-per-second every iteration of their development -- they're not getting much faster.

The problem with threads is that they can introduce a lot of complications to a project. Construct is able to forego multi-threading as a network server because it is primarily IO-bound. The benefits to a single-thread design in both performance and simplicity cannot be overstated. This is why Construct stands to benefit the most from features which allow more work to be accomplished at every individual computing cycle.

Construct undertook several endeavors over the summer which directly leverage platforms featuring SSE, AVX, AVX-512, etc. As an added bonus, our approach is designed to effortlessly port to ARM's Scalable Vector Extensions (SVE) as it becomes available (inside datacenters too!). Our focus has been JSON, Unicode, and finally Base64. Detailed explanations for each of these will need to be discussed in their own posts, but in summary:

• Matrix specifies a canonical form of JSON which is not necessarily the same as the JSON the server receives. Therefor it is imperative that Construct is liberal with what JSON it accepts and correct with its transformation into canonical JSON. Over the summer, a portion of Construct's canonical transform received some optimization to go beyond the default method of character-by-character read-modify-write. As seen in [1], a streaming hardware-accelerated transform now processes at least 16 characters at a time (with more possible) without forward branching except for the parts where transformation needs to occur. One of these transformations is UTF-16 to UTF-8 surrogate conversion, which leads into the second endeavor.

• Construct introduced a completely branch-free Unicode toolset in [2]. Among the functions offered is a custom transform of UTF-16 surrogate pairs to UTF-8 sequences. Using the new vector registers, Construct can transform two UTF-16 surrogates in parallel to output two UTF-8 sequences, or two UTF-16 surrogates in a pair to output one UTF-8 sequence. Unfortunately, these specific functions were written at fixed vector widths, so more work needs to be done to really take advantage of the widest hardware. Each surrogate is 6 bytes, and a surrogate pair is 12 bytes; therefor we cannot make use of the last 4 bytes of a 16 byte vector. However, with a little more work this approach can be extended to a 64 byte vector, capable of decoding 5 surrogate pairs and 10 individual surrogates in parallel!

• Professor Daniel Lemire recently published a paper about fast Base64 from hardware acceleration in [3]. This approach is extremely elegant on the 64-byte-wide AVX-512 system. Prior to Construct's implementation of this in [4], the Boost library base64 encoding and decoding took roughly 20 and 25 cycles per character respectively. Our implementation on the same system, an old system with only SSE2 (not even AVX-512!) yields 5 and 6 cycles per character!

All of this helps lay a foundation for Construct to introduce Federated Media Rooms sometime in the future. Currently, Construct stores media in a separate database. Recently there's been work on a separate branch at [5] which stores actual file block inside events using Base64. It is for this reason sub-cycle and branchless JSON parsing and Base64 encoding is essential for maximum performance. The result is worthwhile, as the latency for querying the media database is slower than parsing and decoding the event content already in-hand.

That's all for today. Construct is available at https://github.com/matrix-construct/construct and don't forget to idle and perform #construct:zemos.net / #construct:maunium.net

Thanks!

1. https://github.com/matrix-construct/construct/blob/563f833ab325f27ff9e71af61af427fb02812f90/ircd/json.cc#L3483

2. https://github.com/matrix-construct/construct/blob/563f833ab325f27ff9e71af61af427fb02812f90/ircd/utf.cc

3. https://arxiv.org/abs/1910.05109

4. https://github.com/matrix-construct/construct/blob/563f833ab325f27ff9e71af61af427fb02812f90/ircd/b64.cc

5. https://github.com/jevolk/charybdis/tree/federated_media_rooms

It's really exciting to see Homeserver development ramping up from all angles, and nice that the protocol warts are slowly getting ironed out in the process.

Synapse Deployment 📥️

Kubernetes

Ananace told us:

Before I forget (more) about it, I pushed the 1.20.0 tag for my K8s-optimized container image as well as my Helm chart.

YunoHost

Pierre reported:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.19.3 (1.20.0 available in branch testing)

Element Web integration had been updated to 1.7.5 (1.7.7 available in branch testing)

dacruz21/matrix-chart

Typo Kign announced:

Thanks to Arkaniad, v2.7.0 of my Matrix Helm chart for Kubernetes is released with support for exporting Prometheus metrics. It pairs well with the Synapse Grafana dashboard.

More Kubernetes

Ananace said:

And just pushed the 1.20.1 tag too for my K8s-optimized container image as well as my Helm chart.

Third-Party PowerPC and ARM64 support for Synapse

andreas announced:

The synapse docker image from AVENTER (https://www.aventer.biz), does support PowerPC (ppc64le) and ARM64 architecture now. But at the moment only under the docker tag "ppc". https://hub.docker.com/r/avhost/docker-matrix/tags?page=1&name=ppc We will be happy to get feedback.

As a Synapse developer, it's great to see the community making personal and enterprise Matrix deployments more accessible!

Dept of Clients 📱

FluffyChat

sorunome told us:

Fluffychat 0.19.1 has been released!

Features

• Implemented ignore list

• Jump to events in timeline: When tapping on a reply and when tapping a matrix.to link

• Display messages with up to 10 emotes or emoji bigger

• New design for the chat list and message bubbles

• Implement reactions

• Implement password change

• Implement deactivate user account

Fixes

• Timeline randomly resorting while more history is being fetched

• Automatically request history if the "load more" button is on the screen

Sorunome briefly mentioned afterwards that there is no 0.19.0 due to some accidental messed up tagging, and that it was easiest to just call the new version 0.19.1.

Element Web

Neil offered:

This week we put out a new release candidate 1.7.8-rc.1 highlights include:-

• Secure Backup has been moved out of the registration flow to a toast when you first encounter an E2EE room, which simplifies the new user experience

• Added options to hide various UI features when hosting a custom Element ...along with various smaller fixes.

Aside from that the work to improve the widget experience continues with modal widget next on the agenda.

Next week we will continue to improve widgets and add some more instrumentation into the app.

Hydrogen

Bruno said:

Released 0.1.0 (and 0.1.1) this week with read-only session backup enabled 🎉 Also doing more work to make Hydrogen work on IE11 on Windows 7 (it does already for Windows 10), Safari and other browsers where you get TransactionInactiveError during login, hope to release that soon.

I need to give Hydrogren a shot myself, the quick speeds and low RAM usage are really attractive.

Element-iOS

Manu offered:

This week, room settings have been updated with a new intermediate screen. The codebase saw the introduction of an AppCoordinator (in swift) which will help us to have a better control on navigation within the app AND to use swift from end to end.

Dept of SDKs and Frameworks 🧰

Polyjuice Client 🧙

uhoreg told us:

Polyjuice Client v0.3.0 has been released. This release includes many breaking changes. ("Breaking" in the sense of API changes, rather than the sync process suddenly failing to work, which was already featured in a previous release.) This release also includes many changes, such as the client managing more bookkeeping, detecting if it got logged out, and supporting more Matrix features. See the release notes for more information.

Igor

uhoreg reported:

Igor is a bot framework for Elixir. Igor v0.2.0 has been released. The main change is updating it to use Polyjuice Client 0.3.0.

Dept of Bots 🤖

Matrix-Architect

erdnaxeli announced:

Hi!

I present you a new project I've been working on for some time. It's a bot that allows you to use the admin API through matrix, by typing commands to the bot. The inspiration comes from the OperServ-like bots that allow IRC operators to administrate an IRC server.

The bot exposes some of the available admin APIs and aims to provide some more high level commands by combining different APIs. There is currently one "high level command", !room garbage-collect, which allow you to purge from your HS all rooms without local users.

The project is written in crystal and is my first project in this language. It should be stable in normal conditions, but don't hesitate to fill issues. A Docker image is provided for more convenience.

I hope you will find this project useful 🙂

https://github.com/erdnaxeli/matrix-architect

People administering their Synapse deployment through Matrix itself! How deep does the rabbit hole go?

Songwhip bot

Tulir reported:

benpa wanted a bot that replies with a songwhip.com link whenever someone sends a music link (youtube, spotify, apple music, etc), so I wrote a small maubot plugin to do that: https://github.com/maubot/songwhip

It's available at @songwhip:maunium.net

I'm also desperately in need of this for the 10 open.spotify.com links that get thrown at me every week. Thanks Tulir!

Cyberbot

jj reported:

There's a new bot! Cyberbot.

It supports E2EE and can be easily extended with Python plugins. Can be used e.g. for GitLab notifications, automated user invites, room creation, and of course can be programmed to react to any message posted in a room.

Dept of Interesting Projects 🛰️

matrix-gotify

sorunome said:

Soru made a new thing, matrix-gotify. It is a gotify plugin to receive matrix notifications. This means, you can now receive matrix push notifications on your android phone self-hosted without the need of google services! That is why putting the full event in the push notification is actually not a privacy leak in this case.

This plugin could also be used to receive push notifications on any other kind of device that gotify supports.

Please note that this is independent of any matrix client - you don't even need to run one to be able to use this!

I asked whether an OpenPush-like solution be built on top of this , and Sorunome responded that someone had already started working on just that! https://github.com/gotify/android/pull/115

This would allow other matrix clients on your phone to get their push notifications through your gotify client, instead of needing to run a process each. Great for battery life without proprietary Google blobs!

Homeservers on-the-go

js got Synapse running in a car on a German highway:

the setup is a cigarette lighter to 2x 230V converter, one being used to power the RockPro64, the other being used to power my notebook. My notebook is connected to the hotspot from my phone, while also being connected to a USB ethernet, the other end of which is plugged into the RockPro64.

The notebook then acts as a gateway, as well as SSHing to a tiny VPS with -R, to get a public port, and forwarding that traffic to the RockPro64.

Maybe one day we'll all have an embedded Synapse homeserver in our cars. P2P E2EE car comms anyone?

Matrix VoIP Tester

reivilibre announced:

https://github.com/matrix-org/voip-tester

I have been through the perils of setting up a TURN server and having VoIP continue to fail, whilst spending hours scratching my head and staring at Wireshark without getting anywhere. When I was given free reign over project choice last year during my internship, I chose to start a tool that tests your homeserver's VoIP configuration, hoping it would be useful to both me and the community at large.

It saw some progress but I never managed to iron out some of the 'last few things' or put a pretty front on it — until about 2 weeks ago, when I got some more time to do it.

There are known issues — such as the scores being very arbitrary, wrong and misleading; it crashes Chromium every time (on my machine) and it completely misbehaves in Brave Browser (on my machine). It may also not be the prettiest (but hopefully it is at least somewhat inoffensive). I hope to work on these annoying issues soon.

I have deployed a test instance at https://test.voip.librepush.net — it can be tried in a WebRTC-supporting browser (probably Firefox if you want half a chance). Please don't take it to heart if you get a Fail or Poor score — it's probably not your fault. :)

Oliver was interning with us at Element this Summer. Fixing up and releasing this VoIP tester was part of it, but he's still working on it in his spare time even though his internship has finished :)

Through this I found out that I forgot to turn on my turnserver on my homeserver again. Thanks Oliver!

Dept of Guides 🧭

German-Language Guide for Getting Start with Matrix

Samuel told us:

I started an article series about Matrix on my german blog. The goal is to make it easier for new users to get started with Matrix. https://blog.sp-codes.de/werde-teil-der-matrix-matrix-teil-1/

Guides like these are the essential entrypoints to a project for a lot of users. The more, in different languages, the merrier!

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Ping Graphs by Timo™️

timo told us:

Here we look at how fast ping bots respond.

I changed the formatting of the plot a bit to make it more readable. Note that it is now using a log scale which allows us to see more data at the same time.

Non-Synapse Ping Room

Tulir reported:

Now that we have multiple somewhat federating Matrix homeserver implementations, I decided to make a ping room where all the echo bots are hosted on second-gen homeservers: #ping-no-synapse:maunium.net. Synapse users are still allowed to join and !ping, but all the pongs will come from non-Synapse servers.

Based on observations in that room so far, the new server implementations don't like eachother very much.

Final Thoughts 💭

• Magnus Svensson was left speechless by the amazing projects that we got to see this week.

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Matrix Live 🎙

Dept of Status of Matrix 🌡️

Element review by TechRadar

TechRadar have published a fairly thorough Element secure messenger review. This covers the Web, Android and iOS clients, and appears to use the matrix.org server for signup.

The review is very positive, awarding 4.5/5, and concludes:

The Element messenger platform scores highly for its approach to security and its commitment to decentralization, and it's definitely going to be of interest to businesses wanting control over their own chats – as well as plenty of individual users as well.

Dept of Spec 📜

Spec

anoa told us:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

• No MSCs were merged this week.

MSCs in Final Comment Period:

• MSC1960: OpenID information exchange with widgets (merge)

New MSCs:

• MSC2778: Providing authentication method for appservice users

• MSC2775: Lazy loading over federation

• MSC2774: Expose the widget ID to the widget

• [WIP] MSC2773: Room kinds

Spec Core Team

In terms of Spec Core Team MSC focus for this week, MSC1960 has made it into FCP, so this week our focus is MSC2414.

Dept of Servers 🏢

Synapse

Neil announced:

This week we put out two point releases to fix critical bugs. At the very least ensure that you have upgraded to 1.19.2 which is a security release but you may as well go the whole hog and upgrade to the hot off the press 1.19.3. We will release 1.20.0 early next week.

Aside from that we continue to work on performance analysis and the sharding of the event persister continues - it’s proving to be a really tough job but we are getting there. We’ve also been making progress on the room knocking implementation

This week we say goodbye to Oliver (reivilibre) as a leaving present he fixed a long standing bug that prevented servers catching up after a federating outage and as we speak is furiously trying to finish a service to test TURN configuration. Thanks Oliver!

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander said:

Not much has happened this week, as the Dendrite team have been taking some time off. However, we still have a couple of changes to report:

• Support for rejected events has been merged

• Work on soft-fail has begun (although not complete yet)

• A new mechanism for avoiding SQLite parameter limits was contributed (thanks HenrikSolver!)

Spec compliance is on the up:

• Client-Server APIs: 56%, same as last week

• Server-Server APIs: 74%, up from 71% last week

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo announced:

Welcome back! This week we made some groundbreaking progress - this is not a joke, we found a bug in Synapse while breaking multiple Matrix rooms, causing the Matrix team a lot of unnecessary stress. Let me explain: The good news is that Conduit is starting to federate now. This means that you should be able to join all public rooms of the Matrix network and exchange messages. Note that Conduit does not do all the checks it should be doing yet making it stop sending messages from time to time as well as that advanced features like loading the history, syncing temporary data like read receipts or accepting invites are not implemented yet.

The bad news is that, while Synapse is happy to accept Conduit's messages when it is already part of the room, joining into one of the rooms Conduit servers are part of didn't work because of an event validation bug. The Matrix team did an excellent job at fixing this bug and releasing a Synapse patch the same day, but the damage has been done making a few rooms inaccessible to old Synapse servers. Thanks to everyone who supports me on "Liberapay" (https://liberapay.com/timokoesters) or Bitcoin!

Synapse Deployment 📥️

Synapse on NetBSD

js reported:

I ported Synapse + its dependencies to NetBSD and it is now in pkgsrc as chat/matrix-synapse. This means that Synapse is now easily installable on any operating system that supports pkgsrc (which is many, e.g. NetBSD, Linux, macOS, Solaris, AIX, Haiku, …). And while at it, I also ported mautrix-hangouts + dependencies (in pkgsrc as chat/mautrix-hangouts).

YunoHost

Pierre told us:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.19.1 (1.19.2 available in branch testing)

Element Web integration had been updated to 1.7.5 (1.7.7 available in branch testing)

Kubernetes

Ananace offered:

Both image tags and a new chart version are hereby pushed for 1.19.2 for my Synapse image and Helm chart

then later

Just pushed the updates for the K8s-optimized Synapse image and chart for 1.19.3

Dept of Bridges 🌉

matrix-appservice-bridge hits 2.0.0ʳᶜ¹

Half-Shot said:

Hey bridge developer enthusiasts! Myself and ChristianP (bridge crew of matrix.org) have been working hard on freshening up the matrix-appservice-bridge library. It's no secret that it was using "classic" Javascript rather than ES6 syntax, was poorly documented in places and there were no types at all to make use of.

But that's all changed now, the 2.0.0-rc1 release brings fresh types, new convienence methods, a total swichover from Bluebird Promises to native Promises and much much more. If you are looking to quickly get set up writing bridges, it's never been easier.

Check out the slack-starter project to get your first taste of bridge excellence and see what you think.

As this is an RC, feedback is valuable and logging bugs helps us all. Please do so if you encounter any issues 😄

Bridge Encryption Support

Half-Shot reported:

Hi encryption fans. This week the bridge team has been churning away at adding support for encrypted rooms in the matrix-appservice-bridge library, to enable support in matrix-appservice-irc, matrix-appservice-slack, and anyone else using the library. This is probably the most exciting feature the library has seen in a long time!

The feature will be a simple config toggle, so existing bridges will have to do very little work to support it.

The first phase plans to use Pantalaimon with bridge users syncing like real users, but eventually the hope is that this will be all built into the library. Come check out this issue to keep track of progress.

Dept of Clients 📱

Element Web

Neil announced:

• Released 1.7.6 and 1.7.7 highlights include

  • Redesigned right panel where top right actions are now revealed via the i icon

  • Widgets can now be opened in the right panel

  • Widgets won't be visible in the Apps Drawer (top of timeline) by default (except Jitsi) - you need to pin them from the right panel

  • Widgets can now be resized

• A new version of matrix.to is now live you can check it out here, bug reports welcome

• A new feature to defer e2ee set up until the user actually wishes to use an e2ee channel.

• A series of config flags to remove various features from the UI thereby simplifying the experience.

Mirage

miruka said:

0.6.3 and 0.6.4 were released this week, and I'm now working on desktop notifications support and push rules control.

Added

• Add a system tray icon.

  A left click will bring up the Mirage window, middle will quit the application and right will show a menu with these

  options.

• Add a closeMimizesToTray setting to the config file, defaults to false.

  Controls whether closing the Mirage window will leave it running in the system tray, or fully quit the application.

• Add a discrete read marker indicator to messages, shows how many people

  have this event as their last seen one in the room.
  A way to see who read the message and when will be added in the future.

• Themes: add chat.message.localEcho and chat.message.readCounter color properties

• Add a zoom setting, defaults to 1.0

• Add a lexicalRoomSorting setting, to sort rooms by their name instead of

  recent activity.
  A restart is needed to apply changes to this setting.

Changed

• Restrict Mirage to a single instance per config folder, trying to launch a

  new window will instead focus the existing one.
  The MIRAGE_CONFIG_DIR and MIRAGE_DATA_DIR environment variables can be

  set to run different "profiles" in parallel.

• Reduce the visible lag when opening a chat page, switching rooms should be

  a lot smoother

• When using the focusPreviousMessage and focusNextMessage keybinds, if no

  message is focused and the timeline has been scrolled up, focus the message in the center of the view instead of returning to the

  bottom of the timeline and focusing the last one.

• Don't re-center the room list on clicks by default.

  This prevents the list from jumping around every time a room is selected.
  The previous behavior can be restored with the new centerRoomListOnClick

  setting.

• Show a better terminal error message than "Component is not ready" when the

  window creation fails, giving details on what went wrong in the code

• If an account's access token is invalid (e.g. our session was signed out

  externally), say so with a popup and cleanly remove it from the UI, instead of spamming the user with errors.

• Rename message context menu option "Debug this event" to just "Debug"

• Unify up/down and (shift+)Tab navigation for the account Sessions page

• Changes to the UI scale/zoom via keybinds are now persisted across restarts

• Themes: uiScale is now bound to window.settings.zoom.

  This change is necessary to keep the zoom keybinds working.

Fixed

• A bunch of stuff, see the full changelog for 0.6.3 and 0.6.4

Element Android 1.0.7

benoit offered:

Element Android 1.0.7 has been released (still stuck in Google pipes writting those lines), and will be quickly followed by 1.0.8 to fix a problem with cross signing verification.

We are working to implement search of messages in a room and we keep on fighting bugs and improving performance of the application.

We are also iterating on the home screen, to improve user experience, and try to satisfy users with a few rooms as well as users with several hundreds of rooms.

Element-iOS

Manu said:

This week, we released 1.0.12 with several bug fixes including the one for background crashes due to PushKit

Hydrogen

Bruno told us:

Released 0.037 on Monday with some polish for E2EE:

• remove outdated devices when querying

• update room summary with decrypted events, also when retrying decryption

• various crash protections

• keep megolm session with earliest index when receiving a room key we already know about

• add new icon

• fix issue where olm would be loaded twice when login failed

• show decryption errors in timeline

• show encryption enabled tile in timeline

The rest of the week was implementing session backup, which I hoped would be ready to release today but it needs some more polish unfortunately. In the meantime, here's a GIF teaser:

FluffyChat

Thank you to Krille, who told us:

FluffyChat version 0.18.0 is out now in F-Droid, Testflight and soon the PlayStore:

Features

• Added translations: Armenian, Turkish, Chinese (Simplified), Estonian
• Url-ify matrix identifiers
• Use server-side generated thumbnails in cleartext rooms
• Add option to send images in their original resolution
• Add additional confirmation for sending files & share intents
• Add option to opt-in to report issues / crashes to sentry
• Write keys to online key backup, fully implementing online key backup

Changes

• Tapping links, pills, etc. now does stuff
• Better handling of sending messages in bad network
• Better recovery of "keys not cached"
• E2EE is enabled again

Fixes:

• Various html rendering and url-ifying fixes
• Added support for blurhashes
• Image viewer now eventually displays the original image, not only the thumbnail

Dept of SDKs and Frameworks 🧰

Hemppa

Hemppa the bot is a multi-purpose Matrix bot for writing new functionality easily with Python.

Cos said:

Hemppa gained a small but big feature for making fediverse better: per-room Mastodon tooting support. This means that you can set up a Mastodon account in your room and anyone with moderator rights can toot to that account. This way you and your roommates can super easily send toots to the world directly from Matrix. Mastodon (unlike the big alternative) supports RSS feeds out of the box so it's very easy to subscribe to users and hashtags with stock RSS bot. Toot #hemppathebot if you like it! Another new contribution worth mentioning is welcome which can send welcome messages to users registering on the server or joining a room. https://github.com/vranki/hemppa

Ruby

Ananace said:

Another week, another Ruby SDK release. This time adding a little fix to avoid duplicate state events being passed twice to the application if identical ones arrive in both state and timeline, also adds a global state event handler and moves room event handling to not depend on room instances themselves.

Asked how much the SDK is used in production, Ananace said:

a little, we've hooked Matrix into our server orchestration / configuration management system TheForeman - which is a Rails application. Also got a colleague who's doing a bot to linkify internal ticket IDs.

Ruma

iinuwa told us:

Thanks to @q-b and @jtescher we have implemented more of the federation endpoints (bringing us up to about 85% completion) and every (one) endpoint of the Push Gateway API!

We fixed a bug where federation endpoints expected access tokens instead of server signatures for authentication. In the future, we may add an API that requires signatures on requests.

Finally, we've also continued progress on the non-exhaustive type updates for backwards compatibility; we expect to finish this within the coming weeks.

New Contributors

• @jtescher, not new but forgot to mention earlier!

• @williewillus

Dept of Bots 🤖

Matrix Reminder Bot

anoa announced:

v0.2.0 of matrix-reminder-bot has been released! This releases includes lots of bugfixes, updates and polishing. Find the list below:

Features

• Better support for command prefixes other than the default !.

• Just writing !silence now silences the currently active alarm.

• The bot will now print the correct syntax of a command if the user fails to follow it.

• The bot will reply to events if it cannot decrypt them, as well as offer helpful tips that both the user and bot operator can try to fix things.

Bugfixes

• Timezones. They should finally work correctly! ...no 100% guarantees though.

• Alarms were a bit broken. They're fixed now.

• Fix commands with formatting and newlines not being picked up by the bot.

• Fix non-latin characters preventing reminders from being deleted.

Internal changes

• Add a dev-optimised Dockerfile for quicker iteration during development.

• Better wording revolving alarms. They're just reminders that alarm repeatedly when they go off.

• Log why the bot is unable to start.

• Don't print "Unknown help topic" in case the user is trying to ask another bot for help.

• The config dict is now a singleton.

• Type hints everywhere!

Additionally, the minimum Python version is now 3.6. matrix-reminder-bot is made with nio-template.

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Dept of Ping Extra:

timo announced:

Normally the ping shows how long other servers take to receive the sending-server's message. Some servers receive it quickly, some take hours. All this flows into the statistic of the sending server. I wrote a small program that's the other way around. Instead of measuring the sending-server's ping, it associates each measurement with an echo-bot (instead of the sending server), so this plot shows which maubots are the fastest to respond (it's probably not accurate at all, because most of the pings were started on conduit servers this week, but it's interesting nevertheless):

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Open Tech Will Save Us #6

From the schedule:

• Ag3m, from La Quadrature du Net joins to present "Some thoughts on moderation and censorship in a decentralised world".
• Sean DuBois (Sean-Der), WebRTC-knower and author of WebRTC for the Curious discusses his recent work in the space. https://webrtcforthecurious.com
• Damir Jelić presents the latest news on the Matrix Rust SDK

Open Tech Will Save Us is also available in audio form as a podcast in all the usual places!

Dept of Status of Matrix 🌡️

GSOC project reports

We had a bumper summer this year, with SIX students successfully completing their projects. So far three project reports have been published:

• GSOC report: E2E encryption for go-neb
• GSOC report: Adding Features in End-to-End encryption for Nheko-Reborn
• GSOC report: Enabling E2EE in Opsdroid Matrix Connector

Expect to see the next three reports next week.

Linköping University

Alexander Olofsson announced:

Since we suddenly have multiple projects that've separately gone on to trial our new and shiny Matrix-on-location for multiple respective use-cases, from conferences to anonymous - encrypted - support chats, we've had to end our soft-launch period very quickly.

Oh dear.

For minutes, confusion reigned in the chat about just what this meant! Fortunately, clarity arrived:

To clarify; Ending the soft-launch period means that we've now gone on to a full launch, with public announcements and everything else that includes

\o/

Dept of Spec 📜

anoa told us:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Closed MSCs:

• [WIPish] MSC1776: Implementing peeking via /sync

Merged MSCs:

• No MSCs were merged this week.

MSCs in Final Comment Period:

• No MSCs are in FCP.

New MSCs:

• MSC2771: Bookmarks
• MSC2772: Notifications for Jitsi Calls

Spec Core Team

In terms of Spec Core Team MSC focus for this week, the widgets spec and related MSCs are still being completed, however is formally removed from the focus list for this week. In its place we've put something in a very similar vein: MSC1960 (OpenID Connect exchange for widgets).

We had another long overdue Spec Core Team retro this week. On the whole it felt very productive, and we've hopefully helped identify some key issues and actions going forward to help internally organise the way the team spends time on MSCs (read: unblocking review progress from the team).

We're still working out our exact strategy asynchronously at the moment, so expect more details to follow soon.

Dept of Servers 🏢

Synapse

Neil reported:

This week we put out a new release candidate for Synapse 1.20.0, highlights include shadow ban support, more async/awaiting and unread message counts being added to sync responses to support upcoming notifications work.

Aside from that Erik has been continuing to get event persistence sharding ready for shipping. He already has a working version, but currently all workers will run at the speed for the slowest instance. Once that is fixed we will put it on matrix.org to see how it performs.

Patrick has finished up on asyncing all the things, and is now studying flame graphs to figure out where all the cycles on the main process are going.

Andrew has been taking a look at the knocking MSC, expecting concrete progress in the next few weeks.

Brendan has been working notifications support and is currently in VoIP land helping Dave on trying to level up our VoIP support.

Finally Oliver has dusted off his TURN tester from last Summer to get it working before he returns to university. The idea is that it will act a bit like the federation tester for but for TURN configuration.

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo announced:

Hi everyone, Devin R and I are working on state resolution, hopefully we have something exciting to show you next week. I also worked on the remaining key backup endpoints, so for example deleting backups works as expected now. Thanks to everyone who supports me on Liberapay or Bitcoin!

Dendrite

Dendrite is a next-generation homeserver written in Go

kegan told us:

This week we have been continuing work on the beta task list:

• The current state server has been removed.

• Joining rooms over federation is now more reliable as a single faulty event will no longer fail the request.

• Support for database migrations has been added.

• Various bug fixes to reduce the number of database is locked errors on SQLite.

• Experimental support for peeking has been added - See MSC2753

• DENDRITE_TRACE_SQL will now additionally trace unsafe goroutine writes that do not use ExclusiveWriter.

• The README has been updated with hardware requirements.

Spec compliance remains the same this week:

• Client-Server APIs: 56%, same as last week.

• Server-Server APIs: 71%, same as last week.

Dept of Clients 📱

Hydrogen

It's been a big week for Bruno:

Released 0.0.36 with end-to-end encryption enabled! 🎉
Most of this week was spent on performance improvements for IE11, where asm.js runs quite slow. Decryption is run in parallel on 4 workers in that browser, giving a massive speed-up and added responsiveness (the UI would freeze for 7 seconds when opening a room prior to this). There is still some polishing to do on sharing room keys, which I will get to next week, but all in all, it should work well. Next week I also want to get started on supporting key backup, so you can also read your encrypted history from before you logged in.

Get the latest: https://github.com/vector-im/hydrogen-web/

Element Web

Neil (same fellow from earlier) reported:

Element 1.7.6-rc.1 is now available at https://staging.element.io:

• Redesigned right panel where top right actions are now revealed via the i icon
• Widgets can now be opened in the right panel
• Widgets won't be visible in the Apps Drawer (top of timeline) by default (except Jitsi) - you need to pin them from the right panel
• Widgets can now be resized

Coming up:

• Continue work on deferred cross-signing setup
• Our new Matrix.to should get it’s first release
• General bug fixes around threepid invites

Element for Nextcloud

Gary Kim said:

Element for Nextcloud (formerly Riot Chat for Nextcloud) has released v0.6.5 and v0.6.6. In this update, Element Web was updated to v1.7.5 and support for adding a custom integration server in the config through the settings UI and support for Nextcloud 20 was added.

Join the development Matrix room at #elementfornextcloud-general:garykim.dev.

Check out the source code here.

Element Android 1.0.6

benoit told us:

We have released Element Android 1.0.6. Now we are working on sync mode for F-Droid version of the application (i.e. without FCM). We are also improving the experience with 1-1 calls.

SchildiChat for Android

SpiritCroc said:

Element v1.0.6 has been merged into SchildiChat.

Furthermore, following Schildi-specific fixes and improvements are included in the latest update:

• Fixed message bubbles clipping italic text

• Less wasted bubble space in some scenarios

• Deleted messages are now hidden by default in the chat overview

• Display the year in the chat list for old chats

• The inclusion of chats without notification in the overview's unread counter, which I added last week, is now optional

To be clear, Element v1.0.6 has been upstreamed into SchildiChat, the projects have not merged.

Element-iOS

Manu told us:

This week, we released 1.0.10 which provides PIN protection and the return of the incoming native call screen. Then, we started to update room creation flows with new icons and behaviors for creation buttons and a new screen for room creation.

Maunium sticker picker

This isn't strictly a client, but it's in the clients section.

Tulir announced:

Getting stuff into the spec or even implemented in Elements is very slow, so while we wait for proper native sticker packs and pickers (i.e. not in an iframe/webview), I decided to make a better sticker picker widget: https://github.com/maunium/stickerpicker

The goal is to be as fast and simple as possible. The picker and sticker packs are just some static files that can be served anywhere (no compilation required). It doesn't require self-hosting anything except said static files, and it has been confirmed to work on all three Elements.

There's a Telegram import script that's used to reupload stickers to Matrix and generate the static sticker pack files that the picker widget uses. It's technically also possible to make the sticker pack files yourself, but there's no easy way to do that yet.

Due to the simplicity of the picker there's no authentication, which also means everyone sees the same sticker packs. One way to get around that limitation is to have a unique URL for each user. I'll probably write a bot to manage copies of the picker for multiple users in the future.

Other future improvements include remembering frequently used stickers (surprisingly, all three Elements keep localStorage for widgets), importing Telegram animated stickers and searching for stickers by emoji/name

later more words appeared from Tulir:

Since the initial announcement, my sticker picker has received some more features:

• Added a slider to configure the number of stickers shown per row

• Frequently used stickers now show up at the top

• Imported all the packs from Scalar (the default integration manager)

A model of productivity, Tulir also announced:

my sticker picker has dark theme support too now

Dept of SDKs and Frameworks 🧰

Ruby

Ananace said:

Just bumped the Matrix Ruby SDK to version 2.1.2, fixing another bug with server discovery as well as the fact that state events haven't been provided for the application using the SDK. Oops.

matrix-appservice-rs

Lieuwe announced:

matrix-appservice-rs got some love!

Though still a long way to go before it is comparable to the likes of matrix-appservice-node and matrix-appservice-bridge, I did improve some little things.

• Updated to the latest (git) version of Ruma, and only depend on ruma instead of all subcrates.

• Small memory improvements: using references in MappingId and shrinking the MappingDict when initializing.

• Some more documentation.

• Cleaned up the code a bit.

There is also now a specific room for this crate: #matrix-appservice-rs:lieuwe.xyz, feel free to ask for support or tell if you've suggestions.

quotient

kitsune said:

libQuotient 0.6.1 is out, with fixes for bugs found during the work on the next Quaternion release and also an optimisation in the way profiles of all the users are managed - noticeable when opening large rooms (like our lovely HQ). Quaternion 0.0.9.5 beta is coming real soon now (TM), with some big features being coded in and merged as I write this.

...and the release notes are here: https://github.com/quotient-im/libQuotient/releases/tag/0.6.1

Dept of Ops 🛠

mnotify

stefan said:

I created mnotify which is a CLI based matrix client for automation tasks in shell scripts. Currently it supports login (via password), logout, sending messages, simple syncing, "get"ting the synapse admin api, creating a room, inviting users to a room, and joining a room.

Dept of Services 🚀

publiclist.anchel.nl

Mr. Wimpy offered:

https://publiclist.anchel.nl has been updated. The publiclist contains several public homeservers who can be used freely. Also listed some communities and roomdirectories which can be found at homeservers, last but not least public useable bridges can be found. The homeserver list contains now also uptime percentage and ip info. There is also a json that can be used by client builders.

Dept of Bots 🤖

Bots galore

Alexander Olofsson reported:

And in completely unrelated news, I also just tagged version 1.0.0 of an invite bot for Matrix, which helps in doing bulk invites to communities and community-associated rooms.

A single bulk-invite (MXID or 3PID) to a main room will be propagated to invites into the community (if wanted), as well as invites to all rooms linked to the community, once the invited user joins the main room.

Dept of Events and Talks 🗣️

Perth Linux user group Matrix presentation

PC-Admin offered:

I did a talk called 'Matrix in 2020' with the Perth Linux user group, parts of the video are missing but the audio is solid

I think we did a good job summarising how far Matrix has come, we also preview the idea our new Matrix hosting company!

https://www.youtube.com/watch?v=jTutFZzFizw

From the YouTube description:

Michael Collins talks about the Matrix, covering the new features, perthchat.org, and ChatOasis an upcoming FOSS Matrix hosting company.

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!