This Week in Matrix 2020-09-04

2020-09-04 — This Week in Matrix — Ben Parsons

Matrix Live 🎙

  • Kegan discusses Complement, testing suite for Homeservers
  • Steve from iOS: FaceID and PIN protection
  • ChristianP shows "Easy bridge creation with matrix-appservice-bridge"
  • Bruno demos Hydrogen e2ee features

Dept of Status of Matrix 🌡️

Next step in world domination

Alexander Olofsson reported:

The Matrix service at Linköping University has now passed its soft launch state and headed straight into full and proper deployment territory. (Though we're still not being overly vocal about it, don't want to crash our deployment on day one after all.)

They're simply announcing it on TWIM, perhaps the most high-profile place in the world!

We're doing a little bit of advertising of it to computer clubs and the like though, to slowly build up the users.

But then:

And just to add another slight update: already we have at least one conference planning on trying out the new Matrix server as well, so the user count might just end up exploding soon anyway.

\o/

Dept of Spec 📜

sorunome asks for attention on MSC2545:

As a lot of people in TWIM were originally interested in it, soru just wanted to say that MSC2545 (emotes) is no longer WIP, it is ready for review

Spec

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus for this week, we're sticking with the widget theme, but also adding something new. Travis is writing up a mahoosive widgets PR in addition to some widget MSCs. We're also adding on MSC2753 (peeking /sync), MSC2444 (peeking federation), which will both unlock some very exciting upcoming features (such as x-as-rooms)!

2020-09-04-gKF3z-stacked_area_chart.png

Dept of Servers 🏢

Synapse

anoa said:

Erik has made an initial, naive implementation of a sharded Event Persister worker - the non-sharded version currently being one of the biggest bottlenecks on matrix.org right now. It still needs a bit of polish to be production ready, and Sytest will need some updating to support testing it, but otherwise progress is looking good!

Brendan is polishing his unread counts implementations, while Rich is writing up the push rules replacement tech in an MSC. Rich has also been optimizing the device code in Synapse to both simplify the code, as well as cut down on unnecessary device data being sent over federation.

Patrick has finished converting Synapse's database code to be fully async! This has the benefit of making profiling the matrix.org Synapse process much more sane, and we're already starting to find a few little places where Synapse is burning CPU unnecessarily! Now, Synapse won't be fully async'd until Twisted itself is, and we drop support for older Twisted versions that aren't, which is not soon. However the majority of the code that developers touch and that is executed being async has massive benefits either way 🙂

Andrew has been working on having a confirmation pop up after users click the link in their email, to help prevent phishing scams. He's also trying to finally make Synapse spec-compliant in accepting identifier dictionaries during user-interactive authentication.

Olivier has been continuing to help Synapse recover missed messages better after it has been offline for an extended period of time, as well as various fixes to push.

Expect a fresh release with all these changes soon!

As someone who had my own server outage this week I'm looking forward to seeing this!

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander said:

This week we have been continuing on our bug-hunt mission and have been working on refactoring some significant parts of the codebase to simplify the API surfaces and avoid race conditions.

Changes this week include:

  • The current state server is in the process of being deprecated, with this functionality being folded into the roomserver

  • Support for rate limiting on certain CS API endpoints has been added

  • Support for password changes has been added, allowing users to change their own password

  • /sync no longer incorrectly lists some rooms in the leave section, which caused problems with Element Web and other clients

  • Roomserver input events are now handled in FIFO order with per-room workers

  • Roomserver input, query and perform packages are now split up nicely

  • The federation sender and keyserver now delay some startup tasks to allow the HTTP listeners to start more quickly

  • The federation sender is now used for backfilling and getting missing events

  • Some more SQLite writer bugs have been fixed

  • Room information in the roomserver is now handled in a common structure

  • The public rooms API now correctly searches remote servers (thanks rohitmohan96)

  • Initial support for Peeking (MSC2753) and Peeking over Federation (MSC2444) is in progress

Spec compliance is improving:

  • Client-Server APIs: 56%, up from 55% last week

  • Server-Server APIs: 71%, same as last week

Last week, we updated some of our documentation and created a whole new set of easy first issues and medium to hard issues for contributors.

If contributing to Dendrite sounds like something you would be interested in, please take a look at these issues and join us in #dendrite-dev:matrix.org! There's also #dendrite:matrix.org for general Dendrite chat and updates.

Synapse Deployment 📥️

YunoHost

Pierre announced:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.18.0 (1.19.1 available in branch testing)

Element Web integration had been updated to 1.7.4 (1.7.5 available in branch testing)

Dept of Bridges 🌉

tomsg-matrix

Lieuwe told us:

Introducing a new bridge to the family, tomsg-matrix!

A friend of mine preferred using his own chat protocol, but having a chat protocol is useless without having people to talk to. Since I prefer using Matrix I thought it would be a fun exercise to try and create a bridge using Rust.

The bridge uses the newly created (still very WIP) matrix-appservice-rs. A Rust library that builds on top of the Ruma suite, the goal is for it to be comparable to matrix-appservice-node and matrix-appservice-bridge.

The code for tomsg-matrix is open source, but be prepared, I hastily pushed it to GitHub without cleaning it up.

Come chat to us over at #tomsg:lieuwe.xyz, hoping that it doesn't break. :P

Making your own chat protocol in a world of well established options seems, frankly- oh right.

matrix-sms-bridge

Benedict told us:

We are currently working on an Android-App to use an old smartphone as SMS-Gateway for matrix-sms-bridge, because using gammu in production (>50 SMS/day) seems not to be very reliable with our USB-modem.

im_sender_service, for zabbix bridging and more

progserega said:

I create service for sending corporate information: some service (for example zabbix) can send message to this service (over http-api) and service will create room (if needed), invite user and send message to this room. Service will change status of message in his database to readed when user will read this message. Also service can send any changes statuses of this message by callback url. Editing old messages also supported: for example zabbix can send UID problem to this service and it is will edit old "problem message" to "problem resolved".

Url: https://github.com/progserega/im_sender_service

Dept of Clients 📱

Ditto

Ditto is a Matrix client for iOS & Android

Annie offered:

Ditto development is back from hiatus!

I'm currently in the process of rebuilding Ditto - partly to incorporate rn-matrix, and partly to support a WEB VERSION

The new repo can be found here (https://gitlab.com/ditto-chat/ditto) (there's a new repo mostly because getting react-native-web to play nicely with the rest of the app was a challenge, to say the least)

As a sneak peek, enjoy this rough version (https://ditto-test.netlify.app) of Ditto for web :)

Expect to see what I'm calling Ditto's 2.0.0 drop on TestFlight and Google Play in a few weeks.

Links:

Matrix Room

Big exciting things happening!

Hydrogen

Bruno offered:

Good progress this week, I just got end-to-end encryption working. The code still needs some cleanup and polishing to be able to release it in the wild, but expect a release somewhere next week with E2EE enabled.

YESSSS - see Matrix Live for a demo.

2020-09-04-t4hyh-image.png

gomuks

gomuks is a terminal based Matrix client written in Go. Source on GitHub

Tulir said:

gomuks now supports importing and exporting message decryption keys. It even supports exporting the keys for a specific room, e.g. for sharing them with someone else. To help with sharing exported key files with someone else, gomuks also got support for uploading media.

Next I'll try to figure out how nikofil's cross-signing/SSSS PR for mautrix-go works and use that in gomuks.

v0.2.0 was just released with these features!

NovaChat

eric told us:

NovaChat is a new Matrix-based desktop client that aggregates all your chat networks into one app. We now have integrated bridges for Whatsapp, Twitter, Telegram, Hangouts, Slack, Messenger, Android Messages (SMS), Skype, Discord, Instagram and IRC.

Sept 4 Updates:

  • Continuous UI improvements. Everything is much denser now. See screenshot below.

  • New bridge alert: tulir just completed an Android Messages SMS bridge (which is a total hack but it works!) for NovaChat.

  • Added Discord Bridge. Thanks for building this Sorunome!

In the works...

  • Still working on our iMessage bridge, using jailbroken iPhone 4s

  • Marking messages as 'unread'

Sign up for the NovaChat beta. Or send me a DM @eric:nova.chat!

2020-09-04-iEKfJ-Screenshot-20200904072010-980x1069.png

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) announced:

  • Lurkki added an emoji completer to the message area! This was a super requested feature and it is pretty awesome, that someone picked this up and implemented it! This means you can finally write :fire: and have it expand to 🔥! Seems like this will also lead to a few more improvements to our other completers down the line (user, room, etc completions), so thank you very much Lurkki!

  • I've been mostly reviewing PRs this week (and trying to get some other work in line, like MSCs, synapse stuff, etc). With this hopefully Chethans work on cross-signing will be merged soon, which will give us most of the building blocks to have cross-signing in Nheko (his work is basically only missing Secure Secret Storage and Sharing). We hope to fix the few remaining build issues in the next week or so.

  • On the side I have been cleaning some UI stuff to simplify our room settings dialog and allow advanced users to hide/show certain event types in the timeline. This will probably take a bit more time to be finished though.

2020-09-04-KUPpB-clipboard.png

Fluffychat

sorunome offered:

Fluffychat 0.17.0 is released!

IMPORTANT: Due to legal reasons (USA cryptography export), end-to-end encryption has been disabled in the google play and ios builds for now. They will be re-enabled once we got all that figured out.

The F-Droid builds still have e2ee. As the play store and fdroid builds use the same build signature, you can simply install the fdroid version over the other one. More information on that here: https://fluffychat.im/en/fdroid.html

Features

  • Pin and unpin chats

  • Implement event aggregations

  • Implement message edits

  • Render reactions

  • Add / Remove reactions by tapping on existing reactions

Fixes:

  • Don't re-render the room list nearly as often, increasing performance

  • Various fixes for sending messages on bad networks

  • Design tweeks and fixes

  • Various performance fixes and improvements

Definitely hoping to see these issues resolved.

SchildiChat for Android

SpiritCroc offered:

I removed SchildiChat from the Google Play for now, due to the complications of providing encryption-capable software from US servers. Huge thanks to the FluffyChat team for providing insight into their investigations concerning the US export regulations!

On the positive side, SchildiChat is now finally available on F-Droid from the official repos.

Furthermore, I have set up my own F-Droid repository, where you can get updates that are compatible with the Google Play Store variant and which support push notifications.

On the implementation side, I can report the following:

  • Element 1.0.5 has been merged

  • The "low priority" category now remembers its expand/collapse-state across restarts

  • New setting to hide member state changes and reactions from the chat overview

  • The category unread counter badge now also displays the number of chats with new messages in case none of these has notifications enabled

Element-iOS

Manu announced:

This week, we should have fixed the background crash that was happening in TestFlight 1.0.7. TestFlight 1.0.9 has been submitted yesterday.

In parallel, we started to modernise the AppDelegate class, the entry point of the app. This is the very first step of several improvements we want to do (this is not only to rewrite it in Swift).

Element Android

benoit reported:

We have just merged a few big PRs to develop: add emails and phone numbers to Matrix account, rework of upload attachment management, and other stuff related to 1-1 calls. It will live on develop for the week-end and we will prepare the release v1.0.6 at the beginning of next week.

Dept of SDKs and Frameworks 🧰

rn-matrix

rn-matrix is a React Native SDK for matrix

Annie offered:

Built on the matrix-js-sdk, this NPM package will help any React Native developer drop matrix into their app.

Been working on this since the beginning of the summer - this package is split into two, one for the data and one with default UI components for those who want to drop chat into their app quickly (not required).

Would love feedback on docs, installation, and usage, so try it out and drop me a line!

Links: Docs (start here!) | Matrix Room

NB, this update should be read together with the Ditto update above

Ruma

iinuwa said:

Over the past couple of weeks, jplatte has been hard at work making sure we can introduce new fields to request and response types as non-breaking updates. Thanks to @nicholaswyoung, consumers can opt-in to breaking changes to ensure compliance with the spec on each update.

Besides that, we implemented another endpoint for the Federation API, bringing us up to 20/31 endpoints implemented.

New Contributors

Dept of Ops 🛠

matrix-docker-ansible-deploy

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi reported:

matrix-docker-ansible-deploy can now help you set up matrix-registration - an application that lets you keep your Matrix server's registration private, but still allow certain users (those having a unique registration link) to register by themselves.

See our Setting up matrix-registration documentation page to get started.

Dept of Events and Talks 🗣️

Matrix talk @ FrOSCon 2020 (in German)

Oleg reported:

I did a talk on Matrix bridges at the annual Free Open Source Conference.

Apparently

The interest was high. Some didn't know, that bridges exist in Matrix.

!!! Well thank you Oleg for letting them know.

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1fairydust.space376
2neko.dev444.5
3tchncs.de565.5
4autodie.be958
5matrix.vgorcum.com1124
6maescool.be1435
7lermer.nl1989
8utzutzutz.net2207
9bau-ha.us2965
10rollyourown.xyz3013

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!