This Week in Matrix 2020-05-15
15.05.2020 00:00 — This Week in Matrix — Ben Parsons🔗Open Tech Will Save Us 🎙
Second edition of OTWSU took place this week! Chance to watch it if you haven't already, or (pro-tip), watch it again.
- Mike Hoye from Mozilla talks about "Verbs in the fediverse"
- Eugen (@[email protected]) discusses "how we keep mastodon.social running at scale" as a Q&A
- ... and a new VIP tour of Dendrite, the next-gen Matrix Homeserver, from our own neilalexander and Kegan!
🔗Dept of Status of Matrix 🌡
🔗Bundeswehr switching to Matrix
Oleg said:
German army reported in December 2019 their intentions for Matrix.
Now they decided to replace Stashcat with Matrix in this autumn.
Around 50'000 account are planned.
Riot will be used as a Matrix client. The E2EE by default and device Cross-Signing will be extensively used.
Full article (in German): https://www.heise.de/newsticker/meldung/Bundeswehr-setzt-kuenftig-auf-Matrix-als-Messenger-4719474.html
🔗Mozilla are happy with their deployment!
About a year after this tweet, four months after deployment of our open source @matrixdotorg instance, community participation in the Mozilla project has increased from 4x to 10x across all the projects that previously relied on IRC.https://t.co/E5HRsfHkzk
— mhoye (@mhoye) May 12, 2020
For more from Mike check out the OTSWU recording above, he's on first.
🔗Dept of Spec 📜
🔗Spec
anoa announced:
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
🔗MSC Status
Merged MSCs:
- No MSCs were merged this week.
MSCs in Final Comment Period:
New MSCs:
🔗Spec Core Team
In terms of Spec Core Team MSC focus for this week, MSC2526 has entered FCP. We'd like to keep the E2E MSCs in flight in focus, but are also conscious that room v6 is coming up. We're sticking the MSCs for the latter alongside the existing E2E ones in the focus queue, for a total of 5. Those MSCs are: MSC2366 (verification flows), MSC2399 (UISI messaging), MSC2540 (canonical JSON validation), MSC2209 (notifications auth rules), and MSC2240 (room version 6).
🔗Dept of Servers 🏢
🔗Dendrite / gomatrixserverlib
Dendrite is a next-generation homeserver written in Go
Neil Alexander announced:
Dendrite now implements
/get_missing_events, which improves inbound federation reliability significantlyState resolution v2 implementation in gomatrixserverlib now uses significantly less overall memory and performs far fewer allocations
A skeleton keyserver component has been added - it returns no keys for now but that is enough to satisfy Riot when creating DMs
Canonical alias is now set if needed when the room is created
Reduced the number of HTTP 500s returned by the media API
Sync API topology logic and tokens have been reworked
Server keys are now cached in memory using an LRU cache, regardless of database or fetchers used
Federated joins now attempt all listed servers more reliably
Federated joins in response to an incoming invite should now work, particularly in cases where the room had previously been joined and left
Some Prometheus statistics are now available for the LRU caches
Sync API storage layer has been deduplicated quite a bit
Display name and avatar URL are now sent in join content again after a regression
Fixed a bug in the membership updater when matching added and removed state
Fixed a bug where a response to a federated device query would not contain all required fields
Fixed a bug where
/synccould panic after joining a room
Get a real demo of Dendrite from Open Tech Will Save Us #2
🔗Conduit
Conduit is a Matrix homeserver written in Rust https://conduit.rs
timo said:
This week I worked on presence and end to end encryption, but they are not working yet.
Meanwhile, Valkum got sytest integration working! This means each time a new commit is pushed to a branch, CI automatically picks up on that and lets a lot of tests run on the homeserver - the same tests that synapse uses. Currently we have 8% of all tests positive on Conduit.
Client-Server APIs: 8% (56/667 tests) *------------------ Registration : 53% (17/32 tests) Login : 26% (5/19 tests) V1 CS APIs : 10% (3/30 tests) Misc CS APIs : 100% (1/1 tests) Profile : 33% (2/6 tests) Devices : 17% (2/12 tests) Presence : 0% (0/10 tests) Create Room : 31% (4/13 tests) Sync API : 1% (1/85 tests) Room Membership : 0% (0/15 tests) Room State APIs : 0% (0/14 tests) Public Room APIs : 0% (0/7 tests) Room Aliases : 0% (0/15 tests)At the same time, MTRNord tried to integrate opentracing into Conduit and Paul worked on a docker file to make installation easier.
Thanks to my supporters on Liberapay.
See the bottom of this post for some more interesting numbers!
🔗Synapse
anoa offered:
Synapse v1.13.0rc2 shipped this week with the following notable improvements:
SSO support in User-Interactive Authentication sessions
Cross-signing device list bug fixes
The ability to run in worker mode using redis as a backend (which will soon be considered the default transport for Synapse workers)
Some admin api improvements and additions
...as well as all the usual bug fixes and especially performance improvements. See the changelog for the full details and please test if you can! A full release is expected early next week.
Speaking of next week: work will involve getting room version 6 ready to ship, alongside even more performance work - especially moving event processing off the master process (the code for which is already running successfully on Half-Shot's own homeserver)!
🔗Dept of Bridges 🌉
🔗mx-puppet-steam
Icewind told us:
I've been working on a new puppet bridge for steam chat mx-puppet-steam, while I would still consider it to be in alpha state, most of the things needed for day to day chatting are functional.
It bridges both one on one and group messages, shows the proper name, avatar and online status of your frends and handles embedded images in one on one chats.
I have a docker image for it if that helps: https://hub.docker.com/r/icewind1991/mx-puppet-steam
🔗mautrix-whatsapp
Tulir reported:
As promised last week, mautrix-whatsapp now has end-to-bridge encryption (currently on the
e2bebranch, will be merged to master soon™). It works mostly the same way as the other bridges.
🔗Dept of Clients 📱
It's a big, big week for client news! We must take a deeper look at some of these projects soon!
🔗Ditto v0.4.1 Released!
Annie offered:
🔗What is Ditto?
Ditto is a mobile client for iOS and Android! It's available on TestFlight and Google Play (beta) - download link available on the website below.
🔗What's New?
🔗Added
Reactions support! 😄🎉
Add a visual hint if a message is sending or could not be sent
Send pending messages when the connection to the homeserver is back
🔗Changed
Update to React Native 0.62.2
Major performance improvements!
🔗Fixed
Redacted events have a proper message
Switched push gateway to https
🔗Links
🔗Matrix-TV
Half-Shot said:
Link: https://github.com/Half-Shot/matrix-tv
I got really bored in lockdown, as we all have. I stared at my TV, and it stared back at me. While it's definitely a privacy nightmare to start developing apps for these things, I couldn't help myself. I got stuck in, and secured myself a LG Developer Account.
The LG set of smart TVs use WebOS, as like the name suggests, runs apps in it's special variant of Chromium. I was lucky to be running a recent OS and had access to Chromium 53! And NodeJS 0.12!! Sarcasm aside, there are some very troubling versions of WebOS with much older versions of the browser, so I was happy to be developing against something with strong ES6 support. Despite the absolutely awful onboarding experience, the WebOS SDK itself was pretty good.
matrix-tvwas written to be a general large screen matrix client, so it talks to the TV's own libraries through an abstraction layer :)Anyway, this client is the culmination of a lazy Sunday afternoon's work. It features very basic support for viewing a room, but I hope to expand it at some point. It supports running in a web browser and the aforementioned OS, but support for more variants is easy to add. I'm looking into how to design for the large screen beyond making fonts bigger, because the unique constraints on memory and UI are fun challenge.
🔗Nheko
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
Nico (@deepbluev7:neko.dev) announced:
Nheko now supports SSO. If your server uses SSO, you may want to test that, if it works for you, but I did successfully sign into the Mozilla homeserver.
We improved hinting on the login and registration pages, so it should be easier to enter correct credentials.
Replies now have a max height by default. If you want to see the full reply, you need to click on it to navigate to the reply.
Nheko now has a red unlocked padlock, when an unencrypted message is sent in an encrypted room.
red_sky has been hacking along on reactions. Displaying them is now in master, for sending he is currently reworking the emoji picker, so that it can be attached to the timeline more easily and it is easier to search, etc.
Nheko has it's own weblate btw, if anyone feels like translating a bit of stuff: https://weblate.nheko.im/engage/nheko/?utm_source=widget
🔗Pattle: version 0.16.0
Pattle is a client written for Flutter using Dart
Wilko said:
A new version has been made available and will become available on Google Play and TestFlight!
Because of corona this might take longer than usual.
🔗Changes
Huge performance improvements! All SDK processing now happens in a different isolate (comparable to a thread)!
Also huge start up time improvement! When opening the app you should see your chats in less than or equal to a second! (will vary per device)
You'll now see two checkmarks when a message is read!
Add Dutch translations! (thanks to Louis!)
You can logout now!
An error along with a stack trace is now shown when the initial sync fails. You can use it to inform me and make an issue!
For the launcher icons vectors are now used on Android when it's supported
Contributing is now easier, you don't have to sign a CLA anymore!
🔗Get Pattle
- Android
- iOS (TestFlight)
🔗Issues or suggestions
If you stumble upon any issues or would like to suggest features, please do so here!
You can login via GitHub and Gitlab.com!
🔗Follow development
Follow development in #app:pattle.im!
🔗RiotX
benoit reported:
We are making progress on Identity server feature implementation (PR in review). Also we are working to integrated call with WebRTC, integration manager (widget, Jitsi, sticker) and we are stabilizing the application, especially on the cross-signing part.
Expect a new corrective release at the end of the day!
🔗gomuks
gomuks is a terminal based Matrix client written in Go. Source on GitHub
Tulir offered:
I was able to solve a few annoying bugs like message history disappearing and switching rooms taking time (it turned out that read receipts were being sent synchronously), so I decided to release gomuks v0.1.0.
🔗fluffychat
krille said:
FluffyChat 0.13.1 is out with:
New status feature
HTML rendering of messages
Markdown support
Enhanced chat list design
New translations (Polish, Hungarian)
Lots of minor fixes and refactoring
🔗Nio
Nio is a client written with SwiftUI
kilian told us:
Since the last updates, there's been a few cool new features and changes added to Nio:
message text field supports multiple lines of input
edit badges instead of "(edited)"
markdown message rendering
spanish translation
blurhash support for incoming images (so find some Nheko-using friends! 😄)
sending images
and a few minor and internal fixes
Join us in #niochat:matrix.org for discussion and more fun 😊
🔗Riot-Android 0.9.11
benoit reported:
Riot-Android: Version 0.9.11 (with Matrix SDK 0.9.34) has been released on the PlayStore, and should be available for all during the week-end. It contains fixes on the identity server feature (the "missing token" thing), and also support Jitsi call with configured domain.
🔗Riot-iOS
Manu offered:
On the cross-signing side, we have been working on the implementation of SSSS (to recover all private keys from the HS) and on UI polishments.
On the maintenance side, the new push mechanism from Apple has been implemented but it needs more work to work properly.
🔗Riot integration with NextCloud
Riot-web integration with NextCloud exists! Source at https://github.com/gary-kim/riotchat.
🔗Dept of SDKs and Frameworks 🧰
🔗matrix-spring-boot-sdk
Benedict offered:
This SDK is now on maven central. I also introduced support for implementing Application Services. The project can be found here: https://github.com/benkuly/matrix-spring-boot-sdk . It will be used by Matrix-SMS-Bridge, which only needs one telephone number. The latter will be released next week, when everything works fine (https://github.com/benkuly/matrix-sms-bridge).
🔗Matrix Dart SDK: v0.31.0
Wilko said:
Add the ability to defer all processing to a different isolate
When you use
isolated: trueonHomeserver.loginorMyUser.fromStore, everything will be handled for you and the SDK works the same.This only works on platforms with the Dart VM (not web).
Updaters are now globally registered perMyUserid.This means for now only one
UpdaterperMyUseris allowed.Updaters are created when aMyUserinstance is retrieved viaHomeserver.loginorMyUser.fromStoreAdd ability to specify what rooms to load from store when retrieving user
The
previousBatchparam fromRoom.copyWithis removed (wasn't used)Add ability to override context in
MyUser.copyWithandMyUser.merge
UpdaterandHomeserverare less coupled nowWhen a
Contextis changed withcopyWith, all child objects'contexts are also changedFix bug in read receipt parsing (thanks to MatMaul!)
Sync errors are now pushed to the
updatesstreamFix access token not being passed to logout API
Add
typetoRequestUpdate, so you can see which type of callthe update was caused by
Moor has been updated
And other minor fixes!
🔗Dept of Bots 🤖
🔗JupyBot - First proof of concept
rdi said:
🔗What is JupyBot
The purpose of the bot is to provide a link with Jupyter kernels running either on your local computer or somewhere else on a server. You can access your kernels through the bot with the chat interface. The kernels will answer to your commands through the same interface.
Since this is still a proof of concepts, the number of features is limited (command line text, string and pictures are the only output formats supported. No widget, no funny javascript) and they are few safeguards against infinite loops and other entertaining bugs.
🔗TODO
This is right now just a proof of concept. We want to see if such a bot can be useful for people and how. What features would be interesting, what can be integrated in the present clients,... Therefore, in the short term, we want to focus on the possibility of easy testing for users and easy contributions for people who want to add their ideas. On the mid- and long term, a lot of ideas can be realized; it will depend on the feedback. These are just some examples. Short term:
proper error catching
fallback for non supported Jupyter formats on riot.im (i.e. a lot)
testing on termux on a tablet or smartphone
solve the problem with nio-matrix and upload in chunks of images.
Dockerized version
Middle term:
- use of opsdroid as a framework
Longer term:
local server integrated to locally command the bot service.
make full use of jupyter capabilities with a client suited for it (for instance a Jupyterlab plugin)
🔗Links
room for testing: #jupybot1:matrix.alkemata.com
room for discussing: #jupybot:matrix.alkemata.com
github repository: https://github.com/albanatita/JupyBot
🔗snowstorm-matrix
lub reported:
I made a little bot that scrapes https://news.blizzard.com/en-us , because they've disabled their RSS feeds for some time now. It regularly scrapes all posts of a configured category and sends messages for new articles to all rooms the bot is a member of. It works like a RSS bot, except the source is a website instead of a RSS feed.
In the readme there are also some public instances, which you can just invite instead of hosting your own bot.
🔗midori jabbot matrix.org
Torrey Searle told us:
I've added matrix support into a chat bot, you can see the chat bot in action at #midori-dev:hackerspaces.be and it's project page is here https://github.com/midoricorp/jabbot.
it's main highlight is anybody can add new commands to it in chat using its scripting language
any any existing bot command can be used as a function in new scripts you write
The scripting language to add new commands can be found here https://github.com/midoricorp/script
you can do things like
!script print "HELLO world";
or make a new command
!script sub hello { print "HELLO World"; }
that would result in a new command !hello being made
print also has 2 output stream, one for rich format and one for basic formatting
!script print "Hello World!"; print HTML "Hello <b>World</b>";
🔗Dept of Events and Talks 🗣
Matrix will be featured in a Rust London meetup on the 27th May 2020.
🔗New Public Rooms 🏟
Ludwig offered:
Hey, I have created a room for designers working on matrix stuff to join: #matrix-design:disko.media
The goal is to improve cohesion and collaboration for all things design (UI, UX, interaction...) Over time, it should become a place for people to
ask design-specific questions
share cool stuff and be praised for all the good work you're doing
develop and document a common understanding of the semantics of matrix
make sure the user experience is amazing, even as we all figure out how to federate. Probably most designers work on single clients right now, but there are important UX questions that can only be answered ecosystem-wide. Maybe we'll end up with a design guide for people to use alongside the spec.
🔗Final Thoughts 💭
Let's test homeservers against sytest, then unfairly pit them against one another! This week we're comparing Conduit and Dendrite - let's see the results!
| Name | Conduit | Dendrite |
|---|---|---|
| Federation APIs | 1% (1/103 tests) | 27% (28/103 tests) |
| State APIs | 0% (0/13 tests) | 46% (6/13 tests) |
| Key API | 17% (1/6 tests) | 33% (2/6 tests) |
| Query API | 0% (0/5 tests) | 80% (4/5 tests) |
| send_join API | 0% (0/8 tests) | 38% (3/8 tests) |
| make_join API | 0% (0/2 tests) | 50% (1/2 tests) |
| Auth | 0% (0/19 tests) | 5% (1/19 tests) |
| room versions | 0% (0/7 tests) | 29% (2/7 tests) |
| Federation API | 0% (0/14 tests) | 36% (5/14 tests) |
| get_missing_events API | 0% (0/6 tests) | 33% (2/6 tests) |
| Backfill API | 0% (0/4 tests) | 0% (0/4 tests) |
| Invite API | 0% (0/10 tests) | 10% (1/10 tests) |
| send_leave API | 0% (0/1 tests) | 0% (0/1 tests) |
| Public Room API | 0% (0/1 tests) | 100% (1/1 tests) |
| Device Key APIs | 0% (0/7 tests) | 0% (0/7 tests) |
| Non-Spec API | 0% (0/50 tests) | 0% (0/50 tests) |
| Client-Server APIs | 8% (56/667 tests) | 33% (225/672 tests) |
| Registration | 53% (17/32 tests) | 62% (20/32 tests) |
| Login | 26% (5/19 tests) | 32% (6/19 tests) |
| V1 CS APIs | 10% (3/30 tests) | 10% (3/30 tests) |
| Misc CS APIs | 100% (1/1 tests) | 100% (1/1 tests) |
| Profile | 33% (2/6 tests) | 100% (6/6 tests) |
| Devices | 17% (2/12 tests) | 17% (2/12 tests) |
| Presence | 0% (0/10 tests) | 0% (0/10 tests) |
| Create Room | 31% (4/13 tests) | 92% (12/13 tests) |
| Sync API | 1% (1/85 tests) | 40% (34/85 tests) |
| Room Membership | 0% (0/15 tests) | 47% (7/15 tests) |
| Room State APIs | 0% (0/14 tests) | 36% (5/14 tests) |
| Public Room APIs | 0% (0/7 tests) | 14% (1/7 tests) |
| Room Aliases | 0% (0/15 tests) | 67% (10/15 tests) |
| Joining Rooms | 0% (0/8 tests) | 75% (6/8 tests) |
| Leaving Rooms | 0% (0/1 tests) | 100% (1/1 tests) |
| Inviting users to Rooms | 0% (0/14 tests) | 7% (1/14 tests) |
| Banning users | 0% (0/5 tests) | 20% (1/5 tests) |
| Sending events | 0% (0/3 tests) | 100% (3/3 tests) |
| Getting events for Rooms | 0% (0/10 tests) | 60% (6/10 tests) |
| Typing API | 0% (0/5 tests) | 0% (0/5 tests) |
| Receipts | 0% (0/4 tests) | 0% (0/4 tests) |
| Read markers | 0% (0/1 tests) | 100% (1/1 tests) |
| Media APIs | 10% (2/21 tests) | 24% (5/21 tests) |
| Capabilities API | 0% (0/2 tests) | 50% (1/2 tests) |
| Logout | 0% (0/4 tests) | 75% (3/4 tests) |
| Push APIs | 21% (12/58 tests) | 8% (5/59 tests) |
| Account APIs | 20% (2/10 tests) | 40% (4/10 tests) |
| Ephemeral Events | 0% (0/1 tests) | 0% (0/1 tests) |
| Power Levels | 0% (0/2 tests) | 83% (5/6 tests) |
| Redaction | 0% (0/5 tests) | 0% (0/5 tests) |
| Third-Party ID APIs | 16% (3/19 tests) | 16% (3/19 tests) |
| Guest APIs | 0% (0/35 tests) | 49% (17/35 tests) |
| Room Auth | 0% (0/25 tests) | 52% (13/25 tests) |
| Forget APIs | 0% (0/4 tests) | 25% (1/4 tests) |
| Context APIs | 0% (0/4 tests) | 0% (0/4 tests) |
| Room Upgrade APIs | 0% (0/21 tests) | 0% (0/21 tests) |
| Room Versions | 0% (0/40 tests) | 75% (30/40 tests) |
| Device Keys | 3% (1/37 tests) | 3% (1/37 tests) |
| Tagging APIs | 0% (0/8 tests) | 75% (6/8 tests) |
| Search APIs | 0% (0/6 tests) | 0% (0/6 tests) |
| OpenID API | 0% (0/3 tests) | 0% (0/3 tests) |
| Send-to-Device APIs | 0% (0/12 tests) | 0% (0/12 tests) |
| Server Admin API | 0% (0/1 tests) | 0% (0/1 tests) |
| Ignore Users | 0% (0/3 tests) | 0% (0/3 tests) |
| User Directory APIs | 0% (0/11 tests) | 0% (0/11 tests) |
| Application Services API | 4% (1/25 tests) | 20% (5/25 tests) |
🔗That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
.png)
