This Week in Matrix 2021-03-05

05.03.2021 19:45 โ€” This Week in Matrix โ€” Ben Parsons
Last update: 05.03.2021 19:23

Matrix Live ๐ŸŽ™

Selection of demos this week, including: Element Home, Hydrogen, Nheko and matrix-notepad.

Dept of Status of Matrix ๐ŸŒก๏ธ

Element Home

Element announced Element Home, a new way to get your own homeserver. Read more about it on their blog, or watch Matrix Live for a real demo.

Dept of Spec ๐Ÿ“œ

Spec

anoa said:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Spec Core Team

This week the Spec Core Team worked on MSC2858 (sso ID providers), MSC2403 (knocking) and MSC2746 (voip signalling), as well as held another retrospective of the past month. Discussion mainly focused on process and the new spec redesign. Speaking of which...

New Spec Platform

The new spec has launched! ๐ŸŽ‰ Check it out at https://spec.matrix.org/unstable! Release versions will be landing sometime soon. Bear with us as we finalise the new release process.

Huge thanks to wbamberg for working hard on bringing the idea to a reality. The Matrix.org foundation will continue maintenance of the platform and continue to make it even better for developers in the coming months.

We've already got a number of bug reports that need fixing. If you find anything that's a bit off with the redesign, please file an issue at https://github.com/matrix-org/matrix-doc. Thanks!

2021-03-05-LWU1H-stacked_area_chart.png

Dept of Servers ๐Ÿข

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timokoesters said:

  • Support for Jaeger to find performance bottlenecks

  • Improve initial sync ~2x

  • Implement /receipt

  • Use transaction ids properly when sending transactions

  • Improve sending code (less likely to get stuck)

  • Small improvements to README.md and https://conduit.rs

  • Updated milestone + lots of new issues: https://gitlab.com/famedly/conduit/-/milestones/3

  • Appservice support improvements

    • no longer query for aliases outside of their namespace

    • no longer send irrelevant events

  • Temporary fix for duplicate events (will be fixed properly with state res)

2021-03-05-c078w-image.png

Thanks to Famedly and everyone who supports me on Liberapay or Bitcoin!

matrix-media-repo

matrix-media-repo is a highly customizable multi-domain media repository for Matrix

TravisR told us:

v1.2.3 is out now with a highish priority fix if you have audio thumbnails enabled. Other goodies include an X-Robots-Tag header and some artifact repairs on gif and apng thumbnails. See https://github.com/turt2live/matrix-media-repo/releases/tag/v1.2.3 for the full changelog.

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander said:

Earlier this week we released Dendrite 0.3.11, and since then we have merged a significant number of appservice fixes to master which will go into the next release. With these, bridges should stand a much better chance of working with Dendrite.

Changes include:

  • Events are now properly sent to interested appservices

  • Appservice namespace exclusivity is now respected properly

  • Users starting with _ can now be registered by appservices correctly

  • Room alias queries for appservices have been improved and should work now

We also upgraded a number of dependencies.

Spec compliance has improved slightly:

  • Client-server APIs: 60%, up from 56% last week

  • Server-server APIs: 82%, up from 80% last week

As always, please feel free to join us in #dendrite:matrix.org for general Dendrite chat, and #dendrite-dev:matrix.org if you are interested in contributing!

Spec compliance is really progressing!

Synapse

Synapse is a popular homeserver written in Python.

callahad told us:

We've heard some feedback that Synapse's release cadenceโ€”an RC every other week, with releases in betweenโ€”can feel a bit overwhelming. In response, I'm going to try to write a little less about each RC and instead simply refer to the 1.29.0rc1 Release Notes from yesterday.

We'll shout a bit more when that matures into a formal release early next week. ๐Ÿ™‚

On non-release weeks I'll try to spend a little more time focusing on the behind-the-scenes aspects of developing Synapse in our TWiM updates. If there's anything in particular you'd like hear about, let me know!

Speaking of, we're experimenting with using GitHub Discussions for... discussions.

We don't want to go full-on mailing list, but it might be nice to have a place to for slower, asynchronous conversations that don't quite fit into the more rapid pace of a typical Matrix room.

Our first two discussion topics ask about:

Please do weigh in!

Dept of Bridges ๐ŸŒ‰

GroupMe bridge (matrix-groupme-go)

karmanyaahm offered:

I'm working on a GroupMe bridge I made based on tulir's mautrix-whatsapp and mautrix-go.

I aim to add features like double puppeting, end to bridge encryption, etc over time, but an alpha release is available now. I would love to get bug reports and testing info to fix them.

If anyone's running/testing this please join #groupme-go-bridge:malhotra.cc to chat about it

https://github.com/karmanyaahm/matrix-groupme-go

Another recent GroupMe bridge effort: https://gitlab.com/robintown/mx-puppet-groupme

matrix-appservice-bridge v2.6.0-rc1

Half-Shot announced:

A PSA this time around. Please test this release and update as soon as you can when we go to release on Monday next week. We've included a change to correct how users are registered against the homeserver to comply with the spec (Synapse accepts a /register call without a type=m.login.application, and versions 2.5.0 and below of the matrix-appservice-bridge sent these invalid requests). I'm hoping to eventually retire this behaviour from Synapse, so the more people that upgrade the better!

Also this PR is the Synapse-side change to remove this behaviour. If your appservice implementation uses this behaviour, please chime in.

matrix-appservice-irc hits v0.25.0(-rc1)

Half-Shot told us:

Hey folks, this release contains a bunch of tiny-but-useful bugfixes that you'd hate to miss out on. The full release comes next week but if you want to help us test early, come and get it while it's hot! This release includes the all important fixes from matrix-appservice-bridge v2.6.0-rc1 which fix a spec compliance issue.

Dept of Clients ๐Ÿ“ฑ

Hydrogen

A minimal Matrix chat client, focused on performance, offline functionality, and broad browser support. https://github.com/vector-im/hydrogen-web/

Bruno said:

Multiple releases this week with lots of bug fixes:

  • make catchup sync faster, a fix for the biggest slowdown

  • correct profile info on messages during sending

  • messages don't wait for remote echo to be marked as sent

  • make storage on iOS more reliable, this should hopefully be the last of storage errors on this platform

  • fix file up/downloads on iOS

  • fixed an issue where room keys never got shared if you talk into an e2ee room for the very first time since login and you happen to be offline

  • more little fixes.

Fixes, fixes, fixes. Working on another catchup sync slowdown, and then hope to focus soon a new features!

Element Clients

Updates from the teams

VoIP

  • Fixed a bug causing connectivity failures on web
  • Fixed bug preventing calls from connecting on iOS
  • Moved VoIP call button to the top on web, and fixed bug resulting from the fact they no longer turn into the hangup button

Web

  • Added Edge to supported browsers

  • E2EE Olm session deadlock issue (primarily affecting large accounts during startup) has been resolved

  • 1.7.22 was released on Monday (1 Mar)

    • Fixed user content sandbox security issue, please upgrade!
    • Improved Jitsi conference names
    • Disabled chat effects when reduced motion preferred
  • 1.7.23-rc.1 next Wednesday (10 Mar)

iOS

  • Release 1.2.3 has been rejected by Apple

  • Release 1.2.5 is available on App Store, it contains fixes:

    • SSO flow crash fix
    • An new option to filter NSFW public rooms (on by default)
    • Fix a crash after login/password account creation when a grace is active
    • Plus fixes from releases 1.2.2 and 1.2.3

Android

  • Release 1.1.0 is now in production on the stores.
  • We are working to improve the global performance of the application. Faster initial sync, faster sending of messages, faster timeline, etc.

Element Web on IPFS

TR_SLimey reported:

I mentioned a while back that I've put Element Web on IPFS.

Since then, I unfortunately had to change its DNSLink domain, so I decided to post about it again, with the new addresses and some minor updates. Here are the former:

  • ipns://element-web.0x1a8510f2.space/ (Main DNSLink address)

  • ipns://k2k4r8mx8muu913j5oy3e4zvt77xtks8xgxuamlv0ltzwx95lcgk2glw (IPNS hash which the above address points to)

  • ipfs://bafybeievxs6qzzbfbn6xcbz3xwxhjzxkffjbpwbcc3gikrac6ytqkkutva (IPFS hash of the current snapshot of the above)

In other news, I've updated Element on IPFS to v1.7.22 and made a tiny script to easily update it in future. In case it's of use to anyone, it's right here:


#!/bin/bash

if [ "$#" -ne 2 ]; then

        echo -e "Invalid number of arguments (got $# but 2 required)!\nUsage: $0 <element web folder> <release tag to download>" >&2
        exit 1

fi

wget -c "https://github.com/vector-im/element-web/releases/download/$2/element-$2.tar.gz" -O - | tar --overwrite --strip-components=1 -xzv -C $1 element-$2

ipfs add -r -p $1

echo "Now use: 'ipfs name publish -key=<publish key> <folder ID>' to publish on IPFS"

NeoChat

Carl Schwan announced:

We are currently refactoring the NeoChat room page. Nothing merged yet but scrolling is now smoother, we added swipe gestures to reply on mobile, and more importantly chat bubbles are coming :) Here is a small video of the new timeline, feedback very much welcome.

Mirage

A fancy, customizable, keyboard-operable Matrix chat client for encrypted and decentralized communication. Written in Qt/QML and Python, currently in alpha. https://github.com/mirukana/mirage

miruka said:

0.7.0 has finally been released, including:

  • Push rules and notifications support:

    • Native desktop notifications

    • Sound effect playback

    • Notifications can be globally turned on and off in the running client within a click

    • Context menu options in the room pane to control a room's notification settings

    • Push rule editor in account settings:

      • Control for any rule whether matching messages are

        marked as unread, highlighted, trigger a desktop notification, sound, window alert/flash, or any combination of those actions

      • Create custom rules targeting a particular room, message sender,

        messages containing certain words or messages matching advanced conditions

  • New Python-based configuration system replacing the previous rudimentary JSON settings file

  • Config files and themes are now automatically reloaded when changed on disk

  • Rooms in the left pane can now be pinned to the top of the list

  • Support for drag-and-dropping text and files to upload in chats

  • Tooltips for the message read counters, listing who has read a message and when

  • The chat header now indicates when messages are selected in the timeline,

    and offers copy/redact/clear selection buttons

  • Visible indicator when downloading files

  • Command-line arguments parsing and a --start-in-tray option

  • Support for HTTP and SOCKS5 proxies

  • Hovering on stability percentages in the public homeserver list

    now shows more detailed tooltips about the server's recent downtimes

  • And a lot of other fixes and improvements: full changelog

Dept of SDKs and Frameworks ๐Ÿงฐ

libkazv (and the Kazv Project)

libkazv is a sans-io C++ (gnu++17) client library

tusooa reported:

libkazv is a sans-io C++ (gnu++17) client library

built upon lager. Along with it there is kazv, a kirigami/qml client, and a

forward bot between matrix and tencent qq. Talk to us on #kazv:tusooa.xyz .

Updates

In the past 1 week we ("tusooa and her longcat," we hope someday we can

remove this annotation :P):

  1. Published API documentation on the web.
  1. Together with that, added a "getting started" guide with an example >w<
  1. Made matrix-tencent-forward-bot support directly forwarding images

    from the matrix room.

  1. Colourized sender nicks in messages for matrix-tencent-forward-bot.

Dept of Services ๐Ÿš€

#matrix-admins-edu-german:matrix.org

Marcus Schopen (the fellow with the festive mugs) reported:

Together with Anthony Rhode from Element we have published the room #matrix-admins-edu-german:matrix.org for Matrix Admins at German-speaking educational institutions. Here we want to discuss topics about Matrix in an academic context. The room is open to the public and everyone is very welcome.

How is it not #matrix-admins-edu-deutsche:matrix.org!? This is a welcome public room for the growing number of Matrix Admins in Germany!

Dept of Interesting Projects ๐Ÿ›ฐ๏ธ

Cactus Comments

carl announced:

I am very excited to launch Cactus Comments (https://cactus.chat) today together with @asbjorn:olli.ng!

Cactus Comments is a federated, web-embeddable comment system. And of course, we federate by building on Matrix!

On the frontend - your website - we embed a Matrix client written in Elm that renders Matrix rooms as comment sections. The client is designed to be especially easy to embed on static sites. Once you log in to the client, it will only communicate directly with your Matrix server of choice.

On the backend, an application service is used to create rooms on-demand and help with moderation. It is built with Python and is completely stateless. It does this by modelling all state in native Matrix objects without any custom events!

The entire thing is free and open source software and we would love it if you were to contribute. We really hope some of you will give it a try!

Join our matrix room: #cactus:cactus.chat

Read the introductory blog post: https://cactus.chat/blog/hello-cactus-comments/ Try the demo: https://cactus.chat/demo/

Source code: https://gitlab.com/cactus-comments

What a project! Let's see how we can integrate this with the Matrix.org blog!

Matrix Queue Processor

reivilibre reported:

With things on the Internet vanishing, I've been wanting to start saving copies of interesting web pages and videos to my server-at-home.

This really doesn't involve much more than just running youtube-dl or archivebox on the relevant URLs, but I needed a better way than having to e-mail myself reminders to download thingsโ€ฆ

I decided to make Matrix Queue Processor.

You can run it occasionally (or set it up to run in a cron job!) and it will then read all outstanding messages and feed the text to a given command, such as youtube-dl.

You can choose for it to delete processed messages, or tick them off (โœ…). If the command wasn't successful, it will instead reply with the exit code.

It's not particularly sophisticated but it might at least be a small starting point for anyone who wanted to make something similar, and it is end-to-end encryption capable because it uses the Python matrix-nio SDK.

See this example of use with youtube-dl; it fails on the second message because it's not a video page URL:

2021-03-05-6K_RG-Screenshot_20210301_074617.png

I like this a lot - there are so many uses for this project.

Hummingbard

ahq said:

Hummingbard updates:

  • Supports nested spaces under user profiles.

  • Enhanced sidebar/navigation.

  • Explore spaces modal.

  • Code cleanup, Bug fixes and UI tweaks.

https://github.com/hummingbard/hummingbard

Have you checkout out hummingbard yet? This is a very promising-looking project, I recommend reading this introduction to get started.

Dept of Ping ๐Ÿ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1envs.net387
2maescool.be464
3neko.dev494
4voolik.pw572.5
5fosil.eu577
6thomcat.rocks623
7kif.rocks702
8dodsorf.as722.5
9aria-net.org743
10matrix.sp-codes.de753

That's all I know ๐Ÿ

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2021-02-26

26.02.2021 00:00 โ€” This Week in Matrix โ€” Ben Parsons

Matrix Live ๐ŸŽ™

Dept of Status of Matrix ๐ŸŒก๏ธ

Decentralized Systems and Network Services Research Group output

Florian told us:

The article of my research group Decentralized Systems and Network Services Research Group from the KASTEL Institute of Information Security and Dependability and me, Analysis of the Matrix Event Graph Replicated Data Type, was published in the Open Access journal IEEE Access! ๐Ÿ“ฐ ๐Ÿฅณ Open Access here means that the article is freely available for you to read, under CC-BY.

2021-02-26-x72VR-MEG_with_Chat.png

In layman's terms, we showed that the Event Graph provides a strong form of eventual consistency even when new events are applied in arbitrary order. In addition, this consistency guarantee can still be provided even if all other servers in a room are malicious, which is unexpected compared to the honest majority required for the usual Distributed Ledger Technology. We also looked at the problem of โ€œIf servers randomly select at which parents at the bottom of the Event Graph they append new events, and the number of parents for an event is limited, how can you be sure that the Event Graph does not grow arbitrarily wide?โ€œ, which is something that puzzled me since the beginning of my Ph.D. two years ago. It is easy simulate a given setup and see that the width shrinks really fast, but it is also easy to see that arbitrary growth has a likelihood larger than zero. As this was astonishingly hard to grasp mathematically, we were very happy that we could recruit Prof. Dr. Norbert Henze from the Institute of Stochastics as co-author to provide evidence that you can expect the Event Graph to shrink to a normal width quite quickly.

Let me quote parts of the abstract for some more technical details:

[โ€ฆ] By comparison with traditional decentralized communication systems, Matrix replaces pure message passing with a replicated data structure. This data structure, which we extract and call the Matrix Event Graph (MEG), depicts the causal history of messages. We show that this MEG represents an interesting and important replicated data type for decentralized applications that are based on causal histories of publish-subscribe events: First, we prove that the MEG is a Conflict-Free Replicated Data Type for causal histories and, thus, provides Strong Eventual Consistency (SEC). With SEC being among the best known achievable trade-offs in the scope of the well-known CAP theorem, the MEG provides a powerful consistency guarantee while being available during network partition. Second, we discuss the implications of byzantine attackers on the data typeโ€™s properties. We note that the MEG, as it does not strive for consensus or strong consistency, can cope with n > f environments with n participants, of which f are byzantine. Furthermore, we analyze scalability: Using Markov chains, we study the number of forward extremities of the MEG over time and observe an almost optimal evolution. [โ€ฆ] With the properties shown, a MEG represents a promising element in the set of data structures for decentralized applications, but with distinct trade-offs compared to traditional blockchains and distributed ledger technologies.

Dept of Spec ๐Ÿ“œ

anoa offered:

Interrupting spec update!

Spec

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Closed MSCs:

MSCs in Final Comment Period:

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus for this week, there's been activity from the core team on MSC2516 (voice messages), MSC2312 (matrix URI scheme) and MSC3026/MSC3032 (presence improvements).

New Spec Platform

PR updates on matrix-doc has been refactored and straightened out, and is up for review. We're reviewing changes to our infrastructure on matrix.org in parallel which should result on spec.matrix.org hosting the unstable spec sometime early next week!

2021-02-26-c2XcZ-stacked_area_chart.png

Dept of Servers ๐Ÿข

Synapse

Synapse is a popular homeserver written in Python.

callahad announced:

Hey, it's Friday!

We released Synapse 1.28 yesterday, with the bulk of effort going into stability improvements and bug fixes, and an emphasis on further improving social login.

We're also working to more clearly communicate our prioritization of issues, starting with documenting our triage and labeling process for the core Synapse repository on GitHub.

(Still very much a work in progress, but starting to shape up nicely.)

We're also on track to maintain our fortnightly cadence of Release Candidates, with 1.29.0rc1 due out in just a few days. I look forward to telling you all about it next week!

Homeserver Deployment ๐Ÿ“ฅ๏ธ

Kubernetes

Ananace offered:

Another week, another bump of the Kubernetes chart and image of Synapse (to 1.28.0), still haven't had time to fix it to be image agnostic - the Synapse team is just developing far too fast.

Dept of Bridges ๐ŸŒ‰

Gitter

Eric Eastwood announced:

We've added a "Chat via Matrix" button next to the join button in a Gitter room so you can jump straight into Matrix and start chatting that way. We've also added a Matrix.to link in the chat header so you can switch off to your favorite Matrix client at any time.

2021-02-26-vvZtd-Photoshop_2021-02-25_18-52-05.png

New LINE messenger bridge

fair offered:

matrix-appservice-line: A bridge for LINE messenger based on running LINE's Chrome extension in Puppeteer. Based on tulir 's mautrix-amp, as per Eric Migicovsky's suggestion ๐Ÿ™‚

Currently very rough around the edges, but message sending & receiving in 1:1 chats works, as does conversation syncing & backfilling.

Repo: git://miscworks.net/matrix-appservice-line
Discussion: #matrix-appservice-line:miscworks.net

Dept of Clients ๐Ÿ“ฑ

Neochat 1.1

Carl Schwan announced:

We released NeoChat 1.1 containing 2 months of new features and bugfixes. Most notably message editing, sticker events support, rewritten login screen, SSO support and more commands. Check it out at https://carlschwan.eu/2021/02/23/neochat-1.1/

2021-02-26-jJwf3-image.png

In addition to the 1.1 release from earlier, aa13q added support for fancy animation for the ๐ŸŽ†, ๐ŸŽ‰ and ๐ŸŒจ๏ธ emojis. This feature can be disabled.

2021-02-26-euMEn-Screenshot_20210226_144943.png

aa13q shared this video too!

neochat pinephone poorly recorded effects demo ๐Ÿ™‚ https://i.imgur.com/Z9vF9px.mp4

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE and intends to be full featured and nice to look at

2021-02-26-M_N-6-clipboard.png

Nico (@deepbluev7:neko.dev) sent a teaser image first, then added:

As seen from the image, you can now share your screen. This currently only works on X11, since there are no cross platform APIs for that, so look out for that, if you run into issues.

Jedi18 also polished the UX a bit more and fixed bugs, so registration should now work properly again and leaving or joining rooms should place you in the right rooms after that. We also fixed some inconsistencies, where text could be misaligned, when writing long messages.

Quaternion

kitsune reported:

Version 0.0.95 beta4 is out, with significantly better Markdown support (code fences work now, and the overall formatting is better), further improvements to the shuttle dial and other smaller fixes. The release notes and some packages are at their usual place at GitHub; the Flatpak is also coming real-soon-now.

No (or almost no) changes in strings are expected before 0.0.95 from now, so translation contributions are very welcome at Lokalise. German, Polish and Spanish translations are quite advanced but need just a few more entries; the French translation is just about half-way; and there are many prominent languages not even started. You can help!

Element Clients

Updates from the team:

Delight

  • Social Login
    • After shipping Social Login on matrix.org, weโ€™re working on general stability to harden the feature, and get better visibility on stability.
  • Spaces
    • On Android, weโ€™re making good progress on creating and managing spaces, first focusing on public space use cases. Otherwise, weโ€™re also polishing implementations on web & iOS.

Web

  • 1.7.22 RC on staging, includes
    • Improved Jitsi conference names
    • Disabled chat effects when reduced motion preferred
  • 1.7.22 planned for release on Monday (1 Mar)

iOS

  • 1.2.2 RC currently available in TestFlight
  • 1.2.3 RC is coming
  • Will be released on Monday:
    • Mainly bug fixes around e2ee and VoIP

Android

  • 1.1.0 released with a huge update on VoIP
  • Initial sync is now more reliable for big accounts, consuming less RAM

Watch The Matrix, for Apple Watch

Doug told us:

Hey TWIM. I'd like to present Watch The Matrix, a matrix client for watchOS that I've been working on over the last few weeks. Currently can view rooms, receive new messages and react to them:

Find the code: https://github.com/pixlwave/Watch-The-Matrix

Dept of SDKs and Frameworks ๐Ÿงฐ

libQuotient

kitsune reported:

libQuotient 0.6.5 is out, featuring fixes in rich replies handling and the updated Matrix URIs support, as per the latest iteration of MSC2312, enabling shorter, nicer Reddit-style URIs such as matrix:r/thisweekinmatrix:matrix.org.

libkazv (and the Kazv Project)

tusooa offered:

libkazv is a sans-io C++ (gnu++17) client library

built upon lager. Along with it there is kazv, a kirigami/qml client, and a

forward bot between matrix and tencent qq. Talk to us on #kazv:tusooa.xyz.

Updates

In the past (how many?) weeks we ("tusooa and her longcat," we hope someday we can

remove this annotation :P):

  1. Released libkazv 0.1.0 and matrix-tencent-forward-bot 0.1.0.

    Those are stable enough to run as daemon, at least for days.

  1. Added related ebuilds to the overlay.
  1. Implemented streaming uploads for libkazv.
  1. Made matrix-tencent-forward-bot support directly forwarding images

    to the matrix room.

Opsdroid 0.21

Cadair reported:

opsdroid is an easy to use chatops framework with excellent matrix support.

0.21 Release

This week opsdroid 0.21 has been released, it's an incremental release but includes a couple of nice matrix related features and fixes. The highlights are:

  • Support authorization by access token in addition to user name and password for the matrix connector. #1707.

  • Fix parsing a sync response with more than one message in it. #1715.

  • Minor fixes to the matrix documentation. Thanks to @anoadragon453 @xDiesel76 for these.

For a full list of the other changes see the release notes.

If you have any questions or want to get involved come say hi in #opsdroid-general:matrix.org, raise an issue on GitHub and checkout the docs.

Caridina, Crystal library for Matrix

erdnaxeli said:

Caridina, a Crystal library for Matrix, got some new releases, which bring it to v0.10.2. It now supports the last version of Crystal (0.36.1) and redacted events, which I discovered the hard way were making the lib crash. Internally a big part of the lib was rewritten using macros, which makes the code a lot clearer but the API doc too. Thanks to Ruma actually for giving me inspiration about implementing events using macros.

Again, if you like Crystal and want to write some matrix bot, this library might interest you. Feel free to open issues about bugs or feature requests! There is now a room #caridina:cervoi.se to talk about it.

Dept of Bots ๐Ÿค–

taavi

select announced:

So I had a minute to release my open q&a bot taavi.

I always wanted to write a chatbot that does cool things, so checkmark for that one on the list. In addition to that I recently failed a job interview. For their technical skill test (I did not fail this :P), I got a task for which I learned how to use puppeteer, then saw a post on reddit about Tuxi and together with my previous project that included Matrix (Audius), I knew what I had to do: create Taavi.

I have created this bot as a "weekend project", literally one day for the code and one day for documentation. Surely it's far from perfect but I don't have any big plans with it so farโ€ฆ If you think I should add some features, create an issue and let me know. I could imagine adding other websites / services to it like last.fm, do you have any ideas?

I gave my daughter the task to choose a name for a bot, that can answer any question in the world, and she came up with Taavi, nice!

https://gitlab.com/s3lect/taavi-bot

Ansible role for installing Maubot via Docker

jaywink said:

I created an Ansible role for installing Maubot via Docker - https://git.feneas.org/jaywink/ansible-maubot . Currently it only supports an Sqlite database, but pull requests are welcome for more configuration. Supports a pre-created client account.

Dept of Interesting Projects ๐Ÿ›ฐ๏ธ

Hummingbard

ahq reported:

I've written a post introducing Hummingbard. https://hummingbard.com/hummingbard/introducing-hummingbard

This looks really interesting!

Final Thoughts ๐Ÿ’ญ

JCG said:

Not sure if TWIM worthy, but here's a nice article about the recent flood of people migrating away from whatsapp, and why people should've migrated to matrix instead of Signal.

(In German)

There is also a sympathetic response that has been discussed on HN.

Seirdy said:

i've been getting some feedback from other matrix users on a blog post i wrote that outlined a few concerns I had about the Matrix ecosystem and steps for improvement. idk if it'd be relevant in TWIM, but I noticed that "Final Thoughts" sometimes linked blog posts so I thought I'd share: https://seirdy.one/2021/02/23/keeping-platforms-open.html

Dept of Ping ๐Ÿ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1rollyourown.xyz420
2neko.dev465.5
3maunium.net480
4phys.ethz.ch481
5envs.net492.5
6mailstation.de738
7blackline.xyz891
8kif.rocks902
9casavant.org932
10aria-net.org1028

That's all I know ๐Ÿ

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.28.0 released

25.02.2021 00:00 โ€” Releases โ€” Dan Callahan

Synapse 1.28.0 is now available!

This release comes with several further improvements to the user experience of single sign-on and numerous bugfixes and stability improvements.

For admins, Synapse 1.28 adds a new Admin API for retrieving event context and implements new spam checker hooks which enable checking file uploads and remote downloads. We've also improved memory usage of media repository workers.

Lastly, we have marked an undocumented Admin API for deprecation. If any of your tools use /_synapse/admin/v1/users/<user_id> to get account information, please replace that with the V2 List Accounts API, which has been available since Synapse 1.7.0.

There are no special upgrade instructions for 1.28.0. See the full changelog for more details on what's in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including arya2331, auscompgeek, bubu, compu42, dklimpel, dykstranet, and shadowjonathan.

We'd also like to thank yoric for thoroughly reviewing and re-organizing the Synapse CONTRIBUTING.md file.

This Week in Matrix 2021-02-19

19.02.2021 00:00 โ€” This Week in Matrix โ€” Ben Parsons

Matrix Live ๐ŸŽ™

Dept of Spec ๐Ÿ“œ

Spec

anoa offered:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

New MSCs:

Spec Core Team

The Spec Core Team has spent this week working on getting MSC2312 (Matrix URI Scheme) over the line, which is really exciting! Otherwise the team has been reviewing MSC2304 (knocking).

New Spec Platform

Work is currently underway in migrating to the new spec platform. A PR for updating the CI of matrix-org/matrix-doc to build the new spec is up, and an equivalent for matrix-org/matrix.org should be up next week.

2021-02-19-QCQcX-stacked_area_chart.png

MSC3013

sorunome said:

Soru has been working on experimenting with / designing a way to improve push systems within the matrix ecosystem, so that ideally clients can display full event contents in a notification and dispose of specific push notifications without the need for any extra HTTP calls. This lead her to making MSC3013 (encrypted push), so that no gateway is able to read event contents pushed down to the client and MSC3014, so that the client can know which rooms have been read on other devices, allowing them to dispose push notifications.

This is more than just MSCs already, soru already got encrypted push in a patched synapse and a patched fluffychat working locally, and it is working super great!

Dunno soru is just really excited about this somehow >~< ๐ŸฆŠ

Dept of Servers ๐Ÿข

Hedwig: push gateway

krille reported:

Let's talk more about Hedwig: This is a push gateway for matrix and an alternative to Sygnal. Hedwig is completely written from scratch in Rust (Yes! Rust!) and offers the basic functionality to forward matrix push notifications to Firebase Cloud Messaging. That's it. Nothing more. It is intended to be as lightweight as possible and so it is very fast and stable. Just change a simple config.toml file and run the binary. There are docs for how you can configure a proxy in Apache2 and an example systemD service file. Currently it only supports event_id_only formatted notifications but maybe, in the future, we expand this functionality.

Learn more: https://gitlab.com/famedly/services/hedwig

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander reported:

This week we released Dendrite 0.3.10, and have committed a couple more fixes to master:

  • The database is now only queried for remote servers once in /send

  • The oldest cache entries are now gradually evicted, saving a small amount of memory

  • The destination queues in the federation sender are now not stored forever, saving some memory

  • A bug where a non-excluded event could be accidentally excluded from /sync has been fixed

  • Some dependency fixes have been merged

Sytest compliance:

  • Client-server APIs: 60%

  • Server-server APIs: 82%

As always, feel free to join us in #dendrite:matrix.org for general Dendrite chat or #dendrite-dev:matrix.org for development discussion.

Synapse

Synapse is a popular homeserver written in Python.

callahad reported:

Synapse 1.27 is out, and you can read all about it on the Matrix Blog! This is the version that powered FOSDEM, and we're super proud of it. Especially the much improved support for multiple social login providers, which debuted at FOSDEM and is now enabled on matrix.org.

In addition to all the performance, stability, and social login improvements, there are two important things to note:

  1. If you use SSO, you will need update your configuration before upgrading; please see the link above for details.
  2. We were unable to build Docker images for ARM platforms in time for this release, and will not build images for ARMv7 in future releases.
  • To avoid unexpected breakage, we have not updated the latest tag on Docker Hub for this release; if you want Synapse 1.27, you'll need to specifically reference the v1.27.0 tag.

  • Images for ARM64 (and an updated latest tag) will return with the next Synapse release.

It took us a bit longer than usual to get 1.27 out the door, so we're also pleased to announce that 1.28.0rc1 is also now available, and we expect to release it early next week. Keep your eyes peeled for more on that next week!

Homeserver Deployment ๐Ÿ“ฅ๏ธ

Kubernetes

Ananace announced:

New chart versions have been pushed for Element 1.7.21 and Synapse 1.27.0.

The rejiggering to allow arbitrary images for the Kubernetes chart got delayed due to lack of time, but 1.27.0 should hopefully be the last version with that limitation.

Dept of Clients ๐Ÿ“ฑ

Nio

Nio is a client written with SwiftUI

kilian offered:

Nio has seen some great improvements since the last update, mostly thanks to gaining a new main contributor \o/

  • History loading now works (seems like a rather essential messenger feature ๐Ÿ˜„)

  • New conversations can be started (with a slick UI)

  • There's a preliminary (but well-working!) build for macOS on Catalyst

  • As a first step towards E2EE support Nio can now participate in encrypted sessions while running

  • Many new translations (thanks to all the contributors on translate.element.io!)

  • Lots of additional bugfixes and improvements

Join us in #niochat:matrix.org for more details ๐Ÿ˜Š

FluffyChat

FluffyChat is a cute cross-platform matrix client. It is available for Android, iOS, Web and Desktop.

krille said:

FluffyChat 0.27.0 is out. Not with the new design but with some minor design improvements like the return of the presence list top bar (or whatever you would name it!). Also this release brings a lot of bug fixes, working bootstrapping and the switch to our new Push Gateway (in Rust) we have invented inside of Famedly. You can find it here: https://gitlab.com/famedly/services/hedwig

FluffyChat on Pinephone

Lukas Lihotzki said:

FluffyChat runs on the PinePhone (Flutter Linux ARM64). I've built it with flutter master on the PinePhone itself, because cross-compiling is not yet supported. Currently, Flutter Linux on the PinePhone only works with X11. It crashes with Wayland.

I am on the brink of buying one of these!

2021-02-19-2Lo26-pine.png

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) told us:

We had a fun week this time. Today I tested trilenes screen sharing code, which seems to work pretty well on X11, although it only shares the full screen atm. I expect window sharing to land next week though. We also ran into some issues with matrix.org enabling optional SSO login, which locked some people out of their accounts, since Nheko preferred SSO login in that case... This is now fixed and in such cases you will be offered both login flows.

Jedi18 also added a room name completer, which means you can finally complete aliases when typing them! The room settings were also ported to Qml and now actually show the room topic. Apart from that there were various UX fixes here ad there, like hovering actually working properly now, emote messages formatting correctly in notifications and more. We also adapted the matrix: uri support to the latest changes, although the MSC is a bit volatile atm, so it may need some more adaptions over time.

I think that's all, have a nice weekend everyone!

Hydrogen

Bruno announced:

I'm back working on Hydrogen, and started to tackle logging to be able to diagnose problems better. I've opted for a structured log format in JSON that gives a tree of log nodes for an entire operation (e.g. sync, or sending a message, ...) which will be more compact and should also be easier to find patterns in than a pure line based format. I had hoped to get that out in a release by today but didn't get as far as I hoped, so should get released early next week.

2021-02-19-VTLG5-image.png

Yes! Hydrogen progresses!

Element

Neil told us:

Delight

  • Social Login

    • Social log-in has shipped to matrix.org, we are seeing about 40% of new users opting for a social method. Next up will be landing some robustness improvements as well as automatic avatar import.
  • Spaces

    • We are aiming to get a basic version up and ready over the next few weeks. The idea is to have something behind a labs flag that will work for Spaces containing public rooms only. After that we will think about more complex cases such as Spaces containing private rooms.

VoIP

  • Fix for crash on Android caused by VoIP tiles

  • Design updates on iOS for in-call view and dial pad

  • Couple of tweaks to web that may help 1:1 call connectivity

Web

  • Element 1.7.21 was released on Monday (15 Feb)

iOS

  • Element 1.2.1 was released on Friday (12 Feb)

  • We now use XcodeGen to generate Xcode project files. Please check new instructions to build the project. It is only on develop for the moment

  • We made an update to libolm so that it accepts an external pickle key to encrypt its data when serialising it. Element-iOS uses the Keychain to provide this key.

  • We started to review and merge pending PRs in our repos but we still have work. Thanks for your contributions and patience

Android

  • We are preparing the release 1.1.0 which contains the work on VoIP

  • This release will also contain a lot of bug fixes

Dept of SDKs and Frameworks ๐Ÿงฐ

ruma

Ruma is a set of Rust library crates around Matrix

jplatte said:

This week,

Dept of Ops ๐Ÿ› 

matrix-docker-ansible-deploy: Synapse workers

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi told us:

After lots and lots of work (done over many months by Marcel Partap, Max Klenk, a few others from the Technical University of Dresden, Germany and various other contributors), support for Synapse workers has finally landed in matrix-docker-ansible-deploy. ๐Ÿš€

Learn more from our changelog announcement.

matrix-docker-ansible-deploy: GroupMe bridge

Slavi returned to say:

Thanks to Cody Neiman, matrix-docker-ansible-deploy can now install the mx-puppet-groupme bridge for bridging to GroupMe.

This brings the total number of bridges supported by the playbook up to 18. See all supported bridges here.

To get started, follow our Setting up MX Puppet GroupMe docs.

ansible-matrix-collection from tntclaus

tntclaus announced:

Collection of Ansible modules to manage Matrix Homeserver instance. Only tested with Synapse but may (partially) work with the other Matrix Homeserver implementations.

Provides modules to manage users, rooms and communities.

https://github.com/eraga/ansible-matrix-collection

synadm

jojo said:

synadm is growing! Pull requests have been submitted, issues been raised, bugs fixed and quite some user community seems to have evolved around it as itโ€™s regularly mentioned on #synapse:matrix.org

Since itโ€™s first announcement on TWIM in December, notable things happening in the project are:

  • Massive internal overhaul, tidying up and refactoring

  • Media Admin API support

  • Make Room Admin API support

  • Delete Group (community) API support

  • 4 available output formats (json, yaml, pprint, human)

  • Python requests debugging switch

  • Textual Error responses of Synapse now showing in output

  • Licensed under GPL-3

Have a look at the latest release notes for a complete list of new features and fixes: https://github.com/JOJ0/synadm/releases

Thanks a lot to contributors @hpd, @kaiyou, @schwindp, @aaronraimist and @rht! Keep those PR's and ideas coming! You have been a great help!

Next features on the roadmap are "Purge History API", regular matrix client calls support and login as a user. Help me prioritise features by raising an issue: https://github.com/JOJ0/synadm

Still, I am basically a one man show, and certainly any contributions are warmly welcomed! Even as a non-programmer you can help by keeping the implementation-status list up to date. If you find an admin API missing, please take a minute and use the integrated edit functionality github provides and add it to the list: https://github.com/JOJ0/synadm#implementation-status--commands-list. It helps contributors to quickly find out what feature they could code!

I am also thinking about finding people who'd like to join the team, code new features and help maintain codebase, review PR's, get push access and so on. Spread the word! Join #synadm:peek-a-boo.at!

Matrix in the News ๐Ÿ“ฐ

Shine reported:

Dunno if twim worthy? Proton mentioned Matrix in https://protonmail.com/blog/whatsapp-alternatives/

Seems to be worthy, it's been included anyway!

Final Thoughts ๐Ÿ’ญ

Nico (@deepbluev7:neko.dev) offered:

Oh, and maybe as a side note: Since Steam voice chat failed me one too many times now, I now use Nheko for voice chat, if I just play with one other player. It is more reliable and most people I play with do have a voip capable Matrix client installed!

Using own product ๐Ÿ˜€ ๐Ÿ‘

TR_SLimey said:

It seems neochat has made it into the Plasma 5.21 preview video, albeit not quite as one might expect :P

2021-02-19-1248X-Screenshot_20210217-193527.png

Here's the video link: https://youtu.be/ahEWG4JCA1w

That indeed is an unusual way to get featured, well done NeoChat!

Dept of Ping ๐Ÿ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1envs.net437
2midov.pl493
3kif.rocks582
4nicoll.xyz638
5maunium.net708
6rollyourown.xyz785
7utzutzutz.net912
8blackline.xyz975.5
9shortestpath.dev1194
10roeckx.be1516

That's all I know ๐Ÿ

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.27.0 released

18.02.2021 23:25 โ€” Releases โ€” Dan Callahan

Synapse 1.27.0 is now available!

We're especially proud of this release, as this is the version of Synapse that powered FOSDEM 2021 on Matrix. As such, our main focus was on stability, performance, and long-awaited support for social login.

What's New

To our surprise, nearly half of all people who created accounts on the FOSDEM homeserver did so via a social login method. Full support for those methods is included in Synapse 1.27.0, and already available for all users on the matrix.org homeserver.

We've also changed how we use Redis in larger deployments, making Synapse more resilient to lost connections and eliminating delays when restarting with multiple federation senders.

Our Server Admin APIs saw a few tweaks, including new APIs to query and delete forward extremities or list the current state of a room.

See the full changelog for more.

Breaking Changes for SSO

If you use Single Sign-On (SSO) via SAML, OAuth2, or OpenID Connect you must adjust your provider's configuration before upgrading to Synapse 1.27.0, as some endpoint URLs have changed. See the upgrading notes for more information.

Dropping ARMv7 Docker Images

We were unable to produce ARM-based Docker images for this release due to problems with cross-compilation. As a result, we have made the difficult decision to cease building 32-bit ARMv7 Docker images as part of our release process. We will resume publishing ARM64 images with the next Synapse release.

Users on ARMv7 platforms (most notably Raspberry Pis) should consider building images locally using Synapse's Dockerfile or switching to installing Synapse directly as a Python module. Users with Raspberry Pi 3's or newer also have the option of installing a 64-bit Linux distribution and using an ARM64 Docker image.

Thank you to our contributors

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including dklimpel, intelfx, jcgruenhage, Oliver-Hanikel, rht, and y-pankaj.

How we hosted FOSDEM 2021 on Matrix

15.02.2021 00:00 โ€” General โ€” Matthew Hodgson

Hi all,

Just over a week ago we had the honour of using Matrix to host FOSDEM: the world's largest free & open source software conference. It's taken us a little while to write up the experience given we had to recover and catch up on business as usual... but better late than never, here's an overview of what it takes to run a ~30K attendee conference on Matrix!

[confetti and firework easter-eggs explode over the closing keynote of FOSDEM 2021]

First of all, a quick (re)introduction to Matrix for any newcomers: Matrix is an open source project which defines an open standard protocol for decentralised communication. The global Matrix network makes up at least 28M Matrix IDs spread over around 60K servers. For FOSDEM, we set up a fosdem.org server to host newcomers, provided by Element Matrix Services (EMS) - Element being the startup formed by the Matrix core team to help fund Matrix development.

The most unique thing about Matrix is that conversations get replicated across all servers whose users are present in the conversation, so there's never a single point of control or failure for a conversation (much as git repositories get replicated between all contributors). And so hosting FOSDEM in Matrix meant that everyone already on Matrix (including users bridged to Matrix from IRC, XMPP, Slack, Discord etc) could attend directly - in addition to users signing up for the first time on the FOSDEM server. Therefore the chat around FOSDEM 2021 now exists for posterity on all the Matrix servers whose users who participated; and we hope that the fosdem.org server will hang around for the benefit of all the newcomers for the foreseeable so they don't lose their accounts!

Talking of which: the vital stats of the weekend were as follows:

  • We saw almost 30K local users on the FOSDEM server + 4K remote users from elsewhere in Matrix.
  • There were 24,826 guests (read-only invisible users) on the FOSDEM server.
  • There were 8,060 distinct users actively joined to the public FOSDEM rooms...
  • ...of which 3,827 registered on the FOSDEM server. (This is a bit of an eye-opener: over 50% of the actively participating attendees for FOSDEM were already on Matrix!)
  • These numbers don't count users who were viewing the livestreams directly, but only those who were attending via Matrix.

Given last year's FOSDEM had roughly 8,500 in-person attendees at the Universitรฉ libre de Bruxelles, this feels like a pretty good outcome :)

Graphwise, local user activity on the FOSDEM server looked like this:

How was it built?

There were four main components on the Matrix side:

  1. A horizontally-scalable Matrix server deployment (Synapse hosted in EMS)
  2. A Jitsi cluster for the video conferencing, used to host all the Q&A sessions, hallway sessions, stands, and other adhoc video conferences
  3. An elastically scalable Jibri cluster used to livestream the Jitsi conferences both to the official FOSDEM livestreams and to provide a local preview of the conference on Matrix (to avoid the Jitsis getting overloaded with folks who just want to view)
  4. conference-bot - a Matrix bot which orchestrated the overall conference on Matrix, written from scratch for FOSDEM by TravisR, consuming the schedule from FOSDEM and maintaining all the necessary rooms with the correct permissions, widgets, invites, etc.

Architecturally, it looked like this:

On the clientside, we made heavy use of widgets: the ability to embed arbitrary web content as iframes into Matrix chatrooms. (Widgets currently exist as a set of proposals for the Matrix spec, which have been preemptively implemented in Element.)

For instance, the conference-bot created Matrix rooms for all the FOSDEM devrooms with a predefined widget for viewing the official FOSDEM livestream for that room, pointing at the appropriate HLS stream at stream.fosdem.org - which looked like this:

Each devroom also had a schedule widget available on the righthand side, visualising the schedule of that room - huge thanks to Hato and Steffen and folks at Nordeck for putting this together at the last minute; it enormously helped navigate the devrooms (and even had a live countdown to help you track where you were at in the schedule!)

Each devroom was also available via IRC on Freenode via a dedicated bridge (#fosdem-...) and via XMPP.

The bot also created rooms for each and every talk at FOSDEM (all 666 of them), as the space where the speaker and host could hang out in advance; watch the talk together, and then broadcast the Q&A session. At the end of the talk slot, the bot then transformed the talk room into a 'hallway' for the talk, and advertised it to the audience in the devroom, so folks could pose follow-on questions to the speaker as so often happens in real life at FOSDEM. The speaker's view of the talk rooms looked like this:

On the right-hand side you can see a "scoreboard" - a simple widget which tracked which messages in the devroom had been most upvoted, to help select questions for the Q&A session. On the left-hand side you can see a hybrid Jitsi/livestream widget used to coordinate between the speaker & host. By default, the widget showed the local livestream of the video call - if you clicked 'join conference' you'd join the Jitsi itself. This stopped view-only users from overloading the Jitsi once the room became public.

The widgets themselves were hosted by the bot (you can see them at https://github.com/matrix-org/conference-bot/tree/main/web). Meanwhile the chat.fosdem.org webclient itself ended up being identical to mainline Element Web 1.7.19, other than FOSDEM branding and being configured to hook the 'video call' button up to the hybrid Jitsi/livestream widget rather than a plain Jitsi.

Meanwhile, for conferencing we hosted an off-the-shelf Jitsi cluster sized to ~100 concurrent conferences, and for the Jibri livestreaming we set up an elastic scalable cluster using AWS Auto Scaling Groups. Jibri is essentially a Chromium which views the Jitsi webapp, running in a headless X server whose framebuffer and ALSA audio is hooked up to an ffmpeg process which livestreams to the appropriate destination - so we chose to run a separate VM for every concurrent livestream to keep them isolated from each other. The Jibri ffmpegs compressed the livestream to RTMP and relayed it to our nginx, which in turn relayed it to FOSDEM's livestreaming infrastructure for use in the official stream, as well as relaying it back to the local video preview in the Matrix livestream/video widget.

Here's a screengrab of the Jitsi/Jibri Grafana dashboard during the first day of the conference, showing 46 concurrent conferences in action, with 25 spare jibris in the scaling group cluster ready for action if needed :)

There was also an explosion of changes to Element itself to try to make things go as smoothly as possible. Probably the most important one was implementing Social Login - giving single-click registration for attendees who were happy to piggyback on existing identity providers (GitHub, GitLab, Google, Apple and Facebook) rather then signing up natively in Matrix:

This was a real epic to get together (and is also an important part of achieving parity between Gitter and Element) - and seems to have been surprisingly successful for FOSDEM. Almost 50% of users who signed up on the FOSDEM server did so via social login! We should also be turning it on for the matrix.org server this week.

Finally, on the Matrix server side, we ran a cluster of synapse worker processes (1 federation inbound, reader and sender, 1 pusher, 1 initial sync worker, 10 synchrotrons, 1 event persister, 1 event creator, 4 general purpose client readers, 1 typing worker and 1 user directory) within Kubernetes on EMS. These were hooked up for horizontal scalability as follows:

The sort of traffic we saw (from day 2) looked like this:

How did it go?

Overall, people seem to have had a good time. Some folks have even been kind enough to call it the best online event they've been to :) This probably reflects the fact that FOSDEM rocks no matter what - and that Matrix is an inherently social medium, built by and for open source communities (after all, the whole Matrix ecosystem is developed over Matrix!). Also, Matrix being an open network means that folks could join from all over, so the social dynamics already present in Matrix spilled over into FOSDEM - and we even saw a bunch of people spin up their own servers to participate; literally sharing the hosting responsibility themselves. Finally, having critical infrastructure rooms available such as #beerevent:fosdem.org, #cafe:fosdem.org and #food-trucks:fosdem.org probably helped as well.

That said, we did have some production incidents which impacted the event. The most serious one was on Saturday morning, where it transpired that some of the endpoints hosted on the main Synapse process were taking way more CPU than we'd anticipated - most importantly the /groups endpoints which handle traffic relating to communities (the legacy way of defining groups of rooms in Matrix). One of the last things we'd done to prepare for the conference on Friday night was to create a +fosdem:fosdem.org community which spanned all 1000 public rooms in the conference, as well as add the +staff:fosdem.org community to all of those rooms - and unfortunately we didn't anticipate how popular these would be. As a result we had to do some emergency rebalancing of endpoints, spinning up new workers and reconfiguring the loadbalancer to relieve load off the main process.

Ironically the Matrix server was largely working okay during this timeframe, given event-sending no longer passes through the main process - but the most serious impact was that the conference bot was unable to boot due to hitting a wide range of endpoints on startup as it syncs with the conference, some of which were timing out. This in turn impacted widgets, which had been hosted by the bot for convenience, meaning that the Jitsi conferences for stands and talk Q&A were unavailable (even though the Jitsi/Jibri cluster was fine). This was solved by lunchtime on Saturday: we are really sorry for folks whose Q&As or conferences got caught in this. On the plus side, we spotted that many affected rooms just added their own widgets for their own Jitsis or BBBs to continue with minimal distraction - effectively manually taking over from the bot.

The other main incident was briefly first thing on Sunday morning, where two Jibri livestreams ended up trying to broadcast video to the same RTMP URL (potentially due to a race when rapidly removing and re-adding the jitsi/livestream widget for one of the stands). This caused a cascading failure which briefly impacted all RTMP streams, but was solved within about 30 minutes. We also had a more minor problem with the active speaker recognition malfunctioning in Jitsi on Sunday (apparently a risk when using SCTP rather than Websockets as a transport within Jitsi) - this was solved around lunchtime. Again, we're really sorry if you were impacted by this. We've learned a lot from the experience, and if we end up doing this again we will make sure these failure modes don't repeat!

Other things we'd change if we have the chance to do it again include:

  • Providing a to-the-second countdown via a widget in the talk room so the speaker & host can see precisely when they're going 'live' in the devroom (and when precisely they're going to be cut in favour of the next talk)
  • Providing a scratch-pad of some kind in the talk room so the host & speaker can track which questions they want to answer, and which they've already answered
  • Keep the questions scoreboard and scratchpad visible to the speaker/host after their Q&A has finished so they can keep answering the questions in the per-talk room, and advertise the per-talk room more effectively.
  • Use Spaces rather than Communities to group the rooms together and automatically provide a structured room directory! (Like this!)
  • Use threads (once they land!) to help structure conversations in the devroom (perhaps these could even replace the hallway rooms?)
  • Make the schedule widgets easier to find, and have more of them around the place
  • Make room directory easier to find.
  • Give the option of recording the video in the per-talk and stands for posterity
  • Provide more tools to stands to help organise demos etc.

So, there you have it. We hope that this shows that it's possible to host successful large-scale conferences on Matrix using an entirely open source stack, and we hope that other events will be inspired to go online via Matrix! We should give a big shout out to HOPE, who independently trailblazed running conferences on Matrix last year and inspired us to make FOSDEM work.

If you want to know more, we also did a talk about FOSDEM-on-Matrix in this month's Open Tech Will Save Us meetup and the Building Massive Virtual Communities on Matrix talk at FOSDEM went into more detail too. Our historical Taking FOSDEM online via Matrix blog has been somewhat overtaken by events but gives further context still.

Finally, huge thanks to FOSDEM for letting Matrix host the social side of the conference. This was a big bet, starting from scratch with our offer to help back in September, and we hope it paid off. Also, thanks to all the folks at Element who bust a gut to pull it together - and to the FOSDEM organisers, who were a real pleasure to work with.

Let's hope that FOSDEM 2022 will be back in person at ULB - but whatever happens, the infrastructure we built this year will be available if ever needed in future.

This Week in Matrix 2021-02-12

12.02.2021 00:00 โ€” This Week in Matrix โ€” Ben Parsons

Matrix Live ๐ŸŽ™

This week we have Neil presenting P2P Matrix over Bluetooth. It's titled "P2P Matrix Bluetooth Low Energy demo", but I thought Neil and Matthew brought a pretty good energy to the video!

Dept of Status of Matrix ๐ŸŒก๏ธ

Famedly in the media

Nico (@deepbluev7:neko.dev) announced:

Famedly (Matrix based application for German hospitals and clinics) seems to be getting positive feedback so far: https://medtech-zwo.de/aktuelles/nachrichten/nachrichten/famedly-ueberzeugt-mit-messenger-dienst-im-krankenhaus.html

Article in German, but my favourite parts are:

  • They used the word "Anti-Datenkrake" to describe how data is handled in the application, which basically is a metaphor for Famedly being the opposite of the usual corporations trying to pry their tentacles into everyones personal data. See also: https://en.wiktionary.org/wiki/datenkraken

  • They actually use [matrix]โ€“Protokoll to refer to the Matrix protocol. The open and standardized Matrix protocol seems to be a big selling factor.

Dept of Spec ๐Ÿ“œ

Spec

anoa said:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Spec Core Team

Some chatter about the MSC2403 (knocking) as FCP has been proposed. Otherwise work on MSC1772 (Spaces) as that nears production and some encryption work with MSC2874 (single SSSS).

The team is carefully debilitating MSC2997, which spurred out of a conversation during FOSDEM2021. What sizes to include? What font to use? What colour do we paint this bike shed? All incredibly complex questions.

New Spec Platform

Will Bamberg has done an incredible job with getting both an entire redesign and an internal refactoring to the Matrix Spec. Remaining work includes ironing out some remaining minor blockers (https://github.com/matrix-org/matrix-doc/pull/3002, https://github.com/matrix-org/matrix-doc/pull/3003), merging changelog representation with respect to the current idea of a global Matrix Spec version number (https://github.com/matrix-org/matrix-doc/pull/2992) and then finally hooking up all the CI/CD to deploy it all (my job!).

We're planning to present the new spec on a new domain; running it in parallel with the old design for a bit before eventually retiring the latter.

2021-02-12-B00Xp-stacked_area_chart.png

Dept of Servers ๐Ÿข

CONSTRUCT

grin shared this note from Jason:

This week in Matrix Construct added Application Services support. I've been using the Mautrix Telegram bridge on a daily basis, and it's been a pleasure to develop with. Expect more bridges to be tested in the coming weeks. I'll also be working on completing the documentation for how to setup a bridge in Construct. Just to give some insight: The server has the ability to manage all of your bridges as sub-processes. In any client, just create a room for your bridge and enter its configuration there. Voila! The bridge's console appears right in that room and can even highlight you about errors!

Join us at #construct:zemos.net to learn more about bridges and lend a hand.

Jason

2021-02-12-eXJxK-Screenshotfrom2020-10-2504-23-40.png

Synapse

Synapse is a popular homeserver written in Python.

callahad announced:

We survived FOSDEM! Last week was relatively slower in Synapse land as we paused to catch our breath, but we do have news!

First, releases. Synapse 1.27.0rc2 is out now (changelog), and we anticipate a full 1.27 release early next week. This is basically the branch that FOSDEM ran, so we think it's a good one.

Second, keeping with our biweekly development cadence, the first 1.28 release candidate is due out midweek next week, so keep your eyes peeled for that. Given FOSDEM, most of our recent focus has been on stability, bug fixes, and social login. We hope to knock your socks off with more ambitious features soon. ๐Ÿ˜‰

Homeserver Deployment ๐Ÿ“ฅ๏ธ

YunoHost

Pierre told us:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.25.0 (1.26.0 available in branch testing)

Element Web integration had been updated to 1.7.16 (1.7.20 available in branch testing)

Dept of Clients ๐Ÿ“ฑ

SchildiChat Web/Desktop

qg offered:

SchildiChat is a fork of Element, which focuses on UI changes such as message bubbles and a unified chat list for both direct messages and groups, which is a more familiar approach to users of other popular instant messengers.

Thanks to @michi7801 we can now provide builds for macOS!! ๐ŸŽ

Code on GitHub: https://github.com/SchildiChat/schildichat-desktop

Matrix room: #schildichat-web:matrix.org

2021-02-12-5B1P9-Bildschirmfoto2021-02-12um10.48.41.png

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) announced:

We just landed basic support for edits. Which means edits are now applied to the original message instead of showing the fallback and you can edit your messages. You can also change a text message to a video (or the other way around) or change, who you replied to, but the latter fails to render in some clients properly, so be aware of that. Edits also send a secondary relation format, which makes handling edits easier, but I still haven't written up the MSC for that... The UI is not final yet and you can't show edit history yet, but apart from that it may or may not work fine.

There were also a lot of bugfixes like broken kerning in some cases, improved focus handling when clicking buttons in the timeline, Duplex devices should now work in calls, notifications now work on Haiku and a few more.

Now please stop asking for edits! :D

Cos discovers the /addwidget command in Element Web

Cos offered:

Public service announcement: Widgets are awesome. You can use /addwidget <url> in Element Web to add any web page in a iframe to the sidebar of Element Web. They get default name "Custom Widget" but you can rename them from /devtools menu. Very useful for group calendars, status pages and other info related to Matrix rooms.

Element

Neil announced:

Delight

  • Social Login

    • Weโ€™ve concluded most of the development on Social Login, and tested it last weekend on the FOSDEM deployments. We found that over 50% of new accounts used it to register! Weโ€™ll be deploying Social Login to matrix.org in the near future after polishing some final pieces on iOS.
  • Spaces

    • Weโ€™re iterating on the user experience for Spaces, first focusing on setting up, sharing and joining public spaces.

VoIP

  • VoIP branches merged to develop on iOS and & Android - please test voice & video calling if you have the android develop build, or testflight for iOS will be coming soon. More complex scenarios with multiple devices are particularly useful to test, eg. do other devices stop ringing when you answer on a different one?

Web

  • 1.7.21-rc.1 is now available at https://staging.element.io, including:

    • Fixed screen sharing in VoIP calls

    • Added window vs. full screen sharing option in VoIP calls

    • Fixed event permalinks to show link text instead of a room pill

  • 1.7.21 planned for release on Monday (15 Feb)

  • Element Desktop nightly builds are now using Electron 11

    • Please report any issues you notice so we can address them before the next Element release

iOS

  • 1.2.1 has been submitted to Apple. TestFlight should be available from tomorrow:

    • All new VoIP stuff described above

    • Handle User-Interactive Authentication for delete device and adding user 3pid.

    • Improve grace period management and setup Cross-Signing after login if possible when using SSO.

  • XcodeGen integration into the Element-iOS codebase is almost complete. From next week, we will not have merge conflicts on the pbxproj files. This also means better control on build settings.

Android

  • Initial sync work improvements: performance is far better regarding RAM usage and stability (especially true for large accounts), though duration is still a concern.

  • Started working on running integration tests via CI.

  • 1.0.17 released, fixing a join over federation bug. Expect 1.0.18 next week.

Dept of SDKs and Frameworks ๐Ÿงฐ

Ruma

Ruma is a set of Rust library crates around Matrix

jplatte offered:

In the last two weeks,

Dept of Ops ๐Ÿ› 

matrix-docker-ansible-deploy

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi told us:

Thanks to @Peetz0r, matrix-docker-ansible-deploy can now install a bunch of tools for monitoring your Matrix server: the Prometheus time-series database server, the Prometheus node-exporter host metrics exporter, and the Grafana web UI.

To install these services, follow our Enabling metrics and graphs (Prometheus, Grafana) for your Matrix server docs page.

Dept of Interesting Projects ๐Ÿ›ฐ๏ธ

SpaceHub

MTRNord offered:

This is a super early announcement.

As now Spaces exist and are kind of testable if you know how I went ahead to plan organizing my 200 rooms I currently have into spaces and realized soon that I will have A LOT of nested Spaces.

So I am now starting to write a tool to generate these spaces easily from a folder structure and exporting them.

You may ask why not a yaml or json format: Simply to more easily allow forking of existing collections. A folder structure is much easier to do a diff over in git than file content.

Feel free to join #space-hub:nordgedanken.dev for discussions or have an eye on https://github.com/matrix-spaces-generator where at the time of writing this only an example of the spaces folder structure exists.

Keymaker

MTRNord reported:

Small update time:

We are currently starting to add some documentation to it over at https://docs.joinmatrix.rocks/users/what-is-keymaker.html

It at time of writing still very barebones but over the next few weeks the goal is to add the Code of Conduct as well as a server admin guide on how to use this.

Dept of Guides ๐Ÿงญ

matrix-appservice-irc sprouts a book

Half-Shot said:

Hi folks! I'm happy to say that after many years of fielding IRC bridge support questions, the bridge team has now written the book on it. A markdown book! Using the latest mdbook tech we've written a draft of a usage guide and a server administration guide for the bridge and would value feedback on how we can make it better. This could be typos that I have strewn about it, or suggesting new content.

The draft is hosted at:

https://matrix-org.github.io/matrix-appservice-irc/latest

and the PR is:

https://github.com/matrix-org/matrix-appservice-irc/pull/1235

Final Thoughts ๐Ÿ’ญ

FluffyChat is a cute cross-platform matrix client. It is available for Android, iOS, Web and Desktop.

Shine reported:

If you have a 3D printer and want to express your love for Fluffychat with a nice badge, you can download the necessary files here ^_^ https://www.thingiverse.com/thing:4550331/files

Freecad source file included for those who need some modifications.

(Yes, it's older, but I never TWIM'd it ๐Ÿคฆโ€โ™€๏ธ)

sorunome shared:

2021-02-12-cPftU-img_20210208_160724.jpg

Dept of Ping ๐Ÿ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1envs.net347.5
2syn.steimel.info480
3matrix.xn--kll-sna.net491
4etke.cc682.5
5maunium.net692
6maescool.be708.5
7jupadr.de741
8matrix.godfreyhendrix.com1072.5
9shortestpath.dev1093.5
10queersin.space1180

That's all I know ๐Ÿ

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2021-02-05

05.02.2021 00:00 โ€” This Week in Matrix โ€” Ben Parsons

Matrix Live ๐ŸŽ™

It's a great week for FluffyChat! I chatted with the authors, Krille and Soru, to get the latest news.

Dept of Status of Matrix ๐ŸŒก๏ธ

FOSDEM 2021

FOSDEM is happening this year, regardless of pandemics or anything else! Actually FOSDEM in my experience is a huge super-spreader event even in normal years, so it may be refreshing to know that this year it will be fully virtual, and hosted using Matrix and Jitsi!

Matrix have several relevant talks happening:

Plus, there is a bonus talk from Brendan:

  • Mental health and free software Sunday @ 13:50 (in which Brendan talks about mental health issues in the context of working on free software, which may or may not relate to people, though certainly not me, pinging him at all hours to ask about customer integrations and features)

Plus, perhaps the most important part of FOSDEM: FOSDEM 2021 Matrix T-Shirt (Limited Edition) is now available

T-Shirt

Dept of Spec ๐Ÿ“œ

Spec

anoa announced:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

  • There were no new MSCs this week.

Spec Core Team

The Spec Core Team was focused on implementation and FOSDEM this week. Next week we'll be back getting through the backlog board.

2021-02-05-gpxy6-stacked_area_chart.png

Dept of Servers ๐Ÿข

Federation Tester

Neil Alexander told us:

federationtester.matrix.org has been updated today so that it now works properly with servers running TLS 1.3!

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timokoesters told us:

We're coming closer to correct federation every day! Currently we are reviewing and improving Devin R's MR and looking for mistakes, we will be giving it a try soon.

Meanwhile I worked on:

  • Push rule settings, making it finally possible to mute rooms

  • Appservice compatibility, I'm currently trying to get the mautrix-hangouts bridge to work

  • Optimizations, incremental sync in big rooms (rooms with >3k members) went from 1.3s to 13 ms, (100x improvement)

Thanks to Famedly and everyone who supports me on Liberapay or Bitcoin!

Synapse

Synapse is a popular homeserver written in Python.

callahad said:

FOSDEM is tomorrow! We're all very excited to be powering the fosdem.org homeserver, and we hope you'll join in over the weekend.

Earlier this week we released Synapse 1.27.0rc1, though the final release is currently blocked on resolving a regression in Synapse 1.26 (#9264) which could cause logins to fail if your public_baseurl was unset in your Synapse configuration. The workaround is to ensure that value is correctly set for your deployment.

Otherwise, we're quite excited to get 1.27 out the door, as it:

  • Solves an issue (#9172) where federation senders could take many minutes to catch up after restarts.

  • Contains countless improvements to our support for social login methods, many of which will be live for FOSDEM users this weekend.

We've also included the usual host of bug fixes and Admin API additions, but more on those when the actual release happens. You can preview the release notes on GitHub.

See you at FOSDEM!

Dept of Bridges ๐ŸŒ‰

mautrix-imessage

Tulir announced:

My iMessage bridge is starting to be usable now. Text and files work in both directions and tapbacks and replies work iMessage->Matrix. Currently it only supports running on a Mac, the iOS option will come later. The bridge doesn't require opening ports to the Mac, it uses a simple proxy that turns the usual HTTP appservice transactions into a websocket instead.

The bridge itself can be found at https://github.com/tulir/mautrix-imessage and the websocket proxy is at https://github.com/tulir/mautrix-wsproxy. The Matrix room is #imessage:maunium.net

mautrix-signal

Tulir told us:

I made the first release for mautrix-signal. No particular reason, but it's v0.1.0 now. Recent changes include:

  • Better profile syncing (Signal profile names should sync when restarting)

  • Support for Signal profile avatars

  • Support for bridging stickers from Signal

Gitter

Eric Eastwood said:

FOSDEM is happening this weekend! Come see our FOSDEM talk in the Real Time Communications track on Saturday around how we made Gitter speak Matrix and how you can add Matrix support to your own app: https://fosdem.org/2021/schedule/event/matrix_gitter/

*--

We're also working on MSC2716 to support backfilling the massive archive of historical messages on Gitter over on Matrix. We're prototyping the MSC on Synapse and have a suite of Complement tests to verify that the homeserver implementation is working according to the specification.

The DAG visualization (made with TARDIS) shows that we're inserting messages 1-3 between A and B:

2021-02-05-qytMG-106591475-f094ce00-6513-11eb-8022-d9df1a229edc.png

matrix-appservice-irc reaches 0.24.0-rc1

Half-Shot said:

Another release! Wow. We're also testing 0.24.0-rc1 which has quite a few big ticket items this week. Off the top of the list is a new command to !plumb rooms dynamically in admin rooms, provided you have permissions in the bridge (happy days self hosters!). We've also added the feature to warn if your messages don't get through in your admin room. Finally, there are a bunch of small bug fixes and one large one that should make the bridge much more resistant to netsplits, so that it should recover in a much safer way should it get hit by a ton of traffic. Please test, and if all goes well the formal release will be out next week!

https://github.com/matrix-org/matrix-appservice-irc/releases/tag/0.24.0-rc1

matrix-appservice-slack hits 1.7.0-rc1

Half-Shot announced:

Happy friday bridge users! Today we're releasing 1.7.0-rc1 of the Slack bridge which contains some more critical encryption fixes for our experimental encrypted rooms support. We've also fixed a few serious bugs where DMs would not be persisted to the database in some cases. Please test, and let us know how you get on!

https://github.com/matrix-org/matrix-appservice-slack/releases/tag/1.7.0-rc1

Dept of Clients ๐Ÿ“ฑ

FluffyChat

FluffyChat is a cute cross-platform matrix client. It is available for Android, iOS, Web and Desktop.

krille announced:

FluffyChat has been approved for the iOS AppStore! Thanks to benpa and matrix.org for the help with this! You guys are awesome!

https://apps.apple.com/de/app/fluffychat/id1551469600โ€‹

Fluffychat F-Droid

FluffyChat is a cute cross-platform matrix client. It is available for Android, iOS, Web and Desktop.

MTRNord told us:

Fluffychat Fdroid now also has a nightly repository: https://fluffychat.im/en/fdroid.html

It gets a new update on each commit, offers fast access to new features, but at the same time is less stable than the normal fdroid repository.

Check it out at: https://fluffychat.im/en/fdroid.html

In case of issues open a issue at: https://gitlab.com/famedly/fluffychat/-/issues To directly try it use: fdroidrepos://nightly.fdroid.fluffychat.im/?fingerprint=21A469657300576478B623DF99D8EB889A80BCD939ACA60A4074741BEAEC397D as a direct way of adding the repo if you have fdroid already installed

Syphon updated

ereio said:

It's been a little over 6 months since I've posted an update for Syphon in TWIM, but a lot has changed since then! There

have been several releases since and a lot of groundwork to get the client further out of Alpha. We've also had people start to contribute in various ways, including with PRs, and I can't thank people enough for all their hard work and support.

Big thanks to TR_SLimey whose contributions went out in this release! All that said, I'll try to give updates here more often.

Version 0.1.6 was just released this week and includes:

New Features:

  • SSO Login & Signup

  • Reactions / Emoji Support ๐ŸŽ‰

  • View Edited Messages (Condensed)

  • Swipe To Reply

  • Cross Server Room/Topic Search

  • Forgot / Reset Password Flow

Performance

  • More caching and cold storage saving/loading improvements

  • Parsing all matrix data from separate threads

Bug Fixes

  • fixed e2ee decryption issue - cache related

  • fixed read receipt bugs

  • fixed various caching performance issues

  • fixed failed syncing on most self-hosted homeservers

  • fixed settings permanence issues

  • fixed new direct chat with invalid user_ids

Since it's been a while, if anyone has any feedback, questions, or concerns, feel free to chat in the #syphon:matrix.org room, message me @ereio:matrix.org, or email me at [email protected].

https://github.com/syphon-org/syphon

If you'd like to help out with the project in any way, just ask. We'd love to have your support ๐Ÿ˜ƒ

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

red_skyโ˜„๏ธ offered:

Nheko now has an optional privacy screen! The timeline will optionally blur when the nheko window loses focus. The timeout for when the blur effect should be applied is configurable in a number of seconds between 0 and 3600 (1 hour).

Nico (@deepbluev7:neko.dev) added:

Cheers, some update probably mentioned it already, but FOSDEM is near. Make sure to check out the Matrix stand! RedSky and I (Nico) will probably be there at some random times as well as other cool people from the Matrix-verse! We even made a small promo video about Nheko!

In other news, lorendb extended the drop zone for media to the full timeline, the privacy screen got merged, some more emojis should now display properly, you can now change your avatar and displayname globally or per room and clicking a matrix link in your browser should now actually open the link in Nheko (we were missing the %u in the .desktop file).

That's all, see you at FOSDEM!

Frost

fabian offered:

I have been learning about matrix the last few months. The result of that is frost a very basic GUI matrix client written in Go using Gio and a CS library. Highlights of the project so far are:

  • Works on Linux, Windows, macOS and Android (although I only test on Linux and often break the other platforms)

  • Should work on WASM and iOS with not too much effort

  • Very, very limited E2EE support. (If the session is manually verified by other clients, events can be encrypted and decrypted.

  • Library that handles storage, encryption and some other things with a opinionated but simple to use (I hope) interface.

I started this project to understand matrix and improve my Go skills (which are not that great) and if somebody wants to play around with it and give me some feedback on it, I would very much appreciate it. Meet me in #frost-dev:intothecyber.space

Fractal

Alexandre Franke reported:

Former GSoC intern and long time Fractal contributor Julian Sparber started on Monday working full time on Fractal thanks to a grant from NlNet. He is funded for the next six months and weโ€™re looking forward to all the amazing work he will do. You can read more details on his blog.

and then, Alexandre Franke reappeared:

Apart from that grant, weโ€™ve done a fair chunk of work since the previous piece of news two months ago. Here are the highlights:

Donโ€™t believe me? Well, go try out our nightly flatpak build!

2021-02-05-hedvP-image.png

Konheko

Nico (@deepbluev7:neko.dev) said:

I've updated my Sailfish client to the API changes in the latest Sailfish early access release. You will need to install the new version on Sailfish 4.0.1.45, since some private APIs got removed. No functional changes though (I hope).

NeoChat

Carl Schwan reported:

NeoChat is continuing to make progress. First of all Tobias, Nicolas and Hannah have been working on packaging NeoChat for many platforms. We now have nightly builds for Android, Windows, macOS, AppImages and Flatpak. There is still work to be done to improve the quality of the builds and fixing some platform incompatibility bugs. Nicolas has also been working on optimizing the performance and the RAM usage, fixing a few bottlenecks. Tobias finally landed his big revamp of the login flow and we now support login using single sign-on. Myself (Carl) worked on E2EE support in NeoChat and Quotient. This week we finished our Qt bindings for libOlm and started adding device key tracking and more!

2021-02-05-iNCYz-image.png

Dept of Bots ๐Ÿค–

Timetracking Bot

MTRNord offered:

3 Hotfixes for the timetracking bot:

v0.4.1

Fixes

  • Make sure !version correctly gets build on CI

  • Make sure we compare to local UTC::now

  • Check for arg len in stats command

  • Sanitize input arguments before parsing them

Refactoring

  • Use https://github.com/MTRNord/mrsbfh framework

v0.4.2

Fixes

  • Make tests compile again

  • Descriptions without explicit time now accept again more than just the first word

Features

  • !stats assumes out now if out is still missing for the last entry

v0.4.3

Fixes

  • Ensure that the tests compile always reliable

Hemppa

Hemppa the Bot is a multipurpose bot for writing modules super easily in Python.

Cos reported:

Hemppa the bot is a general purpose bot for writing modules as easily as possible in Python. This week Hemppa got new module that can be used for hackerspace asset management using Github issues and special labels. In practice it can figure out which machines are working or broken and also list issues per physical space. Creating issues with proper labels and other extra features to come soon. https://github.com/vranki/hemppa / #hemppa:hacklab.fi

Dept of Interesting Projects ๐Ÿ›ฐ๏ธ

MatrixFS

Ananace told us:

Since there was talk about it, I went through and updated / cleaned up the MatrixFS code a bit; Got it working with tail -f (and probably other applications that poll based on mtime/size changes), added a GC that drops unaccessed data from memory after a while, and reduced the communication overhead slightly

Matrix in the News ๐Ÿ“ฐ

Element Android: interesting times

Element Android had a fun time last weekend, getting temporarily removed from the Play Store. Google were somewhat responsive in getting it back up though, and overall Matrix received a nice boost of attention from the tech press as a result!

Gizmodo, ArsTechnica, Engadget, Next INpact, Android Police and Protocol all posted sympathetic coverage of the situation.

Dept of Sadness โ˜น๏ธ

Daydream

MTRNord said:

After a long time of no message from it I am officially ending that project.

The Code probably will stay up but it will not get revived by me. Feel free to fork it however.

Sourcecode can be found at https://github.com/daydream-mx/

Dept of Ping ๐Ÿ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1vdhlogintest.matrix.org377.5
2nicoll.xyz429
3envs.net431
4mchus.pro569
5weimann.digital624
6kif.rocks689
7fje.cz696
8aria-net.org696
9fairydust.space724
10jel-tech.com797

That's all I know ๐Ÿ

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2021-01-29

29.01.2021 00:00 โ€” This Week in Matrix โ€” Ben Parsons

Matrix Live ๐ŸŽ™

Dept of Status of Matrix ๐ŸŒก๏ธ

Christian Bruchatz created this map of currently live Matrix instances at German-ish universities!

The map is growing... after being published, more and more universities signal they also use Matrix... https://doc.matrix.tu-dresden.de/images/federation_map.svg within https://doc.matrix.tu-dresden.de/en/why/

2021-01-29-YevH--image.png

Dept of Spec ๐Ÿ“œ

Spec

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

New MSCs:

  • There were no new MSCs this week.

Spec Core Team

As announced last week, we're now using the Spec Core Team Backlog board to communicate progress. Check it out if you haven't seen it yet, and ask questions in #matrix-spec:matrix.org if you have any ๐Ÿ™‚

2021-01-29-QpAp6-stacked_area_chart.png

Dept of Servers ๐Ÿข

Synapse

Synapse is a popular homeserver written in Python.

callahad announced:

One week 'til FOSDEM!

In addition to several talks specifically about Matrix, Synapse's own Brendan Abolivier will be speaking in the Community devroom on Mental health and free software: What I wish Iโ€™d been told before I got into free software, and more. Check it out!

We're also excited to announce Synapse 1.26.0's release! As mentioned last week, the two biggest new features are a faster algorithm for state resolution and initial support for enabling multiple OpenID Connect providers. Additionally, we've sped up redaction in large rooms and made it possible to run more APIs on workers. See the full changelog for details.

We expect to publish a release candidate for 1.27.0 early next week, but we're trying to be conservative about large changes ahead of FOSDEM, so knocking โ€” which is coming! โ€” will have to wait until 1.28.0... ๐Ÿšช

Dendrite 0.3.8

Dendrite 0.3.8 was released this week! Simple changelog this time:

A well-known lookup regression in version 0.3.7 has been fixed

Homeserver Deployment ๐Ÿ“ฅ๏ธ

Kubernetes

Ananace announced:

Just released the 1.26.0 version of the K8s-optimized image and chart.

(Not sure how this message will do though, been down the last few days for a database upgrade)

Success: the message did great!

Dept of Bridges ๐ŸŒ‰

matrix-appservice-bridge arrives at 2.5.0-rc1

Half-Shot announced:

Hey folks! It's been a while since we've had an update but things are always manic at bridge HQ. This release is a bit of a whopper containing quite a few bugfixes (including improvements to the experimental encrypted bridge support). We've also updated to Typescript 4.1 which is a bit stricter on those promise return types. Finally, you can now create intents on the bridge and use it's components without starting the HTTP listener which has been a longstanding issue for as long as I've been maintaining.

So go out there and test this release, all going well we should ๐Ÿšข early next week!

If only you knew what else this guy was working on. More to be shared soon-ish. I hope!

Dept of Clients ๐Ÿ“ฑ

FluffyChat

FluffyChat is a cute cross-platform matrix client. It is available for Android, iOS, Web and Desktop.

krille said:

FluffyChat 0.26.1 has been released today with updated translations, support for Unified Push (https://github.com/UnifiedPush/) and some bug fixes. The Linux Desktop version is fully functional again and the app is much more stable than ever now. We are working hard to bring FluffyChat to the official iOS AppStore next week! :-)

I reinstalled Fluffychat this week as I attempted to support some less-technically-minded friends, and was VERY impressed. If you've tried FluffyChat in the past but missed recent versions I suggest you have another go.

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) said:

On Wednesday we released version 0.8.1 of Nheko! It's mostly a bugfix release of 0.8.0, but it does bring some new features here and there:

[0.8.1] -- 2021-01-27

Features

  • /plain and /md commands to override the current markdown setting. (contributed by lorendb)

  • Allow persistent hiding of rooms with a specific tag (or from a community) via a context menu.

  • Allow open media messages in an external program immediately. (contributed by rnhmjoj)

Improvements

  • Use async dbus connection for notifications. (contributed by lorendb)

  • Update Hungarian translations. (contributed by maxigaz)

  • Update Finnish translations. (contributed by Priit)

  • Update Malayalam translations. (contributed by vachan-maker)

  • Update Dutch translations. (contributed by Glael)

  • Store splitter size across restarts.

  • Add a border around the completer. (contributed by lorendb)

  • Request keys for messages with unknown message indices (once per restart, when they are shown).

  • Move the database location to XDG_DATA_DIR. (contributed by rnhmjoj)

  • Reload the timeline after key backup import.

  • Autoclose completer on space, when there are no matches.

  • Make completer only react, when the mouse cursor is moved.

Bugfixes

  • Fix unhandled exception, when a device has no keys.

  • Fix some cmake warnings regarding GNUInstallDirs.

  • Fix tags being broken. If you have no tags showing up, you may want to logout and login again.

  • Fix versionOk being called on the wrong thread. (contributed by Jedi18)

  • Fix font tags showing up in media message filenames.

  • Fix user profile in dark themes showing the wrong colors. (contributed by lorendb)

  • Fix emoji category switching on old Qt versions. (contributed by lorendb)

  • Fix old messages being replayed after a limited timeline.

  • Fix empty secrets being returned from the wallet breaking verification.

  • Make matrix link chat invites create a direct chat.

  • Fix focus handling on room change or reply button clicks.

  • Fix username completion deleting the character before it.

Flathub and other repo updates may take a bit, but in the mean time you can try building it yourself or one of our provided stable downloads. Thank you for all the feedback so far and I'm glad a lot of people seem to be enjoying it so far! All the feedback, bugfixes and issues you provided certainly will help improving Nheko for everyone!

Then, later...

Apart from the current releases, we have a few things cooking right now:

Edits

Yes, I have tried to avoid them for long, but I am finally working on it. It's a bit different to how Element and others do it, but it also generates a fallback to be compatible with the current MSCs where possible. I've grown a bit tired of having to explain myself everytime, why I did not want to support edits (yet), so I'm experimenting with a few solutions now. Expect a blog post summarizing those experiments in the future and maybe an MSC.

Polish

Jedi18 and LorenDB are both busy polishing up the rough corners in Nheko, like finally being able to set your username and avatar in the UI as well as being able to copy text via right click, jdenticons and other fancy stuff. We also had a few contributions in the last release, which should improve the UX a lot.

GSoC

Furthermore we will be participating in GSoC (if matrix.org gets accepted) and will probably mentor a student, that implements a few of the smaller features, which users may be missing after using Element for long.

I think that is all I wanted to share. I hope you enjoy the release and if you have any issues or feature requests, tell us in #nheko:nheko.im or open an issue on Github!

Quaternion

kitsune announced:

Since the project is already fairly rich in features, I decided that it's timely (some would say - way overdue) to bump up the version a bit. In that light, the next Quaternion beta is numbered 0.0.95 beta 3 (no four-number version any more) and it's out now! Release notes list notable changes since (0.0.9.5) beta 2, with two most significant being ability to enter messages in Markdown and the overhauled timeline that allows you to easily discern between already read and yet unread messages and also quickly scroll to the read marker.

2021-01-29-KtwNn-Screenshotfrom2021-01-2621-15-53.png

Element

Neil reported on behalf of the teams:

Delight

  • Social Login

    • Final polishing and bug hunting across all platforms. We are targeting 5 options to begin with (Gitlab, Github, Facebook, Google and Apple) and will hopefully be ready to start setting these up on homeservers next week.
  • Spaces

    • Lots of polishing on Element web, you can get all the latest in the matrix live demo session next week!

VoIP

  • Added some debugging to web to help debug call connectivity failures

  • Fixed a compatibility regression in web with voip v0 clients (ie. element android / ios) - dโ€™oh!

  • Android: work on getting call audio routing correct on various different devices

  • Design coming up to speed to support on implementation and ongoing UI improvements

  • iOS on holiday

Web

  • Off cycle 1.7.18 release for VoIP compat bug

  • Various tweaks to prepare for FOSDEM

  • Element Web 1.7.19-rc.1 is now available at https://staging.element.io, including:

    • Allowed guest users to see widgets

    • Standardised security terminology to reduce confusion

    • Added ability to pin widgets for everyone in the room

iOS

  • We made several iterations since the last App Store release (1.1.3) but Element-iOS 1.1.6 is now in the store.

  • We made some improvements to use less RAM in the background sync module that manages push notifications.

  • We continued to improve performance in E2EE to speed up message sending using pre-sharing keys and re-sharing keys methods. Element-iOS now automatically rejects share requests from not verified devices. It does not send anymore such requests if it is not verified.

Android

  • We are working on improving the initial /sync management. The first objective is to reduce RAM usage. Then, we will improve the time to display the room list.

Dept of SDKs and Frameworks ๐Ÿงฐ

Ruma

Ruma is a set of Rust library crates around Matrix

jplatte told us:

In the last two weeks,

Also, @jamtwister imported a bunch of endpoints from @florianjacob's synadminctl into synapse-admin-api.

State Resolution

It's been a really long time since an update on ruma/state-res but work has continued!

  • Thanks to a push from @gnieto the state-res crate no longer has its own PDU type, we are now using a trait.

Conduit continues its use of state-res, which has helped workout the kinks and buggy bits.

Dept of Interesting Projects ๐Ÿ›ฐ๏ธ

Keymaker

MTRNord told us:

Just a small dev update on things that currently happen over at the Serverlist project "Keymaker" and its companion bot.

Follow it in detail at #serverlist:nordgedanken.dev or https://github.com/keymaker-mx/

  • The registration bot finally is starting to get its admin backend -> The registration probably happens in a few weeks from now

  • Keymaker has now a admin login page (using matrix as auth provider) to allow a web UI for admins to manage servers (WIP)

  • The bot successfully writes servers upon registration to the database. This is the main task of the bot besides the capability to administrate the page using the page

  • The bot now is able to be used by admins in an admin room which also gets notifications if a user registered a new server.

Matrix in the News ๐Ÿ“ฐ

Beeper has continued to make headlines, and is now being reported around the world:

Andy announced:

Beeper was mentioned in one of the two biggest argentinian newspapers! ๐Ÿ˜ฎ๐Ÿ™Œ https://www.lanacion.com.ar/tecnologia/todo-en-uno-este-es-beeper-el-chat-universal-que-promete-usar-whatsapp-signal-y-telegram-en-un-solo-nid22012021/

There was a pause, as we wondered how the other biggest Argentinian newspaper would react, and it was worth the wait! Andy soon announced:

And now the other biggest newspaper of Argentina wrote an article about Beeper (Matrix is mentioned about half-through the article)!

https://www.clarin.com/tecnologia/beeper-app-mensajeria-unifica-chats-whatsapp-telegram-signal-instagram_0_1bX5iXnxw.html

Anything that goes in Argentina is sure to make waves in Poland, and so it was that Polish media picked up on the story.

TR_SLimey reported:

Beeper has made it to Polish media too, guys :P

https://spidersweb.pl/2021/01/beeper-imessage-na-androidzie.html

The title: "A new multi-messaging platform has arrived. It's creators will send you an old iPhone so you can even use iMessage"

Final Thoughts ๐Ÿ’ญ

Discussing the sharing of code and ideas with Tobias from NeoChat, Nico (@deepbluev7:neko.dev) said:

There really is no stealing, it's more helping each other out and improving things for everyone, users and developers alike. We are not corporations competing, we are one free software community :3

Which I thought was very astute, and decided to include here.

Dept of Ping ๐Ÿ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1imninja.net428
2envs.net437
3helderferreira.io557
4casavant.org605
5aria-net.org635.5
6chat.bobrtc.tel723.5
7feneas.org724
8nuclearlemons.uk805
9t2l.io855
10matrix.sp-codes.de885

That's all I know ๐Ÿ

It's FOSDEM next weekend, and we are manically preparing for it. Should be a good one!

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.26.0 released

28.01.2021 00:00 โ€” Releases โ€” Dan Callahan

Synapse 1.26.0 is now available!

Note: This release includes a new database schema version. If you need to roll back to Synapse 1.25.0, you will also need to follow the associated database downgrade instructions.

In addition to a truckload of refactoring and general improvements, Synapse 1.26.0 includes three major new features:

  1. A brand new algorithm for calculating the auth chain difference, which should dramatically improve worst case performance during state resolution (#8622).
  2. Initial support for enabling multiple OpenID Connect providers, paving the way for proper multi-provider social login workflows.
  3. A significant speed-up to redaction performance in large rooms.

It also brings several improvements to Admin APIs:

We've also made it possible to offload several additional APIs to worker processes, including read receipts and account data persistence, further improving Synapse's scalability.

See the full changelog for more.

Lastly, a reminder: we have deprecated Python 3.5 and PostgreSQL 9.5 and will cease support at the end of March. Due to deprecations in our Python tooling, we were unable to produce a binary package for Ubuntu 16.04 LTS (Xenial) in time for this release. We have resolved this for 1.27.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including 0xflotus, chris-ruecker, dklimpel, emelie-qis, jerinjtitus, and tzyl.