This Week in Matrix 2021-06-04

04.06.2021 20:37 — This Week in Matrix Ben Parsons
Last update: 04.06.2021 19:10

Matrix Live 🎙

Dept of Status of Matrix 🌡️

Wired UK feature article

Wired UK have published a feature on Matrix in their print edition this month. We'll be sure to link to it when it's made available online!

Wired Article

German-universities poll

jfkimmes shared:

I just learned that in a poll of 89 universities in Germany, Matrix ranked third place in the chat category already.

The source is in only available in German, unfortunately: https://zenodo.org/record/4817795

However, the conclusion list (first table) may be understandable from context. It lists the top three solutions per category with their respective number of universities using it.

Oleg clarified:

The evaluation was "which solution are you using".

Florian added:

The Instant Messaging part starts at slide 15, the first chart on that slide is "which solution do you use", the second is "How content are you with the solution?", with Matrix having the best average of all solutions, namely ~8.8/10.

also JCG:

What's also noteworthy: Those using matrix are the happiest with the solution

\o/

Dept of Spec 📜

Spec

anoa offered:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.

MSC Status

New MSCs:

MSCs with proposed Final Comment Period:

  • No MSCs entered proposed FCP state this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

Merged MSCs:

  • No MSCs were merged this week.

Spec Updates

This week the Spec Core Team has been reviewing various Spaces MSCs, most recently MSC3230 (Space ordering). We're also hoping to square away the aggregations MSCs (message editing, reactions, etc) once and for all, though this will likely take a concerted effort from a few members to pull off.

Finally MSC3231 is a (currently draft status) MSC from Callum, one of Matrix.org's GSoC students this year! His project aims to allow native token-based registration to homeservers (the idea is so that you can generate a few tokens from your registration-disabled homeserver and hand them out to a few trusted friends and family members).

And finally, work still continues on finishing up the technical portions of the new release process for the spec. As mentioned last week, we've attempted to split the work up over multiple people in order to get it done quicker. Slowly but surely...

2021-06-04-I-r8I-stacked_area_chart.png

Dept of Servers 🏢

Synapse

Synapse is a popular homeserver written in Python.

callahad said:

Hello from the Synapse team! A short update for a short workweek (thanks, bank holidays!):

  • Synapse 1.35 was released this week! The Spaces flag is on by default, a bunch of bugs were fixed, and we've landed many of the prerequisites to eliminating RAM spikes on room joins.

  • 📚 We have new docs! 📚 Anoa converted our docs to build with mdbook (#10086), and you can now browse them at https://matrix-org.github.io/synapse/! Check it out and let us know what you think. (Note: Not all of the pages have been converted from reStructuredText to Markdown yet, so some might render a bit strangely, but the structure is there!)

Catch you next Friday! 👋

2021-06-04-bILM1-image.png

synapse-media-proxy

f0x announced:

Another round of updates on my smart caching media proxy. After refactoring a lot (as always), I implemented thumbnailing! Now the only big feature left to add is url previewing. I also have a test deployment configured on media.pixie.town now, so you can try fetching a bit of remote media through there, or view this submissions screenshot

metrics

Got started implementing a Prometeus /metrics endpoint, with a rudimentary Grafana dashboard for my test installation.

comparison with matrix-media-repo

While they both implement Matrix' media endpoints, they serve rather different niches, where matrix-media-repo fully decouples the media repo aspect, my proxy cooperates with Synapse's filesystem and database, to speed up operation while ultimately making it a seamless drop-in and removal process.

also :P

2021-06-04-WnfPb-image.png

Homeserver Deployment 📥️

Kubernetes

Ananace offered:

And another weekly installment of Kubernetes Helm Chart (and deprecated Docker image) updates, tracking the Synapse releases this week (1.35.0/1).

Dept of Bridges 🌉

Heisenbridge demo video

hifi shared this great demonstation video of Heisenbridge:

Half-Shot hit us late with a pair of updates:

Security release for the matrix-appservice-irc and matrix-appservice-bridge library

Hello. This week we've released an update to the https://github.com/matrix-org/matrix-appservice-bridge/ library containing a security fix for room upgrade handling. The security report will come later, but for now we advise anyone using the room upgrade handler feature to upgrade to 2.6.1. By the same token, we would also advise all IRC bridge admins to update their bridge to 0.26.1.

The Libera.chat bridge is still ongoing

Howdy folks. As you've likely seen over the last few days, we're still hard at work getting the final pegs in place for the libera.chat bridge. As usual, you can start using the bridge now while it's in beta by going to #<foo>:libera.chat, but we're hoping to have the thing stable by next week. Catch us in #libera-matrix:libera.chat for the juicy gossip about it.

matrix-puppeteer-line

Fair reported:

matrix-puppeteer-line: A bridge for LINE Messenger based on running LINE's Chrome extension in Puppeteer.

  • Send a bridge notice when getting unexpectedly logged out of LINE, to warn you to log in again.
  • Improve resiliency of LINE user avatar syncing.
  • Properly support syncing LINE rooms with participants who aren't in your LINE friends list (This was harder than it sounds...!)

These changes (and ones before it) will be merged to master once I reorganize some messy commits.

The next big task is still to fix outbound read receipts (i.e. to make it so that the bridge syncing a message doesn't make your > LINE contacts think you actually read that message). Once that is done, I'll consider the bridge to be in beta.

Discussion: #matrix-puppeteer-line:miscworks.net Issue page: https://src.miscworks.net/fair/matrix-puppeteer-line/issues

Dept of Clients 📱

Thunderbird Matrix support

freaktechnik announced:

Thunderbird now has Matrix support based on matrix-js-sdk enabled in the Nightly builds.

The star feature is probably that we support multiple Matrix accounts in the same client. Right now all your unencrypted rooms with text messages should work fine. While we think we won't destroy your account's state, it's still recommended to use a testing account with it. During account setup, it will ask you for a password, even if the homeserver supports SSO. If you intend to log in through SSO, just leave the password field blank.

We're not quite at the point where we support all the things you love about chatting with Matrix. Many of the missing features and polish to make communication successful are tracked in this meta bug. The goal for that milestone is to enable Matrix in our Beta builds.

You can get a Thunderbird Nightly build at the bottom of thunderbird.net by switching from "Beta Channel" to "Nightly Channel". If you run into bugs with the Matrix integration, please report them through this form. When filing a bug, please include debug logs. You can copy the debug logs for the account by going to the "Show Accounts" dialog, right clicking the account and selecting "Copy Debug Log". Note that the debug log may contain information from any of your conversations, so you might want to check the contents before posting it anywhere.

Also, check out Matrix Live!

NeoChat

Carl Schwan announced:

NeoChat 1.2, our third major release, was released this week bringing many improvements to the timeline and text input component. If you missed it, you can read the announcement here: https://carlschwan.eu/2021/06/01/neochat-1.2-bubbles-better-text-editing-and-more/ and we even have a nice release video :) https://www.youtube.com/watch?v=4lcH4tm6uTk

Other than that, we started working on an integration with KDE web shortcuts functionality to quickly search selected text on the web: https://invent.kde.org/network/neochat/-/merge_requests/279.

2021-06-04-KGrID-image.png

Nheko

Nico (@deepbluev7:neko.dev) told us:

Callum, our GSoC student, after spending some time on Synapse, had now his first go at Nheko's codebase. He implemented, that you can now just enter the server name on registration instead of the full URL. This means entering conduit.rs or matrix.org works now nicely, since those servers are actually hosted at a different URL. He's now working on the Token Registration MSC, which he will implement in Synapse and Nheko, so exciting times ahead!

We also had a small contribution from pcworld, who fixed that if you only viewed the room list in the narrow layout, you would not get notifications for the last selected room.

I'll leave you with some words, that you may have heard a few times already: "Watch this space for next weeks update!"

Fractal

Alexandre Franke reported:

A dozen merge requests have been integrated in our fractal-next branch since last week.

Amongst the more trivial changes, Julian made sure rooms are added to the sidebar in batch (!737) to improve performances, added in-app notifications for invite errors (!760), added a menu entry to leave rooms (!769), and implemented display of user and room avatars (!770). We also gained a right-click-menu entry to display event sources thanks to Kévin (!766).

Element Clients

Updates provided by the teams.

Delight team

  • We’re continuing progress on implementing Blurhash on Web & Android to improve the image loading experience, especially on low bandwidth
  • On Spaces, we’ve started working on the ability to drag and drop to re-order Spaces, along with improving adding aliases to public Spaces

Web

  • 1.7.30 RC on staging
    • Improved layout performance in the timeline and room list
    • Refined the message action bar UI
  • Continuing to improve application performance
    • Recent focus on minimising browser layout work when things change
    • Reducing DOM size
  • Working on Apple silicon desktop builds

iOS

  • 1.4.0 is available on the public TestFlight. We expect to make it available on the App Store on Monday. It has:
    • Performance improvements
    • Crash fixes
    • New languages: Esperanto, Portuguese (Brazil), Kabyle, Norwegian, Swedish, Japanese and Welsh.
    • There are some API breaks in MatrixSDK due to those performance improvements.
    • We have now a MXLog module with log levels! It is now possible to disable all logs from MatrixSDK
  • We continued to work on performance and stability and will continue to for the coming sprint period: https://github.com/vector-im/element-ios/milestone/55

Android

  • 1.1.8 has been released to production, and 1.1.9 has been released to beta on the PlayStore
  • We are currently working with the design team on the light and dark theme of the application, especially colors and text appearance. Lots of cleanup to do...

Hydrogen

A minimal Matrix chat client, focused on performance, offline functionality, and broad browser support. https://github.com/vector-im/hydrogen-web/

Bruno announced:

Released Hydrogen 0.1.56 this week, with redactions. In the meantime, I've been making good progress on reactions, which should hopefully get released early next week. Midhun has made good progress on the right panel, ironing out the last bugs.

Here's a sneak preview of reactions (with slow network to show off the local echo animation):

2021-06-04-0GlkZ-hydrogen-reactions-preview.gif

kazv

tusooa reported on kazv:

kazv is a matrix client based on libkazv. Talk to us on #kazv:tusooa.xyz.

Updates

I guess it's a long time from our last twim. Here's what is going on in that time:

  • We used fluent for translations. https://lily.kazv.moe/kazv/kazv/-/merge_requests/1
  • We supported read and save client state. https://lily.kazv.moe/kazv/kazv/-/merge_requests/2
  • A work-in-progress, but we are displaying some common event types; there are even chat bubbles (>w<) Check out a screenshot below: (yes, and we got a new logo)

kavz

Dept of Events and Talks 🗣️

Matrix @ FrOSCon this year

Oleg said:

On August 21-22 the annual Free and Open Source Conference (short FrOSCon) will take place. Usually the conference takes place in a German University of applied Sciences Bonn Rhine Sieg. This year it will be virtual. On the positive side - we don't need to travel.

As German Matrix community grows this is a great opportunity to meet each other and hack together.

Matrix Dev Room

We are planing to do a virtual Dev Room this year. The idea is to exchange on the latest Matrix development and projects, get to know each other and drink <your_favorite_beverage> (virtually) together. 😉

To make it happen we need your help!

Dev Room is living from talks and workshops - this is your chance to present your Matrix project or to do a workshop!

Language: preferably German, but English is also ok

Submission is until 2021-06-11, but please give us feedback ASAP so we can create a plan now.

If it's your first talk or workshop some free of charge coaching is included. 😉

Also help in organizing the Dev Room (moderation, timekeeping) is needed.

Matrix Open Source booth

It was a great place to chit-chat and to get your in-depth answers regarding Matrix at FOSDEM this year. 👍️

We also planing to have a virtual booth at FrOSCon.

We need your support in answering questions about Matrix or just to have a good time.

Get in touch

If you want to take part please contact @oleg:fiksel.info (or [email protected]) ASAP to add you to the Dev Room participants list.

BTW: we also have a #FrOSCon:fiksel.info room

Dept of Interesting Projects 🛰️

Circles

cvwright reported:

  • Set up a new (virtual) EU homeserver in Frankfurt, so European folks can join the beta starting next week

  • Use StoreKit to detect when a user is in Europe and should therefore use the EU homeserver, both for GDPR and to minimize latency

  • Made some progress toward supporting m.video messages

  • Experimented with using the new iOS photo picker to better protect users' privacy Homepage: https://kombuchaprivacy.com/circles/ Source code: https://github.com/KombuchaPrivacy/circles-ios

You can also support circles on Kickstarter.

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1lama-corp.space492
2trolla.us526
3int21.dev569
4fosil.eu727
5d0.ee728
6nordgedanken.dev740.5
7feneas.org805.5
8maescool.be1073.5
9matrix.sp-codes.de1077
10coffespot.com1490

#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1dendrite01.fiksel.info1152
2dendrite.s3cr3t.me1386
3weber.world8297.5

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.35.0 released

01.06.2021 00:00 — Releases Dan Callahan

Synapse 1.35.0 is out! This release focused on improving internals as we drive toward better memory performance during room joins, but more on that below.

Update: Synapse 1.35.1 was published on Thursday, June 3rd. It resolves a bug (#10109) which mistakenly listed invite-only rooms in the Spaces summary.

We'd also like to call the attention of client developers to a deprecation: The unstable prefixes used during development of MSC2858: Multiple SSO Identity Providers will be removed from Synapse 1.38, due out in August. Please ensure your client supports the stable identifiers for this feature.

Spaces: On by Default

Following the successful release of Synapse 1.34, the experimental Spaces flag is now enabled by default. If you had manually enabled the experimental_features: { spaces_enabled: true } flag in your homeserver configuration, you may now remove it.

Bug Squashing

This release of Synapse fixes an issue which could cause federated room joins to fail when the join response exceeded a size limit which was too low (#10082). We've also improved what Synapse logs when it drops a connection in similar circumstances (#10091), which should aid diagnosis if a similar issue were to arise in the future.

GitHub user thermaq contributed a fix (#10014) for a bug which could cause user presence state to become stale.

Lastly our OpenTracing support now allows for profiling end-to-end performance on a per-user basis (#9978).

An Update on Room Joins

We've been hammering away at shrinking Synapse's memory footprint when joining large / complex rooms, and while we're not there yet, the end is in sight! In particular, this release includes many internal refactorings, including using ijson to parse the JSON response to /send_join (#9958), clearing the way for substantial improvements.

Memory usage still spikes because we're effectively doing the same work with a different library, but ijson's design allows for iterative parsing. This will pay dividends once we modify the code downstream of /send_join to take advantage of it.

Concretely, Erik Johnston has an experimental branch of Synapse which completely eliminates the memory spike:

Memory usage graph for Synapse 1.33, 1.35, and an experimental branch

The remaining work is centered on splitting that branch into self-contained, reviewable pull requests, like a rewrite of the Synapse Keyring class (#10035). After that's merged, we'll need to make one further change to properly batch up work, at which point we should attain the efficiency gains from Erik's experiment.

Everything Else

GitHub user savyajha contributed a security hardened systemd unit file which effectively sandboxes Synapse (#9803). While not enabled by default, we'd encourage security conscious users to review the example file and associated documentation.

Please see the Release Notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including dklimpel, jerinjtitus, junquera, lonyeon, savyajha, and thermaq.

This Week in Matrix 2021-05-28

28.05.2021 21:38 — This Week in Matrix Andrew Morgan
Last update: 28.05.2021 21:14

Matrix Live 🎙


A classic "Matthew & Amandine" episode. This week: Spaces, Reputation, Low-Bandwidth, P2P & more!

Dept of Spec 📜

Spec

anoa gifted us with:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.

MSC Status

Lots of new MSCs appearing this week!

New MSCs:

MSCs with proposed Final Comment Period:

  • No MSCs entered proposed FCP state this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

Merged MSCs:

  • No MSCs were merged this week.

Spec Updates

The Spec Core Team is working towards finalising the remaining spec PRs and the new spec release process. Good news is there's only one final spec PR to go! Bad news is it's probably going to be one of the hardest :)

Otherwise the team was pleased to see that noticeable progress is being made on the MSC backlog. But that doesn't mean we get to rest on our laurels!

2021-05-28-u8l6Y-stacked_area_chart.png

Dept of Servers 🏢

Synapse

Synapse is a popular homeserver written in Python.

callahad said:

We have interns! In addition to Matrix.org's participation in Google Summer of Code, Element has funded two Outreachy interns to work on backend projects, and we're overjoyed to welcome the following folks to Matrix:

  • Shay (Outreachy) is focused on modernizing Sydent and Sygnal, our reference implementations of a Matrix Identity Server and a Matrix Push Gateway respectively.
  • Meenal (Outreachy) is helping us improve Complement, and modern replacement for SyTest, our homeserver integration test suite.
  • Callum (GSoC) is working on adding support for restricting homeserver registration to users with pre-shared invite codes.

Welcome Callum, Meenal, and Shay!

We're also working to unify and standardize the module API which is used to extend Synapse, and your feedback would be very welcome. Similarly, if you have any experience with adding custom modules to Docker images or Debian packages and have thoughts on how we should best support that (or just examples of projects which do that well), please weigh in at #9944.

Lastly, we're looking forward to the release of Synapse 1.35 which will bring significant improvements to memory use during room joins, but that's for next week. 😉

A very warm welcome to all of our Outreachy interns and Google Summer of Code students this year! You can see the full list of GSoC students in our Google Summer of Code 2021 blogpost.

synapse-media-proxy

f0x announced:

I've been working on a smart media worker/proxy, to offload initial spikes in traffic for homeservers behind slower network uplinks.

There's a <remote> component that sits on a small vps, handling the upload and download endpoints (and soon, thumbnails), with an in-memory cache to quickly respond to requests. Media still gets forwarded to the <local> component where it gets stored in Synapse's media folder with an accompanying database entry, so all media is still stored long-term in the way Synapse expects it to be, so the proxy can be removed at any point.

Currently not production ready but you're welcome to snoop around the repo :)

https://git.pixie.town/f0x/synapse-media-proxy

Seems like quite a useful project! Other adventures in splitting out the media server from the homeserver include TravisR's matrix-media-repo project.

Conduit

Conduit is a Matrix homeserver written in Rust.

timokoesters reported the latest updates since last week:

  • Feature: Implement /claim

  • Feature: Handle incoming to-device events

  • Feature: Implement /query

  • Improvement: Faster signing key storage

  • Improvement: Documentation for Appservices

  • Fix: State resolution bugs

  • Fix: Respond to unauthorized PDUs with FORBIDDEN

  • Fix: Forward errors from remote servers (e.g. unsupported room version)

Timo also shared with us a milestone dashboard for Conduit being production-ready: https://gitlab.com/famedly/conduit/-/milestones/3. It's exciting to see the multi-implementation Matrix homeserver landscape really beginning to take shape!

Homeserver Deployment 📥️

Kubernetes

Ananace announced:

Another weekly Kubernetes Chart update, this time only for element-web though - which was updated to 1.7.29.

Dept of Bridges 🌉

libera.chat IRC network news

Half-Shot tells us:

Hey folks, some of you (well, frankly most of you) wanted to hear about our plans for libera.chat. I've been hard at work this week building a brand new bridge which is hosted on the libera.chat homeserver and it's nearly ready to go. We're just making a few final preparations before taking it out of beta, but it's currently available as a portaling bridge. You can currently reach any channel on libera.chat by joining #<channel_name>:libera.chat. You can also add the homeserver to your room directory in your Matrix client to search all the bridged rooms.

We're also hanging out in #libera-matrix:libera.chat, if you have any questions. Of course, the bridge is proving to be very popular so please be patient :)

Definitely one of the most sought-after Matrix bridges lately. Props to Half-Shot, Libera.Chat and everyone else for getting it all together in such short notice.

Heisenbridge Updates

hifi told us:

Plumbing IRC channels 🧑‍🔧

Somewhat experimental feature that has landed this week is the ability to use single puppeting in a public Matrix room to bridge IRC channels.

This support comes through a new PLUMB admin command for network rooms that will ask the bridge bot to join a public Matrix room and an IRC channel.

On IRC side there will be only a single user/bot that will relay messages from Matrix users. This is a stop-gap between fully puppeting Matrix users on IRC and doesn't need explicit permission from an IRC network to use by channel operators as long as bots are allowed.

On Matrix side puppets will join and leave based on their presence on the IRC channel. The bridge bot does not need any special permissions in a room as it doesn't do permission synchronization so it can be kept without any power levels.

Unplumbing a room is as simple as kicking the bridge out and it will take the puppets with it.

As this is a fairly new feature it does have some rough edges so feedback is greatly appreciated.

With all this said, if you're in the need to bridge any IRC channel on any network with little friction, Heisenbridge should have you covered!

SASL support 🔒

If you're using Heisenbridge to connect to Libera.Chat from a network block that has SASL enforcement enabled (like some VPS providers) you can now configure SASL credentials to authenticate at connection time to bypass this restriction.

This only implements SASL PLAIN for now. Certificate authentication may be looked at in the future.

But that's not all

  • Identd will now always provide a unique ident for everyone

  • Improved detection for stalled connections

  • IRC user ghosting issues fixed for good 🤞

  • Server TLS cert checking fixes, self-signed support

  • Initial bridge DM invites should go through more reliably

  • QUIT command to leave from all networks in one go

  • Automatic Docker Hub images from master (amd64 and arm64)

In addition Heisenbridge is now featured on matrix.org 🥳

matrix-puppeteer-line Updates

matrix-puppeteer-line is a bridge for LINE Messenger based on running LINE's Chrome extension in Puppeteer.

Fair offered:

This week brings a variety of assorted bugfixes & feature improvements in the testing branch. Included are:

  • Support for m.sticker events for LINE stickers (with an option to use m.image if preferred)

  • Clear resolution of LINE emojis, and a config option to scale them (since they're tiny by default)

  • Less frequent re-downloading of already-seen LINE stickers/emoji (there are limits to this, but it's still an improvement)

  • Option to disable syncing LINE stickers/emoji (in case re-hosting LINE imagery is a liability)

  • More reliable message syncing (should never get "Decrypting message..." or invisible images/stickers/emoji anymore)

  • Fix crashes during inbound message & read receipt syncing of multi-user rooms

Like last time, testing is much appreciated 😀

Remaining big tasks are:

  • Rework message syncing so that receiving a message doesn't require "viewing" the LINE chat in the Puppeteer-controlled browser, which will make LINE send a read receipt even though you may not actually have read the message yourself (I mentioned this a while ago; it's still a todo)

  • Support for multiple bridge users at once (A public instance of the bridge will only be considered once multi-user support is ready)

Discussion: #matrix-puppeteer-line:miscworks.net

Issue page: https://src.miscworks.net/fair/matrix-puppeteer-line/issues

Dept of Clients 📱

NeoChat Updates

NeoChat is a Matrix client written using KDE technologies, most notably the Kirigami UI framework.

Carl Schwan reported:

The past few weeks have been busy with fixing bugs. As a result, we plan to release NeoChat 1.2 next week. You can help by testing the Flatpak beta version of NeoChat: flatpak install https://flathub.org/beta-repo/appstream/org.kde.neochat.flatpakref.

Aside from bug fixing, Tobias worked a bit on E2EE encryption in Quotient and Noah added a fancy typing indicator.

2021-05-28-jvofQ-image.png

Nheko Updates

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE and intends to be full featured and nice to look at

AppAraat told us:

  • I added mnemonic shortcut keys to the context menu. Now you can react and reply even faster! (Thanks Nico for all the guidance :))

  • Nheko is now available on the Chocolatey package manager for Windows, yay!

2021-05-28-GwdUf-clipboard.png

FluffyChat Updates

FluffyChat is the cutest cross-platform matrix client. It is available for Android, iOS, Web and Desktop.

krille announced:

FluffyChat 0.31.1 has been released with a lot of bug fixes, stability improvements, minor redesigns and a lot of refactoring under the hood.

Feature

  • Cute animation for hiding the + button in inputbar

  • Improved chat bubble design and splash animations

  • Zoom page transition on Android

  • SSO is now an option even if the server supports password login

Fixes

  • "Pick an image" button in emote settings doesn't do anything

  • Emoji picker

  • Status bar and system navigation bar theme

  • Open URIs and receive sharing intent on iOS

  • SSO on iOS

  • Workaround for iOS not clearing notifications

  • Send read markers

2021-05-28-wPAPg-scaled_screenshot_20210522-101152_fluffychat.jpg

Fractal Updates

Fractal is a Matrix client written in Rust, and one of the first to use the matrix-rust-sdk!

Alexandre Franke offered:

Since last week, a couple things landed in fractal-next. Julian added support for accepting/rejecting invites (!754/#759). Kévin did an internal restructuring of how we handle the SyncResponse in !759. The visible consequence for users is that room categories (also known as tags) are updated directly in Fractal when the user changes it from an other client. He also switched from comrak to ruma’s markdown in !756. It shouldn’t have any impact for users, but should mean less code in the end on our side and more consistent code as well.

2021-05-28-Hlji9-image.png

Element Client Updates

Crafted by the Element teams.

Delight

  • Spaces: We’ve been listening to, investigating, triaging and organising feedback from the beta so far. There’s a lot to get through, but thanks to everyone who has submitted feedback in-app in Element so far! The insight has been invaluable and is instrumental in shaping up our next milestones.
  • Papercuts: We’re also organising and fixing small issues throughout Element, biasing for highly visible issues which have the greatest impact and reach, affectionately titled ‘papercuts’. Expect more info on these soon, but for now a fun one to highlight is we’re implementing blurhash on Web & Android (with iOS to follow soon after) to improve previewing and viewing images, especially when on low bandwidth.

Web

  • 1.7.29 released
    • Improved startup performance by focusing decryption on recent rooms
    • Fixed reaction duplication
  • Sorting out why we sometimes see "missing translations" hopefully for good this time
  • Continuing to improve application performance
  • Started work on Apple silicon desktop builds

iOS

  • We continued to work on technical subjects:
    • Stabilisation and performance improvements
    • New logging system. It will be possible to disable all logs for MatrixSDK, MatrixKit and Element-iOS
    • The code that manages application navigation in Element-iOS has been updated. It will allow us to add a slide menu to display things like user spaces.

Android

  • Element Android 1.1.8 has been released on the beta track of the Google PlayStore. It will probably be pushed to production at the beginning of next week. The Android matrix SDK2 has also been released.
  • This week, we implemented the ability to change the network when looking in the room directory. Also gitter.im has been added to the default network list. See some screenshots in https://github.com/vector-im/element-android/pull/3419
  • Lots of improvements regarding Spaces have also landed to develop.
  • Besides that, we are still fixing issues and perform regular maintenance on the project

Dept of SDKs and Frameworks 🧰

autodiscover-server-configuration

f0x said:

I forked autodiscover-client-configuration to implement the Server-Server counterpart of getting the API endpoint from a server_name.

Package published on NPM: https://www.npmjs.com/package/@f0x52/autodiscover-server-configuration

Repository: https://git.pixie.town/f0x/autodiscover-server-configuration.git

Dept of Services 🚀

GoMatrixHosting is a SaaS Matrix hosting platform.

Michael reported:

We are finally open for business. Start your own Matrix service today with GoMatrixHosting!

https://www.gomatrixhosting.com/2021/05/23/gomatrixhosting-is-here/

And quite soon after:

GoMatrixHosting v0.4.7 has arrived!

  • reinvented database purging section, separated final shrinking that causes downtime.
  • fixed AWX issue causing rust-synapse-compress-state section to not execute.
  • added more reliable script method of generating the total room list.

Check us out on GitLab: https://gitlab.com/GoMatrixHosting/

Or come say hello at: #general:gomatrixhosting.com

Cactus Comments 🌵

Cactus Comments is a federated comment system for the open web built on Matrix.

Asbjørn reported:

Web Client v0.10.0 is out!

  • Allow commenting with Markdown.

  • Pluralize time units properly.

  • Enforce maximum nesting depth of 100 when sanitizing org.matrix.custom.html-formatted messages.

/ipns/latest.cactus.chat is updated to point to the latest release, so sites linking there should already be using the new version.

Come play with the demo: https://cactus.chat/demo

Join our Matrix room: #cactus:cactus.chat

Beeper Update

Beeper is a Matrix client and service that makes it easy to connect all your digital communications in one place.

Eric Migicovsky told us:

Been a while since our last update! The biggest development is that we've hired some fantastic members of the community including SpiritCroc from SchildiChat to work on Beeper Android and Kilian from Nio to work on Beeper iOS.

  • Beeper Desktop reached 2.0.0 with much improved performance and UI. Video Walkthrough

  • Released Beeper Android to Play Store

  • Switched domains from Beeperhq.com to the shiny new beeper.com

Projects in the works

  • Beeper iOS (SoonTM)

  • Adding Infinite backfill to all bridges.

  • Improvements to iMessage bridge (adding tapbacks and reply support)

We are hiring React, iOS, Android and SRE/Devops engineers. If you're interested, send me a DM! 100% remote, we'll hire you anywhere you are.

2021-05-28-y-Bs5-image.png

Eric's Matrix ID is @eric:beeper.com. Reach out if you're interested in getting involved!

Dept of Bots 🤖

Airlock, the space invite bot

Robin announced:

If you want to use spaces to run an invite-only community, but can't wait for knocking and restricted room membership to make their way into a stable room version, you may be interested in Airlock, a very simple bot that I wrote to automatically invite people to a space's rooms and subspaces when they join.

It requires zero configuration—just invite the bot to your space and all of the rooms and subspaces within it, make sure it has invite permissions, and you're good to go! You're welcome to use my hosted version @airlock:townsendandsmith.ml, or run it yourself.

It's great to see people building useful tools to bridge the gap as Spaces inches closer out of beta.

Dept of Interesting Projects 🛰️

Host your blog on Matrix

evol told us:

About a month ago I came up with the idea that it would be interesting to use Matrix as an API for a blog. Sometime later it turned out that it's possible, and one thing led to another, and now I have a blog hosted on Matrix 😃

Long story short, the blog is a space and each post is a room within that space. This allows you to read the posts in your Matrix client of choice and also have discussions right underneath the content.

You can read more details on how it's done on the blog itself (incl. links to ugly code on my Github): https://evolved.systems/hosting-a-blog-on-matrix/. If you want to chat about this, join #blog:evolved.systems !

Longtime readers may remember Luke Barnard's take on the concept of Blogs built on Matrix back in 2019 with Journal. It's really exciting to see a modern version of this concept - and using Spaces no less!

Server_Stats Updates

Server_Stats is a project to make statistics about the Matrix network neatly catalogued and browsable.

MTRNord said:

Space List

The tool now has a separate list for rooms that are spaces at https://serverstats.nordgedanken.dev/spaces this allows you to find Spaces easily and fast :)

Other Updates

  • A bug in the sdk combined with appservices did initially have for some rooms the ban not recognized. This is now fixed and those rooms are hidden again from the API. Sorry for the inconvenience.

  • Safari should now also work after fixing a setting.

Check out the Source at: https://github.com/MTRNord/server_stats or join the room at #server_stats:nordgedanken.dev for questions :)

Circles

Circles markets itself as a private, end-to-end encrypted social network built on top of Matrix.

farribeiro announced:

Good news folks

The Circles beta is open source

https://github.com/KombuchaPrivacy/circles-ios

It's AGPL for now

But automatically transitions to dual-license under AGPL / Apache 2.0 when 'Murica turns 250 years old. (July 4, 2026, for my international friends)

I love seeing Matrix projects dipping into the social networking sphere. It's an untapped market!

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1rollyourown.xyz534
2maunium.net548.5
3trolla.us746
4matrix.sp-codes.de750
5xethos.net760
6blackline.xyz952.5
7fab.network1010
8lo.hn1096
9dendrite.foxomy.com1125
103icn.net1220

#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1dendrite01.fiksel.info935
2dendrite.foxomy.com1154.5
3dendrite.s3cr3t.me3392

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2021-05-21

21.05.2021 00:00 — This Week in Matrix Ben Parsons

Matrix Live 🎙

Dept of Spec 📜

Spec

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.

MSC Status

New MSCs:

MSCs with proposed Final Comment Period:

  • No MSCs entered proposed FCP state this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

Merged MSCs:

  • No MSCs were merged this week.

Closed MSCs:

Merged Spec PRs

Spec Updates

Travis rolled out MSC3202 to aid in the quest for end-to-bridge encryption without having to rely on /sync. Otherwise the Spec Core Team has been reviewing MSCs and focusing on the last remaining spec PRs needed before cutting a new spec release. Watch this space!

Matrix Spec Change Proposals status chart

Awesome to be able to VISIBLY see the progress in that graph!

Dept of Servers 🏢

Synapse

Synapse is a popular homeserver written in Python.

callahad offered:

The big news this week was the launch of Spaces, supported by the release of Synapse 1.34.0.

However, there were quite a few other fixes, improvements, and additions: I'd encourage you to read the release announcement for those.

But more than anything, we're hard at work realizing the room-join memory improvements we talked about in last week's Matrix Live, and we expect to have some exciting news to share next week. Concretely: the pull request to support incremental parsing of join responses was merged, and several pull requests (#10017, #10018, #10035) to simplify and slim down the code for verifying event signatures are in review.

See you next week! 🚀

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timokoesters said:

Here are some Conduit news!

  • Feature: Handle incoming read receipts over federation

  • Feature: Send read receipts over federation (MVP done)

  • Improvement: Make UIAA (user-interactive authentication API) work like in Synapse (this gives us better client support)

  • Refactor: Sending code refactored to make EDUs over federation possible

  • Fix: Remove old presence events when restarting Conduit

  • Fix: Array bounds bug in server_server.rs

  • Fix: Power level content override adds to the default event instead of replacing it

Homeserver Deployment 📥️

Kubernetes

Ananace said:

And this week brings another instance of the regularly scheduled Kubernetes updates, with Element-web going to 1.7.28 and Synapse having been bumped to 1.34.0 on both the deprecated image and the chart.

Dept of Bridges 🌉

libera.chat IRC network news

Half-Shot offered:

Howdy folks! I'm sure many of you read the news about the new libera.chat IRC network starting up, and many of you also reached out (via pretty much every channel available) to ask about a potential bridge. libera.chat are doing a remarkably good job of getting spun up but are hyper busy with all the inbound users and need a bit more time, so in the meantime I would ask that everyone subscribe to https://github.com/matrix-org/matrix-appservice-irc/issues/1324 to listen for changes as they happen.

The matrix.org bridge team are on the case! Expect news in the near future :)

Dept of Clients 📱

Element Client Updates

Submitted by the teams

Delight

  • Spaces are now live! 🚀 Test them on Element Web, Desktop & Android (iOS is coming soon!)
  • In short, Spaces are a new way to group rooms and people together, and are slated to replace legacy groups/communities
  • We’d love your feedback! Either submitted in app or in #spaces-feedback:matrix.org
  • Read the full blog post

Web

  • 1.7.28 released
    • New spaces Beta (new way of grouping rooms and people)
    • Added support for slash commands working in edits
  • 1.7.29-rc.1 on staging
    • Improved startup performance by focusing decryption on recent rooms
    • Fixed reaction duplication
  • Continuing to pursue application performance generally
  • Improving day to day developer experience with more TypeScript conversion

iOS

  • 1.3.9 has been published on the app store
  • Some performance improvements have been merged on develop:
    • Reduce the number of decryptions. A decryption takes about 5ms on iPhone X. On an account with 500 rooms this allows us to skip thousands of decryptions on an initial sync
    • Those decryptions do not happen anymore on the main thread

Android

  • Lots of dependency upgrade following the release with Space (1.1.7).
  • Next release candidate, 1.1.8 will also contain improvements on Spaces.
  • We have set up towncrier flow to better handle changelog generation.
  • Also Element Android project is now using GitHub actions, but it cannot run the integration tests for the moment.

Fractal

Alexandre Franke offered:

As announced in the general Matrix GSoC post, we are lucky to get two interns this year again. Alejandro (whose previous work includes switching to matrix-rust-sdk for syncs) will implement multi-account support, while Kai (also a regular contributor by now) will help bring our application rewrite to the same functionality level as our current nightly and stable versions. Both projects will build on top of Fractal Next.

Julian came back from his vacation and immediately got quite busy reviewing and merging contributions from others. Kevin (also known as zecakeh) and him made good progress on several fronts:

  • Room handling was improved. We now keep track of whether the user joined them, left them, or has been invited to them, as well as their categories (favourites, low priority…).

  • The sidebar uses these categories and now uses a single list with GtkFilterListModels instead of one list per category. The greatly simplifies code for things like moving a room from one category to another.

  • History style has been tweaked, state events and timestamps have been added.

  • Markdown can be enabled from the newly implemented popover and the message composer offers syntax highlighting, although a bug currently makes it so that sent messages are still not sent with a formatted_body.

  • A persistent state store is used to load rooms on startup while sync is in progress.

Last but not least, new contributor Raatty implemented room name and topic display in the header bar.

2021-05-21-zHc3a-image.png

Hydrogen

A minimal Matrix chat client, focused on performance, offline functionality, and broad browser support. https://github.com/vector-im/hydrogen-web/

Bruno announced:

Released 0.1.53 with support for linking to room this week, which should allow to add Hydrogen to matrix.to soon. Also started working on redactions, which is useful in itself and also lays the foundations for reactions and edits coming next.

Nheko reviewed

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE and intends to be full featured and nice to look at

Check out this flattering portrait of Nheko on the Brodie Robertson "Linux Tips & Tricks" channel.

https://odysee.com/@BrodieRobertson:5/nheko-reborn-lightweight-native-matrix:8

Dept of SDKs and Frameworks 🧰

Ruma

Ruma is a set of Rust library crates around Matrix

jplatte offered:

We've been quiet in the past weeks, but certainly not due to lack of activity! So many awesome things have happened, I don't even know where to start. I guess I'll go chronologically:

We published so many new crate releases. Most of our crates didn't see non-alpha releases for almost a year and now we've finally reset that counter! Now you can depend on ruma 0.1 and get bug fixes and new functionality through cargo update without having to worry about breaking changes. Huge thanks to @zecakeh for automating our release process, without that this would have taken so much longer.

We got initial spaces support! I'm super excited that we're now enabling client authors to take spaces into account, even if "taking into account" will initially just mean filtering out space rooms from their room lists. Code-wise, this was actually a relatively small change, with ruma-client-api receiving support for room types and ruma-events receiving support for the event types m.space.child and m.space.parent.

We're swapping out ruma-signatures' crypto library. Since ring, which is what we've been using so far, doesn't currently support cross-compilation to WASM, there's long been some interest to exchange it for something else for webbrowser homeserver experiments. A viable alternative library was identified a while ago and now @ShadowJonathan has taken on the task of implementing the switch.

In addition to unblocking the WASM usecase, this work revealed a bug in ring, which generated invalid PKCS#8 documents. We are pretty confident that once the switch away from ring (including a small compatibility shim to convert the invalid documents), a certain class of signature verification failures that Conduit has been getting will be fixed.

We're taking part in GSoC again. As you might have seen in the earlier blog post, two students are working on Ruma as part of GSoC this year: @Frinksy (for the first time) and @DevinR528 (for the second time).

Apart from that,

Dept of Ops 🛠

matrix-docker-ansible-deploy

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi told us:

Thanks to Aaron Raimist, matrix-docker-ansible-deploy now supports Hydrogen - a new lightweight matrix client with legacy and mobile browser support.

By default, we still install Element, as Hydrogen is still not fully-featured. Still, people who'd like to try Hydrogen out can now install it via the playbook.

Additional details are available in Setting up Hydrogen.

Also...

Thanks to Toni Spets (hifi), matrix-docker-ansible-deploy now supports bridging to IRC using yet another bridge (besides matrix-appservice-irc), called Heisenbridge.

To get started, see our Setting up Heisenbridge bouncer-style IRC bridging documentation.

Dept of Services 🚀

etke.cc updated

Aine reported:

etke.cc updates

etke.cc is "spantaleev/matrix-docker-ansible-deploy as a service" and we got some updates:

  • New services: miniflux, wireguard, languagetool, email service for you domain (based on awesome Migadu) and consultations

  • Website redesign - made with ❤ ourselves instead of old generic template

  • announcements room migrated to public room without encryption, but with invites and previews

Dept of Bots 🤖

Community -> Space Bot

TravisR said:

If you're trying out all those cool Spaces (Element Blog) and have some old Communities/Groups hanging around, I have the bot for you.

spacebot makes quick work of the conversion by translating as much as possible to the Space structure. Simply DM the bot and say !convert +group:example.org and it'll go off and make your Space.

The bot doesn't invite everyone from your Community to your Space, giving you a chance to configure the Space further before advertising it within your rooms.

The source can be found on GitHub if you feel like running your own or seeing how simple it is under the hood. #spacebot:t2bot.io is a great place to get help if you're having problems getting it going, and #help:t2bot.io would be glad to assist you if the t2bot.io deployment doesn't do the right thing.

P.S.: Spaces are currently considered Beta in Element, so they might not be perfect just yet. #spaces:riot.ovh is the room to join if you have questions about Spaces.

Hemppa

Hemppa the Bot is a multipurpose bot for writing modules super easily in Python.

Cos offered:

Hemppa the bot is a general purpose Matrix bot written in Python. This week it gained two useful admin tools: Kick by wildcard (for cleaning up zombies after bridge decomission) and making tombstones to point room to a new one. https://github.com/vranki/hemppa

Dept of Interesting Projects 🛰️

raspberry-noaa-v2 matrix support

Cadair said:

A little bit different, but matrix related, this week I added Matrix push support to the raspberry-noaa-v2 project. This is a project which runs on a raspberry pi to automate receiving, decoding and post processing of images over radio from weather satellites. It's easy to build a cheap ground station for this with little more than a USB SDR dongle and a metal coat hanger. If you want to see the pictures I am decoding you can join #weather:cadair.com.

Cadair, living in the north of England, will only see this symbol unfortunately 🌧️

Server_Stats Voyager Project

MTRNord announced:

The Project now has a proper webpage at https://serverstats.nordgedanken.dev/ !

Newly added Features are:

  • A List of all the rooms that are found

  • A list to find the links (either outgoing or incoming) for a room. So a 1 level deep view into the graph as a table.

  • An FAQ with some information about the project, known data issues and more

  • A websocket at wss://serverstats.nordgedanken.dev/ws that allows you to get the updates directly pushed as soon as a room lands in the db. Effectively the fastest way to get data updates.

Planned features for the webpage

  • AR and VR Graphs

  • A Space Finder that allows you to find the discovered spaces

Be aware that this page is very new and therefor might have bugs. It also isn't yet mobile optimized.

For questions, suggestions or fast updates check the #server_stats:nordgedanken.dev Room or for questions regarding issues with the bot in a specific room feel free to write a DM to MTRNord :)

Possible Bug that needs manual fixing

Due to a bug with my homeserver or synapse it happens for some rooms that synapse generates a lot of join events (Lots of "server_stats bot made no change" messages). If you see this please write a message to MTRNord so I can manually prevent my bot from trying to join the room over and over again as I cannot easily detect this kind of issue before joining.

The issue can be followed at https://github.com/matrix-org/synapse/issues/10021

midnight from cvwright

cvwright offered:

This might be of interest for TWIM. It's a little web app that supports token-based registration in the Matrix UIAA. Sort of like matrix-registration, but aiming for compliance with the UIAA spec

https://github.com/KombuchaPrivacy/midnight

Dept of Built on Matrix 🏗️

Circles, new social network on top of Matrix

cvwright offered:

Circles is a new project to build an E2E encrypted social network on top of Matrix. https://www.kombuchaprivacy.com/circles/ https://www.kickstarter.com/projects/cvwright/circles-a-secure-social-app-for-friends-and-family

It's really early days for this project, but please check it out!

Final Thoughts 💭

farribeiro reported:

The fedora project is discussing which domain to take in https://discussion.fedoraproject.org/t/so-what-domains-should-we-use-for-our-matrix-server/29842

Un-burying the lede: Fedora are making the jump to Matrix!

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1the-apothecary.club463.5
2fosil.eu539
3helderferreira.io545.5
4trolla.us579
5zheng.fi639
6alra.uk702
7feline.support759.5
8thomcat.rocks770
9jae.fi777.5
10roeckx.be1033

#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1lankolol.de392.5
2zemos.net2251

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Google Summer of Code 2021

20.05.2021 00:00 — GSOC Ben Parsons

Google Summer of Code 2021 participants have been announced! This year Matrix has been assigned seven students.

Projects

R Midhun Suresh: Right Sidebar for Hydrogen client

R Midhun Suresh from the Mar Baselios College of Engineering & Technology in Trivandrum, India will be working on Hydrogen this summer, mentored by Bruno Windels. He will be working on adding a right panel to the room view, including a member list and room information. He will be blogging at https://midhunsureshr.github.io throughout the project.

Devin Ragotzy: Ruma's Automated Checks

My name is Devin Ragotzy. I am a student at Western Michigan University in Kalamazoo, Michigan, studying computer science. I was lucky enough to work last summer on Ruma and have continued to contribute to the project. I was accepted to work on Ruma's automated checks project, mentored by Isaiah Inuwa, Jonas Platte, Timo Kösters. The goal of the project is to create a linter capable of enforcing Ruma-specific style and practices. I hope to get this tool to a working state by the end of this summers GSoC!

Abhinav Krishna C K: First-Class Email Bridge

Abhinav Krishna C K from NSS College of Engineering in Palakkad, India will be working on Building First-Class email bridge for Matrix this summer mentored by Half-Shot and tulir. This will enable Matrix to be connected with Email by translating incoming SMTP traffic to Matrix messages, and then bridging Matrix messages back into emails.

Frinksy: Extend Ruma's API coverage

My name is Adam Blanchet, and I am a student from the University of York in the UK.
I am happy to say that I have quite a few mentors: Isaiah Inuwa, Jonas Platte, Timo Kösters and Nico from Nheko.
My project is to extend Ruma's API coverage. I'll be doing a few things: finishing coverage of the Identity Service API, adding the "knock" feature to Ruma and finishing the implementation of QR code verification. If time allows it I will also work on implementing other MSCs or features such as "Event notification attributes and actions". I hope that my work will help enable other Rust-based Matrix projects, such as Conduit and Fractal, to implement more features.

Timo added:

Hello, I am Timo Kösters. I study Computer Science in Germany and spend most of the remaining time developing Conduit, a Matrix homeserver built on top of Ruma. I use Ruma all the time and will be mentoring Adam Blanchet to make it even better.

Vladyslav Hnatiuk: PyQuotient

My name is Vladyslav Hnatiuk, I'm a student of Vienna University of Technology and my project is PyQuotient.
The aim is to simplify creating of a Matrix Qt-based clients in Python by providing Qt-based SDK and avoid writing a large part of functionality manually. And to not reinvent the wheel PyQuotient will be bindings for the existing library libQuotient that provides SDK for Matrix for C++ applications. I'll be mentored by kitsune, the author of libQuotient and also libQuotient-based Matrix IM client Quaternion. I hope PyQuotient will facilitate the development of Matrix clients in Python with Qt, and it will be a small contribution to the promotion of Matrix, especially in Python world.

kitsune added:

If this experience proves to be successful, there’s a good chance Quaternion will eventually switch to Python.

Callum Brown: Token Authenticated Registration

Hi there, I'm Callum, a Londoner who'll be starting a physics degree in September. For GSoC I'll be working on adding Token Authenticated Registration to Matrix. This will allow homeserver admins to restrict who can sign-up by requiring a token to be submitted during registration. I run a small homeserver for friends and family, but don't have the resources to make registration public, so I have wanted this feature integrated into Matrix servers and clients for quite a while! I'll be working with Nico, anoa, and red_sky to write an MSC, implement the server side in Synapse, and the client side in Nheko. Thanks to the mentors and Matrix.org for the opportunity to work on this!

You can follow along with this project's progress throughout the program at https://calcuode.com/matrix-gsoc/.

Nico, mentor added:

Nico here, one of the Mentors. Personally I am super excited about this project! I have been using Matrix for a while now and I think Nheko is pretty good by now. But there is still a barrier, if I want my friends and family to use Matrix: They can't easily sign up! I have tried creating them accounts and telling them to change their passwords, having a dedicated registration page or just telling them to just use a different server, but nothing of that made me happy and it added friction to the already hard process of getting someone to try a new messenger! As such I am super excited for this, because it will make signing up your friends and family to your personal instance, without it having to be public, sooooo much simpler!

Jaiwanth: Exporting Conversations From Element

Jaiwanth Vemula from the IIT Kharagpur University in India will be working on Exporting Conversations in Element this summer, mentored by Michael (t3chguy). This work will enable users to easily export their conversations for archival or sharing, this is a feature which has been missed in Element for a very long time!

Fractal

There are also two projects under the GNOME organisation with students working on Fractal, a Matrix client for GNOME.

  • Alejandro Domínguez: Fractal: Multi account support
  • Kai A. Hiller: Fractal NEXT

Students become Mentors

I asked: how many of those who are mentors this year have ever been GSOC students? The answer is that this year four of the mentors were once GSoC students themselves!

EOF

That's all! Seven projects in 2021 is awesome, and we're looking forward to seeing updates from the students!

How the UK's Online Safety Bill threatens Matrix

19.05.2021 15:47 — Tech Denise Almeida
Last update: 19.05.2021 14:48

Last week the UK government published a draft of the proposed Online Safety Bill, after having initially introduced formal proposals for said bill in early 2020. With this post we aim to shed some light on its potential impacts and explain why we think that this bill - despite having great intentions - may actually be setting a dangerous precedent when it comes to our rights to privacy, freedom of expression and self determination.

The proposed bill aims to provide a legal framework to address illegal and harmful content online. This focus on “not illegal, but harmful” content is at the centre of our concerns - it puts responsibility on organisations themselves to arbitrarily decide what might be harmful, without any legal backing. The bill itself does not actually provide a definition of harmful, instead relying on service providers to assess and decide on this. This requirement to identify what is “likely to be harmful” applies to all users, children and adults. Our question here is - would you trust a service provider to decide what might be harmful to you and your children, with zero input from you as a user?

Additionally, the bill incentivises the use of privacy-invasive age verification processes which come with their own set of problems. This complete disregard of people’s right to privacy is a reflection of the privileged perspectives of those in charge of the drafting of this bill, which fails to acknowledge how actually harmful it would be for certain groups of the population to have their real life identity associated with their online identity.

Our view of the world, and of the internet, is largely different from the one presented by this bill. Now, this categorically does not mean we don’t care about online safety (it is quite literally our bread and butter) - we just fundamentally disagree with the approach taken.

Whilst we sympathise with the government’s desire to show action in this space and to do something about children’s safety (everyone’s safety really), we cannot possibly agree with the methods.

Back in October of 2020 we presented our proposed approach to online safety - ironically also in response to a government proposal, albeit about encryption backdoors. In it, we briefly discussed the dangers of absolute determinations of morality from a single cultural perspective:

As uncomfortable as it may be, one man’s terrorist is another man’s freedom fighter, and different jurisdictions have different laws - and it’s not up to the Matrix.org Foundation to play God and adjudicate.

We now find ourselves reading a piece of legislation that essentially demands these determinations from tech companies. The beauty of the human experience lies with its diversity and when we force technology companies to make calls about what is right or wrong - or what is “likely to have adverse psychological or physical impacts” on children - we end up in a dangerous place of centralising and regulating relative morals. Worst of all, when the consequence of getting it wrong is criminal liability for senior managers what do we think will happen?

Regardless of how omnipresent it is in our daily lives, technology is still not a solution for human problems. Forcing organisations to be judge and jury of human morals for the sake of “free speech” will, ironically, have severe consequences on free speech, as risk profiles will change for fear of liability.

Forcing a “duty of care” responsibility on organisations which operate online will not only drown small and medium sized companies in administrative tasks and costs, it will further accentuate the existing monopolies by Big Tech. Plainly, Big Tech can afford the regulatory burden - small start-ups can’t. Future creators will have their wings clipped from the offset and we might just miss out on new ideas and projects for fear of legal repercussions. This is a threat to the technology sector, particularly those building on emerging technologies like Matrix. In some ways, it is a threat to democracy and some of the freedoms this bill claims to protect.

These are, quite frankly, steps towards an authoritarian dystopia. If Trust & Safety managers start censoring something as natural as a nipple on the off chance it might cause “adverse psychological impacts” on children, whose freedom of expression are we actually protecting here?

More specifically on the issue of content moderation: the impact assessment provided by the government alongside this bill predicts that the additional costs for companies directly related to the bill will be in the billions, over the course of 10 years. The cost for the government? £400k, in every proposed policy option. Our question is - why are these responsibilities being placed on tech companies, when evidently this is a societal problem?

We are not saying it is up to the government to single-handedly end the existence of Child Sexual Abuse and Exploitation (CSAE) or extremist content online. What we are saying is that it takes more than content filtering, risk assessments and (faulty) age verification processes for it to end. More funding for tech literacy organisations and schools, to give children (and parents) the tools to stay safe is the first thing that comes to mind. Further investment in law enforcement cyber units and the judicial system, improving tech companies’ routes for abuse reporting and allowing the actual judges to do the judging seems pretty sensible too. What is absolutely egregious is the degradation of the digital rights of the majority, due to the wrongdoings of a few.

Our goal with this post is not to be dramatic or alarmist. However, we want to add our voices to the countless digital rights campaigners, individuals and organisations that have been raising the alarm since the early days of this bill. Just like with coercive control and abuse, the degradation of our rights does not happen all at once. It is a slippery slope that starts with something as (seemingly) innocuous as mandatory content scanning for CSAE content and ends with authoritarian surveillance infrastructure. It is our duty to put a stop to this before it even begins.

Twitter card image credit from Brazil, which feels all too familiar right now.

The Matrix Space Beta!

17.05.2021 17:50 — Tech Matthew Hodgson
Last update: 17.05.2021 17:35

Hi all,

As many know, over the years we've experimented with how to let users locate and curate sets of users and rooms in Matrix. Back in Nov 2017 we added 'groups' (aka 'communities') as a custom mechanism for this - introducing identifiers beginning with a + symbol to represent sets of rooms and users, like +matrix:matrix.org.

However, it rapidly became obvious that Communities had some major shortcomings. They ended up being an extensive and entirely new API surface (designed around letting you dynamically bridge the membership of a group through to a single source of truth like LDAP) - while in practice groups have enormous overlap with rooms: managing membership, inviting by email, access control, power levels, names, topics, avatars, etc. Meanwhile the custom groups API re-invented the wheel for things like pushing updates to the client (causing a whole suite of problems). So clients and servers alike ended up reimplementing large chunks of similar functionality for both rooms and groups.

And so almost before Communities were born, we started thinking about whether it would make more sense to model them as a special type of room, rather than being their own custom primitive. MSC1215 had the first thoughts on this in 2017, and then a formal proposal emerged at MSC1772 in Jan 2019. We started working on this in earnest at the end of 2020, and christened the new way of handling groups of rooms and users as... Spaces!

Spaces work as follows:

  • You can designate specific rooms as 'spaces', which contain other rooms.
  • You can have a nested hierarchy of spaces.
  • You can rapidly navigate around that hierarchy using the new 'space summary' (aka space-nav) API - MSC2946.
  • Spaces can be shared with other people publicly, or invite-only, or private for your own curation purposes.
  • Rooms can appear in multiple places in the hierarchy.
  • You can have 'secret' spaces where you group your own personal rooms and spaces into an existing hierarchy.

Today, we're ridiculously excited to be launching Space support as a beta in matrix-react-sdk and matrix-android-sdk2 (and thus Element Web/Desktop and Element Android) and Synapse 1.34.0 - so head over to your nearest Element, make sure it's connected to the latest Synapse (and that Synapse has Spaces enabled in its config) and find some Space to explore! #community:matrix.org might be a good start :)

The beta today gives us the bare essentials: and we haven't yet finished space-based access controls such as setting powerlevels in rooms based on space membership (MSC2962) or limiting who can join a room based on their space membership (MSC3083) - but these will be coming asap. We also need to figure out how to implement Flair on top of Spaces rather than Communities.

This is also a bit of a turning point in Matrix's architecture: we are now using rooms more and more as a generic way of modelling new features in Matrix. For instance, rooms could be used as a structured way of storing files (MSC3089); Reputation data (MSC2313) is stored in rooms; Threads can be stored in rooms (MSC2836); Extensible Profiles are proposed as rooms too (MSC1769). As such, this pushes us towards ensuring rooms are as lightweight as possible in Matrix - and that things like sync and changing profile scale independently of the number of rooms you're in. Spaces effectively gives us a way of creating a global decentralised filesystem hierarchy on top of Matrix - grouping the existing rooms of all flavours into an epic multiplayer tree of realtime data. It's like USENET had a baby with the Web!

For lots more info from the Element perspective, head over to the Element blog. Finally, the point of the beta is to gather feedback and fix bugs - so please go wild in Element reporting your first impressions and help us make Spaces as awesome as they deserve to be!

Thanks for flying Matrix into Space;

Matthew & the whole Spaces (and Matrix) team.

Synapse 1.34.0 released

17.05.2021 16:45 — Releases Dan Callahan
Last update: 17.05.2021 15:13

Synapse 1.34.0 is now available, and it's loaded with new features and performance improvements.

Note: This release deprecates and replaces the room_invite_state_types configuration option. If you've customized that for your homeserver, please review the Upgrade Notes.

We've also marked the v1 room deletion Admin API as deprecated. Instead of sending a POST to a path ending in /delete, administrators are encourage to instead send an HTTP DELETE to /_synapse/admin/v1/rooms/<room_id>. Thanks to ThibF for implementing this (#9889).

Spaces

The highlight of this release is support for Spaces, now that MSC1772: Matrix Spaces has merged into the Matrix spec!

Synapse also has support for MSC2946: Spaces Summary and MSC3083: Restricting room membership based on space membership, but these are off by default as they're still under development. To enable these experimental MSCs, set experimental_features: { spaces_enabled: true } in your homeserver configuration. These are enabled on the matrix.org homeserver, and we encourage you to experiment with Spaces there and let us know in the Spaces Feedback Room if you encounter any issues.

Memory and Caching

Memory consumption and caching have been a major focus of the Synapse team this quarter, and we've made significant strides:

  • Synapse has a new gc_min_interval configuration option with reasonable defaults to prevent Python's garbage collector from running too frequently and thrashing when a large homeserver has its collection thresholds set too low.
  • Synapse will report memory allocation stats to Prometheus when using jemalloc.
  • Synapse will also measure Redis cache response times and report those to Prometheus.
  • For debugging, Synapse can optionally track the memory use of each LruCache.

We have a few more tricks up our sleeves; to learn more about how we're planning to improve the memory cost of joining large rooms, check out last week's Matrix Live.

Other Fixes and Improvements

We've also landed significant improvements to:

  • Sending events when Redis is available (#9905, #9950, #9951)
  • Joining large rooms when presence is enabled (#9910, #9916)
  • Backfilling history in large rooms (#9935)

...and fixed bugs that:

  • Prevented cross-account m.room_key_request messages from being delivered (#9961, #9965)
  • Incorrectly applied room creation / invitation rate limits to users and app services which should have been exempt (#9968)

The health check on our Docker images now responds more quickly upon successful startup thanks to improvements by maquis196 (#9913), and for especially privacy-conscious homeservers, device names can now be shielded over federation thanks to a contribution by aaronraimist (#9945).

New Access Token Format

Inspired by GitHub's new token format, we've restructured Synapse's access token format to follow the pattern:

syt_<UnpaddedBase64MXIDLocalPart>_<20RandomLetters>_<CRC32>

So a token for @example:matrix.org might look like:

syt_ZXhhbXBsZQ_KfJetOcLWEKCvYdKnQLV_0i3W80

Existing tokens remain valid; this is just for new tokens. We hope the new format reduces network overhead while also making it easier identify misplaced tokens in logs and repositories.

Everything Else

See the Upgrading Instructions and Release Notes for further information on this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including aaronraimist, maquis196, and ThibF.

This Week in Matrix 2021-05-14

14.05.2021 00:00 — This Week in Matrix Ben Parsons

Matrix Live 🎙

This week Dan talks to Erik about the state of Synapse generally, and in particular the work Erik has been doing for performance.

Plus! Bonus content, demos! Half-Shot on the MS Teams bridge work, and Hubert previews Matrix over MLS!

Dept of Spec 📜

Matthew reported:

Some combination of me, Kegan, Bruno and neilalexander have been working on v3 of the CS /sync API. (Today's /sync API in matrix is v2; v1 was the old /events API). We're not yet at the point of publishing a draft or MSC, but it's coming soon. It's really exciting work which flips Matrix around so that sync scales independently of the number of rooms you're in - and it's at last possible to write rich clients which only ever sync the bare minimum data needed to work: i.e. lazy loading eeeeeeeeverything. Watch this space :)

anoa told us:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.

MSC Status

New MSCs:

MSCs with proposed Final Comment Period:

  • No MSCs entered proposed FCP state this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

Merged MSCs:

Merged Spec PRs

Here are all the actual MSC-related changes made to the raw Matrix spec this week!

Spec Updates

There's been activity from the Spec Core Team on a number of different MSCs, such as MSC3189 (per-room/space profile data). Additionally a spec PR for Matrix URI schemes has been getting feedback and is moving forward at a quick pace! There was also some feedback from the team on MSC2448 (blurhashes) which I'll get around to answering shortly 🙂.

Otherwise I think this week was a bit implementation-heavy for the team (the Spec Core Team is a task in addition to our full-time jobs). Hopefully next week will grant us more of a breather.

2021-05-14-wdmML-stacked_area_chart.png

Dept of Servers 🏢

Synapse

callahad announced:

Hello TWiMmers! The bulk of our update is in Matrix Live today, so go check out the video above ☝️ to hear about how we're reducing the amount of memory it takes to join large rooms, and why joins take so much memory in the first place.

Otherwise, we're mainly getting ready for the public debut of Spaces as a beta feature, but more on that next week... 😉

Oh, and before we go: please make sure your Synapse is up to date! We released 1.33.2 on Tuesday, which contains a low severity security fix.

Homeserver Deployment 📥️

Kubernetes

Ananace offered:

Another installation of the regularly scheduled Kubernetes Helm Chart updates (and another bump of the deprecated Synapse image). Now up to Synapse 1.33.2 and Element Web 1.7.27.

Dept of Bridges 🌉

matrix-puppeteer-line

Fair told us:

matrix-puppeteer-line: A bridge for LINE Messenger based on running LINE's Chrome extension in Puppeteer.

This week was spent on stability improvements & bug fixes.

Calling for testers! The bridge is at a point where it's mostly usable, but it still has quite a few blindspots. If anyone is willing to try it out & report issues, it would be a great help!

Discussion: Matrix-LINE Puppeteer bridge Issue page: https://src.miscworks.net/fair/matrix-puppeteer-line/issues

Shout-outs to @lecris:lecris.me for being an early tester & having given a lot of useful feedback 🙂

Gitter

Eric Eastwood reported:

Last time, we updated you on starting DM conversations with Gitter users from Matrix. Now we have the other side of this complete! From Gitter, you can now start a one to one conversation with someone you see from Matrix 😀

Just hover over their avatar to bring up the user popover, then press the "Chat privately" button. 🗣

2021-05-14-VYQjI-chrome_2021-05-12_18-38-44.png

matrix-appservice-irc 0.26.0

Half-Shot reported:

Goooooooood afternoon folks and happy Friday! This week we're announcing the 0.26.0 release of the matrix-appservice-irc bridge which contains precious goodies:

  • You can now disable the kick behaviour of the bridge on Matrix users if you are running a personal bridge, so losing your IRC connection no longer results in a kick.

  • You can now remove bridges from rooms by using the admin room, so no need to use the provisioning API or modify the DB.

  • We've added a new feature to allow you to specify bridge options on a per room basis using room state. At the moment you can modify the limits of the automatic pastebin system but more features like reply formats are to come!

As always please come tell us about it in #irc:matrix.org and make sure to check out the new docs if you get a bit stuck.

Dept of Clients 📱

Hydrogen

A minimal Matrix chat client, focused on performance, offline functionality, and broad browser support. https://github.com/vector-im/hydrogen-web/

Bruno said:

Hydrogen can now leave rooms and forget archived rooms. URLs are now also clickable in the timeline. Get the full details in the release notes!

2021-05-14-iIgrd-image.png

Element Clients

Updates from the teams

Web

  • Element Web 1.7.28 is up on staging, targeting Monday for release.
    • New spaces Beta (new way of grouping rooms and people)
    • Added support for slash commands working in edits
  • On develop:
    • Voice messages are nearing completion - enable the labs flag and give it a go :)
    • Performance improvements to app startup time. Let us know if you run into any issues!

iOS

  • 1.3.7 is available on TestFlight. It should be on the App Store on Monday. Spaces are not yet available on Element-iOS but the app offers minimal support. The release contains a fix for background crashed due to PushKit
  • At the platform level, we are still improving stability and performance:
    • Decryption operations to be moved outside the main thread
    • More robust on initial sync
    • etc

Android

Delight

  • “Spaces are coming” (I had heard something about that - BP)

Dept of SDKs and Frameworks 🧰

matrix-spring-boot-sdk 0.5.0

Benedict announced:

matrix-spring-boot-sdk 0.5.0 is released and uses Trixnity now.

Dept of Services 🚀

GoMatrixHosting v0.4.6 released!

Michael offered:

https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1046/files

Just had a new release, GoMatrixHosting v0.4.6


* Add database purge section.

* Add markdown copy of User manual.
* Tweak compress state find rooms timeout.

* Update compile AWX instructions for custom branding and radius fix.

Visit #general:gomatrixhosting.com to join in the discussion.

GoMatrixHosting offer a great way to get your own homeserver without needing to run it yourself. Check them out!

Matrix in the News 📰

farribeiro shared:

Fedora promoting the matrix https://fedoramagazine.org/access-freenode-using-matrix-clients/

Final Thoughts 💭

From Dept of Humour in #twim:matrix.org 😅:

  • R1 Airport: Maybe I want to see if I can build some type of bridge between Matrix and CUPS.
  • hifi: the sad part is that would probably be more reliable than just printing directly

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1neko.dev422
2xirion.net471
3matrix.kalleeen.fi507
4t2bot.io516
5alra.uk556
6thomcat.rocks767
7matrix.vgorcum.com800.5
8matrix.2kgwf.fi817
9nordgedanken.dev828.5
10trolla.us878

#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1dendrite01.fiksel.info810.5
2dendrite.s3cr3t.me816
3lankolol.de931

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!