This Week in Matrix 2021-11-26

🔗Matrix Live 🎙

🔗Dept of Spec 📜

anoa says

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.

🔗MSC Status

New MSCs:

• MSC3523: Timeboxed/ranged relations endpoint
• MSC3517: "Mention" Pushrule
• MSC3510: Let users kick/ban/demote each other

MSCs with proposed Final Comment Period:

• No MSCs entered proposed FCP state this week.

MSCs in Final Comment Period:

• No MSCs are in FCP.

Merged MSCs:

• MSC2674: Event Relationships

Closed MSCs:

• MSC3006: Bot Interactions

🔗Spec Updates

MSC2675 (serverside aggregations) is getting lots of updates from Bruno in order to align the proposal with what is currently implemented in the wild (as it's easier to iterate on incremental improvements from a starting point grounded in reality). This MSC is a bit of a special case though, as it was implemented with stable prefixes before the MSC landed (in the before times...).

Regardless, thank you very much to Bruno for going through and finally untangling and help land aggregations in the spec! This MSC is one of four which describe how aggregations should work in Matrix, and it's great to see them finally being properly spec'd, especially as further features start to be built on top of them (such as threading!).

And finally, a further thank you to community members @ankur12-1610, @Dominaezzz for an OpenAPI schema fix!

🔗Random Spec of the Week

The random spec of the week is... MSC3395: Synthetic appservice events!

This proposal aims to allow appservices to get ever greater visibility into what is happening on the homeserver, while still maintaining full process separation.

A lot of the time solutions to complex problems require knowing when a user has registered or logged in, with what name/3pids etc, when users change their emails, etc. Hooking into these non-room-based actions can help with developing useful features. Maybe you want an appservice that plays a sound effect whenever a user signs up to your homeserver!

These days, that is often achieved by homeserver implementation-specific solutions, such as modules in Synapse. Being able to notify of these events using a standard API shape would be hugely beneficial to generalisation of projects.

So give the proposal a look over and review if that interests you!

Some exciting MSCs this week, I've been waiting for server-side aggregations for long!

🔗Dept of Servers 🏢

🔗Synapse ↗

Synapse is the reference homeserver for Matrix

callahad announces

As predicted last week, we released Synapse 1.47.1 on Tuesday. This is a security release which fixes an issue with Synapse's built-in media repository. Admins are strongly encouraged to upgrade.

Otherwise, relatively slow week: a handful of of the team have been away (Happy Thanksgiving, Americans!), but we did release Synapse 1.48.0rc1. Most importantly, this release candidate includes changes to improve the efficiency of large background updates from past releases, which should significantly reduce database load when upgrading. It also adds support for the /v3 APIs defined in version 1.1 of the Matrix specification.

We'll talk more about 1.48 when it's formally released next week, but as always, we appreciate folks trying out the release candidates and letting us know how they behave.

Administrators, keep your users safe: update as soon as you can!

🔗Homeserver Deployment 📥️

🔗Helm Chart ↗

Matrix Kubernetes applications packaged into helm charts

Ananace announces

A bit of an earlier update this week, but I wanted to make sure to note that my Hem Charts have been updated to matrix-synapse 1.47.1 for the security fix - and element-web has also been bumped to 1.9.5

🔗Dept of Bridges 🌉

🔗mautrix-googlechat

tulir reports

mautrix-googlechat has seen lots of improvements over the past few days. New features include:

• Bridging edits, deletions, reactions, formatting and read receipts in both directions (even /rainbow somewhat works from Matrix)
• Bridging typing notifications and any types of files from Matrix to Google Chat
• Bridging Google Meet links from Google Chat to Matrix
• Syncing group members from Google Chat

(edits and deletions are only available on Google Workspace accounts, not normal accounts. I have no idea why they did that, but that's just how Google Chat works 🤷)

There's still a bug where it sometimes silently stops receiving messages, which I'm currently trying to solve (or work around). After that I'll make a v0.3.0 release. Backfilling history may also happen in the near future

🔗Matrix Webhook Receiver ↗

An add-on for the matrix-appservice-webhooks bridge. Webhooks are essentially web interfaces for applications to "push" data to. The bridge can receive messages in a certain format, which is nice if the notifying app can be configured. Often it cannot.

kim announces

Matrix Webhook Receiver

Do you like to receive notifications in matrix? Matrix Webhook Receiver (MWR) is an add-on for the matrix-appservice-webhooks bridge. Webhooks are essentially web interfaces for applications to "push" data to. The bridge can receive messages in a certain format, which is nice if the notifying app can be configured. Often it cannot.

This is where MWR comes in: It can receive any (JSON) content, optionally reformat it nicely (customizable!), and forward it to the webhooks bridge which will post it to a room for you. If you are running any software service, there is a good chance it can notify you via webhooks!

Right now, several example configurations exist, ready for you to use:

• GitHub
• GitLab (including Community Edition/self hosted)
• Ansible Tower/AWX
• Grafana Alerts

For example, here is the GitHub webhook as seen in #matrix-webhook-receiver:matrix.org:

More examples of apps currently in development: Prometheus Alertmanager, Jellyfin

It is also easy to use to send messages from the commandline with standard tools (curl), e.g. for your cron jobs! See the README for an example.

Some other webhook matrix things exist, but often require

• ➖ admin access to server (appservices)
• ➖ installing and running and maintaining a specific bot per service
• ➖ create and set up accounts and passwords for each bot to use

In contrast, MWR requires:

• ➖ matrix-appservice-webhooks set up on your server by your admin
• ➖ some knowledge of how to send POST requests for one time setup. I want to improve that, suggestions are welcome!
• ➕ MWR can be installed and run by anyone who can access the bridge
• ➕ one single MWR supports any amount of notifying apps!
• ➕ you don't need admin access to the server to add more apps
• ➕ no need to develop a bot/plugin to support new apps, just write a quick jinja template
• ➕ multiple people can use a single MWR instance by sharing HTTP basic auth credentials

Other features:

• can post into encrypted rooms (see readme)
• automatically generated API docs
• message formatting presets including html markup or m.notice
• arbitrarily customizable webhook URLs
• list and manage currently installed webhooks in your browser (screenshot)

Links:

• Repo
• #matrix-webhook-receiver:matrix.org

What a massive update! At this rate I won't need to leave my Matrix client for anything!

🔗matrix-hookshot ↗

A multi purpose multi platform bridge, formerly known as matrix-github

Half-Shot says

Hey folks! Some exciting new news on the bridge front: I've renamed matrix-github to matrix-hookshot to better reflect it's not-just-GitHub-ness. That's not all though, as there are new features too:

• The bridge now supports Rust as a companion language (we're aiming to rewrite critical sections in rust). Some parts of the formatting code have already been rewritten.
• The bridge now supports JIRA (full puppeting!)
• The bridge now supports generic webhooks too, with the ability to write custom handling code inside the state event to process these hooks into pretty messages.
• Basic support for GitHub discussions.

In the works:

• A provisioning API to hook into integration managers
• More GitLab support
• Better GitHub discussions support

We're not quite ready for a 0.2.0 release, but please check us out at https://github.com/Half-Shot/matrix-hookshot.

An interesting update, and Half-Shot even demoes it in today's Matrix Live!

🔗Dept of Clients 📱

🔗Nheko ↗

Desktop client for Matrix using Qt and C++17.

Nico says

After the 0.9.0 release last week, we have of course been busy fixing all the bugs different people reported. Messing around with the sticker pack editor and then leaving room should not make Nheko crash anymore. The problems where the flatpak has issues starting on Gnome systems are still under investigation. We thought we had a solution, but that seems to have broken other stuff! 💥

Apart from that we have been doing some after release party cleanup. Apart from some refactorings, you can now filter your rooms on whether they are a direct chat or not in the sidebar. This is in addition to the filters we already had for favourites, spaces and your other personal tags. User colors should also now be much less biased towards blue and jdenticons should have more variance. Expect the next release to be a much more colorful experience!

Speaking of colors, Twily made this awesome ZX Spectrum inspired logo after we changed our Gitlab bot to be more colorful! Check it out:

Gorgeous, I love it!

🔗Hydrogen ↗

Hydrogen is a lightweight matrix client with legacy and mobile browser support

Bruno says

We're still distracted with SDK work and other things less visible for users, but this week we've also released 0.2.22 that fixes login on Element One (and other servers using SSO login and not yet supporting the experimental dehydrated devices).

🔗Element ↗

Everything related to Element but not strictly bound to a client

kittykat says

Threads

• On Web we’ve been working on Notifications and Badges, making sure no message goes unread.
• On Mobile we’re building out the new Threads Panel so you can easily see all the Threads in a room.

Polls

• The Polls team is making great progress, focusing their efforts on creating and voting on all platforms.
• User testing sessions are coming up!

Community Testing

• Bug squash session on Web on Tuesday 30 November at 17:00 UTC, join the call at #element-community-testing:matrix.org
• Upcoming plans include testing sessions on Polls and IA.

🔗Element Web/Desktop ↗

Secure and independent communication, connected via Matrix. Come talk with us in #element-web:matrix.org!

kittykat announces

• Work continues on Information Architecture: head over to Sidebar settings on develop.element.io to see what the team are currently working on.

🔗Element iOS ↗

Secure and independent communication for iOS, connected via Matrix. Come talk with us in #element-ios:matrix.org!

kittykat announces

• We’re still working hard on replacing Matomo with PostHog to improve how we collect analytics data and making the MatrixKit obsolete.
• Adding more features to Spaces and improvements to Spaces performance is also happening.

🔗Element Android ↗

Secure and independent communication for Android, connected via Matrix. Come talk with us in #element-android:matrix.org!

kittykat announces

• Voice message drafts and other improvements to this feature are underway.
• We’re also looking to integrate PostHog on Android.

🔗Dept of SDKs and Frameworks 🧰

🔗simplematrixbotlib ↗

simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio.

krazykirby99999 says

simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio. Version 2.4.0 provides several new features and a fix.

🔗New Features:

• Newlines are now supported when sending markdown messages.
• The msgtype of text and markdown messages can now be specified. Text and markdown messages can now optionally be sent as "m.notice" to avoid alerting everybody of the new message. The default msgtype will continue to be "m.text".

🔗New Fixes:

• Fixed issue where the homeserver was hardcoded in an http request.

Example usage is shown below:

import simplematrixbotlib as botlib

creds = botlib.Creds("https://home.server", "user", "pass")
bot = botlib.Bot(creds)
PREFIX = '!'


@bot.listener.on_message_event
async def echo(room, message):
    match = botlib.MessageMatch(room, message, bot, PREFIX)

    if match.is_not_from_this_bot() and match.prefix() and match.command(
            "echo"):
            
        response = " ".join(arg for arg in match.args())
        await bot.api.send_text_message(room.room_id, response, "m.notice") ## Uses the msgtype of m.notice instead of m.text

bot.run()

A thank you to HarHarLinks for their contributions to version 2.4.0!

Request additional features here.

View source on Github View package on PyPi View docs on readthedocs.io https://matrix.to/#/#simplematrixbotlib:matrix.org

🔗jOlm ↗

Olm bindings for Java

brevilo reports

This week saw three releases of jOlm which fix a native memory management issue, an Olm API (buffer) issue and add a few other improvements. Everyone is strongly encouraged to update to the latest release.

🔗Notes:

• ✅ Bugfix and maintenance releases
• ✅ Up to date with Olm 3.2.6

🔗Changelog:

• Fixed the backing store retention for all Olm instances
• Fixed a buffer issue in InboundGroupSession.decrypt()
• Ensured conversions of variable native strings are trimmed
• Centralized conversions to canonical JSON
• Completed initial set of unit tests (effectively full coverage now)

Cheers!

The only Java bindings of Olm to my knowledge, that's some very valuable work here!

🔗Dept of Ops 🛠

🔗synadm ↗

Command line admin tool for Synapse (Matrix reference homeserver)

jojo reports

synadm v0.32 is out!

My personal favorites of the new features are:

• Dates and times in several subcommands are translate the admin API's UNIX epoch timestamps to a human readable format.
• synadm room list now displays room aliases (#room:your.homeserver)

Read the full release notes here: https://github.com/JOJ0/synadm/releases/tag/v0.32

synadm is very useful as a homeserver administrator. Thanks JOJ0

🔗Dept of Bots 🤖

🔗maubot ↗

A plugin-based Matrix bot system.

tulir reports

maubot v0.2.0 was released last weekend. Highlights:

• Enabling encryption should be much easier: the device ID can be entered in the web UI or you can just do mbc auth --update-client to automatically log in and store the access token and device ID in maubot.
• mbc auth can now log in with SSO.
• The standalone mode for running a single plugin with a static config is now mostly functional and somewhat documented.

Also, I finally took a day to figure out Sphinx/autodoc and made some decent-looking autogenerated docs for mautrix-python. I'll probably extend that to generate maubot-specific API references too eventually.

🔗Dept of Interesting Projects 🛰️

🔗MinesTRIX ↗

A privacy focused social media based on MATRIX

Henri Carnot announces

Quick update on MinesTRIX (a privacy focused social media based on MATRIX). This week was focused on performance and stability.

• Changed database to use Fluffybox, this should greatly improve performances on web (thanks Famedly !)
• Scrolling through the posts of a profile now properly request history.
• Friend suggestions are now sorted according to the sum of user appearance in all rooms. Naïve, but it's the first step.
• Chat page has also been redesigned. Now support replies and reactions. Chat settings now display room avatar and fetch user list from server.
• Bug affecting MinesTRIX profile creation has been fixed. Login process should be way more stable now.
• Minestrix rooms sync has been rewritten to take into account sync events to rebuild the list.
• Debug page now allow forcing sorting rooms.
• Various post display enhancement (links are now clickable, thanks kellya!)

Come chat with us : #minestrix:carnot.cc

That's one exciting project, I can't wait to see how far it's going!

🔗Sign in with Matrix

Mish says

Federated sign-in component for your web app (using Matrix)

This week's update:

• Gained 180 stars on GitHub since release (thanks!)
• Added login states, accessible from the API
• Sign out
• Added CSS styling via variables
• Updated demo

more on https://github.com/mishushakov/signin-with-matrix

As last week, a note to keep in mind that this is a community project and that there is a MSC to make Matrix more OAuth2 friendly. More on that very soon!

🔗Dept of Built on Matrix 🏗️

🔗Matrix Forms

Mish reports

First release of "Matrix Forms", a project which redirects form submissions to designated Matrix rooms

Features:

• Server-side, no additional JavaScript
• Rich formatting
• Many forms on same instance
• File uploads
• Templates
• CORS
• Metadata accessible for bots

Can be installed using NodeJS or Docker

• demo: https://mishushakov.github.io/matrix-forms
• code: https://github.com/mishushakov/matrix-forms

🔗Final Thoughts 💭

🔗Server_Stats

MTRNord says

Today is a good day for those calling me a spy, someone not wanting to care about privacy and for those who did publicly harass me for server_stats.

People using the API likely already noticed it wasn't reachable for a while. Effective immediately I am currently leaving all rooms the bot is part of. This will take days or even months considering this are 6397 rooms at the time of writing. I am not going into the motives of why I am shutting it down. It comes down to personal reasons.

There won't be any dump of the data. The source will be kept public. Note though if anyone ever tries to run it that you need about 600GB of space for synapse, a lot of CPU, a lot of RAM and plenty of workers as this can easily crash synapse.

Server_Stats was an incredibly useful project. It pains me a lot to see it go, but it pains me even further that its author got harassed. This is not an acceptable behaviour, and we are better than that as a community. Thanks for this incredible project MTRNord, it's been both exciting and useful.

🔗Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

🔗#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	boba.best	413
2	kapsi.fi	508.5
3	maescool.be	528
4	envs.net	540.5
5	converser.eu	547
6	matrix.org	577
7	thesilentlink.org	838
8	matrix.markshorten.co.uk	1068
9	aria-net.org	1217
10	trygve.me	2137

🔗#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	dendrite.neilalexander.dev	430.5
2	grin.hu	555.5
3	matrix.org	1027
4	dendrite.matrix.org	1056
5	matrix.awesomesheep48.me	1191
6	sspaeth.de	1251.5
7	0x1a8510f2.space	3350.5
8	spooks.cyou	10963.5

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!