Matrix logo

We are happy to launch The Matrix Conference on Oct 15-18 in Strasbourg, France. Learn more about it, or submit a proposal!

This Week in Matrix 2022-01-21

2022-01-21 β€” This Week in Matrix β€” Thib

πŸ”—Matrix Live πŸŽ™

πŸ”—Dept of Spec πŸ“œ

anoa reports

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.

πŸ”—MSC Status

New MSCs:

MSCs with proposed Final Comment Period:

MSCs in Final Comment Period:

  • No MSCs are in FCP.

Merged MSCs:

πŸ”—Spec Updates

Work on preparing the release of Matrix v1.2 is currently underway. As of today, the Spec Core Team is aiming for a release of Matrix v1.2 on February 2nd.

If you know of any MSCs which you believe should be included in Matrix v1.2, but haven't started/finished Final Comment Period yet, please bring them up in the #sct-office:matrix.org, and we'll take a look. Thanks!

πŸ”—Random MSC of the Week

The random MSC of the week is... MSC3635: Early Media for VoIP!

When I first came across this MSC, I didn't know what "Early Media" as a concept referred to. Even after skimming the MSC... I still didn't really know. But I'll demystify it here in case it peaks your interest and you would like to learn more.

Early media is essentially any media that is exchanged between the moment you start a call, to the moment the other side picks up (a connection is established). For instance, the ringing you hear while waiting for someone to pick up (called "ringback tones"), or the "busy" tone you hear when the line is busy. Those audio bits are known as "early media". Video can also fall into this category (though that's less common).

This MSC would essentially allow Matrix to introduce these concepts in Matrix-only calls (though these days the client just plays sounds while connecting), but more crucially would allow Matrix to interoperate with other protocols (like SIP) that expect to handle these features.

So yay, more interoperability and bridge compatibility!

πŸ”—Dept of Servers 🏒

πŸ”—Synapse (website)

Synapse is the reference homeserver for Matrix

callahad says

After a somewhat bumpy process, we released Synapse 1.50 on Tuesday! The big thing you need to be aware of is that we're turning off support for Python 3.6 and PostgreSQL 9.6, including Linux distributions which ship with those versions by default (like Ubuntu 18.04 LTS). Please make sure your infrastructure is up-to-date.

I'm personally very excited that we tracked down a bug which could cause device list updates to get lost when being sent over federation. When device lists fall out of sync, it can cause failures when attempting to decrypt messages, since the keys may not have been sent to all of the user's devices.

We've also made quite a lot of progress in allowing Application Services to support end-to-end encryption via the still-experimental MSC3202.

For MSCs which have been merged into the Matrix Spec, we now implement MSC3419, allowing guests to send state events into rooms, and we now use stable identifiers for cross-signing and fallback keys per MSC1756 and MSC2732.

Looking to the future: We're aiming to release 1.51 next week so it has plenty of time to burn in before we host FOSDEM 2022. This is a pretty quick turnaround from 1.50, but we'll return to our usual fortnightly release cadence for subsequent releases. To that end, 1.51.0rc1 is out today; give it a shot. πŸ˜‰

πŸ”—Conduit (website)

Conduit is a simple, fast and reliable chat server powered by Matrix

Timo ⚑️ announces

Hello again! In the last week we continued to optimize the rocksdb backend and Conduit in general, trying to get it as memory efficient as possible. Using valgrind we could see that memory was not getting released in some cases.

We found out that switching to the jemalloc allocator completely got rid of this problem and memory usage seems to be a lot more stable now!

All of this is currently available on the next branch. We are preparing to make a v0.3.0 release soon!

Join #conduit:fachschaften.org to help us test it.

Thanks to everyone who supports me on Liberapay or Bitcoin!

πŸ”—Homeserver Deployment πŸ“₯️

πŸ”—Helm Chart (website)

Matrix Kubernetes applications packaged into helm charts

Ananace announces

And back to regular form, my Helm Charts have gotten some upgrades again; element-web got upgraded to 1.9.9, and matrix-synapse got both 1.50.0 and 1.50.1 (the configuration generated by the chart shouldn't be affected by the .0 bug, but always good to upgrade)

πŸ”—Dept of Clients πŸ“±

πŸ”—Watch The Matrix (website)

A watchOS client for Matrix

Doug announces

It's been a while since I reported any updates on Watch The Matrix.

The following new features have been added:

  • Message bubbles are properly left/right aligned now.
  • Support for displaying images.
  • Replies are nicely formatted now.
  • Ability to send messages and replies (inc FlickType support).
  • Redacted reactions are now properly hidden.

πŸ”—Sailtrix (website)

Sailtrix is a matrix client for SailfishOS

HengYeDev announces

This week, version 1.3.7.1 has been released.

Updates:

  • Support favorite rooms
  • Add View Source
  • Add useful active cover
  • Merge better notification support by @razcampagne

Fixes:

  • Fix bug of images and files not displaying in encrypted rooms
  • Add UI placeholder when there are no rooms

πŸ”—Element (website)

Everything related to Element but not strictly bound to a client

Danielle announces

Threads

  • To keep your timeline clean and ordered we’ll soon be introducing threaded messages to Element. Currently the team is working hard on the fallback solution, and polishing up the user interface.
    • Threads is already in Labs on Web, so go ahead and check it out!
    • For mobile, we’re hoping to release Threads to Labs in the next few weeks.

Polls

  • Get ready to start asking more questions… Polls are nearly ready to go! You’ll be able to ask folks things like; their favourite superhero, which day you should grab lunch, or even the best way to make tea (milk first, always). β˜•οΈ
    • If you just can’t wait β€˜til we’re ready to launch, you can enable Polls from Labs.

Location Sharing

  • Location sharing is almost here! You’ll soon find a new setting to turn this on, which will give you the location sharing icon in your composer and can be used to tell people exactly where you are!
    • The setting will be in the next release on iOS and Android, and will start β€œoff” by default. Once the feature is settled in, we’ll turn it on by default.

Community testing

  • Join #element-community-testing:matrix.org to help out with the testing
  • We will be testing the FOSDEM conferencing setup (Element web + widgets) on Monday at 17:00 UTC
  • Element Desktop (Nightly) has had an Electron update, join us to test it on Tuesday at 16:00 UTC

πŸ”—Element Web/Desktop (website)

Secure and independent communication, connected via Matrix. Come talk with us in #element-web:matrix.org!

Danielle reports

  • We’re moving forwards with our PostHog Analytics implementation and are super excited to start to get to know how our users experience Element. Remember; it’s off by default and you have to opt-in to share.
  • 2 testing sessions happening this week, join at #element-community-testing:matrix.org

In labs (you can enable labs in settings on develop.element.io or on Nightly)

  • With the coolest project name by far, the Bubbles team is working hard to bring you message bubbles ASAP! These should land in the next week or so but are available in Labs today.

πŸ”—Element iOS (website)

Secure and independent communication for iOS, connected via Matrix. Come talk with us in #element-ios:matrix.org!

Danielle says

  • The integration of analytics tracking has been included in the most recent version of the app. Using PostHog Analytics we’ll be able to make informed product decisions for our app as we’ll have more visibility into the usefulness of each feature.
    • Remember; Analytics is opt-in and you don’t have to share any info with us. If you choose to opt-in we’ll start to learn how users use Element and how we can simplify your experiences.

In development:

  • Work ongoing on Spaces support, finished improving room long press interactions for Spaces and reviews of space creation changes are nearing completion. Work ongoing on Space settings.
  • We’re currently building a simplified first time user experience, the first piece of which will be released in the next few weeks!
  • The team is working to update to Xcode 13 / iOS 15.

πŸ”—Element Android (website)

Secure and independent communication for Android, connected via Matrix. Come talk with us in #element-android:matrix.org!

Danielle reports

  • We’ve had some trouble with the stability of our releases this week but the team’s been working hard to get it all ironed out. Our latest update to the app store fixes some bugs and includes the option to enable analytics.
    • Analytics? Yes! Knowing how our users traverse our app, and understanding this cross-platform, will help us to tailor to your needs and make impactful improvements. If you don’t want to send anonymised event info to Element, no problem! Just say no. If you change your mind, there’s a toggle in Settings.

In development:

  • Message Bubbles? Yes, please! With a week of successful testing internally we’re nearly ready to release message bubbles into the wild. We’re excited to see what you think.
  • Element’s first time user experience could use a little help, so the team have been working on improvements to our sign up flow that will hopefully reduce confusion for newbies.

πŸ”—Dept of Non Chat Clients πŸŽ›οΈ

πŸ”—Matrix Wrench (website)

Matrix Wrench is a web client to tweak Matrix rooms.

ChristianP says

New feature: HTTP status in the network log.

Next up: Bulk editing of rooms for updating room power levels or aliases in masses. If you're a maintainer of a homeserver, space or bridge, please let me know your use cases.

πŸ”—export matrix messages (website)

A commandline utility to export matrix messages

Aine reports

It's a small cli tool that does exactly what it says - exports matrix messages from a room. As example you can check etke.cc/news - that page and all items on it generated by emm from #news:etke.cc room.

The tool gracefully supports room aliases, message edits, custom templates (check the contrib/ dir for example) and 2 export modes - single (all messages exported to a single file) or multi (each message exported to separate file, that's how etke.cc/news works)

Source code

πŸ”—Circles (website)

E2E encrypted social networking built on Matrix. Safe, private sharing for your friends, family, and community.

cvwright announces

Circles is a project to build a privacy-respecting, end-to-end encrypted social network on top of Matrix. It was originally built out of the desire for a safer way to share baby photos with friends and family, but it can be used by anyone who wants easy sharing combined with strong security.

Recent news:

  • The Circles iOS app is back in beta on TestFlight. Builds 0.99 (6) and 0.99 (8) are rolling out now.
  • The latest updates fix some bugs in Circles' use of Matrix's encrypted recovery feature to improve the reliability of E2E encryption.
  • FUTO, the new company behind Circles, is hiring an Android developer to help us bring Circles to Android. Interested candidates should send a resume and cover letter to [email protected].

The (old) Circles homepage: https://kombuchaprivacy.com/circles The code on Github: https://github.com/KombuchaPrivacy/circles-ios

πŸ”—Dept of Bots πŸ€–

πŸ”—Honoroit (website)

A Matrix helpdesk bot

Aine reports

long time no see! Today I come with a new internal etke.cc tool that publicly available, because open source matters.

Honoroit is a helpdesk matrix bot with end-to-end encryption support, that utilizes MSC3440 (Threading) to act as a proxy between a customer (any matrix user) and your backoffice (users added in special room), each customer's room = thread in a backoffice room, where multiple operators can send messages to the same customer at once.

Pretty bad description, I know - check the source code to see screenshots.

Updates:

  • the v0.9.2 release brings the fallback reply-to mode, so even on matrix clients that doesn't support threads yet you can use it with good ol' replies.
  • the v0.9.3 release fixes commands parsing in the reply-to mode and adds prefixes to the thread topics

source code, #honoroit:etke.cc

πŸ”—Dept of Interesting Projects πŸ›°οΈ

πŸ”—matrix-art (website)

A Devianart Fork based on Matrix

MTRNord (they/them) says

Matrix Art received some minor changes since the last twim post:

  • You can now get a rss feed of the posts at https://art.midnightthoughts.space/posts.rss (This is in the same format as devianart does their feeds)
  • Mobile should be mostly fixed
  • The About-info isn't hardcoded anymore but now uses an event type.
  • The page now uses the Roboto Font instead of the default font, which helps with readability.
  • Matrix Art cmes with a basic Text Logo now
  • Links to the profile page are now easier to notice
  • Blurhashes are now supported for images
  • The repo now has the Apache-2 License applied
  • Dependencies have been pinned
  • Various small design improvements were made
  • Images now have size hints so the page doesnt jump around as much when loading
  • Lots and lots of metadata was added to each page for SEO

Check it out at https://art.midnightthoughts.space Check the Code out at https://github.com/MTRNord/matrix-art Or join the Chat at #matrix-art:nordgedanken.dev

Planned for this week is to add registration and usage of external profiles. As soon as that works I am also going to make uploading images work. So with a bit of luck in the next TWIM anyone is able to post their images :)

πŸ”—Dept of Guides 🧭

Γ‰ibhear (on matrix.org) reports

Hi TWIM. I published a long-planned, long-in-the-making, long-winded report of how I containerised my matrix-synapse homeserver and its PostgreSQL database in order to get ahead of the application dependency deprecations. This was something I couldn't find for myself, so I put this together to help anyone else who might need it. (And it does contain a TL;DR section!)

Helder Ferreira says

I’ve created [blog post|(https://helderferreira.io/matrix-well-known-with-cloudflare/) explaining the way to host the static well-known files using cloudflare workers gaining speed and stability

πŸ”—Matrix in the News πŸ“°

Brendan Abolivier says

During the holidays I recorded a podcast about Matrix and Element with AnDaolVras/La Cantine Brestoise, which is a French non-profit organising tech events and operating a coworking space in Brest, France. The episode (in French, sorry!) came out on Monday and can be found on their website, on YouTube as well as on most podcast platforms πŸ™‚

πŸ”—Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

πŸ”—#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1envs.net517
2talkthings.net687
3aria-net.org821
4reactos.org869
5matrix.xn--kll-sna.net1109
6trygve.me1881
7minecraftchest1-matrix.loca.lt1922.5
8matx.myecloud.org2453
9thomcat.rocks2949
10asra.gr3546.5

πŸ”—#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1sspaeth.de367
2dendrite.matrix.org423
3rustybever.be606
4conduit.rs908.5
5dendrite.neilalexander.dev926
6matrix.awesomesheep48.me1160.5
7dendrite.beckmeyer.us1856
8conduit.cyberdi.sk3547

πŸ”—That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.50 released

2022-01-18 β€” Releases β€” Brendan Abolivier
Last update: 2022-01-18 17:58

Welcome all for the first Synapse release of 2022: Synapse 1.50!

Note that, as per our platform dependency deprecation policy, Synapse no longer supports Python 3.6 and PostgreSQL 9.6 as of this version. As a result, we have also stopped shipping Debian packages for Ubuntu 18.04 LTS (Bionic Beaver), as it ships with Python 3.6.

As a reminder, please note that Ubuntu 21.04 (Hirsute Hippo) reaches its own end of life on January 20, 2022. Past this date we will stop producing new packages for Ubuntu 21.04.

πŸ”—Encrypted application services

Application services (sometimes called "appservices"), are privileged processes that can interact with a Matrix homeserver in a way a normal user cannot. This is especially useful for bridges, as it allows them to register and puppet multiple users on the homeserver to replicate activity from other platforms.

One of the main shortcomings of application services currently is that they do not support end-to-end encryption. This means that messages sent through a bridge are never encrypted and always visible by the homeserver.

We've recently started work to tackle this issue in the form of MSC3202. A first part of implementing this MSC (allowing application services to masquerade as specific devices) has landed in this release of Synapse; work is still ongoing towards a full implementation, so watch this space!

πŸ”—Improved reliability on device list updates

While working on this release, we identified a long-standing bug that could prevent Synapse from sending device lists update over federation if the server had a high number of active users and/or users with a lot of devices connected to their account.

This bug was introduced back in Synapse 1.0.0, and meant that the homeserver would miss some device list updates when communicating with other homeservers if the amount of updates to send was too high. In practice, this means users on remote homeservers could see outdated device information for other users (including outdated device verification statuses).

Synapse 1.50 includes a fix to this bug. This should contribute towards making the propagation of device list updates more reliable.

πŸ”—Everything else

This release introduces support for MSC3419, which allows guest users to send arbitrary state events into a room. This will be especially useful to the ongoing work on group VoIP calls, which involves having users send new state events into the room to signal their participation in a call.

We've also stabilised identifiers for cross-signing and fallback keys now that MSC1756 and MSC2732 have been merged into the Matrix spec.

On the documentation side of things, the page on setting up and configuring a TURN server has been updated to feature instructions on how to deal with NATs. This is a much welcome addition as configuring TURN is something a lot of Synapse admins struggle with!

Please see the Synapse release notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including Dirk Klimpel, Donny Johnson and AndrewFerr.

Note: An issue preventing client logins (#11763) was identified immediately following the release of Synapse 1.50.0. We released Synapse 1.50.1 the same day with a fix for this issue.

This Week in Matrix 2022-01-14

2022-01-14 β€” This Week in Matrix β€” Thib

πŸ”—Matrix Live πŸŽ™

πŸ”—Dept of Spec πŸ“œ

anoa announces

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.

πŸ”—MSC Status

New MSCs:

MSCs with proposed Final Comment Period:

MSCs in Final Comment Period:

Closed MSCs:

πŸ”—Spec Updates

Hot on the heels (relatively speaking) of v1.1 of the Spec being released in November, v1.2 is now on the horizon! As a reminder, we're working towards quarterly releases of the spec going forwards - no hard dates yet though.

While this is certainly an improvement in speed, when it comes to writing software and updating implementations: quarterly spec updates may actually seem too fast. This is OK; implementations of the spec are not expected to update as soon as a new spec release is published. Rather, it is more realistic to expect that the ecosystem updates gradually over the course of the next few months/year after the release.

Look forward to more concrete dates soon!

πŸ”—Random MSC of the Week

The random MSC of the week is... MSC2702: Specifying semantics for Content-Disposition on media!

This looks like a small, but useful improvement to the spec that never really received much love. Feel free to give it some!

πŸ”—Dept of Servers 🏒

πŸ”—Synapse (website)

Synapse is the reference homeserver for Matrix

anoa says

Happy Friday everyone! Today the Synapse team has released Synapse 1.50.0rc2. It fixes a particularly nasty federation-breaking regression that crept in during 1.50.0rc1. If you're currently running 1.50.0rc1, we implore you to update to 1.50.0rc2 as soon as possible!

In other news, a reminder that as of Synapse 1.50.0rc1, we have ended support for Python 3.6 and PostgreSQL 9.6 as per our dependency deprecation policy, as upstream has marked them as end-of-life.

We'll let the community test 1.50.0rc2 over the weekend to ensure that no other regressions have emerged (please test if you can)! If not, expect Synapse 1.50 to land sometime early next week.

πŸ”—Conduit (website)

Conduit is a simple, fast and reliable chat server powered by Matrix

Timo ⚑️ reports

It has been a long time since my last TWIM post, so here's a summary of all exciting news:

Features:

  • We now support RocksDB as a backend. Migrating to RocksDB improved both my memory and IO usage a lot.
  • You can select the database backend at runtime. No need to recompile a special version anymore.
  • Lazy loading works. This means initial syncs are significantly faster (1-2 minutes instead of an hour!)
  • Voice calls are now supported (requires configuring a TURN server)
  • The report feature is now implemented and sends a report into the admin room

Improvements:

  • Much faster state resolution
  • Batch inserts for events
  • Better appservice docs
  • Lots of bug fixes

All of this is currently available on the next branch. We are preparing to make a v0.3.0 release soon!

Join #conduit:fachschaften.org to help us test it.

Thanks to everyone who supports me on Liberapay or Bitcoin!

πŸ”—Dept of Bridges πŸŒ‰

πŸ”—Heisenbridge (website)

Heisenbridge is a bouncer-style Matrix IRC bridge.

hifi announces

Release v1.10.0 πŸ₯³

  • RELAYMSG sending support πŸš€
  • Allow forwarding all IRC noise to network room 🀫
  • Support ZNC self-message caps βœ…
  • Support CHGHOST caps (prevents leave+join on host change) βœ…
  • Fix owner auto-registration (regression) πŸ›
  • Upgrade to Mautrix 0.14 ⬆️

Sending RELAYMSGs have been requested for a while so now we do support that if the cap is added to request list. This makes plumbs on networks that support it work nicer. Receiving RELAYMSGs has not been implemented yet so they show up as external like before.

If you prefer keeping your IRC rooms clean without any IRC noise (mode changes etc.) you can now use a new FORWARD command in the network room to make all such events happen in the network room instead. This affects in-room commands as well.

ZNC users can now enjoy messages coming from your own clone in IRC rooms (including PMs!) if you wish so by keeping the znc.in/self-message cap enabled. This isn't extremely well tested yet but feedback welcome.

Not a breaking change but the caps support will cause connections to networks that ignore CAP requests take a few seconds longer unless you remove all the default caps for said network and it will never try requesting them again.

Mautrix 0.14 upgrade bumps the minimum version as well so packages beware.

Get some global warming from GitHub, PyPI or matrix-docker-ansible-deploy!

Thanks!

πŸ”—Dept of Clients πŸ“±

πŸ”—Nheko (website)

Desktop client for Matrix using Qt and C++17.

Nico announces

We rewrote the whole settings page (as a stepping stone to 100% QML). Please test it and complain about everything I broke! We also tried to organize it a bit better so that similar settings are grouped together and tried to use the right controls for the right things. Best case you didn't notice anything. ;-)

And we also now show which profile a notification is for on KDE. I don't think other DEs have such a feature, so those will just show the generic Nheko as always.

πŸ”—Hydrogen (website)

Hydrogen is a lightweight matrix client with legacy and mobile browser support

Bruno reports

We're about to release 0.2.23 after somewhat of a release hiatus. We've been working on two fronts:

  • Get the SDK out! Hydrogen was always meant to be easy to embedded, reuse in parts, and make customized versions of. But until now, actually doing that was quite challenging. Now that the SDK is out, you can use the hydrogen-view-sdk package in your projects and use parts of Hydrogen in your application. It's still early days, and we're still working out what symbols should be exported (finding a balance between APIs we want to support and utility), etc. Expect updates in the coming weeks. We're also not yet promising API stability for now, some APIs will very likely still change. Once we hit 1.0, we won't change things from underneath you anymore without increasing the major version.
  • Rich reply previews: as part of providing minimal support for threads (representing them as replies), we're switching from using the embedded reply fallback and actually looking up the replied-to message. One benefit is that reply previews will now updated when they are redacted (and edited, once we support that :).
  • Also fix some minor other things, like loading images when they are only partially visible, and a very basic location tile.

πŸ”—Element (website)

Everything related to Element but not strictly bound to a client

kittykat announces

Threads

  • Issue with thread panel rendering timeline has been fixed
  • Reviewing plans for backward compatibility with clients that don’t support threaded messages

Polls

  • iOS and Android starting on the next phase of development, look out for poll editing and other exciting changes! You can enable polls in the Settings, under Labs.

Community Testing

Analytics

  • You can now opt in to sending anonymous analytics data in the user settings on all platforms. Please enable it in the β€œSecurity & Privacy” menu, under β€œAnalytics” to help us understand better how Element apps are used.
  • Mobile app users will see an opt-in screen on first startup on Android in 1.3.14 and iOS in 1.6.12

πŸ”—Element Web/Desktop (website)

Secure and independent communication, connected via Matrix. Come talk with us in #element-web:matrix.org!

kittykat announces

  • Maximised widgets merged into the release candidate and on track for the next release (see the Beyond Chat section for more info on this feature)
  • Fix code blocks being wrongly wrapped causing the line numbers to misalign, this was a regression in 1.9.8
  • Fixed ability to edit horizontal rules in markdown
  • Fixed some edge cases around Spaces not updating properly causing rooms to show up in the wrong Space
  • Add ability to cancel an outbound message during its encryption phase
  • Fix wrongly sending typing indicator when restoring a draft when changing room
  • Replace kick terminology with remove to be more inclusive
  • In labs (you can enable labs in Settings on develop.element.io or on Nightly)
    • Polishing bubbles layout
    • Upgrade to Electron 16 for Element Nightly - we are expecting this to help resolve some of the issues caused by Electron. Intending to release to production in 1.9.10 (two weeks away)

J. Ryan Stinnett announces

The maximised widgets feature in Element Web that Timo K. has been working on is now available for everyone on develop, and it's on track for the next web release. πŸš€ It's a great match for widget-based collaborative editors, dashboards, games, and more. The widget becomes the focus of the room. You can optionally show chat from the room in a side panel, allowing easy discussion of the document / dashboard / game.

To try it out, add a widget to a room in the usual way, then look for the new "maximise" button in the room info panel's widget section. Please let us know in #beyond-chat:matrix.org or in issues if you have any feedback. πŸ˜„

πŸ”—Element iOS (website)

Secure and independent communication for iOS, connected via Matrix. Come talk with us in #element-ios:matrix.org!

kittykat reports

  • Working on improving app startup speed (note that this is the green spinner, not the account sync)
  • Issue with home view in dark mode was reported this morning and fixed this afternoon
  • In development:

πŸ”—Element Android (website)

Secure and independent communication for Android, connected via Matrix. Come talk with us in #element-android:matrix.org!

kittykat says

  • First time user experiences (FTUE) changes are starting to land, including changes to the login screen which will be available in next week’s release.
  • Replace kick terminology with remove to be more inclusive
  • Resolved issue with stuck event in the timeline, which will be released with 1.3.16

πŸ”—Beeper (website)

All you chats in one app.

Brad Murray announces

Beeper is a universal chat app built on top of Matrix. We've created 12+ open source Matrix bridges and integrated them into an easy to use all-in-one service which does not require setting up your own homeserver. You can learn more at beeper.com.

Our team is growing! We’re now at 25 people, all remote around the world. Recent additions include hifi (creator of Heisenbridge) and Finn (creator of signald).

We are hiring many full-time remote roles including:

  • Bridge developers
  • iOS engineers
  • Product designers

Learn more here and apply through that site, or message @eric:beeper.com

Beeper Desktop

  • We recently released a Beeper Desktop update with a new room list and a ton of UI improvements. Check out the video below!
    • Made it easier to triage your inbox by moving unread dots to left
    • Made the list of connected bridges more compact
    • New link previews
    • Loads of bug fixes including improvements to scrolling

Beeper Bridges

  • We introduced a new version of our iMessage bridge designed for Mac OS computers with SIP disabled. https://docs.mau.fi/bridges/go/imessage/mac-nosip/setup.html Now includes support for:

    • Tapbacks
    • SMS via Continuity
    • Threaded replies
    • Synced read states to other iDevices and

    Most importantly, we launched a cloud Mac service that allows you to use iMessage with Beeper if you do not have access to an always-on Mac computer. Included with your Beeper subscription!

  • Voice messages are bridged in native format in both directions for all Beeper bridges

  • Signal bridge now supports disappearing messages

  • All Instagram message types are now bridged

  • Added support for Telegram message reactions

  • WhatsApp bridge now backfills 3 months worth of chats

Beeper Android

  • Coming soon: A recent addition to our Android team has added encrypted search by massaging seshat into the app. Expect an upstream PR for this into Element Android as well!

πŸ”—Dept of SDKs and Frameworks 🧰

πŸ”—Trixnity (website)

Multiplatform Kotlin SDK for Matrix

Benedict says

Trixnity version 1.0.0 is out! I decided to make it 1.0.0 not because it is out of beta, but because it has most features you need to build a usable client. That means: cross signing has landed into Trixnity! The next big features will be room key backup and push notification.

Here is the changelog:

  • fully support cross signing
  • load members of rooms in an async way (you don't need to catch errors anymore)
  • reactive displayname and avatar url of the logged-in user
  • reactive avatar url for rooms
  • reactive is-direct-room state for rooms
  • better Android support for thumbnails
  • make part of device keys API public
  • merge SecureStore into Store (encrypt your database if you want to keep secrets secure)
  • move trixnity-client-api model-classes into separate module to use them e.g. for a matrix server implementation (thanks to @NicolasJouanin)
  • fixed long standing bug of wrong room name calculation
  • account data events are handled as if they have a key (like state events) to bypass inconsistency in the spec (maybe this will lead into a MSC)

πŸ”—ruby-matrix-sdk (website)

Ruby SDK for the Matrix communication protocol

Ananace says

Just cut a new release of the Ruby Matrix SDK, with 2.5.0 there's some preliminary support for Matrix 1.1 and the client/v3 API, the information for room knocking is exposed properly, some threading issues have had workarounds applied, and a bunch of fixes have been applied.

Additional thanks go out to the nice people submitting PRs on GitHub and the members of the #ruby-matrix-sdk:kittenface.studio room.

Ananace says

Currently running my own ping bot with it, I'm doing the GitHub (and hopefully also GitLab at some point) releasetracker bot, I've got my definitely not suitable for production MatrixFS, there's a notification module for TheForeman which we use at the university. And I've got some MSC hacks as well, like the MSC2108 testbed.

πŸ”—matrix-crdt (website)

Use Matrix as a backend for local-first applications with the Matrix-CRDT Yjs provider.

yousefed reports

I just open sourced a library called Matrix-CRDT: https://github.com/yousefED/matrix-crdt - feedback very welcome! It allows you to use Matrix as a backend for decentralized, local-first collaborative apps. Above you see a collaborative rich text editor (like Google Docs) powered by Matrix!

You can try the Rich text editor here: https://bup9l.csb.app (see also the links in the repo, e.g. there is also a collaborative todo-list example)

J. Ryan Stinnett adds

If you have a distributed data structure and an algorithm that ensures all participants end up with the same result when their actions are combined, then that's effectively a CRDT as well. For Matrix, there a few research papers like https://arxiv.org/abs/2011.06488 which examine the CRDT-like properties.

πŸ”—Dept of Bots πŸ€–

πŸ”—GH-Bot (website)

The worst (according to its author!) but simplest webhook bot for GitHub and Matrix.

Jae Lo Presti announces

First stable release of the gh-bot which is a simple webhook to Matrix bot made in Python. As of now, the bot supports webhooks from Github, Gitlab and Gitea (although support for this one is very light). Some other projects might do that way better than this bot but this is a learning project, to learn step by step how client interact with servers. The next step would be to add a CRT.sh integration to get notified when a certificate is issued for a certain range of domains (which is something we want since we saw an IRC bot do the same).

πŸ”—Dept of Interesting Projects πŸ›°οΈ

πŸ”—matrix-art (website)

A Devianart Fork based on Matrix

MTRNord (they/them) says

Matrix-Art is a new social network prototype on Matrix.

It is a direct Devianart style clone. It currently has a focus on only images but is going to get extended to other media types eventually. I am doing this as a toy project, so it may sometimes have slow progress. The goal is covering the main functions Devianart provides (Posting, Sharing, Profiles, Following, Collections, Comments) as well as integrations to other social networks on matrix using MSC3639.

A lot of things are currently still missing, but I am trying to get uploading working next :)

An instance is hosted at https://art.midnightthoughts.space/ and the code is at https://github.com/MTRNord/matrix-art

Currently, the interface is limited to viewing basic data from a matrix room. In the future, there are plans to have an open registration as well as login with any account to use it. :)

Note however that I suggest against using your personal account just yet, even though the login works, as it may have unexpected room joins at this time. This is known and expected, but not production ready.

Also, at this time, there are no plans of e2ee support. This may get added after the main features are finished.

For questions, feel free to join #matrix-art:nordgedanken.dev :)

πŸ”—Dept of Guides 🧭

Stas'M says

I just wrote an article about my experience using Matrix, and the question... is it worth posting in the news? The article is written in Russian and published on Habr... and since it's pretty big I'm not sure about translating it to English πŸ˜…

Here is the link: https://habr.com/ru/post/599777/ - it also mentions TWIM.

πŸ”—Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

πŸ”—#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1envs.net442
2mtx.koyax.org584
3kapsi.fi698
4finallycoffee.eu877.5
5aria-net.org1032
6trygve.me1049
7fff.chat1528
8samlord.me1551
9bitcoin.ninja1598
10kreatea.space1845.5

πŸ”—#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1conduit.supercable.onl397.5
2rustybever.be655
3dendrite.thomcat.rocks672.5
4dendrite.matrix.org794.5
5dendrite.neilalexander.dev958
6matrix.spooks.cyou1405
7matrix.awesomesheep48.me1534
8conduit.cyberdi.sk3041
9dendrite.beckmeyer.us6883

πŸ”—That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2022-01-07

2022-01-07 β€” This Week in Matrix β€” Thib
Last update: 2022-01-07 19:18

πŸ”—Matrix Live πŸŽ™

What a pleasure it is to be back with the community for the new year! This week Erlend is detailing what Commune is, and I'm flabbergasted by how well thought-through the project is.

πŸ”—Dept of we've been away for holidays

This week in Matrix should be called Three Weeks in Matrix, since there hasn't been TWIM updates during the holiday season. Nico has published a first and second communitwim while I was away. All the news reports since the last official TWIM still made it to the post you're currently reading!

πŸ”—Dept of Spec πŸ“œ

TravisR says

Your regular spec person, anoa, is out today so you're stuck with me, not-anoa. This time I got the script to work though πŸ˜‡

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

πŸ”—MSC Status

Note: This is for the last 3 weeks given TWIM holiday :)

Merged MSCs:

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Closed MSCs: Sadly, not every MSC makes it to the end.

πŸ”—Spec Core Team

In terms of Spec Core Team MSC focus for this week the last little bit, we've been working on getting MSCs merged into the formal spec itself in preparation for v1.2 this quarter. We expect the release to happen quite soon and to contain Spaces, room versions 8 and 9, and refresh tokens (no spec PR yet) in addition to all the other wonderful stuff which has landed.

The merged MSCs can be seen over at https://spec.matrix.org/unstable/ where they're queued up for the next release. Look out for Added in v1.2 labels throughout the spec, denoting what is new and exciting.

πŸ”—Random MSC of the week

The random MSC this week is MSC2192: Inline widgets (I promise it was random, after a few re-rolls). Inline widgets are a concept that allows for rich functionality in the timeline without needing to necessarily specify an explicit event type. For example, video embeds, minigames, etc could all be represented by inline widgets instead of dedicated event types.

The MSC needs updating to handle MSC1767: Extensible Events (and friends), but once there it could be a very powerful bit of functionality, with free fallback thanks to Extensible Events.

πŸ”—The graph

Here's a stacked graph of MSC progress:

Kegan announces

Sliding sync (aka sync v3) is described in MSC3575 - it's still very early days for the proposal which means a lot can change: and YOU can be a part of that. Please take a look at the MSC and provide any and all feedback, be it on the names of keys, format of JSON objects, or the kinds of operations that can be performed. I'm particularly interested in feedback from client developers who have complex room list sort orders or room list filtering requirements and bot/bridge developers who typically don't have a visible room list UI. Any and all feedback at this stage is welcome, visit #sliding-sync:matrix.org to join the discussion.

πŸ”—Dept of Servers 🏒

πŸ”—Synapse (website)

Synapse is the reference homeserver for Matrix

callahad announces

We're back! New year, new release candidate: we published Synapse 1.50.0rc1 today, marking the end of life for our Python 3.6 and PostgreSQL 9.6 support. We've also extracted some shared utilities into their own package, matrix-common, which is used by Synapse, Sygnal, and Sydent.

We'll talk more about what's in 1.50 next week when it formally releases.

The Synapse team is expecting to spend most of January on behind-the-scenes work as we gear up to virtually host FOSDEM again! Members of the team will be presenting two talks this year: Shay on Events for the Uninitiated, and Brendan on Beyond the Matrix: Extend the capabilities of your Synapse homeserver. Check out the full devroom schedule here.

πŸ”—matrix-media-repo (website)

Matrix media repository with multi-domain in mind.

TravisR says

v1.2.9 v1.2.10 got released as a maintenance update. With early support for Matrix 1.1, S3 storage classes, and blurhash fixes it's worth the upgrade though there are other goodies - check out the changelog, and report bugs to the issue tracker πŸ™‚

πŸ”—Homeserver Deployment πŸ“₯️

πŸ”—Helm Chart (website)

Matrix Kubernetes applications packaged into helm charts

Ananace reports

And as a end-of-year update, my Helm Charts have gotten updated yet again. With element-web ending up on 1.9.8, matrix-synapse on 1.49.2 (and .1 before that), and matrix-media-repo on 1.2.10

πŸ”—Dendrite Helm Chart (website)

Helm Chart to deploy Dendrite on Kubernetes

jonnobrow reports

The first iteration of a Dendrite Helm Chart has been released under the k8s-at-home charts repository. It currently supports a full monolithic deployment and requires minimal configuration to get up and running (just need to generate a matrix key and mount it as per the instructions).

If polylith is your thing then I recommend the chart by S7evinK for now, although it is likely these charts will merge in the future.

The chart and documentation are available here: https://github.com/k8s-at-home/charts/tree/master/charts/incubator/dendrite

πŸ”—Dept of Bridges πŸŒ‰

πŸ”—Matrix Webhook Receiver (website)

An add-on for the matrix-appservice-webhooks bridge. Webhooks are essentially web interfaces for applications to "push" data to. The bridge can receive messages in a certain format, which is nice if the notifying app can be configured. Often it cannot.

kim says

matrix-webhook-receiver has hit 1.1.0! Give feedback or talk to us about it over in #matrix-webhook-receiver:matrix.org! πŸ™‚

πŸ”—Features

  • Advanced Templating! It is now possible to set format and msgtype based on arbitrary values from the webhook JSON via Jinja2. Shoutout to qg for suggesting this and giving their feedback!
  • compatibility with matrix-appservice-webhooks forks that read avatarUrl instead of avatar_url
  • allow mxc:// URL avatars
  • improvements to the GUI, including automatically resizing text areas and msc:// avatar URL preview
  • improvements to templates/examples including making use of above features
  • more documentation, including Tips & Tricks and Related Projects

πŸ”—Notable Fixes

Full Changelog: https://github.com/HarHarLinks/matrix-webhook-receiver/compare/1.0.0...1.1.0

πŸ”—matrix-hookshot (website)

A multi purpose multi platform bridge, formerly known as matrix-github

Half-Shot says

Howdy folks, it is the time of the year where everyone scarpers! Anyway, perfect time to announce that matrix-hookshot has gotten it's first major release!. 1.0.0 is here!

For those not in the know, the hookshot bridge is used to bridge GitHub, GitLab, JIRA and Generic Webhooks into Matrix rooms. It doesn't just bridge into existing rooms, but can also spawn dynamic rooms based on aliases, send you your notifications in a DM and do lots of other wonderful things!

The notable changes from the 0.1.0 release are:

  • The bridge has now been renamed from matrix-github to matrix-hookshot.
  • Now supports JIRA and Generic Webhooks in addition to GitHub and GitLab.
  • Includes new commands and metrics reporting.
  • Includes complete documentation.

You can get involved and start playing with it by checking out the release here

And that will be my last TWIM entry of the year. Have a good one and stay safe all 🐢

That was in 2021! Half-Shot is back in 2022 with more news

Hey folks! It was only last year we released the 1.0 release of hookshot, after many years of work to get it that far. I'm happy to announce that this week we've got another release. There are a number of buxfixes and improved documentation pieces landing (special shoutout to HarHarLinks for ensuring the docs are competent). The highlights are as follows:

  • Added support for Figma webhooks. this also means the archival of my old project matrix-figma
  • Support GitLab wiki page change events.
  • Added a new script validate-config which allows you to check your config file for simple errors. Handy for people writing ansible roles!
  • Add support for a html key on generic webhooks to set the HTML content of a Matrix message.

The project can be found over at https://github.com/Half-Shot/matrix-hookshot/, with pretty pretty docs at https://half-shot.github.io/matrix-hookshot. We're also in #hookshot:half-shot.uk if you prefer using Matrix to learn about these things!

πŸ”—Heisenbridge (website)

Heisenbridge is a bouncer-style Matrix IRC bridge.

hifi announces

Heisenbridge roundup!

Release v1.8.0 v1.8.1 v1.8.2 v1.9.0 πŸ₯³

  • Spaces support πŸŒƒ
  • Sort NAMES reply nicks πŸ”€
  • UNPLUMB network command to force unplumbing without being in the room
  • Proper SASL external with CertFP with mechanism override option (see notes)
  • Disconnect and cleanup from networks that have no rooms open ♻️
  • Reply (and reject) DM requests to ghosts with QUERY command β†ͺ️
  • Try to keep IRC users in the room at all costs if they are on the IRC channel
  • Fix assumption of all IRC replies to have arguments
  • Prevent accidental namespace changes to cause mayhem
  • Finally convert from homegrown Matrix API stuff to Mautrix
  • Bump Mautrix requirement to 0.12=>0.14
  • Conduit support was broken in 1.8.x but fixed again in 1.9.0, sorry

Finally there's network level spaces support with a new SPACE command. This creates a new bridge controlled space for the network and automatically manages rooms in and out. There's an issue/feature with Element that all rooms that have been converted to DMs with /converttodm will appear in all bridge spaces. The workaround is to convert them back to regular rooms.

CertFP SASL has been updated to do SASL external flow by default. If you are upgrading and have used CertFP with OFTC you need to run SASL --mechanism=none for it to connect again.

Abandoned networks where the user has left all rooms including the network room will now automatically disconnect and cleanup. This is more in line what people would expect and prevents idle connections from hanging around.

Get your third vaccination from GitHub, PyPI or matrix-docker-ansible-deploy!

Thanks!

πŸ”—Dept of Clients πŸ“±

πŸ”—Nheko (website)

Desktop client for Matrix using Qt and C++17.

Nico announces

We just released 0.9.1!

This is a small bug fix release. If you reported an issue, there is a 15% chance it is fixed now! This release also supports pinned messages, although those will only show up after someone changed the pinned messages in a room currently (we didn't want to force a full resync just for such a small feature). The spaces list is also now nested, Nheko offers you quick access to your recently used reactions and Nheko will show you your direct chats in the sidebar. Apart from that there are quite a few bugfixes and smaller improvements, you can find the full changelog and downloads here: https://github.com/Nheko-Reborn/nheko/releases/tag/v0.9.1

Thank you everyone, who helped shape this release!

Nico reports

Nheko now is a lot more efficient. We now use one Threadpool instead of 3, got rid of more than 60% of the allocations when scrolling through messages, layout half as much content when scrolling, blurhashes decode in 10% of the time and jdenticons allocate ~10% as much temporary buffers. We also deleted around 1000 lines of unused code. Tooltips also shouldn't steal the mouse focus when scrolling anymore, which could lead to sudden stalls when scrolling.

Additionally edits now replace existing notifications, tastytea added a manpage and fixed blurry or incorrectly sized custom emoji, you can now send custom emotes via the inline completer using ~ and completers now show a scrollable list with more Elements than before. Advanced users can also now opt into an insecure client side secrets storage via a hidden setting.

Nico says

We're baaaaack! Anyway, last week Drake just translated all of Nheko into Spanish, you can now zoom in and pan in the image viewer, emojis shouldn't split up into their segments anymore and Nheko should always be sending the qualified version of it (according the to the unicode test files, not by just appending FE0F). Blurhashes should be even faster still and we now have support for running the call event loop on macOS and Windows (although call support is still disabled there for now).

We are also now working to restructure our README. It has a lot of outdated stuff in it and you really want to see screenshots pretty early in the README! You can sneak a peak here: https://nheko.im/nheko-reborn/nheko/-/tree/README_updates

πŸ”—Element (website)

Everything related to Element but not strictly bound to a client

Danielle reports

Welcome to a new year at Element!

Threads

  • This week the Threads team has focussed their efforts on improving some of the smaller details and pesky bugs we’ve found in our first rounds of testing.
    • If you’d like to help us test Threads we’ll be asking for help in the Community Testing room over the next few weeks. Join#element-community-testing:matrix.org to stay in the loop.

Polls

  • Polls is available in Labs on all clients! While we know there are some minor improvements to be made, we’re proud of where we are and would love for you to start using it!

Community Testing

  • Our next session will be on Wednesday, 12th January at 16:00 UTC (17:00 CET). We will be testing the new release candidates on all three platforms! Join us at#element-community-testing:matrix.org

πŸ”—Element Web/Desktop (website)

Secure and independent communication, connected via Matrix. Come talk with us in #element-web:matrix.org!

Danielle reports

  • Work continues on the integration of PostHog analytics. With your explicit permissions we’ll receive anonymous usage data that will allow us to understand the areas of Element that are helpful (or not). This info will help fuel things like our Information Architecture project.
  • In labs (you can enable labs in settings on develop.element.io or on Nightly)
    • Information Architecture improvements are still being worked on - try it out by enabling things like the new Spotlight search and Breadcrumbs.
    • Message Bubbles are improving; We’ve been hard at work preparing them for a release in the coming weeks.

πŸ”—Element iOS (website)

Secure and independent communication for iOS, connected via Matrix. Come talk with us in #element-ios:matrix.org!

Danielle announces

  • During the Christmas break we increase the speed of the app launch by 7x! Once stabilised this update will land with you.
  • Analytics changes have been merged into the RC and opt-in will be available soon.
  • In development:
    • More Spaces improvements are underway; You’ll soon be able to add rooms to Spaces, update room settings, and have room interactions on the β€˜long-press’.
    • Improved onboarding! We’re hoping to make the first few steps in Element easy by simplifying some of the first tasks users take in the app, including signing up.
    • We’ve started building Message Bubbles on iOS. This will help to distinguish the messages you send from the messages you receive.

πŸ”—Element Android (website)

Secure and independent communication for Android, connected via Matrix. Come talk with us in #element-android:matrix.org!

Danielle says

  • For Android we now have a PR review board to increase visibility into the state of PRs submitted by external contributors!
  • The new Opt-in screen for PostHog analytics tracking will be released soon.
  • Work on Message Bubbles has started and we can’t wait to share it with you!
    • If you have any feedback or questions about Messages Bubbles, please let us know now.
  • We’re also working on improving the sign up and sign in experience with some micro-improvements landing in the product soon.
  • We’re sorry for the delay of Element on F-Droid store, it will be available next week.

πŸ”—Dept of Non Chat Clients πŸŽ›οΈ

πŸ”—Populus Viewer (website)

A Social Annotation Tool Powered by Matrix

gleachkr says

A lot has happened with populus-viewer!

  • We now implement MSC2574! This MSC proposes a standard format for marking up resources (PDFs, other document formats, audiovisual media, websites, geospatial data...) using Matrix.
  • Room avatars are now displayed in the welcome view for PDF rooms
  • There's now a UI for using spaces to manage and share PDF collections
  • Reactions now work like element-web: click to mirror an existing reaction or redact your previous reaction

Next, I'm aiming to implement MSC3592, which proposes a standard format for basic markup on PDFs. Stay tuned! And if you're interested in learning more, come visit #opentower:matrix.org.

πŸ”—Matrix Wrench (website)

Matrix Wrench is a web client to tweak Matrix rooms.

ChristianP announces

Matrix Wrench is a web client to tweak Matrix rooms. After formerly calling it Matrix Navigator or Matrix Screwdriver, I finally settled on the name Matrix Wrench. Β―\_(ツ)_/Β― Version v0.2.0 comes with a Network Log which displays curl equivalents for all network requests. It also allows to easily add and remove room aliases. https://gitlab.com/jaller94/matrix-wrench/

πŸ”—matrix-streamchat (website)

Matrix powered stream overlay for OBS, to integrate live chat in your favorite (selfhosted) streaming setups.

f0x says

there's a pink theme and irc styling for matrix-streamchat now

πŸ”—Cactus Comments (website)

Cactus Comments is a federated comment system for the web, based on the Matrix protocol.

AsbjΓΈrn reports

Cactus Comments is a comment system for the open web, built on Matrix.

We released version v0.11.0 of the client!

The client has been relicensed to LGPL. This means that you can now use Cactus Comments in non-GPL compatible projects. But mainly, this release brings a bunch of CSS improvements: introducing automatic dark mode, and making it easier to change colors with CSS variables.

Here's the changelog:

  • Relicense from GPLv3 to LGPLv3.
  • Rewrite large parts of the stylesheet to use flexbox.
  • Introduce CSS variables to the stylesheet.
  • .dark and .light CSS classes with default values for dark/light mode.
  • Bugfix: "View More" button no longer blinks when auto-refreshing short comment sections.

v0.11.0 changelog and IPFS links here.

/ipns/latest.cactus.chat is updated to point to the latest release, so sites linking there should already be using the new version.

Try out a live demo: cactus.chat/demo

Join our Matrix room: #cactus:cactus.chat

πŸ”—Dept of SDKs and Frameworks 🧰

πŸ”—simplematrixbotlib (website)

simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio.

krazykirby99999 reports

Happy New Year!

Since the last post of TWIM, versions 2.5.1 and 2.6.0 of simplematrixbotlib were released.

πŸ”—Version 2.5.1 Released!

πŸ”—New Fixes:

  • Fixed #101 'Api' object has no attribute 'async_client'

πŸ”—Version 2.6.0 Released!

πŸ”—New Features:

  • A listener for handling m.reaction events has been added. Bot developers can now use Listener.on_reaction_event to smoothly handle reactions.

Example usage is shown below:

"""
Example Usage:

random_user
      !echo something

random_user2
      *reacts with πŸ‘οΈ

echo_reaction_bot
      Reaction: πŸ‘οΈ
"""

import simplematrixbotlib as botlib

creds = botlib.Creds("https://example.com", "echo_reaction_bot", "password")
bot = botlib.Bot(creds)


@bot.listener.on_reaction_event
async def echo_reaction(room, event, reaction):
    resp_message = f"Reaction: {reaction}"
    await bot.api.send_text_message(room.room_id, resp_message)


bot.run()

Request additional features here.

Source: https://github.com/KrazyKirby99999/simple-matrix-bot-lib

Package: https://pypi.org/project/simplematrixbotlib/

Documentation: https://simple-matrix-bot-lib.readthedocs.io/en/latest/

Matrix Room: https://matrix.to/#/#simplematrixbotlib:matrix.org

πŸ”—matrix-bot-sdk (website)

A TypeScript/JavaScript SDK for Matrix bots

TravisR announces

matrix-bot-sdk has had a v0.6.0-beta.3 release with beta support for crypto! It even includes documentation!

The crypto is considered beta quality at the moment: good enough to use for somewhat unimportant bots, but not fully recommended for production just yet. With that being said, I'm interested in bugs you run into - please use the issue tracker if you run into crypto not working.

Tutorials for the crypto setup are at https://turt2live.github.io/matrix-bot-sdk/tutorial-encryption.html

Note for appservice support to work then you'll need a Synapse with these PRs enabled (may require manual merge too):

  • https://github.com/matrix-org/synapse/pull/11538
  • https://github.com/matrix-org/synapse/pull/11617
  • https://github.com/matrix-org/synapse/pull/11215

For non-linux platforms, the rust-sdk will try to build itself which means you might need a working Rust stack. The Rust SDK repo itself has more information:

  • https://github.com/matrix-org/matrix-rust-sdk/blob/travis/node-bindings/crates/matrix-sdk-crypto-nodejs/README.md
  • https://github.com/matrix-org/matrix-rust-sdk/blob/travis/node-bindings/README.md

Bots and appservices don't automatically support encryption, but adding encryption should be easy. The Rust SDK dependency is required in either case, sorry.

πŸ”—Complement (website)

Matrix compliance test suite

Kegan reports

Complement has seem some updates this week in the test output that is produced onto the CLI. Details on how to add this to your CI process are contained in the README. Here's the difference when viewed using Github Actions:

Before:

After:

πŸ”—Polyjuice (website)

Elixir libraries related to the Matrix communications protocol.

uhoreg reports

Polyjuice Newt, the newest addition to the Polyjuice project, is an Elixir binding for vodozemac, the new Olm/Megolm implementation in Rust. At the time of writing, Polyjuice Newt supports encryption and decryption using Olm and Megolm, but by the time you read this next year, it may also support the SAS verification functions and pickling/unpickling.

πŸ”—Dept of Ops πŸ› 

πŸ”—NixOS Deployment (website)

Matrix packaging for NixOS

piegames announces

I don't think we've previously had any Nix/NixOS/nixpkgs related entries in TWIM, so I'll start ^^

The current unstable channel has extended its Matrix ecosystem support to also include Heisenbridge and Conduit packages and modules. This makes it super easy to deploy any of those services: For example, my configuration for Heisenbridge is 21 lines long, and Conduit is only 11 lines. You can browse the available configuration options online: services.matrix-conduit, services.heisenbridge (note that some of them are freeform and simply forward to the upstream configuration).

For those that are not into NixOS, a module is the code that turns the declarative configuration files into your running system setup. As an example, if you enable services.heisenbridge the following things are done for you:

  • Create a new heisenbridge user and group for the service
  • Create and manage the registration file for the homeserver (i.e. automatically regenerate it after the configuration changed)
  • Create a systemd service that runs the heisenbridge command with the requested bridge configuration. The unit also sets a few systemd security hardening options.

For support, join our Matrix space at #community:nixos.org and its Matrix-Nix channel: #matrix-nix:transformierende-gesellschaft.org

πŸ”—matrix-docker-ansible-deploy (website)

Matrix server setup using Ansible and Docker

Slavi says

Thanks to Aine of etke.cc, matrix-docker-ansible-deploy can now help you set up Honoroit - a helpdesk bot.

See our Setting up Honoroit documentation to get started.

Slavi says

Thanks to Aine of etke.cc, matrix-docker-ansible-deploy now supports Cinny - a new simple, elegant and secure Matrix client.

To try it out, see our Setting up Cinny documentation page.

Slavi announces

At matrix-docker-ansible-deploy, we believe that 2022 will be the year of the non-Synapse Matrix server!

Jip J. Dekker did the initial work of adding Dendrite support to the playbook back in January 2021. Lots of work (and time) later, Dendrite support is finally ready for testing.

The playbook was previously quite Synapse-centric, but can now accommodate multiple homeserver implementations, with Synapse still remaining the default.

To learn more, see our changelog entry about Dendrite.

Slavi announces

Thanks to Matthew Cengia and Shreyas Ajjarapu, matrix-docker-ansible-deploy can now bridge to Twitter using the mautrix-twitter bridge.

Note: the playbook already supports another Twitter bridge - mx-puppet-twitter.

To get started with this bridge, see Setting up Mautrix Twitter bridging in our documentation.

This brings the total number of bridges supported by the playbook up to 22. See all supported bridges here.

πŸ”—matrix-commit (website)

A Github Action for sending messages to a Matrix Room.

krazykirby99999 says

πŸ”—Version 1.2.2 Released!

A Github Action for sending messages to a Matrix Room. Changes should apply automatically if you are using tag v1.

πŸ”—New Features:

  • Link to repository added
  • Link to commit added

πŸ”—Before:

πŸ”—After:

πŸ”—Dept of Guides 🧭

Austin Huang says

A new Matrix guide has come into town: https://joinmatrix.org

In the hopes to expand Matrix's reach to the non-technical population, this guide is intended to give quick directions on how to use Matrix, as well as clear comparison between Matrix and other dominant platforms.

The pages are available on GitHub. Open to contributions!

πŸ”—Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

πŸ”—#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1envs.net599.5
2thomcat.rocks768
3matrix.spooks.cyou862
4matrix.nicfab.it955
5aria-net.org1300
6aletheia.moe1611.5
7trygve.me1617
8mtx.koyax.org1703.5
9rollyourown.xyz1808
10microelectro.de1996

πŸ”—#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1matrix.spooks.cyou366
2dendrite.matrix.org440
3sspaeth.de466
4rustybever.be546.5
5grin.hu966
6matrix.awesomesheep48.me1294
7dendrite.neilalexander.dev1537
8dendrite.beckmeyer.us1744

πŸ”—That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

The Mega Matrix Holiday Special 2021

2021-12-22 β€” General, Holiday Special β€” Matthew Hodgson
Last update: 2021-12-22 17:54

Hi all,

If you’re reading this - congratulations; you made it through another year :) Every winter we sit down and review Matrix’s progress over the last twelve months, and look forward to the next - for it’s all too easy to get lost in the day-to-day development and fail to realise how much the overall project is evolving, especially when it’s one as large and ambitious as Matrix!

Looking back at 2021, it’s unbelievable how much stuff has been going on in the core team (as you can tell by the length of this post - sorry!). There’s been a really interesting mix of activity too - between massive improvements to the core functionality and baseline features that Matrix provides, and also major breakthroughs on next generation work. But first, let’s check out what’s been happening in the wider ecosystem…

Continue reading…

This Week in Matrix 2021-12-17

2021-12-17 β€” This Week in Matrix β€” Thib

πŸ”—Matrix Live πŸŽ™

A supercharged episode this week with five super exciting demos!

πŸ”—Dept of Spec πŸ“œ

TravisR reports

Hey all, it's me again, not-anoa, with your spec update. I don't have a graph for you this week, but I do have curated content which hopefully holds you over until next week πŸ™‚

This week we've seen a few new MSCs get opened up:

As demonstrated, a few of them are follow-on work from aggregations. A lot of the work is an effort to get MSC2675 - Serverside aggregations of message relationships through FCP - it's changed quite a bit in the last week, so if you reviewed it before then please give it a quick read!

We're also gearing up for MSC1767 - Extensible event types & fallback in Matrix receiving attention early next year. This is currently being evaluated by features like MSC3245 - Voice messages, MSC3381 - Polls, and MSC3488 - Static location sharing. If all goes according to plan, the first pieces of Extensible Events will land soon.

Thib has caught me quite close to the deadline on TWIM, so while I don't have a random MSC for you this week I do recommend some light reading around aggregations and extensible events - these are both major features for Matrix and help shape the future for other, even more exciting, features.

Nico announces

πŸ”—Random MSC

Since the script didn't run, I manually generated random numbers until I actually got one, that matches an MSC and is not a draft!

So the random MSC is: MSC2461: Proposal for Authenticated Content Repository API

The goal of this MSC is to restrict who can access files in your content repository, which is used to share files, images, voice messages and more on Matrix. Currently this is secured by using random identifiers, but this MSC wants to add a few more restrictions: A signed-in user can still access everything, but if you don't provide an access token, when downloading media, you either can access no media, only media that was sent from that server, only media which the server already downloaded or everything. The level of access can be set by the server admin.

I think this would be really nice to have, but it has various challenges when implementing it in clients and when it needs to work across federation. It could limit how much random users can fill up your disk though, which is especially important for small server admins!

πŸ”—Dept of Servers 🏒

πŸ”—Synapse β†—

Synapse is the reference homeserver for Matrix

callahad reports

Happy holidays! This week we released Synapse 1.49.0, our last planned release of the year. This is the last release that supports Python 3.6 or PostgreSQL 9.6; if you've not upgraded, now is the time!

Most notably, this release includes stable support for MSC2918: Refresh tokens. This is a more secure alternative to long-lived access tokens, and we'd encourage clients to implement support for refresh tokens as described in the MSC.

We also released Sygnal 0.11, which includes loads of bugfixes. For the Element-managed Sygnal instances, this release has reduced our daily Sentry error rate by over 99%, dramatically improving the signal-to-noise ratio of our monitoring.

See you in January with Synapse 1.50! πŸŽ„

πŸ”—Sydent β†—

Sydent is the reference Matrix Identity server. It provides a lookup service, so that you can find a Matrix user via their email address or phone number (if they have chosen to share it).

dmr announces

I've just published the third and final post on Sydent's type annotations (part 1, part 2). This one is more reflective and tries to quantify our efforts: how well have we done?

Thanks for reading---I hope it's been useful.

πŸ”—Homeserver Deployment πŸ“₯️

πŸ”—Helm Chart β†—

Matrix Kubernetes applications packaged into helm charts

Ananace announces

And yet again more updates have been done on my Helm Charts - bringing element-web up to 1.9.7 and matrix-synapse up to 1.49.0

πŸ”—Dept of Clients πŸ“±

πŸ”—Nheko β†—

Desktop client for Matrix using Qt and C++17.

Nico reports

We are planning to have a small release next week, that fixes a few issues with the 0.9.0 release. It would be lovely if some of you could test one of our nightlies or could check if the translations for a language you speak are up to date in our weblate.

Some of the fixes this week include another crash fix for handling matrix links from your browser, notification bubbles that can show values over 9000(!), better preview images for sticker and emote packs created in Nheko, allowing you to click links in replies, a few layout and click area fixes. Nheko now also keeps track of your latest reactions and gives you easy access to them in the hover menu.

Nheko now also finally supports pinned messages! Most of you probably don't know, but that feature has pretty much always been in the matrix spec, but very few clients expose it. Today Nheko joins that rank! It's part of our goal to provide better support for building communities. Topics can be quite limiting, because they can only contain plain text. Pinned messages allow for much more creative freedom! They can also be encrypted, while state events currently never are, but the key for that isn't reshared, so currently experience in encrypted rooms is a tradeoff. Maybe we'll go for an encrypted description event in the future, but for now this seems to be a good solution to bridge the gap.

Let's hope the current master branch is good and we'll have a release with ALL THE FIXES next week! And thank you everyone, who already translated and reported issues! It took less than 10 hours to have 5 languages updated to 100%! Last year we didn't even have that many languages at 100%! You guys are AMAZING! <3

πŸ”—Element β†—

Everything related to Element but not strictly bound to a client

kittykat announces

Development has kicked off for location sharing! Watch this space for more news in the new year.

πŸ”—Threads

  • List of threads in a room is now more accurate and viewing very long threads has improved with the integration with the homeserver APIs. (These APIs are not enabled on matrix.org yet.)
  • Threads on mobile platforms are catching up to Web, with many changes in review.
  • Iterated on the design for restricted history threads and search results across all platforms.
  • Design for thread previews in room list has been improved for mobile platforms.
  • Improved Android bottom sheet expandable and scrollable behaviour design.
  • We’ve created MSC3567 to fix some edge cases with API calls

πŸ”—Polls

  • Only one Android bug is outstanding!
  • Gathering feedback to incorporate into the next phase of development, join us for the community testing session on Monday at 17:00 in #element-community-testing:matrix.org

πŸ”—VoIP

  • Continuing with polishing & bug fixing for full mesh calling app. One remaining bug somewhere causing members to not connect properly. Finalising how much of registration & login we can/want to implement for the short term until it’s replaced by OAuth login.

πŸ”—Community testing sessions

  • Tested all three Release Candidates (RCs) at the same time for the first time! Did not find any new web issues when testing first time user experiences and basic interactions on Web. We found 6 new issues on Android and 9 on iOS.
  • Tried out the Information Architecture changes on web with the Delight team. Very exciting to see these changes available in Labs already!
  • Closed 33 out of 60 re-tested encryption issues on Web and prioritised a few to be considered for upcoming work.
  • Next sessions, join #element-community-testing:matrix.org :
    • 17:00 UTC / 18:00 CET on Monday 20th December to test the new Polls feature
    • 16:30 UTC / 17:30 CET on Tuesday 21st December to squash more encryption bugs!

πŸ”—Element Web/Desktop β†—

Secure and independent communication, connected via Matrix. Come talk with us in #element-web:matrix.org!

kittykat announces

  • We’ve been working on auto-generating code and documentation for events raised by our client analytics (code here, PR for documentation generation here). This allows us to publish a comprehensive list of everything our analytics capture, which is great both for end users and for people doing analysis.
  • In labs (you can enable labs features in settings on develop.element.io or on Nightly)
    • First milestone reached on Information Architecture! To try it out, enable β€œThreaded messaging”, β€œUse new room breadcrumbs” and β€œNew spotlight search experience” in the Labs settings.
    • We’re actively collecting feedback on IA to review in the new year.
    • Starting work on Message Bubble defects

πŸ”—Element iOS β†—

Secure and independent communication for iOS, connected via Matrix. Come talk with us in #element-ios:matrix.org!

kittykat reports

  • Fixed an issue around some voice messages not playing in bridged rooms.
  • Polls changes are in this week’s release candidate (RC) and will be available behind a Labs flag in the next release.
  • Analytics changes have been merged into the RC and opt-in will be available soon.
  • In development:
    • Spaces is coming closer to completion: we’re there on space creation, adding rooms to spaces, space management and more. Release coming in the new year!
    • Work on implementing new login flow continues, with more improvements incoming.

πŸ”—Element Android β†—

Secure and independent communication for Android, connected via Matrix. Come talk with us in #element-android:matrix.org!

kittykat announces

  • Improvements to the timeline performance (faster display, faster scroll) after updates to the way we store timeline events.
  • Analytics framework has been merged, opt-in request will be shown to users once more translations have landed. For now, you can enable it in the settings.
  • Work on Message Bubbles has started!

πŸ”—Cinny β†—

Cinny is a Matrix client focused on simplicity, elegance and security

ajbura says

πŸ”—Cinny v1.6.0

πŸ”—Features

  • Room Timeline
    • Add pagination in room timeline
    • Replies link back to original message event
    • Use formatted_body to parse markdown
    • Support rich replies
    • Separation of read/unread messages in the room
    • Typing outside of an input box should focus the message field
    • Spoiler display support
    • User pill display support
    • Custom emoji display support
    • Performance improvements
  • Export E2E key for decrypting history in another client
  • Replaced go-to commands with Room search modal (Ctrl + k)
  • Remember people panel state
  • Twemojified all kind of text (except inputs)
  • Add option to hide membership and user events from timeline
  • Messages now span to full viewport width
  • Add animation on hover in sidebar/avatar

πŸ”—Bugs

  • Fix defer typing notifications until it can't be a command
  • Fix checkbox in register page
  • Fix app sending read receipt in background
  • Fix crash on creating room
  • Fix dark theme colors

πŸ”—Security update

Find more about Cinny at https://cinny.in/ Join our channel at: #cinny:matrix.org Github: https://github.com/ajbura/cinny Twitter: https://twitter.com/@cinnyapp

πŸ”—Dept of Non Clients πŸŽ›οΈ

πŸ”—Matrix Highlight β†—

A decentralized and federated way of annotating the web based on Matrix.

Daniel announces

I've been working on a matrix-based tool for highlighting and annotating websites. By building on top of matrix, we can effectively have a decentralized, federated and collaborative way to leave notes and highlights on pages. I wrote a brief introduction on my blog, as well as made a little bit of a simple demo video. Here's a copy-pasted list of planned and existing features:

  • Current: Create and send website annotations over Matrix.
  • Current: Store data in a decentralized and federated manner.
  • Current: Share highlights with other users, including those on other servers.
  • Current: Group annotations together and create multiple annotation groups
  • Planned: Use Matrix's End-to-End encryption to ensure the secure transmission and storage of highlight data.
  • Planned: Leverage the new m.thread MSC to allow users to comment on and discuss highlights.
  • Planned: Use something like ArchiveBox to cache the current version of a website and prevent annotations from breaking.
  • Planned Highlight PDFs in addition to web pages.

Come join #matrix-highlight:matrix.danilafe.com to receive updates about the project!

I haven't published the code just yet, but I'm going to as soon as the tool is in better shape.

Daniel announces

Matrix highlight for probably the last time this week. Highlight comments and self-editing are implemented, though I'm not sure I'll stick with this particular model.

πŸ”—Populus Viewer β†—

A Social Annotation Tool Powered by Matrix

gleachkr announces

I've been teaching a class this semester using a tool built on the matrix-js-sdk and tentatively entitled populus-viewer. Populus-viewer uses Matrix as a backend for the social annotation of PDFs, with the goal of helping matrix become a platform for teaching and scholarly collaboration. If you're interested in learning more, or adopting populus-viewer in your teaching, come visit #opentower:matrix.org!

Populus-Viewer currently supports:

  • Annotation of PDFs with highlights and pin-drops
  • Matrix conversations based on annotations
  • Audio and video messages
  • Replies, reactions, and redactions
  • Markdown for rich text
  • LaTeX for mathematical notation
  • Typing notifications
  • Synchronized reading position across devices
  • SSO, with single-click links for embedding in an LMS like Canvas or Blackboard.

As the project develops, I'm hoping to continue to polish the reading experience, and to add support for other mime types (audio and video especially).

πŸ”—matrix-streamchat β†—

Matrix powered stream overlay for OBS, to integrate live chat in your favorite (selfhosted) streaming setups.

f0x announces

TWIM I got started on the chat part of matrix-streamchat, to provide a lightweight embeddable Matrix client to be used alongside streams in Owncast and PeerTube. It will use guest access, and lots more features to come like extensive custom emote support. For now refactoring a bunch of things first before adding more flashy things, but who knows, you might see me do it live on https://stream.pixie.town

πŸ”—Dept of SDKs and Frameworks 🧰

πŸ”—vodozemac β†—

An implementation of Olm and Megolm in pure Rust.

Matthew says

Introducing vodozemac (https://github.com/matrix-org/vodozemac) - a rewrite of libolm in Rust by poljar and dkasak! The intention is for this to become the reference Olm implementation going forwards, and to get it audited asap (and benefit from all of Rust’s nice safety and parallelism features, and better crypto primitives!)

πŸ”—simplematrixbotlib β†—

simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio.

krazykirby99999 says

πŸ”—Version 2.5.0 Released!

simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio. Version 2.5.0 adds improvements to the config feature.

πŸ”—Feature Changes:

  • Add allow/block lists: This allows bot developers to specify allow/block lists of users who have permission to interact with the bot using regex.
  • Permissions can checked with Match.is_from_allowed_user(), which lets the bot developer choose which responses are restricted.
  • The allow/block lists can by modified at runtime via the Config.add_allowlist(), Config.remove_allowlist(), Config.add_blocklist(), and Config.remove_blocklist() methods.

A thank you to HarHarLinks for their contributions to version 2.5.0!

Request additional features here.

View source on Github View package on PyPi View docs on readthedocs.io https://matrix.to/#/#simplematrixbotlib:matrix.org

πŸ”—Dimension β†—

An open source integration manager for matrix clients, like Element.

TravisR announces

Dimension, an integration manager alternative for Element, has received a bunch of updates over the last couple weeks:

  • Added (early) support for matrix-hookshot's GitHub, Jira, and Webhooks bridging.
  • Most of a redesign complete to make it feel more like an Element UI rather than something special and third party.

If you're interested in helping out in getting the redesign finished, please check out https://github.com/turt2live/matrix-dimension/issues/458 which has reference mockups and linked issues. The major parts are the "complex bots" (Travis CI, RSS, etc) and the sticker integration. Unfortunately, I don't have enough free time to work on it myself in the near term, but will get back to it eventually πŸ™‚

And now, a complementary screenshot of the Goodβ„’ parts:

πŸ”—Dept of Ops πŸ› 

πŸ”—matrix-commit β†—

A Github Action for sending messages to a Matrix Room.

krazykirby99999 reports

πŸ”—Example Usage:

# .github/workflows/matrix-commit.yml
on:
  push:
    branches:
      - master

jobs:
  matrix_action_job:
    runs-on: ubuntu-latest
    name: Send Message to Matrix Room
    steps:

    - name: Checkout
      uses: actions/checkout@v2

    - name: matrix-commit
      uses: krazykirby99999/matrix-commit@v1

      with:
        homeserver: ${{ secrets.BOT_HOMESERVER }}
        username: ${{ secrets.BOT_USERNAME }}
        access_token: ${{ secrets.BOT_ACCESS_TOKEN }}

        room_id: ${{ secrets.ROOM_ID }}
        message: "#### New Commit:"

πŸ”—Notes:

πŸ”—Syntax:

  • The homeserver should be in the form of https://domain.tld
  • The username should be the username, not the user id. (krazykirby99999, not @krazykirby99999:matrix.org)
  • The room_id should be the internal room id of the room, not the published address. (!QQpfJfZvqxbCfeDgCj:matrix.org, not #thisweekinmatrix:matrix.org) This can be found under Room Options > Advanced > Room Information in the Element Client.

πŸ”—Other

  • If the room_id is not specified, the bot will send the message to all joined rooms.
  • If the message is not specified, it will default to Commit:.
  • The bot will join all invited rooms upon the start of an action.

Contributions are welcome - https://github.com/KrazyKirby99999/matrix-commit

Example Image

πŸ”—Dept of Bots πŸ€–

πŸ”—matrix-imposter-bot β†—

A Matrix appservice for relaying messages.

mr_johnson22 says

matrix-imposter-bot - A bot that uses your account to repeat other people's messages. This gives relay-bot capabilities to puppet-only bridges.

I made this project a while ago to hack in a relay mode to the mautrix Facebook bridge. But as of this week, that bridge supports relaying natively! πŸŽ‰ Thus, my main motivation for maintaining imposter-bot is obsolete, and the project will be on indefinite hiatus.

With that said, it can (mostly) still be used to add relay support to any bridges that don't yet support a relay mode themselves--but native relay support is always better!

Thanks to everyone who's shown interest in the project, and to tulir for making such great bridges!

πŸ”—Dept of Interesting Projects πŸ›°οΈ

πŸ”—ChatStat β†—

An R package To Gather Stats From Chat Platforms

Gwmngilfen announces

A project has started to re-implement the venerable mIRCstats for Matrix! It's in very early stages, right now it only does "getting a data-frame of events for a list of rooms" and has no actual visualisations baked in yet. However, we're moving quickly, and I hope to have some initial easy-to-use viz in place over the Christmas break.

The project is written in R (because I am an R user, and its good for data and viz work :P) and you can find it here. If you're new to R and want to give it a go, check out the extremely brief howto I just wrote here. I look forward to all the ways you will tell me it's broken!

πŸ”—Dept of Built on Matrix πŸ—οΈ

πŸ”—Saint Petersburg Widget (A Board Game build on Matrix)

Timo K. announces

Board games are great. And Matrix and its widget api turned out to be an excellent environment to create collaborative board games. With some really impressive conditions:

  • I don't have to maintain a server with a database.
  • I don't have to create a custom account systems user need to register. They play with their matrix account, which also makes accessibility great. Someone invites you in a room with the game and you can play!
  • I just need to host one static file and ppl will be able to play as long as that static site exists.

This project tries to be two things. A tech demo and inspiration to what is possible with widgets (Especially, with the changes on how widgets can be displayed in element (Check the "matrix live" Demos! 😊) ) Second it should serve as source and resource. For ideas and solutions on how no trust games can be executed without server (third party) side logic. And, for the ones interested, also as a resource on how widgets are implemented.

Last but not least the game Saint Petersburg is really fun. It takes a couple of minutes to grasp the rules but it is one of those games where there are so many things that can be considered with simple rules that it becomes more and more exciting with each round. So I really invite you to check out the rules and give it a try. Its best to start in the Git Repo or join the this room: #st-petersburg-auth:matrix.org

To put it simple, the widget works like this: The game state is stored in the room state and is updated through the widget directly. This of course raises questions: How is it still possible to prohibit users from cheating and manually changing parameters like, how much money they own. Everyone (who has the permission) is always able to send whatever state events they want? How is it possible to draw random cards if there is no third party involved. Could I not just send a state event with the cards that I hope are going to be drawn and are beneficial for me. Can we make card drawing deterministic? Not really since then everyone know what is going to happen. Which kind of breaks the game...

I would be super happy, if someone is interested and wants to find answer to the questions above by checking out the README.

πŸ”—Matrix in the News πŸ“°

kim says

There is an article (exclusive to paid subscribers) in the German tech news/magazine website heise.de about "Running your own messaging service using the matrix server" https://www.heise.de/ratgeber/Eigener-Chatserver-Mit-dem-Matrix-Server-einen-Messaging-Dienst-betreiben-6289020.html

I'm not a subscriber but sounds like they go over how to set up using the Ansible playbook #matrix-docker-ansible-deploy:devture.com

πŸ”—Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.

πŸ”—#ping:maunium.net

Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1envs.net451.5
2cri.epita.fr1019
3aria-net.org1446.5
4flueren.eu1576
5matrix.home.boris-wach.de3181
6trygve.me3519
7utzutzutz.net4038
8matx.myecloud.org4855.5
9rollyourown.xyz4989.5
10grimneko.de5127

πŸ”—#ping-no-synapse:maunium.net

Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1rustybever.be461
2conduit.supercable.onl524
3dendrite.supercable.onl539
4conduit.cyberdi.sk953.5
5matrix.awesomesheep48.me1434.5
6s2.toldi.eu1593
70x1a8510f2.space2125
8dendrite.beckmeyer.us8497

πŸ”—That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Type coverage for Sydent: evaluation

2021-12-17 β€” Tech β€” David Robertson

This is the third in a series of three posts which discuss recent work to improve type annotations in Sydent, the reference Matrix Identity server. Last time we discussed the mechanics of how we added type coverage. Now I want to reflect on how well we did. What information and guarantees did we gain from mypy? How could we track our progress and measure the effect of our work? And lastly, what other tools are out there apart from mypy?

πŸ”—The best parts of --strict

While the primary goal was to improve Sydent's coverage and robustness, to some extent this was an experiment too. How much could we get out of typing and static analysis, if we really invested in thorough annotations? Sydent is a small project that would make for a good testbed! I decided my goal would be to get Sydent passing mypy under --strict mode. This is a command line option which turns on a number of extra checks (though not everything); it feels similar to passing -Wall -Wextra -Werror to gcc. It's a little extreme, but Sydent is a small project and this would be a good chance to see how hard it would be. In my view, the most useful options implied by strict mode were as follows.

πŸ”—--check-untyped-defs

By default, mypy will only analyze the implementation of functions that are annotated. On the one hand, without annotations for inputs and the return type, it's going to be hard for mypy to thoroughly check the soundness of your function. On the other, it can still do good work with the type information it has from other sources. Mypy can

  • infer the type of literals, e.g. deducing x: str from x = "hello";
  • lookup the return types of standard library calls, via typeshed; and similarly
  • lookup the return types of any annotated functions in your code or dependencies.

The information is already available for free: we may as well try to use it to spot problems.

πŸ”—--disallow-untyped-defs and friends

This flag forces you to fully annotate every function. There are less extreme versions available, e.g. --disallow-incomplete-defs; but I think this is a good option to ensure full coverage of your module. It means you can rely on mypy's error output as a to-do list.

One downside to this: sometimes I felt like I was writing obvious boilerplate annotations, e.g.

    def __str__(self) -> str:
        ...

There was one particular example of this that crops up a lot. Mypy has a special exception for a class's __init__ and __init_subclass__ methods. If a return type annotation is missing, it will assume these functions return None instead of Any. (See here for its implementation.) This is normally compatible with --disallow-untyped-defs and --disallow-incomplete-defs, with one exception. If your __init__ function takes no arguments other than self, mypy won't consider it annotated, and you'll need to write -> None explicitly.

It's also worth mentioning --disallow-untyped-calls, which will cause an error if an annotated function calls an unannotated function. Again, it helps to ensure that mypy has a complete picture of the types in your function's implementation. It also helps to highlight dependenciesβ€”if you see errors from this, it might be more practical to annotate the functions and modules it's calling first.

πŸ”—--warn-return-any

If I've written a function and annotated it to return an int, mypy will rightly complain if its implementation actually goes on to return a str.

def foo() -> int:
    # error: Incompatible return value type (got "str", expected "int")
    return "hello"

If mypy isn't sure what type I'm returning though, i.e. if I'm returning an expression of type Any, then by default mypy will trust that we've done the right thing.

def i_promise_this_is_an_int():
    return "hello"

def bar() -> int:
    reveal_type(i_promise_this_is_an_int()) # Any
    return i_promise_this_is_an_int()

Enabling --warn-return-any will disable this behaviour; to make this error pass we'll have to prove to mypy that i_promise_this_is_an_int() really is an int. Sometimes that will be the case, and an extra annotation will provide the necessary proof. At other times (like in this example), investigation will prove that there really is a bug!

πŸ”—--strict-equality

This is a bit like a limited form of gcc's -Wtautological-compare. Mypy will report and reject equality tests between incompatible types. If mypy can spot that an equality is always False, there's a good chance of there being a bug in your program, or else an incorrect annotation.

I'm not sure how general this check is, since users can define their own types with their own rules for equality by overriding eq. Perhaps it only applies to built-in types?

πŸ”—Quantifying coverage

It was important to have some way to numerically evaluate our efforts to improve type coverage. It's a fairly abstract piece of work: there's nothing user-visible about it, unless we happen to discover a bug and fix it.

The most obvious metric is the number of total errors reported by mypy. Before the recent sprint, we had roughly 600 errors total.

dmr on titan in sydent on ξ‚  HEAD (3dde3ad) via 🐍 v3.9.7 (env)
2021-11-08 12:35:37 βœ”  $ mypy --strict sydent
Found 635 errors in 59 files (checked 78 source files)

This is a decent way to measure your progress when working on a particular module or package, but it's not perfect, because the errors aren't independent. Fixing one could fix another ten or reveal another twentyβ€”the numeric value can be erratic.

πŸ”—Reports

I found mypy's various reports to be a better approach here. There were three reports I found particularly useful.

πŸ”—--html-report

This produces a main index page showing the "imprecision" of each module. At the bottom of the table is a total imprecision value across the entire project.

HTML report, index page. A table showing each module's precision and number of lines of code.

The precision for each module is broken down line-by-line and colour-coded accordingly, which is useful for getting an intuition for what makes a line imprecise. More on that shortly.

HTML report, module page. Most lines of source code are highlighted green; a minority are highlighted red.

πŸ”—--txt-report

This reproduces the index page from the html report as a plain text file. It's slightly easier to parseβ€”that's how I got the data for the precision line graphs in part one of this series. That was a quick and dirty hack, though; a proper analysis of precision probably ought to read from the json or xml output formats. Here's a truncated sample:

+-----------------------------------+-------------------+----------+
| Module                            | Imprecision       | Lines    |
+-----------------------------------+-------------------+----------+
| sydent                            |   0.00% imprecise |    1 LOC |
| sydent.config                     |   0.00% imprecise |  266 LOC |
| sydent.config._base               |   0.00% imprecise |   31 LOC |
| sydent.config.crypto              |  15.94% imprecise |   69 LOC |
| ...                               |               ... |      ... |
| sydent.validators                 |   0.00% imprecise |   61 LOC |
| sydent.validators.common          |   7.35% imprecise |   68 LOC |
| sydent.validators.emailvalidator  |   1.30% imprecise |  154 LOC |
| sydent.validators.msisdnvalidator |   1.34% imprecise |  149 LOC |
+-----------------------------------+-------------------+----------+
| Total                             |   5.95% imprecise | 9707 LOC |
+-----------------------------------+-------------------+----------+

πŸ”—--any-exprs-report

Selecting this option generate two reports: any-exprs.txt and types-of-anys.txt. The latter is interesting to understand where the Anys come from, but the former is more useful for quantifying the progress of typing. Another sample:

                  Name   Anys   Exprs   Coverage
-------------------------------------------------
                sydent      0       2    100.00%
         sydent.config      0     185    100.00%
   sydent.config._base      0       3    100.00%
  sydent.config.crypto     34      80     57.50%
sydent.config.database      0       8    100.00%
   sydent.config.email      0      86    100.00%
                   ...    ...     ...        ...
-------------------------------------------------
                 Total    544   11366     95.21%

The breakdown in types-of-anys.txt has more gory detail. I found the "Unimported" column particularly interesting: it lets us see how exposed we are to a lack of typing in our dependencies.

                             Name   Unannotated   Explicit   Unimported   Omitted Generics   Error   Special Form   Implementation Artifact
-------------------------------------------------------------------------------------------------------------------------------------------
                           sydent             0          0            0                  0       0              0                         0
                    sydent.config             0          3            0                  0       0              0                         0
              sydent.config._base             0          0            0                  0       0              0                         0
                              ...           ...        ...          ...                ...     ...            ...                       ...
        sydent.util.versionstring             0         80            0                  0       0              0                         0
                sydent.validators             0          4            0                  0       0              0                         0
         sydent.validators.common             0         20            0                  0       0              0                         0
 sydent.validators.emailvalidator             0          8            0                  0       0              0                         0
sydent.validators.msisdnvalidator             0          8            0                  0       0              0                         0
-------------------------------------------------------------------------------------------------------------------------------------------
                            Total             9       1276          273                  0      37              0                        17

πŸ”—The meaning of precision

There are two metrics I chose to focus on:

  • the proportion of "imprecise" lines across the project; I also used the complement, precision = 100% - imprecision, and
  • the proportion of expressions whose type is not Any.

These are plotted in the graph at the top of this writeup. I could see that precision and the proportion of typed expressions were correlated, but I didn't understand how they differed. I couldn't see an explanation in the mypy docs, so I went digging into the mypy source code. My understanding is as follows.

  1. There are five kinds of precision. Full details are visible in the --lineprecision-report.
  2. Two kinds of precision, EMPTY and UNANALYZED convey no information, because there's nothing to analyze.
  3. A line is marked as precise, imprecise or any based on the expressions it uses.
    • An expression that has type Any leads to precision ANY.
    • I think an expression that involves Any but is not Any counts as imprecise. For instance, Dict[str, Any].
    • Remaining expressions have precision PRECISE.
  4. A line's precision is the worst of all its expressions' precisions.
    • ANY is worse than IMPRECISE, which is worse than PRECISE.

The "imprecision" number reported by mypy counts the number of lines classified as IMPRECISE or ANY.

On balance, my preferred metric is the line-level (im)precision percentage. There wasn't much difference between the two in my experience, but the colour-coded visualisation in the HTML report is a neat feature to have. Maybe in the future there could be a version of the HTML report that colour-codes each expression?

πŸ”—The larger typing ecosystem

There are plenty of articles out there about the typing. As well as the mypy blog itself, see Daniele Varrazzo's post on psycopg3 (2020), Dropbox's blog post (2019), Zulip's blog post (2016), Glyph's blog post on Protocols (2020) and a follow-up comparing them to zope.interface (2021) and Nylas's blog (2019). I'm sure there's plenty more out there.

It's worth highlighting the typeshed project, which maintains stubs for the standard library, plus popular third-party libraries. I submitted a PR to add a single type hintβ€”it was a very pleasant experience! Microsoft has an incubator of sorts for stubs too.

πŸ”—Other typecheckers

After the sprint to improve coverage, I spent a short amount of time trying the alternative type checkers out there. Mypy isn't the only typechecker out thereβ€”other companies have built and open-sourced their own tools, with different strengths, weaknesses and goals. This is by no means an authoritative, exhaustive surveyβ€”just my quick notes.

πŸ”—Pyre (Facebook)

  • I couldn't work out how to configure paths to resolve import errors; in the end, I wasn't able to process much of Sydent's source code.
  • Couldn't get it to process annotations like syd: "Sydent" where "Sydent" is an import guarded by TYPE_CHECKING.
  • No plugin system that I could see. That said, it has a separate mode/program for running "Taint Analysis" to spot security issues.
  • Seemed stricter by default compared to mypy: there was less inference of types.
  • Also has its own strict mode.

πŸ”—Pyright (Microsoft)

  • Didn't seem to recognise getLogger as being imported from logging. Not sure what happened thereβ€”maybe something wrong with its bundled version of typeshed?
  • In a few places, Sydent uses urllib.parse.quote but only imports urllib. We must be unintentionally relying on our dependencies to import urllib.parse somewhere! Mypy didn't complain about this; pyright did.
  • Seemed to give a better explanations of why complex types were incompatible. For example:
    /home/dmr/workspace/sydent/sydent/replication/pusher.py
   /home/dmr/workspace/sydent/sydent/replication/pusher.py:77:16 - error: Expression of type "DeferredList" cannot be assigned to return type "Deferred[List[Tuple[bool, None]]]"
     TypeVar "_DeferredResultT@Deferred" is contravariant
       TypeVar "_T@list" is invariant
         Tuple entry 2 is incorrect type
           Type "None" cannot be assigned to type "_DeferredResultT@_DeferredListResultItemT" (reportGeneralTypeIssues)
 /home/dmr/workspace/sydent/sydent/sms/openmarket.py
   /home/dmr/workspace/sydent/sydent/sms/openmarket.py:93:13 - error: Argument of type "dict[_KT@dict, list[bytes]]" cannot be assigned to parameter "rawHeaders" of type "Mapping[AnyStr@__init__, Sequence[AnyStr@__init__]] | None" in function "__init__"
     Type "dict[_KT@dict, list[bytes]]" cannot be assigned to type "Mapping[AnyStr@__init__, Sequence[AnyStr@__init__]] | None"
       TypeVar "_KT@Mapping" is invariant
         Type "_KT@dict" is incompatible with constrained type variable "AnyStr"
       Type cannot be assigned to type "None" (reportGeneralTypeIssues)
    This would have been really helpful when interpreting mypy's error reports; I'd love to see something like it in mypy. Here's another example where I tried running against a Synapse file.
    /home/dmr/workspace/synapse/synapse/storage/databases/main/cache.py
/home/dmr/workspace/synapse/synapse/storage/databases/main/cache.py:103:53 - error: Expression of type "list[tuple[Unknown, Tuple[Unknown, ...]]]" cannot be assigned to declared type "List[Tuple[int, _CacheData]]"
Β Β TypeVar "_T@list" is invariant
Β Β Β Β Tuple entry 2 is incorrect type
Β Β Β Β Β Β Tuple size mismatch; expected 3 but received indeterminate number (reportGeneralTypeIssues)
    This is really valuable information. It's worth considering Pyright as an option to get a second opinion!
  • It looks like Pyright's name for Any is Unknown. I think that does a better job of emphasising that Unknown won't be type checked. I'd certainly be more reluctant to type x: Unknown versus x: Any!
  • Pyright is the machinery behind Pylance, which drives VS Code's Python extension. That alone probably makes it worthy of more eyes.
  • Seemed like it was the best-placed alternative typechecker to challenge mypy (the de-facto standard).

πŸ”—Pytype (Google)

  • Google internal? Seems to be maintained by one person semiregularly by "syncing" from Google.
  • Apparently contains a script merge-pyi to annotate a source file given a stub file.
  • No support for TypedDict: as soon as it saw one in Sydent, it stopped all analysis.
  • No Python 3.10 support (according to the README anyway).
  • I think it might use a different kind of typing semantics; its typing FAQ speaks of "descriptive typing" and a more lenient approach.

πŸ”—PyCharm

PyCharm has its own means to typechecking code as you write it. It's definitely caught bugs before, and having the instant feedback as you type is really nice! I have seen it struggle with zope.interface and some uses of Generics though.

πŸ”—Runtime uses of annotations

When annotations were first introduced, they were a generic means to associate Python objects with parts of a program. (It was only later that the community agreed that we really want to use them to annotate types). These annotations are available at runtime in the __annotations__ attribute. There's also a helper function in typing which will help resolve forward references.

>>> from typing import get_type_hints
>>> def foo(x: int) -> str: pass
...
>>> get_type_hints(foo)
{'x': <class 'int'>, 'return': <class 'str'>}

Programs and libraries are free to use these annotations at runtime as they see fit. The most well-known examples are probably dataclasses, attrs with auto_attribs=True and Pydantic. I'd be interested to learn if anyone else is consuming annotations at runtime!

πŸ”—Summary

All in all, in a two-week sprint we were able to get Sydent's mypy coverage from a precision of 83% up to 94%. Our work would have spotted the bytes-versus-strings bug; we understand why the missing await wasn't detected. We fixed other small bugs too as part of the process. As well as fix bugs, I've hopefully made the source code clearer for future readers (but that one is hard to quantify).

There's room to spin out contributions upstream too. I submitted two PRs to twisted upstream; have started to work on annotations for pynacl in my spare time; and submitted a quick fix to typeshed.

Looking forward, I think we'd get a quick gain from ensuring that our smaller libraries (signedjson, canonicaljson) are annotated. We'll be sticking with mypy for nowβ€”the mypy-zope plugin is crucial given our reliance on twisted. We're also working to improve Sygnal and Synapseβ€”though not to the extreme standard of --strict across everything.

I'd say the biggest outstanding hole is our processing of JSON objects. There's too much Dict[str, Any] flying around. The ideal for me would be to define dataclass or attr.s class C, and be able to deserialise a JSON object to C, including automatic (deep) type checking. Pydantic sounds really close to what we want, but I'm told it will by default gladly interpret the json string "42" as the Python integer 42, which isn't what we'd like. More investigation needed there. There are other avenues to explore too, like jsonschema-typed, typedload or attrs-strict.

To end, I'd like to add a few personal thoughts. Having types available in the source code is definitely A Good Thing. But there is a part of me that wonders if it might have been worth writing our projects in a language which incorporates types from day one. There are always trade-offs, of course: runtime performance, build times, iteration speed, ease of onboarding new contributors, ease of deployment, availability of libraries, ability to shoot yourself in the foot... the list goes on.

On a more upbeat note, adding typing is a great way to get familiar with new source code. It involves a mixture of reading, cross-referencing, deduction, analysis, all across a wide variety of files. It'd be a lot easier to type as you write from the get-go, but typing after the fact is still a worthy use of time and effort.

Many thanks for reading! If you've got any corrections, comments or queries, I'm available on Matrix at @dmrobertson:matrix.org.

On Matrix and the log4j vulnerabilities

2021-12-15 β€” Security β€” Matrix Security Team

There is currently a lot of buzz and uncertainty around a number of vulnerabilities discovered in the log4j library in the Java ecosystem. These vulnerabilities are collectively known as "Log4Shell" and currently encompass CVE-2021-44228 and CVE-2021-45046.

First and foremost, there are to our knowledge no Matrix homeservers written in Java. Synapse, the canonical implementation developed by the Matrix Foundation and the implementation that is backing matrix.org, is written in Python and thus unaffected. P2P Matrix relies on Dendrite, our next-gen homeserver which is written in Go and is unaffected. Conduit, a community homeserver, is written in Rust and also unaffected. Supporting components like Sygnal and Sydent are written in Python and unaffected.

There are two components that are commonly used in the Matrix ecosystem that do rely on Java. These are Jitsi, specifically the Jitsi Videobridge for VoIP, and signald used by the Signal bridge. Both components pull in log4j as part of their (transitive) dependencies. We're not aware of other bridges that are dependent on Java-based components.

For both of these projects updates have been published that integrate log4j 2.15.0 covering the initial CVE and we're currently waiting for additional updates to be published that integrate log4j 2.16.0 to cover the second. In the meantime, we've put all mitigations we are aware of in place on our systems and we strongly recommend everyone do the same.

For what mitigations to put in place, we recommend following the recommendations provided by LunaSec. They also provide a lot of background information on the vulnerabilities and how to audit for them.

Please keep an eye out for releases from the Jitsi and signald projects and follow their upgrade instructions to update your own deployments as soon as possible.

Synapse 1.49.0 released

2021-12-14 β€” Releases β€” Brendan Abolivier

Synapse 1.49.0 is out now!

πŸ”—Platform deprecations

Synapse 1.49.0 is the last version of Synapse to officially support Python 3.6 and PostgreSQL 9.6. This follows our platform dependency deprecation policy.

As a consequence of this, Synapse 1.49.0 is the last version of Synapse to support Ubuntu 18.04 LTS (Bionic Beaver), as it ships with Python 3.6.

On the topic of supported Ubuntu releases, please note that Ubuntu 21.04 (Hirsute Hippo) reaches its own end of life on January 20, 2022. Past this date we will stop producing new packages for Ubuntu 21.04.

πŸ”—Improved documentation

Up until now, a lot of very useful information was stored on the Synapse repo's wiki, which wasn't well advertised nor well reviewed.

With this release, we have migrated most of this information to Synapse's documentation website, so all the information you need to set up, maintain and troubleshoot a Synapse instance lives at the same place. Included in these new pages are the server admin FAQ and a guide to Synapse's Grafana dashboard.

The media repository documentation has also been updated with a lot of details about how Synapse stores media files.

πŸ”—Refresh tokens

When a Matrix client needs to authenticate a request to a homeserver, it uses what is called an access token. Sometimes server administrators might not want a user's access token to live forever (e.g. for security reasons). To address this concern, MSC2918 introduces the concept of refresh tokens to Matrix.

Initial support for refresh tokens in Synapse was introduced in version 1.38.0. Synapse 1.49.0 finalises and stabilises this implementation, allowing any client that supports this feature to use it as it is currently described in the related MSC.

πŸ”—Everything else

This release introduces the last changes needed to Synapse for basic threading support. It also introduces support for MSC3030, which allows clients to jump to a specific date in a room's history (expect a sneak peek of this in the next episode of Matrix Live!).

Another interesting point is the addition of a couple of admin APIs for federation. More specifically, they allow you to visualise all of the other homeservers your Synapse instance has been interacting with, as well as how successful the last attempts at communicating with them have been.

Please see the Synapse release notes for a complete list of changes in this release.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including Dirk Klimpel, Maximilian Bosch and Tulir Asokan.

πŸ”—Till next year

This is the last release of Synapse of 2021! The Synapse team will take a break for the holidays, pushing the next release of Synapse (1.50.0) to January 11, 2022.

We'd like to thank everyone who has been using Synapse, contributing to it, and/or supporting us for the past year, and we hope to see you again in 2022! πŸŽ†

Disclosure: buffer overflow in libolm and matrix-js-sdk

2021-12-13 β€” Security β€” Matrix Security Team
Last update: 2021-12-13 16:11

Today we are releasing security updates to libolm, matrix-js-sdk, and several clients including Element Web / Desktop. Users are encouraged to upgrade as soon as possible. This resolves the pre-disclosure issued on December 3rd.

Fixed library versions are:

Client versions incorporating the fixes are:

These releases mitigate a buffer overflow in olm_session_describe, a libolm debugging function used by matrix-js-sdk in its end-to-end encryption (E2EE) implementation. If you rely on matrix-js-sdk for E2EE, you are affected. This vulnerability has been assigned CVE-2021-44538.

Clients which do not use matrix-js-sdk for E2EE, like FluffyChat or Element Android / iOS, are not affected.

This issue has been present since the introduction of the olm_session_describe function in October 2019 (commits: libolm, matrix-js-sdk).

We do not believe it is practical to successfully exploit this issue. However, upgrading remains important as the overflow can be triggered remotely.

Separately from the above vulnerability, we noticed during an internal audit that the libolm bindings in matrix-js-sdk were not zeroing out certain arrays containing entropy for cryptographic operations. This causes the entropy to remain resident in memory longer than necessary. As a defense-in-depth measure, this release of libolm now proactively overwrites those arrays when it is safe to do so.

Lastly, we are also taking this opportunity to update the version of Electron bundled with Element Desktop, pulling in the latest backported security fixes there.

The buffer overflow was found and reported by GitHub user @brevilo in the course of developing jOlm, a library of Java bindings to libolm; thank you. If you believe you've discovered a security vulnerability in Matrix or its implementations, please see our Security Disclosure Policy for how to get in touch.