For the past two years, FOSDEM couldn’t happen in-person. Fortunately we could
help, and Matrix hosted the world's largest free & open source software
conference online! This year we were finally back in-person… but not only.
The set-up we have arranged in 2021
and polished in 2022
has proved to be robust and served us well during the pandemic. Returning to an
in-person conference didn’t mean we had to throw it all away… quite the
opposite!
The Matrix Foundation is a candidate this year again to the GSoC programme. If you intend to mentor a student around your Matrix project, please ping me (@thib:ergaster.org) in the #gsoc:matrix.org room. Your project doesn't have to be set in stone yet: we need to have a good estimate of the number of mentors and projects applying before FOSDEM (by the end of next week).
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
This week and the week afterwards, the Spec Core Team are mostly focused on improvements to Matrix that we'd like to show off at FOSDEM 2023 this year! That consists of MSCs related to Faster Room Joins (MSC3943 among others) and Extensible Events (MSC1767).
This defines an algorithm and a data structure that can be used to order one's top-level list of Spaces and have that order sync across all of their clients. Rooms and spaces within a Space continue to have their order defined by an order key (and failing that, the origin_server_ts field) in the corresponding m.space.child event of their parent's Space's state.
I won't get into the details of the algorithm here (or its criticisms), but feel free to jump into the MSC and take a look!
Back in July I started a discussion on wikidata for adding a matrix space property, after much discussion in the wikidata community (lead mostly by tgr) we instead landed on a Matrix room property. This now enables slightly more accurate semantics when describing matrix rooms belonging to organizations, projects, and people on wikidata.
Wikidata is a knowledge base available under a free license and using standard machine-parsable data to add information to what is known as the "semantic web", this allows querying for information like for example: Organizations with matrix rooms
As the rest of wikimedia's projects it's open for contributions!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
As you can tell from the above MSC list, Extensible Events continues to charge forwards, with Travis working busily away at replicating all of the existing event functionality (plus new functionality - image albums anyone?) in a world containing Extensible Events. As always, take a look at the core MSC (MSC1767) for a background on what Extensible Events is, and why it's so exciting.
This week has also seen room version 10 become the default recommended room version in the spec! As a reminder, v10 brings the ability to have a room that's both knockable and restricted at once, as well as more strictly enforces the types of power level values.
Otherwise we've seen lots of movement in other areas of the spec. Expect to see some work done around push rules (which have historically been rather complicated and fiddly...) and notifications in the days to come.
I remember this MSC fondly. It was originally born out of MSC3489's need to allow any user in the room to send m.beacon_info state events. This can easily be achieved today by lowering the required power level of m.beacon_info to the default user level. However, you then run into the issue of any user being able to edit any other user's m.beacon_info event!
Thus this MSC attempts to modify the state events permission model so that users can "own" certain state events that they send. We already somewhat have this functionality - if you put your Matrix ID as the state ID for any state event, only you or users with a power level higher than you can edit it.
Sadly this little trick (which we use for m.room.member events) doesn't work in the case of live location sharing, as the feature demands the ability to share location from multiple devices at once. Hence, trying to send two m.beacon_info events with the same state key would overwrite each other.
This MSC attempts to expand the functionality by modifying the definition so that a user "owns" a state event if the state key starts with their Matrix ID. Problem solved... for the most part!
Do check out the MSC if you have some use cases in mind that would benefit from something like this.
Since the last few official Matrix holiday updates didn't mention as many of the cool community projects as I would have liked, I tried to work with the community to publish a community side review of 2022 as well as possibly some small teasers of what 2023 will bring. There are a lot of very varied updates, since everyone seems to have tackled the challenge differently, but I hope you you enjoy the result as much as I did: https://blog.neko.dev/posts/matrix-year-in-review-2022.html
A few days later we also published the same blog post on matrix.org, with a few typo fixes and cleanups: https://matrix.org/blog/2023/01/03/matrix-community-year-in-review-2022
This was a bit shot notice, so I would like to extend my gratitude to everyone who contributed and took some time in probably one of the busiest periods in a year! For the same reason, I hope you can excuse if one of your favourite projects is missing. If you have anything that is sorely missing, feel free to reach out in #year-in-2022:neko.dev and maybe I can amend the blog post.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
After a lull from the holiday period, work has continued on different parts of the spec. MSC3706 has merged, which furthers the spec side of the work to make joining rooms faster in Matrix (see MSC3902 for the overview).
MSC3938 has also been merged to the spec. The proposal removes a deprecated keyId field and cleans up the endpoint by disallowing trailing slashes.
Sliding Sync (MSC3575) is the next generation of sync - how Matrix clients receive new data from their homeserver. The spec side of the feature has been designed to be modular, with different extensions of spec provided different functionality. MSC3885 is one of those extensions, and defines how To-Device Messages (how different user devices talk directly to each other) would be requested by a Matrix client from the homeserver.
This proposal doesn't appear to have had too much review from the community yet - so feel free to check it out if faster Matrix clients appeal to you!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
Both new MSCs this week are from @uhoreg, and look pretty great from a spec polish perspective (I've been a fan with doing away of public device names for ages). Check them out!
Otherwise, MSC3898 (Native Matrix VoIP signalling) received some review and attention this week.
This MSC would allow users to actually delete account data they've set 🎉 Currently you cannot delete top-level keys from your account data, only remove their content.
Deleted account data items will come down /sync as having their content updated to {}, which clients that implement MSC3391 will take to mean that the account data item has been deleted. Clients that don't implement MSC3391 will store the content as {}, which commonly means an unused account data item anyhow.
Yours truly is currently writing an implementation for this in Synapse, and so far implementing the spec as written seems sound!
It's Friday which means it's time to talk about the newest Synapse release, v1.73.0! A few details:
Please note that this version removes legacy Prometheus metric names entirely. This also means that the enable_legacy_metrics configuration option has been removed; it will no longer be possible to re-enable the legacy metric names.
If you use metrics and have not yet updated your Grafana dashboard(s), Prometheus console(s) or alerting rule(s), please consider doing so when upgrading to this version.
In addition, a few notable features and bugfixes are:
Move MSC3030 /timestamp_to_event endpoints to stable v1 location (/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>, /_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>).
Reduce database load of Client-Server endpoints which return bundled aggregations.
Fix a long-standing bug where paginating from the start of a room did not work. Contributed by @gnunicorn.
Fix a bug introduced in Synapse 0.9 where Synapse would fail to fetch server keys whose IDs contain a forward slash
plus much more! You can read more about this release here: https://matrix.org/blog/posts.
So... some people are distracting me by asking me to play Valheim all the time since Mistlands was in public beta (it is released now). To make that easier to synchronize, especially if not everyone can be online all the time, I wrote a small bridge: https://nheko.im/nheko-reborn/valheimmatrix
Your own messages are
bridged from Valheim to Matrix, while other peoples messages are bridged
(privately) from Matrix to Valheim. This prevents loops and does nice puppeting,
but it means everyone needs to run their own bridge. It also means you can see
other peoples messages on Matrix, as long as they have the mod enabled, even if
you are not signed in yourself.
It is a very hacky experiment, but it works, so... why not show it off? Definitely check out the game, it is great!
We've also been plugging along on less visible changes during this time like bug-fixes for bridging DM's and fleshing out more of the bridge scenarios like leaving or deleting a room and bans to ensure things stay in sync. We also started bridging private conversations to Matrix behind the scenes back in February so we have a break-away point-in-time to import back from when we do the massive history import to backfill everything. Please note, that you still can't access these private rooms on Matrix but this will come later either in the form of inviting MXID's from the Gitter side or when we give access to the gitter.im homeserver itself to self-manage from that side.
Apart from playing Valheim, I at least also did some productive things! (Apart from everyone else who has been helping squashing bugs and jugendhackt of course.)
Nheko now supports MSC3664. Support for this is automatically enabled if it is enabled server side. This allows you to get notifications for events that relate to another event. For example you might want to get notifications for a reply to one of your messages. So you need to check if the replied to event was sent by you. Similar things can be done for relations like "notify me for reactions to my posts in rooms with less than 20 users".
By default only notifications for replies are enabled. You might not notice a difference, since by default you get notified for them because the fallback includes your username, but not all bridges implement that correctly and for me the reply fallback has been one of the biggest sources of bugs, so I'd like to get rid of it long term. (Also some users disable username mentions, because their name is too generic to be useful, in which case this can help as well since it actually checks the sender.)
Threads has made great progress and is nearly ready for release into the wild! There’s a few bugs remaining that we want to smash before we turn the feature on by default for all users.
Work on Element X is making great progress, we’re even looking at how to improve the iPad and desktop view for users on larger screens.
Element Android 1.5.11 has been released, it includes fixes for some crashes that had been reported via rageshake.
Threads has made great progress and is nearly ready for release into the wild! There’s a few bugs remaining that we want to smash before we turn the feature on by default for all users.
Please note the release cycle will be slightly different over the holidays!
Work has begun on Android X and fast progress is being made.
Currently we’re looking at performance on the room list and the timeline view, implementing a memory cache to handle things more efficiently.
We are skipping one release cycle over the festive holidays: all apps will receive a release on 20th December (with threads coming out of Beta!) and the following RC will come out on 10th January leading up to a release on 17th January!
Earlier this week, we have release the EFFEKTIO white paper, outlining what we are working on, why we think this matters, where we intend to head with it and what perspective we come from. We’ve published this 30-pager as a Google Doc with comments activated (if you have any feedback on it) as well as with a 2-page executive summary for the busy. You can also join us in #foyer:effektio.org if you’d like to discuss it with us.
For a couple of weeks now, we have a fresh nightly build available whenever changes have been merged within the 24 hours prior. Unfortunately the Android package, with mobile being the main target the UI has been developed for, can’t be installed as is (you need to clone and build it yourself for now), the Desktop App for Linux, Windows and Mac do give you an impression what we are after. Mind you that a) other than the chat all sections are still mainly UI mocked and b) it is still in heavy development: things break sometimes.
On the app itself, we have spent a lot of time on getting the chat right. The app’s core was upgraded to matrix-sdk 0.6+ to get the latest timeline API in, which required quite some refactoring. But allowed us to implement reactions, edits and replies. Encrypted Messages are still somewhat unreliable though. Prior to the nightly builds we’ve also taken some time to clean up the UI to give it a more polished look and make clear which things are simply not enabled yet. On the non-chat side of things, we’ve mostly worked on UI things (check out the ToDo-Sections and Pins), while we were waiting for the synapse fix on reading the timeline from the beginning, which is now released in 1.73. Happy to get back to implementing the state machines soon.
You want updates more frequently and close to when they actually happen? Join our #news:effektio.org or hang out with the devs in our #tech:effektio.org matrix room.
libolm 3.2.14 has been released. This is maintenance release. The main change was improvements in the TypeScript types. There were also some improvements in the Python packaging, which should mean that the pre-built Python package for Linux is compatible with more distributions. Unfortunately, there are some issues with the i686 and aarch64 Linux builds, meaning that those packages are not currently available for the current version. However, if the previous version works for you, there is no reason to upgrade. We are investigating the issue, and may provide new packages later.
I am building an Matrix SDK in Elm but I've never created or published a library before. This is a practice library to see what I can expect. You can install the library using
elm install noordstar/elm-matrix-webhooks
And you can send a webhook message using a function like this:
import Matrix.Webhooks as Hook
send : String -> Cmd msg
send = Hook.sendMessage toMsg webhook
-- send "Hi TWIM!"
-- send "I support **Markdown!**"
-- send "That's cool, isn't it?"
There is currently no room for this library. Just contact me @bram:noordstar.me for the library or visit #matrix-webhook:tetaneutral.net for questions about the Webhook API.
The latter came up when investigating, whether we could use sanakirja, the pijul database backend, as a multi-process crypto store backend for mobile. This investigation is still on-going.
Further than an investigation is the uniffi async support. After completing the Python backend, Swift is also passing all tests now. Additional Docs should clarify most questions. We are as excited about this as the Mozillians seem to be. Next up: Kotlin backend.
👉 Wanna hack on matrix rust? Go check out our help wanted tagged issues and join our matrix channel at Matrix Rust SDK.
I built a ChatGPT bot for Matrix! It does exactly what you think it does: you send a message, and ChatGPT will respond. It's very cool being able to talk to ChatGPT while on a mobile device.
It sends read receipts when messages are received by the bot, and the bot sends a typing indicator as ChatGPT is "thinking" 😀
It currently only works in unencrypted chats. I haven't managed to get encryption using matrix-bot-sdk working reliably yet. If it's meant to be working, tips or PRs are very welcome! I've really enjoyed tinkering withmatrix-bot-sdk on a few projects so far.
Keep in mind any text you send it will go to the closed source servers of OpenAI, and it is unlikely to stay free forever.
The Matrix.org Foundation has a stand at FOSDEM! Come in Brussels on February 4 and 5 to meet us in person, see the fun stuff we're doing, show us the nice things you are doing and grab cool merch.
We have also received proposals for a little more than a full day of devroom! We only have half a day of in-person devroom. We will (try to?) broadcast the in-person devroom in the Matrix room of course, and then the conference bot will schedule the talks we couldn't fit in the in-person slots.
Matrix Community Summit Berlin 2022 Podcast (German)
Another week, another episode, another community member interviewed.
Meet Robert, who spoke with Christian, about creating a CMS system on top of matrix.
episode link: https://spotifyanchor-web.app.link/e/p8aiO8ABCvb
rss feed: https://anchor.fm/s/cdb34188/podcast/rss
I hope you enjoy this week's interview and learn what other people in the community are up to.
Dept of Ping
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
This episode of Open Tech Will Save Us was a special on Moderation. Our guest was Jim from the Matrix.org Foundation's Trust & Safety Team.
Can moderation be automated? Can it be apolitical? How does it scale? Are decentralised systems inherently more insecure than centrlised ones? Let's find out!
You can now officially join the Matrix Foundation as an organisational or individual member in order to sustainably support core Matrix development, help steer the direction of the protocol and how best to fund it. In order to run the Governing Board and the overall work of the Foundation, we are also hiring an Executive Director.
A lot of this week was spent on implementation by the SCT, hence little movement in the bullet points above. However, outside of label changes, we've seen lots of progress on Extensible Events work from Travis as we continue to lay the groundwork for the IETF MIMI initiative.
We also continue to receive lots of fixes to the spec text itself via PRs to https://github.com/matrix-org/matrix-spec. Shout outs to zecakeh, dylhack, and uhoreg for their contributions!
This MSC aims to provide a solution to remotely toggling do-not-disturb across your various Matrix clients without needing to set it on each device manually. It does so by using account data which is synced to each client. Upon a client seeing data intended for them, they will silence notifications locally until told to do so otherwise (either remotely or by the user interacting with the client directly).
Quite a neat idea, and adds more powerful functionality to the "device manager" of a client. Also handy if your other clients are things like embedded systems!
As we barrel towards the end of the year, the Synapse team is hard at work
making Synapse faster, more stable and filled with features! This week we released v1.73.0rc2. Notable work
includes a significant speed-up to the /messages path. In addition you can find:
Improved DB performance by reducing amount of data that gets read in device_lists_changes_in_room.
Support for handling avatar in SSO login. Contributed by @ashfame
Unstable support for an Extensible Events room version (org.matrix.msc1767.10) via MSC1767, MSC3931, MSC3932,
and MSC3933.
A bugfix for a long-standing bug where the List media admin API would fail when processing an image with broken
thumbnail information
and a whole host of other bugfixes, features and improvements. Head over to https://github.com/matrix-org/synapse/releases
to check it out.
It's been a while since the last TWIM update, but the Slack bridge has been steadily improving! See https://github.com/matrix-org/matrix-appservice-slack/releases/ for a slew of updates.
Highlights include:
DMs are now consistently assigned with the name & avatar of the Slack user you're chatting with.
Display names of Slack users should be kept up-to-date more reliably.
Support is added for bridging Slack threads with MSC3440 m.thread relations.
The usability of bridge bot admin commands (like whoami and link) has been improved.
Some cases of inbound Slack messages being dropped have been fixed.
Hookshot 2.5.0 is now ready for your eyes and your processors!
This week we bring you the very last (all fingers being crossed) hookshot release not to support e2ee encryption. Our resident expert is busy working away on supporting it and is a couple of bugfixes away from providing it to you all (https://github.com/matrix-org/matrix-hookshot/pull/299#issuecomment-1327124799).
In the meantime, this release was so good we couldn't keep it from you any longer. This release is packed with quality of life improvements, and general bugfixes / stability. Deploy away!
The highlights are:
GitHub assign command now automatically chooses the top issue and/or authenticated user if not provided.
The RSS feed connection no longer feels the need to spam you when GitHub fails to respond within 10s. It now waits for 30s.
Hookshot now supports creating GitLab connections without automatically provisioning a webhook. When this happens, the bot will tell you to talk to your admin to setup the webhook manually.
SchildiChat is a fork of Element that focuses on UI changes such as message bubbles and a unified chat list for both direct messages and groups, which is a more familiar approach to users of other popular instant messengers.
Version 1.5.8.sc62 of SchildiChat-Android got released this week, which adds a few new interesting features on top of Element:
Reworked reply rendering, to follow the rich replies specification. This improves the following:
Update the replied-to message in case the original message gets edited or deleted
Fix replies edited on desktop not being shown as replies on Android
Fix replies edited on Android being shown on Desktop with the replied-to event twice
Fix some weird rendering issues, like lists in a reply being completely broken
Nicer design, using the original sender's color, and a maximum height for the replied-to event
Render the replied-to event also for media replies, for example when the mautrix-telegram bridge sends an image reply
In the future, this will also allow us to render replied-to images instead of just writing "Image" in there, but this one is still on the TODO-list since I didn't want to delay a release further.
We now allow sending all kinds of room-specific or account-specific global custom emotes as per MSC2545 (but you still need to configure them with a different client before sending).
For those about to ask if I'll upstream these changes to Element: I'm currently not planning to do so, but you are free to pick my changes yourself and create a PR. For rich replies, see also here.
Help us test Threads on Wednesday December 7th from 15:00 to 16:30 GMT! We need the most help testing on Android and iOS but if you can help test on Web/Desktop that's cool too! And don't worry if you can't make it, we'll have folks around most of the day so please come out and help us test whenever you can manage. See you in the community testing room at #element-community-testing:matrix.org!
📬 Useful feature time: The spaces button is now badged whenever you have unread messages or invites that aren’t visible in the current space.
📝 The Rich Text Editor has seen more bugs and edge cases fixed and support for links and lists is underway.
🏗️ On the ElementX side we have been working on a Reactions picker, the timeline rewrite is almost finished, work on a split view layout for iPad continues and we’re finishing up a shiny new Room Details screen.
Element Android 1.5.10 has been released for testers on the PlayStore (still in review by Google at this time). It should be pushed to production next week. It includes a new full screen mode for the Rich Text Editor and lots of other changes. The full changelog is here.
The team is making progress to integrate the Crypto Rust SDK in the main project and generate the application ElementR.
On Android Element X, which is the new Android app written with Jetpack Compose on top of the Matrix Rust SDK, it is now possible to edit events and to reply to events. Html is also now rendered in the timeline. We will define in the coming weeks what will be the plan for the future regarding this project.
Announcing Feline Matrix Assistant. This is a PowerShell 7 cross platform script that aims to gather a few somewhat common matrix tasks in one place with a somewhat user friendly interface.
It was designed to enable users on Windows to easily access things like custom powerlevel rooms and manual room upgrades that normally require using Linux to run bash scripts if one does not want to figure out how to make the web requests using their preferred method themselves.
One of the other reasons i made this is just because i wanted to get more familiar the CS API and because i needed it my self because im currently a Windows user.
The Third Room team have been invited to present Third Room at SIGGRAPH Asia 2022 next week! https://sa2022.siggraph.org/en/presentation/?id=realcur_102&sess=sess143
Dept of SDKs and Frameworks 🧰
Yet Another TypeScript Bot SDK
As named per the author, don't blame Thib for this name!
Why? As an alternative to matrix-bot-sdk, this library will be strongly typed with simplicity in mind to make sure there is no overhead to writing the perfect Matrix bot.
This library also focuses on having browser support and no dependencies around NodeJS (and least amount of dependencies in general)
AppService functionality will be shipped in a separate package as an extension of this base package.
There is plenty of dependency injection for: Logging, how HTTP comms are done, and caching (come ask about the Redis cache layer!)
Come join our Matrix room to help this library grow and give input!
Trixnity 3.0.0 is released. This release contains all changes of the beta versions. We additionally fixed two bugs regarding encrypted edited messages.
I'm currently working on vodozemac integration. For this I implemented uniffi-kotlin-multiplatform-bindings to bridge Rust and Kotlin Multiplatform.
The last few weeks of Sliding Sync in the various mobile clients is showing its fruits: Sliding Sync offline caching and session recovery was merged this week, giving you the latest state of your sliding sync in milliseconds without ever having touched the network now. You need to activate it via a setting on the SlidingSyncBuilder - FFI is exposed as well. This PR also adds recovery when the server rejected our request upon a failing position, indicating our session was reset. Previously, you needed to reconstruct the entire sliding-sync with all its state yourself, now this does it internally and transparently for the API user. Additionally, we merged a fix for invited rooms, which caused a panic on FFI usage, and have one other fix to the sliding-sync state post cache coming up.
Talking about fixes, the retry code path had a bug, which caused retries even when the API clearly stated we needed to change request, which is fixed now, and the memory store won't continue to keep all media cached anymore. More fixes are being pulled out of the demo branch and coming up as PRs next week.
We can also report amazing news from the Async-UniFFI-front: we have the first async-uniffi-swift code running now. Exciting times!
👉 ️ Wanna hack on matrix rust? Go check out our help wanted tagged issues and join our matrix channel at Matrix Rust SDK.
Introducing a new way to run Synapse on top of Kubernetes !
The current capabilities of the Synapse Operator:
Deploy Synapse: the operator can generate a default homeserver.yaml, or it can work with a custom homeserver.yaml, provided by the user.
Deploy the Heisenbridge and the mautrix-signal bridge: the operator automatically (re)configures Synapse to add the corresponding app_services.
Deploy a PostgreSQL database for Synapse (depends on the postgres-operator).
Once the operator is deployed, the Synapse resource becomes available. Deploying a basic instance goes as simple as:
$ cat << EOF > synapse.yaml
apiVersion: synapse.opdev.io/v1alpha1
kind: Synapse
metadata:
name: my-synapse
spec:
homeserver:
values:
serverName: example.com
reportStats: true
EOF
$ kubectl apply -f synapse.yaml
synapse.synapse.opdev.io/my-synapse created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
my-synapse-84d56d54df-c699k 1/1 Running 0 63s
The operator takes care of the deployment and lifecycle management of the Synapse instance. Similarly, the Heisenbridge and MautrixSignal resources can also be deployed alongside Synapse and are managed by the operator.
This operator is written in Go. To know more about the project, please visit https://github.com/opdev/synapse-operator, there are explanations on how to deploy the operator, and examples demonstrating the main features. Or ping me on Matrix if you want to have a chat about it ! I'd love to collect some feedbacks !
The clock is ticking! We're already in December, only a few days left to submit your talk proposal for our in-person Matrix devroom at FOSDEM. All the details to answer the call are in our FOSDEM 2023 CfP
Meet other matrix users, chat about Matrix, the rest, and everything else, discuss your Matrix ideas, sign each other in persona, and maybe spice the evening with a good mate or beer.
Every first Wednesday of the month in the c-base at 8pm ('til the next pandemic).
Matrix Community Summit Berlin 2022 Podcast (English episode)
Meet Charles (cvwright), Project creator and Lead engineer of Circles. He and I spoke about how he got to know about Matrix, how encryption became more common on the Internet and how he started Circles – an app, built with Matrix, to privately connect with family and friends.
I hope you enjoy this week's interview and learn what other people in the community are up to.
From next week on we're back to episodes in German – but another English episode is on its way!
Dept of Ping
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Interesting paper analysing the impact of decentralisation (and particularly Matrix) on legislation around content moderation: https://www.thecgo.org/research/the-decentralized-web-and-the-future-of-section-230/
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
Off the back of the release of Matrix v1.5 last week (see the blog post if you missed it), work on the next release has now begun.
Continued review on MSC3706 (extensions to /send_join for partial state) and MSC3852 (user agent information on /devices) by the SCT was seen. MSC3743 (standard error responses for unknown endpoints) was also merged to the unstable spec!
This MSC calls out the issue that today application services, especially those that bridge third-party networks, need to re-upload all bridged media to the homeserver in order for users to view it. This can create a large strain on disk resources for homeservers that bridge third-party networks and is a problem that is worth considering.
The approach the MSC takes to solving this is to allow application services to reserve an exclusive namespace for MXC URL's, such as mxc://example.com/_discord_.*. When a user's client attempted to request a media item that falls within the exclusive namespace, the homeserver would request the media from the application service (which in turn could pull it from the third-party network's CDN) rather than the homeserver's local media repo.
Seems like a cool idea! Check out the MSC and leave your thoughts if you're interested or have suggestions.
plus a batch of bugfixes. As ever, we aim to cut the next release candidate for Synapse 1.73 on the coming Tuesday. It should include the work that landed on develop this week. Of note:
The biggest release so far. Hmm... let me try again
THE Biggest Release (so far)!
The last few weeks we at etke.cc enhanced and optimized Postmoogle to make it as close to common email services as possible (it's not a full Gmail/Outlook/%You-name-it% replacement for a regular user yet, some features are missing, but we're pushing it towards that goal!)
The most notable changes:
automatic ban list (can be enabled by postmoogle admin) - no more nasty spammers will ever bother your host and if they will - Postmoogle will cut the wires drop TCP connections before even talking with them
automatic greylisting (can be configured by postmoogle admin) - good SMTP software must resend emails (and 99.99% of them do), RFC said.
internal email queue to re-send failed emails (with 45x errors) automatically - because Postmoogle is good SMTP software and it follows RFC
bridging email threads ⇿ matrix threads/reply-tos (for new emails) - that's a hard one. TL;DR: reply to email in matrix room (either by matrix thread or matrix reply-to) = email reply on email side, works both ways
multi-domain mode with the ability to select a domain for sending emails per room/per mailbox (all other domains will act as aliases)
!pm send and thread replies send multipart emails by default (both HTML from formatted markdown and plaintext in the same email)
lots of other enhancements and fixes under the moogle
You may have noticed it has been a bit quiet from our side lately. Partially that is because I was just busy fixing issues in our new features on the Synapse side (or implementing MSCs in Synapse). But we also just have been fixing boring old issues in our project, that have been not annoying enough to get attention. For example there was a warning when configuring the project with CMake, which now shouldn't happen anymore, but also is just really boring to talk about.
So maybe some more exciting stuff, in theory, if your distribution builds curl with the correct flags, Nheko now supports connecting to servers over HTTP/3. This in itself is not that useful, but it might improve how Nheko behaves when switching connections or when using mobile data.
Nheko now also shows a completer for /commands (stuff like /me, /react or /sovietflip).
You can also now invert the Enter setting, so that you can send using Shift+Enter (thanks to LordMZTE) and you can update your status message over D-Bus (thanks LorenDB), which allows you to show what song you are listening to or what project you are working on, if you enable the D-Bus API and implement that as an addon in your music player or IDE.
Also lots of boring cleanups going on behind the scenes still, so expect cool stuff to be slower for the next few weeks.
You might be aware that we, the NeoChat team, are currently working on implementing end-to-end encryption in NeoChat. If you're interested in that work, you should read the blog post i've written about it at https://tobiasfella.de/posts/neochat-e2ee/
Back after a bit of a hiatus working on call support (coming soon!) and non-hydrogen things which we're wrapping up now. Today and last Friday we've added sticky date headers to the timeline, which just got released in 0.3.5. Check it out if you haven't already at https://hydrogen.element.io
We’ve recently made improvements to the password recovery flow, meaning that you no longer choose your password before verifying your email… We now follow a more conventional approach (and have parity with mobile flows) that will make it easier for those struggling to sign in.
We’ve also continued to add tests and increase overall test coverage!
In labs (you can enable labs features in settings on develop.element.io or on Nightly):
Rich text editor: We’ve been working on lists and ‘ctr+backspace’ support
Threads: Improvements to performance and notifications continue but this week we’ve mostly put our efforts on testing as we’re prepping for the release of Threads out of Labs!
Trinity is an experimental bot framework written in Rust and using matrix-rust-sdk, as well as commands / modules compiled to WebAssembly, with convenient developer features like modules hot-reload.
we have experimental support for using Trinity as a library, thanks to @gnunicorn! So experimental that the public API may break at any point without warnings, and it hasn't even been published, but if you feel adventurous, take a look.
Flaki from Suborbital and Yours Truly have had a small and cozy chat about Trinity, where we talk about WebAssembly and Matrix, try to set up a new instance of the bot (!), find a bug (!!) but at the end it works (!!!). Youtube link if you're teased!
Matrix-Webhooks goal is to provide Rich, Embedded webhooks for Matrix as an alternative to Discord Webhooks, it's compatible with discord payload and API, so very easy to integrate into services like github, gitlab, grafana, uptime kumi etc ...
You can find the source code here : https://github.com/rednaks/matrix-webhooks
Or you can try it or use it here : https://matrix-webhooks.com/
You can also join our Matrix space where we share announcements, and have general discussions about matrix-webhooks inorder to improve it : https://matrix.to/#/#matrix-webhooks.com:matrix.org
It's been a busy week for the matrix-sdk team. As the week before, a lot of time was (well) spent on fixes and smaller improvements for mobile integration, during which a few bugs and problems where discovered and squashed. While this is ongoing, some of them already found their way back into mainline or at least have PRs pending for it: Like the bug in the read marker tracking which sometimes caused a crash 12, a problem with timeline construction being async or some minor fixes on the new xtask for swift. We've also noticed a significant regression in the automatic retry mechanism, which would just continue to spin for ever if the server gave a fully warranted error code back - if you experienced this, this PR should be solving it for you.
A significant milestone was achieved this week on Async UniFFI. After getting a first PoC ready the week before, Ivan packaged it all up and sent a draft PR upstream with async python code calling an async rust code over FFI and interpreting the rusty result. So cool! Swift and kotlin targets are next.
👉 Wanna hack on matrix rust? Go check out our help wanted tagged issues and join our matrix channel at Matrix Rust SDK.
As that blog post by Tobias mentions, releasing libQuotient 0.7 is the immediate next step - and in line with that, 0.7 RC has been published! Not much has changed since two betas; despite a very big piece of E2EE introduced in 0.7 release cycle, things have been mostly ironed out earlier in the year. Packagers are welcome to try their tools on the new release. Be mindful though that E2EE has just been introduced and wasn't widely tested; besides, a few pieces are still missing (read the blog post!); so while it's probably reasonable to switch E2EE on for packaging libQuotient, it's NOT recommended to do the same in client applications, unless accompanied by warnings and disclaimers all over the place. The release notes are still shallow and boring - the longer text will come with the final release.
matrix-docker-ansible-deploy has received lots of optimizations to cut down playbook runtime, sponsored by the etke.cc managed Matrix hosting service. The etke.cc Ansible playbook (which is an extension of matrix-docker-ansible-deploy) is growing to support more and more services (besides just Matrix), so the Matrix playbook being leaner prevents runtimes from becoming too slow and improves the customer experience.
The playbook now runs about 2x-5x faster, thanks to:
various optimizations (using include_tasks instead of import_tasks in all Ansible roles)
the introduction of additional install-only tags (--tags=install-all or --tags=install-COMPONENT) which skip slow and often unnecessary to run uninstall tasks
Hi everyone, we have released Mjolnir v1.6.1 (via v1.6.0)
Which includes:
Always echo policy list changes. List changes are now always enabled, whereas before they where only shown with config.verboseLogging. Mjolnir now no longer hides changes made by the same mjolnir account, providing the user with feedback for changes they have made to policy lists after using the ban/unban commands.
Policy lists created by mjolnir will now be done so with support for MSC3784.
Mjolnir now supports specifying the config file with the argument --mjolnir-config. It is highly recommended that you do this as opposed to relying on the environment variable NODE_ENV. The documentation for running with docker and from source have been updated accordingly.
Fix the WordList protection which was matching every message.
Rework the banning and unbanning of entities in PolicyLists.
Unbanning no longer searches for rules by state_key but by entity.
Users with longer mxids can now be banned.
Mjolnir no longer waits for the next client sync before applying server ACL and member bans after using the ban command
Improve the clarity of the unban command. Users marked as banned by the Flooding and FirstMessageIsMedia protections can now be unbanned with the ban command. Using the unban command without true will now warn the user that room level bans will not be overridden.
Replaced acceptInvitesFromGroup with acceptInvitesFromSpace.
Fix config.protectAllJoinedRooms leaking into explicitly protected rooms and mjolnir trying to protect watched lists in some circumstances. You would only have been impacted by this if you ran latest.
Dept of Events and Talks 🗣️
Matrix Community Summit Berlin 2022 Podcast (German)
The Community Summit 2022 in Berlin was a lot of fun and we've gotten to know a lot of community members and their projects. Some of these people we were able to interview and will release the recordings as a weekly podcast. From now on, each Friday an episode will be released for a total of 8 episodes.
Today (18th November), we're starting with Alex and Valentin. We talked about the summit, what went well and what activities to add next time. Furthermore, the two shared their passion for home server performance. Give it a listen!
A part of our chaotic decentralized intergalactic community will be a meeting in the rooms of "ETI Schauspielschule Berlin" at Hacking in Parallel//// this year.
There are several units and indiduals who will be there spread (into) the matrix ...
You can enter proposals until 2022-11-27 23:42 (Europe/Berlin). https://pretalx.c3voc.de/hip-berlin-2022/cfp
The Matrix space for the event is a good general place to get in touch.
https://matrix.to/#/#hip-space:1312.media
We also prepare for having a Matrix Operation C~~enter~~haos at the Chaos Communication Camp in Summer. This will take place from August 15 to 19, 2023 at Ziegeleipark Mildenberg near Berlin, Germany.
If you want to join forces of the and need a ticket voucher for the HIP. Drop in at:
https://matrix.to/#/#matrix-moc:datanauten.de
If you can / will not make it with you physical body to Berlin. No worries. You should start connecting the dots.
The XRevent.Labs Operation Group invites you to xrelog22. Discussions, lectures, music, art, performance and the joint creation of independent multiverses will take place from December 28 to 30, 2022. We not only want to raise socio-political, ethical and creative questions, but also actively develop own spaces and content for an interactive, cross-reality live experience.
Anyone who is interested in joining or visiting us can do so either with pre-registration for the real stage at FTZ - Digital Reality at HAW in Hamburg or virtually in the digital-twin at XRevent platform, or the Intergalactic-Chaos-Communication-Broadcast-Studio, aka icc|bs. Your contributions can be pre-recorded, streamed live via a conference tool or presented in the real space on site. Either way, get in touch and we’ll find a solution. Please test yourself and wear a mask if you visit us in person.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
Matrix v1.5 was released on November 17th, 2022! 🎉
While not as feature-packed as previous releases (v1.5 was intended as a maintenance release), it's still worth checking out! Included are two new MSCs that have landed, as well as loads of little bugfixes and clarifications to the spec text itself. Thanks to all who helped fix issues since v1.4!
As may be apparent from the new MSCs this week, the SCT has been looking at extensible events to iron out some of the finer details of the system. They're not necessarily in the best position to review at the moment, but keep an eye on this space for when review would be most welcome 🙂
This MSC attempts to define a method for clients to indicate to their homeserver that they would like to have animated version if possible when requesting a thumbnail of some media. This would allow clients to provide a toggle to the user for whether media in the timeline should be animated, or whether room and user avatars should be animated, etc. Potentially separate options for each!
Today, there is no way for the client to indicate per-request whether they would like to receive an animated version of a thumbnail. This leaves the choice up to the homeserver, which gives individual users much less control.
The MSC also defines mimetypes that animated thumbnails should return. And it has an implementation in matrix-media-repo!
Check it out if animated avatars/thumbnails peaks your interest.
Another week has gone by and we here at Synapse Inc have been working hard on
faster room joins, bugfixes, and other fun items to make Synapse faster and
better, in addition to releasing v1.72.0rc1. Just a few highlights of the release are:
Adding experimental support for MSC3912: Relation-based redactions.
Adding an Admin API endpoint for user lookup based on third-party ID (3PID).
Fixing a background database update, introduced in Synapse 1.64.0, which could cause poor database performance.
Improving the performance of /context in large rooms.
In addition, please note that we now only support PostgreSQL 11+, as PostgreSQL 10 has reached end-of-life.
As always, if you'd like to know more about this release, hop on over to the release page (https://github.com/matrix-org/synapse/releases) and take a look!
FluffyChat 1.7.0 has been released. It features a new way to work with spaces via a bottom navigation bar. A lot of work has also been done under the hood to make the app faster and more stable. The main color has slightly changed and the design got some finetuning.
Please be aware that it can take some time until it receives all app stores. The Snap Store currently waits for a review, while the arm64 version of Flatpak has recently failed to build.
Read the full changelog here: https://gitlab.com/famedly/fluffychat/-/blob/main/CHANGELOG.md
Ement.el, a Matrix client for the GNU Emacs text editor and Lisp environment, has been updated to v0.5. Since the last version announced in TWIM, these improvements have been made:
Additions
Present "joined-and-left" and "rejoined-and-left" membership event pairs as such.
Process and show rooms' canonical alias events.
Changes
The taxy.el-based room list, with programmable, smart grouping, is now the default ement-room-list. (The old, tabulated-list-mode-based room list is available as ement-tabulated-room-list.)
When selecting a room to view with completion, don't offer spaces.
When selecting a room with completion, empty aliases and topics are omitted instead of being displayed as nil.
This week has been exciting for ElementX (our brand new Element iOS client based on the Matrix Rust SDK) which gained a brand new navigation system and a new user notification system, support for displaying file messages in the timeline and playing back video ones, as well as sending states for new messages and an improved bubble layout.
We have also started working on an iPad specific layout, improved our logging components, improved hardware keyboard handling and fixed a few annoying issues
In Element land we have a made a handful of bug fixes which will hit the store next week and work has been continuing on threads, the new rich text editor, the device manager and our new crypto stack
Element Android 1.5.8 RC is available on the PlayStore for testers.
The team is working on the device manager, the Rich Text Editor (to add more edition options: links, quote, etc.), and the voice broadcast.
On the Element Android X side, we are also making progress. We can now change the server (the server has to support sliding sync though). The room list is displayed and the timeline with the Rich Text editor too. We are now working to add more features such as supporting more Event types in the timeline, etc. And the app got a new monochrome icon!
Since our last update, Populus viewer has added two features, both a bit overdue:
Populus viewer now supports .well-known discovery, so you just need to know your matrix server name, rather than the full subdomain in order to connect.
Populus viewer now supports the "restricted" join rule for annotations, so that annotation membership can be restricted to members of the underlying resource space. This will likely be the default for newly created annotations in the future.
Our server (at populus.open-tower.com) has also been rebuilt and reconfigured a bit, and will likely be converted from a test-bed to a stable home for annotation-based discussions in the coming weeks. As always, if you want to give us feedback, keep track of progress, or talk about the future of social annotation on Matrix, join us at #opentower:matrix.org!
As many of you know, we are working on using MLS, an upcoming IETF standard for end-to-end encryption, in Matrix. We've set up a page, Are We MLS Yet to track the progress of this project.
Elm is a functional programming language that compiles to JavaScript with the promise that their generated JS code will NEVER raise any runtime errors and show high performance.
Despite the fact that certain implementations like Cactus Chat already use Elm, I could not find an SDK for it. That's why I've started to write one: https://git.noordstar.me/bram/Elm-Matrix
The SDK is currently in early development. The /v3/sync endpoint is functional and there are already a few other ways to interact with the API, so the repository can already be tested by developers who are familiar with Elm. If you aren't that familiar with the language, it is better to wait until I release the first complete version.
Trinity is an experimental bot framework written in Rust and using matrix-rust-sdk, as well as
commands / modules compiled to WebAssembly, with convenient developer features like modules
hot-reload.
Two weeks of updates, because I've been slightly late last week:
Trinity now uses WebAssembly components as created with
cargo-components and
wit-bindgen under the hood for the Matrix
modules. This is quite experimental and early stage, but this allows multiple exciting things in
some not-so-distant future: switch API implementations at runtime, more security by isolating
specific APIs, using source languages for the modules that can be compiled with wit-bindgen, etc.
In particular, we've made a proof-of-concept for a bot module written in C (!) and compiled to
WebAssembly, and we've been very close to having a bot module written in Java using the excellent
TeaVM project.
New APIs have been added for the modules:
a simple key-value store, that is backed by the redb pure
Rust project, which comes with great performance.
modules can now specific help messages as well as admin commands (that will be available only
to a predefined admin user).
Which then allowed to write a shiny new Mastodon module: configure it in a room with a set of
credentials, a Mastodon URL instance, define which users are allowed to use it, and then post
statuses from the room using !toot message.
If you're interested to follow the project, feel free to join our Matrix
room or get the code on
Github.
Major breakthroughs have happened behind the scenes this week: we were able to get a simple python3-asyncio to poll our rust-side future and receive its result via uniffi. While there isn't much to show for the outside world, this is a major step towards having async-uniffi. The team is super excited about this development.
A few weeks ago I started experimenting with extending my moderation tools with a basic ML model that can give me early warnings on common spam.
As a result https://github.com/MTRNord/matrix-spam-ml was made. It contains a basic ML model using tensorflow based on telegram spam from matrix and a standard SMS dataset. It provides a HTTP api which can also be found at https://spam.midnightthoughts.space and a Synapse and mjolnir integration for both sampling training data and using it as automated spam reactions and is planned for the future.
Feel free to hop by in #matrix-spam-ml:midnightthoughts.space for any further questions :) also please note that the repo is using LFS and as it contains trained models it is pretty huge when cloning with LFS data.
axon.sh version 0.22.0 has been released, adding support for retrieving event reports from your homeserver, and looking up local users by third-party ID (requires Synapse 1.72.0rc1 or later).
Please try it out and report your experience in #axon:matrix.thisisjoes.site!
This year, the Matrix.org Foundation is excited to host the first ever Matrix.org Foundation and Community devroom in person at FOSDEM. Half a day of talks, demos and workshops around Matrix itself and projects built on top of Matrix.
We have a physical devroom on Sunday 5 morning, and will follow with details for the online devroom as soon as we get them! You can read the CfP for the physical devroom here: https://matrix.org/blog/2022/11/16/call-for-participation-for-the-fosdem-2023-matrix-devroom
Last week's TWIM presented a GBA emulator that was written and implemented in JavaScript. Due to some obscure Rust dependencies that weren't compatible with my server setup, I decided to rewrite the program in Python.
The Python implementation can be found at bram/pyboy-to-matrix. It only supports GameBoy games (no GBA), but it only needs one process (or Docker container) to run both the emulator and the Matrix bot.
If you're unfamiliar with last week's post, the GameBoy emulator is a program that emulates games and then interacts with you on Matrix. You can send commands and the emulator will show you what happened in the game.
This year, the Matrix.org Foundation is excited to host the first ever
Matrix.org Foundation and Community devroom in person at FOSDEM. Half a day of
talks, demos and workshops around Matrix itself and projects built on top of
Matrix.
We encourage people working on the Matrix protocol or building on it in an open
source project to submit a proposal! Note that companies are welcome to talk
about the Matrix details of their open source projects, but marketing talks are
not welcome.
This call for participation is only about the physical devroom. A separate CfP
will be issued for the online devroom once there are more details about it.
Key dates are:
Conference dates 4-5 February, 2023 In person
Matrix Devroom date: Sunday 5 morning in person, online devroom to be
announced
Submission deadline: Monday 5th December
Announcement of selected talks: Thursday 15th December
You must be available in person to present your talk for the physical devroom.
Talk Details
The talks can follow one of thee two formats for the physical devroom
20 min talk + 10 min Q&A, for topics that can be covered briefly
50 min talk + 10 min Q&A for more complex subjects which need more focus
We strongly encourage you to prepare a demo when it makes sense, so people can
actually see what your work looks like in practice!
Of course, the proposal must respect the FOSDEM terms as well:
The conference language is English. All content must relate to Free and Open
Source Software. By participating in the event you agree to the publication of
your recordings, slides and other content provided under the same licence as
all FOSDEM content (CC-BY).
We expect to receive more requests than we have slots available. The devroom
organisers will be reviewing the proposals and accepting them based on the
potential positive impact the project has on Matrix (as defined in by the
Mission section of https://matrix.org/foundation).
If a project proposal has been turned down, it doesn't mean we don't believe it
has good potential. Maintainers are invited to join the #twim:matrix.org
Matrix room to give it some visibility.
