we're starting a policy and regulation blog series over on the Foundation's blog. Over the next few months I'll be covering various pieces of legislation that are already in place, as well as incoming regulation, and what it all means for Matrix.
The new FluffyChat has several performance improvements, including a fix which should speed up the whole app if you have a lot of megolm sessions.
Also FluffyChat v1.21.0 includes a new search and gallery feature for chats. You can now search for specific messages or browse the shared photos in a conversation.
Also please note that the default network request timeout has been changed. Before FluffyChat ran into a timeout if the server needed more than 30 seconds to send the initial sync. This was probably too optimistic so the timeout has now been set to 30 minutes. This should allow much more users with very large accounts to log in. Feel free to share your feedback.
QR Code login works is completed we are just waiting for matrix server to fully support native OIDC login, before rolling out the feature
Message queue work is also almost completed, now messages will be automatically queued for resending when internet connection comes back, without the need of taking any manual action
We are also working on storing and restoring composer drafts for each room, so no need to worry about losing anything that has not been sent yet if you browse another room or close your app
Great news also for element call integration, we are implementing native call notifications (with ringing), and a call log for DMs. Also starting a call will also be displayed as an event in the timeline.
We have added a quick implementation of sharing, so you can now send text and media from other applications.
QR Code login works is completed we are just waiting for matrix server to fully support native OIDC login, before rolling out the feature
Message queue work has been started, messages will be automatically queued for resending when internet connection comes back, without the need of taking any manual action.
Great news also for element call integration, we are implementing native call notifications (with ringing). Also starting a call will also be displayed as an event in the timeline.
Way back in December I TWIM'd I was working on a new project called complement-crypto which aims to write an extenstive, exhaustive set of end-to-end E2EE tests for Matrix clients. The aim of this work is to ensure encryption in Matrix Rust SDK in particular is robust to all kinds of failure modes: from connectivity blips over federation, server restarts and corrupted data, to actively malicious homeservers. Work has progressed significantly in the intervening 6 months:
The test suite runs in both Rust SDK and JS SDK CI pipelines to detect regressions. This required a lot of work enabling conditional compilation (so you don't need to run JS code in rust SDK) and improving test reliability to a sufficient standard that it can be relied upon.
It now has full federation support, spinning up sliding sync proxies for both homeservers if needed.
It has a comprehensive RPC client process to test ungraceful shutdowns (e.g what happens if your client gets SIGKILL'd?)
When things do go wrong, you have powerful debugging tools thanks to using mitmproxy for all communications, in addition to client log files and container logs. The use of mitmproxy is not only useful for debugging, but is also used as a test synchronisation primitive to reduce flakiness (e.g do X when you see the HTTP response to Y), as an assertion tool (e.g ensure the client has uploaded one-time keys by looking for the request), and as an adversarial tool (e.g modify the response returned to the client).
it has allowed us to test the SDKs to reproduce tricky "unable to decrypt" (UTD) scenarios which we couldn't otherwise replicate. The NSE bug was reproduced via a complement-crypto test which semi-randomly performed high-level operations like restarting clients and sending messages in a multiprocess environment to try to trip up the SDK code.
it has allowed us to write regression tests for UTDs such as:
adding targets beyond rust/JS SDK (matrix-bot-sdk, mautrix, and others). A perfect client can still receive UTDs if the sending client didn't encrypt correctly, and the sender may be on a different codebase.
This has been just one part of our work reducing UTD errors. Another significant part has been aggressively analysing bug reports in the wild to identify common failure modes. To that end, please report any 'unable to decrypt' messages you see and include log files with your report. Please also try to get the sender to submit a bug report. I may reach out to you if I need additional information. These bug reports not only guide our priorities, but have also helped us identify obscure failure modes which we cannot reproduce via complement-crypto.
It's been decided to split the event rewrite up into multiple phases/parts. The original vision of the changes turned out to be far too optimistic, and we've run into some language-related challenges trying to implement them.
We'll still try to implement those changes, but this might depend on compile-time source generation or similar endeavours.
If this sounds like a fun challenge to you, or want to discuss and shape the semantics of these changes, feel free to join in!
Some policy changes, and some new projects have been started within the LibMatrix source tree:
New workflow: Use of feature branches
Keeping up to date with LibMatrix is hard, because the chance of updating to a partially finished commit is too high.
To address this, we're moving to a "main branch is stable" model, with partial implementations on deciated branches so developers can keep up and prepare for those changes.
New subproject: Writing of full documentation
We want to improve the developer experience by offering documentation of features right in your editor, without having to reference the backing source. This should enable users who are less familiar with Matrix' internal structures to still be able to implement their deepest wishes.
New subproject: Workspace and structural cleanup
The LibMatrix source tree became a mess as the project grew. We want to restructure it to be maintainable, and also far more friendly to those who might want to contribute. If this sounds like something you'd like to help with or would love to offer feedback on, please do feel free to reach out at #libmatrix:rory.gay!
Splitting up of subproject: Event handling rewrite
Due to the large amount of complexity and work, we've decided to first start rewriting the inner implementations of events.
This should massively improve the end user experience, as custom event content is no longer lost if edited, as well as no longer having to worry about desyncs between actual event content and what you might see in an editor!
Next to this, you will likely see performance improvements when dealing with accessing the contents of events!
Longer-term changes still include static typing of event collections, especially around single content type collections such as fetching a room's member list, which should also see a large performance uplift when fetching!
All dependencies have been updated. End users should not notice any differences as this is a maintenance task, but this does potentially enable some new usecases and squashes some bugs and security risks originating from outdated dependencies.
We have also expanded our team, in particular on the community growth side, and are about to open another position for a product designer ... if that's you or someone you know, stay tuned!
As of today, 9397 Matrix federateable servers have been discovered by matrixrooms.info, 2860 (30.4%) of them are publishing their rooms directory over federation.
The published directories contain 160776 rooms.
See you next week, and be sure to stop by #twim:matrix.org with your updates!
To learn more about how to prepare an entry for TWIM check out the TWIM guide.
The Foundation needs you
The Matrix.org Foundation is a non-profit and only relies
on donations to operate. Its core mission is to maintain
the Matrix Specification, but it does much more than that.
It maintains the matrix.org homeserver and hosts several
bridges for free. It fights for our collective rights to
digital privacy and dignity.
