🔗Matrix 1.0

Hi all,

We are very excited to announce the first fully stable release of the Matrix protocol and specification across all APIs - as well as the Synapse 1.0 reference implementation which implements the full Matrix 1.0 API surface.

This means that after just over 5 years since the initial work on Matrix began, we are proud to have finally exited beta!! This is the conclusion of the work which we announced at FOSDEM 2019 when we cut the first stable release of the Server-Server API and began the Synapse 0.99 release series in anticipation of releasing a 1.0.

Now, before you get too excited, it’s critical to understand that Matrix 1.0 is all about providing a stable, self-consistent, self-contained and secure version of the standard which anyone should be able to use to independently implement production-grade Matrix clients, servers, bots and bridges etc. It does not mean that all planned or possible features in Matrix are now specified and implemented, but that the most important core of the protocol is a well-defined stable platform for everyone to build on.

On the Synapse side, our focus has been exclusively on ensuring that Synapse correctly implements Matrix 1.0, to provide a stable and secure basis for participating in Matrix without risk of room corruption or other nastinesses. However, we have deliberately not focused on performance or features in the 1.0 release - so I’m afraid that synapse’s RAM footprint will not have got significantly better, and your favourite long-awaited features (automatically defragmenting rooms with lots of forward extremities, configurable message retention, admin management web-interface etc) have not yet landed. In other words, this is the opposite of the Riot 1.0 release (where the entire app was redesigned and radically improved its performance and UX) - instead, we have adopted the mantra to make it work, make it work right, and then (finally) make it fast. You can read the full release notes here. It’s also worth looking at the full changelog through the Synapse 0.99 release series to see the massive amount of polishing that’s been going on here.

All this means that the main headline features which land in Matrix 1.0 are vitally important but relatively dry:

• Using X.509 certificates to trust servers rather than perspective notaries, to simplify and improve server-side trust. This is a breaking change across Matrix, and we’ve given the community several months now to ensure their homeservers run a valid TLS certificate. See MSC1711 for full details, and the 2 week warning we gave. As of ~9am UTC today, the matrix.org homeserver is running Synapse 1.0 and enforcing valid TLS certificates - the transition has begun (and so far we haven’t spotted any major breakage :). Thank you to everyone who got ready in advance!
• Using .well-known URIs to discover servers, in case you can’t get a valid TLS certificate for your server’s domain.
• Switching to room version 4 by default for creating new rooms. This fixes the most important defects that the core room algorithm has historically encountered, particularly:
  • The new State Resolution algorithm to fix the Hotel California bug and many others: State Resolution Reloaded
  • Collision resistant event IDs
• Specifying the ability to upgrade between room versions
• Full specification of lazy loading room members
• Short Authentication String (Emoji!) interactive verification of E2EE devices
• ...and lots and lots and lots of bugfixes and spec omission fixes.

That said, there is a lot of really exciting stuff in flight right now which sadly didn’t stabilise in time for Matrix 1.0, but will be landing as fast as we can finalise it now that 1.0 is at last out the door. This includes:

• Editable messages! (These are in Synapse 1.0 and Riot already, but still stabilising so not enabled by default)
• Reactions! (Similarly these are in develop)
• Threading!! (We’ve planted the seeds for this in the new ‘aggregations’ support which powers edits & reactions - but full thread support is still a bit further out).
• Cross-signed verification for end-to-end encryption (This is on a branch, but due to land any day now). We’ve also held off merging E2E backups into the Matrix 1.0 spec until cross-signing lands, given it may change the backup behaviour a bit. Once this is done, we can seriously talk about turning on E2E by default everywhere.
• Live-tracking of room statistics and state in Synapse! (This is in Synapse 1.0 already if you check out the new room_stats and room_state tables, but we need to provide a nice admin interface for it).
• Support for smaller footprint homeservers by reducing memory usage and stopping them from joining overly complex rooms.

Then stuff which we haven’t yet started, but is now unlocked by the 1.0 release:

• Fixing extremities build-up (and so massively improving performance)
• Rewriting Communities. Groups/Communities deliberately didn’t land in Matrix 1.0 as the current implementation has issues we want to fix first. MSC1772 has the details.
• Rewritten room directory using the new room stats/state tables to be super-speedy.
• Super-speedy incremental state resolution
• Removing MXIDs from events (MSC1228)

Just to give a quick taster of the shape of things to come, here’s RiotX/Android, the all-new Riot client for Android, showing off Edits & Reactions in the wild…

...and here’s a screenshot of the final test jig for cross-signing devices in end-to-end encryption, so you will never have to manually verify new devices for a trusted user ever again! We demoed a very early version of this at FOSDEM, but this here is the testing harness for real deal, after several iterations of the spec and implementation to nail down the model. + means the device/user's cross-signing key is trusted, T means it's TOFU:

So, there you have it - welcome to Matrix 1.0, and we look forward to our backlog of feature work now landing!

Massive massive thanks to everyone who has stuck with the project over the years and helped support and grow Matrix - little did we think back in May 2014 that it’d take us this long to exit beta, but hopefully you’ll agree that it’s been worth it :)

Talking of which, we were looking through the photos we took from the first ever session hacking on Matrix back in May 2014…

...suffice it to say that of the architectural options, we went with #3 in the end...

...and that nowadays we actually know how power levels work, in excruciating and (hopefully) well-specified detail :)

There has been an absolutely enormous amount of work to pull Matrix 1.0 together - both on the spec side (thanks to the Spec Core Team for corralling proposals, and everyone who's contributed proposals, and particularly to Travis for editing it all) and the implementation side (thanks to the whole Synapse team for the tedious task of cleaning up everything that was needed for 1.0). And of course, huge thanks go to everyone who has been helping test and debug the Synapse 1.0 release candidates, or just supporting the project to get to this point :)

🔗The Matrix.org Foundation

Finally, as promised, alongside Matrix 1.0, we are very happy to announce the official launch of the finalised Matrix.org Foundation!

This has been a long-running project to ensure that Matrix’s future is governed by a neutral non-profit custodian for the benefit of everyone in the Matrix ecosystem. We started the process nearly a year ago back with the initial proposal Towards Open Governance of Matrix.org, and then legally incorporated the Foundation in October, and published the final governance proposal in January.

As of today the Foundation is finalised and operational, and all the assets for Matrix.org have been transferred from New Vector (the startup we formed in 2017 to hire the core Matrix team). In fact you may already have seen Matrix.org Foundation notices popping up all over the Matrix codebase (as all of New Vector’s work on the public Matrix codebase for the foreseeable is being assigned to the Matrix.org Foundation).

Most importantly, we’re excited to introduce the Guardians of the Matrix.org Foundation. The Guardians are the legal directors of the non-profit Foundation, and are responsible for ensuring that the Foundation (and by extension the Spec Core Team) keeps on mission and neutrally protects the development of Matrix. Guardians are typically independent of the commercial Matrix ecosystem and may even not be members of today’s Matrix community, but are deeply aligned with the mission of the project. Guardians are selected to be respected and trusted by the wider community to uphold the guiding principles of the Foundation and keep the other Guardians honest.

We have started the Foundation with five Guardians - two being the original founders of the Matrix project (Matthew and Amandine) and three being entirely independent, thus ensuring the original Matrix team forms a minority which can be kept in check by the rest of the Guardians. The new Guardians are:

• Prof. Jon Crowcroft - Marconi Professor of Communications Systems in the Computer Lab at the University of Cambridge and the Turing Institute. Jon is a pioneer in the field of decentralised communication, and a fellow of the Royal Society, the ACM, the British Computer Society, the Institution of Engineering and Technology, the Royal Academy of Engineering and the Institute of Electrical and Electronics Engineers.

  Jon is a global expert in decentralisation and data privacy, and is excellently placed to help ensure Matrix stays true to its ideals.

• Ross Schulman - Ross is a senior counsel and senior policy technologist at New America’s Open Technology Institute, where he focuses on internet measurement, emerging technologies, surveillance, and decentralization. Prior to joining OTI, Ross worked for Google.

  Ross brings a unique perspective as a tech- and decentralisation-savvy lawyer to the Foundation, as well as being one of the first non-developers in the Matrix community to run his own homeserver. Ross has been known to walk around Mozfest clutching a battery-powered Synapse in a box, promoting decentralised communication for all.

• Dr. Jutta Steiner - As co-founder and CEO of Parity Technologies, Jutta is dedicated to building a better internet - Web 3.0 - where users’ privacy & control come first. Parity Technologies is a leader in the blockchain space – known to many as the creator of one of the most popular Ethereum clients, it is also the creator of two ambitious new blockchain technlogies, Polkadot and Substrate, that make it easier to experiment and innovate on scalability, encryption and governance.

  Parity has been pioneering Matrix enterprise use since the moment they decided to rely on Matrix for their internal and external communication back in 2016, and now run their own high-volume deployment, with end-to-end encryption enabled by default. Jutta represents organisations who are professionally dependent on Matrix day-to-day, as well as bringing her unique experiences around decentralisation and ensuring that Web 3.0 will be a fair web for all.

We’d like to offer a very warm welcome to the new Guardians, and thank them profusely for giving up their time to join the Foundation and help ensure Matrix stays on course for the years to come.

For the full update on the Foundation, please check out the new website content at https://matrix.org/foundation which should tell you everything you could possibly want to know about the Foundation, the Guardians, the Foundation’s legal Articles of Association, and the day-to-day Rules which define the Open Governance process.

🔗And finally…

Matrix 1.0 has been a bit of an epic to release, but puts us on a much stronger footing for the future.

However, it’s very unlikely that we’d have made it this far if most of the core dev team wasn’t able to work on Matrix as their day job. Right now we are actively looking for large-scale donations to the Matrix.org Foundation (and/or investment in New Vector) to ensure that the team can maintain as tight a focus on core Matrix work as possible, and to ensure the project realises its full potential. While Matrix is growing faster than ever, this perversely means we have more and more distractions - whether that’s keeping the Matrix.org server safe and operational, or handling support requests from the community, or helping new members of the ecosystem get up and running. If you would like Matrix to succeed, please get in touch if you’d like to sponsor work, prioritise features, get support contracts, or otherwise support the project. We’re particularly interested in sponsorship around decentralised reputation work (e.g. publishing a global room directory which users can filter based on their preferences).

Finally, huge thanks to everyone who has continued to support us through thick and thin on Patreon, Liberapay or other platforms. Every little helps here, both in terms of practically keeping the lights on, and also inspiring larger donations & financial support.

So: thank you once again for flying Matrix. We hope you enjoy 1.0, and we look forward to everything else landing on the horizon!

- Matthew, Amandine & the whole Matrix.org Team.

Well here it is: Synapse 1.0.

Synapse 1.0 is the reference implementation of the Matrix 1.0 spec. The goal of the release overall has been to focus on security and stability, such that we can officially declare Synapse (and Matrix) out of beta and recommended for production use. This means changing the default room protocol version used for new rooms to be v4, which includes the new state resolution algorithm, as well as collision-resistant event IDs, which are now formatted to be URL safe.

Synapse 1.0 also ships with support for the upcoming v5 room protocol (which enforces honouring server key validity periods), but this will not be used as the default for new rooms until a sufficient number of servers support it.

Please note that Synapse 1.0 does not include significant performance work or new features - our focus has been almost exclusively on providing a reference implementation of the Matrix 1.0 protocol. But having cleared our backlog on security/stability issues we will finally be now unblocked to pursue work around reducing RAM footprint, eliminating forward-extremity build up, and shipping new features like Edits, Reactions & E2E cross-signing support.

As part of the security work, Synapse 1.0 contains a breaking change that requires a valid TLS certificate on the federation API endpoint. Servers that do not configure their certificate will no longer be able to federate post 1.0.

It is also worth noting that Synapse 1.0.0 is the last release that will support Python 2.x and Postgres 9.4. For more information see here but the TL;DR is that you should upgrade asap.

This release has been a long time coming. Many thanks indeed to everyone who helped test the release candidates and provided feedback along the way.

Synapse 1.0 is just one component of a larger Matrix 1.0 release, which you can read all about here.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. Also, check out our Synapse installation guide page

The changelog since 0.99.5 follows:

🔗Synapse 1.0.0 (2019-06-11)

🔗Bugfixes

• Fix bug where attempting to send transactions with large number of EDUs can fail. (#5418)

🔗Improved Documentation

• Expand the federation guide to include relevant content from the MSC1711 FAQ (#5419)

🔗Internal Changes

• Move password reset links to /_matrix/client/unstable namespace. (#5424)

🔗Synapse 1.0.0rc3 (2019-06-10)

Security: Fix authentication bug introduced in 1.0.0rc1. Please upgrade to rc3 immediately

🔗Synapse 1.0.0rc2 (2019-06-10)

🔗Bugfixes

• Remove redundant warning about key server response validation. (#5392)
• Fix bug where old keys stored in the database with a null valid until timestamp caused all verification requests for that key to fail. (#5415)
• Fix excessive memory using with default federation_verify_certificates: true configuration. (#5417)

🔗Synapse 1.0.0rc1 (2019-06-07)

🔗Features

• Synapse now more efficiently collates room statistics. (#4338, #5260, #5324)
• Add experimental support for relations (aka reactions and edits). (#5220)
• Ability to configure default room version. (#5223, #5249)
• Allow configuring a range for the account validity startup job. (#5276)
• CAS login will now hit the r0 API, not the deprecated v1 one. (#5286)
• Validate federation server TLS certificates by default (implements MSC1711). (#5359)
• Update /_matrix/client/versions to reference support for r0.5.0. (#5360)
• Add a script to generate new signing-key files. (#5361)
• Update upgrade and installation guides ahead of 1.0. (#5371)
• Replace the perspectives configuration section with trusted_key_servers, and make validating the signatures on responses optional (since TLS will do this job for us). (#5374)
• Add ability to perform password reset via email without trusting the identity server. (#5377)
• Set default room version to v4. (#5379)

🔗Bugfixes

• Fixes client-server API not sending "m.heroes" to lazy-load /sync requests when a rooms name or its canonical alias are empty. Thanks to @dnaf for this work! (#5089)
• Prevent federation device list updates breaking when processing multiple updates at once. (#5156)
• Fix worker registration bug caused by ClientReaderSlavedStore being unable to see get_profileinfo. (#5200)
• Fix race when backfilling in rooms with worker mode. (#5221)
• Fix appservice timestamp massaging. (#5233)
• Ensure that server_keys fetched via a notary server are correctly signed. (#5251)
• Show the correct error when logging out and access token is missing. (#5256)
• Fix error code when there is an invalid parameter on /_matrix/client/r0/publicRooms (#5257)
• Fix error when downloading thumbnail with missing width/height parameter. (#5258)
• Fix schema update for account validity. (#5268)
• Fix bug where we leaked extremities when we soft failed events, leading to performance degradation. (#5274, #5278, #5291)
• Fix "db txn 'update_presence' from sentinel context" log messages. (#5275)
• Fix dropped logcontexts during high outbound traffic. (#5277)
• Fix a bug where it is not possible to get events in the federation format with the request GET /_matrix/client/r0/rooms/{roomId}/messages. (#5293)
• Fix performance problems with the rooms stats background update. (#5294)
• Fix noisy 'no key for server' logs. (#5300)
• Fix bug where a notary server would sometimes forget old keys. (#5307)
• Prevent users from setting huge displaynames and avatar URLs. (#5309)
• Fix handling of failures when processing incoming events where calling /event_auth on remote server fails. (#5317)
• Ensure that we have an up-to-date copy of the signing key when validating incoming federation requests. (#5321)
• Fix various problems which made the signing-key notary server time out for some requests. (#5333)
• Fix bug which would make certain operations (such as room joins) block for 20 minutes while attemoting to fetch verification keys. (#5334)
• Fix a bug where we could rapidly mark a server as unreachable even though it was only down for a few minutes. (#5335, #5340)
• Fix a bug where account validity renewal emails could only be sent when email notifs were enabled. (#5341)
• Fix failure when fetching batches of events during backfill, etc. (#5342)
• Add a new room version where the timestamps on events are checked against the validity periods on signing keys. (#5348, #5354)
• Fix room stats and presence background updates to correctly handle missing events. (#5352)
• Include left members in room summaries' heroes. (#5355)
• Fix federation_custom_ca_list configuration option. (#5362)
• Fix missing logcontext warnings on shutdown. (#5369)

🔗Improved Documentation

• Fix docs on resetting the user directory. (#5282)
• Fix notes about ACME in the MSC1711 faq. (#5357)

🔗Internal Changes

• Synapse will now serve the experimental "room complexity" API endpoint. (#5216)
• The base classes for the v1 and v2_alpha REST APIs have been unified. (#5226, #5328)
• Simplifications and comments in do_auth. (#5227)
• Remove urllib3 pin as requests 2.22.0 has been released supporting urllib3 1.25.2. (#5230)
• Preparatory work for key-validity features. (#5232, #5234, #5235, #5236, #5237, #5244, #5250, #5296, #5299, #5343, #5347, #5356)
• Specify the type of reCAPTCHA key to use. (#5283)
• Improve sample config for monthly active user blocking. (#5284)
• Remove spurious debug from MatrixFederationHttpClient.get_json. (#5287)
• Improve logging for logcontext leaks. (#5288)
• Clarify that the admin change password API logs the user out. (#5303)
• New installs will now use the v54 full schema, rather than the full schema v14 and applying incremental updates to v54. (#5320)
• Improve docstrings on MatrixFederationClient. (#5332)
• Clean up FederationClient.get_events for clarity. (#5344)
• Various improvements to debug logging. (#5353)
• Don't run CI build checks until sample config check has passed. (#5370)
• Automatically retry buildkite builds (max twice) when an agent is lost. (#5380)

🔗Dept of Spec 📜

We are sooo close to Matrix 1.0. So close! The PR which announces the arrival of v1.0 exists already.

TravisR said:

Work towards 1.0 is progressing nicely - still on track for a Monday release! View our progress at https://github.com/matrix-org/matrix-doc/projects/2?fullscreen=true

🔗Dept of Servers 🏢

🔗Synapse

Neil, who oversees the creation of Synapse:

Quiet week, not much happening, the only real progress has been shipping a Synapse 1.0 release candidate :) Joking aside we’d really appreciate your help in testing it ahead of the full release next week. Get the RC here https://github.com/matrix-org/synapse/releases/tag/v1.0.0rc1

Also friends don’t let friends operate a Synapse install without a valid certificate on the federation API. Post 1.0, servers that are not compliant will be locked out of the federation. While the copy needs updating to reflect 1.0’s imminent release, our handy FAQ (https://github.com/matrix-org/synapse/blob/master/docs/MSC1711_certificates_FAQ.md) has all the details.

🔗Construct

Construct, a C++ homeserver has been doing testing this week:

I'd like to give special thanks to Ddanyspin97 (Danilo Spinella) for working on building construct with the musl C library so we can enjoy extremely small VM images, as well as Black Hat for building on Alpine linux also using muslc and Thomas Lewis for starting work on the FreeBSD build.

🔗Ruma

Ruma, a homeserver written in Rust, has a new surge of activity following the Rust language syntax developments last week. This Week in Ruma received it's first edition in ~6 months!

Jimmy, project lead:

the ruma homeserver is now targeting stable rust, which means the entire project is now on stable rust

🔗New project: load balancer for synchrotron workers

Soru:

[I] wrote a proof-of-concept load balancer for synchrotron workers....well, proof-of-concept because it isn't tested much, yet!

It routes users to synchrotrons based on their user ID to maximize performance, as per worker docs and, if a synchrotron catches fire it'll also automatically start shuffling workers around.
The repo can be found here, for support just poke soru wherever she is

Contributions are welcome!

Join the newly created room at #matrix-synchrotron-balancer:sorunome.de to discuss.

🔗Dept of SDKs and Frameworks 🧰

🔗Ruby SDK

Ananace:

Just released version 1.1.0 of the Ruby SDK, which includes a few more and improved CS API endpoints, better room alias handling, lazy loading of join rules and guest access, and the forgotten - oops - handling of room avatars.

And released version 1.1.1 of the Ruby SDK too, fixing another embarrassing mistake with not including the S2S API methods correctly, which lost the server version retrieval method (The only currently implemented part of the S2S API)

🔗Dept of Clients 📱

🔗RiotX

Are you in the know about Matrix? If not, prepare to learn about RiotX, a completely re-written Matrix client for Android. This new version has a Kotlin SDK.

Benoit, from the team:

Many things to announce this week on RiotX:

• Crypto is starting to work! We can now decrypt message and send encrypted messages. The whole feature is not implemented yet, but we will merge the PR at the beginning next week before going further. Thanks François for the great work!
• On the timeline, message composed by only emoji are now displayed bigger (as in other Riot clients)
• We can now see details and actions of each message, including state event.
• We can also see the detailed list of reactions for a particular message, and reaction display rendering has been improved. More details of what has been improved by Valere can be found in this PR description: https://github.com/vector-im/riotX-android/pull/168.
• Local echo on message edition has been improved
• The 3 main themes has been implemented (Light, Dark Black). Status theme is still there but has not been tested yet. There are still some little issues which will be fixed in the next coming days.
• The new Home is on develop, composed by a catchup tab, a direct message tab and a group room tab. The drawer is now used to display the groups hierarchy.
• A debug signature has been added to the git repository, so the APK built by buildkite can now be installed and replace a previously installed APK.

There is still lots of work to do, but we are quite confident to deliver a first alpha version of RiotX to a larger audience at the end of June.

🔗famedly talk

We've heard about Famedly before, and here they are with an announcement about their new client:

After some months of work our app 'famedly talk' is now able to send and receive messages. We are now concentrating on implementing the most important functions in a stable manner.

🔗continuum

yuforia:

fix some missing avatars and names of rooms. Normally continuum saves all useful data to disk when syncing, but the previous version missed some of it, the newer version will query the server for the missing pieces

🔗Fractal

This week in Fractal: Alexandre Franke noticed nightly was broken on all three of the computers he was using it on. He bisected and found out when the regression was introduced, but still needs to figure out why it broke and how to fix it.

The symptom is syncing forever on startup and never actually syncing. The commit introducing the borkage is this one.

🔗Riot Web

• Working on polish for reactions and edits
• Improving configuration of the default homeserver
• Added Albanian and West Flemish languages thanks to translation contributors

🔗Riot iOS

• Working on reaction local echo.
• Working on message edition.
• Restrict reactions to room messages.

🔗Riot Android

Nothing special this week.

DID YOU SEE THE RIOTX UPDATE ABOVE?

🔗Dept of Bridges 🌉

🔗matrix-appservice-irc 0.12.0 released

Half-Shot:

matrix-appservice-irc got a release 0.12.0. This is a smaller release, as it's based upon 0.12.0-rc2 which was released back in March (when exams hit). In the meantime, working towards a speedy 0.13.0 release :)

Changelog for 0.12.0:

The bridge now supports upgrading rooms, and will follow room upgrades to the new room.
Added support for the RoomLinkValidator, which allows admins to manually configure rules about plumbing rooms.
A dockerfile is now included.
Add support for "feature flags", allowing users to dynamically enable/disable bridge features for their account.
Add command "!bridgeversion"

High-level plan for 0.13.0:

Fix the most serious P1 bugs like not dropping messages, handling PARTs better, dropping support for older versions of node, and generally better UX.

🔗Dept of Services 🚀

🔗matrixservers.net

Atreatis provided an update to matrixservers.net:

I've open sourced the website of https://matrixservers.net and I am open to contributions towards the website through the github repository: https://github.com/Atreatis/matrixservers
Currently I have refactored matrixservers.net from Hugo to Jekyll which was totally worth it at all times

Join #matrixservers:novaim.com for more.

🔗www.matrix-wiki.org

sanlox:

I launched https://www.matrix-wiki.org as a place to collect information, lists, guides and manuals related to matrix. I hope as a community we can work together on this to create a central knowledge base for the matrix protocol.

🔗Final thoughts 💭

I spent so much of my week migrating matrix.org to a great new design (from Nad), and am really happy with the result. I'm also really happy with Gatsby, the SSG we've chosen. All these things have flaws but overall it works well.

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

🔗Matrix Live - Half-Shot and Sorunome discuss the Discord-Matrix Bridge 🎙

🔗Dept of Spec 📜

TravisR:

Alongside Synapse’s 1.0 release, the Matrix Specification is also getting a 1.0 on June 10th. In practice this means a fresh release of the Client-Server, Server-Server, Application Service, and Identity Service specifications. The Push Specification is just so simple it hasn’t needed an update yet.

Work has been happening to get all the specs in shape for 1.0 - follow along at https://github.com/matrix-org/matrix-doc/projects/2?fullscreen=true

🔗Dept of Servers 🏢

🔗Synapse

Main news in synapse-land this week is that we released Synapse v0.99.5.2, which fixes a big performance regression which was introduced v0.99.5. Folks who are currently on v0.99.5/v0.99.5.1 are advised to upgrade asap.

Otherwise, work has been continuing apace towards a v1.0 release on 10th June.

🔗Construct

Construct is a homeserver implementation in C++:

• Construct has added support for Server Access Control Lists (m.room.server_acl). The exact behavior of the implementation is configurable by the server administrator, and is even more comprehensive than Synapse. In Construct, ACLs can be configured to cover incoming EDUs, outgoing events, and indirectly fetched events.
• The GraphicsMagick library has been utilized to implement the media thumbnailing feature called for by the specification. This is an optional dependency written in C and offers a rich API which as been extremely fun to work with. In the future, expect your Construct to be capable of things like annotating images with the Server Command Line which can be used to generate memes in your chatroom or write custom text on top of stickers!
• On the backend we have devoted significant effort to getting the most out of dynamic-linking to simplify the architecture of a large and modular free-software project. This effort makes adding features and contributions to Construct extremely straightforward. I will enumerate the basics for those familiar with developing C software: We are now loading all modules at runtime with the flags RTLD_GLOBAL|RTLD_DEEPBIND|RTLD_LAZY and we build with the flag --weak-unresolved-symbols; during execution we hook the error handling in ld.so to throw our own C++ exception directly from the PLT slot of a missing symbol. In summary this allows features to be added to Construct by simply creating an interface in a header and definitions in a loadable module, while still enjoying optional loading and unloading of the module at runtime. This is a significant feat that remains unrealized in most existing projects around the free software ecosystem!

🔗Dept of SDKs and Frameworks 🧰

🔗Dept of Clients 📱

🔗nheko 0.6.4 and 0.7.0 roadmap

red_sky updated us on nheko:

Just wanted to pop in and mention that 0.6.4 is out this week. It comes with a decent number of bug fixes.

And then:

Nheko 0.7.0 roadmap is out: https://github.com/Nheko-Reborn/nheko/milestone/2
Also, Nheko-Reborn has a new team member, with MonokelPinguin joining the team : https://github.com/MonokelPinguin. (goes by Nico in the #nheko-reborn room)

el0 pointed out that nheko is available on flathub: https://flathub.org/apps/details/io.github.NhekoReborn.Nheko

🔗continuum, JavaFX client

yuforia:

continuum modified the JSON parser to save a copy of the source. Now fields that are not standard or not yet supported are displayed when you right-click to view the source. For example, "age" and "transaction_id" are not used by the app, but are available when viewing the raw JSON

🔗Neo

f0x has been working on Neo:

Neo can now finally fetch old backlog, with a fancy css loading animation

The Neo room has moved to #neo:pixie.town, and the site+deployment are being moved to https://neo.pixie.town

🔗Fractal 🦶

Alexandre Franke:

Fractal got initial support for being kicked, implemented by first time contributor Isaac Curtis.

We also joined the new call at https://stopthemingmy.app

🔗Riot Web

Continued work on edits and reactions. Also ongoing work on sdk-level support for cross-signing. Released Riot 1.2.0 (and 1.2.1).

🔗Dept of Bridges 🌉

🔗mautrix-telegram, mautrix-facebook, mautrix-hangouts, mautrix-whatsapp

Can you believe it, tulir had a busy week working on his various bridges:

mautrix-telegram now has native Matrix edit support in both directions and basic Prometheus metrics. I'll probably start making 0.6.0 RCs after improving the metrics a bit.

mautrix-facebook and mautrix-hangouts got some bugfixes and options to configure the displaynames of the ghost users on Matrix (like adding (FB) or (Hangouts) suffixes).

mautrix-whatsapp's Matrix puppeting was finished (meaning typing notifs, read receipts and presence updates can now be bridged to WhatsApp) and unexpected disconnection handling was fixed (automatic reconnections should now usually work again). Backfilling needed some hacks to work with Matrix puppeting. The problem is that the real Matrix users have rate limits and can't send timestamp-massaged messages. I decided to just invite the default ghost user to the room while backfilling and make it leave afterwards so that private chat room names and such would stay intact. Some clients like Riot Android/iOS sometimes do weird things, such as moving the room out of "People", but that shouldn't be too bad.

🔗Dept of Ops 🛠

🔗matrix-docker-ansible-deploy makes Synapse optional

Slavi:

matrix-docker-ansible-deploy has seen some work in becoming more flexible.

Synapse is not a requirement anymore. People can now use the playbook to install any of the other services, wiring them to a Synapse instance installed in another way. This is mostly useful for installing bridges or other auxiliary services without going "all in" on the playbook. Not having Synapse as a fixed requirement also opens the door for installing alternative homeservers, when such become available in the future.

On a similar note, to ease maintainability, all bridges have been moved to separate Ansible roles (outside of the matrix-synapse role). There have been lots of improvements related to bridge configuration with even more likely to come soon.

🔗0.99.5.2 kubernetes-optimized images

Ananace had 0.99.5.2 kubernetes-optimized images ready to go very quickly.

🔗Dept of Services 🚀

🔗Matrix User Documentation launches 🎉

Aaron (formerly known as Aaron Raimist) has been a major help to the Matrix project, and has recently launched something new:

I have started another Matrix documentation project, Matrix User Documentation. I know, there are several however none of them aim to actually be a handbook for using Riot or any other clients. I would like for this to become a place where you can find "The (unofficial) Riot Handbook" or "The (unofficial) Quaternion Handbook" as well as a place where you can find answers to more of your FAQs.

It's still very early, only a few answers are written so far and none of the handbooks have been started but I would welcome any community assistance. You can view what I have so far at: http://mud.raim.ist/en/latest/

You can contribute some documentation at: https://github.com/aaronraimist/matrix-user-documentation and join our room: #matrix-user-documentation:raim.ist

🔗matrixservers.net

We welcomed Atreatis to Matrix recently. They are creating a new directory service:

Someone mentioned earlier that there is a need for a list of public servers on Github so people can find and add their public matrix servers. Which combines well with what I am building at the moment. Hopefully you guys don't mind helping me out on creating the list and spreading the word. The repository can be found here: https://github.com/Atreatis/matrix-servers

🔗Final thoughts 💭

Matrix Shell Suite is a promising looking project which could deserve more attention.

Thank You to PyConWeb and especially to Hardy for letting richvdh and myself host TWO Matrix sessions at your event last weekend - was a great time and I hope everyone enjoyed the presentations and workshops.

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

0.99.5.2 contains a critical performance fix following a regression that was introduced in 0.99.5. Affected servers will have experienced increased CPU and RAM usage with a knock on effect of generally sluggish performance.

Separately, we are also looking into reports relating to further performance degradations that may have been introduced as part of 0.99.5, though consider the 0.99.5.2 fix to be a significant improvement on previous 0.99.5.x releases.

Please upgrade asap.

You can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. Also, check out our Synapse installation guide page.

🔗Synapse v0.99.5.2 Changelog

🔗Bugfixes

• Fix bug where we leaked extremities when we soft failed events, leading to performance degradation. (#5274, #5278, #5291)

Hi all,

After lots of refinements, polishing and a few distractions we’re finally at the point of announcing the final timeline for both Matrix 1.0 and Synapse 1.0! We are targeting Monday 10th June as our release date - please consider this your two week warning!

This is the end game of the process we began back in February when we released the first stable release of the Server-Server API at FOSDEM, and started the Synapse 0.99 release series to prepare for 1.0.

Matrix 1.0 refers to the upcoming set of API releases which provides a matched set of stable and secure APIs across all of Matrix - at which point the project (at last) exits beta! In practice, this will be Client-Server API 0.5 (including final membership lazy loading, E2E backups and interactive verification and lots more), SS API 0.2 (including server key validity period fixes and associated v5 room protocol) and any other spec updates. The next 2 weeks will see a flurry of spec activity as we get everything together - you can see the full list and track the progress for the CS 0.5 spec release at https://github.com/matrix-org/matrix-doc/projects/2.

Meanwhile, Synapse 1.0 will be the reference implementation of Matrix 1.0, and so makes the changes required to implement Matrix 1.0 and close all currently known security and stability issues and thus exit beta. This means changing the default room protocol version used for new rooms to be v4, which includes the new state resolution algorithm, as well as collision-resistant event IDs, which are now formatted to be URL safe. Support for v4 rooms shipped in Synapse 0.99.5.1, so please upgrade asap to 0.99.5.1 before 1.0 is released to ease the transition.. Synapse 1.0 will also ship with support for the upcoming v5 room protocol (which enforces honouring server key validity periods), but this will not used as the default for new rooms until sufficient servers are speaking Matrix 1.0.

As part of the security work, Matrix 1.0 and Synapse 1.0 also contains a breaking change that requires a valid TLS certificate on the federation API endpoint. Servers that do not configure their certificate will no longer be able to federate post 1.0

You can check that your server has been correctly configured here and see here for more info on what you need to do. If in doubt head to #synapse:matrix.org.

We've been tracking readiness for the certificate change at https://arewereadyyet.com, at the time of writing 68% of active servers on the federation have valid certificates. We obviously would want that number to be higher, however since the largest installations have upgraded the total number of users who are ready for 1.0 stands at 96%, which we consider to be high enough to release 1.0.

This is not a drill, from here until 10th June we need everyone to not only ensure that their own server is ready, but also to encourage their fellow admins to update as well. With your help we can get everyone over the line!

Thanks everyone for your help to date, especially those providing support in #synapse:matrix.org.

Onwards!

🔗Matrix Live - Wilko, creator of Pattle 🎙

This week I chatted to Wilko, creator of Pattle.

🔗Dept of Servers 🏢

🔗Synapse

Neil:

Folks, big news this week as we announce that Synapse v1.0 is scheduled for release on 10th June - read all about it here

Aside from that we shipped v0.99.5.1 which (hopefully) is the penultimate release ahead of v1.0. Please, everyone upgrade to v0.99.5.1 because it implements rooms v4 which will be the default room version in Synapse v1.0.

0.99.5.1 also contains experimental support for edits and reactions which are currently hidden behind a Riot labs flag.

🔗Dendrite

Brendan:

Some activity has been happening this week in Dendrite-land, with Brendan adding support for Go modules to the project, and anoa adding SyTest runs to the project’s CI. These were two long-awaited maintenance works that will make working on Dendrite much easier in the future!

🔗Dept of SDKs and Frameworks 🧰

libQMatrixClient (that will soon become libQuotient) 0.5.2 has been released, with the sole purpose of fixing a nasty bug unmarking some direct chats when doing initial sync or a clean-cache start up. Everybody on 0.5.x branch is advised to upgrade.

🔗Dept of Clients 📱

🔗QMatrixClient to Quotient

kitsune:

The process of renaming QMatrixClient to Quotient has commenced - expect some turbulence while we're transitioning. The place for the repos is at https://github.com/quotient-im. Note that although the library repo name has changed, the old version of the library will continue releases under the old name (libQMatrixClient), and only in the master branch the library will be renamed. In most cases redirects should bring you home even if you request the old URL (thanks to GitHub); however, people with git repos are strongly advised to update their remotes to new URLs!

In other news:

For platforms that don't have a separate libQMatrixClient package (that is, Windows, macOS, Flatpak and AppImage), Quaternion 0.0.9.4 has been rebuilt with libQMatrixClient 0.5.2 - in the form of Quaternion 0.0.9.4c release.

🔗tangent

tangent is an embeddable HTML client from sanlox:

I added guest login, possibility to disable guest login, possibility to set own message limit on startup and various error messages. Cleaned up the code to make it more consistent and faster. Everything I wanted to have for this tiny embeddable web chat is now there so I'd consider it finished for now.

Check it out at: https://sanlox.dev/tangent/

🔗continuum

yuforia updated their JavaJX client:

minor changes in continuum this week:

• updated controlsfx to version 11, which has better modularization support
• sync issue indicator now uses NotificationPane from controlsfx, so now it has slide-in animation when it appears https://matrix.org/_matrix/media/v1/download/matrix.org/VtPORWFqBamfnuJtPtgEXWBs

🔗Neo now has image, video and general file sending

Fox:

Neo now supports sending multiple images, videos and files at once. You'll get a bar with previews, and the option to remove them from the queue/add more.
There's also been a bunch of changes to how events are handled. Images and videos should be much more robust against missing keys (no thumbnail, no information, etc), and there's basic displaying of the most common state events.
I also added an experimental media repository fallback option, which is disabled by default, and only implemented for room avatars. This allows you to provide a list of alternative homeservers Neo is allowed to try when your own homeserver can't load a piece of media.
Due to the loss of lain.haus, I lost admin access in the Neo room, so keep your eyes out for a new one when my infra is back.
I'm currently not really versioning anything, but I do push significant commits to https://neo.lain.haus/neo for people to try. Once it gets to a more useable state, I'll start adopting semantic versioning.

🔗Spectral update

Black Hat:

You can now paste images from the clipboard in Spectral. It is also possible to change room name and topic in room settings. A new release is pushed to Flathub to address direct room issues. Also there've been discussions about implementing custom room themes and backgrounds.

🔗Riot Web

• Continued work on reactions and edits
• New emoji font added to standardise emoji appearance and assist OS / browser combinations that don’t support emoji by default

🔗Riot Mobile

• Continued work on reactions and edits
• Riot-iOS has a new actions menu on event
• Fix registration with an email

🔗Riot and RiotX available from F-Droid

krombel updated their Riot(X) F-Droid repos:

I finally found some time to update my fdroid repos which provide the development builds of riot and riotx. Now the builds of buildkite are part of the repos.
There are now 4 separate repos: One for each app and flavor. You can have a look at https://fdroid.krombel.de to find out the URL for the version you want to use.

Riot-dev (F-Droid; Repo; Build-Source)
    https://fdroid.krombel.de/riot-dev-fdroid
    https://fdroid.krombel.de/riot-dev-fdroid/fdroid/repo
    Fingerprint: 312E07B9444D0D1B615EBBAAC55EA4E5A54E123C3BEFCCA5D18B5E12DFC95BDC

Riot-dev (GPlay; Repo; Build-Source)
    https://fdroid.krombel.de/riot-dev-gplay
    https://fdroid.krombel.de/riot-dev-gplay/fdroid/repo
    Fingerprint: 81EDF1741A51B944B00B55E307C7AA043623CB646599182A104B895B6B319844

RiotX-dev (F-Droid; Repo; Build-Source)
    https://fdroid.krombel.de/riotx-dev-fdroid
    https://fdroid.krombel.de/riotx-dev-fdroid/fdroid/repo
    Fingerprint: FD146EF30FA9F8F075BDCD9F02F069D22061B1DF7CC90E90821750A7184BF53D
RiotX-dev (GPlay; Repo; Build-Source)

    https://fdroid.krombel.de/riotx-dev-gplay
    https://fdroid.krombel.de/riotx-dev-gplay/fdroid/repo
    Fingerprint: 5564AB4D4BF9461AF7955449246F12D7E792A8D65165EBB2C0E90E65E77D5095

🔗Dept of Bridges 🌉

🔗Major WhatsApp bridging update

tulir made great strides on mautrix-whatsapp this week:

I've been working on mautrix-whatsapp to add history bridging and Matrix puppeting.

• New portals are populated with some history when creating them (exact count is configurable)
• All messages missed during bridge downtime are backfilled
• Creating portals is smarter now:
• When logging in initially, it'll create portals for a few recent chats (count configurable).
• It'll create portals when there are incoming messages as before, but it should no longer create portals for chats that only have old messages.
• The missed message backfilling creates portals when necessary.

Matrix puppeting isn't quite finished yet. It can already use your Matrix account to bridge messages sent from whatsapp mobile, but it doesn't use the account to bridge EDUs (typing notifs, presence, read receipts) yet.

To make the history bridging a bit nicer, I made a PR to fix timestamp massaging in synapse: https://github.com/matrix-org/synapse/pull/5233. Timestamp massaging was removed from the spec in 1.0, but it wasn't intentionally removed from synapse, it just broke due to other changes. It was probably supposed to stay there as an easter egg until there's a proper solution for bridging history.

🔗Dept of Ops 🛠

Ananace pushed updated K8s images for Synapse 0.99.5

Bubu updated synapse on Arch:

Archlinux updated to synapse 0.99.5 as well after the requests, urllib3 incomopatibility thing was sorted out.

Mathijs updated the synapse avhost docker image:

With synapse v 0.99.5.1 the avhost docker image has finally moved to python 3

🔗Dept of Services 🚀

🔗Modular

New widget for Scalar: EtherCalc.

🔗Dept of Bots 🤖

🔗QuatBot

Very very new, let's take a look at QuatBot, which uses libQuotient:

QuatBot is a simple meeting-management bot for use with the Matrix. Taking turns during an online meeting -- and making sure everyone gets to have their say -- takes a bit of organizing, and this bot helps you do that. QuatBot runs as a command-line application.

🔗Dept of Articles 📝

uhoreg wrote a really informative article about key verification:

For those who want to know more about the security behind emoji-based key verification, I've written a blog post about it: https://www.uhoreg.ca/blog/20190514-1146

🔗Dept of GSOC 🎓

🔗GSoC 2019 – Reliable Bridges

Thanks Kai for this introduction to his project:

The Reliable Bridges GSoC project is about implementing a feedback mechanism for the Matrix network in cases where bridges are not able to properly handle messages. Currently clients are unable to know if a message was successfully sent over a bridge. With the new mechanism in place they get the information about errors happening at bridges and can behave accordingly by e.g. notify the user of the failed delivery.

The implementation is foremost focused on the matrix-appservice-dicord bridge which uses the underlying matrix-appservice-bridge. A JS bridge was chosen so that as much code as possible is brought into the SDK and other bridges can profit sooner from the work done.

The first step for implementing the new feature will be the signaling of permanent errors occurring at the bridge (in contrast to temporary failures). They might occur e.g. when the sending account was banned on the bridged foreign network. These permanent errors will be implemented as a new type of PDU originating from the bridge.

After permanent errors are done, the subsequent weeks will see work on temporary failures which might include work on Synapse as well as work on Riot Web to have a client which actually uses the new events. As these events are new features being introduced there will also be a MSC draft. There everyone can check out the proposed solution and tear it apart with their criticism (if applicable 😉).

For discussions related to the GSoC project or when you have some opinions on how the MSC should look like, you are invited to join the Reliable Bridges Matrix room.

🔗That's all I know 🏁

Finally Alan, friend of Matrix and creator of TADHack and TADSummit is doing an "Open Source Telecom Software Survey" - if you could add value to this research by completing it please do so here: http://alanquayle.com/2019/05/open-source-telecom-software-project-survey/

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Okay folks, this is an important one. v0.99.5.1 will be the last release before we ship Synapse v1.0. It is really important that you upgrade to v0.99.5.1 because it implements rooms version 4 - which is the room version that Synapse 1.0 will default to.

This means that Synapse 1.0 servers will create new rooms as version 4 by default and servers that have not upgraded to at least v0.99.5.1 will not be able to join those rooms.

Over the coming days we will announce a release day for Synapse v1.0, the idea is to give admins 2 weeks notice so that anyone yet to configure their federation SSL certificate has time to do so. This is important, failure to configure your certs will mean not being able to federate with v1.0 servers. If you are not sure if you certs are valid, you can test here and read here for more info on what to do.

Aside from room v4, this release also includes the ability to blacklist specific IPs from federating as well as experimental support for edits and reactions. We are not quite ready to mark the feature 'done done', but it is very close. Watch out for news as the feature lands properly.

We're really close to v1.0 now, give us a few more days and we'll announce an official release date.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. Also, check out our Synapse installation guide page

🔗Synapse v0.99.5.1 Changelog (since v0.99.4)

🔗Features

• Add ability to blacklist IP ranges for the federation client. (#5043)
• Ratelimiting configuration for clients sending messages and the federation server has been altered to match login ratelimiting. The old configuration names will continue working. Check the sample config for details of the new names. (#5181)
• Drop support for the undocumented /_matrix/client/v2_alpha API prefix. (#5190)
• Add an option to disable per-room profiles. (#5196)
• Stick an expiration date to any registered user missing one at startup if account validity is enabled. (#5204)
• Add experimental support for relations (aka reactions and edits). (#5209, #5211, #5203, #5212)
• Add a room version 4 which uses a new event ID format, as per MSC2002. (#5210, #5217)

🔗Bugfixes

• Fix image orientation when generating thumbnails (needs pillow>=4.3.0). Contributed by Pau Rodriguez-Estivill. (#5039)
• Exclude soft-failed events from forward-extremity candidates: fixes "No forward extremities left!" error. (#5146)
• Re-order stages in registration flows such that msisdn and email verification are done last. (#5174)
• Fix 3pid guest invites. (#5177)
• Fix a bug where the register endpoint would fail with M_THREEPID_IN_USE instead of returning an account previously registered in the same session. (#5187)
• Prevent registration for user ids that are too long to fit into a state key. Contributed by Reid Anderson. (#5198)
• Fix incompatibility between ACME support and Python 3.5.2. (#5218)
• Fix error handling for rooms whose versions are unknown. (#5219)

🔗Internal Changes

• Make /sync attempt to return device updates for both joined and invited users. Note that this doesn't currently work correctly due to other bugs. (#3484)
• Update tests to consistently be configured via the same code that is used when loading from configuration files. (#5171, #5185)
• Allow client event serialization to be async. (#5183)
• Expose DataStore._get_events as get_events_as_list. (#5184)
• Make generating SQL bounds for pagination generic. (#5191)
• Stop telling people to install the optional dependencies by default. (#5197)

🔗Matrix Live - Reactions and Edits coming to Matrix

If Since you've been chatting on Matrix this week, you'll have noticed some new features rolling out.

In Riot develop, hit Labs in the Settings menu and you'll be able to try out the new Reactions and message editing features.

🔗Dept of Servers

🔗Synapse

Neil said:

This week we shipped 0.99.4 - no stand out ‘oh my gosh’ headlines, just lot lot of bug fixes and perf improvements - get involved. We’ve also been working hard to getting things like reactions and edits going, as well more prep for improving perf for small server instances.

🔗Construct

First Construct update for a few weeks:

Construct has added a server command line available in any room when starting a line with a special character (by default it's '\'). The commands are private so the room doesn't actually see it. More on this next week, or check out #test:zemos.net.

🔗Dept of SDKs and Frameworks

🔗Ruby SDK now 1.0!

Big news from Ananace this week:

Finished getting the test coverage to a reasonable enough percentage to feel comfortable releasing the Ruby Matrix SDK as a nice and stable 1.0 (.0)

This is great! All Rubyists should check it out. Ananace notes:

I still need to set up fixtures for all the API endpoints I'm exposing so I can verify every single API call as well, but I see that as a later thing as that relates mostly to how the other side (the HS) handles my input, not to how the SDK itself handles said input

🔗Dept of Clients

🔗Pattle

Wilko:

A new version of Pattle is available on F-droid!

Changes include:

• Implement direct chats correctly!
  • Use user user avatar as chat avatar if direct
  • Hide user name in direct chat
• Use names of room members if no room name is set (whether the chat is direct or not)
• Add border to left of replied-to messages to easily differentiate them
• Show redaction events!
• Use icons instead of letters if chat has no avatar
  • Use different icons for direct chats than group chats (and in the future public chats)
• Use user color for direct chats if the user has no avatar
• Simplify member change messages ('Pat has joined' -> 'Pat joined', etc)
• Tweak font sizes (thanks to Mathieu Velten)!
• Change date header style (smaller and full caps)

Development happens here and development discussion happens here: #pattle:matrix.org!

To install this release, add the following repo in F-droid:

https://fdroid.pattle.im/?fingerprint=E91F63CA6AE04F8E7EA53E52242EAF8779559209B8A342F152F9E7265E3EA729

And install 'Pattle'.

🔗Riot (various)

This week the various Riot teams (web, iOS, Android) have been spending time implementing reactions/edits, but also:

🔗Riot Android

• PR review from community: Matomo SDK will replace Piwik SDK
• Weblate is up again, a sync has been done.

🔗RiotX (Android)

• Crypto integration is still ongoing
• Reactions are coming soon!
• New home and navigation to rooms development started

🔗tangent

tangent is an exciting young project designed to create a very lightweight browser-based new client. sanlox:

did alot of changes on tangent to make it more stable and performant, only registration and guest login are left to do, git changed to https://gitlab.com/sanlox/tangent

Test the latest at: https://sanlox.dev/tangent/

🔗Spectral timeline UI and more

Black Hat:

• Polished the timeline UI for Spectral.
• Added m.video support.
• Also added drag-and-drop support in Spectral.

🔗Fractal now with keyboard shortcuts

Alexandre Franke:

Ana, first time contributor, added a bunch of cool keyboard shortcuts to ease navigation in Fractal. One can now e.g. go down to the next room with unread messages with ctrl+shift+page down.

🔗continuum updates

yuforia:

• Load message history from server lazily when scrolling
• Loaded messages are always saved to disk, so if you aren't offline for too long, only a few messages will need to be fetched the next time you login
• The screenshot also shows the UI for handling invitations

🔗early-day reactions support in Quaternion

kitsune:

This week was too packed IRL but I took an hour to experiment with reactions that are a rage in riot.im/develop these days. An experimental "kitsune-reactions" branch understands reaction events and allows clients to further process them. A proof-of-concept in Quaternion will land over the weekend.

🔗Scylla

via Aaron Raimist:

Update on Scylla from VaNilLa: Though I am very busy with schoolwork, I found time to fix a long-standing issue: the names of private messages showing up as <No Name>. this is gone now, and rooms are sorted alphabetically to make it easier to navigate.
Try Scylla here: https://scylla.danilafe.com
Come join us in #scylla:riot.danilafe.com to discuss

🔗Dept of Bridges

🔗matrix-appservice-discord 0.5.0

Half-Shot:

Evening, we've just cut the first RC of matrix-appservice-discord 0.5.0. Most of the changes this time around are bugfixes to formatting, internal re-architecturing / performance boosts and shifting more things to the database. Please help test if you run an instance, so we can get a 0.5.0 out swiftly. The next release has a lot of features planned for it 😉

Also - congratulations to Half-Shot on finishing all his final-year exams! How will you fill your time now?

🔗mautrix-telegram/mautrix-hangouts/mautrix-facebook/mautrix-whatsapp

Think tulir's been busy this week?

Lots of small changes in my projects:

• mautrix-telegram had a bunch of bugs fixed, like multiline messages from some clients not being bridged correctly.
• mautrix-hangouts got matrix->hangouts image bridging and some bugfixes.
• mautrix-facebook also got some bugfixes.
• mautrix-whatsapp now informs the user about connection problems rather than crashing and has commands to try to reconnect. It also now bridges redactions in both directions.

🔗Dept of Ops

🔗kubernetes from Ananace

Ananace updated their k8s packaging for Synapse to 0.99.4: https://github.com/ananace/matrix-synapse

🔗Debian Packages

andrewsh announced the Debian Matrix Packagers Team has its own dedicated blog

for everything about Matrix on Debian, join #debian-matrix:matrix.org

🔗silvio/avhost synapse docker container

Mathijs:

the silvio/avhost synapse docker container is making progress in moving to python 3, the container is made smaller and no longer runs synapse as root

There are some important differences between this and https://github.com/matrix-org/synapse/tree/master/contrib/docker:

it puts all the configuration files, logfiles and media files in the volume, you don't really configure it with environmental variables but just by editing the homeserver.yaml file in the volume, and it also contains coturn

🔗Dept of Bots

Half-Shot created a "reactbot":

Hot on the heels of the rapid developments of the reaction work, I've written a bot that automatically reacts when spotting certain phrases inside rooms. Honestly I have no idea what the use case is for this, but it exists and I currently use it to stick the :dog: reaction on every dog related event. https://github.com/Half-Shot/matrix-reactbot

The bot is also now used in #twim:matrix.org, whhere it performs the needed work of adding red circles to submissions.

tulir's karma maubot now also supports reactions and redactions, and the maubot sed plugin now underlines changes in messages.

🔗Dept of Status of Matrix

🔗t2bot.io has launched a new website

TravisR:

t2bot.io has launched a new website, surpassed a milestone of 300k total bridged users (70k of those are active monthly), and launched 2 new early-beta-quality bridges. If you're looking to try out tulir's latest Hangouts or Facebook Messenger bridge then t2bot.io is an option for that, assuming you don't mind the occasional bug, missing feature, or problem. Check out https://t2bot.io/hangouts/ and https://t2bot.io/messenger/ for setup instructions.

🔗That's all I know

See you next week, and be sure to stop by #twim:matrix.org with your updates!

PS Massive thanks to Aaron Raimist for doing the needful to make full articles appear in RSS!

Hey ho Synapse release day.

0.99.4 is a maintenance release collecting together all of the bug fixes and performance improvements over the past few weeks, additionally there is further support for the upcoming 1.0 release (more info coming soon). One thing worth calling out is how many community contributions have made their way into 0.99.4, take a look at the change log for details, but many thanks to everyone submitting PRs, keep them coming!

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. Also, check out our Synapse installation guide page

🔗Synapse 0.99.4 Changelog

🔗Features

• Add systemd-python to the optional dependencies to enable logging to the systemd journal. Install with pip install matrix-synapse[systemd]. (#4339)
• Add a default .m.rule.tombstone push rule. (#4867)
• Add ability for password provider modules to bind email addresses to users upon registration. (#4947)
• Implementation of MSC1711 including config options for requiring valid TLS certificates for federation traffic, the ability to disable TLS validation for specific domains, and the ability to specify your own list of CA certificates. (#4967)
• Remove presence list support as per MSC 1819. (#4989)
• Reduce CPU usage starting pushers during start up. (#4991)
• Add a delete group admin API. (#5002)
• Add config option to block users from looking up 3PIDs. (#5010)
• Add context to phonehome stats. (#5020)
• Configure the example systemd units to have a log identifier of matrix-synapse instead of the executable name, python. Contributed by Christoph Müller. (#5023)
• Add time-based account expiration. (#5027, #5047, #5073, #5116)
• Add support for handling /versions, /voip and /push_rules client endpoints to client_reader worker. (#5063, #5065, #5070)
• Add an configuration option to require authentication on /publicRooms and /profile endpoints. (#5083)
• Move admin APIs to /_synapse/admin/v1. (The old paths are retained for backwards-compatibility, for now). (#5119)
• Implement an admin API for sending server notices. Many thanks to @krombel who provided a foundation for this work. (#5121, #5142)

🔗Bugfixes

• Avoid redundant URL encoding of redirect URL for SSO login in the fallback login page. Fixes a regression introduced in #4220. Contributed by Marcel Fabian Krüger ("zaugin"). (#4555)
• Fix bug where presence updates were sent to all servers in a room when a new server joined, rather than to just the new server. (#4942, #5103)
• Fix sync bug which made accepting invites unreliable in worker-mode synapses. (#4955, #4956)
• start.sh: Fix the --no-rate-limit option for messages and make it bypass rate limit on registration and login too. (#4981)
• Transfer related groups on room upgrade. (#4990)
• Prevent the ability to kick users from a room they aren't in. (#4999)
• Fix issue #4596 so synapse_port_db script works with --curses option on Python 3. Contributed by Anders Jensen-Waud [email protected]. (#5003)
• Clients timing out/disappearing while downloading from the media repository will now no longer log a spurious "Producer was not unregistered" message. (#5009)
• Fix "cannot import name execute_batch" error with postgres. (#5032)
• Fix disappearing exceptions in manhole. (#5035)
• Workaround bug in twisted where attempting too many concurrent DNS requests could cause it to hang due to running out of file descriptors. (#5037)
• Make sure we're not registering the same 3pid twice on registration. (#5071)
• Don't crash on lack of expiry templates. (#5077)
• Fix the ratelimting on third party invites. (#5104)
• Add some missing limitations to room alias creation. (#5124, #5128)
• Limit the number of EDUs in transactions to 100 as expected by synapse. Thanks to @superboum for this work! (#5138)
• Fix bogus imports in unit tests. (#5154)

🔗Internal Changes

• Add test to verify threepid auth check added in #4435. (#4474)
• Fix/improve some docstrings in the replication code. (#4949)
• Split synapse.replication.tcp.streams into smaller files. (#4953)
• Refactor replication row generation/parsing. (#4954)
• Run black to clean up formatting on synapse/storage/roommember.py and synapse/storage/events.py. (#4959)
• Remove log line for password via the admin API. (#4965)
• Fix typo in TLS filenames in docker/README.md. Also add the '-p' commandline option to the 'docker run' example. Contributed by Jurrie Overgoor. (#4968)
• Refactor room version definitions. (#4969)
• Reduce log level of .well-known/matrix/client responses. (#4972)
• Add config.signing_key_path that can be read by synapse.config utility. (#4974)
• Track which identity server is used when binding a threepid and use that for unbinding, as per MSC1915. (#4982)
• Rewrite KeyringTestCase as a HomeserverTestCase. (#4985)
• README updates: Corrected the default POSTGRES_USER. Added port forwarding hint in TLS section. (#4987)
• Remove a number of unused tables from the database schema. (#4992, #5028, #5033)
• Run black on the remainder of synapse/storage/. (#4996)
• Fix grammar in get_current_users_in_room and give it a docstring. (#4998)
• Clean up some code in the server-key Keyring. (#5001)
• Convert SYNAPSE_NO_TLS Docker variable to boolean for user friendliness. Contributed by Gabriel Eckerson. (#5005)
• Refactor synapse.storage._base._simple_select_list_paginate. (#5007)
• Store the notary server name correctly in server_keys_json. (#5024)
• Rewrite Datastore.get_server_verify_keys to reduce the number of database transactions. (#5030)
• Remove extraneous period from copyright headers. (#5046)
• Update documentation for where to get Synapse packages. (#5067)
• Add workarounds for pep-517 install errors. (#5098)
• Improve logging when event-signature checks fail. (#5100)
• Factor out an "assert_requester_is_admin" function. (#5120)
• Remove the requirement to authenticate for /admin/server_version. (#5122)
• Prevent an exception from being raised in a IResolutionReceiver and use a more generic error message for blacklisted URL previews. (#5155)
• Run black on the tests directory. (#5170)
• Fix CI after new release of isort. (#5179)