🔗New matrix.org Security Disclosure Policy and moderation pages

Check out the Matrix Official Security Disclosure Policy, which also features a "Hall of Fame": https://matrix.org/security-disclosure-policy/

We also have an official moderation guide now at https://matrix.org/docs/guides/moderation

🔗Servers

🔗Synapse

🔗Last week

Work progresses on reactions support, expect a more formal MSC rsn, but for check out https://github.com/matrix-org/matrix-doc/blob/matthew/msc1849/proposals/1849-aggregations.md for more details. Aside from that we now have an API to send server notices, support is coming for blacklisting IP ranges for federation traffic and finally we published a security disclosure policy and hall of fame https://matrix.org/security-disclosure-policy/. We’ve also been tracking down some device management bugs that prevent e2e message decryption as well as fixing some security bugs.

🔗Next week

More reactions work, device management bug hunting and server key validity support.

🔗Crypto

• Olm 3.1.2 bugfix release.
• Pantalaimon: Initial SAS support through panctl http://webmshare.com/play/QeBY1
• Pantalaimon: Support for sending out key requests.

🔗Clients

There are some BIG (capital and bold!) client releases this week, let's take a look

🔗Quaternion 0.0.9.4

Big release for Quaternion this week, following lots of work on libQMatrixClient. kitsune reports that:

Quaternion is now officially at version 0.0.9.4! Optional native scrollbar for the timeline, files uploading, initial support of matrix.to links (the foundation for future Matrix URIs), first complete translations, and much much more - the long list is here: https://github.com/QMatrixClient/Quaternion/releases/tag/0.0.9.4. Packagers are advised to take a look at the building and packaging section of the release notes: there are a few updates for you there.

🔗Spectral

Continuing the QT-client theme, Black Hat has released a new version of Spectral to FlatHub, join #spectral:matrix.org for more:

A new version of Spectral is released on Flathub this week! The last release is half a year ago, and there are a lot of changes since then, including a better UI, bug fixes and performance improvements.

Changelog:

Increase minimum Qt version to 5.12.
Redesigned UI.
Emoji and username auto completion in text input.
Respect server-side notification settings.
Emoji picker.
Fix font size in HiDPI.
Improved reply UI/UX and rich reply UI.
Sending/receiving typing notifications.
Switch to hoedown for parsing markdown.
Display notification count.
Responsive UI.
Fix #2(Room/People separator issue).
Infer device name from system information.
Image caching.

And of course various performance improvements. For my account Spectral takes ~35MB RAM initially, compared to 45~50MB before.

🔗Pattle new release

Wilko came to announce a new Pattle release, that's the Flutter client:

A new release of Pattle has been pushed to F-droid! Lots of different changes this release, including:

• Viewing images!
• Actual local echo (message is immediately placed in the timeline)
• Sent state indicators are now shown next to the message time (clock for still being sent and a checkmark for sent)
• Show member change events! (x has joined, left, invited, etc)
• Thumbnails are now used for chat avatars, which should improve performance a bit
• Other small fixes :)

Check out a video demo at here.

🔗continuum

yuforia reports that:

This week in continuum:

• Remove support for embedded webview and open external browsers when necessary instead. Many users actually prefer it this way. And dependency on a fairly large native module is gone.
• Improvements on the emoji input. Removed some style classes and tweaked some sizes to make the appearance more compact and flat. https://matrix.org/_matrix/media/v1/download/matrix.org/PvFFPAvoDhiHghsyeJnWVyAK

🔗Riot Web

Initial UI work on reactions and editing, nothing useable yet though.

🔗Riot iOS

• Release of 0.8.5 :
  • Grouped notifications
  • Interactive device verification (by emoji)
  • WebRTC and Jitsi libs updates
  • min iOS version is now 10 (Jitsi constraints)
• Initial UI work on reactions, nothing useable yet though.

🔗Riot Android

Benoit from the Riot team:

• CI has been configured on Buildkite for the Android matrix sdk, riot and riotx.
• The Android sdk can now be integrate as a grate dependency via Jitpack.
• Riot 0.9.1 has been released with sas feature included! Device verification is easier with riot web and riot Android users, soon with riot ios users. Will be on production channel on Monday.
• We have finally pushed the security fix on GH.
• Also working on the crypto modularization to integrate it on RiotX.
• We will concentrate our effort on RiotX now

🔗RiotX (Android)

• Working on reactions

🔗Bridges

🔗matrix-appservice-discord searching for a new maintainer

Sorunome reports that:

We (matrix-appservice-discord) are looking for a new maintainer!

What are we looking for in a maintainer?
You wouldn't have to be super active, however a few minutes per day would be greatly appreciated. You'd be responsible of reviewing PRs! Because of this not being afraid of Typescript and not being afraid of Discord is required - no need to be at pro-level!

So we are basically looking for someone who can help us look over PRs and maybe do some themself. If you think you are interested, please message us in our matrix room!

🔗Bridging Facebook Messenger

tulir:

I made a new Facebook Messenger bridge: https://github.com/tulir/mautrix-facebook / #facebook:maunium.net. Currently the main difference to matrix-puppet-facebook is multi-user support like my WhatsApp and Telegram bridges. The bridge is Python and the code structure is similar to mautrix-telegram, so I'll probably eventually create a generic bridge library out of the common parts.

🔗SDKs and Frameworks

🔗Ruby SDK

Ananace reports that:

Working on the Ruby SDK again, planning to try and get it to the point where I feel comfortable calling it a provisionary 1.0 release. I want to propose additions to various software using it after all, and that tends to look better if it's 1.0 - or close. 😃

🔗That's all I know

That's all for this week folks, come chat in #twim:matrix.org for more, or to share what you've been working on.

PS this interesting-looking project is not ready for public eyes on it yet, so please refrain from checking out the code and discussing with the author in #tangent:matrix.org

🔗Hello and welcome

No Ben this week. I'm afraid this means no matrix live, but fear not your Ben orientated programming will resume next week.

🔗Things that have happened

🔗Riot Android

• New Riot.im application has been delivered to the PlayStore: https://play.google.com/store/apps/details?id=im.vector.app. It replaces the previous app. More details here: https://medium.com/@RiotChat/riot-im-android-security-update-2b3f655ad739
• François and Benoit were at AndroidMakers Paris on Tuesday and Wednesday. We’ve seen plenty of interesting conferences and come back with many ideas to improve Riot UX/UI/Implementation/testing/etc.
• SAS device verification review is over, will be merged once we have the tagged OLM library.

🔗RiotX (Android)

• Valere has started working on reactions (which will be implemented only on RiotX for Android).
• François has worked on merging membership Events in the timeline

🔗Riot iOS

• Interactive device verification has been merged
• Jitsi and WebRTC updates have been merged
• Fix bugs in order to prepare a release

🔗Crypto

uhoreg reports that:

olm 3.1.0 has been released. This release adds new functions to help with SAS-based key verification (a.k.a. emoji-based verification) and with cross-signing. The Python bindings are also now available on pypi, so you can install it using "pip install python-olm", though you need the olm library and development files installed first.

Also

• Python-olm available on pypi.
• Initial SAS verification support for nio.

🔗Spectral

Black Hat reports that:

Spectral's redesign continues, featuring a beautiful responsive UI(not kirigami yet, sorry) and more functionalities. Legacy UIs such as the room detail panel are changed to fit into redesigned UI better. Basic room upgrade support is added, allowing you to switch between the old room and the new room. Room settings and user detail dialogs are added. You can also ignore users in the user detail dialog.

🔗matrix-nsfw

A Black Hat double header this week:

matrix-nsfw has been ported from Golang to Rust. The backend machine learning framework is also switched to Tensorflow, giving a major performance boost. For anyone that doesn't know what matrix-nsfw is, it is a bot-like utility that detects NSFW images in a room. The new repo is at https://gitlab.com/b0/matrix-nsfw-rust

🔗Neo

Sometimes a picture tells a thousands rainbow coloured words. Thanks Foks

🔗Ruma

Jimmy reports that:

All of Ruma's libraries (but not yet the homeserver itself) are now targeting stable Rust!

🔗Continiuum

yuforia reports that:

• If a room member is not visible on screen, updating their name doesn't require switching to the main UI thread

• Apply formatting when viewing the json source of an event
• Reuse GUI components to improve performance, update content of views instead of creating new ones
• Use a hash set to avoid going through the list of room members in some cases * Move more of local storage into the database: names and avatars of users and rooms, room membership, recently used accounts, etc.
• Placeholder avatars are made with GUI components and instead of generated bitmap images
• Switch to gradle multi-project build to modularize

After switching from plaintext files to an embedded database, some components are still in the process of being rewritten, coming next week: load messages from server on demand when scrolling, if they are not yet stored in database; add support for invitations;

🔗matrix-appservice-discord

Sorunome reports that:

matrix-appservice-discord work has finally resumed! The PRs for both migrating the room and user store to SQL have been merged, and many awesome new things should follow up soon!

🔗Synapse

The reactions and edits API is taking shape, we’re making progress on our small homeserver setup, and we’re hunting a new set of device key management bugs that came to light in the absence of matrix.org.

We’ve been a bit disrupted these past few weeks, but work towards Synapse 1.0 continues and we’ll soon be ready to offer a release candidate.

🔗miniVector

Curious Cat reports that: >miniVector 0.8.29 is available on Goldy and Froid (and are unaffected by the Riot/Android re-release drama).

🔗The 'Incident'

We know you'll have a bunch of questions, we'll be publishing a full post-mortem next week. Thanks for bearing with us.

🔗The End

See you next week, it will be Bentastic I can assure you. Be sure to stop by #twim:matrix.org with your updates!

🔗Overview

We became aware today of a flaw in sydent’s validation of email addresses which can lead to a failure to correctly limit registration to a given email domain. This only affects people who run their own sydent, and are relying on allowed_local_3pid in their synapse config. We’d like to thank @fs0c131y for bringing it to our attention on Twitter this morning. We are not aware of this being exploited in the wild other than the initial report.

If you are running your own sydent, and limiting signup for your server using the allowed_local_3pids configuration option, then you need to upgrade your sydent immediately to Sydent 1.0.2.

Meanwhile, if you have been relying on the allowed_local_3pids configuration option to restrict access to your homeserver, you may wish to check your homeserver’s user_threepids table for malformed email addresses and your sydent’s database as follows:

$ sqlite3 sydent.db 
sqlite> select count(*) from global_threepid_associations where address like '%@%@%';
0

$ psql matrix
matrix=> select count(*) from user_threepids where address like '%@%@%';
 count 
-------
     0

If the queries return more than 0 results, please let us know at [email protected] - otherwise you are fine.

🔗Details

A flaw existed in sydent whereby it was possible to bypass the requirement specified in synapse’s allowed_local_3pids option, which restricts that users may only register with an email address matching a specific format.

This relied on two things:

1. sydent uses python's email.utils.parseaddr function to parse the input email address before sending validation mail to it, but it turns out that if you hand parseaddr an malformed email address of form [email protected]@c.com, it silently discards the @c.com prefix without error. The result of this is that if one requested a validation token for '[email protected]@important.com', the token would be sent to '[email protected]', but the address '[email protected]@important.com' would be marked as validated. This release fixes this behaviour by asserting that the parsed email address is the same as the input email address.
2. synapse's checking of email addresses relies on regular expressions in the home server configuration file. synapse does not validate email addresses before checking them against these regular expressions, so naive regular expressions will detect the second domain in email addresses such as the above, causing them to pass the check.

You can get sydent 1.0.2 from https://github.com/matrix-org/sydent/releases/tag/v1.0.2.

🔗Welcome to the new blog!

Check out the new digs! We're happy with this newly deployed blog, and all the old and loveable content is right down there. If you find issues, let me know. You may remember Nad, from previous editions of Matrix Live - huge thanks to him for his work on the design and upkeep of this new deployment.

🔗Notes from the downtimes

🔗Github tokens

jaywink:

Due to the security incident, all GitHub access tokens for the Scalar GitHub integration were cleared. This means that if you have a GitHub bot in the channel and want to use the !github bot commands, you need to re-login to github via the integration manager menu. Note, existing webhooks are untouched and should work fine without re-authenticating.

🔗Bridges

Half-Shot:

From the matrix.org bridge team, we are resurrecting bridges as fast as possible. Currently running are the freenode, slack, gitter and gimpnet (now hosted on gnome.org) with more to come today and next week.
We have the snoonet and oftc irc bridges back. Mozilla is coming soon hopefully this weekend too!

Whoop! Mozilla is actually up now!

🔗Pattle new release on F-Droid

Wilko:

A big release of Pattle has just been pushed to the F-droid repo! Changes include:

• Display names are now shown
• You can now click on chats and view them!
• Messages are grouped by time and sender (see screenshots)
• Add fancy transition animation and ripples to chat messages (see video)
• Use Sentry for error reporting (only Android version and device model is sent, along with the stacktrace of the error)

Also, please note that if you have a matrix.org you probably have reinstall the app if you're logged in because of the recent matrix.org incident (because there's no logout button yet and no detection for invalidated access tokens)

There has actually been a release since, which includes:

message sending and viewing image history!

🔗libQMatrixClient 0.5.1.2 && Quaternion 0.0.9.4

kitsune is talking about the long road to 0.0.9.4:

libQMatrixClient 0.5.1.2 has been released, with all the remaining bugfixes for Quaternion 0.0.9.4 that's coming any day soon now. The release notes are here: https://github.com/QMatrixClient/libqmatrixclient/releases/tag/0.5.1.2

Quaternion 0.0.9.4 RC3, the last one before the release that will happen in the nearest days, is out. Release notes can be found at https://github.com/QMatrixClient/Quaternion/releases/tag/0.0.9.4-rc3. Translators, you literally have hours to add your translations for 0.0.9.4!

🔗neo

I reimplemented the matrix sdk into Neo, so it works nicer. Colors and font look nicer (base16-tomorrow, Open Sans), and there's text message sending, with localecho!
I also fixed a bug where react would recycle displayname-components across rooms, attributing them to the wrong messages

a video at https://lain.haus/_matrix/media/v1/download/lain.haus/VfshWRfaNUnpGQbdkyYczxvd

Go test Neo at: https://neo.lain.haus/neo

🔗matrix-registration update

ZerataX:

it's been a long while, but I've finally come around to improving on matrix-registration
For those of you, who have forgotten what this project is about, it basically lets you invite people to your homeserver with tokens, e.g. https://homeserver.tld/register?token=DoubleWizardSky
This whole update was about making the project more user friendly.
I made a new default registration page that requires 0 setup and you can install the project right from pypi with pip, so you don't even need to clone the repo any longer.
check a live example here: https://chat.dmnd.sh/register
and to play around with the api you can can go over to the github page: https://zeratax.github.io/matrix-registration/demo.html?token=ColorWhiskeyExpand
channel: #matrix-registration:dmnd.sh
github: https://github.com/zeratax/matrix-registration

Video available here.

🔗matrix-media-repo now has s3 support

TravisR:

matrix-media-repo now has s3 (and s3-like) support, making it easier to archive older media or use minimal disk space. See the new datastores option in the config and the admin docs ( https://github.com/turt2live/matrix-media-repo/blob/master/docs/admin.md#datastore-management ) for more information.

🔗Dimension

TravisR:

Dimension has been updated to more safely handle when upstream integration managers (like Scalar) are offline. Instead of crashing or breaking in various ways, it'll report which integrations are not accessible.
As well, due to recent events, if you use matrix.org bots or bridges in Dimension then go the the admin section and log everyone out using the red button. Dimension caches upstream tokens and isn't smart enough to realize that they are no longer valid, which means they need clearing. Clients should automatically handle getting new tokens in the background.

🔗Riot iOS

• Interactive device verification is still in progress
• Update WebRTC and Jitsi libs (in progress)

🔗Riot Android

*We are finalizing SAS device verification feature *We are preparing a release to fix some issues

🔗RiotX (Android)

• We can now clear the cache of the application (Riot legacy feature)
• Sync filter has been implemented
• François is working on thumbnail of video (upload and preview) in the timeline

🔗That's all I know

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Update: for the full story here, please see the post mortem.

🔗Here's what you need to know.

TL;DR: An attacker gained access to the servers hosting Matrix.org. The intruder had access to the production databases, potentially giving them access to unencrypted message data, password hashes and access tokens. As a precaution, if you're a matrix.org user you should change your password now.

The matrix.org homeserver has been rebuilt and is running securely; bridges and other ancillary services (e.g. this blog) will follow as soon as possible. Modular.im homeservers have not been affected by this outage.

🔗The security breach is not a Matrix issue.

The hacker exploited a vulnerability in our production infrastructure (specifically a slightly outdated version of Jenkins). Homeservers other than matrix.org are unaffected.

🔗How does this affect me?

We have invalidated all of the active access tokens for users on Matrix.org - all users have been logged out.

Users with Matrix.org accounts should:

• Change your password now - no plaintext Matrix passwords were leaked, but weak passwords could still be cracked from the hashed passwords
• Change your NickServ password (if you're using IRC bridging) - there's no evidence bridge credentials were compromised, but if you have given the IRC bridges credentials to your NickServ account we would recommend changing this password

And as a reminder, it's good practice to:

• Review your device list regularly - make sure you recognise all of the devices connected to your account
• Always make sure you enable E2E encryption for private conversations

🔗What user data has been accessed?

Forensics are ongoing; so far we've found no evidence of large quantities of data being downloaded. The attacker did have access to the production database, so unencrypted content (including private messages, password hashes and access tokens) may be compromised.

🔗What has not been affected?

• Source code and packages have not been impacted based on our initial investigations. However, we will be replacing signing keys as a precaution.
• Modular.im servers are not affected, based on our initial analysis
• Identity server data does not appear to have been compromised

The target appeared to be internal credentials for onward exploits, not end user information from the matrix.org homeserver.

🔗You might have lost access to your encrypted messages.

As we had to log out all users from matrix.org, if you do not have backups of your encryption keys you will not be able to read your encrypted conversation history. However, if you use server-side encryption key backup (the default in Riot these days) or take manual key backups, you’ll be okay.

This was a difficult choice to make. We weighed the risk of some users losing access to encrypted messages against that of all users' accounts being vulnerable to hijack via the compromised access tokens. We hope you can see why we made the decision to prioritise account integrity over access to encrypted messages, but we're sorry for the inconvenience this may have caused.

🔗What happened?

We were using Jenkins for continuous integration (automatically testing our software). The version of Jenkins we were using had a vulnerability (CVE-2019-1003000, CVE-2019-1003001, CVE-2019-1003002) which allowed an attacker to hijack credentials (forwarded ssh keys), giving access to our production infrastructure. Thanks to @jaikeysarraf for drawing this to our attention.

🔗Timeline

March 13th Updated 2019-04-12 11:00 UTC

• Attacker compromises Jenkins CI server

April 4th Updated 2019-04-12 11:00 UTC

• Attacker gains access to production infrastructure by hijacking a forwarded SSH agent logging into the compromised Jenkins worker

April 9th

• Jenkins vulnerability brought to our attention by @jaikeysarraf

April 10th

• Investigation identified the compromised machines and the full scope of the attack
• Jenkins was removed
• Attacker's access to compromised machines was removed

April 11th

• Matrix.org was taken offline and production infrastructure fully rebuilt
• Having fully flushed out the attacker, external communication was published informing users and advising on next steps
• Matrix.org homeserver restored, with bridges and ancillary services (e.g. this blog) following as soon as possible

🔗Update 2019-04-12

At around 5am UTC on Apr 12, the attacker used a cloudflare API key to repoint DNS for matrix.org to a defacement website (https://github.com/matrixnotorg/matrixnotorg.github.io). The API key was known compromised in the original attack, and during the rebuild the key was theoretically replaced. However, unfortunately only personal keys were rotated, enabling the defacement. We are currently doublechecking that all compromised secrets have been rotated.

The rebuilt infrastructure itself is secure, however, and the DNS issue has been solved without further abuse. If you have already changed your password, you do not need to do so again.

The defacement confirms that encrypted password hashes were exfiltrated from the production database, so it is even more important for everyone to change their password. We will shortly be messaging and emailing all users to announce the breach and advise them to change their passwords. We will also look at ways of non-destructively forcing a password reset at next login.

The attacker has also posted github issues detailing some of their actions and suggested remediations at https://github.com/matrix-org/matrix.org/issues/created_by/matrixnotorg.

This confirms that GPG keys used for signing packages were compromised. These keys are used for signing the synapse debian repository (AD0592FE47F0DF61), and releases of Riot/Web (E019645248E8F4A1). Both keys have now been revoked. The window of compromise for the keys started from April 4th; there have been no Synapse releases since then. There has been one release of Riot/Web (1.0.7), however as the key was passphrased and based on our initial analysis of the release, we believe it to be secure.

🔗What are we doing to prevent this in future?

Once things are back up and running we will retrospect on this incident in detail to identify the changes we need to make. We will provide a proper postmortem, including follow-up steps; meanwhile we are obviously going to take measures to improve the security of our production infrastructure, including patching services more aggressively and more regular vulnerability scans.

TL;DR DON'T PANIC - Synapse 1.0 will support Postgres 9.4 and Python 2.7

Folks, this is an update to explain that we will be shortly deprecating Synapse support for Postgres 9.4 and Python 2.x.

🔗What are we doing?

From the dates described below, we will no longer guarantee support for deprecated versions. This means that Synapse may continue to work with these versions but we will not make any attempt to ensure compatibility and will remove old library versions from our CI.

🔗When is this happening?

Synapse 1.0 will continue to support both technologies, but subsequent releases may not:-

For Python, we shared that we would discontinue to Python 2.x support from April 1st 2019, so for the first release that follows 1.0 we do not guarantee Python 2.x support.

For Postgres, will give server admins 6 weeks to upgrade to a newer version, and will guarantee support up until 20th May 2019.

🔗Why would you do this to us?

We have multiple reasons, but broadly:-

• We want to make use of new language features not supported in old versions. This will enable us to continue to improve the performance and maintainability of Synapse.
• Python 2.x overall will be end of life'd at the end of the year. Postgres 9.4's final release will follow 2 months later on 13th February 2020.
• Since very few server admins still use these technologies on the wild, providing support is costly and we want to reduce our overall maintenance load.

🔗La la la I am ignoring you - what will happen?

You will be able to upgrade to Synapse 1.0, but will likely experience incompatibilities that prevent you upgrading further. Seriously, you really need to upgrade.

🔗Okay, but I have questions, where should I go?

Come and say Hi in #synapse:matrix.org and we'll do our best to help you.

🔗Matrix Live S3E20 FluffyChat creator Krille on Matrix, Ubuntu Touch and Matrix-for-Healthcare

🔗Synapse, the road to 1.0

Neil, Synapse overlord:

Last week we carried on with Synapse 1.0 work, in particular server key signing validity, a config option to verify federation certificates to support MSC1711 and support for 3PID unbinding APIs (MSC1915). Outside of that we've been thinking about how to improve room upgrades for private rooms and landed the preparatory work for a refactor of the room directory.

Coming up next, we'll continue to bash out 1.0 blockers, Hawkowl has started work on optimising for small hosted homeservers, and anoa will be working on the new super fast room directory. Finally Erik has started work on aggregations support so clients will be able to offer things like edits and reactions ?

https://www.arewereadyyet.com/ is now just shy of 64%, keep going.

🔗Dendrite

Dockerfile for sytest running against Dendrite approved. Close to being able to run Dendrite against sytest. Also using sytest's new test white/blacklist functionality to include a list of tests that Dendrite is known to pass in the repo. When people make new PRs that allow Dendrite to pass new tests, they can also append the names of the tests to the testfile to help automatically track Dendrite's improving progress. Look forward to seeing further progress post Synapse 1.0.

🔗libQMatrixClient 0.5.1 released,

kitsune:

libQMatrixClient 0.5.1 is out! This is a minor release with bugfixes and small improvements, a base for Quaternion 0.0.9.4, RC of which is coming this weekend. Full release notes here

and to follow up:

Quaternion 0.0.9.4 Release Candidate is available for all those who don't need a "release" seal of approval to use applications. Get out, grab it, report bugs that could still sneak in: https://github.com/QMatrixClient/Quaternion/releases/tag/0.0.9.4-rc. A separate call out to translators - it's a great moment to teach Quaternion a few more phrases in your language!

🔗.NET Core Synchrotron

Half-Shot:

This weekend I took a brief pause from doing matrix work, and then went back into it by finishing the feature set for the .NET Core Synchrotron. It's "feature complete" meaning all specced features of sync are in this branch, excluding legacy endpoints. There are a few bugs logged against it and it's really only useful as a super early test, while I fix some of the p1 bugs.

The federation sender continues to be reliable and stable, and we are prepping for the release of synapse 0.99.3 which made a few changes to the replication stream. For more info, please see https://github.com/turt2live/synapse-netcore-workers

🔗Pattle has now moved to Flutter

Wilko has officially discontinued the previous version of Pattle, and is focused on their Flutter SDK and new version of Pattle based on it:

After some silence here but a lot of development, Pattle has now moved to Flutter. The new app is available in the same F-droid repository, you can just update. Note that you will have to log in again. If there are issues with updating, try removing the old app and installing the new one.

Also note that this version is still very basic and does not have all features of the old Pattle unfortunately. The reason that I chose to already replace it on the repo is that the old app will not receive updates anymore.

The current features are logging in and an overview of all chats. What is new is that data is now stored locally, so you won't have to do an initial sync everytime you open the app (which was the case with old app). What's also new is that I've decided to use more red, because.. I like red.

There is still a lot to do! Please notify me of anything that you think is missing, even though some things may be obvious (sorting chats based on date, chat avatars), some may not be as obvious to me!

If you'd like to contribute, the new repo is here. Note that many contributions will probably start at the SDK level. Pattle uses my own Dart SDK, which you can find here.

There is at the moment a very specific bug in Flutter, where on Android 7.1 with release builds, the app crashes when building a list widget. If you use Android 7.1 (like me) and you crash after logging in, that's the reason. It seems I can't do anything about it, sorry.

🔗libaqueous (Matrix Dart SDK) and Aqueous client

Black Hat is continuing work on libaqueous, a Matrix Dart SDK.

Aqueous now comes with room categories and history messages support. Also, a lot of code refactoring is going on to change the backend store from plain json file to sqlite, which should improve the performance a lot. The room is at #libaqueous:encom.eu.org

🔗Riot Web

• Working towards bringing breadcrumbs out of labs
• Adding image preview to file upload UX
• Stability fixes for timeline
• Defending against issues introduced when using Riot Web on a machine which is running out of disk space

🔗Riot iOS

• Grouped notifications are coming! We have added some logs to track why the notification setting is sometimes disables (#2348).
• SAS device verification is still in progress but it should be released next week.

🔗Riot Android

• Three releases this week, to fix several bugs with the new EventStreamServiceX, and to fix issue with the Jitsi conference call (Riot.im 0.8.29). FDroid users should receive the upgrade soon.
• Users are happy with the new notifications!
• Valere is still working on SAS, we are reaching the end :).

🔗RiotX (Android)

• It's now possible to signout from RiotX.
• The whole setting screen has been imported from Riot, as well as the Themes and some other small features (launcher icon, etc.).
• Benoit is working on room creation, sdk side, and François is working on image upload.
• We should start to integrate new design from Nad next week.

🔗continuum (JavaFX desktop client)

yuforia has continued work on their client:

in continuum, moving storage of messages from plain text files to a database, simplifying code and improving performance

🔗neo

f0x:

Neo now has .well-known implemented for login :D
video showcasing the different outcomes:

🔗Synapse K8s images

Ananace has updated his k8s images for Synapse 0.99.3, and has been continuing work on other parts of the project:

I'll be writing a Helm chart for 1.0 as well, just haven't had time to do so yet.
I need to update all the references to TLS as well, as that's supposed to be left to kube-lego or cert-manager for a modern install

🔗Federation Tester

Brendan and anoa have been working on the Federation Tester:

Following up on last week's effort, a number of issues on the Matrix federation tester, both on the backend and the frontend side, have been fixed, including fixing certificate validity checks, not following CNAME records in SRV records, fixing display of the SRV lookup in the UI, preventing some crashes, and a few more.

All outstanding issues on both the federation tester's backend and its frontend that could have preventing people to test efficiently how their server is performing in the context of Synapse's 1.0 release have now been fixed and deployed on https://matrix.org/federationtester/.

🔗Synapse and Riot bounties

Aaron Raimist reports:

A few bounties have been added to Synapse and Riot recently to encourage community members to work on those projects. Although they aren't large enough to actually pay for development, I believe they can serve as a small push to help an already interested developer justify spending some time working on these projects.

• $400 bounty on threading in Riot thanks to GitHub user @fallerOfFalls, expires November 21, 2019
• $200 bounty to fixing stuck invites, expires on January 1, 2020
• $200 bounty to fix Riot forgetting the name of rooms, AKA the "Aaron and 5246 others" bug, expires on January 1, 2020

🔗Still more

pantalaimon has been seen in the wild, even appearing to run on Android. We'll get a little closer to the project and see how it's going in a future edition.

jj announced a new Python SDK for Matrix:

new, alpha quality, matrix python client for asyncio and with support for e2e encryption: https://github.com/SFTtech/aiomatrix
Polishing is needed but the basics do work already :)

L'autre from Calculate Linux announced that the project is now using Matrix for their communications, you can find their rooms in their newly created community: +list:calculate-linux.org.

George Lucas György Lukács Georg Lukas brutally skewered Matrix with an April Fool's post suggesting that yaxim, an XMPP client for Android, will begin to support Matrix.

Matrix listed as one of the "Top Ten Tools" for Open Science practitioners

the whole point of Matrix is that its a protocol which many different suppliers can use so you can avoid nasty lock ins

That is certainly an advantage!

🔗That's all I know

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Hey hey, a Synapse release for you today.

The big news in 0.99.3 is that the user directory has been completely re-written and should now be much more performant - this will benefit all installations, but especially those housing larger servers.

Aside from that we continue our 1.0 preparations and relatedly we've improved our docs, in particular to explain how .well-known works. On the perf side we've added rate limiting to login and register endpoints as well as now batching up read receipts to send over federation.

I've said it before, and I'll say it again:-

The most important thing that admins should know is that prior to 1.0 landing later this month, it is essential that the federation API has a valid TLS certificate - self signed certificates will no longer be accepted. For more details see our handy guide. Failure to do this will result in being unable to federate with other 1.0 servers.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. Also, check out our new Synapse installation guide page.

🔗Synapse 0.99.3 changelog

🔗Features

• The user directory has been rewritten to make it faster, with less chance of falling behind on a large server. (#4537, #4846, #4864, #4887, #4900, #4944)
• Add configurable rate limiting to the /register endpoint. (#4735, #4804)
• Move server key queries to federation reader. (#4757)
• Add support for /account/3pid REST endpoint to client_reader worker. (#4759)
• Add an endpoint to the admin API for querying the server version. Contributed by Joseph Weston. (#4772)
• Include a default configuration file in the 'docs' directory. (#4791, #4801)
• Synapse is now permissive about trailing slashes on some of its federation endpoints, allowing zero or more to be present. (#4793)
• Add support for /keys/query and /keys/changes REST endpoints to client_reader worker. (#4796)
• Add checks to incoming events over federation for events evading auth (aka "soft fail"). (#4814)
• Add configurable rate limiting to the /login endpoint. (#4821, #4865)
• Remove trailing slashes from certain outbound federation requests. Retry if receiving a 404. Context: #3622. (#4840)
• Allow passing --daemonize flags to workers in the same way as with master. (#4853)
• Batch up outgoing read-receipts to reduce federation traffic. (#4890, #4927)
• Add option to disable searching the user directory. (#4895)
• Add option to disable searching of local and remote public room lists. (#4896)
• Add ability for password providers to login/register a user via 3PID (email, phone). (#4931)

🔗Bugfixes

• Fix a bug where media with spaces in the name would get a corrupted name. (#2090)
• Fix attempting to paginate in rooms where server cannot see any events, to avoid unnecessarily pulling in lots of redacted events. (#4699)
• 'event_id' is now a required parameter in federated state requests, as per the matrix spec. (#4740)
• Fix tightloop over connecting to replication server. (#4749)
• Fix parsing of Content-Disposition headers on remote media requests and URL previews. (#4763)
• Fix incorrect log about not persisting duplicate state event. (#4776)
• Fix v4v6 option in HAProxy example config. Contributed by Flakebi. (#4790)
• Handle batch updates in worker replication protocol. (#4792)
• Fix bug where we didn't correctly throttle sending of USER_IP commands over replication. (#4818)
• Fix potential race in handling missing updates in device list updates. (#4829)
• Fix bug where synapse expected an un-specced prev_state field on state events. (#4837)
• Transfer a user's notification settings (push rules) on room upgrade. (#4838)
• fix test_auto_create_auto_join_where_no_consent. (#4886)
• Fix a bug where hs_disabled_message was sometimes not correctly enforced. (#4888)
• Fix bug in shutdown room admin API where it would fail if a user in the room hadn't consented to the privacy policy. (#4904)
• Fix bug where blocked world-readable rooms were still peekable. (#4908)

🔗Internal Changes

• Add a systemd setup that supports synapse workers. Contributed by Luca Corbatto. (#4662)
• Change from TravisCI to Buildkite for CI. (#4752)
• When presence is disabled don't send over replication. (#4757)
• Minor docstring fixes for MatrixFederationAgent. (#4765)
• Optimise EDU transmission for the federation_sender worker. (#4770)
• Update test_typing to use HomeserverTestCase. (#4771)
• Update URLs for riot.im icons and logos in the default notification templates. (#4779)
• Removed unnecessary $ from some federation endpoint path regexes. (#4794)
• Remove link to deleted title in README. (#4795)
• Clean up read-receipt handling. (#4797)
• Add some debug about processing read receipts. (#4798)
• Clean up some replication code. (#4799)
• Add some docstrings. (#4815)
• Add debug logger to try and track down #4422. (#4816)
• Make shutdown API send explanation message to room after users have been forced joined. (#4817)
• Update example_log_config.yaml. (#4820)
• Document the generate option for the docker image. (#4824)
• Fix check-newsfragment for debian-only changes. (#4825)
• Add some debug logging for device list updates to help with #4828. (#4828)
• Improve federation documentation, specifically .well-known support. Many thanks to @vaab. (#4832)
• Disable captcha registration by default in unit tests. (#4839)
• Add stuff back to the .gitignore. (#4843)
• Clarify what registration_shared_secret allows for. (#4844)
• Correctly log expected errors when fetching server keys. (#4847)
• Update install docs to explicitly state a full-chain (not just the top-level) TLS certificate must be provided to Synapse. This caused some people's Synapse ports to appear correct in a browser but still (rightfully so) upset the federation tester. (#4849)
• Move client read-receipt processing to federation sender worker. (#4852)
• Refactor federation TransactionQueue. (#4855)
• Comment out most options in the generated config. (#4863)
• Fix yaml library warnings by using safe_load. (#4869)
• Update Apache setup to remove location syntax. Thanks to @cwmke! (#4870)
• Reinstate test case that runs unit tests against oldest supported dependencies. (#4879)
• Update link to federation docs. (#4881)
• fix test_auto_create_auto_join_where_no_consent. (#4886)
• Use a regular HomeServerConfig object for unit tests rater than a Mock. (#4889)
• Add some notes about tuning postgres for larger deployments. (#4895)
• Add a config option for torture-testing worker replication. (#4902)
• Log requests which are simulated by the unit tests. (#4905)
• Allow newsfragments to end with exclamation marks. Exciting! (#4912)
• Refactor some more tests to use HomeserverTestCase. (#4913)
• Refactor out the state deltas portion of the user directory store and handler. (#4917)
• Fix nginx example in ACME doc. (#4923)
• Use an explicit dbname for postgres connections in the tests. (#4928)
• Fix ClientReplicationStreamProtocol.__str__(). (#4929)

🔗Matrix Live

Tom & Matthew talk through the newly published Riot/Web roadmap! Be sure to read Tom's blog post which gets into his thinking for the roadmap too.

🔗Matrix.org homeserver notice

The matrix.org homeserver will be down for essential database maintenance between 12:00 and 12:30 UTC on 2019-04-01

🔗Synapse

Neil, Synapse overseer, reports on the acceleration towards 1.0:

We have much improved sytest support for Synapse worker mode, Hawkowl landed a brand new super fast super shiny user directory implementation, Brendan fixed a bunch of niggles in the federation tester and we finally got to the bottom of our sync caching bug that obscured accepting invites (thanks richvdh for sticking with it). Erik solved our presence spamming bug and anoa fixed some LDAP auth bugs and submitted a PR for certificate checking - one of our final blockers for Synapse 1.0. For all the latest checkout out our latest release candidate 0.99.3rc1.
Next week, we'll ship 0.99.3, we'll be looking at server key validity periods, adding a 3PID unbind API and starting work on tuning low powered Synapse installations.
https://www.arewereadyyet.com/ is still rising so keep banging to everyone you know that they ensure their federation certificates are valid.

🔗Pantalaimon

poljar gave this update on his new project Pantalaimon:

Pantalaimon is a new project that acts as a reverse proxy for clients to connect to. Once a client is connected Pantalaimon handles end to end encryption for the client transparently.
The project is only a week old but already at a working prototype phase. A demo can be found here: http://webmshare.com/play/Qn4wg

This is a huge gain! Use of this project, as you will see in the video, permits any Matrix client to support End-to-End encryption of messages, by handling encryption/decryption in a daemon rather than in the client.

Watch the video, and you'll see gomuks, which does not support encryption, chatting in an encrypted room. Also, go chat in #pantalaimon:matrix.org and get involved!

🔗Construct

Report from the Construct team:

The matrix Construct server has made significant progress this week implementing the 1.0 specification and is very close to a 1.0 release! Special thanks to our star tester Yan Minari and expert Matrix consultants Tulir and Max and C++ developers Jason of Zemos and Mujx of nheko for all their hard work to make this happen. The Construct now fully supports IPv6 and is ready to participate in fully end-to-to-end encrypted ipv6 networks like #yggdrasil:matrix.org and #cjdns:matrix.org.
This week also saw additions for .well-known support, m.fully_read and even DNS caching inside a matrix room shared between servers (which is really cool if you ask me).

I'd like to recognize our testers for their invaluable contributions this week, only a fraction of which are documented here: https://github.com/matrix-construct/construct/issues leading to the 1.0 release.

The Construct is written in C++ for maximum performance and scalability. It is the first fully federating server after the Synapse reference implementation. Your contributions in code and participation are essential to bring Construct to its upcoming release; get involved at https://github.com/matrix-construct/construct and in #zemos-test:matrix.org.

🔗Riot Web

From the team:

We've been upgrading and optimising our Jitsi instance so people should see more reliable video conference calls, especially if they avoid connecting from Firefox over poor connections.
We've been squashing scroll jumps (where the timeline pops out of position unexpectedly due to images loading, etc.). We've come up with a radical reimplementation of the timeline (which should be imperceptible, except it doesn't jump) - try it out on https://riot.im/develop now.

Bruno, who has been working on scroll functional for Riot deserves a call-out - scrolling in new Riot web works great, and he may now be the most qualified dev on scroll: anchor outside of Mozilla or Google…

You can now get all Riot updates from #riot-web-announcements:matrix.org, the official room of Riot web announcements! You can especially get this update, which was also featured on Twitter:

Attention Riot Web Admins! We reset Scalar tokens to address a potential security vuln. with some clients - if you run your own Riot instance please upgrade to at least v1.0.4 to keep using integrations (widgets, sticker picker, any bots and bridges configured through Scalar).

— Riot.im (@RiotChat) March 27, 2019

Please upgrade your Riot web instances!

Additionally, Notkea has packaged Riot desktop for NixOS.

🔗Riot iOS

Another case of UTD (https://github.com/vector-im/riot-ios/issues/2320) has been fixed. Some logs have been added to track push notifications that disabled themselves. SAS verification implementation is still in progress.

UTD = "Unable to Decrypt", messages as seen in Riot

🔗Riot Android

• Rework of EventStreamService, to fix many issues (crash) reported by users. The feature is available on the develop branch.
• Also we are trying to upgrade Jitsi library, to fix other errors reported by users.
• SAS verification implementation is still in progress.

Also, you may have already seen the use of Android 9-style notifications, featuring "Mark as read" + "Quick reply" buttons. This addition has started to make Riot Android my client of choice for burning through notifying rooms.

🔗RiotX (Android)

• Phase 1 started!
• Backporting RageShake feature, with better handshake detection.
• We are still fighting bug on the timeline rendering.

🔗nheko packaging

@adridg:matrix.org:

Quick note that nheko (v 0.6.3) is now packaged for FreeBSD as well. The C++ code was fine (we use Clang, that does trip up some people) but I have Opinions about the CMake code (in particular that the find code for nlohmann/json.hpp, lmdb++.h and tweeny.h could use a lot of work -- if I feel perky I may come up with a PR). Thanks for the good work!

🔗Dart SDKs

Black Hat has been using Dart and Flutter this week, and is making progress with an SDK and reference client:

I am trying to use libaqueous(Matrix Dart SDK) to create a cross-platform matrix client for Android and iOS.
Almost nothing works except Already a significant number of client features are available: login/logout, room list, basic timeline and message posting

Flutter is quite good for rapidly building mobile apps. ? Strong-typed json serialization/deserialization in Dart is a bit difficult at first, but I managed to solve that.
the repo is here, the SDK is here, and the room for discussion is at #libaqueous:encom.eu.org

Debug build apk from Gitlab CI is here

🔗Notes on manually upgrading rooms, from TravisR

TravisR has written a brief guide to manually upgrading rooms:

https://gist.github.com/turt2live/a99c8e794d6115d4ddfaadb72aabf063

Note that you generally won't need to do this.

🔗That's all I know

See you next week, and be sure to stop by #twim:matrix.org with your updates!

🔗Matrix Live with Kitsune

I chatted with Kitsune, maintainer of libQMatrixClient, Quaternion and Spec Core Team member. We talked about the history and future of these projects, platform preferences, the importance of decentralisation and more.

🔗Synapse and the road to 1.0

Neil, Synapse-dev wrangler #1:

Huge thanks to everyone who has helped increase the number of ‘1.0 ready' synapse installs. If you don't know what this means, see our blog. https://arewereadyyet.com reports > 60% adoption on a per server basis, and high 90s on a per user basis. We are now really close to being able to ship a 1.0 release candidate and start the 2 week countdown before releasing 1.0 proper.

This week we have focussed on performance, richvdh has been working on batching of outgoing read receipts and hawkowl shipped a much more performant implementation of user search. Erik has been putting the state compressor through its paces, he saw one room compress down to 1% of its original size. Andrew has been focusing on ensuring spec compliance on various Synapse endpoints and is currently looking at some bugs in the federation tester.

🔗NixOS-Synapse setup

Jonas Schürmann shared his config for Synapse on NixOS:

I open sourced my NixOS-Synapse setup: https://gitlab.com/MazeChaZer/nixops-servers/tree/master/matrix

#NixOS:matrix.org for more.

🔗Pattle, lots of progress and Dart SDK planned

Lots of news from Pattle, the project aiming to create clients for multiple mobile platforms. All quotes below from Wilko.

Pattle for Android now has an F-droid repository: https://fdroid.pattle.im/

It's the most up to date version straight from the pipeline, so not necessarily stable.

You can now scroll up and load history

As part of the project, Wilko also announced the intention to create a Dart SDK:

I'm currently working on a Matrix SDK for Dart, which will be used for the Pattle rewrite in Flutter. The reason for this is outlined here. To summarize: My goal for Pattle is to release for not only Android but also iOS and web, with a single code base. Other reasons include that Android native development is just bad, and the fact that I'm not too happy with the official Matrix Android SDK design and documentation. I will maintain and also add some features to the current version of Pattle, to try things out. The Flutter version will replace the native one once the features are in sync.

🔗Riot iOS

We've released a hotfix release 0.8.4 on App Store that fixes following issues:

• Unable to open a file attachment of a room message.
• Unable to share a file with whitespace in filename.

We are working on iOS 12 notifications features.

🔗Riot Android

• Improving new notification system
• Implementing SAS verification

🔗RiotX (Android)

• Setup CI
• Finishing Phase 0

🔗Dimension sticker packs

TravisR has introduced sticker packs for Dimension:

Dimension now has beta support for custom sticker packs. To create a sticker pack:

1. Start a conversation with @stickers:t2bot.io
2. Say !stickers newpack and follow the prompts.
3. Share the URL with your friends.

To add the sticker pack to your sticker picker:

1. Make sure you're using Dimension as your integration manager.
   • Not sure if you are? Click the 4 squares in the top right of the room in Riot - if the dialog is titled "Dimension", you're all set :)
   • If you aren't, check out https://dimension.t2bot.io
2. Click on the smiley face to the right of where you type messages then the Edit button in the top right.
3. Paste the URL of the sticker pack into the box and click "Add Stickerpack".
4. Start using your new stickers.

These instructions are also available at https://github.com/turt2live/matrix-dimension/blob/master/docs/custom_stickerpacks.md as is the admin/operator guide for running your own sticker bot (you're not stuck with using t2bot.io unless you want to be).
Custom sticker packs are still beta while the proposals to share this with the wider Matrix ecosystem are still works in progress. This serves as a proof of concept to see how crazy of an idea it is to have stickerpacks-as-rooms (yes, they're just plain Matrix rooms under the hood) and what needs ironing out before moving ahead with the MSC.

🔗matrix-notification-profile-manager

Brendan has created a notification profile manager:

Over the past few weeks, I grew a bit fed up of always having to turn on and off every notification rule each time I'm having a slight change in what I'm working on or depending on what mood I was in (e.g. want to focus only on work-related stuff and nothing else, don't want to hear about work at all, somewhere in the middle, etc.), so I built a notification profile manager for Matrix. It's available both as a command line interface or a Go package in case people want to build on top of it.

It allows one to take a snapshot of their current notifications settings and save that as a profile, so that this profile can be applied later. It also allows one to delete a profile or list the existing profiles (more features to be added as time goes by). In order to make the whole thing interoperable with other projects building on top of the Go package, it also uses the user's account data on the user's Matrix homeserver to store profiles.

🔗neo v4: iris

F0x has recommenced development on the Matrix client Neo:

After discussion following https://cyberia.social/@foks/101785513826000032 I've resumed development on Neo. Suggestions are very much welcomed on the pad and mastodon thread
I'm implementing components one by one now, with just mocked events. Actual Matrix integration will come when the gui components are ready.

Neo is now partly integrated with matrix-js-sdk because I grew tired of having to write my own mock events. There's a basic authentication flow, with 0 error handling, and parsing of m.text and m.image events

Check out https://git.lain.haus/f0x/iris for the source code and see a working version at https://neo.lain.haus/neo

f0x also said:

it would be much appreciated to get some feedback on how I implemented matrix-js-sdk, had to do some weird stuff to make React update

I'll take a look but would love anyone with more experience of the SDK to get in touch with the f-man.

🔗solari[z|s]ed wallpaper

ma1uta shared this awesome Matrix wallpaper, created by @kolonsky:092918.xyz via https://linux.pictures/projects/solarized-dark-wallpaper

🔗mautrix-telegram 0.5.0 released

tulir is up again with his Telegram bridge:

mautrix-telegram 0.5.0 was released after I finally fixed the bug that was causing the bridge database to lock up. It turned out to be a single line of ORM usage that I had missed while converting everything to use SQLAlchemy Core. The full release notes are at https://github.com/tulir/mautrix-telegram/releases/tag/v0.5.0. I also released v0.5.1 to fix a bug that made the DBMS migration script not work and Python 3.5 compatibility. I wouldn't recommend using Python 3.5 though, I'm going to drop support for it some time after Debian 10 is released

🔗Shevski mentions Matrix on BBC Radio 4

Shevski from the #redecentralize organisation was interviewed on Radio 4, and mentioned Matrix:

Yesterday on @BBCRadio4 I talked about the problems of centralisation, how to regulate big tech, personal data stores and @redecentralize! You can listen online here: https://t.co/xGHK812lYa with #Web30 section starting 1:50 mins in ? (I'm from 7mins ish) #redecentralize pic.twitter.com/m0dazBH7Dk

— Irina (Ira) Bolychevsky (@shevski) March 22, 2019

🔗Extras

Scylla has been receiving regular updates, no release to report but the project keeps going. #scylla:riot.danilafe.com

This React Native client is not announced so don't look at it

My pal Doug shared his frustration with the broader state of the chat ecosystem after reading that teens now use Google Docs for their chat needs (and not the chat feature, the document itself!)

I love that you've got Amazon with their Alexa messaging, Apple in their own bubble with iMessage, Facebook doing chat 3-ways and Google trying 50 ways to do chat. And that's before you even get into things like Slack, Discord, Telegram, Riot, Line, WeChat etc etc. And then the current hot way to chat is in a document... I mean, how much money is being invested in trying to create the perfect chat solution?

For the record, we were chatting over Matrix…

🔗That's all I know

That's the news, if you have something to say, or something to add, then you should go to #twim:matrix.org and share it. If you have other projects to discuss, come share them. If you'd prefer to come quietly, my door is always open: @benpa:matrix.org.

See you here next week, and on Matrix!