Hi all,

We've just released Synapse 1.7.2 - a minor point release to address two regressions which snuck into 1.7.0 and 1.7.1. Sorry for the upgrade faff; hopefully we will be back to a saner release cadence shortly. Reminder that if you are on 1.7.0 or earlier you should upgrade asap as 1.7.1 contained security fixes.

Get the new release from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.

The changelog since 1.7.1 is:

🔗Synapse 1.7.2 (2019-12-20)

This release fixes some regressions introduced in Synapse 1.7.0 and 1.7.1.

🔗Bugfixes

• Fix a regression introduced in Synapse 1.7.1 which caused errors when attempting to backfill rooms over federation. (#6576)
• Fix a bug introduced in Synapse 1.7.0 which caused an error on startup when upgrading from versions before 1.3.0. (#6578)

Hi all,

We’re incredibly excited that Mozilla just announced that they’ve selected Matrix as the successor to IRC as the communication platform for the public Mozilla community!! This comes off the back of a formal 1-month trial in September to evaluate various options side by side, and now New Vector will be helping Mozilla get their homeserver up and running on the Modular.im hosting platform over the coming weeks - and federating openly with the rest of the open global Matrix network! :)

We have always been massive fans of Mozilla: they have been an excellent role model as champions of the open web, open standards, not to mention open source - and it’s fair to say that Mozilla has been a major inspiration to how Matrix has evolved (Riot aspires to be to Matrix what Firefox is to the Web: a flagship open source app which provides an accessible friendly interface into an open standard network). It’s very reassuring to see that Mozillians from the trial recognise the alignment and have converged on Matrix as the way forward - it’s a massive win for the open web and standards-based communication in general.

It’s worth noting that we’ve also always been massive fans of IRC, and Matrix is unashamedly derivative of IRC in capabilities and culture, while broadening the scope to decentralised synchronisation and relaying of any kind of data. For context, the genesis of the team which eventually spawned Matrix was on a student IRC server ~20 years ago - and subsequently everything we’ve worked on (up to Matrix) was coordinated exclusively through IRC. We even used to give conference talks on how to run your project/company off IRC. I can’t really overstate how fundamental IRC is to our history - and we still keep our private IRC network online for old time’s sake (albeit bridged to Matrix). The very first protocol bridge we built for Matrix back in 2015 was for IRC - and Moznet and Freenode were the first public bridges we turned on. As of right now, /stats u on Moznet says that there are 4950 connected users, of which 1724 (so 35%) are actually Matrix users connected via the Moznet bridge - effectively using Matrix as a big decentralised IRC bouncer in the sky.

All of this is to say is that we deeply understand how dependent Mozilla has been on IRC over the years, and that we built Matrix to be a worthy successor which tries to capture all the best bits of IRC while providing much richer primitives (E2E encryption, openly federated decentralised chatrooms, arbitrary data sync, HTTP API, VoIP, etc). It’s also worth noting that even though Moznet is being turned off, matrix-ircd exists as a very promising project that exposes any Matrix homeserver as an ircd - so for all you IRC die-hards, Moznet can absolutely live on in the afterlife! (matrix-ircd is still alpha right now, but it’s a relatively modest amount of Rust and PRs are very welcome - if you grok IRC it should be a really really fun project to contribute to).

In other news, the trial in September was an amazing opportunity to gather feedback first-hand from a wide range of Mozillians as they gave Riot and Matrix a spin, often for the first time - and it was a lot of fun to take that feedback and rapidly act on it to improve the app. For instance, having direct expert feedback on our screenreader support meant that we were able to radically improve our accessibility, and we’ve kept up the momentum on this since the trial (regardless of the outcome) with Mozilla & Riot devs hacking together with the aim of making Riot the most accessible communication app out there without exception. Huge thanks to Marco Zehe for all his guidance (and PRs), as well as the rest of #a11y:matrix.org!

Meanwhile, Riot’s UX continues to mature in general. One of our two primary projects right now is to improve First Time User Experience (FTUE) - i.e. making our UX as smooth and polished and predictable as possible, especially as seen by new users. This project had just kicked off in September as the Mozilla trial began, and some of the major improvements to the Room Directory and Room Creation flow which subsequently landed in Riot/Web 1.5 were prioritised directly based on Mozillian feedback. Since the trial we’ve been focusing more on our other primary project (getting E2E Encryption enabled by default), but we will be back on FTUE asap - particularly to incorporate all the feedback we anticipate as Mozilla goes live! We are absolutely determined for Riot to have as good if not better UX than the likes of Slack or Discord. New Vector is also actively hiring more designers to come work fulltime on Riot’s UI and UX as we shift Riot’s focus from being developer-led to design-led - if this sounds interesting, please get in touch! And finally, everything is of course open source and PRs are genuinely appreciated to keep Riot heading in the right direction (please just check first if they change the UI/UX).

Finally, in case you’re dreading having to use a graphical chat client like Riot, the Mozilla instance will of course be accessible to any Matrix client that floats your boat - for instance, weechat-matrix also got a spurt of development to support Mozilla IAM single-sign-on so that commandline junkies can get their fix too. (It’s worth noting that weechat-matrix really is an incredibly fully featured and usable client - complete with full end-to-end encryption support. If you haven’t tried it, you’re missing out).

So, to conclude: it has been indescribably valuable to have the expertise and enthusiasm of the Mozilla community in contributing feedback and fixes to Riot (and even building new Matrix bots!). Huge thanks to everyone who invested their time and energy participating in the trial and for their trust in concluding that Matrix was the way forward. We see this as a massive responsibility and honour to help power the wider Mozilla community, and we will do everything we can to make it as successful as conceivably possible :)

To the future of an open web, with even more open communications!

Matthew, Amandine & the whole Matrix & Riot team :)

P.S. we’ve come a long way since Matrix was first proposed for Mozilla :D

Hi folks; today we are releasing Synapse 1.7.1.

This is a security release which fixes some problems which affected all previous versions of Synapse. We advise all admins whose servers are open to public federation to upgrade as soon as possible.

Full details follow, but the most important change improves event authorization, thereby preventing the ability to add certain events to a given room erroneously.

You can get the new release from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.

The changelog since 1.7.0 follows:

🔗Security updates

• Fix a bug which could cause room events to be incorrectly authorized using events from a different room. (#6501, #6503, #6521, #6524, #6530, #6531)
• Fix a bug causing responses to the /context client endpoint to not use the pruned version of the event. (#6553)
• Fix a cause of state resets in room versions 2 onwards. (#6556, #6560)

🔗Bugfixes

• Fix a bug which could cause the federation server to incorrectly return errors when handling certain obscure event graphs. (#6526, #6527)

Hello people, it’s Synapse 1.7.0 time.

This release includes some long requested features, most notably the ability to automatically delete message data after a predefined period. For more details take a look at the config here ─ it should be pretty self explanatory.

Another significant change this release is to explicitly set room directories to be private by default. Previously it was possible to inadvertently configure the directory to be visible to arbitrary Matrix servers and the internet in general.

This means that for those admins who want their room directories to be publicly searchable (matrix.org for instance) they need to explicitly say so in the config. For more details see the upgrade notes and our blog post explaining the situation in greater detail.

We also have early support for ephemeral messages, as well as the ability to specify a reason when rejecting an invite (amongst other actions).

Aside from all of that, we want to let you know about some changes on the horizon. Currently Synapse runs Sqlite by default. This is great in that it gets new admins going quickly without needing to install and configure Postgres. The downside of using Sqlite is that it offers very poor performance, especially once a server tries to join the federation. In truth Sqlite is only really there to demonstrate the service, but for anything other than the most trivial cases it is essential to migrate to Postgres.

Over the past few months we’ve been working to improve the migration path to Postgres such that finally we feel confident to actively encourage admins to migrate. What’s more, in a future release we will forcibly prevent SQLite-backed servers federating unless the admin explicitly sets a config flag to show that they understand the trade-off they are making.

Overall we see these changes as something that will improve everyone’s experience of the matrix federation. We’ll talk more about this closer to the time, but please expect a change in the coming months and if you are running SQLite, consider this a nudge to get yourself migrated.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page.

The changelog since 1.6.1 follows:

🔗Synapse 1.7.0 (2019-12-13)

This release changes the default settings so that only local authenticated users can query the server's room directory. See the upgrade notes for details.

Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11.

Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available here. A future release of synapse will, by default, disable federation for servers using SQLite.

No significant changes since 1.7.0rc2.

🔗Synapse 1.7.0rc2 (2019-12-11)

🔗Bugfixes

• Fix incorrect error message for invalid requests when setting user's avatar URL. (#6497)
• Fix support for SQLite 3.7. (#6499)
• Fix regression where sending email push would not work when using a pusher worker. (#6507, #6509)

🔗Synapse 1.7.0rc1 (2019-12-09)

🔗Features

• Implement per-room message retention policies. (#5815, #6436)
• Add etag and count fields to key backup endpoints to help clients guess if there are new keys. (#5858)
• Add /admin/v2/users endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. (#5925)
• Require User-Interactive Authentication for /account/3pid/add, meaning the user's password will be required to add a third-party ID to their account. (#6119)
• Implement the /_matrix/federation/unstable/net.atleastfornow/state/<context> API as drafted in MSC2314. (#6176)
• Configure privacy-preserving settings by default for the room directory. (#6355)
• Add ephemeral messages support by partially implementing MSC2228. (#6409)
• Add support for MSC 2367, which allows specifying a reason on all membership events. (#6434)

🔗Bugfixes

• Transfer non-standard power levels on room upgrade. (#6237)
• Fix error from the Pillow library when uploading RGBA images. (#6241)
• Correctly apply the event filter to the state, events_before and events_after fields in the response to /context requests. (#6329)
• Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. (#6332)
• Prevent account data syncs getting lost across TCP replication. (#6333)
• Fix bug: TypeError in register_user() while using LDAP auth module. (#6406)
• Fix an intermittent exception when handling read-receipts. (#6408)
• Fix broken guest registration when there are existing blocks of numeric user IDs. (#6420)
• Fix startup error when http proxy is defined. (#6421)
• Fix error when using synapse_port_db on a vanilla synapse db. (#6449)
• Fix uploading multiple cross signing signatures for the same user. (#6451)
• Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. (#6462)
• Fix synapse_port_db not exiting with a 0 code if something went wrong during the port process. (#6470)
• Improve sanity-checking when receiving events over federation. (#6472)
• Fix inaccurate per-block Prometheus metrics. (#6491)
• Fix small performance regression for sending invites. (#6493)
• Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. (#6494)

🔗Improved Documentation

• Update documentation and variables in user contributed systemd reference file. (#6369, #6490)
• Fix link in the user directory documentation. (#6388)
• Add build instructions to the docker readme. (#6390)
• Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. (#6443)
• Write some docs for the quarantine_media api. (#6458)
• Convert CONTRIBUTING.rst to markdown (among other small fixes). (#6461)

🔗Deprecations and Removals

• Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. (#5925)
• Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. (#6488)

🔗Internal Changes

• Add benchmarks for structured logging and improve output performance. (#6266)
• Improve the performance of outputting structured logging. (#6322)
• Refactor some code in the event authentication path for clarity. (#6343, #6468, #6480)
• Clean up some unnecessary quotation marks around the codebase. (#6362)
• Complain on startup instead of 500'ing during runtime when public_baseurl isn't set when necessary. (#6379)
• Add a test scenario to make sure room history purges don't break /messages in the future. (#6392)
• Clarifications for the email configuration settings. (#6423)
• Add more tests to the blacklist when running in worker mode. (#6429)
• Refactor data store layer to support multiple databases in the future. (#6454, #6464, #6469, #6487)
• Port synapse.rest.client.v1 to async/await. (#6482)
• Port synapse.rest.client.v2_alpha to async/await. (#6483)
• Port SyncHandler to async/await. (#6484)

🔗Matrix Live 🎙

🔗Dept of Status of Matrix 🌡

Matthew notes:

We'd like to welcome Twitter to the world of decentralised communication protocols after Jack Dorsey's announcement this week that Twitter is building a decentralised social media team. It seems that the constraints they're working with are to focus on decentralised reputation (supporting different content filtering algorithms), incentive models (presumably some kind of token) and avoiding consensus-based standards processes. It's worth noting that we've been working on decentralised reputation stuff in Matrix for a while now - of which MSC2313 - Moderation policies as rooms is the most concrete result so far, and it's great to see Twitter thinking about how to adopted different filtering mechanisms for their content. It sounds as if they're approaching this from a blockchain/incentives angle however, so it remains to be seen whether they'll be interested in our work - especially as Matrix doesn't have a microblogging client yet (but only because nobody has made one yet). We'll be trying to talk to them whatever to see if we can be of use, eitherway :)

🔗Dept of Spec 📜

🔗Spec

anoa said:

Here's your weekly update for what happened in spec land!

While it may look quiet from the state changes list, there's actually been a flurry of activity on MSC2376 and MSC2385 (for disabling URL previews on a per-message basis), MSC2380 (for a method of querying the metadata of a piece of media without downloading it) and MSC2346 (for showing metadata about the bridges that are currently active in the room)! Now's the time to jump in if you want to have your say!

Updates:

Merged MSCs

• No MSCs were merged this week.

MSCs in Final Comment Period

• No MSCs in FCP.

New MSCs

• MSC2385: Disable URL Previews, alternative method

Spec Core Team:

The Spec Core Team is on the same track as last week with no specific 3 MSC focus, but working on bringing up a lot of MSCs across the board.

🔗Dept of Servers 🏢

🔗Synapse

Neil announced:

Synapse 1.7.0 is out, check out all the details here, admins can now specify message retention policies at a server and room level. We also changed the defaults for the room directory to be privacy preserving by default.

Next up we’re taking a look at support for redacting room alias events 1, 2 as well as porting Sydent to python 3.

🔗Deploying Synapse

Several packaging projects have been updated to deploy the new version:

• matrix-docker-ansible-deploy from Slavi
• avhost/docker-matrix and mvgorcum/docker-matrix from Mathijs
• Kubernetes from Ananace
• multi-arch synapse docker image from Black Hat
• also from the Debian repos thanks to @andrewsh:matrix.org

🔗Dept of Clients 📱

🔗Fractal

Alexandre Franke told us:

We gained the ability to save spellcheck language per room, which makes me quite happy as I keep switching between English speaking and French speaking ones and was growing tired of those red underlines and having to switch manually every time.

Data is stored in /user/{}/rooms/{}/account_data/org.gnome.fractal.language.

🔗Continuum updates

yuforia said:

Continuum, Kotlin client for the desktop:

• Updated to Kotlin Json and HTTP libraries, removed Moshi and Retrofit from dependencies

• When there is an error when loading notifications, one can click to retry or view the cause

🔗nheko

Nico said:

nheko mostly fixed bugs regarding the new file encryption this week and did some organizational stuff:

• We fixed a compliance issue, where Riot couldn't decrypt our media
• You can now actually see your encrypted images, when you sent them
• We fixed some tests regarding our session key export
• We fixed our coverage of our automated tests
• We did some prepwork for device verification
• A few minor usability fixes and code cleanups

🔗RiotX v0.10.0

benoit told us:

RiotX v0.10.0 has been released on Tuesday, with some bug fixes and a new Breadcrumbs drawer to switch between rooms super super fast. Give it a try! Now we are implementing workflow when the access token get invalidated, with SoftLogout support. Also, we are still working on improving the initial sync management, which can be a long task on big account, and that causes some problem with the current implementation. Among various other subjects: matrix.to support, room profile screen, verification in DM, cleanup dependencies to reach the F-Droid store, we are quite busy!

🔗Riot-iOS 0.10.4

Manu reported:

This week, we released Riot-iOS 0.10.4 with a couple of hot fixes on device verification. In parallel, we have been still working on verification by DM both on UI and SDK sides. As a collateral effect, the aggregation of m.reference has been implemented in the SDK. This means the SDK is now ready for message threading !

🔗Riot Web + Cross-Signing

Ryan said:

Cross-signing keys and secret storage can be created in Settings for advanced users with the cross-signing feature flag enabled in labs, though please keep in mind that these features are still in development. More work is still needed to change to verifying users instead of devices. More accessibility fixes have landed as well.

Matthew added:

as part of all the work around cross-signing, we're shifting device verification to happen in the context of DMs so verification is done per-user rather than per-device, and so you can track your verification history and generally massively improve the UX. valere made a great video of how this is shaping up between RiotX and Riot/Web...

Check out the video here

🔗Dept of Ops 🛠

🔗ma1sd 2.2.1 released

ma1uta said:

Release 2.2.0 of the ma1sd (fork mxisd) https://github.com/ma1uta/ma1sd/releases/tag/2.2.0 Changes:

• support of the MSC2140 (hash lookup)
• support of the MSC2134 (API v2)

and then...

ma1sd hotfix 2.2.1 released with a lot of bugfixes. Also the v2 API (MSC2140) was disabled by default because it breaks backward compatibility in lookup behaviour.

🔗Dept of Bots 🤖

🔗Matrix bot functionality in Python

Cos reported:

I have created a modular bot for writing Matrix bot functionality in Python easily. It already has bunch of modules ranging from weather to calendar integration and more will come. Even the location bot from last week's TWIM is now implemented as a module. I hope you find it useful. PR's of new modules are always welcome. https://github.com/vranki/hemppa

🔗Mlrdb, LDAP sync bot, announced

@dalang:mc.kircheneuenburg.de offered:

A Bot to sync LDAP groups to matrix rooms. Rooms will be created automatically and group member changes are reflected in the matrix rooms. The bot is currently in beta and documentation will be added in the next weeks. New features for simple integration will be added soon. Have a look at the repo: https://git.sr.ht/~davidlang/mlrbd

🔗Dept of Interesting Projects 🛰

🔗mautrixfs - Matrix client as a FUSE filesystem

Tulir offered:

I've started a new project: mautrixfs is a Matrix client as a FUSE filesystem. It's very WIP and currently only supports reading events by ID. I'm hoping to have something more useful in a week or two.

tulir later added:

for media uploads, I just realized that my asynchronous uploads MSC would make it significantly easier to implement. It could have a file you read to allocate a mxc uri and then you could simply write the data to the file corresponding to that mxc uri

🔗Matrix in the News 📰

🔗Matrix featured in Skrolli

Cos offered:

Finnish computer culture paper magazine Skrolli published two articles spanning 5 pages about Matrix in latest edition. For non-subscribers the digital edition readable with mobile app is free for limited time. Read more (in Finnish) at https://skrolli.fi/

🔗Dept of Ping 🏓

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

🔗Matrix Live 🎙

I chatted to Kilian from the new Nio project about his progress on a new iOS client.

🔗Dept of Status of Matrix 🌡

Yesterday we announced the availability of zipped hoodies! The resulting avalanche of orders was a shock for our local Post Office, who watched with surprise as I burst in, laden with packages like a mule.

🔗Dept of Spec 📜

anoa offered:

Here's your news for what happened in spec-land last week!

Merged MSCs

• MSC2367: Add reason field to all membership events
• MSC2324: Facilitating early releases of software dependent on spec

MSCs in Final Comment Period

No MSCs are in FCP...

New MSCs

• MSC2380: Matrix Media Information API
  • This is a markdownification of MSC1310
• MSC2370: Resolve URL API
• MSC2375: Appservice Invite States
• MSC2376: Disable URL Previews

TravisR confirmed:

The spec core team is focused on implementing a bunch of MSCs so that we can make progress with the spec.

🔗Dept of Servers 🏢

🔗Synapse

Neil told us:

This week we've mainly been tying up loose ends on the matrix.org data centre migration. Outside of that one interesting feature to land on develop is that of configurable message retention. It means that as an admin you can set a default period after which messages will be removed from the server (a year say), what’s more the default is configurable on a per room basis via state events. Note, this feature applies to messages only, not media (on its way!). Configurable message retention will be make it into the next versioned release.

🔗matrix-synapse-ldap3

anoa said:

After 2 long years, matrix-synapse-ldap3 has finally gained another release: v0.1.4. If you want to allow users to sign in to Synapse using their LDAP accounts, then this is the module for you.

Get the release notes here: https://github.com/matrix-org/matrix-synapse-ldap3/releases/tag/v0.1.4

🔗Dept of Clients 📱

🔗Fractal

Alexandre Franke offered:

4.2.2 got released! Get it now off Flathub.

New features:

• Translation strings have been updated

Bugfixes:

• Newlines now work when markdown is enabled
• Account settings can be accessed again
• Redaction is working again
• Compatibility with HTTP-only (non HTTPS) servers has been restored
• Some crashes fixed

Under the hood:

• Code refactor is still going

As if all that wasn't enough, how about this image of Fractal running on a Librem5, from Adrien Plazas:

🔗nheko

Nico offered:

Nheko just merged support for encrypted files in e2ee chats. This allows you to send and receive encrypted files and even play videos and other media in the new timeline. Those changes are still on the development branch, but will be included in the next release, which should be 0.7.0. If you want to try it out, you can get development builds here.

🔗koma

yuforia announced:

koma, the Kotlin library behind the desktop client Continuum:

• Rewrite all Moshi json adapters with kotlinx serializers, replace API interface declared using Retrofit with HTTP requests constructed using ktor. The majority of the changes needed to enable multiplatform support is now made.

🔗RiotX 0.9.1

benoit reported:

Released v0.9.1 on Thursday! This version includes a new login flow and account creation flow. Read marker has been reworked, to simplify user experience on mobile. Also you can type and send user pills.

We are now working on room and user profiles, and matrix.to link handling. Verification in DM development is progressing well. Also we implemented a first version of breadcrumbs, to switch between recent rooms very quickly.

🔗Riot-iOS

Manu announced:

Riot-iOS 0.10.3 is on its way to Apple review. The release note is: This release includes bug fixes and improvements, like:

• Ask permission before opening a widget and the integration manager.
• Display room name, user name and user avatar in the Jitsi conference screen.
• Support Welsh and Italian languages.

🔗Messagerie

Manu reported:

let me share Messagerie: https://github.com/manuroe/messagerie, an iOS app and, maybe, in the future, a MacOS app.

Messagerie is a chat app written to test SwiftUI and Combine to make a real and, hopefully, useful app. Messagerie misses a lot of things but it has:

• Support of Matrix using SwiftMatrixSDK
• E2EE
• Multi-account (swipe the navigation bar to switch accounts)
• Dark mode and auto-sizing fonts (well, they come for free)
• 4 screens at the moment: a Matrix login screen, a room list, a room screen and a very early beginning of a SwiftUI text composer to post text message

This is a personal and a side project, but RiotX-iOS will probably take benefit from it.

🔗Riot Web means Cross Signing!

Ryan announced:

Riot Web is largely focused on nudging cross-signing forward over the last week. Key verification and cross-signing setup continues to make progress. We've also fixed a few bugs and merged some accessibility improvements.

🔗Dept of Ops 🛠

🔗matrix-docker-ansible-deploy

Slavi reported:

matrix-docker-ansible-deploy has been made more flexible in terms of how its services can be exposed to the web. Reverse-proxying using another local (in-machine or on-the-network) webserver should be easier. SSL-termination using Traefik is now also a possibility, although we're yet to confirm it as a success and document exactly how to do it. Read more in the CHANGELOG.

🔗riot-desktop-builder docker image

Mathijs announced:

I created a dockerfile and corresponding docker image that can build the develop branch of riot-desktop for both windows and linux.

🔗Dept of Bots 🤖

🔗msc-chatbot

anoa announced:

msc_resolver2 is back and now isn't annoying with all new features like:

• actually showing you the MSC title when it links it
• not posting the same links multiple times
• not posting a link when you've already linked the msc in the original message
• ignoring edits
• ignoring content of replies

rejoice! Code is https://github.com/anoadragon453/msc-chatbot

🔗malobot - send or receive location messages

Cos said:

It would be cool to be able to share location in Matrix rooms. This bot by cos solves the egg part in chicken and egg problem of having no clients able to send or receive location messages. Next we need a mobile client that can send location updates. Which one will be the first? https://github.com/vranki/malobot/

🔗Dept of Ping 🏓

🔗Final Thoughts 💭

swedneck made a room for discussing how matrix could be used to implement forums: #matrix-forums:permaweb.io

Mathijs pointed out that there is a Matrix feature which enables spoilers.

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Matrix merch is some of the most sought-after apparel and adhesive labeling available in the world today. We supply discerning customers with t-shirts, hoodies and stickers which remind them which decentralised, federated, end-to-end encrypted communications protocol is the best choice.

This is all very well, but we wondered whether it would be possible to have a warm, hooded garment which promotes the wearers preference of chat system, and also has a zip on the front.

Today, we’re proud to announce a breakthrough in this area: Matrix Zipped Hoodies are available now from shop.matrix.org!

Grab yours now from shop.matrix.org!

🔗Matrix Live 🎙

🔗Dept of Spec 📜

🔗Spec

anoa announced:

Here's what happened in spec land this week:

Merged MSCs:

• MSC 1756: cross-signing devices using a master identity key
• MSC 2313: Moderation policies as rooms (ban lists)

Final Comment Period:

• MSC 2324: Facilitating early releases of software dependent on spec
• MSC 2367: Add reason field to all membership events

New MSCs:

• MSC 2336: Key verification flow addition: m.key.verification.accept

The Spec Core Team have chosen the following three MSCs to focus on for next week: MSC 1270 (URL resolver in media), MSC 2366 (accept in verification), and MSC 1849 (aggregations/edits)

🔗Dept of Servers 🏢

🔗Synapse v1.6.1

Neil offered:

This week we put out Synapse 1.6.0, and 1.6.1 checkout all the juicy details in the blog post. Aside from that message retention and ephemeral message support continues and we expect the latter to merge early next week.

The next big thing we’ll be looking at is sharding out the Synapse master process so that instances running in worker mode can make full use of the CPU power available. This will make a big difference to matrix.org.

🔗Deploying Synapse

Several packaging projects have been updated to deploy the new version

• matrix-docker-ansible-deploy from Slavi
• avhost/docker-matrix and mvgorcum/docker-matrix from Mathijs
  • Mathijs was also happy to remind everyone of Docker itself
• Kubernetes from Ananace
• multi-arch synapse docker image from Black Hat

Additionally, from JCG:

Synapse 1.6.1 has been packaged for VoidLinux, FreeBSD and Alpine Linux, with NixOS waiting to have the PR updating it to 1.6.1 merged. Synapse 1.6.0 has been packaged for Debian Unstable and Ubuntu 20.04.

🔗Docker for synapse-ldap

JCG announced:

The container image with the updated LDAP auth provider over at https://gitlab.com/famedly/container/synapse-ldap/container_registry has been updated to 1.6.0 too.

🔗Matrix over Yggdrasil

Neil Alexander:

Timothée has been working on a university project to integrate the Yggdrasil library into the CoAP proxy, which allows Matrix homeservers to federate over a pure Yggdrasil connection instead of using IP. The Yggdrasil portion gives full reachability and traffic forwarding between nodes in the mesh even in complicated topologies, and end-to-end encryption as an additional benefit

As a reminder,

Yggdrasil is a proof-of-concept mesh network that is designed to avoid the scaling issues that we've seen in the past with existing mesh systems. It uses a spanning tree-based topology and aims to make all nodes in the mesh fully routable, even at massive scale

If you'd like to know more, come chat to the folk in #yggdrasil:matrix.org, and read https://yggdrasil-network.github.io

🔗Department of Matrix jobs 👩‍💻

New Vector (the startup which the original Matrix team founded in order to hire folks to work on Matrix as their day job) are currently hiring people so if you ever wanted to work on Matrix full time get in touch.

Neil says:

You can read all about it here but we are particularly keen to speak to people who want to hack on Synapse or work in Operations.

We are remote friendly though find it easier to hire people in some territories than others, so if you have any questions just ask.

You can apply via the site, or alternatively reach out to @neilj:matrix.org (engineering) or @nadonomy:matrix.org (design).

On top of this, Neil Alexander, creator of Seaglass and maintainer of Yggdrasil, will be joining New Vector. He said:

I shall most likely be working with the backend team on Synapse/Dendrite and I think there's a couple of other things like the coap-proxy too

🔗Dept of Bridges 🌉

🔗mx-puppet-bridge

Tulir said:

I made two pull requests to Sorunome's mx-puppet-bridge projects:

• A relaybot mode in mx-puppet-bridge. Basically means any mx-puppet-bridge can be used as a relaybot bridge.
• OAuth login flow for mx-puppet-slack. Should be useful for those in slack workspaces that disallow legacy tokens.

🔗Matrix-Appservice-IRC

Half-Shot offered:

My fellow comrades, today we have released 0.14.0-rc1 of the IRC bridge. The changes are massive and vast, and frankly it probably could have been done in 2 or 3 releases. At any rate, this release contains support for PostgreSQL Datastores and Sentry monitoring, amongst other small quality of life changes. The bridge has also had a total refactor using Typescript, and it's a little bit nicer to look at now.

🔗Dept of Clients 📱

🔗Redesign of Ditto is out!

Annie told us:

• Ditto is now redesigned with a fresh new look!
• Functionality:
  • Login, send / receive messages, logout
• On Deck:
  • Notifications, writing a new message (in that order)

Come chat about UX and things you'd like to see in Ditto!
Matrix room: #ditto:elequin.io

iOS: https://testflight.apple.com/join/9M0ERlKd
Website: https://dittochat.org
Feedback: https://plan.dittochat.org

swedneck asked if being purple in colour, the project used libpurple? The answer is no, it uses React Native, and he was widely rebuked for doing so.

🔗nio, new client for iOS

kilian offered:

I spent the last few days building my Matrix client Nio 😄 Apple just approved a very early first alpha for TestFlight distribution. It really doesn't do a lot aside from account authentication and displaying recent chats and messages. It is able to handle e2e encryption, but unfortunately doesn't persist the encryption keys right now (meaning it loses them and re-requests them from other clients on being restarted). It's built on SwiftUI and runs on iOS (iPhone and iPad). The app will likely not run as-is on macOS in the future, but I'd love to also build a separate version of Nio for macOS once the iOS app is functional.

Website: https://nio.chat Repo: https://github.com/kiliankoe/nio TestFlight: https://testflight.apple.com/join/KlXr3kKz

Note that this is not connected to the existing Matrix project, matrix-nio.

🔗Continuum and koma, client and library in Kotlin

yuforia reported:

koma, Kotlin library:

• Start replacing Retrofit with Ktor, which is JetBrain's HTTP library for building projects that can be compiled to Native, JVM, and JavaScript. Currently, coexistence of both libraries causes the package size of applications based on Koma to increase to some extent, this will no longer be the case after the transition is complete.

Continuum, desktop client based on koma, version 0.9.31:

• Implement minimal XML parsing without adding additional dependencies to extract user ID and name from formatted_body used by Riot
• Display mentioned users with highlight and avatar.

🔗Riot Web

Ryan reported:

• Continued work on setting up cross-signing and secret storage keys in labs
• Device verification is moving to the user info panel and happening via DMs also in labs
• Fixes for system theme and read receipts
• Improved signing for Windows builds
• Converting tests towards native promises

🔗Riot-iOS

From Manu:

We have a new bottom sheet that we are going to use a lot in the coming designs. We used it in the widget permission screen. Meanwhile we are working on user verification by DM

🔗Riot-android

From Manu:

We made a release this week with all the work around privacy for the use of an Integration Manager and widgets.

🔗RiotX

From Manu:

All flows for registration and login are now supported. Please test them on your specific homeserver configuration in case we miss something. We are also working on user verification by DM.

🔗Dept of Ops 🛠

🔗Docker

Mathijs reported:

mvgorcum/docker-matrix:v1.6.1 is now on docker-hub

🔗matrix-docker-ansible-deploy riot-web image

Slavi announced:

matrix-docker-ansible-deploy has switched from the bubuntux/riot-web Docker image to the official vectorim/riot-web image. There should be no visible changes for users. We're just hoping for faster releases by keeping closer to upstream.

🔗Dept of Ping 🏓

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.6.0 has landed and is here to brighten your day!

1.6.0's most notable feature is that of label based filtering. It allows for messages to be tagged with a given label so that clients can filter on the label, this means that clients can subscribe to specific topics in a room, such as #lunch.

Completely separately, from here on in new rooms will be version 5 by default, all this means in practice is that servers will respect server signing key validity periods. This won't make a lot of difference in day to day operation, but it is an important security consideration and we now have sufficient penetration across the federation to make version 5 the default.

Aside from that there are a bunch of bug fixes and improvements, including fixing a bug that in some cases prevented messages being decrypted shortly after a restart (#6363) and generally improving the room upgrade experience (#6232, #6235).

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page

The changelog since 1.5.1 follows:

🔗Synapse 1.6.0 (2019-11-26)

🔗Bugfixes

• Fix phone home stats reporting. (#6418)

🔗Synapse 1.6.0rc2 (2019-11-25)

🔗Bugfixes

• Fix a bug which could cause the background database update handler for event labels to get stuck in a loop raising exceptions. (#6407)

🔗Synapse 1.6.0rc1 (2019-11-20)

🔗Features

• Add federation support for cross-signing. (#5727)
• Increase default room version from 4 to 5, thereby enforcing server key validity period checks. (#6220)
• Add support for outbound http proxying via http_proxy/HTTPS_PROXY env vars. (#6238)
• Implement label-based filtering on /sync and /messages (MSC2326). (#6301, #6310, #6340)

🔗Bugfixes

• Fix LruCache callback deduplication for Python 3.8. Contributed by @V02460. (#6213)
• Remove a room from a server's public rooms list on room upgrade. (#6232, #6235)
• Delete keys from key backup when deleting backup versions. (#6253)
• Make notification of cross-signing signatures work with workers. (#6254)
• Fix exception when remote servers attempt to join a room that they're not allowed to join. (#6278)
• Prevent errors from appearing on Synapse startup if git is not installed. (#6284)
• Appservice requests will no longer contain a double slash prefix when the appservice url provided ends in a slash. (#6306)
• Fix /purge_room admin API. (#6307)
• Fix the hidden field in the devices table for SQLite versions prior to 3.23.0. (#6313)
• Fix bug which caused rejected events to be persisted with the wrong room state. (#6320)
• Fix bug where rc_login ratelimiting would prematurely kick in. (#6335)
• Prevent the server taking a long time to start up when guest registration is enabled. (#6338)
• Fix bug where upgrading a guest account to a full user would fail when account validity is enabled. (#6359)
• Fix to_device stream ID getting reset every time Synapse restarts, which had the potential to cause unable to decrypt errors. (#6363)
• Fix permission denied error when trying to generate a config file with the docker image. (#6389)

🔗Improved Documentation

• Contributor documentation now mentions script to run linters. (#6164)
• Modify CAPTCHA_SETUP.md to update the terms private key and public key to secret key and site key respectively. Contributed by Yash Jipkate. (#6257)
• Update INSTALL.md Email section to talk about account_threepid_delegates. (#6272)
• Fix a small typo in account_threepid_delegates configuration option. (#6273)

🔗Internal Changes

• Add a CI job to test the synapse_port_db script. (#6140, #6276)
• Convert EventContext to an attrs. (#6218)
• Move persist_events out from main data store. (#6240, #6300)
• Reduce verbosity of user/room stats. (#6250)
• Reduce impact of debug logging. (#6251)
• Expose some homeserver functionality to spam checkers. (#6259)
• Change cache descriptors to always return deferreds. (#6263, #6291)
• Fix incorrect comment regarding the functionality of an if statement. (#6269)
• Update CI to run isort over the scripts and scripts-dev directories. (#6270)
• Replace every instance of logger.warn method with logger.warning as the former is deprecated. (#6271, #6314)
• Port replication http server endpoints to async/await. (#6274)
• Port room rest handlers to async/await. (#6275)
• Remove redundant CLI parameters on CI's flake8 step. (#6277)
• Port federation_server.py to async/await. (#6279)
• Port receipt and read markers to async/wait. (#6280)
• Split out state storage into separate data store. (#6294, #6295)
• Refactor EventContext for clarity. (#6298)
• Update the version of black used to 19.10b0. (#6304)
• Add some documentation about worker replication. (#6305)
• Move admin endpoints into separate files. Contributed by Awesome Technologies Innovationslabor GmbH. (#6308)
• Document the use of lint.sh for code style enforcement & extend it to run on specified paths only. (#6312)
• Add optional python dependencies and dependant binary libraries to snapcraft packaging. (#6317)
• Remove the dependency on psutil and replace functionality with the stdlib resource module. (#6318, #6336)
• Improve documentation for EventContext fields. (#6319)
• Add some checks that we aren't using state from rejected events. (#6330)
• Add continuous integration for python 3.8. (#6341)
• Correct spacing/case of various instances of the word "homeserver". (#6357)
• Temporarily blacklist the failing unit test PurgeRoomTestCase.test_purge_room. (#6361)

🔗Matrix Live 🎙

OpenPush is Marcus' project to remove the need for Google to be involved with push notifications on Android. We chat in some detail here about his work, the Prototype Fund, and other great topics.

🔗Dept of Status of Matrix 🌡

🔗matrix.org server migration

Neil reported:

Those following closely will know that the matrix.org home server has been having some problems with our hosting provider. This really came down to I/O provision and stability therein. It turns out that running a homeserver is harder when it can’t talk to the db.

We have now fully migrated to our new provider (with improved hardware specs) and you should notice everything feeling much much snappier.

https://twitter.com/matrixdotorg/status/1197828358664589312?s=20

https://twitter.com/matrixdotorg/status/1197576886278393856?s=20

🔗Public rooms upgraded

Neil reported:

We have also taking the time to upgrade all of our community rooms to modern room versions. This should mean an end to state resets and other unintuitive behaviour associated with large public rooms.

🔗kitsune, international man of mystery

kitsune offered:

I have been dismantling my habitat in Japan and will spend a couple of weeks in Moscow, Russia before moving further west to the Netherlands. Due to this, expect very low activity on Quotient front in December; but I still intend to release the first beta of libQuotient 0.6 in the remaining week, breaking the half-year span without releases.

Good luck!

🔗Dept of Spec 📜

🔗Spec

anoa told us:

Here's your weekly spec update on what happened last week in spec land.

New MSCs:

• MSC 2359: E2EE VoIP Conferencing [WIP]
• MSC 2356: Bulk /joined_members endpoint

Entered Final Comment Period:

• MSC 1756: cross-signing devices using a master identity key
• MSC 2313: Moderation policies as rooms (ban lists)

Merged:

• MSC 2284: Making the identity server optional during discovery

This week, the Spec Core Team is focusing on the following 3 MSCs: 2241 (DM key verification), 2324 (new spec process), and 2326 (label-based filtering).

🔗Dept of Servers 🏢

🔗Synapse v1.6.0rc1

Neil told us:

This week released Synapse v1.6.0rc1 and will release the real deal next week. 1.6.0 contains a lot of ground work for e2ee cross signing, supporting multiple databases (to aid db sharing) as well as a bunch of bug fixes and perf improvements.

Aside from that we’ve been working on room retention support and ephemeral messages which should be ready to merge rsn.

Finally we’ve been working on improving config granularity for caching, such that individual caches can be configured via homeserver.yaml. Experimenting with this approach to caching has proved to be very powerful in tuning performance, expect to see it on mainline shortly. Further down the line we'd like to make it more dynamic so that manual tuning is unnecessary.

This is also now available from Mathijs' docker hub repo.

🔗Dept of Bridges 🌉

🔗mautrix-facebook

Tulir reported:

Facebook decided to break everything and switch from long polling to MQTT over websockets, but mautrix-facebook has already been updated with initial support for the new protocol. It's still a bit buggy though, e.g. reconnecting after a disconnection doesn't seem to work properly

🔗Dept of Clients 📱

🔗Brawl 0.0.12

Bruno offered:

I have started deploying Brawl, you can try it at https://bwindels.github.io/brawl/

Also few small new features and fixes since the last TWIM mention a while back:

• make initial sync faster: it uses the lazy load members feature to not load a lot of room state on initial sync as that isn't used yet anyways.
• more useful room list: room aliases and are now recognized for the room name, and DMs are named after the user id.
• some smaller bugs fixed

🔗Continuum and koma progress

yuforia announced:

koma, Kotlin library:

• Update Kotlin to latest release 1.3.60
• Start making use of kotlinx.serialization, making the first step toward multi-platform support (which enables a Kotlin project to be compiled to native, JavaScript, or JVM)

Continuum, desktop client based on koma:

• Implement pop-up and sound, which will be used to show notifications

https://matrix.org/_matrix/media/r0/download/matrix.org/TJlhyKntaXngkHJvdgPIhOsK

🔗Riot iOS

Manu announced:

We are almost done in our privacy work around integrations and integrations manager. While we were working on widgets, we made some improvements on them. They now have a menu with some actions (refresh, open in Browser, remove). The jitsi widget now displays the room name, user avatar and name.

🔗Seshat

@poljar:matrix.org told us:

The PR for matrix-react-sdk has finally landed, the PR for riot-web needed some documentation and is waiting for final review. Work on the UI for our indexer inside of riot has started and some more functionality to load events that are files has been added inside of Seshat as well.

🔗Riot Android

@valere35:matrix.org reported:

Privacy work related to Integration Manager about to be released (SDK + App). Also a couple of bug fixed (including the infinite app restart after SSO token expiration)

🔗riot-web

Bruno reported:

riot web just gained in-app notifications for verification requests on /develop. See PR at https://github.com/matrix-org/matrix-react-sdk/pull/3661 . All of this is behind the verification over DM labs flag!

🔗matrix-notepad

Check out these docs, very nice!

@kb1rd:kb1rd.net said:

matrix-notepad It's been a slow two weeks for matrix-notepad since I've been a bit busy.

• The core "Logootish" algorithm was documented
• "Logootish" was separated out into its own repo (logootish-js) and converted to TypeScript.
• TypeDoc was used to document the logootish-js algorithm
• Fixed some wonderful spelling errors in the algorithm code
• Nothing has changed for the main Matrix Notepad repo, so there's no user difference. It just makes the core algorithm a bit easier to read.

My plan in the future is basically to work out rich text and JSON object collaboration (clearly, this is far away!) and create some kind of "universal client" that can load up web apps to use the algorithm in a single Matrix room. The result would be that it's much easier to create collaboration apps.
Obviously, that's a far-off goal, but my point in documenting the algo is to get ahead of the game a bit

🔗Dept of SDKs and Frameworks 🧰

🔗matrix-bot-sdk v0.4.0 release

TravisR announced:

v0.4.0 has been released of the matrix-bot-sdk. Last week was beta 15, and now it's out for real. There's quite a few changes from v0.3.9, but here's an overview:

• Support for encryption through Pantalaimon

• Support for metrics

• Finished support for appservices

• Better support for bridges (storage, mappings, and utility functions)

• Classes for events and other structures in Matrix

• Utility classes for permalinks, profile caching, Synapse admin APIs, and server ACLs (globs)

• Improved documentation

• A template repository for bots

• Improved logging support

• Various bug fixes and improvements

Give it a test, and report any issues in #matrix-bot-sdk:t2bot.io ! The full diff is available here: https://github.com/turt2live/matrix-bot-sdk/compare/v0.3.9...v0.4.0

🔗simplematrixlib, new Python library

@swedneck:permaweb.io said:

I made a really simple matrix library for python, and a couple of utility scripts that make use of it! https://gitlab.com/Swedneck/simplematrixlib, https://gitlab.com/Swedneck/py-matrix-utils

🔗Dept of Bots 🤖

🔗maubot

Tulir announced:

Maubot will now automatically follow room upgrades. In addition to that, plugins storing room IDs in the database have been updated to automatically update the database when the room is upgraded.

In other maubot news, the PR by lorico to rewrite my old GitLab bot as maubot plugin was finally merged.

🔗Dept of Ping 🏓

🔗Final Thoughts 💭

Impressive progress on the PinePhone, Martijn Braam has been showing off various apps running, particularly interesting is that they are showing Quaternion.

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!