The whole Matrix project is racing towards enabling e2ee by default. Synapse is no different and v1.10.0 contains multiple e2ee UX improvements, as well as a bug fix that prevented cross signing requests over federation to work reliably.
If any of your users are on the bleeding edge and have already started using cross signing (by enabling labs flags in Riot), then it will be necessary for them to force Synapse to re-send device updates by renaming all of their devices.
We've also included a temporary fix to address alias abuse. The idea is that until #6898 lands, servers will refrain from sharing events of type m.room.aliases with clients. Most admins will not be affected, but if you are present in rooms subject to alias abuse, then upgrading provides a pragmatic short term solution.
Finally, as of this release Synapse validates client_secret parameters in the Client-Server API as per the spec. See #6766 for details.
Get the new release from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.
WARNING to client developers: As of this release Synapse validates client_secret parameters in the Client-Server API as per the spec. See #6766 for details.
Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS). (#6734)
Minor fixes to PUT /_synapse/admin/v2/users admin api. (#6761)
Validate client_secret parameter using the regex provided by the Client-Server API, temporarily allowing : characters for older clients. The : character will be removed in a future release. (#6767)
Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key). (#6771)
Every month (in theory), we do an in-office wrap-up of some interesting project work from the team. This Matrix Live was recorded this afternoon in the Matrix Office, and features Valere explaining Cross-Signing, and Matthew demonstrating P2P/dendrite. (Apologies for the incorrect recorded screen at times! Video came straight from a live demo!)
I (anoa) have been re-writing mscbot as it hasn't gotten much development love in the past year and there's lots of features we'd like to implement. Expect to see some of that next week. First feature the new implementation has is being able to track who is in the Spec Core Team via github teams instead of a config file.
The Spec Core Team is working on MSC2260, MSC2261, and a QR-via-SAS thing which uhoreg is currently working on (this means less complicated, easier-to-scan QR codes for verification, which may have the side effect of FOSDEM demos being slightly less hilarious).
FOSDEM happened, Matrix had a great time! Was awesome to see so many people at the stand, and the talks, and even at our social on Saturday night.
Congratulations to everyone who was able to buy a limited edition Matrix FOSDEM 2020 t-shirt! Check out our round-up here.
Quite a lot has been happening in the Dendrite space, both in terms of general development, usability improvements and also further P2P experimentation post-FOSDEM. The main highlights are:
We have decided to require Go 1.13 for Dendrite going forward (our CI and tools have been updated to reflect this requirement)
Some federation testing has taken place between Dendrite and Synapse and it's looking reasonably good bar a few bugs
Initial support for distinguishing room versions has been added - this is the foundation work for being able to implement state resolution v2 and later room versions soon
The /capabilities endpoint has been implemented and now contains information about m.room_versions
Account data GET endpoints have been implemented
Guest registration should now be working - thanks to prateek2211 for the contribution!
The public rooms API is now wired up for roomserver events and (mostly) works for local rooms
Some more /sync fixes are in now
SQLite work is continuing thanks to Kegan's on-going efforts
We've also been really encouraged by the number of people at FOSDEM who came to speak to us about Dendrite and P2P. We're hoping to strategise further about the P2P work further in the coming weeks once Dendrite is in better shape, so watch this space!
This week we’ve been working on alias auth rules, acme v2 support and moving groups apis onto a worker to aid performance. Sorry matrix.org has been a bit slow these past few days, we’re working on it and shipped some fixes earlier today.
Coming up are more perf improvements, more alias rules, getting sydent running in production on python 3. We’ll also ship v1.10.0 which contains some cross signing bug fixes.
Good people, 0.15.0 is out!. Please go ahead and read the changes and see if there is something you like :). We will be shipping this out to all the other bridges in due course :)
I released v0.7.1 that includes some minor bugfixes, then added a !tg backfill command to backfill all history since the last bridged message. Future improvements may include more control over how many messages to backfill, automatically backfilling for new portals and freezing incoming message handling when a backfill is in progress.
There's a new maubot plugin to post http.cat pictures to a room. It's installed into Cat Disruptor 6000 and can of course be self-hosted too. It has one command: !http \<status\>, that gets https://http.cat/<status>.jpg and posts it to the room.
Some time ago I also made this thing: https://github.com/tulir/mautrix-manager. It might theoretically do something useful in the future, such as allowing users to log into bridges from a single website instead of by talking to many bots. Currently it only does telegram and maybe facebook, but at least it has a fancy login UI. It could also support being embedded into Riot as a widget and/or integration manager at some point, since I made the auth thing with Matrix's OpenID feature and MSC1961.
The Quotient project is back from hibernation now that its lead developer settled in another half of the globe; commits and PRs start moving around again. Thanks to all those patiently waiting!
Riot-Android (yes, the legacy app!): We have fixed the issue with SSO login with Firefox accounts, and issue with Video Call. Expect a release at the beginning of next week.
RiotX: we are still stabilizing the cross-signing implementation. Many other fixes are coming. Also SSO issue with Firefox accounts has been fixed, and we will also release a new version of the application next week.
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
We're just back from an incredible time at FOSDEM 2020 - Europe's biggest Free & Open Source Software conference. Huge huge thanks to everyone who came to our talks (sorry if you couldn't get in :/), came to talk to us at the stand, or flagged us down to give feedback, chase PRs, file bugs, or just say thanks. Thanks also to FOSDEM to accepting all of our talks this year, and to the FOSDEM organisers for pulling together yet another amazing event :)
We'll do a proper blog write-up on enabling E2E encryption by default, cross-signing, and all the other E2E encryption work that's been going on once we ship the stable release - but as of Saturday(!) it has landed on Riot/Web Develop, RiotX/Android (0.14.2) and Riot/iOS develop TestFlight, but we're still debugging and we need a bit longer before cutting the final releases.
So, until then, please take a look at the videos if you missed the livestream or weren't at the event!
This weekend Matrix will be well represented at FOSDEM! We'll have a stand (come visit!), but also note that we have THREE speaking slots (all on Sunday):
This week we’ve been continuing our E2EE UX bug hunt, getting Sydent ready for python 3, working on alias rules (MSC2260, 2261) and fixing some bugs in our ACME support.
Next week, we’ll ship the new improved Sydent, keep going with alias rules and dust off our master process sharding plans.
Just pushed updated 1.9.1 tags for the K8s-optimized Synapse images, including support - and example config - for running with securityContext features enabled. With a separate media repo, it's now fully possible to run these images without root, and in completely read-only environment.
The synapse 1.9.1 release images are available on avhost/docker-matrix:v1.9.1 and mvgorcum/docker-matrix:v1.9.1 The latter image also contains simple-antispam
way more tests (still not good enough test coverage, though!)
enhanced reaction support (reaction deletions/redactions are now properly bridged from and to matrix)
presence status now sets im.vector.user_status event to be in line with riots status labs feature (why doesn't riot just use the spec on presence status here?!)
add a message deduplicator class that protocol implementations with echo back can easily implement
fix various small bugs
As always, if you have any questions check out the support channel and please consider to donate! (Thanks sooooo much to everyone who donated so far!) 🦊
matrix-slack-parser is, similar to matrix-discord-parser, a parser for slack->matrix and matrix->slack messages. It has been created over this week and does proper parsing (meaning HTML parsing and slack blocks, etc.). mx-puppet-slack already implements this, which leaves matrix-appservice-slack to follow suit! (cadair expressed interest, so looking forward to that!)
Nheko has added support kicking, inviting and banning/unbanning via the commands /kick, /invite, /ban and /unban with an optional reason specified after the userid. Kick and Ban buttons were also enabled for the userprofile. Furthermore some shortcuts for switching rooms (Ctrl+Up/Down) and paginating the timeline (PgUp/PgDown) were added. The shortcuts may change in the future though, so if those don't work for you, please give us feedback in #nheko-reborn:matrix.org or via the issue!
I started a new Pluggable Chatbot (PCB) written in Go. Still early development, but working hard on it. Bot is pluggable, meaning more functionality can be easily added to it. Check it out at https://github.com/srados/pcb.
Currently there are two examples available: echo and uppercase.
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
I know it's a bit naughty to post rooms in here, but for developers looking to play around with Vue.js (with matrix), we've created a support room over at #vue:half-shot.uk
A quick bug fix release that affects admins making use of monthly active user limits. You will know if you are affected because 1.9.0 will not start up. Apologies if you were bitten by this one!
Get the new release from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.
A fairly big PR was merged into Dendrite yesterday (and some more fixes today) which gets /sync working better and adds the /messages endpoint, so it's now actually possible to paginate around in channels and to see room history. This massively improves the user experience when logging into a Dendrite homeserver.
In addition, another small PR was merged to fix Riot user login in the normal login workflow (not registration) and the public rooms API has been wired up for roomserver events.
A couple of changes have also been merged into gomatrixserverlib, mostly fixing the creation of /backfill requests and allowing the use of custom http.Transports for federation traffic.
This week we shipped 1.9.0 which contained new admin API endpoints as well as bug fixes for existing ones. Aside from that, we’ve been continuing on our E2EE UX bug hunt and python3ing sydent.
Next week, we’ll keep going on the e2ee ux bug hunt and sydent work. As well as dusting off MSCs 22602261 to improve room aliases management.
Hello wannabe IRC and Matrix users! Today the bridge team bring you the 0.14.X series of the irc bridge. This is the mammoth release featuring all the Typescript changes and postgres stuff. The Postgres changes aren't as essential as they are for Synapse, but useful if you run a ~20k Freenode bridge.
The highlights are:
Typescript is now used everywhere across the project, bringing a range of refactors.
Postgresql support has landed, while still retaining NeDB support for the present.
Storing your password will now cause a seamless reconnect to the IRC network, allowing you to set a password without having to bother NickServ.
Sentry.io support has landed, to trace errors in the bridge.
Added a reapUsers API endpoint, which allows the bridge to automatically prune users who have not interacted with the bridge for a long time.
Some of these features are still more experimental than others, so the next release hopes to clean up a lot of the outstanding bugs. That's all from us for now :)
matrix-discord-parser is the joint matrix->discord and discord->matrix parser that both mx-puppet-discord and matrix-appservice-discord use. For sending messages matrix->discord optional auto-detection of code language for code blocks using highlight.js has been implemented.
Always wanted to host your own version of Riot Web locally but never came around doing it? Meet Riot WebExt, an unofficial proof of concept Browser Extension that bundles Riot Web, so everything is locally served with the click of a button. Available for Firefox and Chrome!
Responses of the sync API can be received continuously using Flow (async lazy sequence) now. Compared to previous method which used Channel, Flow can be transformed and combined in a much more flexible way, and Flow stops when it's no longer being consumed, so no coroutine will be left running accidentally.
This week, in the cross-signing side, we updated e2e decoration, replacing padlocks with shields. We started working on the new device list in the room member screen.
RiotX: We are making progress on cross-signing and verification by QRCode. Also we are working to improve the network detection and paused sync thread issue. But the most important is: "/rainbow" command will be available soon 🌈 !
We've done a lot this week, and probably even more next week:
Continuing on the cross-signing front with support for QR code verification, verification in the right panel, a bunch of shields to indicate trust, some bootstrapping in the secret store, and more!
A release candidate for 1.5.8-rc.2 (please give this a test on riot.im/staging)
An all-new invite dialog for creating DMs and bringing people into rooms.
tsw from Tampere Hacklab has made a read-only client for information displays. It's still in early development but can already be tried out. https://github.com/tswfi/matrix-viewer/
Are there any French speaking Sys Admins reading this? If that's you then I want you to know that https://apply.workable.com/new-vector/j/766008FB0D/ exists and if you are interested to know more you should DM me. Remote is fine - though full time only.
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Synapse 1.9.0 contains a bunch of new admin APIs as well as bug fixes to existing ones. In particular we have a new end point that allows admins to query which rooms their server participates in.
Aside from that it is worth noting that we have officially dropped support for SQLite versions < 3.11. Though more generally we would recommend that all admins migrate to Postgres for anything other than test and evaluation purposes.
Get the new release from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.
WARNING: As of this release, Synapse no longer supports versions of SQLite before 3.11, and will refuse to start when configured to use an older version. Administrators are recommended to migrate their database to Postgres (see instructions here).
If your Synapse deployment uses workers, note that the reverse-proxy configurations for the synapse.app.media_repository, synapse.app.federation_reader and synapse.app.event_creator workers have changed, with the addition of a few paths (see the updated configurations here). Existing configurations will continue to work.
Fix a bug causing Synapse to not always purge quiet rooms with a low max_lifetime in their message retention policies when running the automated purge jobs. (#6714)
Fix the synapse_port_db not correctly running background updates. Thanks @tadzik for reporting. (#6718)
Fix monthly active user limiting support for worker mode, fixes #4639. (#6742)
Fix bug when setting account_validity to an empty block in the config. Thanks to @Sorunome for reporting. (#6747)
Fix AttributeError: 'NoneType' object has no attribute 'get' in hash_password when configuration has an empty password_config. Contributed by @ivilata. (#6753)
Fix the docker-compose.yaml overriding the entire /etc folder of the container. Contributed by Fabian Meyer. (#6656)
Synapse no longer supports versions of SQLite before 3.11, and will refuse to start when configured to use an older version. Administrators are recommended to migrate their database to Postgres (see instructions here). (#6675)
Don't block processing of incoming EDUs behind processing PDUs in the same transaction. (#6697)
Remove duplicate check for the session query parameter on the /auth/xxx/fallback/web Client-Server endpoint. (#6702)
Attempt to retry sending a transaction when we detect a remote server has come back online, rather than waiting for a transaction to be triggered by new data. (#6706)
Add StateMap type alias to simplify types. (#6715)
Add a DeltaState to track changes to be made to current state during event persistence. (#6716)
Add more logging around message retention policies support. (#6717)
When processing a SAML response, log the assertions for easier configuration. (#6724)
Sometimes it seems like existence is a constant loop. Cycles within cycles are replayed, with only minor differences to help us discern the current and the foregoing.
How can we endure it?
Matrix will be applying to be a Mentoring Organisation, as we have for several years now. This year, we'd like to open the umbrella a little. If you think your (Matrix-related) project could help mentor a student, and that you could offer a meaningful project to that student, please contact me directly. For reference, there were four students under the Matrix heading last year, of which one was assigned to work with Kitsune on encryption support in libQuotient.
Will you be at FOSDEM this year? Matrix will be very well represented, so please come and see us at our stand, plus attend all of the three talks we'll be conducting. Many more details to follow!
I've spent some time this week experimenting with integrating libp2p into dendrite and gomatrixserverlib to understand if we can get link-local homeserver discovery working along with direct or indirect federation. It's still early at the moment but I have had success with early federation requests working over libp2p—hopefully I'll have something interesting to show soon!
This week we’ve been continuing to work on migrating sydent to py3, improving admin tooling and working on fixing e2ee bugs. We’re also looking to make changes to allow admins the power to remove room aliases (MSCs 2260 2261).
matrix-imposter-bot - A bot that uses your account to repeat other people's messages. This gives relay-bot capabilities to puppet-only bridges. For more details, see https://github.com/mrjohnson22/matrix-imposter-bot
I added pretty rendering of replies to nheko. It uses the same rendering as for normal messages, so you will see an image being replied to and you can even play a replied to video. You can also click on the reply to navigate to that part of the history.
RiotX: Release v0.13.0 has been uploaded today to the PlayStore and should follow in F-Droid store. It contains typing notification feature, and new screens for Room profile and room members list, and new screen for Room Member profile. Those new screens are still missing a lots of features, that will be implemented in the next weeks. For F-Droid version, it contains a fix on the broken background sync. We are also making big progress on cross-signing implementation, and we have started to write integration tests on the SDK.
Update network library ktor to latest release 1.3.0
Add option to use authentication with GET /_matrix/client/r0/publicRooms to fix HTTP Unauthorized errors with newer Synapse versions which may have allow_public_rooms_without_auth set to False.
matrix-docker-ansible-deploy can now easily configure automatic Double Puppeting for all of its Mautrix bridges (Facebook, Hangouts, Whatsapp, Telegram). This is possible due to those bridges' integration with matrix-synapse-shared-secret-auth - yet another component that the playbook can install. Our documentation page for each bridge provides relevant information on how to get it enabled.
My Grafana notification ingress gem received the missing piece of functionality for configuring it for posting both m.text as well as the original m.notice messages (as well as defaulting to m.text now as the more common expectation) - for those that want the messages to also provide Matrix notifications instead of just acting as an added history/flow.
This can also be configured per-rule, so that you are able to have multiple notification ingresses configured with different message types.
It's always so lovely when people - other than myself - use my projects, especially when providing feedback on them. So many simple-to-implement features that get forgotten just because they weren't part of the original MVP.
TWIM bot has been updates to use all the new features of opsdroid 0.17 it now supports Edits, both in updating it's database and by editing messages which get posted to #twim_updates:cadair.com . Also it supports 'TWIMing' a post by the user who posted it reacting with ⭕️, this is mainly designed as an easy way to mark images as TWIM posts. 🚀
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
We've had a slight lull from people crawling out from winter holiday hibernation caves, but there's likely more to come as everyone gets back into the swing of things.
Synapse 1.8.0 has arrived, it contains a whole host of bug fixes and tweaks, most notably fixing some long standing problems with search.
More generally we are spending a lot of time improving the e2ee experience ahead of switching on e2ee by default, so watch this space.
Federation fixes have been pushed to gomatrixserverlib and Dendrite's master branches, that include the following tweaks:
Using the v2 endpoints for /send_join and /send_leave
Fixing the resolution of room aliases to room IDs in the Dendrite federation API
A rewrite of the auth chain functions which should be a bit smoother
A couple of other minor tweaks to some of the types
In addition, I've spent the last few days working on Dendrite's storage backends, adding somewhat-hacky support for SQLite and investigating ORM modelling for some of the simpler components, as a part of getting Dendrite to run as a "true monolith" for the P2P work.
🔗cortex (synapse worker in Rust) supports e2ee rooms
Use the Zapier App to integrate your Matrix room with any of Zapier's 1500+ apps, including:
Github
Trello
Pagerduty
Google calendar
Jira
Salesforce
Currently the MVP supports sending messages to a room. Reading messages from a room is future work.
If you're interested providing feedback, requesting a feature, future development, or just understanding how it works, please join the matrix room, #zapier:matrix.org
matrix-zammad now supports reacting to tickets to close them, provided you've set up your puppeted token in the config. It's useful if you get a lot of spam tickets :p
Work has resumed on matrix-presents, newly rewritten in Vue.js! The project was first demoed back in Oggcamp 2018 where I gave a meta presentation on the virtues Matrix for other mediums. This time around, it's being brought back with:
A user interface for managing, joining and creating slideshows.
Control over how slides are advanced (pinned to the presenters view, or unlocked)
Finer control over how slides are laid out, using a fragments system to build slides out of submessages.
This is currently in heavy development and will debut at Fosdem 2020!
RiotX v0.12.0 has been released with some performance improvement, especially on initial sync and on timeline loading time. The release also contains bugfixes and a cleanup in the application settings. A new "developer mode" has been added to show advanced features only to power users. We are now working on the room profile screen, and we are making progress on the cross-signing implementation. Besides that, we are working to make RiotX available on the F-Droid store.
I must say RiotX is getting a lot more stable and reliable recently!
In addition to mentioning Synapse 1.8.0 support, Ananace said:
Synapse 1.8.0 Kubernetes-optimized images are pushed, I've also updated the example manifests as part of some work on making it easier to deploy - expect a MVP of a Helm Chart some time Soon™
Also, to add a bit of context/information to this;
Helm is the de-facto standard package manager for Kubernetes clusters, where a Chart is a package for an application that can be installed and configured. The Helm Chart I'm working on won't be a one-click install to begin with, for the Minimum Viable Product it will require manually generating and storing the Synapse signing key - though I have thoughts on how to later delegate that to a small one-time job that Helm can run if necessary
Opsdroid 0.17 was released in December, it comes with many changes but the main matrix improvements are support for extra event types like Replies, Edits, Reactions, Room Name changes, Room Topic changes, Alias changes, Room avatar changes, Power levels, and support for generic matrix state events. All these events are supported for both sending and implementing skills based on receiving them.
Let's reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Add option limit_profile_requests_to_users_who_share_rooms to prevent requirement of a local user sharing a room with another user to query their profile information. (#6523)
Add an export_signing_key script to extract the public part of signing keys when rotating them. (#6546)
Add experimental config option to specify multiple databases. (#6580)
Raise an error if someone tries to use the log_file config option. (#6626)
Add experimental support for multiple physical databases and split out state storage to separate data store. (#6245, #6510, #6511, #6513, #6564, #6565)
