Synapse 1.3.1 released

2019-08-17 — General — Neil Johnson

Some of you will have been bitten by a bug that prevented Synapse 1.3.0 from starting up correctly. If this is you, please upgrade.

Additionally we have taken the opportunity to fix a dependency bug for our intrepid packagers.

Apologies for the inconvenience.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page

The changelog since 1.3.0 follows:

Synapse 1.3.1 (2019-08-17)

Features

Drop hard dependency on sdnotify python package. (#5871)

Bugfixes

Fix startup issue (hang on ACME provisioning) due to ordering of Twisted reactor startup. Thanks to @chrismoos for supplying the fix. (#5867)

This Week in Matrix 2019-08-16

2019-08-16 — This Week in Matrix — Ben Parsons

Matrix Live 🎙

In which Matthew talks about:

Synapse
perf speedup
installer
turn tester
extremities fix by default
security work
privacy work
Riot
privacy; IS and IM permissions and management. Hashed lookup API. Metadata cleanup (GCing redactions; deleting media; history retention); TURN management
aggregations paused
FTUE next up
Cross-signing
Immutable DMs
...then communities
RiotX
0.3 came out last week - handles DMs; lots of polishing; reduces the APK down to ~10MB!
Bridges
moving stuff into proper managed infrastructure
bifrost should be back soon?
gitter kept breaking but we're actually working with their team to fix that
working out how to automate deploying them on modular hosts

Dept of Spec 📜

Approved MSCs

No MSCs have been approved.

Final Comment Period

In Progress MSCs

Dept of Servers 🏢

Ruma

Jimmy reported:

This Week in Ruma: https://www.ruma.io/news/this-week-in-ruma-2019-08-11/

He was also a guest of the Rustacean Station podcast this week:

Ruma and the Matrix Communication Protocol: An Interview with Jimmy Cuadra (Rustacean Station podcast): https://rustacean-station.org/episode/001-ruma/

Construct CS API progress

Jason said:

Construct implemented event relations from the client-server API. Now clients can view edit history of messages.

Learn more in #construct:zemos.net

Dendrite

anoa:

Progress for Dendrite this week has been a bit starved due to ongoing privacy work from the backend team. However it continues to move forwards with the excellent help of cnly reviewing and merging PRs.
We had 3 authors have pushed 5 commits to master and 6 commits to all branches. On master, 14 files have changed and there have been 298 additions and 121 deletions.
Notably we had trion129’s work on providing a fallback webpage for recaptcha authentication finally merged! This was another early PR that was dragged up from the bottom of the list to be modernised and integrated into the current codebase.
Also this week was some work from cnly on refactoring gomatrixserverlib and exposing some more of its internal constants to deduplicate code between itself and Dendrite.
Finally, we had some more work on an unmerged PR from cnly, which adds some internal query APIs for which servers are currently joined to a room, for use by other components later down the line.
No new Sytests are passing this week. We’re still currently at 172 tests passing out of 761 tests in total.

Matrixmon updated

Matrixmon, the perl C2S testing utility has been updated to v0.2.0 with support for SSL connections.

Synapse

Synapse 1.3.0 was released - check out the blog post.

contains performance improvements to reduce disk I/O and reduce RAM usage. We’ve been running it on matrix.org for a week or so and are really pleased with the results.

Ananace's k8s tooling, Slavi's matrix-docker-ansible-deploy and andrewsh's Debian packages were all updated within an hour of release, which is a great community achievement. 👏

Dept of Bridges 🌉

Major mautrix-* bridges update

tulir said:

mautrix-telegram got some fixes to bugs that appeared during the move to mautrix-python. It also got a manhole similar to the one in synapse, which gives admins access to an interactive Python shell inside the bridge while it's running. The main part of the manhole (server and repl) is in mautrix-python, so you can expect the feature to appear in mautrix-facebook, mautrix-hangouts and possibly maubot in the future.
mautrix-whatsapp and mautrix-hangouts can now automatically add rooms to a user-specific community to help with filtering. mautrix-facebook is also getting that feature in the near future.
mautrix-facebook now has a search command, which means you can now actually initiate conversation with facebook users.

Dept of Clients 📱

Riot Web

From the team:

Privacy work continues
STUN fallback server for assisting with voice & video call negotiation (only used when your homeserver is not configured with it’s own TURN server) changed from Google to Matrix.org server and a prompt has been added to request permission before using the fallback server
Many small steps towards supporting user choice of identity servers and integration managers, including no identity server at all

Bluepill (Sailfish client) status update

Cy8aer said:

Users can now download artifacts from my gitlab account since I got an SDK container from CoDerus running, cross compiling to Sailfish-RPMs in the Gitlab-Ci. But my programming progress on master looks a bit silent because I swap to matrix-nio.

Fractal

Alexandre Franke told us:

Fractal nightlies are building again.

Continuum

yuforia told us:

Continuum:
Reimplement ScrollBar for the message list view to make it more consistent with Riot Web. It has a flat appearance now and only appears when the mouse hovers over the list of messages. https://matrix.org/_matrix/media/r0/download/matrix.org/kVfBsoAsguMmESNThxnUmaTM
Rewrite the list view partially to handle lists with several types of items more efficiently. The message list has text messages, images, membership updates, etc. Each type needs to be rendered differently, but items of the same type could reuse GUI components when they scroll into and out of view.

Dept of Ops 🛠

K8s optimized images of Synapse RCs

Ananace offered:

Since the coming 1.3.0 release is bringing along a whole bunch of perf improvements, (and I want to test them out on my IO starved setup) I'm building K8s optimized images of the RCs. Only the debian version of the image though.

They expanded:

It's basically the regular docker image, only with some changes to make it fit nicer in Kubernetes
And there's no Helm chart at the moment, was poking at that for a bit but took a break from building one as I - back then - had trouble with the amount of data that had to be generated
You can only have one Synapse pod at any given time, as it's not scalable. Though you can scale it with workers (as I do myself)
Database-wise, I'm personally running three node HA clusters using Stolon

Dept of Services 🚀

New public instance: Permaweb.io

swedneck said:

We run a public instance on Permaweb.io for everyone to join. We’re a large community of developers focusing on the decentralized web, with lots of channels dedicated to IPFS. In fact, we even host Riot on IPFS! We make full use of Matrix’s bridging capabilities, to Discord, IRC, Slack, Gitter and Telegram too! Come join us at #general:permaweb.io or register at https://riot.permaweb.io/ (or any other compatible client) and say hi!

Dept of Bots 🤖

Notification/Remind-Me-Bot using matrix-js-bot-sdk

Anton (@antonivan:matrix.org) is working at Matrix Towers this summer for work experience. He's been learning about Matrix and matrix-js-bot-sdk:

This week I made a Notification/Remind-Me-Bot using Travis's bot sdk, it took me 1 and half days, and is able to - set reminder, check reminder, cancel reminder, set interval reminder, cancel interval reminder. It is also able to store multiple reminders and cancel them using the numbers 1 - ∞ . Here is a link to my Github repo - https://github.com/joakimvonanka/Matrix-Remind-me-bot

Dept of Event Videos 📹

Matrix talk @ FrOSCon

Oleg's talk at FrOSCon was apparently really well received:

Video recording from the Matrix talk @ FrOSCon. Video is in German slides in English.
Video: https://media.ccc.de/v/froscon2019-2330-matrix
Slides: https://programm.froscon.de/2019/system/event_attachments/attachments/000/000/579/original/presentation.pdf
Channel: #FrOSCon:fiksel.info

He's also thinking of writing a guide to creating presentations:

If someone is interested on an efficient way of creating presentation using only open source tools I'm planing to write an article on that.
Small spoiler: matrix is one of them 😉

Final Thoughts 💭

This week, I have been mostly thinking about... merch. We blew the cache on stickers, so if you are waiting for stickers specifically and have not heard from me, that is probably why. Please contact me with any questions!

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.3.0 released

2019-08-15 — General — Neil Johnson

Well now, Synapse 1.3.0 is here.

The main thing to know about 1.3.0 is that is contains performance improvements to reduce disk I/O and reduce RAM usage. We’ve been running it on matrix.org for a week or so and are really pleased with the results.

Check out our message send heat map.

Message Send

Other than that there are a bunch of bug fixes and tweaks to generally make things run more smoothly.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Also, check out our Synapse installation guide page

The changelog since 1.2.1 follows:

Synapse 1.3.0 (2019-08-15)

Bugfixes

Fix 500 Internal Server Error on publicRooms when the public room list was cached. (#5851)

Synapse 1.3.0rc1 (2019-08-13)

Features

Use M_USER_DEACTIVATED instead of M_UNKNOWN for errcode when a deactivated user attempts to login. (#5686)
Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify. (#5732)
Synapse will no longer serve any media repo admin endpoints when enable_media_repo is set to False in the configuration. If a media repo worker is used, the admin APIs relating to the media repo will be served from it instead. (#5754, #5848)
Synapse can now be configured to not join remote rooms of a given "complexity" (currently, state events) over federation. This option can be used to prevent adverse performance on resource-constrained homeservers. (#5783)
Allow defining HTML templates to serve the user on account renewal attempt when using the account validity feature. (#5807)

Bugfixes

Fix UISIs during homeserver outage. (#5693, #5789)
Fix stack overflow in server key lookup code. (#5724)
start.sh no longer uses deprecated cli option. (#5725)
Log when we receive an event receipt from an unexpected origin. (#5743)
Fix debian packaging scripts to correctly build sid packages. (#5775)
Correctly handle redactions of redactions. (#5788)
Return 404 instead of 403 when accessing /rooms/{roomId}/event/{eventId} for an event without the appropriate permissions. (#5798)
Fix check that tombstone is a state event in push rules. (#5804)
Fix error when trying to login as a deactivated user when using a worker to handle login. (#5806)
Fix bug where user /sync stream could get wedged in rare circumstances. (#5825)
The purge_remote_media.sh script was fixed. (#5839)

Deprecations and Removals

Synapse now no longer accepts the -v/--verbose, -f/--log-file, or --log-config command line flags, and removes the deprecated verbose and log_file configuration file options. Users of these options should migrate their options into the dedicated log configuration. (#5678, #5729)
Remove non-functional 'expire_access_token' setting. (#5782)

Internal Changes

Make Jaeger fully configurable. (#5694)
Add precautionary measures to prevent future abuse of window.opener in default welcome page. (#5695)
Reduce database IO usage by optimising queries for current membership. (#5706, #5738, #5746, #5752, #5770, #5774, #5792, #5793)
Improve caching when fetching get_filtered_current_state_ids. (#5713)
Don't accept opentracing data from clients. (#5715)
Speed up PostgreSQL unit tests in CI. (#5717)
Update the coding style document. (#5719)
Improve database query performance when recording retry intervals for remote hosts. (#5720)
Add a set of opentracing utils. (#5722)
Cache result of get_version_string to reduce overhead of /version federation requests. (#5730)
Return 'user_type' in admin API user endpoints results. (#5731)
Don't package the sytest test blacklist file. (#5733)
Replace uses of returnValue with plain return, as returnValue is not needed on Python 3. (#5736)
Blacklist some flakey tests in worker mode. (#5740)
Fix some error cases in the caching layer. (#5749)
Add a prometheus metric for pending cache lookups. (#5750)
Stop trying to fetch events with event_id=None. (#5753)
Convert RedactionTestCase to modern test style. (#5768)
Allow looping calls to be given arguments. (#5780)
Set the logs emitted when checking typing and presence timeouts to DEBUG level, not INFO. (#5785)
Remove DelayedCall debugging from the test suite, as it is no longer required in the vast majority of Synapse's tests. (#5787)
Remove some spurious exceptions from the logs where we failed to talk to a remote server. (#5790)
Improve performance when making .well-known requests by sharing the SSL options between requests. (#5794)
Disable codecov GitHub comments on PRs. (#5796)
Don't allow clients to send tombstone events that reference the room it's sent in. (#5801)
Deny redactions of events sent in a different room. (#5802)
Deny sending well known state types as non-state events. (#5805)
Handle incorrectly encoded query params correctly by returning a 400. (#5808)
Handle pusher being deleted during processing rather than logging an exception. (#5809)
Return 502 not 500 when failing to reach any remote server. (#5810)
Reduce global pauses in the events stream caused by expensive state resolution during persistence. (#5826)
Add a lower bound to well-known lookup cache time to avoid repeated lookups. (#5836)
Whitelist history visbility sytests in worker mode tests. (#5843)

This Week in Matrix 2019-08-09

2019-08-09 — This Week in Matrix — Ben Parsons

Matrix Live

Something a little different this week: Erik helped me install Prometheus logging on my own homeserver, and we recorded it for Matrix Live.

Interesting reading: https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.rst

Dept of Status of Matrix 🌡

The Official Merch Store

<blink>YOU NEED MATRIX MERCH</blink>

Yes, it's true: Official Matrix Merch is available now. There are stickers, t-shirts and hoodies all available now, go and claim yours from https://shop.matrix.org.

stickers

<blink>CONSUME</blink>

Free Open Source Conference appearance from Oleg

Oleg:

I will be doing a talk on Matrix this weekend (tomorrow) at the annual Free Open Source Conference (FrOSCon) in Sankt Augustin (Germany). If you are around - come by and get some awesome [matrix] stickers! =)

https://programm.froscon.de/2019/events/2330.html

Visit #FrOSCon2019:froscon2019.fiksel.info to talk about the event.

Dept of Spec 📜

Approved MSCs

[MSC 2134]: MSC2134: Identity Hash Lookups 🎉

Final Comment Period

No MSCs have entered FCP

In Progress MSCs

[MSC 2213]: Rejoinability of private/invite-only rooms [MSC 2214]: Joining upgraded private rooms

Dept of Servers 🏢

Dendrite

Huge update from Andrew:

This week’s stats for Dendrite are in! 4 authors have pushed 11 commits to master and 11 commits to all branches. On master, 48 files have changed and there have been 676 additions and 146 deletions.
Of note are the following merged PRs:
Implement client single event retrieval - #671
You can now request a single event from Dendrite’s DB, assuming you have permission to view it.
Account data events filtering database API and State events filtering database api
Very old PRs from CromFR have been updated and finally merged! These lay the groundwork for providing filters across various API endpoints (used in search, /sync, etc)
Add typingserver service to docker-compose
Another fairly old PR by viccuad (their first for Dendrite!) finally reviewed and merged.
Fix permission and 404 response for alias deletion - #654, Replace membership and visibility values with constants, Implement profile retrieval over federation - fixes #651 (this one is very useful), and Fix transaction IDs in transaction cache have global scope
A myriad of changes and fixes by cnly
We also have a number of PRs that were moved forward but not merged. Add auth fallback endpoint (our oldest PR at the moment!) which we mentioned last week is so very close, but not quite there in time for this TWIM.
In terms of passing Sytests, Dendrite now passes 172 tests, up from 167 last week. This is out of 761 tests in total. So just a small bump (0.6%), but note that things that are crucial for Dendrite development at the moment, such as federating, only cover a small subset of tests, so each passing one can bring us much closer to getting Dendrite usable day-to-day.

Synapse

From Neil:

The big news this week we’ve landed some massive DB improvements (https://github.com/matrix-org/synapse/pull/5706) which should make message sending on matrix.org (and any other server) feel noticeably snappier. :) It'll be in the next release.
Outside of perf, we’ve also been continuing with our privacy project, sydent now supports hashing third party identifiers instead of storing the raw form, and in the coming week we’ll look at minimising the meta data we store. You can track our progress across the full project here (https://vector-im.github.io/feature-dashboard/#/plan?label=privacy-sprint&repo=matrix-org/synapse&repo=vector-im/riot-web&repo=vector-im/riot-ios&repo=vector-im/riot-android&repo=vector-im/riotX-android&repo=matrix-org/matrix-doc&repo=matrix-org/sydent)
Work continues on our installer to make it easier to configure Synapse and we’ll be looking for some community feedback rsn. The room directory revamp is now very close and we hope to have something live on matrix.org in about a week.
Finally we’re working on improving the efficiency of smaller instances sending messages into large rooms, we’re still at the design stage, but it will make a huge difference for anyone self hosting.

andrewsh also informs us:

synapse in up-to-date in Debian unstable — and in buster-backports (uploaded last week)

Ruma

This Week in Ruma: https://www.ruma.io/news/this-week-in-ruma-2019-08-04/

Also, get live-streamed Ruma development: https://www.jimmycuadra.com/posts/twitch-live-streams-of-ruma-development/

Does the video include time spent waiting for async to land in Rust?

signaller - new "lightweight server"

ChronosX88 announced that he is "developing a lightweight Matrix server" - check on their progress at https://github.com/signaller-matrix/signaller, also join #signaller_dev:netwhood.online.

TravisR announces various unstable interesting things

TravisR:

If you fancy running highly experimental software on your homeserver which might break things, I have two things for you:
matrix-key-server ( #matrix-key-server:t2bot.io ) is an implementation of a Matrix key server with notary support. Although it follows the specification, it has not been tested against Synapse. It is however fully featured: check it out on the federation tester ( https://federationtester.matrix.org/#keys.t2host.io ) or by querying it yourself ( https://keys.t2host.io/_matrix/key/v2/query/matrix.org ).
matrix-room-directory-server ( #matrix-room-directory:t2bot.io ) is less experimental than the key server but is still very early days. Currently it only offers the ability to manipulate the federated public room directory for your server, but in future it is planned to be its own standalone directory server (room aliases without having to run a whole homeserver). Check it out by searching the t2bot.io room directory from your client.

Dept of Clients 📱

Continuum progressing!

I love getting updates from yuforia - they've been consistently working on Continuum for some time, and by increments are making a great client.

Reuse the ListView of messages across different rooms to reduce memory usage (Experimental) Remember and refocus the last read message, making it easier to go through all unread messages while switching chat rooms freely.

QMatrixClient is now Quotient

kitsune:

Renaming of QMatrixClient to Quotient has been finally merged to the master branch; Quaternion master uses it from now, too. Packagers are welcome to make test builds and report bugs in #quotient:matrix.org. libQuotient 0.6 beta is coming close now!

Riot iOS

Test on iOS 13 Beta. Beta 6 fixed most bugs discovered on Beta 5
Released 0.9.2 (Waiting for Apple review at the time of writing)
Working on privacy concern.

Riot Android

Working on privacy concern.

RiotX (Android)

Release of v0.3.0 on Friday: full changelog can be found here: https://github.com/vector-im/riotX-android/releases/tag/v0.3.0
We are working on read receipt management, it’s nearly functional

Dept of Identity 🛂

ma1sd 2.1.1

ma1uta announced ma1sd release 2.1.0:

Changes: reworked unbind (MSC1915).
Now ma1sd is compliant with the MSC1915 (unbind).

and then 2.1.1:

ma1sd release 2.1.1 with the security fix and more strict request validation, allows only requests from ma1sd's matrix domains (mirroring bind). Download links: https://github.com/ma1uta/ma1sd/releases/tag/2.1.1 (archives and the deb package) and https://hub.docker.com/r/ma1uta/ma1sd for docker image.

Dept of Bridges 🌉

mautrix-facebook, mautrix-telegram and the tulirverse

tulir has been making big updates to two of his bridges:

mautrix-facebook can now bridge formatting, mentions, replies and reactions in both directions.

There is also a logout command now.

mautrix-telegram's switch to mautrix-python is nearly finished (i.e. it didn't cause any errors for the past few days when testing in production). The main reason for the switch is using one Matrix library for all my python bridges. It also means the bridging code like double puppeting and command handling I shared between mautrix-facebook and mautrix-hangouts is now also used in mautrix-telegram.
Visible changes directly caused by using mautrix-python:
Logs are now colorful.
Python 3.5 is no longer supported.
The bridge will refuse to start without access to the base config file.
Other changes that happened during the switch:
Telegram "Saved Messages" can now be bridged even when using double puppeting.
Mentions on Telegram are marked as read when using double puppeting (messages were already being marked as read, but mentions weren't).
Also, this actually happened last week and the week before that, but anyway: I've moved the CI and docker registry of my active projects to a self-hosted GitLab at mau.dev. Specifically, the CI/docker registry for all four mautrix bridges and maubot and automatic builds for gomuks have been moved. My maubot plugins also have automatic .mbp builds in the CI. For the docker registry, prepending dock.mau.dev/ to the existing image names will work. The old places (docker hub, dl.maunium.net) won't get new builds anymore. The repos on GitHub are still the "canonical" repos, but they're mirrored more or less instantly with maumirror.

He adds:

mautrix-telegram will probably get some sort of history filling in the near-ish future

Also,

I'm going to add some way to put bridged rooms into personal communities for filtering purposes. Not yet sure if it'll be fully built into the bridges or some kind of an external script

IRC bridge RC

Half-Shot:

Hi folks, we've released a RC of the next irc bridge version. Check it out at https://github.com/matrix-org/matrix-appservice-irc/releases/tag/0.13.0-rc1

Email2Matrix

Slavi, famed for his ansible playbooks, announced:

I've just released Email2Matrix - an SMTP server that can relay incoming messages over to Matrix based on mappings defined in a configuration file.
While it can also be installed standalone and work with any Matrix homeserver (see its documentation), the simplest way to install it is using matrix-docker-ansible-deploy and its guide for Email2Matrix installation.
The configuration process (users, rooms, mappings) is quite manual, but it's simple and has worked well for me for the past year and a half. I've only just polished it up a bit, released and integrated with the Ansible playbook, so others could benefit from it too.

matrix-xmpp-filter

mijutu appeared, and announced:

I started a "matrix-xmpp-filter" project. It's like matrix-ircd, but with xmpp instead of irc. It can also filter messages by weekday, time and regular expressions. Target audience is Sailfish phone users, but it could be used with other xmpp clients too (no xmpp MUC support required). https://k2c42.dy.fi/matrix-xmpp-filter.git/ #matrix-xmpp-filter:ellipsis.fi.

matrix-github (GH issues bridge)

Half-Shot:

https://github.com/Half-Shot/matrix-github has had quite a few updates. Notably:
Support for storing users access tokens securely, using a bot command.
Multi-process bridging: The webhook portion of the bridge can be run in a seperate function if needed for performance.
Images and mentions now work in both directions
The readme is now actually helpful, and have also made *.sample.yaml files.
You can now use it via oauth

Dept of Ops 🛠

Ananace:

Currently working on a Puppet module to manage Matrix Synapse installs (both directly and through docker), not quite ready for prime-time yet but watch this spot.

Dept of Bots 🤖

Voice to text bot

progserega announced:

I develop matrix Bot, which converting voice messages to text. It use Yandex Speech API and Yandex API cloud (some as aws) for temporary store voice-data before converting. https://github.com/progserega/voice2textMatrix Bot have such logic:
user add bot to room (for example room with whatsapp bridge users - now bridge support adding bot to such rooms)
bot listen room, and when get voice-message - send it to Yandex-cloud for translate (now support only Russian language).
When translating is success - bot get result text and show it in room as notice, such: "Username said: text"
Bot also receive some command, which allow disable it in this room, or disable translating for user, which send command..

Seeing this, I wondered, "y tho?", luckily progserega was able to explain with a graphic! They say a picture paints a thousand words, and it's clear from the conversation below why he'd want to have speech-to-text capabilities.

Speech to Text

Dept of Services 🚀

The Modular Matrix -> Matrix migration tool now supports migration of power levels to the new user.

That's all I know 🏁

So that's all I have to say to you right now! See you next week, and be sure to stop by #twim:matrix.org with your updates!

Official Matrix Merch Now Available

2019-08-09 — Merch — Ben Parsons

Here at Matrix we’re frequently asked for official merch. While those who choose to sponsor Matrix development on Patreon have always been blessed with branded t-shirts, it was otherwise very difficult to obtain official merch.

Today, those dark times have ended, as we announce the Official Matrix Merch Store!

Now available from shop.matrix.org, we’re offering packs of Stickers, T-Shirts, and Hoodies!

Stickers are provided in three types: hexagonal Matrix and Riot stickers, and rectangular transparent Matrix stickers.

T-Shirts are available in all sizes and in Unisex and Womens editions. We’re using Gildan SoftStyle t-shirts, which are popular amongst printed shirts for being both soft and durable.

Matrix-branded hoodies are something very special. Superbly embroidered, they are comfortable and stylish. On a recent trip to San Francisco, a passerby stopped a Matrix guardian in the street and asked where they could buy a Matrix hoodie without knowing what Matrix was.

You really need one of these.

All merchandise proceeds go to the Matrix.org Foundation, which helps fund Matrix development!

The Official Matrix Merch Store is open now: shop.matrix.org!

This Week in Matrix 2019-08-02

2019-08-02 — This Week in Matrix — Ben Parsons

Matrix Live

I chatted to Rick about the new Small instances available on modular.im.

Dept of Spec 📜

Approved MSCs

No MSCs have been approved.

Final Comment Period

In Progress MSCs

(kudos to anoa for providing this well-formatted summary section)

KaTeX rendering support

Tim:

there's now a pull request in for full native math rendering support (via KaTeX), which will replace the shoddy bot that I made last week (or whenever)
https://github.com/matrix-org/matrix-react-sdk/pull/3251

Dept of Servers 🏢

Synapse

This week we’ve been working on implementing identifier hashing in sydent, a brand new installer for Synapse, we’ve trialled some new db perf improvements which look very promising (more on this soon) and finally very close to an all new implementation of room search.
Next week we’ll push out the all new Sygnal, continue with the installer and room search and pick up some tasks under our privacy umbrella to ensure that Synapse is not holding onto any data that it does not strictly need to.

Dendrite

anoa:

Dendrite progress marches forwards. This week we had:
Room tagging implemented (thanks peekay_46!)
A PR for redactions opened by our GSoC student cnly
A PR for authentication fallback (for when clients don’t have a web browser built in and want to use recaptcha from trion129
This has been in the works for a while, and just has a couple small changes left to go!
Two PRs for filtering database functionality from CromFr
A PR for fixing the scope of transaction IDs in Dendrite’s transaction cache from cnly
Cnly’s GSoC period ends on August 26th. He has been a massive boon to the project’s cadence so far and we hope he will continue even after GSoC ends when he has time :)

Dept of SDKs and Frameworks 🏗

koma, Kotlin Library

yuforia has this update about koma, the library used to power continuum:

Provide automatic deduplication and rate-limiting when downloading media resources. This improves performance of GUI applications, where avatars of many users may appear on screen at once, and some of them may be identical.

libQuotient

kitsune:

libQuotient's master branch now supports sending and receiving reactions, and receiving message edits. We also have another PR in the works from aa13q that would add support for events decryption, as a part of his GSoC endeavour.

Dept of Clients 📱

Fractal 4.2 released

Alexandre Franke announced a new edition of Fractal:

Nightlies of Fractal are currently stuck on a 2 weeks old build because of one of our dependencies… but the long awaited 4.2 stable release is out! You can get it out fresh from flathub as usual.

Changelog:

New features:

    Adaptive window, mobile friendly
    Window size and position are remembered
    Redesigned login
    Spellcheck
    Network proxy support
    Typing notifications
    Badges are shown for operators and moderators
    Keyboard shortcuts for easier navigation across rooms

Better uploads:

    Audio and video files are now tagged correctly
    Image files have a thumbnail
    Various tweaks to the file chooser

Bugfixes:

    Logs actually output something now
    A few issues with invites and direct chats have been resolved
    More reliable scrolling
    Some crashes fixed

Under the hood:

    Code refactor continues
    We’re now using Rust 2018
    Many improvements to the build system and CI

Neo huge update!

joepie91:

Lots of news about Neo!
There's now an additional maintainer (joepie91)!
A lot of the internals have been refactored, so they should be more maintainable and reliable going forward. Also some future work planned on refactoring the state handling using Redux, which should make it much faster to fix issues and add new features in the future. There are even some ideas already for a future plugin API!
Display has been improved in a lot of places. Error messages are now more clearly formatted, images in reply-quotes are now shown as thumbnails rather than just a filename, and the reply-to popup now shows the full event that you're replying to properly. The chat window now correctly sticks to the bottom when you're scrolled to the end and receive new messages, even when an image or video loads - though per-room scroll position restoration isn't done yet.
There's now an experimental compact mode! It uses a more IRC-client-like layout for messages, and generally just fits more content onto the screen. It can be enabled with an experimental flag. A screenshot of the compact mode can be found here.

You can try the compact mode at https://neo.pixie.town/app/?compactMode=true.

Riot Web

Message editing composer now supports undo/redo.

Riot iOS

Reactions: List of who reacted
Native Camera. The embedded camera view has been removed. There is now a new action “Take photo and video”
Privacy: We have been working on implementing terms to display before using integrations

Riot Android

VOIP / Remove default stun server
Make Integration Manager optional (disables widgets, calls)

RiotX (Android)

Feature / Direct Room Creation Flow
Improvement / Local echo of pending edits
Improvement / Seamless transition from timeline to fullscreen image preview
Bug fixes

Pattle

Wilko:

Pattle is now available on Google Play!

Dept of Bridges 🌉

Half-Shot:

Hey folks, first an update from the bridges integrations side. We've brought back snoonet and oftc on the integrations menu so you can once again connect matrix rooms to these networks. Other networks should work as standard.

matrix-appservice-irc

Half-Shot again:

On the matrix-appservice-irc side we've made a few fixes to the handling of IRC modes (things that handle how users should behave) when bridged to IRC. This should hopefully make opping and voicing users more reliable. There are a few other fixes in the pipeline too, so a release isn't too far off.

GitHub issue bridging !?

Half-Shot AGAIN:

I had an adventure this afternoon into the world of bridging again, and have made a Github to Matrix bridge. At the moment you can join aliases and saturate your homeserver with the entire history of a issue or PR. You can also chat to folks on issues in realtime.

Check it out at: https://github.com/Half-Shot/matrix-github

GitHub issue bridging

Dept of Ops 🛠

debian-based K8s-optimized image

Ananace:

Just pushed a debian-based K8s-optimized image as well, it's 150MB larger than the alpine one, but on the other hand it comes with jemalloc support so it's nicer on the RAM instead.

CLI federation tester

kai is someone with no fear of rapid iteration! Versions 1.2, 1.3, 1.4 and 1.5 of his script were made available this week:

I finished version 1.5 of my small command line based federation tester. The bash script relies heavily on popular Unix tools like awk, curl, jq, sed and so on. Compared to https://federationtester.matrix.org, the outputs are a little more detailed and additional tests are performed, especially on the supported SSL protocols. Currently, it has only been tested under Linux.
The current version improves the handling of port in the .well-known and DNS SRV configuration.
You can download it from #cli_federation-tester_de:matrix.chat-secure.de or directly via https://matrix.chat-secure.de/_matrix/media/r0/download/matrix.chat-secure.de/MDwYLdrxRdudmcgDzBEALCzN

Dept of Bots 🤖

jcgruenhage has been working on a bot in relation to the startup he's working with:

I made a bot for issuing Json Web Tokens to matrix users based on what homeserver they are on. It's implemented as a maubot plugin (best way to write bots right now) and licensed under the AGPLv3. You can find it over at https://gitlab.com/famedly/bots/jwt, and talk about it in #jwt-bot:famedly.de
As for the usecase, we want to give people from customer homeservers access to an API component, but don't want to maintain a separate account database. This way, they can request a token from the bot and then use that to interact with the API component