This Week in Matrix

335 posts tagged with "This Week in Matrix" (See all Category)

Atom Feed

This Week in Matrix 2020-09-18

18.09.2020 00:00 β€” This Week in Matrix β€” Ben Parsons

Matrix Live πŸŽ™

Dept of Status of Matrix 🌑️

Element review by TechRadar

TechRadar have published a fairly thorough Element secure messenger review. This covers the Web, Android and iOS clients, and appears to use the matrix.org server for signup.

The review is very positive, awarding 4.5/5, and concludes:

The Element messenger platform scores highly for its approach to security and its commitment to decentralization, and it's definitely going to be of interest to businesses wanting control over their own chats – as well as plenty of individual users as well.

Dept of Spec πŸ“œ

Spec

anoa told us:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus for this week, MSC1960 has made it into FCP, so this week our focus is MSC2414.

2020-09-18-8BoQF-stacked_area_chart.png

Dept of Servers 🏒

Synapse

Neil announced:

This week we put out two point releases to fix critical bugs. At the very least ensure that you have upgraded to 1.19.2 which is a security release but you may as well go the whole hog and upgrade to the hot off the press 1.19.3. We will release 1.20.0 early next week.

Aside from that we continue to work on performance analysis and the sharding of the event persister continues - it’s proving to be a really tough job but we are getting there. We’ve also been making progress on the room knocking implementation

This week we say goodbye to Oliver (reivilibre) as a leaving present he fixed a long standing bug that prevented servers catching up after a federating outage and as we speak is furiously trying to finish a service to test TURN configuration. Thanks Oliver!

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander said:

Not much has happened this week, as the Dendrite team have been taking some time off. However, we still have a couple of changes to report:

  • Support for rejected events has been merged

  • Work on soft-fail has begun (although not complete yet)

  • A new mechanism for avoiding SQLite parameter limits was contributed (thanks HenrikSolver!)

Spec compliance is on the up:

  • Client-Server APIs: 56%, same as last week

  • Server-Server APIs: 74%, up from 71% last week

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo announced:

Welcome back! This week we made some groundbreaking progress - this is not a joke, we found a bug in Synapse while breaking multiple Matrix rooms, causing the Matrix team a lot of unnecessary stress. Let me explain: The good news is that Conduit is starting to federate now. This means that you should be able to join all public rooms of the Matrix network and exchange messages. Note that Conduit does not do all the checks it should be doing yet making it stop sending messages from time to time as well as that advanced features like loading the history, syncing temporary data like read receipts or accepting invites are not implemented yet.

The bad news is that, while Synapse is happy to accept Conduit's messages when it is already part of the room, joining into one of the rooms Conduit servers are part of didn't work because of an event validation bug. The Matrix team did an excellent job at fixing this bug and releasing a Synapse patch the same day, but the damage has been done making a few rooms inaccessible to old Synapse servers. Thanks to everyone who supports me on "Liberapay" (https://liberapay.com/timokoesters) or Bitcoin!

Synapse Deployment πŸ“₯️

Synapse on NetBSD

js reported:

I ported Synapse + its dependencies to NetBSD and it is now in pkgsrc as chat/matrix-synapse. This means that Synapse is now easily installable on any operating system that supports pkgsrc (which is many, e.g. NetBSD, Linux, macOS, Solaris, AIX, Haiku, …). And while at it, I also ported mautrix-hangouts + dependencies (in pkgsrc as chat/mautrix-hangouts).

YunoHost

Pierre told us:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.19.1 (1.19.2 available in branch testing)

Element Web integration had been updated to 1.7.5 (1.7.7 available in branch testing)

Kubernetes

Ananace offered:

Both image tags and a new chart version are hereby pushed for 1.19.2 for my Synapse image and Helm chart

then later

Just pushed the updates for the K8s-optimized Synapse image and chart for 1.19.3

Dept of Bridges πŸŒ‰

matrix-appservice-bridge hits 2.0.0ʳᢜ¹

Half-Shot said:

Hey bridge developer enthusiasts! Myself and ChristianP (bridge crew of matrix.org) have been working hard on freshening up the matrix-appservice-bridge library. It's no secret that it was using "classic" Javascript rather than ES6 syntax, was poorly documented in places and there were no types at all to make use of.

But that's all changed now, the 2.0.0-rc1 release brings fresh types, new convenience methods, a total swichover from Bluebird Promises to native Promises and much much more. If you are looking to quickly get set up writing bridges, it's never been easier.

Check out the slack-starter project to get your first taste of bridge excellence and see what you think.

As this is an RC, feedback is valuable and logging bugs helps us all. Please do so if you encounter any issues πŸ˜„

Bridge Encryption Support

Half-Shot reported:

Hi encryption fans. This week the bridge team has been churning away at adding support for encrypted rooms in the matrix-appservice-bridge library, to enable support in matrix-appservice-irc, matrix-appservice-slack, and anyone else using the library. This is probably the most exciting feature the library has seen in a long time!

The feature will be a simple config toggle, so existing bridges will have to do very little work to support it.

The first phase plans to use Pantalaimon with bridge users syncing like real users, but eventually the hope is that this will be all built into the library. Come check out this issue to keep track of progress.

Dept of Clients πŸ“±

Element Web

Neil announced:

  • Released 1.7.6 and 1.7.7 highlights include

    • Redesigned right panel where top right actions are now revealed via the i icon

    • Widgets can now be opened in the right panel

    • Widgets won't be visible in the Apps Drawer (top of timeline) by default (except Jitsi) - you need to pin them from the right panel

    • Widgets can now be resized

  • A new feature to defer e2ee set up until the user actually wishes to use an e2ee channel.

  • A series of config flags to remove various features from the UI thereby simplifying the experience.

Mirage

miruka said:

0.6.3 and 0.6.4 were released this week, and I'm now working on desktop notifications support and push rules control.

Added

  • Add a system tray icon.

    A left click will bring up the Mirage window, middle will quit the application and right will show a menu with these

    options.

  • Add a closeMimizesToTray setting to the config file, defaults to false.

    Controls whether closing the Mirage window will leave it running in the system tray, or fully quit the application.

  • Add a discrete read marker indicator to messages, shows how many people

    have this event as their last seen one in the room.
    A way to see who read the message and when will be added in the future.

  • Themes: add chat.message.localEcho and chat.message.readCounter color properties

  • Add a zoom setting, defaults to 1.0

  • Add a lexicalRoomSorting setting, to sort rooms by their name instead of

    recent activity.
    A restart is needed to apply changes to this setting.

Changed

  • Restrict Mirage to a single instance per config folder, trying to launch a

    new window will instead focus the existing one.
    The MIRAGE_CONFIG_DIR and MIRAGE_DATA_DIR environment variables can be

    set to run different "profiles" in parallel.

  • Reduce the visible lag when opening a chat page, switching rooms should be

    a lot smoother

  • When using the focusPreviousMessage and focusNextMessage keybinds, if no

    message is focused and the timeline has been scrolled up, focus the message in the center of the view instead of returning to the

    bottom of the timeline and focusing the last one.

  • Don't re-center the room list on clicks by default.

    This prevents the list from jumping around every time a room is selected.
    The previous behavior can be restored with the new centerRoomListOnClick

    setting.

  • Show a better terminal error message than "Component is not ready" when the

    window creation fails, giving details on what went wrong in the code

  • If an account's access token is invalid (e.g. our session was signed out

    externally), say so with a popup and cleanly remove it from the UI, instead of spamming the user with errors.

  • Rename message context menu option "Debug this event" to just "Debug"

  • Unify up/down and (shift+)Tab navigation for the account Sessions page

  • Changes to the UI scale/zoom via keybinds are now persisted across restarts

  • Themes: uiScale is now bound to window.settings.zoom.

    This change is necessary to keep the zoom keybinds working.

Fixed

  • A bunch of stuff, see the full changelog for 0.6.3 and 0.6.4

Element Android 1.0.7

benoit offered:

Element Android 1.0.7 has been released (still stuck in Google pipes writing those lines), and will be quickly followed by 1.0.8 to fix a problem with cross signing verification.

We are working to implement search of messages in a room and we keep on fighting bugs and improving performance of the application.

We are also iterating on the home screen, to improve user experience, and try to satisfy users with a few rooms as well as users with several hundreds of rooms.

Element-iOS

Manu said:

This week, we released 1.0.12 with several bug fixes including the one for background crashes due to PushKit

Hydrogen

Bruno told us:

Released 0.037 on Monday with some polish for E2EE:

  • remove outdated devices when querying

  • update room summary with decrypted events, also when retrying decryption

  • various crash protections

  • keep megolm session with earliest index when receiving a room key we already know about

  • add new icon

  • fix issue where olm would be loaded twice when login failed

  • show decryption errors in timeline

  • show encryption enabled tile in timeline

The rest of the week was implementing session backup, which I hoped would be ready to release today but it needs some more polish unfortunately. In the meantime, here's a GIF teaser:

2020-09-18-uMR7F-session-backup.gif

FluffyChat

Thank you to Krille, who told us:

FluffyChat version 0.18.0 is out now in F-Droid, Testflight and soon the PlayStore:

Features

  • Added translations: Armenian, Turkish, Chinese (Simplified), Estonian
  • Url-ify matrix identifiers
  • Use server-side generated thumbnails in cleartext rooms
  • Add option to send images in their original resolution
  • Add additional confirmation for sending files & share intents
  • Add option to opt-in to report issues / crashes to sentry
  • Write keys to online key backup, fully implementing online key backup

Changes

  • Tapping links, pills, etc. now does stuff
  • Better handling of sending messages in bad network
  • Better recovery of "keys not cached"
  • E2EE is enabled again

Fixes:

  • Various html rendering and url-ifying fixes
  • Added support for blurhashes
  • Image viewer now eventually displays the original image, not only the thumbnail

Dept of SDKs and Frameworks 🧰

Hemppa

Hemppa the bot is a multi-purpose Matrix bot for writing new functionality easily with Python.

Cos said:

Hemppa gained a small but big feature for making fediverse better: per-room Mastodon tooting support. This means that you can set up a Mastodon account in your room and anyone with moderator rights can toot to that account. This way you and your roommates can super easily send toots to the world directly from Matrix. Mastodon (unlike the big alternative) supports RSS feeds out of the box so it's very easy to subscribe to users and hashtags with stock RSS bot. Toot #hemppathebot if you like it! Another new contribution worth mentioning is welcome which can send welcome messages to users registering on the server or joining a room. https://github.com/vranki/hemppa

Ruby

Ananace said:

Another week, another Ruby SDK release. This time adding a little fix to avoid duplicate state events being passed twice to the application if identical ones arrive in both state and timeline, also adds a global state event handler and moves room event handling to not depend on room instances themselves.

Asked how much the SDK is used in production, Ananace said:

a little, we've hooked Matrix into our server orchestration / configuration management system TheForeman - which is a Rails application. Also got a colleague who's doing a bot to linkify internal ticket IDs.

Ruma

iinuwa told us:

Thanks to @q-b and @jtescher we have implemented more of the federation endpoints (bringing us up to about 85% completion) and every (one) endpoint of the Push Gateway API!

We fixed a bug where federation endpoints expected access tokens instead of server signatures for authentication. In the future, we may add an API that requires signatures on requests.

Finally, we've also continued progress on the non-exhaustive type updates for backwards compatibility; we expect to finish this within the coming weeks.

New Contributors

Dept of Bots πŸ€–

Matrix Reminder Bot

anoa announced:

v0.2.0 of matrix-reminder-bot has been released! This releases includes lots of bugfixes, updates and polishing. Find the list below:

Features

  • Better support for command prefixes other than the default !.

  • Just writing !silence now silences the currently active alarm.

  • The bot will now print the correct syntax of a command if the user fails to follow it.

  • The bot will reply to events if it cannot decrypt them, as well as offer helpful tips that both the user and bot operator can try to fix things.

Bugfixes

  • Timezones. They should finally work correctly! ...no 100% guarantees though.

  • Alarms were a bit broken. They're fixed now.

  • Fix commands with formatting and newlines not being picked up by the bot.

  • Fix non-latin characters preventing reminders from being deleted.

Internal changes

  • Add a dev-optimised Dockerfile for quicker iteration during development.

  • Better wording revolving alarms. They're just reminders that alarm repeatedly when they go off.

  • Log why the bot is unable to start.

  • Don't print "Unknown help topic" in case the user is trying to ask another bot for help.

  • The config dict is now a singleton.

  • Type hints everywhere!

Additionally, the minimum Python version is now 3.6. matrix-reminder-bot is made with nio-template.

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1fairydust.space412
2matrix-dev.kapsi.fi1308
3utzutzutz.net1519
4blob.cat1521
5r23s.eu1915.5
6conduit.rs2149
7maescool.be2759
8halogen.city2774.5
9uraziel.de3253
10kapsi.fi3713

Dept of Ping Extra:

timo announced:

Normally the ping shows how long other servers take to receive the sending-server's message. Some servers receive it quickly, some take hours. All this flows into the statistic of the sending server. I wrote a small program that's the other way around. Instead of measuring the sending-server's ping, it associates each measurement with an echo-bot (instead of the sending server), so this plot shows which maubots are the fastest to respond (it's probably not accurate at all, because most of the pings were started on conduit servers this week, but it's interesting nevertheless):

2020-09-18-PRR99-image.png

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2020-09-11

11.09.2020 18:33 β€” This Week in Matrix β€” Ben Parsons

Open Tech Will Save Us #6

From the schedule:

  • Ag3m, from La Quadrature du Net joins to present "Some thoughts on moderation and censorship in a decentralised world".
  • Sean DuBois (Sean-Der), WebRTC-knower and author of WebRTC for the Curious discusses his recent work in the space. https://webrtcforthecurious.com
  • Damir JeliΔ‡ presents the latest news on the Matrix Rust SDK

Open Tech Will Save Us is also available in audio form as a podcast in all the usual places!

Dept of Status of Matrix 🌑️

GSOC project reports

We had a bumper summer this year, with SIX students successfully completing their projects. So far three project reports have been published:

Expect to see the next three reports next week.

LinkΓΆping University

Alexander Olofsson announced:

Since we suddenly have multiple projects that've separately gone on to trial our new and shiny Matrix-on-location for multiple respective use-cases, from conferences to anonymous - encrypted - support chats, we've had to end our soft-launch period very quickly.

Oh dear.

For minutes, confusion reigned in the chat about just what this meant! Fortunately, clarity arrived:

To clarify; Ending the soft-launch period means that we've now gone on to a full launch, with public announcements and everything else that includes

\o/

Dept of Spec πŸ“œ

anoa told us:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Closed MSCs:

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus for this week, the widgets spec and related MSCs are still being completed, however is formally removed from the focus list for this week. In its place we've put something in a very similar vein: MSC1960 (OpenID Connect exchange for widgets).

We had another long overdue Spec Core Team retro this week. On the whole it felt very productive, and we've hopefully helped identify some key issues and actions going forward to help internally organise the way the team spends time on MSCs (read: unblocking review progress from the team).

We're still working out our exact strategy asynchronously at the moment, so expect more details to follow soon.

2020-09-11-yd96P-stacked_area_chart.png

Dept of Servers 🏒

Synapse

Neil reported:

This week we put out a new release candidate for Synapse 1.20.0, highlights include shadow ban support, more async/awaiting and unread message counts being added to sync responses to support upcoming notifications work.

Aside from that Erik has been continuing to get event persistence sharding ready for shipping. He already has a working version, but currently all workers will run at the speed for the slowest instance. Once that is fixed we will put it on matrix.org to see how it performs.

Patrick has finished up on asyncing all the things, and is now studying flame graphs to figure out where all the cycles on the main process are going.

Andrew has been taking a look at the knocking MSC, expecting concrete progress in the next few weeks.

Brendan has been working notifications support and is currently in VoIP land helping Dave on trying to level up our VoIP support.

Finally Oliver has dusted off his TURN tester from last Summer to get it working before he returns to university. The idea is that it will act a bit like the federation tester for but for TURN configuration.

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo announced:

Hi everyone, Devin R and I are working on state resolution, hopefully we have something exciting to show you next week. I also worked on the remaining key backup endpoints, so for example deleting backups works as expected now. Thanks to everyone who supports me on Liberapay or Bitcoin!

Dendrite

Dendrite is a next-generation homeserver written in Go

kegan told us:

This week we have been continuing work on the beta task list:

  • The current state server has been removed.

  • Joining rooms over federation is now more reliable as a single faulty event will no longer fail the request.

  • Support for database migrations has been added.

  • Various bug fixes to reduce the number of database is locked errors on SQLite.

  • Experimental support for peeking has been added - See MSC2753

  • DENDRITE_TRACE_SQL will now additionally trace unsafe goroutine writes that do not use ExclusiveWriter.

  • The README has been updated with hardware requirements.

Spec compliance remains the same this week:

  • Client-Server APIs: 56%, same as last week.

  • Server-Server APIs: 71%, same as last week.

Dept of Clients πŸ“±

Hydrogen

It's been a big week for Bruno:

Released 0.0.36 with end-to-end encryption enabled! πŸŽ‰
Most of this week was spent on performance improvements for IE11, where asm.js runs quite slow. Decryption is run in parallel on 4 workers in that browser, giving a massive speed-up and added responsiveness (the UI would freeze for 7 seconds when opening a room prior to this). There is still some polishing to do on sharing room keys, which I will get to next week, but all in all, it should work well. Next week I also want to get started on supporting key backup, so you can also read your encrypted history from before you logged in.

Get the latest: https://github.com/vector-im/hydrogen-web/

Element Web

Neil (same fellow from earlier) reported:

Element 1.7.6-rc.1 is now available at https://staging.element.io:

  • Redesigned right panel where top right actions are now revealed via the i icon
  • Widgets can now be opened in the right panel
  • Widgets won't be visible in the Apps Drawer (top of timeline) by default (except Jitsi) - you need to pin them from the right panel
  • Widgets can now be resized

Coming up:

  • Continue work on deferred cross-signing setup
  • Our new Matrix.to should get it’s first release
  • General bug fixes around threepid invites

Element for Nextcloud

Gary Kim said:

Element for Nextcloud (formerly Riot Chat for Nextcloud) has released v0.6.5 and v0.6.6. In this update, Element Web was updated to v1.7.5 and support for adding a custom integration server in the config through the settings UI and support for Nextcloud 20 was added.

Join the development Matrix room at #elementfornextcloud-general:garykim.dev.

Check out the source code here.

Element Android 1.0.6

benoit told us:

We have released Element Android 1.0.6. Now we are working on sync mode for F-Droid version of the application (i.e. without FCM). We are also improving the experience with 1-1 calls.

SchildiChat for Android

SpiritCroc said:

Element v1.0.6 has been merged into SchildiChat.

Furthermore, following Schildi-specific fixes and improvements are included in the latest update:

  • Fixed message bubbles clipping italic text

  • Less wasted bubble space in some scenarios

  • Deleted messages are now hidden by default in the chat overview

  • Display the year in the chat list for old chats

  • The inclusion of chats without notification in the overview's unread counter, which I added last week, is now optional

To be clear, Element v1.0.6 has been upstreamed into SchildiChat, the projects have not merged.

Element-iOS

Manu told us:

This week, we released 1.0.10 which provides PIN protection and the return of the incoming native call screen. Then, we started to update room creation flows with new icons and behaviors for creation buttons and a new screen for room creation.

Maunium sticker picker

This isn't strictly a client, but it's in the clients section.

Tulir announced:

Getting stuff into the spec or even implemented in Elements is very slow, so while we wait for proper native sticker packs and pickers (i.e. not in an iframe/webview), I decided to make a better sticker picker widget: https://github.com/maunium/stickerpicker

The goal is to be as fast and simple as possible. The picker and sticker packs are just some static files that can be served anywhere (no compilation required). It doesn't require self-hosting anything except said static files, and it has been confirmed to work on all three Elements.

There's a Telegram import script that's used to reupload stickers to Matrix and generate the static sticker pack files that the picker widget uses. It's technically also possible to make the sticker pack files yourself, but there's no easy way to do that yet.

Due to the simplicity of the picker there's no authentication, which also means everyone sees the same sticker packs. One way to get around that limitation is to have a unique URL for each user. I'll probably write a bot to manage copies of the picker for multiple users in the future.

Other future improvements include remembering frequently used stickers (surprisingly, all three Elements keep localStorage for widgets), importing Telegram animated stickers and searching for stickers by emoji/name

later more words appeared from Tulir:

Since the initial announcement, my sticker picker has received some more features:

  • Added a slider to configure the number of stickers shown per row

  • Frequently used stickers now show up at the top

  • Imported all the packs from Scalar (the default integration manager)

A model of productivity, Tulir also announced:

my sticker picker has dark theme support too now

Dept of SDKs and Frameworks 🧰

Ruby

Ananace said:

Just bumped the Matrix Ruby SDK to version 2.1.2, fixing another bug with server discovery as well as the fact that state events haven't been provided for the application using the SDK. Oops.

matrix-appservice-rs

Lieuwe announced:

matrix-appservice-rs got some love!

Though still a long way to go before it is comparable to the likes of matrix-appservice-node and matrix-appservice-bridge, I did improve some little things.

  • Updated to the latest (git) version of Ruma, and only depend on ruma instead of all subcrates.

  • Small memory improvements: using references in MappingId and shrinking the MappingDict when initializing.

  • Some more documentation.

  • Cleaned up the code a bit.

There is also now a specific room for this crate: #matrix-appservice-rs:lieuwe.xyz, feel free to ask for support or tell if you've suggestions.

quotient

kitsune said:

libQuotient 0.6.1 is out, with fixes for bugs found during the work on the next Quaternion release and also an optimisation in the way profiles of all the users are managed - noticeable when opening large rooms (like our lovely HQ). Quaternion 0.0.9.5 beta is coming real soon now (TM), with some big features being coded in and merged as I write this.

...and the release notes are here: https://github.com/quotient-im/libQuotient/releases/tag/0.6.1

Dept of Ops πŸ› 

mnotify

stefan said:

I created mnotify which is a CLI based matrix client for automation tasks in shell scripts. Currently it supports login (via password), logout, sending messages, simple syncing, "get"ting the synapse admin api, creating a room, inviting users to a room, and joining a room.

Dept of Services πŸš€

publiclist.anchel.nl

Mr. Wimpy offered:

https://publiclist.anchel.nl has been updated. The publiclist contains several public homeservers who can be used freely. Also listed some communities and roomdirectories which can be found at homeservers, last but not least public useable bridges can be found. The homeserver list contains now also uptime percentage and ip info. There is also a json that can be used by client builders.

Dept of Bots πŸ€–

Bots galore

Alexander Olofsson reported:

And in completely unrelated news, I also just tagged version 1.0.0 of an invite bot for Matrix, which helps in doing bulk invites to communities and community-associated rooms.

A single bulk-invite (MXID or 3PID) to a main room will be propagated to invites into the community (if wanted), as well as invites to all rooms linked to the community, once the invited user joins the main room.

Dept of Events and Talks πŸ—£οΈ

Perth Linux user group Matrix presentation

PC-Admin offered:

I did a talk called 'Matrix in 2020' with the Perth Linux user group, parts of the video are missing but the audio is solid

I think we did a good job summarising how far Matrix has come, we also preview the idea our new Matrix hosting company!

https://www.youtube.com/watch?v=jTutFZzFizw

From the YouTube description:

Michael Collins talks about the Matrix, covering the new features, perthchat.org, and ChatOasis an upcoming FOSS Matrix hosting company.

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1fairydust.space325.5
2helderferreira.io614.5
3neko.dev661.5
4kif.rocks718.5
5matrix.vgorcum.com1785.5
6an-atom-in.space2034
7urech.ca2226.5
8ragon.xyz2436
9im.kabi.tk2478
10utzutzutz.net2588.5

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2020-09-04

04.09.2020 20:24 β€” This Week in Matrix β€” Ben Parsons
Last update: 04.09.2020 19:22

Matrix Live πŸŽ™

  • Kegan discusses Complement, testing suite for Homeservers
  • Steve from iOS: FaceID and PIN protection
  • ChristianP shows "Easy bridge creation with matrix-appservice-bridge"
  • Bruno demos Hydrogen e2ee features

Dept of Status of Matrix 🌑️

Next step in world domination

Alexander Olofsson reported:

The Matrix service at LinkΓΆping University has now passed its soft launch state and headed straight into full and proper deployment territory. (Though we're still not being overly vocal about it, don't want to crash our deployment on day one after all.)

They're simply announcing it on TWIM, perhaps the most high-profile place in the world!

We're doing a little bit of advertising of it to computer clubs and the like though, to slowly build up the users.

But then:

And just to add another slight update: already we have at least one conference planning on trying out the new Matrix server as well, so the user count might just end up exploding soon anyway.

\o/

Dept of Spec πŸ“œ

sorunome asks for attention on MSC2545:

As a lot of people in TWIM were originally interested in it, soru just wanted to say that MSC2545 (emotes) is no longer WIP, it is ready for review

Spec

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus for this week, we're sticking with the widget theme, but also adding something new. Travis is writing up a mahoosive widgets PR in addition to some widget MSCs. We're also adding on MSC2753 (peeking /sync), MSC2444 (peeking federation), which will both unlock some very exciting upcoming features (such as x-as-rooms)!

2020-09-04-gKF3z-stacked_area_chart.png

Dept of Servers 🏒

Synapse

anoa said:

Erik has made an initial, naive implementation of a sharded Event Persister worker - the non-sharded version currently being one of the biggest bottlenecks on matrix.org right now. It still needs a bit of polish to be production ready, and Sytest will need some updating to support testing it, but otherwise progress is looking good!

Brendan is polishing his unread counts implementations, while Rich is writing up the push rules replacement tech in an MSC. Rich has also been optimizing the device code in Synapse to both simplify the code, as well as cut down on unnecessary device data being sent over federation.

Patrick has finished converting Synapse's database code to be fully async! This has the benefit of making profiling the matrix.org Synapse process much more sane, and we're already starting to find a few little places where Synapse is burning CPU unnecessarily! Now, Synapse won't be fully async'd until Twisted itself is, and we drop support for older Twisted versions that aren't, which is not soon. However the majority of the code that developers touch and that is executed being async has massive benefits either way πŸ™‚

Andrew has been working on having a confirmation pop up after users click the link in their email, to help prevent phishing scams. He's also trying to finally make Synapse spec-compliant in accepting identifier dictionaries during user-interactive authentication.

Olivier has been continuing to help Synapse recover missed messages better after it has been offline for an extended period of time, as well as various fixes to push.

Expect a fresh release with all these changes soon!

As someone who had my own server outage this week I'm looking forward to seeing this!

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander said:

This week we have been continuing on our bug-hunt mission and have been working on refactoring some significant parts of the codebase to simplify the API surfaces and avoid race conditions.

Changes this week include:

  • The current state server is in the process of being deprecated, with this functionality being folded into the roomserver

  • Support for rate limiting on certain CS API endpoints has been added

  • Support for password changes has been added, allowing users to change their own password

  • /sync no longer incorrectly lists some rooms in the leave section, which caused problems with Element Web and other clients

  • Roomserver input events are now handled in FIFO order with per-room workers

  • Roomserver input, query and perform packages are now split up nicely

  • The federation sender and keyserver now delay some startup tasks to allow the HTTP listeners to start more quickly

  • The federation sender is now used for backfilling and getting missing events

  • Some more SQLite writer bugs have been fixed

  • Room information in the roomserver is now handled in a common structure

  • The public rooms API now correctly searches remote servers (thanks rohitmohan96)

  • Initial support for Peeking (MSC2753) and Peeking over Federation (MSC2444) is in progress

Spec compliance is improving:

  • Client-Server APIs: 56%, up from 55% last week

  • Server-Server APIs: 71%, same as last week

Last week, we updated some of our documentation and created a whole new set of easy first issues and medium to hard issues for contributors.

If contributing to Dendrite sounds like something you would be interested in, please take a look at these issues and join us in #dendrite-dev:matrix.org! There's also #dendrite:matrix.org for general Dendrite chat and updates.

Synapse Deployment πŸ“₯️

YunoHost

Pierre announced:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.18.0 (1.19.1 available in branch testing)

Element Web integration had been updated to 1.7.4 (1.7.5 available in branch testing)

Dept of Bridges πŸŒ‰

tomsg-matrix

Lieuwe told us:

Introducing a new bridge to the family, tomsg-matrix!

A friend of mine preferred using his own chat protocol, but having a chat protocol is useless without having people to talk to. Since I prefer using Matrix I thought it would be a fun exercise to try and create a bridge using Rust.

The bridge uses the newly created (still very WIP) matrix-appservice-rs. A Rust library that builds on top of the Ruma suite, the goal is for it to be comparable to matrix-appservice-node and matrix-appservice-bridge.

The code for tomsg-matrix is open source, but be prepared, I hastily pushed it to GitHub without cleaning it up.

Come chat to us over at #tomsg:lieuwe.xyz, hoping that it doesn't break. :P

Making your own chat protocol in a world of well established options seems, frankly- oh right.

matrix-sms-bridge

Benedict told us:

We are currently working on an Android-App to use an old smartphone as SMS-Gateway for matrix-sms-bridge, because using gammu in production (>50 SMS/day) seems not to be very reliable with our USB-modem.

im_sender_service, for zabbix bridging and more

progserega said:

I create service for sending corporate information: some service (for example zabbix) can send message to this service (over http-api) and service will create room (if needed), invite user and send message to this room. Service will change status of message in his database to readed when user will read this message. Also service can send any changes statuses of this message by callback url. Editing old messages also supported: for example zabbix can send UID problem to this service and it is will edit old "problem message" to "problem resolved".

Url: https://github.com/progserega/im_sender_service

Dept of Clients πŸ“±

Ditto

Ditto is a Matrix client for iOS & Android

Annie offered:

Ditto development is back from hiatus!

I'm currently in the process of rebuilding Ditto - partly to incorporate rn-matrix, and partly to support a WEB VERSION

The new repo can be found here (https://gitlab.com/ditto-chat/ditto) (there's a new repo mostly because getting react-native-web to play nicely with the rest of the app was a challenge, to say the least)

As a sneak peek, enjoy this rough version (https://ditto-test.netlify.app) of Ditto for web :)

Expect to see what I'm calling Ditto's 2.0.0 drop on TestFlight and Google Play in a few weeks.

Links:

Matrix Room

Big exciting things happening!

Hydrogen

Bruno offered:

Good progress this week, I just got end-to-end encryption working. The code still needs some cleanup and polishing to be able to release it in the wild, but expect a release somewhere next week with E2EE enabled.

YESSSS - see Matrix Live for a demo.

2020-09-04-t4hyh-image.png

gomuks

gomuks is a terminal based Matrix client written in Go. Source on GitHub

Tulir said:

gomuks now supports importing and exporting message decryption keys. It even supports exporting the keys for a specific room, e.g. for sharing them with someone else. To help with sharing exported key files with someone else, gomuks also got support for uploading media.

Next I'll try to figure out how nikofil's cross-signing/SSSS PR for mautrix-go works and use that in gomuks.

v0.2.0 was just released with these features!

NovaChat

eric told us:

NovaChat is a new Matrix-based desktop client that aggregates all your chat networks into one app. We now have integrated bridges for Whatsapp, Twitter, Telegram, Hangouts, Slack, Messenger, Android Messages (SMS), Skype, Discord, Instagram and IRC.

Sept 4 Updates:

  • Continuous UI improvements. Everything is much denser now. See screenshot below.

  • New bridge alert: tulir just completed an Android Messages SMS bridge (which is a total hack but it works!) for NovaChat.

  • Added Discord Bridge. Thanks for building this Sorunome!

In the works...

  • Still working on our iMessage bridge, using jailbroken iPhone 4s

  • Marking messages as 'unread'

Sign up for the NovaChat beta. Or send me a DM @eric:nova.chat!

2020-09-04-iEKfJ-Screenshot-20200904072010-980x1069.png

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) announced:

  • Lurkki added an emoji completer to the message area! This was a super requested feature and it is pretty awesome, that someone picked this up and implemented it! This means you can finally write :fire: and have it expand to πŸ”₯! Seems like this will also lead to a few more improvements to our other completers down the line (user, room, etc completions), so thank you very much Lurkki!

  • I've been mostly reviewing PRs this week (and trying to get some other work in line, like MSCs, synapse stuff, etc). With this hopefully Chethans work on cross-signing will be merged soon, which will give us most of the building blocks to have cross-signing in Nheko (his work is basically only missing Secure Secret Storage and Sharing). We hope to fix the few remaining build issues in the next week or so.

  • On the side I have been cleaning some UI stuff to simplify our room settings dialog and allow advanced users to hide/show certain event types in the timeline. This will probably take a bit more time to be finished though.

2020-09-04-KUPpB-clipboard.png

Fluffychat

sorunome offered:

Fluffychat 0.17.0 is released!

IMPORTANT: Due to legal reasons (USA cryptography export), end-to-end encryption has been disabled in the google play and ios builds for now. They will be re-enabled once we got all that figured out.

The F-Droid builds still have e2ee. As the play store and fdroid builds use the same build signature, you can simply install the fdroid version over the other one. More information on that here: https://fluffychat.im/en/fdroid.html

Features

  • Pin and unpin chats

  • Implement event aggregations

  • Implement message edits

  • Render reactions

  • Add / Remove reactions by tapping on existing reactions

Fixes:

  • Don't re-render the room list nearly as often, increasing performance

  • Various fixes for sending messages on bad networks

  • Design tweaks and fixes

  • Various performance fixes and improvements

Definitely hoping to see these issues resolved.

SchildiChat for Android

SpiritCroc offered:

I removed SchildiChat from the Google Play for now, due to the complications of providing encryption-capable software from US servers. Huge thanks to the FluffyChat team for providing insight into their investigations concerning the US export regulations!

On the positive side, SchildiChat is now finally available on F-Droid from the official repos.

Furthermore, I have set up my own F-Droid repository, where you can get updates that are compatible with the Google Play Store variant and which support push notifications.

On the implementation side, I can report the following:

  • Element 1.0.5 has been merged

  • The "low priority" category now remembers its expand/collapse-state across restarts

  • New setting to hide member state changes and reactions from the chat overview

  • The category unread counter badge now also displays the number of chats with new messages in case none of these has notifications enabled

Element-iOS

Manu announced:

This week, we should have fixed the background crash that was happening in TestFlight 1.0.7. TestFlight 1.0.9 has been submitted yesterday.

In parallel, we started to modernise the AppDelegate class, the entry point of the app. This is the very first step of several improvements we want to do (this is not only to rewrite it in Swift).

Element Android

benoit reported:

We have just merged a few big PRs to develop: add emails and phone numbers to Matrix account, rework of upload attachment management, and other stuff related to 1-1 calls. It will live on develop for the week-end and we will prepare the release v1.0.6 at the beginning of next week.

Dept of SDKs and Frameworks 🧰

rn-matrix

rn-matrix is a React Native SDK for matrix

Annie offered:

Built on the matrix-js-sdk, this NPM package will help any React Native developer drop matrix into their app.

Been working on this since the beginning of the summer - this package is split into two, one for the data and one with default UI components for those who want to drop chat into their app quickly (not required).

Would love feedback on docs, installation, and usage, so try it out and drop me a line!

Links: Docs (start here!) | Matrix Room

NB, this update should be read together with the Ditto update above

Ruma

iinuwa said:

Over the past couple of weeks, jplatte has been hard at work making sure we can introduce new fields to request and response types as non-breaking updates. Thanks to @nicholaswyoung, consumers can opt-in to breaking changes to ensure compliance with the spec on each update.

Besides that, we implemented another endpoint for the Federation API, bringing us up to 20/31 endpoints implemented.

New Contributors

Dept of Ops πŸ› 

matrix-docker-ansible-deploy

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi reported:

matrix-docker-ansible-deploy can now help you set up matrix-registration - an application that lets you keep your Matrix server's registration private, but still allow certain users (those having a unique registration link) to register by themselves.

See our Setting up matrix-registration documentation page to get started.

Dept of Events and Talks πŸ—£οΈ

Matrix talk @ FrOSCon 2020 (in German)

Oleg reported:

I did a talk on Matrix bridges at the annual Free Open Source Conference.

Apparently

The interest was high. Some didn't know, that bridges exist in Matrix.

!!! Well thank you Oleg for letting them know.

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1fairydust.space376
2neko.dev444.5
3tchncs.de565.5
4autodie.be958
5matrix.vgorcum.com1124
6maescool.be1435
7lermer.nl1989
8utzutzutz.net2207
9bau-ha.us2965
10rollyourown.xyz3013

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2020-08-28

28.08.2020 00:00 β€” This Week in Matrix β€” Ben Parsons

Matrix Live πŸŽ™

Dept of Spec πŸ“œ

anoa announced:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus for this week, we're taking off MSC2674 and MSC1544 while the author works through feedback, and instead looking at getting widgets in line.

The spec for widgets has so far lived on a collaborative document that hasn't been updated in some time. As a result multiple, separate implementations have sprouted. We'd like to nail this down. So the focus for this week will be: MSC1960 (widget auth mechanics) and MSC1236 (general widgets).

2020-08-28-MSnEZ-stacked_area_chart.png

Dept of GSoC πŸŽ“οΈ

The coding phase of Google Summer of Code is over! That said there is still a bunch of stuff to do, code is still appearing and we're now entering the time of many forms to fill in in order for people to get paid.

A little pre-emptive, but wow it's been a good summer! SIX students, all did well!

matrix-ircd - Call for Testing

jplatte announced:

GSoC coding is over and there's now an async/await ported matrix-ircd! Before merging that into master, it would be nice to have it tested a bit more. To install it, you first need Cargo, Rust's package manager (Installation). With that set up, just run

cargo install --git <https://github.com/matrix-org/matrix-ircd> --branch async_await

Once matrix-ircd is installed, you can run it and connect to the locally spawned server with any IRC client.

HTML embeddable matrix chat

arnav-t said:

For the GSoC project HTML embeddable matrix chat this week -

  • Added an 'auto' option for the theme (read and set the theme from the device)-

  • Added a 'custom' highlight color (meant to be modified by users).

  • Restyled various components.

More news to come from this project - going to be used in a really exciting live event in the near future. Arnav has accidentally found a niche. :D

One more thing: This project needs a new name (riot-embedded I don't like.) If you have ideas you can fling them at arnav-t or post in #twim:matrix.org.

Dept of Servers 🏒

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander told us:

This week has mostly been focused on beta milestone tasks and bug fixes. Changes this week include:

  • Joining rooms which the server already belongs to when invited by a user from another homeserver is now significantly faster

  • The client API no longer refers to the device DB directly, instead using the new user API

  • A number of locking issues have been fixed in the federation sender

  • The /join endpoint now accepts ?server_name=... query parameters

  • The device list updater now obeys the backoff period and blacklisted status from the federation sender

  • Uploaded media will no longer be truncated to 0 bytes when max_file_size_bytes option is set to 0 (for unlimited)

  • The media API now correctly cleans up temporary directories when errors occur

  • The media API now returns unique media IDs and metadata when multiple users upload the same file, whilst maintaining deduplication on disk

  • Rejecting invites over federation multiple times should now work properly

  • Redactions are now permanent and are committed to the database

  • Some caches have been added to the roomserver for room IDs, state key IDs and event type IDs to reduce pressure on the database

  • The redaction endpoint is now spec-compliant

  • Some places where global errors were overwritten have been fixed (thanks oliverpool!)

  • Some state key checks in the roomserver membership updaters have now been tidied up (thanks oliverpool!)

Spec compliance has improved slightly:

  • Client-Server APIs: 55%, up from 54% last week

  • Server-Server APIs: 70%, down from 71% last week due to a single flakey test

We've also updated some of our documentation and created a whole new set of easy first issues and medium to hard issues for contributors.

If contributing to Dendrite sounds like something you would be interested in, please take a look at these issues and join us in #dendrite-dev:matrix.org! There's also #dendrite:matrix.org for general Dendrite chat.

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo said:

Some new features this week!

  • Handle transaction ids to avoid duplicate messages on bad connections

  • Redaction reasons

  • Add remaining key backup endpoints (e.g. deleting backups)

  • Send correct device list changes when leaving and joining e2ee rooms

Also check out our new website (https://conduit.rs) if you haven't already.

Thanks to everyone who supports me on Liberapay or Bitcoin!

Synapse

anoa offered:

This week Erik has been working on sharding the Event Persister worker as we aim further for horizontal scalability for Synapse. Some of the complication comes from ensuring that sysadmin's don't misconfigure things where multiple event persisters persist events in the same room.

2020-08-28-uUl-U-image.png

Patrick has been continuing on the async/await train and has just submitted the final PR which async's our runInteraction method - the one that most database calls go through. Synapse is nearly 100% async! He's also been submitting a flurry of shadow ban PRs, which will be another powerful tool for moderation.

Brendan and Rich are continuing to work on the Notifications rewrite, with Brendan finishing up unread counts, and Rich polishing up designs for the new replacement for push rules.

Andrew (who dat? - ed.) is continuing to work on the knocking MSC and has fleshed out the CS and SS details. There are still some open questions though, so please give feedback if you notice anything off!

Finally, Olivier has been working on making sure you get your missed messages when your homeserver has been offline for a while, as well as various Sygnal fixes and matrix-synapse-ldap3.

Them's the news!

Synapse Deployment πŸ“₯️

YunoHost

Pierre said:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.18.0 (1.19.0 available in branch testing)

Element Web integration had been updated to 1.7.3 (1.7.4 available in branch testing)

Kubernetes

Ananace told us:

Just pushed an update to my matrix-synapse chart, fixing the volume permission configuration - for those who want to run Synapse as non-root without using a storage class that supports setting rights based on fsGroup

and then

Just pushed the 1.19.1 tags for my K8s-optimized container image, and a new helm chart version to go with it.

Did you know that Ananace is running a Matrix server for LinkΓΆpings universitet?
Another True Fact: "universitet" appears to be Swedish for "university".

Dept of Bridges πŸŒ‰

matrix-appservice-mattermost

dalcde offered:

I recently published a new matrix <-> mattermost bridge - matrix-appservice-mattermost. The new bridge already supports many features and is rapidly improving, with the current focus being setting up proper integration tests.

New bridge! This is actually really fully featured considering that we've only just heard about it! Looking forward to seeing more from dalcde and team in future.

Dept of Clients πŸ“±

Daydream

MTRNord offered:

Some small updates on Daydream:

  • The pages https://daydream.im and https://app.daydream.im get finally updated again. We switched from netlify to github pages

  • Devin R fixed a bug with huge numbers in power levels which caused Florian to not being able to join (Fixed in the rust-sdk)

  • Because of the now again working deployment of Daydream parts of the WIP redesign are now available on https://app.daydream.im

  • Github Emoji codes like :gift: Get replaced with unicode on sending allowing basic emoji usage

Also the landing page at https://daydream.im got a fresh new design :)

And we are investigating a switch to the Druid framework to also add support for desktop

Check it out at: https://app.daydream.im or join our Chat at #daydream:nordgedanken.dev

Honestly I didn't realise Florian was a setting in the power levels. Is this in the spec? Very good seeing Rust SDK getting such usage, we will see more from this project I hope!

mxapps

KB1RD offered:

A multi-account Matrix client that allows you to authorize webapps to access parts of your account, designed with collaboration in mind.

This is the webapp I've been talking about for a while that allows webpages limited access to your account. Currently, apps can view the name, avatar, alias, and type of a room that's been opened with the app, and that's just about it.

The big interesting thing about mxapps right now is the architecture. Just about all of the logic is running in a shared worker. This means that you can duplicate as many tabs as you want and it will all act as the same client. It also has multi-account support.

Features coming soon at some point will be full state event support and easy-to-follow docs on how to make an app, plus Vue plugins to make development really easy.

You can checkout the live version on mxapps.kb1rd.net, but it is a bit useless...

There are some work-in-progress docs on the structure on docs.mxapps.kb1rd.net

I'll post an example app in a bit ;)

What a great use of workers! The possibilities for this are really impressive...

Mirage

miruka reported:

Small 0.6.2 release this week, mostly bug fixes:

Changed

  • When replying to a message, you can now press enter without entering any

    text to send it directly (useful to "forward" a message).

  • Sending a file while replying to a message will create a pseudo-reply,

    consisting of an "In reply to" text message with no body, followed by the actual file event.

    This is a workaround to the reply restrictions imposed by the Matrix spec.

  • Composer aliases cannot contain whitespace anymore.

    This includes spaces, hard tabs or newline characters. If an alias from your config still has whitespace, only the first word

    will be taken into account (ignoring any leading or trailing space).

  • Faster server browser loading, now gathers all needed data with a

    single request instead of one for each server

  • Auto-focus the "Join" button on invited room pages

    ((Shift+)Tab can be used to navigate between buttons)

  • Auto-focus the "Forget" button on left room pages

  • Themes: modify chat.message.styleSheet to add some spacing between HTML

    list items, see 48663ae

Fixed

  • Fix @username autocompletion closing if there's more than one character

    after the @

  • Consider the partial text from IME (input method editors) and touch screen

    autocompleting keyboards and for username autocompletion

  • Reset IME state upon autocompleting a username

  • Fix clicking on autocompletion list user not making the username a mention

  • Fix UI freezing when mentioning user lacking a display name

  • Fix mentioning users with blank display name (e.g. only spaces), mention

    them by their user ID

  • Fix text fields/areas unable to be focused on touch screen

  • Fix random chance of profile retrieval requests failing if one of the logged

    in account doesn't federate with other servers (e.g. localhost synapse)

  • Fix composer text saved to disk for the wrong account if that text begins

    by an account alias

  • Servers can potentially return an outdated member list for rooms on initial

    sync, which is one of the possible cause of "Members not synced" error for encrypted rooms.

    When loading the full room list, discard members from the initial sync list that are absent from the full list.

    For those not using the AppImage or Flatpak, this fix requires matrix-nio 0.15.1 or later to take effect.

  • When erasing an account alias inside the composer, send a

    "x isn't typing anymore" notification corresponding to that account

  • Fix potential 403 error on chat pages for invited rooms.

  • Start loading room history immediately when the room join state changes,

    e.g. when clicked "Join" for an invited room page.

Hydrogen

Bruno announced:

Released 0.0.34 this week with some fixes to the room list sorting and fixing several crashes. If you clear your session, it will also sort low priority rooms at the bottom of the room list, even if unread, making it a bit more usable until we add more UX refinements to the room list.

Most of the effort this week however has gone into working on end-to-end encryption, having investigated using the rust-sdk for a couple of days, and now looking into a custom implementation based on libolm. This is well on its way with key upload working and device tracking nearing completion.

Getting e2ee will make this a very compelling client! Remember to watch Matrix Live with Bruno above. If you've watched it, remember we have 100s of episodes of Matrix Live to catch up on!

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) told us:

I just merged the new event store a few days ago. This means Nheko now doesn't keep messages in RAM! Everything is stored on disk and mapped into memory from disk. This may reduce memory usage a tiny bit, but may increase latency while scrolling a bit. Try it out and give your feedback!

For the developers reading this, I collected some ramblings on my blog here: https://blog.neko.dev/posts/nhekos-event-store-v2.html . Yes, I designed that site when I was like 15, I know the dark theme is a bit too much... Just use Firefox and the alternate stylesheet for light mode! Anyway, I'm very interested to hear how other clients implement their event stores!

Element-Android v1.0.5

benoit said:

Element-Android v1.0.5 has been released to the PlayStore. We are now working to improve user experience with 1-1 calls. We will add support to add and remove emails to your Matrix account, and we are still fixing bugs...

We are always fixing bugs!

Element-iOS

Manu has been working hard on iOS:

This week, we made 2 release candidates available through TestFlight. New features are :

  • app access protection with PIN code, TouchID or FaceID
  • the come back of the incoming native call screen

A huge thanks to the community for the feedbacks. It helped us to discover issues like a crash in background due to PushKit. The coming TF (1.0.7) should fix all of them

Dept of SDKs and Frameworks 🧰

mautrix-go

nikofil reported:

A PR has been created for cross-signing and SSSS support! It also supports accepting in-room verification requests and will soon also support creating them, after which it should be ready for reviewing. Afterwards, with some minor changes to clients using the library, cross-signing should be available.

Cross-signing in gomuks and more?

Dept of Bots πŸ€–

catbot

chloe reported:

Github

A Matrix bot using the library matrix-nio and Docker (specifically aiodocker) for running sandboxed code, allowing users to write their own factoid commands which can perform any task required. The bot launches a new instance of itself every time it is invited to a room, where it can then be configured for that channel by a management website. There is also a web IDE for the factoids, so you can edit and test the bot's output.

  1. Bash-like factoid and command input/output redirection

  2. Run untrusted code for PHP, Python, JavaScript and Java in Docker containers

  3. Manage the instance of the bot from a management server (realtime log output over websockets, schedule commands to run, manage trusted devices and more)

  4. Web interface for factoid editing (create, save, edit and test with live output). IDE from ace from cloud9

2020-08-28-ArKGA-testing_factoid.png

2020-08-28-gdnHE-time_example.png

Cool! There are so many bot creation systems for Matrix now!

Final Thoughts πŸ’­

Christian offered:

There is one month left until HacktoberfestπŸŽƒ! Prepare your repos now!

It's a great time to get friends and newcomers involved with open source and your projects. During October, merged PRs to any(!) public GitHub repo will count towards a person's score. With four PRs, they get a very neat looking t-shirt from the organizer DigitalOcean. They can also ask for a tree to be planted instead.

As a maintainer,

  1. Add a "Hacktoberfest" label (color suggestion: #FF7518) on issues to point people to tasks with a clear scope.
  2. Let people know that you're interested in contributions and will review them.
  3. Add the label "invalid" on a PR that's spam or a minor contribution that shouldn't count (fixing typos, blindly upgrading dependencies, etc.).

Matrix benefits from Hacktoberfest, but do keep in mind that this is a commercial project from DigitalOcean.

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1fairydust.space374
2neko.dev526
3loves.shitposting.chat566
4yyyyyyy.ml619
5kif.rocks669
6maescool.be737.5
7matrix.vgorcum.com754
8autodie.be839
9hmlw.me966.5
10fick.es1053.5

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2020-08-21

21.08.2020 00:00 β€” This Week in Matrix β€” Ben Parsons

Matrix Live πŸŽ™

Nico, one of the Nheko client maintainers, discusses the state of the project and shares various opinions about Matrix, chat clients, physics, the local weather and so on.

Dept of Spec πŸ“œ

anoa offered:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Closed MSCs:

MSCs in Final Comment Period:

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus for this week, MSC1960 will be the only one on the list. The two MSCs from last time will be put aside for this week while we focus on implementation.

2020-08-21-WX3fG-stacked_area_chart.png

Dept of GSoC πŸŽ“οΈ

Nheko(GSoC)

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Chethan offered:

  • Room Verification works in Nheko! Still needs some Improvement,working on it.

  • Improving and fixing some caching issues.

  • Fixing some breaking stuff with the new event store Nico is working on.

Dept of Servers 🏒

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander told us:

This week we have largely been working on bug fixes and performance issues.

Changes this week include:

  • The room server now handles input events for multiple rooms in parallel when running with PostgreSQL, which helps a lot with perceived latency

  • Device key queries now obey federation backoffs, so that we don't spend a lot of time making unnecessary requests

  • The key server now only emits changes for /users/devices which reduces database hits significantly

  • The client API now tries to guarantee message ordering on a per-user basis (thanks Anand!)

  • Federated invites are now more reliable and are performed synchronously

  • Invite room metadata is now sent correctly in all cases

  • Accepting and rejecting invites is also now much more reliable

  • A bug has been fixed that caused excessive memory usage in the sync API (bringing the memory usage of my own instance down to less than 100mb)

  • A massive amount of work has been done to reduce database locks in SQLite mode across the entire codebase

  • Some unnecessary code has been removed from the roomserver updaters

Spec compliance has improved slightly:

  • Client-Server APIs: 54%, same as last week

  • Server-Server APIs: 71%, up from 70% last week

As always, feel free to join us in #dendrite:matrix.org for the latest!

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo told us:

While work on federation is going on in the background, here are some other things we worked on this week

Also check out the Romeo and Juliet Benchmark!

Thanks to everyone who supports me on Liberapay or Bitcoin!

Synapse

Neil said:

This week we added some more federation readers to matrix.org which should noticeably improve federation lag.

2020-08-21-2HGwe-Screenshot2020-08-21at14.59.34.png

Patrick has continued on with the async/await-a-thon and we are starting to enter the home stretch. Of the things we care about only runInteraction remains. Once done we can get on with profiling the main process.

2020-08-21-u40Ex-Screenshot2020-08-21at15.08.51.png

On the sharding side the last big milestone is to get the event persister to shard. I know this comes up every week, but we now have a rough working implementation for non-backfilled events. Next steps are to get backfill working, get the tests passing, go through review and then ship. I asked Erik if it would be live on matrix.org within 2 weeks. He didn’t say, at least I didn’t hear him say no…> Aside from that Rich is working on the push rules redesign, the next steps are to design the APIs in more detail and then make a formal spec proposal. Brendan is also in notifications land and his work on room unread counts is going through final review.

Finally, Andrew has been working with Sorunome to dust off the knock feature, both on the spec and the Synapse side.

Romeo and Juliet benchmark to compare Matrix server implementations

timo offered:

This week I created the first(?) Matrix server benchmark and ran it on Synapse, Dendrite and Conduit. The test will go through the entire Romeo and Juliet play and create users for each character and sends messages for each line they say.

The results are very interesting:

Synapse:

     timo         neilalexander    Half-Shot

# default (sqlite)
time 5m0.870s

# postgres:
time 1m46.319s                     1m8s


Dendrite:

# default (sqlite)
time 6m8.802s     0m38.502s

# postgres:
time 2m45.387s    0m53.691s


Conduit:

# default (sled)
time 0m4.184s     0m2.776s         0m2.935s

2020-08-21-KRqaI-Screenshot_20200818_223636.png

There is some question here about how federation and state resolution will affect performance on these tests.

Synapse Deployment πŸ“₯️

There is rather a lot in this section this week (see also Ops πŸ›  section below.) I'll consider making this a more structured (table?) feature in future.

SELinux policy module

0xC0ncord reported:

Hello! Just recently I made public an SELinux policy module I developed for Synapse. This policy is one I developed early on during my trip down the SELinux rabbit hole, and I have been using it in production for some time. At the moment, this policy module is built for the Gentoo SELinux policy (not RHEL/CentOS/Fedora) and does not currently have systemd support, nor has it been tested in all environments, i.e. with workers. I am publishing this policy in the hopes that it will be useful and I am open to any contributions, even those to help tighten the policy if possible. Finally, I have also made public a policy module for coturn, which I am also using in my own personal environment under the same conditions. Both the Synapse policy and the coturn policy can be found on Github. Thank you!

Ansible

JCG said:

The ansible collection famedly.matrix has seen another release, 0.1.2, which updates the element role to 1.7.4 and the synapse role to 1.19.0. As always, get it from Ansible Galaxy, source from Gitlab, and for any questions join #ansible:famedly.de.

Kubernetes

Ananace announced:

Just pushed the 1.19 tags for the K8s-optimized Synapse image, as well as an updated chart version for it that also takes advantage of the added /health endpoint.

Void Linux

JCG announced:

Element 1.7.4 and Synapse 1.19.0 have been packaged for Void Linux, packages are already available from mirrors.

dacruz21/matrix-chart Kubernetes Helm Chart

Typo Kign told us:

v2.5.0 of dacruz21/matrix-chart has been released with Synapse 1.19 support and some new configuration options.

YunoHost

Pierre reported:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration is still 1.18.0, 1.19.0 is available en branch testing

Element Web integration is still 1.7.3, 1.7.4 is available en branch testing

Dept of Bridges πŸŒ‰

Telegram public channel mirror

bo41 reported:

As channels are really popular in Telegram, I wrote a quick and dirty program/bot which can mirror (text and image) public Telegram channels to Matrix rooms. This should make it easier for Telegram users to consider a switch.

Making the rooms read-only via permissions prevents messages from users. It provides a good basis for someone trying to make a real 1to1 mirror. Please fork this project and host some channels publicly ;)

https://git.sr.ht/~bo41/matrix-channel-mirror

This project is functionally quite different from the more established mautrix-telegram: it is purely designed to mirror content from Telegram public rooms.

matrix-appservice-irc release 0.20.1

Half-Shot said:

Hello all, I'm going to be quick off the mark and announce matrix-appservice-irc release 0.20.1! This release contains a ton of bug fixes and quality of life changes that you never knew you were missing. Please update and report back in the usual channels as always. Thanks!

Dept of Clients πŸ“±

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) reported:

I've been working on trying to finish the event store (when I wasn't distracted by video calls with some people). This should allow limited offline use and reduce memory use as well as allow some other features in the future. If you want to try it out, you can build this PR or try out the nightlies with new-event-store in their name.

If you decide to try it out, backup your cache or at least BACKUP YOUR ENCRYPTION KEYS! I won't be responding to feedback immediately, since I am currently hiking, but I should get to it this weekend or next week. Come discuss it in #nheko:ocean.joedonofry.com, if you have some feedback or issues.

See also Matrix Live this week!

Element Android

benoit announced:

We are still fixing issues on Element Android. Hopefully we will release version 1.0.5 of the application and of the Matrix SDK in the coming days!

later, fidèle à sa parole:

Element Android 1.0.5 has been published on the beta channel of the PlayStore. If everything is fine it will be pushed to production next week. I've also published SDK v1.0.5, and push translations of the PlayStore home page of Element to German, Chinese, English US, Hungarian, Portuguese (Brazil), Russian, Slovak, Swedish and Ukrainian.

Element-iOS

steve offered:

This week, we improved customisation for forks. We add possibility to customize default HTTP headers (like "User-Agent") and improve login and room settings screens customisation. We started to improve project documentation. We also made some fixes like fixing registration validation link in email.

Element Web

Ryan announced:

  • Released 1.7.4

    • Updated various links and strings for the new brand

    • Renewed Windows code signing cert

    • Removed rebranding toast

    • All cross-signing keys now shared with other devices during verification

  • This week

    • Mode to require secure backup during registration

    • Continue work on deferred cross-signing setup

    • Light weight Hydrogen client continues, adding key features and design tweaks

    • Notifications continues in background while push rules evolve

    • Matrix.to link previews in progress

Hydrogen

Hydrogen is a web-based client designed to be very lightweight, and with few dependencies. Anyone who wants to discuss Hydrogen can join #hydrogen:matrix.org.

Bruno told us:

Added several features this week:

  • avatars and display names in the timeline and for rooms

  • implemented the logic for the member list, but no UI for this yet

  • room list sorting by last message timestamp, unread state and badge count

  • clear unread state when reading a room

  • some visual tweaks in the timeline and room list

  • bug fixes

If you have checked out hydrogen before, you'll need to clear your session (the little clear button in the session picker) to get correct avatars and display names everywhere.

Next week I'll get started on E2EE, which should make the project a lot more useful. I'll look into leveraging the nice rust matrix-crypto-sdk for this.

2020-08-21-qAi-A-image.png

Mirage 0.6.1

miruka hit us with a late breaking update about Mirage!

Mirage 0.6.1 was released today:

Added

  • SSO authentication support

  • Homeserver browser:

    • To add a new account, you will be asked first to pick one of the listed public server (list data from anchel.nl) or to manually enter a server address
  • Typing in the server address field will also filter the public server list, Up/Down or (Shift+)Tab and Enter can be used to navigate it by keyboard

  • If the address doesn't have a scheme://, auto-detect whether the server supports HTTPS or only HTTP

  • Use the .well-known API if possible to resolve domains to the actual homeserver's URL, e.g. matrix.org resolves to https://matrix-client.matrix.org

  • The server address field will remember the last homeserver that was connected to

  • Room members autocompletion:

    • Type @ followed by one or more characters in the composer, or one or more characters and hit (Shift+)Tab to trigger username/user ID autocompletion

    • Only autocompleted names will be turned into mentions, unlike before where any word in a sent message that happened to be someone's name would mention them

  • Full image viewer for matrix image messages and URL previews:

    • Click on a thumbnail in the timeline to open the image viewer

    • Middle click on a thumbnail (or use the option in the context menu) to open the image externally

    • Left click on the image (mouse only): expand to window size if the image's origin size is smaller than the window, else expand to original size

    • Tap on the image (touch screen/pen only): reveal the info and button bars when auto-hidden (bars will auto-hide only when they overlap with a big enough displayed image)

    • Any mouse movement: reveal auto-hidden bars

    • Double click on the image: toggle full screen

    • Middle click anywhere: open externally

    • Right click anywhere: close the viewer, back to chat

    • Drag when displayed image is bigger than window to pan

    • Wheel to pan up/down, hold shift or alt to pan left/right

    • Ctrl+wheel to control zoom

    • Buttons to control rotation, scale mode, full screen, GIF play/pause and GIF speed

    • New keyboard shortcuts are available for all these actions, see keys.imageViewer in the config file (will be automatically updated when you start Mirage 0.6.1)

  • Add media.openExternallyOnClick setting to swap the new click and middle click on thumbnails behavior

  • Room and member filter fields now support (Shift+)Tab navigation, in addition to Up/Down

  • Add a colored left border to the currently highlighted item in list views (e.g. room list, members list, etc) to improve visibility

  • Themes:

    • Add controls.listView.highlightBorder and controls.listView.highlightBorderThickness properties (can be set to 0)
    • Add the chat.userAutoCompletion section

Changed

  • Messages context menu:

    • Use a cleaner icon for the "Copy text" entry

    • Replace the confusing broken "Copy media address" entry with:

      • Copy media address: visible for non-encrypted media, always copies the HTTP URL

      • Copy local path: always visible for already downloaded media, even if they were downloaded before mirage was started

  • The openMessagesLinks keybind (default Ctrl+O) is renamed to openMessagesLinksOrFiles and can now also open media message files

  • Using the openMessagesLinksOrFiles keybind on a reply will now ignore the matrix.to links contained in the "In reply to XYZ" header

  • Pressing Ctrl+C to copy selected/highlighted non-encrypted media messages will copy their HTTP URL instead of the filename

  • Retry downloading image thumbnails if they fail with a 404 or 500+ server error (uploads sometimes take a few seconds to become available on the server)

  • Non-encrypted media messages are now always downloaded on click and opened with a desktop application (or the image viewer), instead of being opened in a browser

  • Compress thumbnails and clipboard images in a separate process, to avoid blocking every other backend operation while the compression is running

  • Reduce the level of optimization applied to clipboard images, the previous setting was too slow for large PNG (10MB+)

  • Increase applied scrolling velocity when using the scrollPageUp/scrollPageDown keybinds, now similar to how it was before Mirage 0.6.0

  • Don't catch SIGQUIT (Ctrl+\ in terminal) and SIGTERM signals, exit immediately

  • Slightly increase the top/bottom padding to the multi-account bar in the left pane

  • Dependencies: minimum nio version bumped to 0.15.0

Removed

  • Themes: remove unused controls.listView.smallPaneHighlight property

And lots of fixes, see full changelog

Mirage

Dept of SDKs and Frameworks 🧰

Ruma

iinuwa offered:

We wanted to take a little bit of time to show off a state resolution library for Ruma, courtesy of @DevinR528 and @Timo. Ruma has a new crate to make resolving state easier, state-res! The goal of this crate is to be the base for federation, it provides the necessary utilities to resolve state, sort power events (reverse topological ordering), sort events (based on mainline depth), and a lot of event authentication functions. It is still in the early stages of development, but will be hopefully proved out in Conduit.

Other things that happened this week:

  • Added the invite endpoints from the Federation API

  • Started requiring strong types for event-sending endpoints

  • Augmented error types with error-specific fields (see the livestream here)

Speaking of livestreams, @jplatte streams coding sessions from time to time on Twitch. If you're interested in watching some coding streams to get more familiar with the Ruma codebase or learn Rust in general, follow us at [#ruma-livestreams:matrix.org](https://matrix.to/#/#ruma-livestreams:matrix.org)!

Very impressed that these features are landing in Ruma first, with the intention of getting them into Conduit.

mautrix-go

nikofil reported:

  • SSSS and cross-signing are slowly making their way to mautrix-go! A WIP PR exists for both features and is being developed upon. This will allow any clients that use the library to eventually be able to use SSSS and verify the trustworthiness of other devices based on cross-signing.

Ruby

Ananace announced:

Just pushed version 2.1.1 of the Ruby Matrix SDK, mainly including a couple of fixes for various minor issues.

Dept of Ops πŸ› 

matrix-docker-ansible-deploy

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi said:

matrix-docker-ansible-deploy now makes it incredibly easy to use rust-synapse-compress-state to compress the state groups in your Synapse database.

See our Compressing state with rust-synapse-compress-state documentation page to get started.

Dept of Bots πŸ€–

go-neb

nikofil said:

  • Device verification was merged to neb! It needs human intervention to approve that the SAS match, of course.

  • A new service called "cryptotest" was added that allows other clients to exchange e2e-related events with neb to test their capabilities (such as encryption, room key forwarding, SAS verification)

Final Thoughts πŸ’­

Christian told us:

Residents of Germany can apply for the Prototype Fund until the end of September. They are looking for new open source ideas or features and fund about 20 projects twice a year with 47,500 EUR for six months (starting in April for this round).

Previously funded chat-related projects include Briar and OpenPush. It would be great to see a project building on Matrix.

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1fairydust.space345
2maescool.be865
3chatserver.ca985
4nuclearlimes.co.uk1206
5utzutzutz.net1826
6nct08.de1885
7kapsi.fi1994.5
8mailstation.de2324
9envs.net4053
10kittenface.studio4076

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2020-08-14

14.08.2020 00:00 β€” This Week in Matrix β€” Ben Parsons

Matrix Live πŸŽ™

No new Matrix Live this week, we have Open Tech Will Save Us #5:

Also, I recommend Matrix Live from last week, with Timo and Matthew discussing Conduit/Rust/Homeserver dev. Conduit is absolutely motoring, as you will read below.

Dept of Status of Matrix 🌑️

LinkΓΆping University has now soft-launched Matrix

Alexander Olofsson said:

In completely unrelated news to my earlier K8s updates, LinkΓΆping University has now soft-launched Matrix for use both internally - and hopefully with other universities and institutions in the future.

Se the k8s updates from Ananace below. Is this the first .se university offering Matrix? Also, great hostname!

Dept of Spec πŸ“œ

SAS Emoji Translations

TravisR said:

We've finally opened up https://translate.riot.im/projects/matrix-doc/sas-emoji-v1 to translate the emoji used by clients to ensure that users are securely verifying each other. The idea is that clients should always have a consistent name for these emoji, even if developed by completely different teams. If your client supports encryption verification and has translations, please transition to using matrix-doc for your translations: https://github.com/matrix-org/matrix-doc/pull/2728 has some details, and #matrix-spec:matrix.org would love to help out.

2020-08-14-Ly4te-image.png

Spec

anoa said:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Spec Core Team

Some of the team was on holiday last week, hence the noticeable lack of progress πŸ˜‡ In terms of Spec Core Team MSC focus for this week, we're keeping on with the same two as last week: MSC2674 (aggregations pt1: relationships) and MSC1544 (QR code verification).

2020-08-14-i2G3v-stacked_area_chart.png

Dept of GSoC πŸŽ“οΈ

Opsdroid(GSoC)

tyagdit announced:

Matrix database

  • The port to nio is complete and most of the functionality has been worked out

  • Encryption support is being worked on

  • The module now has a delete method

  • Some testing is remaining

  • Check out the progress here

End to End Encryption with matrix

  • Opsdroid will soon get a new release with the new connector

  • For now it can be used through the master branch of opsdroid

Dept of Servers 🏒

Synapse

Neil reported:

We have a new RC out, edited highlights include

  • lots of async/await changes, 27 separate PRs (thanks Patrick)

  • improved logging performance, this is in part to reduce IO but also to reduce the total amount logged. We’d be really interested to get some feedback on if the changes help. You may also want to take a look at our new logging defaults to see if they would help reduce total disk space usage.

  • Further improvements to the Admin API allowing admin to optionally purge rooms on deletion.

Expect 1.19.0 to land early next week.

Aside from that Erik has been continuing with sharding the event persister, this is a big job, so we’re still a little way off having something to ship, but I’ll keep you updated.

Big news this week, after spending a good chunk of time trying to rework notifications to make the UX more intuitive, we have decided that the best (and only) way forward is to completely rethink how the push rules work. Rich is currently redesigning the whole thing, and once done we’ll resume on the client side.

Having spent much of the past 3 months trying to improve the performance of large scale instances we will soon start looking at improving the out of the box experience for smaller installs and are in the early stages of thinking about what that might look like. I’ll update more over time, but the two obvious use cases to target are

  • Federating instance, with 1-5 users, resource constrained.
  • Federating instance with a few hundred users.

Our (relatively small) changes to logging in 1.19.0 are the first signs of this shift. We still need to ship event persistence sharding and also move more functionality from the main process (dependent on async/await), but once done it clears the way for new projects.

2020-08-14-2cR_s-image.png

anoa brought it to our attention that this graph looks like a "ocean-beach waterline dropoff".

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo reported:

This week I worked a bit on federation! Last week we were able to query room directories over federation; This week I looked into what's needed to join one of those rooms:

  • Resolve room aliases over federation

  • Send make_join request when a user tries to join an unknown room

  • Use make_join response to create send_join request (insert origin, origin_server_ts and add hashes and signatures)

After implementing all those things, I could actually see Conduit's join event on Synapse!

2020-08-14-bRuiV-Screenshot_20200814_125755.png

Of course a lot more work is needed to make this really work,

like creating the room on the Conduit side and sending and receiving federation transactions, but this is a start.

Other news:

  • Docker support (thanks to @weasy, @valkum and @paul)

  • Room upgrade support (almost done, thanks to @Faelar)

Thanks to everyone who supports me on Liberapay or Bitcoin!

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander told us:

This week we've been closing off some of the remaining tasks for end-to-end encryption and working on tasks from our beta milestone task list.

Changes this week include:

  • New format for the Dendrite configuration file, which is correctly broken up into configuration per-component

  • Support for server ACLs has been added

  • Significant refactoring of the HTTP routing and API setup to improve flexibility and separation of internal and external APIs

  • A number of improvements to device list handling

  • Key updates are now sent when handling device display name changes

  • Deleted devices are now propagated correctly over federation

Spec compliance is improving:

  • Client-Server APIs: 54%, up from 51% last week

  • Server-Server APIs: 70%, up from 54% last week

As always, feel free to join us in #dendrite:matrix.org for the latest!

I'm enjoying the close rivalry and progress happening in Dendrite and Conduit right now!

Sytest

Neil Alexander reported:

Sytest has now been updated to allow skipping tests for endpoints that have been deprecated in the spec. This may be useful if you are writing a homeserver and have no plans to implement deprecated endpoints, e.g. /initialSync or /events.

Use the command line parameter --exclude-deprecated when starting Sytest to skip these tests!

Synapse Deployment πŸ“₯️

YunoHost

Pierre reported:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.18.0

Element Web integration had been updated to 1.7.3

Kubernetes

Ananace reported:

Just pushed some WIP (v0.1) charts for both Synapse and the matrix-media-repo project, written with Redis, workers/sharding, PVC constraints, and other large deployment thoughts in mind.

Note that these charts - the Synapse one in particular - basically require you to have a fully featured cluster, with a regex-capable ingress, automated cert manager, and RBAC. They do not require you to have a ReadWriteMany capable storage class though, even when using workers.

and then:

To expand on my earlier TWIM, the charts for Synapse, media-repo, and now element-web are all being used in production now so they are also respectively receiving their 1.0's

Dept of Clients πŸ“±

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) said:

So, we've been silent for a bit, but that doesn't mean we haven't been busy, so I have a little bit of a bigger announcement to make today:

Trilene implemented voice call support in Nheko! We just merged this and you can try it out now!

Some caveats apply at the moment, but will be resolved in the near future:

  • Currently you will need to have a patched version of GStreamer. The current releases of GStreamer have a bug when your turnserver uses a password, that contains a ':'. Since this is the default for coturn, I'm guessing this affects everyone.

  • You will need to build Nheko yourself. Currently we do not have GStreamer available in our CI pipeline, so we disabled support for voice calls in those builds (apart from the flatpak build, but that also needs some further validation and a new GStreamer release). We will release proper builds with VOIP support once GStreamer is actually released and available with the fixes we need.

  • There are still some other issues, when relying on a turnserver to connect a call. We will fix them one at a time.

Big shoutout to trilene for implementing all of this! I can finally call people from Nheko, which is super awesome!

This was the first of the bigger features, that have been cooking for a while, in the near future we should also finish the new event store, which reduced memory usage and provides some kind of offline support, and Chethan will probably finish the first iteration of cross-signing soon too. After that we'll focus on the smaller features again. ;-)

BIG update! Looking forward to trying out voice calls. ☎️ Maybe we'll be calling Nico for Matrix Live soon..?

Element-Android

benoit said:

We are still fixing issues following the Element release. We are also working to add Jitsi conference call support.

We have exported a first beta version of the SDK from Element Android. It is available here: https://github.com/matrix-org/matrix-android-sdk2 and can be integrated in any Android app as a regular gradle dependency. We will update the SDK after each release of Element Android.

SchildiChat for Android

SpiritCroc said:

SchildiChat version 1.0.4-sc.12 should be up in the Play Store soon.

It includes following changes since the last TWIM:

  • The single-list overview screen is now optional. If you prefer the tabbed layout for separating direct messages and rooms, as known from Element for Android, but still want SchildiChat's message bubbles, you can get that combination now too!

  • Push notifications have been enabled for the Play Store variant

  • Some theme fixes for old Android versions

Hydrogen

Bruno told us:

Some regular readers might remember, in the past I've put a few entries here about a side-project called Brawl. Two weeks ago, we took Brawl and started building a new client on top of it code-named Hydrogen.

As the name implies, Hydrogen tries to be the lightest Element. It is written entirely in vanilla javascript (no React, no Webpack) for complete control, structured as an MVVM app, leveraging the raw performance of indexeddb. On huge accounts where Element-web uses up to 1GB of heap memory, Hydrogen sips away with a mere 15mb. It is very early days though, and Hydrogen has very little features: you can receive and send unencrypted messages and that's it.

Hydrogen also aims to support a wider range of browsers, from IE11 (particularly for orgs with locked-down machines) all the way to a fully featured PWA that works well on mobile. We should also be able to easily add progressive loading, which proved hard in the past.

Another goal is to make it easy to embed parts of the app (like the room view) into another app or website. It could also be really nice to have a place to play and experiment with new tech for element proper - much as Dendrite has influenced Synapse.

As mentioned, it's early days, and lots of things are missing or broken, so please don't file any issues yet. Feel free to play around with it at https://hydrogen.element.io/ though!

Hydrogen

Find the repo here: https://github.com/vector-im/hydrogen-web/.

Element-iOS

ismailgulek offered:

This week, we've completed some of the tasks from the sprint: https://github.com/vector-im/element-ios/milestone/38.

As a highlight, we've released a new TF with PushKit & CallKit back.

Dept of SDKs and Frameworks 🧰

Ruma

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

iinuwa announced:

Lots of exciting things happened in Ruma this week:

  • We have officially implemented over half (18/31) of the Federation API endpoints!

Special thanks to @agraven and @q-b!

  • We've added the only new endpoint from r0.6.1 of the C2S spec.

  • We have begun work on adding support for borrowed types for requests. This should reduce the memory allocations required for clients to send requests to

a server.

  • Exported some macros to make creating custom endpoints and events that

much easier.

New Contributors

This week, we have gotten help from a few new contributors:

Thanks for your help!

If you're interested in contributing, join the party on GitHub or

#ruma:matrix.org!

Great to see new contributors here.

Dept of Ops πŸ› 

Support for Ansible collections from Famedly

JCG told us:

There's news from ansible land! Ansible recently introduced a new packaging format, the "collection", and we've now started moving our ansible content into collections. This means the matrix modules I twim'ed a few months back are now easier to install than ever before, and the synapse/element roles are included in there as well (thanks to madonius for the contributions there). The roles have been regularly updated for a while now, and the element role isn't even called riot anymore! You can get the collection on Ansible Galaxy, with the source being available over on GitLab. In case of any questions, feel free to hop over into #ansible:famedly.de

If Ansible were a film franchise "Ansible: The Collection" would be the long awaited boxset.

Dept of Bots πŸ€–

Cody

carl said:

cody - the REPL for your matrix room - received support for unit conversions this week. This is done by implementing GNU units as a language.

Chat with cody: @cody:bordum.dk

Join our cody room: #cody:bordum.dk Read the source: https://gitlab.com/carlbordum/matrix-cody/

2020-08-14-cody.png

Seems to work well!

Opsdroid

Cadair reported:

opsdroid is getting close to a release with the matrix connector rewritten to use the matrix-nio library. If you are feeling brave and want to help us iron out any bugs before the release, please install the master branch and test it out for your matrix bots. Please report any issues on the GitHub issue tracker or come talk in #opsdroid-developers:matrix.org. Hopefully, a release with this and lots of other goodies will be coming in a couple of weeks.

Dept of Interesting Projects πŸ›°οΈ

Privee

terry_hello said:

Hi All,

I’ve created a compact and portable home server based on Matrix protocol, it’s a piece of hardware which is easy to use and set up. I’m working on a new client APP similar to other typical messenger GUI, but this server we named it β€˜Privee’ supports any messenger developed with Matrix protocol. I’ve launched a campaign on Kickstarter this week, please come and support us! Thank you! https://www.kickstarter.com/projects/privee/privee?ref=beibnw

Privee is an interesting project alright! The project creators are based in Hong Kong, and are working from a desire to have more control over their communications. The plan is to run homeservers on a small computer physically located with the user, similar to plans outlined by NovaChat.

This is an early-stage project (hence starting on KickStarter.) While exciting and ambitious, I encourage everyone to do their own research! Learn more and chat with terry_hello (who has done a good job handling questions already!) in #Privee:matrix.org.

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1helderferreira.io470
2maescool.be486.5
3matrix.vgorcum.com570.5
4mchus.pro730
5envs.net770
6maunium.net983
7matrix.org1261
8acmelabs.space1282.5
9chatserver.ca1387
10productionservers.net1433.5

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2020-08-07

07.08.2020 00:00 β€” This Week in Matrix β€” Ben Parsons

Matrix Live πŸŽ™

Dept of Spec πŸ“œ

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Dept of GSoC πŸŽ“οΈ

go-neb

nikofil reported:

  • Handle SAS verification by adding an endpoint where the decimal SAS code generated by another device can be sent. If this code matches with go-neb's generated code, the verification succeeds and the device is marked as trusted.

  • Add a cryptotest service which can be used for other clients to test their functionalities related to e2ee. This currently includes a challenge / response, key forwarding functions (key request / key forward) and SAS verification. SSSS and cross-signing are planned for the future. (PR is under review)

Dept of P2P πŸ‘₯

iOS P2P Demo

Dendrite is a next-generation homeserver written in Go

Neil Alexander said:

The Element P2P demo on iOS has seen a number of updates in the last week - now at build 31 - which contains a number of fixes:

  • QUIC session setup is now far more reliable

  • QUIC sessions are now reset when the node's network coordinates change

  • The federation sender now wakes up properly on incoming QUIC sessions

  • Network coordinates are now cached for remote federated nodes, reducing the number of DHT searches and speeding up resuming federation

  • HTTP pools are disabled, which reduces head-of-line blocking problems on stalled sessions

  • A number of timeouts have been lowered

  • Dendrite has been updated to the latest version

If you have an iPhone or iPad and want to play with the demo, grab the public TestFlight!

Dept of Servers 🏒

matrix-media-repo v1.2.0

matrix-media-repo is a highly customizable multi-domain media repository for Matrix

TravisR told us:

v1.2.0 is out now, with a bunch of bug fixes and some added features:

  • Thumbnail support for apng and webp images (thanks Sorunome!)

  • Experimental (but pretty stable) support for Redis as a cache - instructions here

  • Optional oEmbed support for things like previewing tweets - turn it on here

  • Dynamic thumbnail support

  • Per-user quotas

v1.2.0 is also much faster at file uploads, however some areas of matrix-media-repo can still be improved - general performance work is planned for the next release.

Learn more about matrix-media-repo.

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander announced:

We are happy to announce the beta milestone for our first versioned release of Dendrite, which we anticipate to land in the next few weeks. Once these issues are cleared, we'll have made the big breaking changes that we want to make and will have a migration mechanism in place for database schema upgrades.

This will make life significantly easier for anyone who wants to start experimenting with and running their own Dendrite homeservers without unexpected breakages. More updates to follow soon - join us in #dendrite:matrix.org for the latest!

This week, Dendrite's end-to-end encryption support has also been mostly rounded off with the exception of a few minor bugs and edge-cases.

Changes this week include:

  • TLS certificate verification is now enabled by default

  • A new federation_disable_tls_validation config option has been added to override the above

  • Added support for processing inbound device lists from federation

  • Local device list changes are now sent outbound over federation

  • One-time key counts are now sent in /sync

  • Direct server signing key fetches are now limited to 15 seconds

  • State events that fail signature checks on federated room joins are now properly ignored

  • A number of SQLite database lock bugs have been fixed using TransactionWriter

  • A bug when finding joined hosts in the federation sender in SQLite mode has been fixed

  • Logging levels have been adjusted in a number of areas

Spec compliance:

  • Client-Server APIs: 51%, same as last week

  • Server-Server APIs: 54%, up from 53% last week

Asked whether this progress means that matrix.org would be running on Dendrite next year, Neil said he was certain that it would definitely be ready:

Anything is possible

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo reported:

  • Improve long polling when user is invited (thanks @gnieto)

  • Documentation and code cleanup

  • Getting initial server_server code back into shape

Thanks to everyone who supports me on Liberapay or Bitcoin!

Oh and btw look at Matrix Live this week!

Synapse

Patrick offered:

This week in Synapse land we continued with our performance push making further progress on the async/await-athon. We will move into the storage layer this coming week which will leave the caching layer and a few other bits outstanding. Separate work continued to get the event persister into a shardable state. The first step being to replace the event stream token with its own type and then threading that through the code base.

We have also continued to improve on our push capabilities and released a new version of Sygnal (0.8.2) with support for pushing to iOS devices via Firebase.

We are also very close to getting a workable version of the new push rules. The idea is to test it out in the wild before getting it enshrined in the spec.

Next week will see a continued focus on sharding and async/awaiting, more notifications work, logging improvements, and a host of bug fixes!

Dept of Bridges πŸŒ‰

maubot rss plugin

Tulir reported:

I released v0.2.0 of the maubot RSS plugin. The most recent addition is support for JSON feeds, stolen from inspired by FantasyCookie17's jsonfeed-bot.

Other changes since v0.1.0:

  • Per-feed toggle to send posts as m.text instead of m.notice
  • Automatically moving subscriptions across room upgrades
  • Some fixes for handling weird feeds

maubot/rss is a maubot plugin that posts feed updates to Matrix rooms. It supports RSS/Atom/JSON feeds, customizing message templates per-feed/room, and more.

mautrix bridges

Tulir reported:

All my bridges now have options for automatic e2ee key sharing in portal rooms. Currently they don't have verification support (other than manually marking as verified in the database), so enabling key sharing isn't entirely secure yet. I'll probably invent some sort of web verification interface after nikofil adds cross-signing to mautrix-go.

This also means mautrix-python now supports sending and responding to key requests. I'll port the interactive verification and future cross-signing code from mautrix-go to mautrix-python at some point too.

Dept of Clients πŸ“±

Fractal

Alexandre Franke offered:

4.4.0 is out as promised last week! Get it off Flathub now.

After we announced our intention to release, we discovered a regression that prevented joining new rooms because there was a mixup between room aliases and room id during a switch from internal structures to ruma-identifiers. Luckily we managed to fix it in time and make the release as intended.

Full changelog since 4.2.2

New features

  • Videos now have a preview directly in history

  • New videos player: click on a preview to open it

  • Improved audio file support, with a nicer presentation and seeking

  • Redacted messages are removed from history

  • Edited messages are replaced with their new content and show a small icon

  • Translations have been updated

  • Spellcheck language is saved per room

  • New swipe-back gesture to go back from room to room list in narrow view

  • New swipe left and right gesture to navigate through media in media view

  • SOCKS proxy are now also supported

Bugfixes

  • Thumbnail for pasted images have been fixed

  • File type detection for uploads is more reliable

  • Typing notifications are now only sent when text is actually typed in the message input

Under the hood

  • Code refactor is still going

  • Some improvements to logging, including access token removal for privacy

Element Android

valere told us:

Sorry we missed last week TWIM!

  • Element Android 1.0.4 πŸš€ has been released to the store, it contains several bug fixes, and is adding back some moderators functionality (see list of banned users for a room + actions)

The team has been mainly working on stabilisation + refactoring of the current code base.

We are also trying to give more visibility on what is coming next:

  • Pin code protection πŸ”“ (merged on develop)

  • Improvement of big media upload

  • Cancelling send, retries, better upload progress...

  • Call fixes and improvement

  • Improve fdroid polling for notification

The team is partially off, expect some delays in handling of issues and PR reviews

SchildiChat Android

SpiritCroc reported:

SchildiChat is a new fork of Element for Android (previously RiotX), which features some UI changes, to make it more comfortable to use for people who are used to other common instant messengers.

The most important changes are the following:

  • The overview screen lists all chats in one list instead of the tabbed layout in Element

  • Message bubbles (optional)

  • Theme choice based on the system-wide dark/light theme on recent Android versions

It is now available on Google Play, it might appear on F-Droid too in the future.

#schildichat-android:matrix.org is the place to find out more.

New AppImage for the Element Desktop Matrix Client

srevinsaju told us:

As per this issue comment, I was asked to inform this channel of the new AppImage for the Element Desktop Matrix Client. πŸŽ‰. Interested users may get them on appimage.github.io or on my releases page, offering Continuous builds (refreshed every day) for stable and beta builds πŸ˜ƒ

IndieChat announced

indiechat announced:

Hello, World! [IndieChat], a new Android matrix client is in Play Store: https://play.google.com/store/apps/details?id=chat.indie.cam

This is forked from FluffyChat:

I have changed the archive feature. Added a splash screen. Further simplified settings. Login times work faster compared to FC. I am connecting it to my Jitsi instance. which operates p2p and is completely analytics free. Message notifications are analytics free. It will eventually be an extension to a videocall app developed by myself. It adds variety to the ecosystem. It strictly follows license rules, credits the developers and links to their wonderful work. And it is a great thing to do in my spare time ☺️

Element-iOS

Manu said:

This week, we started a new sprint to finalise remaining rebranding and iOS13 SDK issues. Note that have a plan to make CallKit and PushKit back!

In parallel, we have finished to setup the CI for the 3 iOS projects.

Dept of Encryption πŸ”

Device dehydration

uhoreg told us:

I missed out on demo day last week because my demo wasn't ready yet. But I recorded a video, so here is what you would have seen if I had managed to finish on time: https://scitech.video/videos/watch/4aad91c3-30bb-479f-8437-a517354594db

Dept of SDKs and Frameworks 🧰

matrixbz

patrick announced:

simple matrix bots with matrix-nio

Hey all! I built yet another bot library. matrixbz was built with the intention of making it easy to draft matrix bots. Check out the github - I've added some examples where you can build a bot in ~10 lines of python code. matrixbz features:

  • Auth - you can specify user(s) who are authorized to call commands. bot only accepts invites from those users.

  • Cache - you can cache results for particular command invocations.

Check out the package.

Ruma

iinuwa reported:

We have released version 0.0.1 of the ruma crate! Thanks to all of our contributors!

Besides that and continuing to refine ruma-events, here are some other highlights over the past couple weeks:

  • Implemented a few more endpoints implemented on ruma-federation-api

  • Added helper macros for creating identifier types

Dept of Bots πŸ€–

Hemppa

Cos told us:

Hemppa the bot is a general use bot that has lot of modules for various uses and super easy API to create more. Hemppa received support for Mastodon. Now you can login to your Mastodon account and send toots via Matrix. Next major feature would be to subscribe to hashtags and users and get notifications to Matrix rooms. https://github.com/vranki/hemppa#mastodon

HOPEless, the HOPE conference bot

valka said:

The Hackers on Planet Earth conference went almost entirely self-hosted this year due to not being able to hold an in-person event, and used Matrix to go virtual! hopeless was the robot-heart of the conference, providing community+room ACLs for attendees. The bot also has a bunch of other features including:

  • configured admins sending notices to rooms

  • ability to manually and programmatically schedule announcements for scheduled events happening, like workshops or talks

  • the ability for users to re-join rooms and generate invites for themselves as necessary

  • admin commands for inviting attendees and presenters without tickets

  • an attempt to be friendly and solve people's common problems without a real person having to assist (surprisingly effective!)

Many thanks to the nio-template for existing, otherwise this wouldn't have been possible!

anoa (nio-template) creator responded:

Wonderful! I plan to give it some more love soon, including some docker magic to make it easier to run.

If you ever have any feedback about improving it, feel free to post it in #nio-template:amorgan.xyz :)

matrix-diskstation-bot

KB1RD said:

This is just a stupid simple bot based on T r a v i s R's matrix-bot-sdk-bot-template that forwards alerts from a Synology DiskStation NAS to Matrix room(s).

It's on GitHub and Docker Hub

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1fairydust.space426
2privacytools.io438
3saferutar.com487
4maunium.net534
5heitkoetter.net570
6chatcloud.net640
7encom.eu.org658.5
8envs.net697
9matrix.vgorcum.com847
10neko.dev879

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2020-07-31

31.07.2020 20:51 β€” This Week in Matrix β€” Ben Parsons
Last update: 31.07.2020 20:24

Matrix Live πŸŽ™

Dept of Status of Matrix 🌑️

Matrix is being used in the "world's biggest messaging and collaboration" deployment.

[Matrix will] be rolled out by the German education system to provide collaboration tools for half a million seats in the states of Schleswig-Holstein and Hamburg

Casual 500,000 user deployment, very nice!

Dept of Spec πŸ“œ

anoa announced:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Finished FCP:

MSCs in Final Comment Period:

  • No MSCs are in FCP.

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus for this week, we'll be continuing on with MSC2674 (aggregations pt1: relationships) and adding MSC1544 (QR code verification) on top.

2020-07-31-3hFLP-stacked_area_chart.png

Dept of GSoC πŸŽ“οΈ

karlik

karlik announced:

for matrix-ircd

  • Merged update for total move to async / await into development branch, (hopefully) soon to be merged to master

  • Made single-threaded project thread-safe for easily scaling up to potentially many threads

  • Added additional logging for tls and non-tls connections for more clear debugging

Dept of Servers 🏒

E2E Dendrite

Dendrite is a next-generation homeserver written in Go

kegan reported:

E2E Dendrite work is progressing, and we're making a lot of breaking changes ahead of our first stable release which you can expect in the coming weeks.

As for the changes this week:

  • Add a bare bones user directory. This is mainly to benefit the iOS Yggdrasil demo.

  • The dendrite-config.yaml configuration format has been overhauled. It's still in review but expect big changes here!

  • Fixed a bug with SQLite when handling large rooms (thanks @HenrikSolver!)

  • Device lists now work locally, with federation support to come.

Spec compliance is up from last week:

  • Client-Server APIs: 51%, up from 49%.
  • Federation APIs: remains at 53%.

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo announced:

Welcome back! This week was very productive:

  • Ignore users (thanks @devinr528)

  • Long polling (faster and more efficient /sync)

  • Fix and improve presence

  • Room tagging (thanks @gnieto)

  • /publicRooms pagination

  • Refactor endpoints into modules

  • Export endpoints as a library (useful for p2p!)

  • Config options to set the maximum upload size or disable encryption

  • and more!

Thanks to everyone who supports me on Liberapay or Bitcoin!

Synapse

Neil offered:

This week we shipped 1.18.0 which contained support for our worker sharding performance projects that you’ve been hearing about recently.

We also shipped a security and feature release of Sygnal (actually two each…). The main new feature being the ability to run behind an HTTP proxy which is useful in environments that mandate that all traffic go through a proxy.

Aside from that week we continued our async/await-athon and Patrick provided a visual update

2020-07-31-m2KRg-image.png

We’ve been documenting worker support and diving into sharding the event persisters. This is not a small job since it means every other part of the code base that assumes an integer stream id will need to be updated to be aware of multiple stream ids from multiple persisters.

Over in feature land we’ve been working on the notifications project and most of the backend support is done. The project will mean more intuitive defaults as well as a much better client UX in configuring notification settings.

Finally special thanks to Aaron Raimist for a bumper docs PR

Synapse Deployment πŸ“₯️

Kubernetes

Ananace offered:

Just pushed the 1.18.0 K8s-optimized Synapse images, now only doing the debian-based version as like upstream - though still including jemalloc. So latest/v1.18.0 and *-debian will be the same in this version and going forwards. Additionally, the *-debian tags will only remain for a release or two more before disappearing, as they're now superfluous.

Dept of Bridges πŸŒ‰

mautrix-telegram

Tulir offered:

I finally added automatic backfilling options to mautrix-telegram. Like my other bridges, it can backfill old messages when creating portals and missed messages when restarting, and it can disable notifications in the room while backfilling.

I also added bridging for your own read receipts, so if you read a message with another Telegram client, the bridge will mark it as read on your Matrix account with double puppeting.

Dept of Clients πŸ“±

NovaChat

eric reported:

NovaChat is a new Matrix-based desktop client that aggregates all your chat networks into one app

July 31 Updates:

  • New room list design, featuring collapsible 'clusters' (see gif)

  • Enabled encrypted room search (thanks #seshat:matrix.org!)

  • Added [mautrix-twitter]( https://github.com/tulir/mautrix-twitter)

  • Worked around FB Messenger bridge 'forced-logout issue' by routing traffic through proxy on Raspberry Pi at each users home

In the works...

  • Create new clusters of chats in room list, using tags

  • iMessage bridge, using jailbroken iDevice

  • Android Messages bridge

Adding new users weekly. Sign up for the NovaChat beta. Or send me a DM @eric:nova.chat

2020-07-31-yJ-jJ-NovaChat-Newroomlist.gif

gomuks

gomuks is a terminal based Matrix client written in Go. Source on GitHub

Tulir told us:

gomuks can now do interactive verification for e2ee. Most of it is thanks to nikofil's SAS verification PR to mautrix-go, I just made a UI for using it. Currently gomuks can only send verification requests, but I'll add some interface for accepting incoming requests soon.

Related to verification, there are some new commands for verifying/unverifying/blacklisting devices and viewing device lists. There's also a toggle to disable sending messages to unverified devices.

This is super exciting, raises gomuks to the level of a daily driver IMO.

Fractal

Alexandre Franke announced:

We will release version 4.4 next Friday. Now is a good time to test the nightly version and check if it doesn’t break anything with your homeserver compared to 4.2.
The changelog has been sparsely given in previous TWIMs, but you could have a look at https://gitlab.gnome.org/GNOME/fractal/-/commit/463e6bc9988e2d8fffbfc38058f8d0b12cd074d7

Element-iOS, previously Riot-iOS

Manu announced:

We released 1.0.2 this week to the AppStore and on TestFlight. It's mainly a bug fix release after the Element rebranding.

Dept of SDKs and Frameworks 🧰

mautrix-go

nikofil offered:

SAS verification support has been merged! Support for it should also be landing in go-neb soon, both for verifying with the bot's device and for testing with go-neb's client testing service. Next up, looking at SSSS and eventually cross-verification support.

Dept of Ops πŸ› 

YunoHost

Pierre told us:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.15.2 (1.17.0 available in branch testing)

Riot Web integration had been updated to 1.6.8 (Element 1.7.1 available in branch element)

Final Thoughts πŸ’­

Tobi said:

We, some German Synapse admins, have created a survey about the usage of Matrix-Synapse for private statistic purpose (results will be published alike). As suggested in #synapse:matrix.org, this could also be a topic for "TWIM".

Find the survey at http://go.kabi.tk/MatrixQuestionnaire2020.

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1kleinhirn.net366
2fairydust.space383
3tchncs.de394.5
4mjdsystems.ca506
5matrix.org587
6nobelium.no753.5
7pixie.town761
8matrix.linux.pizza1111
9uraziel.de1230.5
10yyyyyyy.ml1314

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2020-07-24

24.07.2020 00:00 β€” This Week in Matrix β€” Ben Parsons

Matrix Live πŸŽ™

Dept of Status of Matrix 🌑️

New guide for client e2ee implementation, including cross-signing

Thanks and congratulations to Sorunome this week for releasing a new guide: Implementing more advanced e2ee features, such as cross-signing. This is a hugely detailed guide detailing the necessary steps to enable Cross-Signing and verification. Thanks as well to uhoreg and poljar for their work in reviewing the text!

New Masters Thesis written on Matrix

Julian Sparber announced:

I wrote a master thesis that tries to create e2e encryption based on Ethereum on top of Matrix. It may not solve all problems but it's an interesting experiment

https://blogs.gnome.org/jsparber/2020/07/20/i-finished-my-masters-degree/

I didn't get to look at this yet but it's exciting!

Dept of Spec πŸ“œ

kitsune (guess who's back!?) said:

I have put the long-awaited MSC2312 out of WIP - this is about matrix: URIs in case you haven't heard - and it's already in active discussion at #uri-scheme-proposal:matrix.org! Join the fun to make it the most awesome! The upcoming Quotient 0.6 has an implementation of those so adventurous Quaternion users can start experimenting right away.

Spec

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Abandoned MSCs:

MSCs in Final Comment Period:

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus, we've reduced last week's list of 5 MSCs down to one, MSC2674 (aggregations part 1: relationships). Next week, we're going to focus entirely on that MSC to make some headway on finally getting aggregations into the spec.

2020-07-24-oRVoi-stacked_area_chart.png

Dept of GSoC πŸŽ“οΈ

HTML embeddable Matrix client

arnav-t announced:

For the GSoC project HTML embeddable Matrix client this week:

  • Added status bar for connection errors

  • Added support for displaying typing notifications

Dept of P2P πŸ‘₯

P2P / Dendrite

Dendrite is a next-generation homeserver written in Go

kegan announced:

The P2P Yggdrasil iOS demo has seen some improvements:

  • Federation sender blacklists are now persisted which is important as the iOS app will be frequently terminated.

  • SQLite3 now uses TransactionWriters in more places to reduce the amount of 'database is locked' issues.

  • Federation sender has had much of its storage code refactored and de-duplicated.

In addition, there has been much work getting Dendrite to support E2E rooms, which it now mostly does.

You can create and join E2E rooms and have a conversation locally, though there's issues with it over federation. In addition, Dendrite currently lacks device lists so adding new devices may result in unable-to-decrypt errors:

  • Send-to-device events now work over federation.

  • Device key uploads and querying now work both locally and over federation.

  • One-time key uploads and claiming now work both locally and over federation.

Spec compliance is up from last week:

  • Client Server APIs: 49%, up from 48%.
  • Federation APIs: 53%, up from 51%.

Doesn't it feel like we'll be using Dendrite in the wild really soon now?

Dept of Servers 🏒

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo offered:

Another quiet week. Next week should be more exciting again.

  • Moved to stable rust (rocket now compiles on stable!)

  • Implement /joined_rooms (thanks @aura)

  • Add max_request_size config option to set the maximum size for file uploads and other requests (thanks @CapsizeGlimmer)

Synapse

Neil said:

This week we continued on our async/await athon, you can track progress here, expect steady progress over the coming weeks.

We dusted off the notifications project working with our pals in the Element client teams to ensure that the push rule defaults make sense. On the flip side we made further improvements to Sygnal and will merge the ability to have it sit behind a proxy rsn.

We are also trying to figure out how media retention limits should work in Synapse.

Up next is to continue our performance push. We need to spend time between pulling things off the master process as well sharding the event persister. We will continue with notifications and maybe finally squash https://github.com/matrix-org/synapse/issues/2528

Dept of Bridges πŸŒ‰

Introducing matrix-figma

Half-Shot reported:

Hello people. It's been a while since I've talked about a new bridge...or actually any bridges. Fear not, now you can hear both! Today I'm announcing the birth of another project matrix-figma. Figma is an online collaborative design tool. It was created to scratch an itch with the Element design team, who wanted to see comments appear in their Matrix rooms in realtime when someone commented on a Figma file.

The bridge works by first joining it to an "admin room", which functions as a permitted user list. The bot can then be invited to any other room so long as the invitee is in that admin room, and can then ask the bot to start directing comment notifications into the room.

The room uses room state to hold configuration, so you can host the bot as a docker container anywhere without the need for any support files!

I can announce that the bridge is now functional for simple use cases such as these, with more interesting functionality in the future.

Please check out https://github.com/Half-Shot/matrix-figma to see if the bridge is right for you :)

slack-matrix-migration

Sven offered:

I opened a PR for https://github.com/Awesome-Technologies/slack-matrix-migration, which makes it compatible with recent Synapse versions and adds new config options allowing to effectively import a Slack workspace history to an existing homeserver via federation.

This Python project was initially intended for a full migration from Slack to a new Matrix homeserver and allows reusing the imported user accounts. We have some research groups using Slack that would like to import their Slack workspace history to our Matrix homeserver. This PR allows to import a (free) Slack export to a fresh and empty Synapse instance that is federated with our main homeserver. We do not need a full "migration": we kick all imported users and invite the existing Matrix users from our homeserver.

Two groups already imported the history and switched to Matrix, one more is in queue and probably more to come.

This is a really, really exciting development, and the context for which Sven is expecting to use it is BIG, I can't wait til we can share more about it!

mautrix-twitter

Tulir said:

The Twitter DM bridge I announced last week mostly works now. It does text and reaction bridging in both directions, twitter->matrix media bridging, end-to-bridge encryption and backfilling. I also added support for it in mautrix-manager for web-based login.

If it mostly works, I guess it's time to mostly start trying it out!

Dept of Clients πŸ“±

Fluffychat

sorunome offered:

Fluffy 0.16.0 has been released! It is already available on F-Droid, Google Play and IOS Testflight will follow. You can also try it out in the webbrowser. Visit https://fluffychat.im

Features

  • Implement web notifications

  • Implement a connection status header

Changes

  • Switch out database engine for faster performance

  • Greatly improve startup time

  • Added languages: Galician, Croatian, Japanese, Russian, Ukrainian - Thanks a lot to all the weblate users!

  • Only show the microg toast once, if you have play services disabled

  • Homeserver URL input now strips trailing whitespace and slash - Thanks @Katerina

  • Also use prev_content to determine profile of a user: This allows the username and avatar of people who left a group to still be displayed

Fixes:

  • Fix not being able to initiate key verification properly

  • Fix message sending being weird on slow networks

  • Fix a few HTML rendering bugs

  • Various other fixes

  • Fix the 12h clock showing 00:15am, instead of 12:15am - Thanks @not_chicken

  • Fix an issue with replies and invalid HTML

  • Fix messages getting lost when retrieving chat history

  • Fix a bug where an incorrect string encoding from the server is assumed

  • Fix a bug where people couldn't log in if they had email notifications enabled

Mirage

miruka reported:

0.6.0 is out today:

Added

  • Room member profiles:

    • Can be accessed by clicking on a user in the room's right pane, or focusing the filter field and navigating with up/down/enter/escape

    • Includes large avatar, display name, user ID, presence info,

      power level control and E2E sessions list

  • E2E Verification:

    • Sessions for room members can now be (manually) verified from their profile

    • Sessions for different accounts within the same client will automatically

      verify each others based on session keys

    • Verifying a session will automatically verify it for all connected accounts,

      as long as the session keys are identical

  • Presence:

    • Added presence (online, unavailable, invisible, offline) and status message control to the accounts context menu in the room list

    • Added togglePresence{Unavailable,Invisible,Offline} keybinds bound by

      default to Ctrl+Alt+{A/U,I,O}

    • Added openPresenceMenu keybind to open the current account's context

      menu, Alt+P by default

    • The room member list is now sorted by power level, then presence, then name

    • The room member list will display presence orbs and last seen time for

      members if the server supports it. Last seen times for offline members are also automatically retrieved as needed.

    • Set logged in accounts offline when closing Mirage

    • Linux/X11 specific: Add auto-away feature configurable by the

      beUnavailableAfterSecondsIdle setting (default 600 for 10mn), can be disabled by setting it to -1.

  • Session sign out: you can now sign out your other sessions from the

    account settings. This currently only supports password authentication.

  • Pasting images via Ctrl+V or composer context menu, shows a preview of

    the image before uploading

  • Added basic keyboard navigation for account settings session list

  • Add a verified devices indicator to encrypted room headers

  • Add experimental support for rendering of inline images and custom emotes in

    messages

  • Add kineticScrollingMaxSpeed and kineticScrollingDeceleration settings

  • When highlighting accounts, rooms or members in lists

    (focus filter field and use up/down), the highlighted item's context menu can now be accessed with the keyboard Menu key

  • Support for Menu key when keyboard-navigating messages in the timeline

  • Add context menus to text field and areas

  • Add a button to quickly expand the room pane when collapsed and focus

    the filter field

  • Clicking on the current tab button for the room pane now fully hides it,

    this can also be toggled with the new toggleHideRoomPane keybind (default Ctrl+Alt+R)

Changed

  • When panes are smaller than their default width due to user resizing or

    window size constraints, focusing certain elements will auto-expand them until the focus is lost: filter fields, member profile and room settings

  • Reduced the default kinetic scrolling speed, which was hardcoded to an

    aggressive 4000 before. This can be restored with the kineticScrollingMaxSpeed setting.

  • Improve key verification popup texts and make the session details copiable

  • Power levels/room permission change events will now show a line of text or

    table containing the details of what exactly changed

  • Messages containing tables will no longer be width-limited

  • Using the sendFileFromPathInClipboard keybind (default Alt+Shift+S)

    now shows a preview of the file if it's an image and asks for confirmation

  • Image messages now show spinners when loading the thumbnail

  • Clicking on a GIF message will now open it externally like other images

    instead of pausing it. A dedicated play/pause button is now displayed in the corner.

And more: full changelog

Riot-iOS, no sorry, Element-iOS

Manu offered:

Last week, we forgot to announce that we renamed the Riot-iOS app to Element but this is not a surprise anymore. The app version is now 1.0.0. This change came with a lot of UI tweaks like new icons and new colors.

Since then, we made bug fixes we will release soon. This week, we also worked on PIN protection for the app, including support of Touch ID and Face ID.

Element Android

Francis offered:

This week has been focused on bug fixing, there will be a release next week. We also started to work on PIN protection for the app.

quotient

kitsune offered:

Quotient 0.6 released! The release notes are somewhat long, since it's been 4 months since the last stable version update, and more than a year since the 0.5 release - thanking two heroes of this release, and giving a nod to the original Quaternion author who happened to contribute using an inverted time machine. Read here: https://github.com/quotient-im/libQuotient/releases/tag/0.6.0

Big news - first release in a year. Do take a look at the full notes, they're very readable. In particular, they call out contributions from Black Hat, Alexey Andreyev and the original project creator, Felix Rohrbach aka @fxrh.

Dept of SDKs and Frameworks 🧰

ruma

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

jplatte said:

We weren't in TWIM for a while, but that doesn't mean no progress! Apart from Devin's constant work on ruma-events, there was also some movement in ruma-identifiers and ruma-client-api, with the former seeing a new release, 0.17.0.

We're planning to do more releases next week, and hopefully a 0.1 release of the ruma crate that provides a simpler entry point by re-exporting everything else, in one of the weeks after that. Stay tuned!

Dept of Ops πŸ› 

matrix-docker-ansible-deploy

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi said:

matrix-docker-ansible-deploy just gained support for its first bot - matrix-reminder-bot.

See our Setting up matrix-reminder-bot documentation to get started.

Then, another report:

matrix-docker-ansible-deploy has gained support for synapse-admin.

See our Setting up Synapse Admin documentation to get started.

Dept of Bots πŸ€–

matrix-wug

dandellion said:

It's been a while since it's come up, but matrix-wug is a bot capable of transcribing characters writable via normal keyboards to scripts from other languages/notation systems

It supports: x-sampa, z-sampa, proto-indo-european, Inuktitut, and IΓ±upiatun.

Now it also supports hiragana! The romanization system is something like Wāpuro rōmaji, but sticks to Nihon-shiki when in doubt.

I want this transcription to be forgiving, so if you find it not transcribing something in a way you'd expect it to, feel free to DM me!

To use it use the hi key with a delimiter like /x/ or []

hi/konnichiwa maatorikusu!/

vΓΈgg is also behind pantalaimon now so it now works in encrypted rooms πŸ˜€

If you need to convert swiggles into a different type of swiggle, dandellion has you covered here.

Jitsi Token Service

kinta reported:

https://gitlab.com/communia/matrix-jitsi-token-service

This bot is not about adapting the current jitsi integration provided by Element IM as widget.

If you have a configured jitsi server to be authenticated with jwt tokens as in https://github.com/jitsi/lib-jitsi-meet/blob/master/doc/tokens.md , this bot will assist you when claiming for a jwt instead of going to jwt.io each time. You'll need to create a user and invite you in each room where you want to create a jitsi room url respecting the matrix power levels in the room.

When the service is running invite the bot and send message !jitsi-jwt

If you are a moderator in the room (alias you have enough power levels to redact messages), then you will get a direct message from bot with the url to jitsi room with the jwt get parameter.

cody 0.7.0

carl told us:

New cody release. @cody:bordum.dk is a chat bot that works as a REPL for your matrix rooms. This week I worked on metrics. I host a Grafana instance with anonymous access allowed, so codys charts are now embedded the #cody:bordum.dk chat room.

Announcing The Very First Release of sMythbot

ssorbom told us:

I am proud to announce the very first release of sMythbot, the Matrix chatbot designed to control your Myth Tv DVR remotely. This release should be considered a tech preview. I welcome feedback and Bug reports. More information on installation and setup is available in the project wiki.

It currently supports the following commands

  • !smythbot help: Display this message

  • !smythbot set mythbackend address: Sets the Myth Tv backend address to use for this room.

  • !smythbot set mythbacked port: Sets the Myth Tv backend port to use for this room.

  • !smythbot view mythbackend address: Allows you to view the Myth Tv backend address set for this room

  • !smythbot view mythbackend port: Allows you to view the Myth Tv backend port set for this room

  • !smythbot view mythbackend info: Allows you to view various pieces of information for the Myth Tv backend connected to this room. It will not work if the address and port are not set.

  • !smythbot display upcoming recordings: Displays the upcoming recordings on your Myth Tv Backend.

  • !smythbot display recorded programs: Displays the recordings from the default recording group that are stored on your Myth Tv Backend.

You can find out more information at my Github Page

I love a project designed to scratch the author's own itch!

Dept of Interesting Projects πŸ›°οΈ

Noteworthy Elements for Mobile Web

balaa said:

The team behind Noteworthy (Matrix over WireGuard overlay networks) has started work on bringing better mobile web support for Element (riot-web).

Noteworthy Elements is a lightweight shim powered by the Ionic Framework thats goal is to bring first-class support for Element running on mobile devices.

What we have so far

  • Usable version of Element wrapped in native iOS / Android app

  • Ability to run multiple instances of Element in a native iOS / Android app (ie connect to multiple home servers simultaneously)

Our initial experimentation has been positive and our goal is to make Element the best mobile client for Matrix with minimal fuss. With minimal changes to Element's codebase our goal is to land (significantly better) support for mobile web in upstream Element. Join us over at #noteworthy:tincan.community to get involved!

2020-07-24-fH9Fl-ScreenShot2020-07-23at5.04.31PM.png

Trust-based Moderation Using Distributed Chat Systems for Transitive Trust Propagation

Florian said:

I just stumbled over the master thesis TrustNet: Trust-based Moderation Using Distributed Chat Systems for Transitive Trust Propagation. While it is mainly targeted at systems like Secure Scuttlebut and Cabal, I think it is also of interest for the people thinking about trust-based moderation for Matrix.

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1fairydust.space309
2bardiharborow.com545
3nicoll.xyz647.5
4grant.org667.5
5elsmussols.net694
6heitkoetter.net871.5
7ragon.xyz876
8acmelabs.space906.5
9opensuse.org915.5
10matrix.vgorcum.com1017

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

This Week in Matrix 2020-07-17

17.07.2020 00:00 β€” This Week in Matrix β€” Ben Parsons

Matrix Live πŸŽ™

Let's hear from Christian about working on his dream projects, his thoughts on bridging, hosting, and the importance and nature of chatbots and assistants.

Dept of Status of Matrix 🌑️

"Element"

This week was the announcement of the much-awaited rebrand: Riot is now Element. In fact, three brands are coalescing into one: Riot and New Vector will be referred to as Element, while the SaaS platform known as Modular.im is now Element Matrix Services.

Note that Matrix is not involved in this change. Matrix is still Matrix, don't worry about that!

CommCon Virtual 2020, with added Matrix

CommCon is an event dedicated to Real-Time Communications. In 2020 they made the difficult decision to go online-only, but had a fear of missing out on the "hallway-track" that is so important to industry events. Their solution was to include a live chat to run alongside their streamed talks!

Matrix was a natural choice for the crowd, but they wanted a way to encourage viewers to join the correct room from their own Matrix clients. To do this, they used the GSOC project from arnav-t - an HTML Embedded client - to present a live scrolling-view of the chat. You can see the result at https://2020.commcon.xyz/live/.

Dept of Spec πŸ“œ

uhoreg said:

anoa is away this week, so no pretty graph, I'm afraid. (this is intolerable - BP)

MSC status

Merged MSCs

  • No MSCs were merged this week.

MSCs in Final Comment Period:

  • No MSCs in Final Comment Period this week.

New MSCs:

Spec Core Team

This week, the Spec Core Team will be focusing on MSC2610 (Removing m.login.oauth2 from User-Interactive Authentication), MSC2611 (Removing m.login.token from User-Interactive Authentication), MSC2663 (Errors for dealing with non-existent push rules), MSC2674 (Event Relationships), and MSC2689 (Fix E2EE for guests).

Dept of GSoC πŸŽ“οΈ

Ruma/Matrix GSoC update

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

devinr528 reported:

The End is Nigh

This week in the ruma/matrix Google Summer of Code project, I worked on refactoring both ruma-api and ruma-events. After moving some of the larger chunks of the ruma_api_macro::api::Api::to_tokens method to helper functions, I spent time removing repetition from the Request/Response code generated by the ruma_api! macro. For ruma-events, the input parsing was changed to only allow valid names for the Any*Event enums. Altering the input parsing had the added benefit of replacing all of the string comparison and manipulation with strongly typed comparison and manipulation.

The final few issues to be resolved before the next crates.io release for ruma-events can happen are related to redacted events. Support for redacted events was added to the Any*Event enums, they now have redacted variants of each event kind. A few follow-up PR's have been merged to fully integrate redacted events into ruma-events, fixing specific event deserialization issues and splitting the UnsignedData struct into Unsigned and RedactedUnsigned.

HTML Embeddable Matrix

arnav-t reported:

This week for the GSoC project of HTML Embeddable Matrix Client:

  • Added a read receipts menu for messages.

  • Minor bug fixes

  • Automatic hyperlinking of URLs in pipeline

See above for details of this project being used in the wild!

mautrix-go

nikofil announced:

  • Added the capabilities to request room keys from other devices, as well as share room keys with other devices that requested them

  • Currently working on device verification using SAS, will then work on key export / import

neb

  • Updated to latest mautrix version with some minor fixes, most notably using m.encryption event parameters

Opsdroid(GSoC)

tyagdit said:

End to End Encryption with matrix

  • The new connector has been fully implemented!

  • Device verification has been put on hold for the time being

  • No extra configuration is required

  • Check out all the changes here

Matrix database

  • Work has begun on porting the matrix database module to matrix-nio and integrating it into opsdroid core

  • It uses matrix rooms as databases

  • Users can store key value pairs as state events in a room

  • Also works with encrypted rooms to allow for encrypted storage

matrix-ircd GSOC

karlik announced:

for matrix-ircd:

  • Merge in futures 0.3 support for matrix and http modules

  • Opened pull request for updating tests, irc module, and bridge module to futures 0.3 & remove several futures dependencies

  • updating several packages TLS packages to their more modern counterparts

Dept of Servers 🏒

matrix-media-repo

matrix-media-repo is a highly customizable multi-domain media repository for Matrix

TravisR said:

matrix-media-repo has just released v1.1.3 with several improvements and bug fixes - please give it a go, and get support in #media-repo:t2bot.io

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo told us:

I also investigated why so many sytests fail and created this issue: https://github.com/matrix-org/sytest/issues/913 Apparently Sytest did not expect a server to optimize their /sync responses as much as Conduit does :P.

Note: The official Conduit server is now reachable at "https://conduit.koesters.xyz". Thanks to everyone who supports me on Liberapay or Bitcoin!

Synapse

Neil said:

A big week for matrix.org performance.

Hot on the heels of shipping shardable federation readers last week, this week we shipped shardable federation senders. There is still plenty of work to do, but together these changes has made a massive difference to federation lag overall and hopefully those of you not on matrix.org are noticing the difference when you talk to matrix.org users.

This graph shows the impact to outbound federation lag.

2020-07-17-01oi5-Screenshot2020-07-17at14.39.02.png

Aside from that we also shipped shardable push and frontend proxy workers as both were starting to max out on CPU as well as a shardable client reader, allowing us to shard registration which was especially important this week :)

Finally we moved typing notifications from the master process and optimised incoming replication queuing to buy us a little more head room.

Next steps are to revisit where all the remaining cycles are going on the master process. To help us profile we are migrating to async/await semantics and Patrick produced this natty graph to track progress.

2020-07-17-Uqa2y-Screenshot2020-07-17at14.38.44.png

Outside of performance we shipped a bug fix to prevent large initial syncs taking out the synchrotrons. The admin api sprouted an end point to list room members (thanks awesome-michaeland the ability to reactivate previously deactivated users.

Coming up we’ll dust off the notifications project which has been put on hold while Riot transmogrified into Element and we’ll continue with chipping away at the master process.

Synapse Deployment πŸ“₯️

Kubernetes

Ananace said:

And another bump of the K8s-optimized Synapse images, this time to 1.17.0

Synapse adoption graphs

We only featured this a month ago, but I want to call attention again to this awesome Synapse version adoption tracking project from Chris . I wanted to know how quickly Synapse 1.17.0 would be the most deployed version (~36 hours), and the answer is right there!

Dept of Bridges πŸŒ‰

mautrix-twitter

Tulir announced:

I started working on a Twitter DM bridge that uses the internal API instead of the complicated official one. It should be easier to set up than bridges using the official API, since there won't be a need to get API keys and forward webhooks. The main risk is that Twitter doesn't like people using the internal API and starts blocking users like Facebook does, but hopefully that doesn't happen.

So far I've only made a simple library for the API, but the bridge should be usable by the next TWIM. The bridge and API library are at https://github.com/tulir/mautrix-twitter and there's a discussion room at #twitter:maunium.net.

Dept of Clients πŸ“±

Element Android 1.0.0

benoit announced:

Element Android 1.0.0 is out! Download or upgrade it from the PlayStore at https://play.google.com/store/apps/details?id=im.vector.app or download it from the F-Droid store at https://f-droid.org/packages/im.vector.app/

  • PlayStore users: Element should come as a regular upgrade of the application Riot-Android, and the upgrade will update your previous account and data, without the need to log in again.

  • F-Droid users: Element is a new app, you have to install it and log in again.

  • RiotX: RiotX will disappear from the PlayStore (it was only a beta application). We will provide a very last update to inform users to install Element Android

Please report any issues at https://github.com/vector-im/riotX-android (which will probably be renamed soon), because Element Android is actually RiotX code! Thanks for all the contributors of RiotX, we still have lots to do to make the app even better and full featured.

Radical

Davo announced:

After seeing all the Element room avatars, I thought to myself, "you know what, #radical-webext:matrix.org needs a new icon"… so I sat down and combined a couple of icons. The result:

Radical icon

Radical also quickly (as always) got updates when Element 1.7.0 and 1.7.1 were released. Huge thank you to stoic for making Radical in the first place.

Element F-Droid repos

krombel announced:

I updated my F-Droid repos which are containing the dev builds to reflect the latest naming change to Element.

The people who formerly used my repos for getting the dev repo of Riot-Android should update to the new repo. It wont be updated to get Element

As always you can grab the F-Droid or GPlay flavor and you can pick the repo which reflects your flavor on https://fdroid.krombel.de

Note: As "the new repo" is internally the same repo as the one of RiotX (and Riot.imX) the people who already used that don't need to update their packet source. They will get the updates to Element via the old repo urls.

Feel free to use it πŸ™‚

2020-07-17-izu7P-Screenshot_20200717-182645__01.jpg

Dept of SDKs and Frameworks 🧰

maubot

Tulir announced:

As promised last week, maubot now supports end-to-end encryption. It's a bit bare at the moment, so you need to manually insert the device ID into the database for it to start working, but other than that it works. Plugins don't need to be changed at all, the framework will handle everything.

While I was adding e2ee to maubot, I also improved mautrix-python's crypto stuff so that it's easier to use it directly as a client library with e2ee. Some day I might even add docs :D

matrix-appservice-cr

Chris Vincent reported:

Related to my previous update here, I've just published an initial version of a Matrix Client-Server client library for the Crystal programming language: https://github.com/cvincent/matrix-client-cr

Is this the first time we've seen tooling for Crystal?

Dept of Ops πŸ› 

matrix-docker-ansible-deploy

This Ansible playbook is meant to easily let you run your own Matrix homeserver.

Slavi announced:

matrix-docker-ansible-deploy is now ready to help you with the Riot -> Element transition.

We have a few ways to handle the migration depending on how much change and breakage you're willing to tolerate. See our Migrating to Element documentation page.

Incredibly thorough migration paths here!

Dept of Bots πŸ€–

Matrix-Alertmanager bot v0.3.0

jaywink offered:

Matrix-Alertmanager bot has a new release of v0.3.0. Highlights are compatibility with AWS Lambda, better error handling if message fails to send and the ability to do a @room mention on firing alerts. Find it here.

cody 0.6.0

carl reported:

REPL for your matrix rooms

This week, we got support for javascript (nodejs) with !js. The code

complexity was considerably reduced, which should make adding new languages much easier in the future. All Python sub-dependencies are now pinned, making

cody builds even more reproducible.

Chat with cody: @cody:bordum.dk

Read the source: https://gitlab.com/carlbordum/matrix-cody

Dept of Interesting Projects πŸ›°οΈ

Noteworthy (Matrix over Wireguard overlay networks)

balaa said:

Noteworthy team (patrick and myself) have open sourced both components (spoke & hub) of Noteworthy (Matrix over Wireguard overlay networks -- https://github.com/decentralabs/noteworthy) we are actively welcoming testers/contributors and working with a handful of projects on incorporating our deployment model over in #noteworthy:tincan.community -- it is also the fastest way (less than a minute) to deploy your own home server! Also, working on incorporating Noteworthy into the popular ansible deployment repo to enable users who don't have access to a publicly accessible to deploy their own home servers.

Dept of Ping πŸ“

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

RankHostnameMedian MS
1fairydust.space390
2exp.farm555.5
3matrix.vgorcum.com768
4tchncs.de810.5
5settgast.org1026
6moritzdietz.com1046
7ragon.xyz1288
8aragon.sh1383
9elcyb.org1755.5
10kapsi.fi2034

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!