This Week in Matrix 2020-08-14

🔗Matrix Live 🎙

No new Matrix Live this week, we have Open Tech Will Save Us #5:

Also, I recommend Matrix Live from last week, with Timo and Matthew discussing Conduit/Rust/Homeserver dev. Conduit is absolutely motoring, as you will read below.

🔗Dept of Status of Matrix 🌡️

🔗Linköping University has now soft-launched Matrix

Alexander Olofsson said:

In completely unrelated news to my earlier K8s updates, Linköping University has now soft-launched Matrix for use both internally - and hopefully with other universities and institutions in the future.

Se the k8s updates from Ananace below. Is this the first .se university offering Matrix? Also, great hostname!

🔗Dept of Spec 📜

🔗SAS Emoji Translations

TravisR said:

We've finally opened up https://translate.riot.im/projects/matrix-doc/sas-emoji-v1 to translate the emoji used by clients to ensure that users are securely verifying each other. The idea is that clients should always have a consistent name for these emoji, even if developed by completely different teams. If your client supports encryption verification and has translations, please transition to using matrix-doc for your translations: https://github.com/matrix-org/matrix-doc/pull/2728 has some details, and #matrix-spec:matrix.org would love to help out.

🔗Spec

anoa said:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

🔗MSC Status

Merged MSCs:

• No MSCs were merged this week.

MSCs in Final Comment Period:

• No MSCs are in FCP.

New MSCs:

• MSC2730: Verifiable forwarded events
• MSC2723: Forwarded message metadata

🔗Spec Core Team

Some of the team was on holiday last week, hence the noticeable lack of progress 😇 In terms of Spec Core Team MSC focus for this week, we're keeping on with the same two as last week: MSC2674 (aggregations pt1: relationships) and MSC1544 (QR code verification).

🔗Dept of GSoC 🎓️

🔗Opsdroid(GSoC)

tyagdit announced:

🔗Matrix database

• The port to nio is complete and most of the functionality has been worked out

• Encryption support is being worked on

• The module now has a delete method

• Some testing is remaining

• Check out the progress here

🔗End to End Encryption with matrix

• Opsdroid will soon get a new release with the new connector

• For now it can be used through the master branch of opsdroid

🔗Dept of Servers 🏢

🔗Synapse

Neil reported:

We have a new RC out, edited highlights include

• lots of async/await changes, 27 separate PRs (thanks Patrick)

• improved logging performance, this is in part to reduce IO but also to reduce the total amount logged. We’d be really interested to get some feedback on if the changes help. You may also want to take a look at our new logging defaults to see if they would help reduce total disk space usage.

• Further improvements to the Admin API allowing admin to optionally purge rooms on deletion.

Expect 1.19.0 to land early next week.

Aside from that Erik has been continuing with sharding the event persister, this is a big job, so we’re still a little way off having something to ship, but I’ll keep you updated.

Big news this week, after spending a good chunk of time trying to rework notifications to make the UX more intuitive, we have decided that the best (and only) way forward is to completely rethink how the push rules work. Rich is currently redesigning the whole thing, and once done we’ll resume on the client side.

Having spent much of the past 3 months trying to improve the performance of large scale instances we will soon start looking at improving the out of the box experience for smaller installs and are in the early stages of thinking about what that might look like. I’ll update more over time, but the two obvious use cases to target are

• Federating instance, with 1-5 users, resource constrained.
• Federating instance with a few hundred users.

Our (relatively small) changes to logging in 1.19.0 are the first signs of this shift. We still need to ship event persistence sharding and also move more functionality from the main process (dependent on async/await), but once done it clears the way for new projects.

anoa brought it to our attention that this graph looks like a "ocean-beach waterline dropoff".

🔗Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo reported:

This week I worked a bit on federation! Last week we were able to query room directories over federation; This week I looked into what's needed to join one of those rooms:

• Resolve room aliases over federation

• Send make_join request when a user tries to join an unknown room

• Use make_join response to create send_join request (insert origin, origin_server_ts and add hashes and signatures)

After implementing all those things, I could actually see Conduit's join event on Synapse!

Of course a lot more work is needed to make this really work,

like creating the room on the Conduit side and sending and receiving federation transactions, but this is a start.

Other news:

• Docker support (thanks to @weasy, @valkum and @paul)

• Room upgrade support (almost done, thanks to @Faelar)

Thanks to everyone who supports me on Liberapay or Bitcoin!

🔗Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander told us:

This week we've been closing off some of the remaining tasks for end-to-end encryption and working on tasks from our beta milestone task list.

Changes this week include:

• New format for the Dendrite configuration file, which is correctly broken up into configuration per-component

• Support for server ACLs has been added

• Significant refactoring of the HTTP routing and API setup to improve flexibility and separation of internal and external APIs

• A number of improvements to device list handling

• Key updates are now sent when handling device display name changes

• Deleted devices are now propagated correctly over federation

Spec compliance is improving:

• Client-Server APIs: 54%, up from 51% last week

• Server-Server APIs: 70%, up from 54% last week

As always, feel free to join us in #dendrite:matrix.org for the latest!

I'm enjoying the close rivalry and progress happening in Dendrite and Conduit right now!

🔗Sytest

Neil Alexander reported:

Sytest has now been updated to allow skipping tests for endpoints that have been deprecated in the spec. This may be useful if you are writing a homeserver and have no plans to implement deprecated endpoints, e.g. /initialSync or /events.

Use the command line parameter --exclude-deprecated when starting Sytest to skip these tests!

🔗Synapse Deployment 📥️

🔗YunoHost

Pierre reported:

YunoHost is an operating system aiming for the simplest administration of a server, and therefore democratize self-hosting.

Synapse integration had been updated to 1.18.0

Element Web integration had been updated to 1.7.3

🔗Kubernetes

Ananace reported:

Just pushed some WIP (v0.1) charts for both Synapse and the matrix-media-repo project, written with Redis, workers/sharding, PVC constraints, and other large deployment thoughts in mind.

Note that these charts - the Synapse one in particular - basically require you to have a fully featured cluster, with a regex-capable ingress, automated cert manager, and RBAC. They do not require you to have a ReadWriteMany capable storage class though, even when using workers.

and then:

To expand on my earlier TWIM, the charts for Synapse, media-repo, and now element-web are all being used in production now so they are also respectively receiving their 1.0's

🔗Dept of Clients 📱

🔗Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) said:

So, we've been silent for a bit, but that doesn't mean we haven't been busy, so I have a little bit of a bigger announcement to make today:

Trilene implemented voice call support in Nheko! We just merged this and you can try it out now!

Some caveats apply at the moment, but will be resolved in the near future:

• Currently you will need to have a patched version of GStreamer. The current releases of GStreamer have a bug when your turnserver uses a password, that contains a ':'. Since this is the default for coturn, I'm guessing this affects everyone.

• You will need to build Nheko yourself. Currently we do not have GStreamer available in our CI pipeline, so we disabled support for voice calls in those builds (apart from the flatpak build, but that also needs some further validation and a new GStreamer release). We will release proper builds with VOIP support once GStreamer is actually released and available with the fixes we need.

• There are still some other issues, when relying on a turnserver to connect a call. We will fix them one at a time.

Big shoutout to trilene for implementing all of this! I can finally call people from Nheko, which is super awesome!

This was the first of the bigger features, that have been cooking for a while, in the near future we should also finish the new event store, which reduced memory usage and provides some kind of offline support, and Chethan will probably finish the first iteration of cross-signing soon too. After that we'll focus on the smaller features again. ;-)

BIG update! Looking forward to trying out voice calls. ☎️ Maybe we'll be calling Nico for Matrix Live soon..?

🔗Element-Android

benoit said:

We are still fixing issues following the Element release. We are also working to add Jitsi conference call support.

We have exported a first beta version of the SDK from Element Android. It is available here: https://github.com/matrix-org/matrix-android-sdk2 and can be integrated in any Android app as a regular gradle dependency. We will update the SDK after each release of Element Android.

🔗SchildiChat for Android

SpiritCroc said:

SchildiChat version 1.0.4-sc.12 should be up in the Play Store soon.

It includes following changes since the last TWIM:

• The single-list overview screen is now optional. If you prefer the tabbed layout for separating direct messages and rooms, as known from Element for Android, but still want SchildiChat's message bubbles, you can get that combination now too!

• Push notifications have been enabled for the Play Store variant

• Some theme fixes for old Android versions

🔗Hydrogen

Bruno told us:

Some regular readers might remember, in the past I've put a few entries here about a side-project called Brawl. Two weeks ago, we took Brawl and started building a new client on top of it code-named Hydrogen.

As the name implies, Hydrogen tries to be the lightest Element. It is written entirely in vanilla javascript (no React, no Webpack) for complete control, structured as an MVVM app, leveraging the raw performance of indexeddb. On huge accounts where Element-web uses up to 1GB of heap memory, Hydrogen sips away with a mere 15mb. It is very early days though, and Hydrogen has very little features: you can receive and send unencrypted messages and that's it.

Hydrogen also aims to support a wider range of browsers, from IE11 (particularly for orgs with locked-down machines) all the way to a fully featured PWA that works well on mobile. We should also be able to easily add progressive loading, which proved hard in the past.

Another goal is to make it easy to embed parts of the app (like the room view) into another app or website. It could also be really nice to have a place to play and experiment with new tech for element proper - much as Dendrite has influenced Synapse.

As mentioned, it's early days, and lots of things are missing or broken, so please don't file any issues yet. Feel free to play around with it at https://hydrogen.element.io/ though!

Find the repo here: https://github.com/vector-im/hydrogen-web/.

🔗Element-iOS

ismailgulek offered:

This week, we've completed some of the tasks from the sprint: https://github.com/vector-im/element-ios/milestone/38.

As a highlight, we've released a new TF with PushKit & CallKit back.

🔗Dept of SDKs and Frameworks 🧰

🔗Ruma

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

iinuwa announced:

Lots of exciting things happened in Ruma this week:

• We have officially implemented over half (18/31) of the Federation API endpoints!

Special thanks to @agraven and @q-b!

• We've added the only new endpoint from r0.6.1 of the C2S spec.

• We have begun work on adding support for borrowed types for requests. This should reduce the memory allocations required for clients to send requests to

a server.

• Exported some macros to make creating custom endpoints and events that

much easier.

🔗New Contributors

This week, we have gotten help from a few new contributors:

• @agraven

• @lieuwex

• @q-b

Thanks for your help!

If you're interested in contributing, join the party on GitHub or

#ruma:matrix.org!

Great to see new contributors here.

🔗Dept of Ops 🛠

🔗Support for Ansible collections from Famedly

JCG told us:

There's news from ansible land! Ansible recently introduced a new packaging format, the "collection", and we've now started moving our ansible content into collections. This means the matrix modules I twim'ed a few months back are now easier to install than ever before, and the synapse/element roles are included in there as well (thanks to madonius for the contributions there). The roles have been regularly updated for a while now, and the element role isn't even called riot anymore! You can get the collection on Ansible Galaxy, with the source being available over on GitLab. In case of any questions, feel free to hop over into #ansible:famedly.de

If Ansible were a film franchise "Ansible: The Collection" would be the long awaited boxset.

🔗Dept of Bots 🤖

🔗Cody

carl said:

cody - the REPL for your matrix room - received support for unit conversions this week. This is done by implementing GNU units as a language.

Chat with cody: @cody:bordum.dk

Join our cody room: #cody:bordum.dk Read the source: https://gitlab.com/carlbordum/matrix-cody/

Seems to work well!

🔗Opsdroid

Cadair reported:

opsdroid is getting close to a release with the matrix connector rewritten to use the matrix-nio library. If you are feeling brave and want to help us iron out any bugs before the release, please install the master branch and test it out for your matrix bots. Please report any issues on the GitHub issue tracker or come talk in #opsdroid-developers:matrix.org. Hopefully, a release with this and lots of other goodies will be coming in a couple of weeks.

🔗Dept of Interesting Projects 🛰️

🔗Privee

terry_hello said:

Hi All,

I’ve created a compact and portable home server based on Matrix protocol, it’s a piece of hardware which is easy to use and set up. I’m working on a new client APP similar to other typical messenger GUI, but this server we named it ‘Privee’ supports any messenger developed with Matrix protocol. I’ve launched a campaign on Kickstarter this week, please come and support us! Thank you! https://www.kickstarter.com/projects/privee/privee?ref=beibnw

Privee is an interesting project alright! The project creators are based in Hong Kong, and are working from a desire to have more control over their communications. The plan is to run homeservers on a small computer physically located with the user, similar to plans outlined by NovaChat.

This is an early-stage project (hence starting on KickStarter.) While exciting and ambitious, I encourage everyone to do their own research! Learn more and chat with terry_hello (who has done a good job handling questions already!) in #Privee:matrix.org.

🔗Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	helderferreira.io	470
2	maescool.be	486.5
3	matrix.vgorcum.com	570.5
4	mchus.pro	730
5	envs.net	770
6	maunium.net	983
7	matrix.org	1261
8	acmelabs.space	1282.5
9	chatserver.ca	1387
10	productionservers.net	1433.5

🔗That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!