GSOC Updates

With the first evaluation date coming up, let's take a look at what some of the GSOC students have been working on.

Zil0 on the matrix-python-sdk

I am working on implementing end-to-end encryption in the Python SDK. As of now, I have done a good part of the encryption and decryption work with Olm and Megolm, enough to allow communicating with Riot in an encrypted room. My next goals include device list tracking and key persistence, which are the main steps left before the implementation starts being usable in a real environment.

apwhitehat on Dendrite

My aim is to achieve feature completion on Dendrite. Have got lots of help from anoa, for which I'm very grateful.

Features worked on this month:

• we made a transactions cache/table for idempotency. So now dendrite will not reprocess events as far as client-server api is concerned.
• added request authentication function which recognises an application service user (partially landed.)
• username auto generation, which let's you register without username (will be used for guest access later.)
• We also added a tokens library based on macaroons, however we might not be using it anytime soon. Since it's been known to be very problematic (on synapse).
• Other small bug fixes not big enough to mention.

Clients

We are rich in client announcements this week!

palaver

Arne has announced v0.1.0 of Palaver:

More news from the JVM-world: I've just released the first Alpha version (v0.1.0) of Palaver, a Desktop client written in Kotlin: https://github.com/MrCustomizer/palaver/releases/tag/v0.1.0. As backend it uses the matrix-java-sdk, the GUI is written in TornadoFX, a Kotlin-Wrapper for JavaFX. The room for this client is #palaver:matrix.localtoast.de. The current features are listed in the changelog: https://github.com/MrCustomizer/palaver/blob/v0.1.0/CHANGELOG.md. And there is a little demo video on youtube, too: https://www.youtube.com/watch?v=AuGuxAPuRIM.

I enjoyed this video enough that I'm embedding it here:

kitsune: GTAD and libQMatrixClient

kitsune has announced GTAD v0.6 and libQMatrixClient v0.3. We've covered GTAD before, but I think it's worth including this note from the readme:

GTAD (Generate Things from an API Description) is a generator of code from a Swagger/OpenAPI specification. Initially made to generate marshalling/unmarshalling C++ code for Matrix CS API, it can be extended to support other API descriptions and other programming languages with static type checking.

kitsune regarding the GTAD 0.6 release:

A new version of API code generator, GTAD, 0.6, has been released today, adding support of variant types, proper dealing with definitions referencing other files ($ref) and more options to override schema names - even defined inline. Most importantly, GTAD has got extensive README.md that describes (most of) the things needed to start writing your own templates! As usual, kitsune will be happy to help those who would like to try it (both with C/C++ projects and other languages).

libQMatrixClient has been benefiting from GTAD over the last 4 months or so - but this is the first release where GTAD is stable and feature-complete enough to be shared with other projects.

#gtad:matrix.org is the new room for GTAD chat and support.

and libQMatrixClient 0.3:

A new version of libQMatrixClient, 0.3, comes out this weekend. This is the first libQMatrixClient release to include (almost) all CS API requests and supplementary (non-event) definitions, thanks to GTAD 0.6. Notably, it now includes jobs to register users, manage devices and keys on the server (no local key management yet - E2E work has just begun). This version also gains centralised request error handling so that clients could deal with problems in a unified way, and support "Consent not given" errors of GDPR fame, so that client authors could automatically open consent pages. You can also trigger logging out of all devices through libQMatrixClient thanks to a very recent addition to CS API spec from TravisR.

Fractal

Last week we linked to Julian's blog notes on the User Settings panel - this has now landed in Fractal master.

nheko

nheko v0.4.3 is now available, from the release notes:

• Overdue fixes for some regressions with regard to widget height introduced in the previous two releases
• The matrix id will be shown on hover on the display name.

riot-web

Riot-web: released an RC for 0.15.5. This RC includes some small bugfixes.

riot-android

We have been working on Riot stability: fixing crashes and adding more tools to control code quality The community on android adds a more readable display of keys and a floating actions menu is coming.

riot-ios

Working on perf. On stability too (like killing build warnings). Users can now re-request keys when they have UTCs.

Other projects

anoa on dendrite

We've got a shiny new application service component which runs as a separate process (if you're doing multiprocess dendrite) that handles all outbound communication to application services. Last week I got event sending working. This week I'm hooking up an internal API for other components (roomserver, c-s api) to talk to the app service component, as we occasionally need to ping AS's to ask about existence of rooms or users. Getting those two endpoints covered is the goal of this week.

ma1uta Full Spec Implementation in Java

ma1uta appeared with an extremely ambitious piece of work. His goal:

I want to create a full spec's implementation on java (all 5 specification). After that I want try to write a homeserver (2 variants: distributed on java+kafka and simple which can run on a light vps). But it's a long-term goal.

So far work has begun on an implementation of the Matrix API in Java:

https://gitlab.com/ma1uta/jeon - another java implementation of the matrix api (client-server, server-server, application server, push server, identity server) using jax-ws. It corresponds the specification on https://matrix.org/docs/spec. On the next step I want dive deep into synapse to parse it's api and fill the gaps of the spec with sending pull request. May be I will write something like a TCK (test compatibility kit).

And an SDK for which there is a sample bot for mastodon:

Also this project has a very early sdk (client, bot). I will fix it after finishing works with the spec.https://gitlab.com/ma1uta/mxtoot - matrix-mastodon bridge. With this bridge you can invite a bot and read your home mastodon's timeline, post messages, reply and boost messages.

There are rooms to follow progress on these projects:

• #mxtoot:matrix.org for Matrix-Mastodon bridge.
• #jeon:matrix.org for Matrix implementation on java.

Also find updates on Mastodon: [email protected]

The Continuing Adventures of Half-Shot and the Discord Bridge

Half-Shot has been at work on matrix-appservice-discord, and has made a v0.2.0 release! Looong feature list below is taken from https://github.com/Half-Shot/matrix-appservice-discord/releases/tag/v0.2.0:

• Support Node v10
• Option to disable @here and @everyone on the bridge - Permission based mentions coming soon.
• Sending large files over the Discord limit now sends them as URLs
• Added support for external and animated emoji - Thanks @Sorunome!
• Initial support for m.sticker
• Sender avatars are now thumbnailed - Thanks @turt2live
• Support for message edits. Thanks @anoadragon453
• Support for custom and animated discord emoji parsing M->D. Thanks @anoadragon453
• Throw discord.js logs into our logging to help debug connection issues
• Warn when turning on encryption that the bridge will no longer work.
• Clarify where to specify protocols in the registration config. Thanks @anoadragon453
• Add attachments into the event store and fix up some of our Inserts. Thanks @eeeeeta
• Fail if the event.content.url is not present.

+ lots of testing and fixes

t3chguy matrix-search including e2e

t3chguy, straight out of exams, went back to working on matrix-search, now with a working release, including e2e chats!

a local Full Text Search engine for Matrix, which will index the events locally and then provide a search api interface for matrix clients to query it matrix-search mimics the standard /search API

toml has already been looking at this and has created a docker container for it: https://github.com/matrix-org/matrix-search/tree/toml#to-run-in-a-docker-container

Ananace on Kubernetes

Switched the Kubernetes-oriented Synapse image over to running on the official one as a base, seems to still work quite well. Even if there were some issues at first due to Alpine and busybox.

I've been building slightly specialized Synapse docker images since 0.25.1 - and running them on my Kubernetes cluster at home. Moved them to being based on the official images now instead of building them on CentOS as I used to.

Synapse 0.31.0 and 0.31.1 released

Changes for Synapse 0.31.0

• Most notable change from v0.30.0 is to switch to python prometheus library to improve system stats reporting. WARNING this changes a number of prometheus metrics in a backwards-incompatible manner.
• There are still many changes being made relating to the Python 3 migration.

For Synapse 0.31.1

v0.31.1 fixes a security bug in the “get_missing_events“ federation API where event visibility rules were not applied correctly.

andrewsh has been packaging synapse for Debian.

Guides index

I'm going to start making a list of guides from around the web to be linked from https://matrix.org. If you have, or know of, useful guides relating to matrix, please let me know or share in #twim:matrix.org.

New rooms

• mentioned above, from ma1uta:
  • #mxtoot:matrix.org for Matrix-Mastodon bridge.
  • #jeon:matrix.org for Matrix implementation on java.
• #gtad:matrix.org is kitsune's room for gtad.
• #palaver:matrix.localtoast.de for Palaver discussion

other points of interest

• things are happening with doorman but I have not checked in for some time
• tulir mentioned that https://github.com/Dhole is working on a matrix Go client and an e2e lib
• Coffee migrated to Gitlab, including: https://gitlab.com/Matrixcoffee/matrix-client-core
• Naming things is hard. neilalexander's previously nameless client and Matrique both came under discussion for naming issues.
• benpa just discovered VS Code markdown preview and can't believe the time he wasted before

The end is nigh...

And here it is. As you may have noticed, I'm leaning much more toward quoting wherever possible, rather than trying to paraphrase. The aim is to keep the content authentic and community-driven, rather than a narrative from one fairly naive observer. The risk is it makes the post awkward to read as it switches voice too frequently. Come to #twim:matrix.org and let me know if you have opinions on this.

Check out Matrix Live: