Matrix at FOSDEM 2019

04.02.2019 00:00 — In the News Matthew Hodgson

Hi all,

We just got back from braving the snow in Brussels at FOSDEM 2019 - Europe's biggest Open Source conference. I think it's fair to say we had an amazing time, with more people than ever before wanting to hang out and talk Matrix and discuss their favourite features (and bugs)!

The big news is that we released r0.1 of Matrix's Server-Server API late on Friday night - our first ever formal stable release of Matrix's Federation API, having addressed the core of the issues which have kept Federation in beta thus far. We'll go into more detail on this in a dedicated blog post, but this marks the first ever time that all of Matrix's APIs have had an official stable release.  All that remains before we declare Matrix out of beta is to release updates of the CS API (0.5) and possibly the IS API (0.2) and then we can formally declare the overall combination as Matrix 1.0 :D

We spoke about SS API r0.1 at length in our main stage FOSDEM talk on Saturday - as well as showing off the Riot Redesign, the E2E Encryption Endgame and giving an update on the French Government deployment of Matrix and the focus it's given us on finally shipping Matrix 1.0! For those who weren't there or missed the livestream, here's the talk!  Slides are available here.

Full house for @ara4n talking about @matrixdotorg and the French State @fosdem It was a packed presentation full of lots exciting progress demos. So sorry for practically yanking you offstage in the end! pic.twitter.com/idshDcSRhv

— Rob Pickering (@RobinJPickering) February 2, 2019

Then, on Sunday we had the opportunity to have a quick 20 minute talk in the Real Time Comms dev room, where we gave a tour of some of the work we've been doing recently to scale Matrix down to working on incredibly low bandwidth networks (100bps or less).  It's literally the opposite of the Matrix 1.0 / France talk in that it's a quick deep dive into a very specific problem area in Matrix - so, if you've been looking forward to Matrix finally having a better transport than HTTPS+JSON, here goes!  Slides are available here.

Full house for @matrixdotorg ? #FOSDEM #RTCsevroom pic.twitter.com/dDQnD3mzmc

— Saúl Ibarra Corretgé (@saghul) February 3, 2019

Huge thanks to everyone who came to the talks, and everyone who came to the stand or grabbed us for a chat! FOSDEM is an amazing way to be reminded in person that folks care about Matrix, and we've come away feeling more determined than ever to make Matrix as great as possible and provide a protocol+network which will replace the increasingly threatened proprietary communication silos. :)

Next up: Matrix 1.0...

This Week in Matrix 2019-02-01

01.02.2019 00:00 — This Week in Matrix Ben Parsons

MSC (spec) updates

r0, the first stable release of the Server-Server (Federation) Specification is extremely close! We of course will make a big splash and let you know when this comes!

Even apart from that, let's take a look at the MSCs (Matrix Spec Changes) that are currently in progress.

Approved MSCs

Final Comment Period

In Progress MSCs

Synapse

Neil and the team have been working frantically on getting a new Synapse release out this week:

This week has been all about gearing up for v0.99.0 and if you would like to help us test it, our latest release candidate lives here
We've taken the decision to bump up to v0.99.0 because it is very much a precursor to v1.0. When v1.0 lands it will contain a breaking change that means all homeservers will need a valid certificate for their server to server endpoint, self signing will no longer be possible.
v0.99.0 contains support to help you do this, but once it lands all admins will need to upgrade, failure to do so will mean losing the ability to federate with > v1.0 servers.
We'll have detailed docs ready to go alongside the full v0.99.0 release, and we plan to leave at least 1 month between v0.99.0 and v1.0, but for now please be aware that the change is coming.
Huge thanks to Rich, Erik, Hawkowl and Anoa for all their work in getting us to rc stage.

nheko_reborn

If you haven't already heard, then I envy that you get to learn about it now: there is a credible new project which forks nheko and seeks to maintain and continue the project.

Nheko-Reborn is a new project headed by red_sky, who told me:

I think I'm getting pretty close to having another release ready. Need to clean up some things here and there and get the CI packages uploading correctly
It'd be good to have more community engagement

Come chat about the project in #nheko-reborn:matrix.org.

Riot iOS

  • New app icons
  • Most of e2ee keys backup screens are done. We are only missing the sign out warning which invited the user to back up their keys before logging out
  • A Beta release is on its way to TestFlight

Riot Android

  • Finishing e2ee keys backup screen
  • Integrate new app icons, and splash
  • Release in Beta
  • Riotx: improvement in room list, read marker management

journal

lukebarnard has continued work on journal, a matrix-backed blog engine:

I've posted another blog on journal about the next steps for the project, including my thoughts on verifying blog content.

https://journal.lukebarnard.co.uk/journal/26-jan-2019

I'm planning to start work on the component that will scrape blog content from a matrix room. This will most likely involve a dedicated bot that syncs new journal blog events, verifies the blog signature and then writes the blog content to a file on the blog server.

I'll probably ramble about the details of that at some point.

matrix-docker-ansible-deploy

Slavi had two big updates for matrix-docker-ansible-deploy this week, improved security and IRC bridging:

matrix-docker-ansible-deploy now has much-improved security. All services now run in containers with a non-root user from the very beginning, without capabilities and with a read-only filesystem. To learn more, see the relevant changelog entry.
Thanks to a contribution by Devon Maloney (@Plailect), matrix-docker-ansible-deploy now has IRC bridging support via matrix-appservice-irc. To learn more, see the Setting up Appservice IRC documentation page.

neo

f0x again, now with an update on neo:

Not too much work on Neo this week, but there's some smooth animations for the roomlist now video, with music, which is surprisingly complicated in css
https://git.lain.haus/f0x/iris

libaqueous - Matrix SDK in Dart

Black Hat is working on a Matrix SDK in Dart:

libaqueous (the Matrix SDK in Dart) is progressing nicely. A reference implementation is also planned.
The repo is at https://gitlab.com/b0/libaqueous, and I also set up a room at #libaqueous:encom.eu.org

ma1uta's Matrix projects: Jeon, JMSDK, matrix-jabber-java-bridge

ma1uta has been working on his ecosystem of Matrix tooling:

Not much changes in jeon, jmsdk and matrix-jabber-java-bridge (mjjb).

  • Jeon: improved mxid with a new validation, support of the 11 java. And preparing the new release.
  • Jmsdk: fixed few bugs.

matrix-jabber-java-bridge

The bridge is being re-written. I excluded the jabber server (still need to setup SRV records) and implementing the xmpp s2s api in the bridge. Done:

  • rfc6120 in s2s part;
  • almost server dialback (XEP-0220);
Remaining:
  • MUC (XEP-0045);
  • stabilize.
New version will allow:
  • 1:1 conversations between matrix and jabber users;
  • group chats by double-puppet mapping rooms to conferences;
  • additionally xmpp users can join directly to rooms via double-puppet bots.
Also I started breaking the bridge to modules. And the next module will be ActivityPub S2S module which allows communicate matrix servers with other fedivers.

mxisd

Max reports that:

mxisd v1.3.0-alpha.3 is out! This one works further towards protecting your privacy and we strongly recommend it if you already are using an alpha release. We have written our stance on privacy with how it affects mxisd here.

We also consider this release to be as stable as v1.2.2. Feel free to upgrade following the Upgrade notes and benefits from all the v1.3.0 work so far, especially the massive improvements on resources usage.

Purism working on Fractal integration

update on @gnome @matrixdotorg client for #PureOS: "I am pleased to announce that over the next week I will be working to make Fractal's UI adaptive for the Librem 5's launch. This contract began last week, and I already have some results to show off." https://t.co/iX47u1Bdb1 pic.twitter.com/Hnr7ZVpYd1


— Purism (@Puri_sm) January 30, 2019

Definitely take a look at the linked blog post for more.

What else is there?

What there mainly is is FOSDEM. A lot of Matrix-folk are currently near me as I write this, sat in a hotel bar in sunny Brussels. Matrix live is not available today, but will incorporate some of the event, which kicks of tomorrow.

If you will be attending, come /join us at the Matrix table, where there will be stickers and t-shirts and merriment, and definitely come and attend the talks:

This Week in Matrix 2019-01-25

25.01.2019 00:00 — This Week in Matrix Ben Parsons

Matrix Live S03E12 - Modular.im

This week I chatted to Rick about the release of Modular, Hosted Homeservers and more. We're pleased to be able to announce the availability of a HipChat migration tool to get people into Matrix.

Latest MSC updates (from anoa's MSC bot)

Approved MSCs

Final Comment Period

In Progress MSCs

Synapse

  • Working furiously towards an r0 spec release. Event ids as hashes (MSC 1659) and S2S API certificates (MSC 1659 ) are very close now - see https://github.com/orgs/matrix-org/projects/8 to track our progress.
  • Since MSC1711 is a breaking change, we will initially ship our next release (v0.35.0) with ACME support to make it easy to provision and renew certificates. The give everyone a month to upgrade and install a cert before we ship Synapse v1.0 which will require that servers have certificates in order to federate. Don't worry, there will be plenty of details on the steps necessary for admins when v0.35.0 lands - watch this space.
  • Finally a raft of db performance improvements, room version upgrade bug fixes, as well taking a look at room directory and user directory efficiency.

matrix-puppet-slack

tom reports that:

matrix-puppet-slack version 1.10.1 has been released, thanks again to @twouters

  • The bridge could not create new Matrix rooms on versions of Synapse after a certain change, because it did not reserve the room alias prefix it used. That's been fixed, but anyone currently running matrix-puppet-slack will need to edit their slack_registration.yaml and restart Synapse. See the Release for instructions on doing so.
  • The bridge will no longer send "Edit: " events when Slack sends it a "message_changed" message, if the message text has not actually changed (Slack sends these events for URL previews, for example, but this just causes duplicate bridged messages). This fix has been a long time coming.

matrix-corporal

Slavi reports that:

matrix-corporal 1.3.0 was released.
It uses a new Matrix API for fetching account data (Synapse >0.34.1 is required), so it performs reconciliation quicker than before.

From the notes:

Reconciliation is now much faster, due to the way we retrieve account data from the Matrix server (no longer doing /sync).
From now on, the minimum requirement for running matrix-corporal is Synapse v0.34.1, as it's the first Synapse release which contains the new API we require (GET /user/{'{'}user_id{'}'}/account_data/{'{'}account_dataType{'}'}).

journal

lukebarnard disappeared from our screens some time ago, but he's back with this news on journal:

I have an update on journal (finally). I've pushed the redesign branch that I've been working on. It contains the web view component of the new architecture and can be used as a generic blog-hosting site (I'll be doing this personally). Feel free to check it out here: https://journal.lukebarnard.co.uk/journal/1-jan-2019

Riot-web

  • Redesign
  • Typing notifications don't make timeline jump anymore
  • Jump to bottom button is overlay now, so we can finally hide the room status bar again most of the time
  • Working on room sublist resizing
  • Work on authentication flow
  • Work on settings

Riot Android and iOS

  • Reskin of Riot is nearly finished. Last known issues have been fixed.
  • Keys backup screen development well underway! We're working to fit everything nicely on mobile platforms - there are some UX/UI specificities to consider.
  • Riotx (new version of riot for Android, built using the upcoming kotlin SDK): more and more event types supported in timeline.

Quaternion

kitsune reports that:

the macOS build for Quaternion 0.0.9.3 turned out to be not complete and fails if the user doesn't have Qt installed. Thanks to Aaron Raimist the build has been now includes a snapshot of Qt 5.11 - if you tried and failed to run the .dmg from https://github.com/QMatrixClient/Quaternion/releases, you can try to use it again.

Thanks also to Aaron for helping populate homebrew with Matrix goodness. Mac users may be interested to know that Seaglass and Spectral are both available in homebrew now.

Neo client

f0x reports that:

Neo is still in the GUI component design stage (the best stage to get involved with feedback!). I have implemented the jdenticon library for avatars, autoscrolling when there are new events, and I've added the Rust code of conduct. https://git.lain.haus/f0x/iris
General vision for this project is to first get as much gui done as possible, before diving in the backend. This will be split into a separate module, with the gui component being as protocol-agnostic as possible, to allow different backend modules for XMPP or IRC as well.

Fractal client

Backend refactoring by Julian Sparber and Alejandro Domínguez. They also got progressed with tag handling, spell checking and lazy loading.

matrix-bot-sdk

TravisR reports that:

the matrix-bot-sdk has received a bunch of updates currently residing on the develop branch. Changes include unit tests covering most of the library, appservice support improvements, handling of room version upgrades, and a bunch of bug fixes.

matrix-bot-sdk was recently updated to have support for Application Services, and is a lighter alternative to the matrix-js-sdk.

cl-matrix

Gnuxie reports that:

I've been working on cl-matrix and I think now it might be in a good condition to talk about it. cl-matrix is a WIP client library written in common lisp, most of the API endpoints have been covered using macros that allow you to copy straight from the spec, here is an example using the send event endpoint:

(define-matrix-endpoint room-send-event (:put)
  ("rooms" room-id "send" event-type txn-id))

this will expand into a function with the signature:

(PUT-ROOM-SEND-EVENT AUTHENTICATION ROOM-ID EVENT-TYPE TXN-ID CONTENT &KEY PARAMETERS)

it also has some basic events defined using deeds that can be issued using the sync endpoint.

Informo

The unknown individual from Informo, vabd told us:

Not much news this week in Informo land, though we have a few specs proposals that are still open for public review, including SCS #19 (rendered version here) which rewrites the specs website's introduction to make it more newcomer-friendly and feature a brief introduction on what Informo is about.
People who either never had a look at the project, or got fed up trying to because of the difficulty to easily understand what we're building, we'd love to read your opinion on this! ?

This is much, much appreciated. For those interested but confused, please take a look.

matrix-autoinvite

CromFr (Thibaut CHARLES) reports that:

matrix-autoinvite is a very basic service that synchronizes joined rooms between users from different servers, by inviting missing users to the room.
I'm using it to invite @CromFr:matrix.org to each Facebook Messenger rooms on my personal homeserver (that has very limited resources) hosting the matrix-puppet-facebook bridge. This way I can chat with people on facebook from a matrix.org account :)

Some relevant news from today: Zuckerberg Plans to Integrate WhatsApp, Instagram and Facebook Messenger

matrix-register-bot

krombel reports that:

A long time ago I started writing matrix-register-bot. Now as there is a new requestor of it I finally implemented the planned features and released 0.1-rc1
Feel free to test it and ask questions or provide feedback in #matrix-register-bot:msg-net.de

Other thoughts

Did you know about status.matrix.org? This handy site lets you know about the status of the matrix.org homeserver. That URL again: status.matrix.org.

linux.conf.au is happening right now in Christchurch, New Zealand (editor's note: is this right? .au or .nz?), and they've been partly using Matrix for their comms.

FOSDEM is in a week! Come chat in #matrix-fosdem:matrix.org. Be there and be square. I'll be there with Matrix swag so ping me if you'll be attending.

When will I see you… again?

I might adjust the schedule a little next week since it's FOSDEM (see above), but as always, stay tuned into #twim:matrix.org for all the biggest news!

This Week in Matrix 2019-01-18

18.01.2019 00:00 — This Week in Matrix Ben Parsons

Matrix Live S3E11

This week I chatted with Jason Robinson about all things decentralisation, especially his projects socialhomethe-federation.info, and feneas.org. Jason has been interested in decentralisation for many years, and had a lot to say about how we can look forward to a more decentralised Internet.

Latest MSC updates (from anoa's MSC bot)

Approved MSCs

[MSC 1229]: Mitigating abuse of the event depth parameter over federation [MSC 1497]: MSC1497: Advertising support of experimental features in the CS API [MSC 1501]: Room version upgrades [MSC 1354]: Widget API extension: Always-on-screen [MSC 1339]: Proposal to add a GET method to read account data

Final Comment Period

MSC 1794: Federation v2 Invite API

In Progress MSCs

[MSC 1796]: improved e2e notifications [MSC 1797]: Proposal for more granular profile error codes [MSC 1607]: Proposal for room alias grammar [MSC 1544]: Key verification using QR codes [MSC 1802]: Standardised federation response formats [MSC 1804]: Advertising capable room versions to clients [MSC 1538]: storing megolm keys serverside [MSC 1681]: cross-signing [MSC 1659]: Proposal: Change Event IDs to Hashes [MSC 1767]: Extensible event types & fallback in Matrix (v2) [MSC 1768]: Proposal to authenticate with public keys [MSC 1769]: Extensible profiles as rooms [MSC 1772]: Groups as rooms (v2) [MSC 1776]: Implementing peeking via /sync [MSC 1640]: MSC: Replace event IDs with hashes [MSC 1777]: peeking over federation

matrix-docker-ansible-deploy

Slavi reports that:

It's a New Year and there have been a lot of improvements to the matrix-docker-ansible-deploy playbook:

Lots of thanks go to Maximus, jcgruenhage and Cadair for helping out!

libQMatrixClient and Quaternion

kitsune reports that:

libQMatrixClient 0.4.2 has been released, fixing a security issue (the library could be tricked into altering the local room state by fake state events - those without state_key). The master branch of the library is updated as well - it is strongly recommended to update to either 0.4.2 or master, depending on which branch you live on.

For those who want to help testing Quaternion or just can't wait to the next release, we now have CI builds collected at bintray: https://bintray.com/qmatrixclient/ci/Quaternion. Linux and macOS are already there, and Windows binaries will also be available any day soon.

matrix-client.el

alphapapa reports that:

matrix-client.el can now send typing notifications.

matrix-puppet-slack

tom reports that:

matrix-puppet-slack v1.10.0 has been released, thanks to Cadair and (once again!) d3m3vilurr!

v1.10.0 fixes Matrix-to-Slack image upload, and no longer sends markdown-formatted @-mention links in the plaintext body of Matrix events; instead, it uses plaintext username the way text-only clients traditionally have it.

The version bump is also the project's 100th commit, and I've updated the supported feature checklist in the README to paint a more comprehensive picture of what is and isn't supported, and link out to the GitHub Issues for some of the unsupported features, in hopes of making life easier for users and encouraging contributions.

opsdroid

SolarDrew reports that:

opsdroid's Matrix connector is now a core part of the library rather than an external addition. This should open up a lot of cool possibilities for doing fancy stuff with bots on matrix. Many thanks to Cadair for helping with this.

mxisd

Maximus reports that:

mxisd has a new alpha release: v1.3.0-alpha.2 - Fixes a set of issues from alpha.1 and is now close to v1.2.2 stability. If you are already on alpha.1, update is highly recommended.

continuum (previously, "koma")

uforia reports that:

made some minor changes to continuum to make the GUI more intuitive. Now when there are no joined rooms, buttons for joining or creating rooms are shown instead of an empty list. And when there are issues with syncing or syncing takes longer than usual, a status bar with options is shown.

mauview

tulir reports that:

I've been working on a new Go TUI library for gomuks: https://github.com/tulir/mauview

Still not quite sure if it's a good idea, but at least the input area component is very nice. It should also make it easier to add more fancy stuff like html tables and selecting messages (for replying/redacting)

Riot iOS

We are still working on the key backup screens. The last bit on the SDK side, the passphrase support, has been implemented this week.

Reskin is almost done. It just needs some small adjustments. We will release a beta soon so that we can gather users feedback.

This week, we have also fixed small but boring UX issues in Riot and we will continue to do so up to the end of the month and FOSDEM!

Riot Android

Reskin is almost done too and available on develop builds.

Benoit has started to implement the key backup passphrase management in the SDK. Valere is still improving push notifications at the code level but also at the display level.

We are working hard to polish the app for FOSDEM.

Modular

Rick reports:

We now have a HipChat migration tool - https://www.modular.im/tools/hipchat-migration. This tool helps migrate a HipChat workspace to your very own Modular Hosted Homeserver. Migration is performed by uploading an exported copy of an existing HipChat workspace so that the tool can automatically re-create all of the users and rooms (including messages and attachments) on your new Matrix homeserver. Once the migration is complete all of the migrated users will be emailed with login instructions, so that they can seamlessly continue chatting where they left off in HipChat. If your organisation (or if you know of an organisation that) has not yet decided what to do when Stride & HipChat is discontinued next month, come and try Modular!

Synapse

The Synapse gang are working flat out towards r0 and a v1.0 release - you can track our progress here (https://github.com/orgs/matrix-org/projects/8). Aside from that if you haven't already done please please upgrade to v0.34.1.1 (https://matrix.org/blog/2019/01/15/further-details-on-critical-security-update-in-synapse-affecting-all-versions-prior-to-0-34-1-cve-2019-5885/)

Dendrite

Brendan reports:

People have been generally happy about the flood of new issues following the completion of the internal audit last week, and a few folks have been opening PRs addressing some of the “good-first-issues” labeled ones, which has been amazing to see, including fixing room joins and a couple of default variables. Thanks a lot to Cnly and Behouba for these!

On my end, this week mainly consisted in reviewing most of these PRs (and merging them when that was possible), while anoa worked on making Dendrite's CI more complete and reliable, especially by configuring it to run sytest against Dendrite which will allow us to better track its compliance to the Matrix specification.

If you're interested in contributing on Dendrite, there's still some “good first issues” open, and we'll be happy to chat and help in #dendrite-dev:matrix.org ?

Synapse in Debian

andrewsh reports that:

the most recent synapse, using Python 3, is available in both stretch-backports and Debian testing

Discord Bridge

Half-Shot reports that:

Discord bridge v0.4.0 is out now, nothing new since rc1 because it was that good. There have been no changes since rc1 because either Sorunome did a stellar job of keeping things stable, or nobody spoke up out of fear :p. Thanks one and all for continuing to run the bridge.

New Dart/Flutter Matrix SDK

Black Hat reports that:

I am prototyping a Matrix client SDK in Dart. It will target Flutter (Android, Fuchsia and iOS), web (AngularDart or other) and Dart VM.

f0x returns to Neo

f0x reports that:

Neo v4: Iris is coming up, focusing on design first. It will be based on React, and the matrix js sdk
I didn't plan ahead enough with old Neo (v3), so it became a bit of a clobbered togeather mess, which I'm trying to prevent this time around
https://git.lain.haus/f0x/iris

That's all for now

Come chat in #twim:matrix.org to discuss what's happening, and especially come and share what YOU'VE been working on!

Ben's favourite projects 2018

17.01.2019 00:00 — Thoughts Ben Parsons

Hi all, Ben here.

Since joining the core team as Developer Advocate last year it's been quite a ride. One of the best things about the job is getting the chance to talk to so many people about their projects and what they would like to see happen in the matrix ecosystem. With so much going on, I just want to say thanks to everyone who has been so welcoming to me and share some of my personal highlights, as I recall them, from 2018!

Clients

Fractal was featured in the very first TWIM, announcing v1.26. Since then, the team have hosted two IRL hackfest events (Strasbourg and Seville - where to next, Stockholm? Salisbury?), engaged two GSOC students and continued to push out releases. At this point, Fractal is a full-featured Matrix client for GNOME.

Matrique became Spectral, and is generally awesome. Apparently the name "Matrique" was chosen because it sounds French, but those who speak the language well revealed that this name was not ideal! The project was re-named "Spectral", and is going strong. I really appreciate the multi-user facility! It's a great looking client, and runs great on macOS too (protip: get more attention from /me by providing a macOS build…)

On which subject, Seaglass is a native macOS client. First announced in June, this client supports E2EE rooms (via matrix-ios-sdk), and is also available on homebrew.

Ubuntu Touch has the most Matrix clients per-user of any platform. UT epitomises the resilience and collaborative spirit of Open Source. It's a true community maintenance effort, and is as friendly a community as you might meet. uMatriks came first, but it's FluffyChat that prompted me to install it on my battered old OnePlus One. FluffyChat is now extremely full-featured, with E2EE support being actively discussed.

In the command line, gomuks appeared and quickly became a competent client, but in terms of sheer enthusiasm and momentum, I must give commendation to matrix-client.el, a newly revived mode for Emacs which turns your editor/OS into a great Matrix Client. I enjoyed using it enough that it began to change my mind about using emacs. Laptops have more than 8mb memory these days anyway.

A culture of bots

There is a tendency in the community to build a bot for everything and anything. This has reached the point where there are multiple flairs available depending on what bots you like to make (silly vs serious.)

TravisR was perhaps the first person I saw to get the obsession, creating

and more…

Cadair even made twimbot, designed to make it easier to consume and produce This Week in Matrix itself.

In June tulir started maubot, a plugin-based bot system built in Python, which now also has a management UI.

All bridges lead to Matrix

Or from Matrix, depending on which way you want to send the message.

Around May, I started to notice another obsession brewing in the community. Bridging is a core part of the Matrix mission, but it was around this time I started seeing it in the wild.

Summer 2018 Half-Shot began working in the Matrix core team, and was hugely productive in maintaining and developing the bridge infrastructure for matrix.org. IRC bridging is far more stable and reliable now than it was a year ago. And yet there are still more bridges - too many to list, so I'm picking the ones I've used and enjoyed.

Discord is bridged by matrix-appservice-discord, handled by Half-Shot, aided and abetted by anoa but with a new maintainer this year, Sorunome. This bridge is now feature-rich and sits at v0.3.1.

tulir's suite of bridges including mautrix-telegram and mautrix-whatsapp are extremely stable and useful - big thank you to TravisR for maintaining t2bot.io and hosting the Telegram bridge too.

SMSMatrix, a phone-hosted bridge is simple and works great for SMS bridging.

Libraries, SDKs, Frameworks

I enjoyed using matrix-bot-sdk for building elizabot (more coverage needed for that!), and the SDK recently received support for application services.

In April, kitsune announced v0.2 of libqmatrixclient describing it as “the first one more or less functional and stable" - confidence! This library now powers both Quaternion and Spectral. QMatrixClient has continued to get updates, plus features including lazy loading and VoIP signalling.

There are a few libs I want to pay more attention to this year, starting with tulir's maubot now that it has been rewritten in Python. I'm also excited to see jmsdk, part of ma1uta's broader ecosystem of Matrix tooling - a Java-based SDK.

Ruma Resurrection

Until around June, Ruma was receiving regular updates. There was a pause as the team waited for Rust async/await to land, and also to get some stability in the Matrix Spec. Still waiting on Rust, but now that the Matrix Spec is stabilising, Ruma is showing signs of life too. I have also been watching other homeserver projects begin to restart, which makes for a great start to 2019.

DSN Traveller by Florian

Matrix was featured as part of a Master's thesis by Florian Jacob.

DSN Traveller tries to get a rough overview of how the Matrix network is structured today. It records how many rooms it finds, how many users and servers take part in those rooms, and how they relate to each other, meaning how many users a server has and of how many rooms it is part of.

Florian's thesis was handed in last August. Source code is available.

All details at https://dsn-traveller.dsn.scc.kit.edu/, room at #dsn-traveller:dsn-traveller.dsn.scc.kit.edu.

Still more

Synapse dominates the homeserver space right now, so if you want to host your own homeserver today it's the obvious choice. Too great a variety of installation guides was doing more harm than good, so Stefan took the initiative to create a definitive community-driven Synapse installation guide, including a room to discuss and improve the text. Find the guide linked from here, and chat about the guide in #synapseguide:matrix.org.

I want to use Matrix, and I want to host my own homeserver. As such, matrix-docker-ansible-deploy is a project I absolutely love. It uses Synapse docker images from the Matrix core team, and combines them with Ansible playbooks written and organised by Slavi. It lets you quickly deploy everything needed for a Synapse homeserver, and it's simple enough that even I can use it.

Construct, a homeserver implementation in C++ began successfully federating with Matrix, work progressed from around April/May.

Having a Matrix-native mode for shields.io (those counter/indicator images you often see at the top of repos) seems like something petty at first, but it's actually a great indicator of the importance of Matrix from the outside. Plus, I love seeing the images at the top of different repos. Thanks Brendan for helping this along.

Two students worked on Matrix-related projects during GSOC 2018.

Something which came in super-helpful for me when testing homeserver installations was f0x's fed-tester. Source code available (obv.)

Thanks for all the projects

Thanks for a great 2018. There was so much to learn about, so much to write about, and so many great community members to meet and chat to! If I didn't mention your project, I'm sorry to have been either forgetful or to not be able to include everything.

If you think I've missed something, or if there's a project I should have included rather than another, or even if you just disagree with my choices, let's discuss it in #twim:matrix.org. See you there, and let's all parade ahead to a productive, open, interoperable 2019!

Further details on Critical Security Update in Synapse affecting all versions prior to 0.34.1 (CVE-2019-5885)

15.01.2019 00:00 — Security Neil Johnson

On Thursday Jan 10th we released a Critical Security Update (Synapse 0.34.0.1/0.34.1.1), which fixes a serious security bug in Synapse 0.34.0 and earlier. Many deployments have now upgraded to 0.34.0.1 or 0.34.1.1, and we now consider it appropriate to disclose more information about the issue, to provide context and encourage the remaining affected servers to upgrade as soon as possible.

In Synapse 0.11 (Nov 2015) we added a configuration parameter called “macaroon_secret_key” which relates to our use of macaroons in authentication. Macaroons are authentication tokens which must be signed by the server which generates them, to prevent them being forged by attackers. “macaroon_secret_key” defines the key which is used for this signature, and it must therefore be kept secret to preserve the security of the server.

If the option is not set, Synapse will attempt to derive a secret key from other secrets specified in the configuration file. However, in all versions of Synapse up to and including 0.34.0, this process was faulty and a predictable value was used instead.

So if, your homeserver.yaml does not contain a macaroon_secret_key, you need to upgrade to 0.34.1.1 or 0.34.0.1 or Debian 0.34.0-3~bpo9+2 immediately to prevent the risk of account hijacking.

The vulnerability affects any Synapse installation which does not have a macaroon_secret_key setting. For example, the Debian and Ubuntu packages from Matrix.org, Debian and Ubuntu include a configuration file without an explicit macaroon_secret_key and must upgrade. Anyone who hasn't updated their config since Nov 2015 or who grandfathered their config from the Debian/Ubuntu packages will likely also be affected.

We are not aware of this vulnerability being exploited in the wild, but if you are running an affected server it may still be wise to check your synapse's user_ips database table for any unexpected access to your server's accounts. You could also check your accounts' device lists (shown under Settings in Riot) for unexpected devices, although this is not as reliable as an attacker could cover their tracks to remove unexpected devices.

We'll publish a full post-mortem of the issue once we are confident that most affected servers have been upgraded.

We'd like to apologise for the inconvenience caused by this - especially to folks who upgraded since Friday who were in practice not affected. Due to the nature of the issue we wanted to minimise details about the issue until people had a chance to upgrade. We also did not follow a planned disclosure procedure because Synapse 0.34.1 already unintentionally disclosed the existence of the bug by fixing it (causing the logout bug for affected users which led us to pull the original Synapse 0.34.1 release).

On the plus side, we are approaching the end of beta for Synapse, and going forwards hope to see much better stability and security across the board.

Thanks again for your patience,

The Matrix.org Team

This Week in Matrix 2019-01-11

11.01.2019 00:00 — This Week in Matrix Neil Johnson

Welcome!

Do not panic, Benpa is away, I repeat, Benpa is away. Nonetheless TWIM lives on!

Spec

Lots of spec work this week, and a shout out to anoa for his magical mscbot that provides pokes, nudges and updates on all things spec. Here's what mscbot had to say about the past week.

Approved MSCs

[MSC 1497]: Advertising support of experimental features in the CS API [MSC 1339]: Proposal to add a GET method to read account data [MSC 1501]: Room version upgrades

Final Comment Period

MSC 1708: .well-known support for server name resolution MSC 1711: X.509 certificate verification for federation connections

New and In Progress MSCs

[MSC 1794]: Federation v2 Invite API [MSC 1796]: Improved e2e notifications [MSC 1797]: Proposal for more granular profile error codes [MSC 1640]: Replace event IDs with hashes [MSC 1776]: Implementing peeking via /sync [MSC 1777]: peeking over federation [MSC 1779]: Proposal for Open Governance for Matrix.org (v2)

(A few may be missing as we're still tweaking mscbot :)

Dendrite

Brendan had this to say:-

The Dendrite audit is over! A bunch of issues have been created on the Dendrite GitHub repository, as well as a project board in order to keep track of everything: https://github.com/matrix-org/dendrite/projects/2 There's a fair amount of issues that have been labeled as “good first issue”, so feel free to pick them up and open pull requests if you're looking into hacking on Dendrite! :)

And whilst we have your attention - here's Brendan & Matthew talking through the audit in this week's Matrix Live!

Synapse

Neil says:-

both contain critical security updates so please update asap for more details, we'll be able to share a bit more about the vuln once admins have had a chance to upgrade.

Meanwhile Hawkowl has been cranking out bug fixes and perf improvements and in particular taking a look at taming the user_ips table.

While Debian packager Andrewsh adds:-

latest synapse (0.34.1.1, Python 3) in Debian, fixing CVE-2019-5885; an update to a previous release fixing this CVE uploaded to stretch-backports, using Python 2. Dependencies for a Python 3 upload approved in stretch-backports, a Python 3 upload of 0.34.1.1 will be following later this week

Riot/iOS

Riot-iOS 0.7.11 has been released, with lots of bug fixes.

We have been working on e2e new screens (like key backup setup) and the re-skinning of the app.

Riot/Android

Working to improve notifications style.

Split screen mode will be supported on next release!

Continuous autofocus on the Camera has been enabled.

Also fighting bugs on registration.

Bridges

Halfshot has this to say:

Matrix-appservice-purple is being renamed to matrix-bifröst, on the basis that we now bridge to things and "burning rainbow bridge" seemed like a good description.

Other things that have happened: Performance improvements, as always. XMPP -> Matrix typing notifications XMPP -> Matrix avatars XMPP -> Matrix uploads * Matrix -> XMPP uploads (via oob)

and then follows up with this:-

As promised, we've got a discord bridge release out today. v0.4.0-rc1 has landed! See the change notes https://github.com/Half-Shot/matrix-appservice-discord/releases/tag/v0.4.0-rc1 . Thank you to Sorunome for doing a huge amount of work on this!
@swedneck reports that:

linuxgaming.life is now running matrix-appservice-discord v0.4.0-rc1.

Matrix.org Foundation

Matthew has a final draft of the Matrix.org Foundation governance document ready: https://github.com/matrix-org/matrix-doc/blob/matthew/msc1779/proposals/1779-open-governance.md. Comments on https://github.com/matrix-org/matrix-doc/pull/1779 would be much appreciated!  We expect to propose merging it next week, and then incorporating it into the final Articles of the foundation.

Riot Web

Loads and loads of work happening on https://riot.im/experimental which is now where all new development is happening as we race towards launching the new design.  Highlights include:

  • All new key verification is implemented! (in olm & matrix-js-sdk).  We're currently hooking up the UX.
  • Online key backup is pretty much finished.
  • Cross-signing is up next.
  • Redesign backlog is progressing (slightly stuck on making the RoomList resizing work nicely, but almost there)
  • Finalising the all new registration/login screens
  • ...and loads of other stuff too.

Meanwhile...

kitsune reports that:

Sending files landed in master branches of libQMatrixClient and Quaternion. Finally you can send your Quaternion screenshots (as any other images, jingles, cat videos etc.) to Matrix using Quaternion ;)

Also, libQMatrixClient is available as a Conan repository, for developers who'd like to use Conan to track dependencies.

progserega reports that:

Hello to all! I am write matrix bot for bridge messages between matrix and social network vk.com (russian analog of facebook). https://github.com/progserega/MatrixVkBot

alphapapa reports that:

matrix-client.el gained a room-list buffer, which can be sorted by unread status, name, number of members, etc, and has a right-click context menu like the room-list sidebar.

matrix-client.el gained right-click context menus in the room sidebar, allowing to set room priority, notifications, etc.

The matrix-client.el git repository has moved to: https://github.com/alphapapa/matrix-client.el

Stanislav N. aka pztrn reports that:

Hey guys, joined here to post another thing that works in Matrix https://gitlab.com/pztrn/check_mk_matrix_notifications it is a script that sends check_mk notifications to Matrix. Check_mk is a "plugin" for Nagios NMS.

Cadair reports that:

It's not my update but I saw this HomeAssistant addon for matrix (https://github.com/hassio-addons/addon-matrix) and wanted to make sure it got a shoutout on TWIM. [Seeing how nobody else has posted it in here, just on twitter etc.]

Morgan McMillian (thrrgilag) reports that:

I published v1.0.1 of the pnut-matrix bridge this week which brings public pnut.io chat rooms to the matrix network. Features include syncing of pnut.io names and avatars, matrix users ability to authorize their pnut.io accounts, and administrative controls for managing linked rooms. Project can be found at https://gitlab.dreamfall.space/thrrgilag/pnut-matrix and discussion is at #pnut-matrix:monkeystew.net

MMJD reports that:

ma1uta's MXToot deserves mention in the blog, and in https://matrix.org/docs/projects/try-matrix-now.html . People should not be wanting of Twitter over Decentralized-Federated F(L)OSS feeds in their Matrix room.

uforia reports that:

in the koma project, the desktop client continuum now does a full sync when the user account doesn't seem to have joined any chat rooms, this way, it can recover from some disk IO errors, or more commonly, unclean shutdowns. A ca-certificates issue with Java 11 on Debian stable was found while running a bot on a headless server, more details and the solution is in the README

vabd reports that:

Our first specs proposal of 2019 just landed in the form of SCS #16, which specifies the data/event structure for trust authorities. This is a big step as TAs play a key role in Informo's trust/reputation system!

In the meantime, we've also opened SCS #19, which proposes a rework of the specs' introduction with the idea to give newcomers a more accessible and immediate way to figure out what Informo is about, and give them some starting points so they can dive deeper into it if interested. It's a rather small one and we'd love people to give it a look so we can aim for the most newcomer-friendly version possible

We've also just opened SCS #21 which specifies a way for a source to change the Matrix user it uses to publish articles (e.g. if it was previously using a server managed by non trustworthy people). As with all of our proposals introducing changes in behaviour, it's open for people to share their comments on it for the next 7 days.

Maximus reports that:

The first alpha release for mxisd v1.3.0 has been released with already major performance improvements. Early testing and reporting about success/failure would be very much appreciated as v1.3.0 will break backward compatibility. We have been running it on our own servers for about a week now and feels really good and stable.

Friedger Müffke reports that:

I just launched OI Chat, a matrix service dedicated to Blockstack users (https://www.producthunt.com/posts/oi-chat).

It is a home server that does not rely on any passwords but on cryptography and user-owned storage.

OI Chat uses one-time logins to verify the ownership of a username that can only be created by the user if they control the blockstack account.

...and that's all this week, folks!  Your normal hand-crafted artisanal benpa confectionery will be back next week.

Critical Security Update: Synapse 0.34.0.1/Synapse 0.34.1.1

10.01.2019 00:00 — Security Neil Johnson

After releasing Synapse v0.34.1, we have become aware of a security vulnerability affecting all previous versions (CVE-2019-5885). v0.34.1 closed the vulnerability but, in some cases, caused users to be logged out of their clients, so we do not recommend v0.34.1 for production use.

Today we release two mitigating versions v0.34.0.1 and v0.34.1.1. Both versions close the vulnerability and will not cause users to be logged out. All installations should be upgraded to one or other immediately.

  • Admins who would otherwise upgrade to v0.34.1 (or those that have already done so) should upgrade to v0.34.1.1.
  • Admins on v0.34.0, who do not wish to bring in new non-security related behaviour, should upgrade to v0.34.0.1.

You can get the new updates for v0.34.0.1 and v0.34.1.1 here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. See also our Synapse installation guide page.

We will publish more details of the vulnerability once admins have had a chance to upgrade. To our knowledge the vulnerability has not been exploited in the wild.

Many thanks for your patience, we are moving ever closer to Synapse reaching v1.0, and fixes like this one edge us ever closer.

Thanks also to the package maintainers who have coordinated with us to ensure distro packages are available for a speedy upgrade!

This Week in Matrix 2019-01-04

04.01.2019 00:00 — This Week in Matrix Ben Parsons

Welcome to 2019

It's been 2019 for several days now, plenty of time to get used to it! Let's get started with the first TWIM of the year!

Matrix Live S3E09 from 35C3

Several Matrix-ers attended 35C3 in Leipzig last month, you can check out Matrix Live recorded from the conference below (also includes some screenshots and other clips of the event), and also watch a talk given by this author titled Matrix, the current status and year to date.

Welcoming Jason Robinson to New Vector

Decentralisation lover and Python fan Jason Robinson joins New Vector!

one of my dreams of working for a company that is a driver and leader in open source and open standards is coming true

matrix-appservice-purple

Hey Half-Shot, what bridges have you worked on this week?

matrix-appservice-purple got soft launched on matrix.org and is happily bridging XMPP and matrix communities together. I am on full bug and feature fixing duty for it and the consensus from both sides is that it's looking pretty awesome.
The matrix-appservice-purple bridge is coming along leaps and bounds, with formatting fixes, presence handling and speedier message delivery both ways. Also a shoutout to the XMPP community for guidling me through the XEP landscape and giving the bridge a thorough testing. :)

Riot Web Experimental

Riot Web Experimental was announced in a blog post last month, and is ready to get some more testing! Note that the name is still "experimental", but to get an image of where things are going, please give it a try!

http://riot.im/experimental

Riot iOS

Photo sharing via the app share extension has been fixed this week.
At the time of writing, a new Riot iOS is baking with all bug fixes made the last month.

Riot Android

Riot 0.8.21 has been released on 01/02 on the PlayStore and on F-Droid.

This version contains:

  • A new notification troubleshoot screen with the possibility to run a diagnostic and to submit bug report. Feedbacks are already coming and we improve this screen incrementally to help users.
  • A new invitations counter on the group icon in the home screen
  • Other bug fixes
We are still working on push/notification reliability. Riot Play Store resources have been translated into 8 languages so far: Basque, Bulgarian, Chinese (Traditional), French, German, Hungarian, Italian and Portuguese (Brazil).

matrix-client.el's many updates

alphapapa provided many updates for matrix-client.el this week, I recommend chatting in #matrix-client.el:matrix.org where the cultists Emacs users and client devs hang out.

matrix-client.el gained more room sorting options and a /priority command to set room priority. It also includes a workaround for a Google Chrome drag-and-drop bug on Linux, so now Chrome users can drag-and-drop URLs, files, and images directly into room buffers to upload them.

matrix-client.el gained a new notifications-buffer feature that shows notifications from multiple rooms in a single list, allowing you to easily monitor multiple rooms at once and jump to events in them.
e.g. I can see messages from #matrix and #twim in the same window, and reply to messages in both rooms from the same prompt

koma project: now continuum-desktop (client) and koma library

uforia from koma announced that the client formerly known as koma is now continuum-desktop:

in the koma project, the desktop client now has continuous integration and prebuilt packages for Mac and Linux; and you can click on image messages to zoom in. A simple weather bot is created reusing the same implementation of matrix client api. Send it the name of a city, and it will fetch the current weather using openweathermap

mxisd

Max, creator of mxisd, announced:

All versions of mxisd dropped support for Riot v0.17.8, introducing a bug affecting many of its features. Any new release integrating this PR will also be dropped of support. mxisd users are strongly encouraged to roll back to v0.17.7

This design concern is noted by the riot-web team and is under investigation.

Dendrite audit progress

Brendan on the progress of the Dendrite audit:

Dendrite's audit is finally coming to an end! I'm happy to say I just finished the “data collection” phase, in which I looked at everything that needs to be either fixed or implemented in Dendrite. This represents 90% of the work and around 3 weeks of full-time work. All that's left to do now is some triaging in the data (which is available here, by the way), into order to have a clear view on what's left to do in the audit. Expect a lot of new issues and a shiny project board appearing on the Dendrite repository next week ?

Informo news

Informo is a project intended to enable information sharing, especially for vulnerable activists. It is enabled by Matrix. vabd, the mystery individual behind the project announced:

Our specs bot, which shouts in Matrix rooms when the state of a proposal to a specs project changes, got an upgrade: it now handles concurrency better, and can now send multiple messages if multiple matching labels are added to the proposal in the same action (before, it just wouldn't know what to do in such an event and would fail silently).

msc-chatbot

The MSC process is the formal process by which changes are submitted to become part of the Matrix specification. anoa has been working on a bot to help with the process:

msc-chatbot now exists. It has commands that let you view the status of current MSCs, as well as a daily summary of MSCs to keep people up to date.

Matrix-Minecraft bridge

TravisR, as if he has time to be working on such things, has announced the revival of his Matrix-Minecraft bridge:

I've brought my Matrix<-->Minecraft bridge back to life in the form of a Bukkit plugin. It's still in the very early stages of development and requires you to compile it yourself to get it, but it is a thing. Check it out on GitHub: https://github.com/turt2live/matrix-minecraft
It'll be designed to work as a public hosted bridge, so someone could use t2bot.io to bridge their minecraft server (for example)

stickerpack dimension migration tool

Dandellion has created a tool for stickerpack creators:

I've finished https://dali99.github.io/stickerpack-dimension-migration-tool/ for a niceish way to make migrations files for custom stickerpacks in dimension

XMPP (as Jabber) turns 20

As noted in The XMPP Newsletter:

Today is Jabber's 20th anniversary! Jabber would later be standardized and renamed to XMPP.

On this subject, it's always worth thinking about the importance of openness and interoperability in messaging. This recent article in Linux Journal is a reminder of the need to avoid proprietary vendor lock in, and mentions both XMPP and Matrix.

We'll meet again…

Come chat in #twim:matrix.org with your Matrix news to be featured in this post. Next Friday there will be another weekly edition, but before then expect to see an edition to the effect "benpa's best-of-the-community 2018".

This Week in Matrix 2018-12-28

28.12.2018 00:00 — This Week in Matrix Ben Parsons

Welcome to the last TWIM of the year

It's the 28th of December as I write this, and I hope you had a good year. Long as it is, I recommend reading Matthew's write-up of the year, as it covers a lot of ground!

Many of the core team have been out of the office this week, but there are still plenty of updates to share from the Matrix ecosystem!

mxisd v1.2.2

Max released mxisd v1.2.2:

mxisd had a Holiday-special release: v1.2.2 before v1.3.0.

This release introduces two new big features:

  • Username login rewriting via 3PID to allow advanced flows, like bypassing the synapse restriction of having numerical usernames for non-guest users
  • Support for multiple Base DNs for LDAP backends
Work has started on v1.3.0 so this is definitely the last release before a non-backward compatible release.

Scylla

VaNilLa has started a new client, a web app build in Elm:

Hi all! I am working on a Matrix client in Elm, and I was recommended to share it here: Scylla

koma-desktop

druig:

koma-desktop is updated to JavaFx 11 and installation is simplified. Dependencies, including native modules can be packaged into one single file, which only needs to be downloaded and run. Java Runtime 11 is the only runtime dependency. Now it's just cross-compilation that needs to be set up before packaged releases can be provided for Mac, Windows, and Linux users.

Prometheus Alertmanager bot for Matrix

Jason Robinson:

I started work on a Prometheus Alertmanager bot for Matrix. The basic idea is that Alertmanager can send webhook alert events to the bot which will then send the formatted events to configured rooms based on the alert receiver. It works, but is still early work in progress. See code and info here: https://git.feneas.org/jaywink/matrix-alertmanager.

Also mirrored on GitHub: https://github.com/jaywink/matrix-alertmanager

Fractal 4.0

Alexandre Franke:

As expected the Fractal team released 4.0 and is already hard at work on the next micro version. We recommend getting it from Flathub like we usually do.

From the release notes:

New features:

  • Enhanced history view with adaptive layout, day divider
  • Reorganised headerbar, app menu merged with user menu
  • Larger display of emoji-only messages
  • Some performance improvements
  • Opening a room jumps to first unread message
Bugfixes:
  • More reliable notifications
  • Fixed display bug for avatars
Under the hood:
  • Large code refactor
  • Logging infrastructure
  • Continuous integration
  • More informative build output

maubot

tulir working on maubot:

The new command handling system in maubot is ready. The new system should be much nicer to use when developing plugins.

Previously maubot had a system that was designed after the improved bot support spec proposal, but it wasn't very nice or pythonic. If/when the proposal or something similar goes through, I'll probably add support for it in the new command handling system.

Next I'll make some developer docs so that other people could actually make their own plugins.

The code is at https://github.com/maubot/maubot and you can ask about maubot in #maubot:maunium.net.

matrix-client.el new features

alphapapa:

Emacs makes it so easy to integrate things. Now you can send org-mode syntax messages with the /org command in matrix-client.el.

Tab-completion of usernames and IDs was added to matrix-client.el.

matrix-to GitHub app

t3chguy has come out of hiding to announce:

https://github.com/apps/matrix-to is a Github App which makes use of their shiny and new Content Attachments API/Webhook. When a matrix.to or view.matrix.org URL is used this app is activated. It adds a little snippet with the Room Title and Topic (if the room is peekable from matrix.org). In future it'll work for event permalinks, but currently there is no support for peeking context/event in Matrix API.

Example can be seen at https://github.com/matrix-org/matrix.to/issues/52#issuecomment-449878490. Idea courtesy of TravisR.

See you next year

So there you have it.

I'm at 35c3 with some known characters from the Matrix world (as well as 15,000 others.) If you're here too, come visit us in our assembly, and also make sure to come to Dijkstra tomorrow to watch me present a look back at on the last year: https://fahrplan.events.ccc.de/congress/2018/Fahrplan/events/9400.html. We have recorded a message for Matrix Live from 35c3, but will post tomorrow with some more footage from the event.

Otherwise, see you next year Matrix fan!