This week I chatted with Jason Robinson about all things decentralisation, especially his projects socialhome, the-federation.info, and feneas.org. Jason has been interested in decentralisation for many years, and had a lot to say about how we can look forward to a more decentralised Internet.
libQMatrixClient 0.4.2 has been released, fixing a security issue (the library could be tricked into altering the local room state by fake state events – those without state_key). The master branch of the library is updated as well – it is strongly recommended to update to either 0.4.2 or master, depending on which branch you live on.
For those who want to help testing Quaternion or just can’t wait to the next release, we now have CI builds collected at bintray: https://bintray.com/qmatrixclient/ci/Quaternion. Linux and macOS are already there, and Windows binaries will also be available any day soon.
v1.10.0 fixes Matrix-to-Slack image upload, and no longer sends markdown-formatted @-mention links in the plaintext body of Matrix events; instead, it uses plaintext username the way text-only clients traditionally have it.
The version bump is also the project’s 100th commit, and I’ve updated the supported feature checklist in the README to paint a more comprehensive picture of what is and isn’t supported, and link out to the GitHub Issues for some of the unsupported features, in hopes of making life easier for users and encouraging contributions.
opsdroid‘s Matrix connector is now a core part of the library rather than an external addition. This should open up a lot of cool possibilities for doing fancy stuff with bots on matrix. Many thanks to Cadair for helping with this.
made some minor changes to continuum to make the GUI more intuitive. Now when there are no joined rooms, buttons for joining or creating rooms are shown instead of an empty list. And when there are issues with syncing or syncing takes longer than usual, a status bar with options is shown.
Still not quite sure if it’s a good idea, but at least the input area component is very nice. It should also make it easier to add more fancy stuff like html tables and selecting messages (for replying/redacting)
We are still working on the key backup screens. The last bit on the SDK side, the passphrase support, has been implemented this week.
Reskin is almost done. It just needs some small adjustments. We will release a beta soon so that we can gather users feedback.
This week, we have also fixed small but boring UX issues in Riot and we will continue to do so up to the end of the month and FOSDEM!
Reskin is almost done too and available on develop builds.
Benoit has started to implement the key backup passphrase management in the SDK. Valere is still improving push notifications at the code level but also at the display level.
We are working hard to polish the app for FOSDEM.
We now have a HipChat migration tool – https://www.modular.im/tools/hipchat-migration. This tool helps migrate a HipChat workspace to your very own Modular Hosted Homeserver. Migration is performed by uploading an exported copy of an existing HipChat workspace so that the tool can automatically re-create all of the users and rooms (including messages and attachments) on your new Matrix homeserver. Once the migration is complete all of the migrated users will be emailed with login instructions, so that they can seamlessly continue chatting where they left off in HipChat. If your organisation (or if you know of an organisation that) has not yet decided what to do when Stride & HipChat is discontinued next month, come and try Modular!
People have been generally happy about the flood of new issues following the completion of the internal audit last week, and a few folks have been opening PRs addressing some of the “good-first-issues” labeled ones, which has been amazing to see, including fixing room joins and a couple of default variables. Thanks a lot to Cnly and Behouba for these!
On my end, this week mainly consisted in reviewing most of these PRs (and merging them when that was possible), while anoa worked on making Dendrite’s CI more complete and reliable, especially by configuring it to run sytest against Dendrite which will allow us to better track its compliance to the Matrix specification.
Discord bridge v0.4.0 is out now, nothing new since rc1 because it was that good. There have been no changes since rc1 because either Sorunome did a stellar job of keeping things stable, or nobody spoke up out of fear :p. Thanks one and all for continuing to run the bridge.
Neo v4: Iris is coming up, focusing on design first. It will be based on React, and the matrix js sdk
I didn’t plan ahead enough with old Neo (v3), so it became a bit of a clobbered togeather mess, which I’m trying to prevent this time around https://git.lain.haus/f0x/iris
That’s all for now
Come chat in #twim:matrix.org to discuss what’s happening, and especially come and share what YOU’VE been working on!
Since joining the core team as Developer Advocate last year it’s been quite a ride. One of the best things about the job is getting the chance to talk to so many people about their projects and what they would like to see happen in the matrix ecosystem. With so much going on, I just want to say thanks to everyone who has been so welcoming to me and share some of my personal highlights, as I recall them, from 2018!
Fractal was featured in the very first TWIM, announcing v1.26. Since then, the team have hosted two IRL hackfest events (Strasbourg and Seville – where to next, Stockholm? Salisbury?), engaged two GSOC students and continued to push out releases. At this point, Fractal is a full-featured Matrix client for GNOME.
Matrique became Spectral, and is generally awesome. Apparently the name “Matrique” was chosen because it sounds French, but those who speak the language well revealed that this name was not ideal! The project was re-named “Spectral”, and is going strong. I really appreciate the multi-user facility! It’s a great looking client, and runs great on macOS too (protip: get more attention from /me by providing a macOS build…)
On which subject, Seaglass is a native macOS client. First announced in June, this client supports E2EE rooms (via matrix-ios-sdk), and is also available on homebrew.
Ubuntu Touch has the most Matrix clients per-user of any platform. UT epitomises the resilience and collaborative spirit of Open Source. It’s a true community maintenance effort, and is as friendly a community as you might meet. uMatriks came first, but it’s FluffyChat that prompted me to install it on my battered old OnePlus One. FluffyChat is now extremely full-featured, with E2EE support being actively discussed.
In the command line, gomuks appeared and quickly became a competent client, but in terms of sheer enthusiasm and momentum, I must give commendation to matrix-client.el, a newly revived mode for Emacs which turns your editor/OS into a great Matrix Client. I enjoyed using it enough that it began to change my mind about using emacs. Laptops have more than 8mb memory these days anyway.
A culture of bots
There is a tendency in the community to build a bot for everything and anything. This has reached the point where there are multiple flairs available depending on what bots you like to make (silly vs serious.)
TravisR was perhaps the first person I saw to get the obsession, creating
In June tulir started maubot, a plugin-based bot system built in Python, which now also has a management UI.
All bridges lead to Matrix
Or from Matrix, depending on which way you want to send the message.
Around May, I started to notice another obsession brewing in the community. Bridging is a core part of the Matrix mission, but it was around this time I started seeing it in the wild.
Summer 2018 Half-Shot began working in the Matrix core team, and was hugely productive in maintaining and developing the bridge infrastructure for matrix.org. IRC bridging is far more stable and reliable now than it was a year ago. And yet there are still more bridges – too many to list, so I’m picking the ones I’ve used and enjoyed.
SMSMatrix, a phone-hosted bridge is simple and works great for SMS bridging.
Libraries, SDKs, Frameworks
I enjoyed using matrix-js-bot-sdk for building elizabot (more coverage needed for that!), and the SDK recently received support for application services.
In April, kitsune announced v0.2 of libqmatrixclient describing it as “the first one more or less functional and stable” – confidence! This library now powers both Quaternion and Spectral. QMatrixClient has continued to get updates, plus features including lazy loading and VoIP signalling.
There are a few libs I want to pay more attention to this year, starting with tulir‘s maubot now that it has been rewritten in Python. I’m also excited to see jmsdk, part of ma1uta‘s broader ecosystem of Matrix tooling – a Java-based SDK.
Until around June, Ruma was receiving regular updates. There was a pause as the team waited for Rust async/await to land, and also to get some stability in the Matrix Spec. Still waiting on Rust, but now that the Matrix Spec is stabilising, Ruma is showing signs of life too. I have also been watching other homeserver projects begin to restart, which makes for a great start to 2019.
DSN Traveller by Florian
Matrix was featured as part of a Master’s thesis by Florian Jacob.
DSN Traveller tries to get a rough overview of how the Matrix network is structured today. It records how many rooms it finds, how many users and servers take part in those rooms, and how they relate to each other, meaning how many users a server has and of how many rooms it is part of.
Synapse dominates the homeserver space right now, so if you want to host your own homeserver today it’s the obvious choice. Too great a variety of installation guides was doing more harm than good, so Stefan took the initiative to create a definitive community-driven Synapse installation guide, including a room to discuss and improve the text. Find the guide linked from here, and chat about the guide in #synapseguide:matrix.org.
I want to use Matrix, and I want to host my own homeserver. As such, matrix-docker-ansible-deploy is a project I absolutely love. It uses Synapse docker images from the Matrix core team, and combines them with Ansible playbooks written and organised by Slavi. It lets you quickly deploy everything needed for a Synapse homeserver, and it’s simple enough that even I can use it.
Having a Matrix-native mode for shields.io (those counter/indicator images you often see at the top of repos) seems like something petty at first, but it’s actually a great indicator of the importance of Matrix from the outside. Plus, I love seeing the images at the top of different repos. Thanks Brendan for helping this along.
Two students worked on Matrix-related projects during GSOC 2018.
Thanks for a great 2018. There was so much to learn about, so much to write about, and so many great community members to meet and chat to! If I didn’t mention your project, I’m sorry to have been either forgetful or to not be able to include everything.
If you think I’ve missed something, or if there’s a project I should have included rather than another, or even if you just disagree with my choices, let’s discuss it in #twim:matrix.org. See you there, and let’s all parade ahead to a productive, open, interoperable 2019!
It’s been 2019 for several days now, plenty of time to get used to it! Let’s get started with the first TWIM of the year!
Matrix Live S3E09 from 35C3
Several Matrix-ers attended 35C3 in Leipzig last month, you can check out Matrix Live recorded from the conference below (also includes some screenshots and other clips of the event), and also watch a talk given by this author titled Matrix, the current status and year to date.
one of my dreams of working for a company that is a driver and leader in open source and open standards is coming true
Hey Half-Shot, what bridges have you worked on this week?
matrix-appservice-purple got soft launched on matrix.org and is happily bridging XMPP and matrix communities together. I am on full bug and feature fixing duty for it and the consensus from both sides is that it’s looking pretty awesome.
The matrix-appservice-purple bridge is coming along leaps and bounds, with formatting fixes, presence handling and speedier message delivey both ways. Also a shoutout to the XMPP community for guidling me through the XEP landscape and giving the bridge a thorough testing. :)
Photo sharing via the app share extension has been fixed this week.
At the time of writing, a new Riot iOS is baking with all bug fixes made the last month.
Riot 0.8.21 has been released on 01/02 on the PlayStore and on F-Droid.
This version contains:
A new notification troubleshoot screen with the possibility to run a diagnostic and to submit bug report. Feedbacks are already coming and we improve this screen incrementally to help users.
A new invitations counter on the group icon in the home screen
Other bug fixes
We are still working on push/notification reliability.
Riot Play Store resources have been translated into 8 languages so far: Basque, Bulgarian, Chinese (Traditional), French, German, Hungarian, Italian and Portuguese (Brazil).
matrix-client.el gained more room sorting options and a /priority command to set room priority. It also includes a workaround for a Google Chrome drag-and-drop bug on Linux, so now Chrome users can drag-and-drop URLs, files, and images directly into room buffers to upload them.
matrix-client.el gained a new notifications-buffer feature that shows notifications from multiple rooms in a single list, allowing you to easily monitor multiple rooms at once and jump to events in them.
e.g. I can see messages from #matrix and #twim in the same window, and reply to messages in both rooms from the same prompt
koma project: now continuum-desktop (client) and koma library
in the koma project, the desktop client now has continous integration and prebuilt packages for Mac and Linux; and you can click on image messages to zoom in. A simple weather bot is created reusing the same implementation of matrix client api. Send it the name of a city, and it will fetch the current weather using openweathermap
All versions of mxisd dropped support for Riot v0.17.8, introducing a bug affecting many of its features. Any new release integrating this PR will also be dropped of support. mxisd users are strongly encouraged to roll back to v0.17.7
This design concern is noted by the riot-web team and is under investigation.
Dendrite audit progress
Brendan on the progress of the Dendrite audit:
Dendrite’s audit is finally coming to an end! I’m happy to say I just finished the “data collection” phase, in which I looked at everything that needs to be either fixed or implemented in Dendrite. This represents 90% of the work and around 3 weeks of full-time work. All that’s left to do now is some triaging in the data (which is available here, by the way), into order to have a clear view on what’s left to do in the audit. Expect a lot of new issues and a shiny project board appearing on the Dendrite repository next week 🙂
Informo is a project intended to enable information sharing, especially for vulnerable activists. It is enabled by Matrix. vabd, the mystery individual behind the project announced:
Our specs bot, which shouts in Matrix rooms when the state of a proposal to a specs project changes, got an upgrade: it now handles concurrency better, and can now send multiple messages if multiple matching labels are added to the proposal in the same action (before, it just wouldn’t know what to do in such an event and would fail silently).
msc-chatbot now exists. It has commands that let you view the status of current MSCs, as well as a daily summary of MSCs to keep people up to date.
TravisR, as if he has time to be working on such things, has announced the revival of his Matrix-Minecraft bridge:
I’ve brought my Matrix<–>Minecraft bridge back to life in the form of a Bukkit plugin. It’s still in the very early stages of development and requires you to compile it yourself to get it, but it is a thing. Check it out on GitHub: https://github.com/turt2live/matrix-minecraft
It’ll be designed to work as a public hosted bridge, so someone could use t2bot.io to bridge their minecraft server (for example)
stickerpack dimension migration tool
Dandellion has created a tool for stickerpack creators:
Today is Jabber’s 20th anniversary! Jabber would later be standardized and renamed to XMPP.
On this subject, it’s always worth thinking about the importance of openness and interoperability in messaging. This recent article in Linux Journal is a reminder of the need to avoid proprietary vendor lock in, and mentions both XMPP and Matrix.
We’ll meet again…
Come chat in #twim:matrix.org with your Matrix news to be featured in this post. Next Friday there will be another weekly edition, but before then expect to see an edition to the effect “benpa’s best-of-the-community 2018”.
koma-desktop is updated to JavaFx 11 and installation is simplified. Dependencies, including native modules can be packaged into one single file, which only needs to be downloaded and run. Java Runtime 11 is the only runtime dependency. Now it’s just cross-compilation that needs to be set up before packaged releases can be provided for Mac, Windows, and Linux users.
I started work on a Prometheus Alertmanager bot for Matrix. The basic idea is that Alertmanager can send webhook alert events to the bot which will then send the formatted events to configured rooms based on the alert receiver. It works, but is still early work in progress. See code and info here: https://git.feneas.org/jaywink/matrix-alertmanager.
This week QMatrixClient project gained some improvements in macOS support (thanks to Matthew and Aaron Raimist!); but most of Kitsune‘s time (aside from dayjob) has been spent on attachments uploading in Quaternion. This required improvement in support of uploading in the library. These changes have now landed in the master branches.
The new command handling system in maubot is ready. The new system should be much nicer to use when developing plugins.
Previously maubot had a system that was designed after the improved bot support spec proposal, but it wasn’t very nice or pythonic. If/when the proposal or something similar goes through, I’ll probably add support for it in the new command handling system.
Next I’ll make some developer docs so that other people could actually make their own plugins.
https://github.com/apps/matrix-to is a Github App which makes use of their shiny and new Content Attachments API/Webhook. When a matrix.to or view.matrix.org URL is used this app is activated. It adds a little snippet with the Room Title and Topic (if the room is peekable from matrix.org). In future it’ll work for event permalinks, but currently there is no support for peeking context/event in Matrix API.
I’m at 35c3 with some known characters from the Matrix world (as well as 15,000 others.) If you’re here too, come visit us in our assembly, and also make sure to come to Dijkstra tomorrow to watch me present a look back at on the last year: https://fahrplan.events.ccc.de/congress/2018/Fahrplan/events/9400.html. We have recorded a message for Matrix Live from 35c3, but will post tomorrow with some more footage from the event.
Matrix Live: Half-Shot talks bridges, and working on libpurple bridging to Matrix
You may have seen that Half-Shot been working fearlessly and tirelessly on bridges for many, many months. In this episode of Matrix Live Half-Shot chats with Matthew about his work, progress and achievements, with a focus on recent matrix-appservice-purple and XMPP work. Audio is not amazing, but worth listening to get acquainted with recent work.
A couple of weeks ago, fr1kin PR’d a nice Matrix badge to the Shields project that tells you how many users are in a public room. There were a few issues with it so I PR’d some changes to make the badge more usable, and it’s now merged and live (as of yesterday evening), with examples available here! 🙂
For instance, here’s a badge for TWIM: https://img.shields.io/matrix/twim:matrix.org.svg 😉
matrix-js-bot-sdk has received early support for application services. Similar to the official matrix-js-sdk, the bot-sdk uses an Intent-based model for making bridges easier to write. Check out the simple example here for more information on how it works.
druig continues work on their JavaFX/kotlin client project:
The matrix client API implementation in koma is being extracted into a new repo, which is going to be a lightweight library for Kotlin.
Seaglass now available on homebrew
Aaron Raimist reports that Seaglass is available on homebrew for macOS:
Installing Seaglass is now easier than ever. If you already use Homebrew to manage other packages, you can now install Seaglass with a simple brew cask install seaglass.
the matrix-docker-ansible-deploy playbook has received some bugfixes and improvements lately. Most importantly, it’s now running the freshly-released Synapse 0.34.0 under Python 3, so memory usage should be much better.
#space:im.kabi.tk for anyone interested in space, rocket launches, satellites, etc.
Are you wondering what NASA’s new Mars Rover is doing? Maybe you live on the west coast of the United States, and you saw that meteor on Wednesday night that came within minutes of a scheduled rocket launch and just after three astronauts left the space station. If any of that sounds interesting to you, feel free to join the room.
A Matrix bot is being tested to send updates about upcoming rocket launches to the room.
We released 0.34.0! This release recommends Python 3 for production and brings with it huge performance improvements. If you’ve been putting upgrading off upgrading your Synapse, now is the time to do so. For more details here is a post that explains what you should expect and a recent Matrix Live interviews Amber (hawkowl) on the subject.
Aside from that we are working furiously towards federation R0 and have a bunch of MSCs to get us ever closer. You can track our progress here.
andrewsh notes that 0.34.0 is also available in the Debian repos:
Synapse 0.34.0~rc2 in Debian since Tuesday, 0.34.0 uploaded today; both use Python 3 only
Dendrite’s internal audit is progressing very well and is getting very close to its end.
What’s left to do for me is check the implementation status of a few Matrix features, and translate those into tagged GitHub issues so that everyone can have as clear of a view as possible on what’s left to work on.
I’m on holiday all of next week, but hopefully will have some good news about that the following week.
That really is it for now
Did you get to the end? What was your favourite section? Come tell us in #twim:matrix.org! Do you have your own update you’d like to add? Same place, come chat in #twim:matrix.org.
Next week is various things. It’s Christmas, which means there will be more hacking and coding happening than usual I expect! It’s also 35c3, which I will be attending, and might affect scheduling next week. Stay tuned in #twim:matrix.org for news, and come join us in #matrix-35c3:matrix.org if you’ll be there and want to meet up!