This week I chatted with Jason Robinson about all things decentralisation, especially his projects socialhome, the-federation.info, and feneas.org. Jason has been interested in decentralisation for many years, and had a lot to say about how we can look forward to a more decentralised Internet.
completely redone way of generating mxisd configuration, letting people unleash the full power of mxisd without us having to explicitly add support for new features
mxisd 1.3.0-ready. We default to a stable mxisd release still (1.2.2), but people can try the much-improved 1.3.0 alpha releases even now (e.g. matrix_mxisd_docker_image: "kamax/mxisd:1.3.0-alpha.2").
libQMatrixClient 0.4.2 has been released, fixing a security issue (the library could be tricked into altering the local room state by fake state events – those without state_key). The master branch of the library is updated as well – it is strongly recommended to update to either 0.4.2 or master, depending on which branch you live on.
For those who want to help testing Quaternion or just can’t wait to the next release, we now have CI builds collected at bintray: https://bintray.com/qmatrixclient/ci/Quaternion. Linux and macOS are already there, and Windows binaries will also be available any day soon.
v1.10.0 fixes Matrix-to-Slack image upload, and no longer sends markdown-formatted @-mention links in the plaintext body of Matrix events; instead, it uses plaintext username the way text-only clients traditionally have it.
The version bump is also the project’s 100th commit, and I’ve updated the supported feature checklist in the README to paint a more comprehensive picture of what is and isn’t supported, and link out to the GitHub Issues for some of the unsupported features, in hopes of making life easier for users and encouraging contributions.
opsdroid‘s Matrix connector is now a core part of the library rather than an external addition. This should open up a lot of cool possibilities for doing fancy stuff with bots on matrix. Many thanks to Cadair for helping with this.
mxisd has a new alpha release: v1.3.0-alpha.2 – Fixes a set of issues from alpha.1 and is now close to v1.2.2 stability. If you are already on alpha.1, update is highly recommended.
made some minor changes to continuum to make the GUI more intuitive. Now when there are no joined rooms, buttons for joining or creating rooms are shown instead of an empty list. And when there are issues with syncing or syncing takes longer than usual, a status bar with options is shown.
Still not quite sure if it’s a good idea, but at least the input area component is very nice. It should also make it easier to add more fancy stuff like html tables and selecting messages (for replying/redacting)
Riot iOS
We are still working on the key backup screens. The last bit on the SDK side, the passphrase support, has been implemented this week.
Reskin is almost done. It just needs some small adjustments. We will release a beta soon so that we can gather users feedback.
This week, we have also fixed small but boring UX issues in Riot and we will continue to do so up to the end of the month and FOSDEM!
Riot Android
Reskin is almost done too and available on develop builds.
Benoit has started to implement the key backup passphrase management in the SDK. Valere is still improving push notifications at the code level but also at the display level.
We are working hard to polish the app for FOSDEM.
Modular
Rick reports:
We now have a HipChat migration tool – https://www.modular.im/tools/hipchat-migration. This tool helps migrate a HipChat workspace to your very own Modular Hosted Homeserver. Migration is performed by uploading an exported copy of an existing HipChat workspace so that the tool can automatically re-create all of the users and rooms (including messages and attachments) on your new Matrix homeserver. Once the migration is complete all of the migrated users will be emailed with login instructions, so that they can seamlessly continue chatting where they left off in HipChat. If your organisation (or if you know of an organisation that) has not yet decided what to do when Stride & HipChat is discontinued next month, come and try Modular!
People have been generally happy about the flood of new issues following the completion of the internal audit last week, and a few folks have been opening PRs addressing some of the “good-first-issues” labeled ones, which has been amazing to see, including fixing room joins and a couple of default variables. Thanks a lot to Cnly and Behouba for these!
On my end, this week mainly consisted in reviewing most of these PRs (and merging them when that was possible), while anoa worked on making Dendrite’s CI more complete and reliable, especially by configuring it to run sytest against Dendrite which will allow us to better track its compliance to the Matrix specification.
Discord bridge v0.4.0 is out now, nothing new since rc1 because it was that good. There have been no changes since rc1 because either Sorunome did a stellar job of keeping things stable, or nobody spoke up out of fear :p. Thanks one and all for continuing to run the bridge.
Neo v4: Iris is coming up, focusing on design first. It will be based on React, and the matrix js sdk
I didn’t plan ahead enough with old Neo (v3), so it became a bit of a clobbered togeather mess, which I’m trying to prevent this time around https://git.lain.haus/f0x/iris
That’s all for now
Come chat in #twim:matrix.org to discuss what’s happening, and especially come and share what YOU’VE been working on!
Do not panic, Benpa is away, I repeat, Benpa is away. Nonetheless TWIM lives on!
Spec
Lots of spec work this week, and a shout out to anoa for his magical mscbot that provides pokes, nudges and updates on all things spec. Here’s what mscbot had to say about the past week.
The Dendrite audit is over! A bunch of issues have been created on the Dendrite GitHub repository, as well as a project board in order to keep track of everything: https://github.com/matrix-org/dendrite/projects/2 There’s a fair amount of issues that have been labeled as “good first issue”, so feel free to pick them up and open pull requests if you’re looking into hacking on Dendrite! :)
And whilst we have your attention – here’s Brendan & Matthew talking through the audit in this week’s Matrix Live!
This week saw the release of v0.34.0.1 and v0.34.1.1 both contain critical security updates so please update asap if you have not already done so. See here for more details, we’ll be able to share a bit more about the vuln once admins have had a chance to upgrade.
Aside from that we’ve been making progress against getting s2s api to r0 and Synapse to v1.0. Rich has been grappling with federation certificates (MSCs 1708, 1711) and Erik is pushing on with event ids as hashes MSC 1640. Meanwhile Hawkowl has been cranking out bug fixes and perf improvements and in particular taking a look at taming the user_ips table.
While Debian packager Andrewsh adds:-
latest synapse (0.34.1.1, Python 3) in Debian, fixing CVE-2019-5885; an update to a previous release fixing this CVE uploaded to stretch-backports, using Python 2. Dependencies for a Python 3 upload approved in stretch-backports, a Python 3 upload of 0.34.1.1 will be following later this week
Riot/iOS
Riot-iOS 0.7.11 has been released, with lots of bug fixes.
We have been working on e2e new screens (like key backup setup) and the re-skinning of the app.
Riot/Android
Working to improve notifications style.
Split screen mode will be supported on next release!
Continuous autofocus on the Camera has been enabled.
Matrix-appservice-purple is being renamed to matrix-bifröst, on the basis that we now bridge to things and “burning rainbow bridge” seemed like a good description.
Other things that have happened: Performance improvements, as always. XMPP -> Matrix typing notifications XMPP -> Matrix avatars XMPP -> Matrix uploads * Matrix -> XMPP uploads (via oob)
Loads and loads of work happening on https://riot.im/experimental which is now where all new development is happening as we race towards launching the new design. Highlights include:
All new key verification is implemented! (in olm & matrix-js-sdk). We’re currently hooking up the UX.
Online key backup is pretty much finished.
Cross-signing is up next.
Redesign backlog is progressing (slightly stuck on making the RoomList resizing work nicely, but almost there)
Sending files landed in master branches of libQMatrixClient and Quaternion. Finally you can send your Quaternion screenshots (as any other images, jingles, cat videos etc.) to Matrix using Quaternion ;)
Also, libQMatrixClient is available as a Conan repository, for developers who’d like to use Conan to track dependencies.
matrix-client.el gained a room-list buffer, which can be sorted by unread status, name, number of members, etc, and has a right-click context menu like the room-list sidebar.
matrix-client.el gained right-click context menus in the room sidebar, allowing to set room priority, notifications, etc.
It’s not my update but I saw this HomeAssistant addon for matrix (https://github.com/hassio-addons/addon-matrix) and wanted to make sure it got a shoutout on TWIM. [Seeing how nobody else has posted it in here, just on twitter etc.]
I published v1.0.1 of the pnut-matrix bridge this week which brings public pnut.io chat rooms to the matrix network. Features include syncing of pnut.io names and avatars, matrix users ability to authorize their pnut.io accounts, and administrative controls for managing linked rooms. Project can be found at https://gitlab.dreamfall.space/thrrgilag/pnut-matrix and discussion is at #pnut-matrix:monkeystew.net
in the koma project, the desktop client continuum now does a full sync when the user account doesn’t seem to have joined any chat rooms, this way, it can recover from some disk IO errors, or more commonly, unclean shutdowns. A ca-certificates issue with Java 11 on Debian stable was found while running a bot on a headless server, more details and the solution is in the README
Our first specs proposal of 2019 just landed in the form of SCS #16, which specifies the data/event structure for trust authorities. This is a big step as TAs play a key role in Informo’s trust/reputation system!
In the meantime, we’ve also opened SCS #19, which proposes a rework of the specs’ introduction with the idea to give newcomers a more accessible and immediate way to figure out what Informo is about, and give them some starting points so they can dive deeper into it if interested. It’s a rather small one and we’d love people to give it a look so we can aim for the most newcomer-friendly version possible
We’ve also just opened SCS #21 which specifies a way for a source to change the Matrix user it uses to publish articles (e.g. if it was previously using a server managed by non trustworthy people). As with all of our proposals introducing changes in behaviour, it’s open for people to share their comments on it for the next 7 days.
The first alpha release for mxisd v1.3.0 has been released with already major performance improvements. Early testing and reporting about success/failure would be very much appreciated as v1.3.0 will break backward compatbility. We have been running it on our own servers for about a week now and feels really good and stable.
It’s been 2019 for several days now, plenty of time to get used to it! Let’s get started with the first TWIM of the year!
Matrix Live S3E09 from 35C3
Several Matrix-ers attended 35C3 in Leipzig last month, you can check out Matrix Live recorded from the conference below (also includes some screenshots and other clips of the event), and also watch a talk given by this author titled Matrix, the current status and year to date.
Welcoming Jason Robinson to New Vector
Decentralisation lover and Python fan Jason Robinson joins New Vector!
one of my dreams of working for a company that is a driver and leader in open source and open standards is coming true
matrix-appservice-purple
Hey Half-Shot, what bridges have you worked on this week?
matrix-appservice-purple got soft launched on matrix.org and is happily bridging XMPP and matrix communities together. I am on full bug and feature fixing duty for it and the consensus from both sides is that it’s looking pretty awesome.
The matrix-appservice-purple bridge is coming along leaps and bounds, with formatting fixes, presence handling and speedier message delivey both ways. Also a shoutout to the XMPP community for guidling me through the XEP landscape and giving the bridge a thorough testing. :)
Photo sharing via the app share extension has been fixed this week.
At the time of writing, a new Riot iOS is baking with all bug fixes made the last month.
Riot Android
Riot 0.8.21 has been released on 01/02 on the PlayStore and on F-Droid.
This version contains:
A new notification troubleshoot screen with the possibility to run a diagnostic and to submit bug report. Feedbacks are already coming and we improve this screen incrementally to help users.
A new invitations counter on the group icon in the home screen
Other bug fixes
We are still working on push/notification reliability.
Riot Play Store resources have been translated into 8 languages so far: Basque, Bulgarian, Chinese (Traditional), French, German, Hungarian, Italian and Portuguese (Brazil).
matrix-client.el gained more room sorting options and a /priority command to set room priority. It also includes a workaround for a Google Chrome drag-and-drop bug on Linux, so now Chrome users can drag-and-drop URLs, files, and images directly into room buffers to upload them.
matrix-client.el gained a new notifications-buffer feature that shows notifications from multiple rooms in a single list, allowing you to easily monitor multiple rooms at once and jump to events in them.
e.g. I can see messages from #matrix and #twim in the same window, and reply to messages in both rooms from the same prompt
koma project: now continuum-desktop (client) and koma library
in the koma project, the desktop client now has continous integration and prebuilt packages for Mac and Linux; and you can click on image messages to zoom in. A simple weather bot is created reusing the same implementation of matrix client api. Send it the name of a city, and it will fetch the current weather using openweathermap
the Fractal team focused on improving code quality (including moving to Rust 2018) with a dozen merge request currently being reviewed (a couple of which have already been approved).
Fractal made incredible strides in 2018, can’t wait to see how they progress in 2019.
All versions of mxisd dropped support for Riot v0.17.8, introducing a bug affecting many of its features. Any new release integrating this PR will also be dropped of support. mxisd users are strongly encouraged to roll back to v0.17.7
This design concern is noted by the riot-web team and is under investigation.
Dendrite audit progress
Brendan on the progress of the Dendrite audit:
Dendrite’s audit is finally coming to an end! I’m happy to say I just finished the “data collection” phase, in which I looked at everything that needs to be either fixed or implemented in Dendrite. This represents 90% of the work and around 3 weeks of full-time work. All that’s left to do now is some triaging in the data (which is available here, by the way), into order to have a clear view on what’s left to do in the audit. Expect a lot of new issues and a shiny project board appearing on the Dendrite repository next week 🙂
Informo news
Informo is a project intended to enable information sharing, especially for vulnerable activists. It is enabled by Matrix. vabd, the mystery individual behind the project announced:
Our specs bot, which shouts in Matrix rooms when the state of a proposal to a specs project changes, got an upgrade: it now handles concurrency better, and can now send multiple messages if multiple matching labels are added to the proposal in the same action (before, it just wouldn’t know what to do in such an event and would fail silently).
msc-chatbot now exists. It has commands that let you view the status of current MSCs, as well as a daily summary of MSCs to keep people up to date.
Matrix-Minecraft bridge
TravisR, as if he has time to be working on such things, has announced the revival of his Matrix-Minecraft bridge:
I’ve brought my Matrix<–>Minecraft bridge back to life in the form of a Bukkit plugin. It’s still in the very early stages of development and requires you to compile it yourself to get it, but it is a thing. Check it out on GitHub: https://github.com/turt2live/matrix-minecraft
It’ll be designed to work as a public hosted bridge, so someone could use t2bot.io to bridge their minecraft server (for example)
stickerpack dimension migration tool
Dandellion has created a tool for stickerpack creators:
Today is Jabber’s 20th anniversary! Jabber would later be standardized and renamed to XMPP.
On this subject, it’s always worth thinking about the importance of openness and interoperability in messaging. This recent article in Linux Journal is a reminder of the need to avoid proprietary vendor lock in, and mentions both XMPP and Matrix.
We’ll meet again…
Come chat in #twim:matrix.org with your Matrix news to be featured in this post. Next Friday there will be another weekly edition, but before then expect to see an edition to the effect “benpa’s best-of-the-community 2018”.
It’s the 28th of December as I write this, and I hope you had a good year. Long as it is, I recommend reading Matthew’s write-up of the year, as it covers a lot of ground!
Many of the core team have been out of the office this week, but there are still plenty of updates to share from the Matrix ecosystem!
koma-desktop is updated to JavaFx 11 and installation is simplified. Dependencies, including native modules can be packaged into one single file, which only needs to be downloaded and run. Java Runtime 11 is the only runtime dependency. Now it’s just cross-compilation that needs to be set up before packaged releases can be provided for Mac, Windows, and Linux users.
I started work on a Prometheus Alertmanager bot for Matrix. The basic idea is that Alertmanager can send webhook alert events to the bot which will then send the formatted events to configured rooms based on the alert receiver. It works, but is still early work in progress. See code and info here: https://git.feneas.org/jaywink/matrix-alertmanager.
This week QMatrixClient project gained some improvements in macOS support (thanks to Matthew and Aaron Raimist!); but most of Kitsune‘s time (aside from dayjob) has been spent on attachments uploading in Quaternion. This required improvement in support of uploading in the library. These changes have now landed in the master branches.
As expected the Fractal team released 4.0 and is already hard at work on the next micro version. We recommend getting it from Flathub like we usually do.
From the release notes:
New features:
Enhanced history view with adaptive layout, day divider
Reorganised headerbar, app menu merged with user menu
The new command handling system in maubot is ready. The new system should be much nicer to use when developing plugins.
Previously maubot had a system that was designed after the improved bot support spec proposal, but it wasn’t very nice or pythonic. If/when the proposal or something similar goes through, I’ll probably add support for it in the new command handling system.
Next I’ll make some developer docs so that other people could actually make their own plugins.
https://github.com/apps/matrix-to is a Github App which makes use of their shiny and new Content Attachments API/Webhook. When a matrix.to or view.matrix.org URL is used this app is activated. It adds a little snippet with the Room Title and Topic (if the room is peekable from matrix.org). In future it’ll work for event permalinks, but currently there is no support for peeking context/event in Matrix API.
I’m at 35c3 with some known characters from the Matrix world (as well as 15,000 others.) If you’re here too, come visit us in our assembly, and also make sure to come to Dijkstra tomorrow to watch me present a look back at on the last year: https://fahrplan.events.ccc.de/congress/2018/Fahrplan/events/9400.html. We have recorded a message for Matrix Live from 35c3, but will post tomorrow with some more footage from the event.
Matrix Live: Half-Shot talks bridges, and working on libpurple bridging to Matrix
You may have seen that Half-Shot been working fearlessly and tirelessly on bridges for many, many months. In this episode of Matrix Live Half-Shot chats with Matthew about his work, progress and achievements, with a focus on recent matrix-appservice-purple and XMPP work. Audio is not amazing, but worth listening to get acquainted with recent work.
The matrix-appservice-purple bridge gained a XMPP specific backend for better performance when you want to do just xmpp bridging. It’s rather quick right now, and needs dogfooding.
A couple of weeks ago, fr1kin PR’d a nice Matrix badge to the Shields project that tells you how many users are in a public room. There were a few issues with it so I PR’d some changes to make the badge more usable, and it’s now merged and live (as of yesterday evening), with examples available here! 🙂
For instance, here’s a badge for TWIM: https://img.shields.io/matrix/twim:matrix.org.svg 😉
matrix-js-bot-sdk has received early support for application services. Similar to the official matrix-js-sdk, the bot-sdk uses an Intent-based model for making bridges easier to write. Check out the simple example here for more information on how it works.
koma-library
druig continues work on their JavaFX/kotlin client project:
The matrix client API implementation in koma is being extracted into a new repo, which is going to be a lightweight library for Kotlin.
Seaglass now available on homebrew
Aaron Raimist reports that Seaglass is available on homebrew for macOS:
Installing Seaglass is now easier than ever. If you already use Homebrew to manage other packages, you can now install Seaglass with a simple brew cask install seaglass.
alphapapa reports from a team who have forked and are maintaining a Matrix client for Emacs:
Many additions and improvements to matrix-client.el (https://github.com/jgkamat/matrix-client-el) recently, including a “standalone client” mode that launches Emacs to look like this.
the matrix-docker-ansible-deploy playbook has received some bugfixes and improvements lately. Most importantly, it’s now running the freshly-released Synapse 0.34.0 under Python 3, so memory usage should be much better.
I started work on matrix: scheme support in libQMatrixClient/Quaternion. Expect more news on this around New Year.
Fractal road to 4.0
Fresh from their hackfest in Seville last week, Alexandre Franke reports:
The Fractal crew spent the week chasing last minutes bugs and made two beta releases (3.99.0 and 3.99.1) in preparation for the big new stable release, 4.0, which is right around the corner.
#space:im.kabi.tk for anyone interested in space, rocket launches, satellites, etc.
Are you wondering what NASA’s new Mars Rover is doing? Maybe you live on the west coast of the United States, and you saw that meteor on Wednesday night that came within minutes of a scheduled rocket launch and just after three astronauts left the space station. If any of that sounds interesting to you, feel free to join the room.
A Matrix bot is being tested to send updates about upcoming rocket launches to the room.
Just a head’s up that I’ve increased the storage capacity of the database. With current projections, the server should be good for another year or two.
The database is also 1ms closer to Synapse and has a faster CPU in it. It probably won’t make a dent in speed on federation, but it is a step forward.
I’ll be rolling out python 3 to the homeserver this week too, which should help a little bit.
Back to reskin. We start to implement e2e backup screens.
Riot Android
From the Riot Android team:
New screen to troubleshoot notification issue has been merged on /develop.
Splitting current Android SDK to separate crypto part is on it’s way. We’re doing this in order to be able to integrate crypto faster in the Riot reboot.
We released 0.34.0! This release recommends Python 3 for production and brings with it huge performance improvements. If you’ve been putting upgrading off upgrading your Synapse, now is the time to do so. For more details here is a post that explains what you should expect and a recent Matrix Live interviews Amber (hawkowl) on the subject.
Aside from that we are working furiously towards federation R0 and have a bunch of MSCs to get us ever closer. You can track our progress here.
andrewsh notes that 0.34.0 is also available in the Debian repos:
Synapse 0.34.0~rc2 in Debian since Tuesday, 0.34.0 uploaded today; both use Python 3 only
Dendrite’s internal audit is progressing very well and is getting very close to its end.
What’s left to do for me is check the implementation status of a few Matrix features, and translate those into tagged GitHub issues so that everyone can have as clear of a view as possible on what’s left to work on.
I’m on holiday all of next week, but hopefully will have some good news about that the following week.
That really is it for now
Did you get to the end? What was your favourite section? Come tell us in #twim:matrix.org! Do you have your own update you’d like to add? Same place, come chat in #twim:matrix.org.
Next week is various things. It’s Christmas, which means there will be more hacking and coding happening than usual I expect! It’s also 35c3, which I will be attending, and might affect scheduling next week. Stay tuned in #twim:matrix.org for news, and come join us in #matrix-35c3:matrix.org if you’ll be there and want to meet up!
Developers and creators of Fractal, the GNOME Matrix client, have been holding a Hackfest this week in Seville. Matthew and I caught up with them on video for Matrix Live this week, and discussed the product improvements they’ve been making and their plans for the next release (due next week!)
Matrix Spec
MSC1466 merged. MSC1466 from Erik describes “Soft Remote Logout”, which avoids loss of encryption keys in circumstances of remote logout.
MSC1730 entered FCP (final comment period, the last step before being merged.) read the proposal
This Week in Ruma
We don’t often get to feature news from Ruma, but this week there is an updated This Week in Ruma.
Ruma is not dead, however, and small improvements have continued over the last year. The Matrix spec has advanced quite a bit and many of the blocking issues for Ruma have been resolved. Rust’s maturity is another story. async/await is still under development and this is the most significant blocking issue to progress on Ruma.
maubot got a command-line tool for building plugins and managing maubot instances. I also added some server-side stuff for easy registration of accounts in the management interface using synapse shared secrets, but the UI for that isn’t ready.
Also, I’m planning on adding some kind of small Matrix client in the management interface for manually managing the added bot clients. That might lead to a separate library that could be used in other projects or embedded in websites.
mxisd hits v1.2.1 with a new maintenance release, fixing bugs and regressions from v1.2.0. Updating to v1.2.1 is strongly encouraged as v1.3.x will contain breaking changes and will not be a straight-forward update like v1.x has been until now.
Clients
Lazy-loading lands in QMatrixClient
kitsune has reported that lazy-loading, a Matrix feature that entails only loading room member details as needed, is now in libQMatrixClient master.
reference implementation in Quaternion will follow up
Black Hat has been testing the feature in Spectral, which uses the library, and says there is a 30%-50% reduction in RAM usage at startup.
Riot Web progress on /experimental and v0.17.8 released
Bruno has been working on /experimental, and I recommend taking a look at the progress there! The next version of Riot Web is closer than ever.
Improved read markers now available on /experimental, needs further tweaking though. Brought back community UX on redesign, other small improvements.
This new version supports the consent of matrix servers terms of service (including GDPR) in the registration flow.
It also contains fixes for the “Empty room” bug, the registration issue on iOS 10, etc.
This new version supports the consent of matrix servers terms of service (including GDPR) in the registration flow.
Many bugfixes
SDK contains KeyBackup
My internal audit of Dendrite is continuing, drawing a more and more precise picture of what’s left to fix and implement. I aim to have it over by the end of the year, or the very early days of 2019 at the latest. Folks can track its progress through https://cloud.abolivier.bzh/index.php/s/qXi2KFjCQk2c6eG
modular.im now has Extra-Large instances available
Due to demand, modular.im Hosted Homeservers now has Extra-Large instances available. If you need to service 1,000+ users on a Matrix homeserver, this is the product for you!
linuxgaming.life homeserver is the number one Matrix homeserver focused on Linux gaming
Recent Comments