Today the European Parliament, the European Council and the European Commission will meet again for a discussion about the Digital Markets Act (DMA). This is the second of three of these meetings, appropriately called trilogues, where each party exposes their stance on a proposed law and the group tries to agree on the final version.
The DMA is a groundbreaking step forward in shaking the hold a few gatekeepers have on users and the market, in particular because it looks to (among others):
Require gatekeepers to allow other services to interoperate with their services
Prevent them to treat their own services and products more favourably (for example by ranking)
Require them to allow users to uninstall any pre-installed software or app
The interoperability obligation is obviously the one on which we’ve kept a particularly close eye, as if it lands well it could take the success of Matrix to the next level completely overnight.
However, whilst in our mind interoperability automatically implies “open standard”, there are actually different ways of implementing it, depending on how far one wants to go. Typical debates here have been between whether to force gatekeepers to maintain open and well documented APIs, or whether to go full swing and mandate an open standard, and every shade in between.
We’ve been lucky to have had the opportunity to talk to policy advisors from different European member states, and it has been pretty fascinating to realise that it was always the same arguments which were being presented back at us, straight from the gatekeepers partyline.
We’ve ended up just listing them in a quick, high level, Myth Debunking exercise and thought it would be useful to actually publish them for everyone to access, so here they are!
MYTH #1 - "It is impossible to have a standard that is open, decentralized and secure at the same time"
⇒ false: HTTPS did it, Matrix did it.
MYTH #3 - "Interoperability is incompatible with end-to-end encryption"
⇒ false: services just have to speak the same language, email has proved this with S/MIME and PGP - where different vendors can and do interoperate with E2EE. It’s even better when the protocol is E2EE by default.
MYTH #4 - "It may work for messaging, but less so for social networks"
⇒ false: it's still about managing content and users. Even though social networks have more varied content, it is already well modelled for their own APIs, ready to be expressed in a common language. The key is in the fallback option on unsupported features, as well as the ability to have moderation tools (more on that later).
MYTH #5 - “Interoperability is not compatible with data privacy”
⇒ false: Interoperability gives the ability to users to choose who is hosting their data and as such choose providers they trust. Besides, the DMA doesn’t live in a vacuum: it will exist alongside horizontal regulations like the GDPR and the Data Act, which give people sufficient control over their data to rectify their choices if they are not happy. Because the possibility of interoperability is there, it does not mean it will become mandatory for users to use it: they will still have their own threat models and will make decisions accordingly, just as they do today. But enshrining interoperability in law will at least ensure gatekeepers need to provide recourse for people to have further control over their data, which will be an improvement from the landscape today.
MYTH #6 - "There is no user need"
⇒ false: most haven't had a taste of interoperable chat/social media (but they know email), others are demanding bridges between services: 25% users of 2 communication apps lose contact with friends because they are using too many apps. And this figure doubles for people using more than 5 apps. There was no demand for cars when they were created: people only wanted faster horses.
MYTH #7 - "There is no demand from European companies"
⇒ false: The fact it is so hard for European companies to remain competitive enough to stay alive means there are few of them to complain about what is killing them! However these companies are gathering to push for interoperability (like the Coalition for Competitive Digital Markets). It will enable them to be more innovative in the product they develop by benefiting from an existing open network rather than being slowed down by having to build one from scratch. Companies will compete on the value they add rather than the size of their network. An open standard also gives an open field for innovation from a business model perspective. The Web is an excellent example of how much an open network fuels innovation and growth.
MYTH #8 - "It is better to require providers to have open and stable APIs than define a single open standard"
⇒ false: this is the best way to leave gatekeepers at the center of the ecosystem as it means that each player has to multiply its effort to interface with every single other player, but every player will only have the resources to interface with a few of its counterparts and will logically default to the bigger ones, effectively not solving the problem. In addition, if providers are not aligned on which encryption to use it will just break end-to-end encryption and create risk for the user in every bridge. In practice the DMA is about forcing the gatekeepers to interoperate only, but we strongly believe that everyone should be interoperating if we are about improving the user’s experience and control, and giving more space to companies to innovate. Limiting it to the gatekeepers is a first step, but only a defensive one.
MYTH #9 - “An open standard limits innovation if it defines a lowest common denominator”
⇒ false: the lowest common denominator should match what users consider as table stakes in a messaging or social media app. Providers can innovate on top by providing different features which go beyond table stakes, for example by targeting niche use cases, like messaging services focused on elderly and disabled users, or focused on healthcare, warehouse workers, or integrated in a CRM for call centers, or creatives… Providers also can implement a profile of the standard which is a subset of its full scope, ensuring the standard remains a highest common denominator..
MYTH #10 - “It will be impossible to moderate social networks built on an open standard”
⇒ false: decentralised networks actually have driven the adoption of much more sophisticated moderation techniques than the coarse approaches of centralised silos. Appropriate moderation means have to be part of the open standard definition, and some are already used in Matrix. It would also empower victims who today have no choice but get in touch with providers one by one. Each provider will also have control over their own users, and users can select providers whose T&Cs are aligned with their ethics. The world is not black and white, unlike what Silicon Valley tries to make us believe.
MYTH #11 - “It will take years before being able to define an open standard”
⇒ you don’t have to: You could leverage existing technologies which are being used by the industry. Matrix, XMPP and ActivityPub exist today. For instance, Matrix has been managed by its own standard body (The Matrix Foundation) and could be ratified by a more established one like IETF, ETSI or W3C if needed.
Obviously the devil will be in the details of the way the final text is formulated, as well as the limits, obligations and controls put in place, but overall it should be an improvement for all European users and companies and we’re looking forward to seeing how today’s trilogue goes!
Welcome to the second installment of our quarterly spec releases. If it feels like it hasn’t been long since our last release, you’re not alone! Our last release was just 3 months ago, introducing the new platform and world we build within.
This improvement in speed might seem too fast, but fret not: implementations are not expected to update as soon as the new spec is published. Rather, it is more realistic to expect that the ecosystem gradually update over the course of the following few months/year. Particularly after the massive release that was v1.1.
So, what’s new in v1.2? With 18 MSCs merged there’s a lot to cover - we’ve picked some notable highlights and recommend the full changelog at the bottom for a complete idea of what’s been going on.
Spaces launched into beta last May, redefining how we can use rooms on Matrix to represent different data structures. Described mostly as MSC1772, Spaces are simply rooms with a specific type in their m.room.create event. With state events being used to define which other rooms (meaning Spaces too) are part of that Space, the possibilities for tree-like structured data become endless.
There’s still quite a lot of work to do in the Spaces space (hah), though we’re excited to see it all land. For instance, MSC3216 and MSC2962 target power level syncing, MSC3219 aims for flair, and MSC3089 looks at file structures using Spaces. We might even be able to replace the public room directory with a server-wide space, making writing clients a little bit easier.
Restricted rooms are new in this release from MSC3083. In these rooms (and therefore Spaces too!), the join rules can be configured such that a member of another room can join without needing an invite. In a typical setup, this means that a Space could be set as invite-only, but all the rooms and Spaces underneath that Space could allow joins for members of the parent Space. When someone wishes to explore the universe you’ve laid out in your Space they can simply join the interesting rooms without having to ask for invites constantly.
This feature changes how membership events are authorized, so is only available in room versions 8 and 9 (both introduced in this release). Room version 9 fixes a relatively rare bug from version 8, so we’d recommend using v9 if you’re planning an upgrade for this functionality.
Further work in this area involves figuring out how to keep membership lists perfectly in sync between the rooms, which is currently done by external tooling. For example, evicting someone from the parent Space could (optionally) evict the user from all the subsequent rooms and Spaces too.
We also need to figure out how to support both knocking and restricted rooms at the same time (oops). MSC3613 and MSC3386 both tackle this problem in different ways and timescales.
A massive shoutout goes to kitsune and the whole community for working on MSC2312, giving us a URI we can pass around outside of Matrix to bring us back in. The early work on this dates all the way back to 2014, the very beginning of Matrix’s development, and has since been marked Provisional by the IANA.
MSCs are how the spec changes in the way it does - adding, fixing, and maintaining features for the whole ecosystem to use. The blog post can’t cover them all, but that doesn’t make them any less important! Check out the full changelog below, and the Spec Change Proposals page for more information on how these MSCs got merged (hint: they submitted a proposal, which anyone can do - take a look at the Matrix Live episode where Matthew covers the proposal process).
Explicitly mention RFC5870 in the definition of m.location events (#3492)
Add 403 M_FORBIDDEN error code to /profile/{userId} as per MSC3550. (#3530)
Clarify the description of the /sync API, including a fix to an ASCII art diagram. (#3543)
Clarify that base_url in client well_known may or may not include trailing slash. (#3562)
Clarify which signature to check when decrypting m.olm.v1.curve25519-aes-sha2 messages. (#3573)
Clarify what "Stripped State" is and what purpose it serves, as per MSC3173. (#3606)
Clarify how to interpret missing one-time key counts. (#3636)
Correct the schema for the responses for various API endpoints. (#3650)
Clarify that group mentions are no longer in the specification. (#3652)
Distinguish between "federation" event format as exchanged by the Federation API, and the "client" event formats as used in the client-server and AS APIs. (#3658)
Fix the rendering of the responses for various API endpoints. (#3674)
Distinguish between "federation" event format as exchanged by the Federation API, and the "client" event formats as used in the client-server and AS APIs. (#3658)
Fix the rendering of the responses for various API endpoints. (#3674)
Correct the documentation for the response value for GET /_matrix/app/v1/thirdparty/protocol/{protocol}. (#3675)
Element Desktop 1.9.6 and earlier depend on a vulnerable version of Electron, leading to a High severity vulnerability in Element Desktop, relating to its functionality for opening downloaded files. If successfully exploited, the vulnerability allows an attacker to open an arbitrary file path on the user's machine using the platform's standard mechanisms, but without the ability to pass additional arguments or data to the program being executed.
However in certain platform configurations, the same vulnerability could allow an attacker to open an arbitrary URL with an arbitrary scheme instead of a file path, again using the platform's standard mechanisms. There has been research demonstrating that the ability to open arbitrary URLs can sometimes lead to arbitrary code execution.
The attack requires user interaction and the exploit is complex. To the best of our knowledge, the vulnerability has never been exploited in the wild.
Patched in 1.9.7 with further hardening done in 1.9.9 to ensure it's harder to exploit even in light of new Electron vulnerabilities. Please upgrade to 1.9.9 as soon as possible. The vulnerability has been assigned CVE-2022-23597.
Kudos to Aaron for adding a GitHub action to matrix.org's repository to check for typos, and taking the time to fix them all! Worry not fellow proofreaders, our post-publication TWIM proofreading tradition is not extinct: some typos are not caught by the CI, such as misspelling of proper nouns (e.g. Devianart) or articles (e.g. "an proofreader" won't make the CI upset)
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
The release of Matrix v1.2 is right around the corner, and is expected to go out on February 2nd. This is in line with our quarterly release cycle for the spec going forwards.
Please reach out to us in #sct-office:matrix.org if there are any last-minute MSCs that haven't started/finished Final Comment Period that you believe should be included!
A concept similar to robots.txt for the web, but for Matrix! Currently the way to inform room-crawling bots in Matrix today is by banning/kicking them from the room. However, this doesn't allow you to blanket prevent bots from joining - nor does it only inform bots that they should access some of the room's data. It's a binary switch.
Consider if you wanted your room to be crawled, but the room topic to not be indexed. This MSC could help with that! Give it a read/review if you're interested.
Have you ever pinged someone by accident, because you replied to a message with their name in it or maybe even a whole room mention? Or are you avoiding replies for that reason? Did the quote of a messages suddenly vanish when someone edited their message in a client? Or are you just in general unhappy with replies only showing what type of message was replied to, instead of showing the actual image being replied to (on mobile for example)? Were you annoyed, that you could only reply with a text message, but not an emote message, an image, a sticker or even a video?
I spent a bunch of time trying to remove the blocker for that in the Matrix specification, you can find the proposal here: https://github.com/matrix-org/matrix-doc/pull/2781
As it turns out, it is not quite as simple, because notifications for replies to your messages rely on a bunch of implicit interactions, which is why I also wrote another MSC to fix that: https://github.com/matrix-org/matrix-doc/pull/3664
I'm a bit unhappy with replies across the matrix ecosystem at the moment and I hope this is a small step towards improving things. It's certainly not as exciting as spaces, voip, threads or stickers, but it is something which affects me every day and I think those small papercuts need some attention too.
I hope some of you share my love for the less exciting stuff!
To add onto Nico's efforts, I think that mentions today are based on too many "false-positives" (ping on displayname or username mention), and i kinda wish for the "just mention them" UX of discord, telegram, whatsapp, and so many other messengers.
So I've also made a proposal that tries to bring some explicitness in that, a new push rule that'll fire on "mention"; https://github.com/matrix-org/matrix-doc/pull/3517
It'll look for any @-MXID mention in the plaintext, or look for an <a>-mention "pill" in the HTML. It is a hack, but it is the most correct way of determining if you've been pinged in matrix, at the moment.
In terms of reply fallbacks - the Spec Core Team gave an overwhelming thumbs up to removing reply fallbacks via MSC2781... but given practically this causes a cascade of other work (defining and implementing push_rules for replies, and defining and implementing push_rules for threads (MSC TBD)) and so as a temporary transitional measure we've instead made reply fallbacks best effort (MSC3676). This means that clients can now reply using non-textual events, and we can use replies as a fallback for threads for non-thread-capable-clients/ASes once MSC3440 lands. To be clear, the intention is still to incorporate all of Nico's excellent contributions on MSC2781 and MSC3664 however.
Hello TWIM! Last week we released Synapse 1.51. This is a great release which includes lots of performance work around sending aggregations down /sync. It also makes Spaces much more reliable: we tracked down a bug with caching which could cause some sub-spaces to randomly appear empty when queried over federation. We also removed the 50 result limit when listing the contents of a Space.
We're also starting to see some promising results experimenting with ways to make room joins faster (MSC2775). It's not quite ready to demo, but we should have something to show off before too long. 🤞
We know that it has been a while coming but today we have released Dendrite 0.6! This version contains a number of significant changes, including switching away from Kafka to NATS JetStream and refactoring a number of other components. We have been making architectural changes recently to help Dendrite scale better in the future and to fix a number of race conditions that were present before. Federation in particular is much smoother and better behaved in this release, with overall lower CPU and memory usage and less resource spikes.
NATS JetStream support, deprecating Kafka and Naffka
The roomserver now being responsible for fetching missing events and state instead of the federation API, which fixes some race conditions
Strict ordering and asynchronous input support in the roomserver input API, which smooths out incoming federation significantly
Consolidated federation API, including functionality from the now-deprecated federation sender and signing key server components
Database-backed device list synchronisation, which is now more reliable
Correct gap checking when fetching missing events and state
Better behaviour and lower resource usage by the /event_auth endpoint
Spec compliance, as measured by Sytest, currently sits at:
Client-server APIs: 65%
Server-server APIs: 94%
This is a highly recommended update so if you are running a Dendrite server, please upgrade! As always, you can join us in #dendrite:matrix.org for Dendrite discussion and announcements.
Hey folks, no releases this week but a general update on a little project of ours. I've been working through the problem of making bridges easier to use and supplimenting our command interfaces with pretty buttons and forms. To that effect, we've landed out first PR on adding widget communication support to the matrix-appservice-bridge library \o/.
You can see some of the details here https://github.com/matrix-org/matrix-appservice-bridge/pull/365
but that's not all. I'm doing a talk about this whole subject at FOSDEM which you can tune into! See the deets at https://fosdem.org/2022/schedule/event/matrix_next_gen_interfaces/
Next to spec work, progress on porting Nheko to 100% qml continues. This week I ported the login and registration pages. I also did some minor cleanups on them and prep work for future improvements (like buttons for each SSO provider or providing email address and registration token inline instead of in a separate popup, if the server requires it). Logging in or registering is often the first thing a user does on a client or maybe even the first thing they do on Matrix! As such I should really spend more effort optimizing that feature, that usually falls off the priority list because I don't use it often!
Once Nheko is 100% Qml we should also see lower memory usage again, here is a screenshot from a Nheko instance with over 100 rooms, that's been running for a few days of active use on Windows:
We also fixed an issue where Nheko could crash the notification service, because it sent images not following the Freedesktop notification specification and tasty spent some time further improving the man page.
Hello folks, it's been a while since we last spoke! We have been focused on the code, but we're long overdue for an update. A lot has happened since November. Fractal-Next is getting closer to feature parity with current Fractal, and even supports new things:
Timeline
Fractal-Next now allows you to open and save sent files
It also displays images, videos and stickers in the timeline
You can also get a better view of media send to the room thanks to the built-in media viewer
It (finally!) supports reactions (displaying them and sending new ones)
User verification
Fractal-Next now supports verification of other users by scanning their QR code, or via emoji
When a user is verified, an icon is displayed next to their username in the list of room members
Room details
The room details show now the members of the room including the power level
General UX
Fractal-Next is better integrated with GNOME's secret management service Seahorse
It supports room upgrades
It also supports inviting users to a room
Users can change the category of rooms in the sidebar via drag and drop or by using the context menu
Release candidate was late as we have more regressions and release blockers than normal: last minute fixes include a couple of crashes when hanging up a call from the picture-in-picture view and in the appearance tab of settings, plus chasing a crashing bug on Linux.
Metaspaces has landed in the release candidate: it will give you a new way to group your favourites, DMs and rooms outside of spaces. You can switch these on in the Quick Settings menu at the bottom left from Monday.
Bubble layout for messages has landed for the upcoming release!
This release will update to Electron 15 which uses a newer Chromium, so if you host your own Jitsi, please make sure it’s up to date, or you may find calls start breaking.
Nightly testing went well on Tuesday, with 47 test cases showing up 15 new issues
In labs (you can enable labs in settings on develop.element.io or on Nightly)
New thread fallback using the reply chain
Better stability when home server supports threads API defined in MSC3440
We are intending to add the new and improved search to the next release candidate on 8th Feb, community testing planned for next week
Find more about Cinny at https://cinny.in/
Join our channel at: https://matrix.to/#/#cinny:matrix.org
Github: https://github.com/ajbura/cinny
Twitter: https://twitter.com/@cinnyapp
Matrix highlight got experimental support for Firefox! It's still a bit crashy, but it shouldn't be much harder to stabilize it, and get the tool working properly there, too. Firefox is my personal browser of choice, and others have requested it, so it's nice that it's on the horizon :)
Also, after the discussion in Matrix Live and with some minor tweaks, I was able to get the extension to play nice with conduit! So that's now a viable alternative if you want to use Matrix Highlight with your own, self-hosted server.
Circles is a project to build a secure, end-to-end encrypted social network using Matrix technology.
This week the primary focus was on updating Circles to work with the latest Matrix iOS SDK. I also successfully built and packaged the first beta release from an Apple M1 Mac.
The next step is to develop and integrate a new, more secure password-based login based on the BS-SPEKE protocol. This will replace Circles' current approach, which is described in MSC3265. Hopefully the new login flow will also be of interest to the broader Matrix community.
Our search continues for an Android developer to help us bring Circles to the world's largest mobile device platform. If you are excited about working with Matrix, Android Studio, and Jetpack Compose, send a resume and cover letter to [email protected].
Trixnity version 1.1.0 has been released! It adds support for room key backup, which makes developing a matrix client a lot more comfortable.
Here is the changelog:
Added key backup support!
RoomService::getAll() returns a reactive list of reactive rooms instead of a reactive list of rooms. This allows to implement huge room lists without performance losses in the UI.
Some reactive data can be accessed in a non reactive way. Note that you will then only get a snapshot of the data.
Prevent sending room keys as to device messages, when they cannot be encrypted (e.g. due to missing one time keys).
Hello again! Halcyon is a python Matrix bot library created with the intention of being easy to install and use.
With this release, we have reached our second release milestone! Check out the roadmap in notes.md (located in the repository), for what is planned for RC3!
I'm really happy about the performance and functionality gains for this release. If you have any questions or bug reports, come and chat with us over in https://matrix.to/#/#halcyon:blackline.xyz
More info at on the project at https://github.com/WesR/Halcyon
Added
Detailed room info is here! The bot will now cache and provide room info with each message, automatically refreshing in the background. Check out usage.md for info on what is provided
get room admin / moderators, topic, related groups and more
Better polling through long polling (Thanks @forden:matrix.org for pointing out this improvement)
send images with the new send_image() function. It includes simple toggles for blurhash and thumbnail generation
More examples in /examples
General roadmap and documentation updates
Fixed
A fix for the slow polling, which might have also caused issues on systems with frequent network drops (Thanks @octt:matrix.org)
Bubo is a friendly little owl (bot) that helps maintain your community. It's been over a year since cutting the last release, so the changelog contains a bunch of things, namely:
Added command to set power levels in rooms.
Added users command to list and create users in a configured Keycloak. Optionally send invites via keycloak-signup, a web app that allows self-registering with Keycloak via short lived tokens.
Add logging to a Matrix room.
Bubo admins and coordinators can now be set based on room memberships.
Add command to make Bubo unlink and part itself from a room.
Add command to list rooms Bubo maintains but is not admin in.
Add command to recreate a room. This follows the upgrade room functionality, but does it without needing admin permissions (basically almost everything except a tombstone event). Useful if for example you have accidentally lost admin in a hundred or so rooms in your community. Which may have happened 😅
The communities command is now deprecated (support for spaces coming in the next version).
Additionally some other changes and fixes, see changelog for full details.
Plans for next up features include syncing with Discourse to replicate spaces from Discourse groups and populate space members from Discourse group memberships (when Discourse shares the same Keycloak SSO provider with the homeserver).
🔗Reaching another feature Milestone – Submitting images via Matrix Art.
Matrix Art now supports logging in with any account (well-known not supported yet. You need to provide the full server address) and submitting images through the interface.
You can now either just log in (No benefit though at this time as comments and follows are missing) or create a Matrix Art Profile.
An Account can also be created later. An Account would create a Profile room at #<mxid> on your server if it doesn't exist.
This currently is only possible by doing a logout and login in again with the box checked. A better way will be added soon.
Editing the Profile is not yet implemented at this time.
These 2 changes should allow some people to experiment with this. Mobile design is not tested at this time.
Submits should work. If you were previously logged in, you will need to log out and back in for some variables to get set correctly.
Registration is still WIP as there is some more moderation prep needed before the server accepts the registrations.
I hope people cheer in and post their images :)
(Also, please use the nsfw flag if required. This will make them hidden on the front page but shown in your profile page. I will block people that don't use it from the instance otherwise. Please be sensible.)
Apparently, a German school forked FluffyChat and is using it for homeschooling: https://www.golem.de/news/matrix-grundschule-forkt-messenger-2201-162562.html
(German only news article)
Sources: https://gitlab.com/hermanncoders/hermannpost
A long time ago, Synapse used to serve a very basic web Matrix client (named "console") that could be used to connect to the homeserver. Server administrators could chose to make it available to their users by configuring a webclient listener.
This web client was removed in Synapse 0.34 back in 2018, but the webclient listener stayed, instead allowing server admins to serve the web client of their choice (or redirect to it) through the web_client_location configuration file.
Synapse 1.53 will remove the webclient listener, as well as the ability to set web_client_location to a static directory (instead of a HTTP(S) URL). See the upgrade notes for more information.
The concept of server-side aggregation in Matrix is defined in MSC2675 and is the ability for homeservers to extend the information included in an event using other events that relate to it. This allows, for example, clients to quickly retrieve the reactions associated with a given message, or its latest edit.
This release includes a number of notable performance improvements to calculating aggregations when responding to /sync requests. We continue to measure and investigate potential performance improvements in this area, which should end up greatly benefiting /sync response times.
FOSDEM, one of the biggest gatherings around free and open-source software in the world, is happening next week! Just like last year, the conference will happen online and will be hosted on Matrix. And just like last year, it will be packed with super interesting Matrix-related (but not only) talks.
One new addition this year is the presence of a whole devroom dedicated to Matrix. It will be hosted on February 6th, and you can already find its whole schedule of talks right here.
This release includes a couple of spaces-related bug fixes, specifically related to the /_matrix/client/v1/room/{roomId}/hierarchy API. One of them in particular targets a bug in spaces that include more than 50 rooms, and should make it much easier to look for a specific room inside a space.
The synapse_review_recent_signups script, which allows homeserver administrators to review recent signups (e.g. in the event of a spam attack), was also improved with an option to exclude virtual users belonging to an application service from the results. See synapse_review_recent_signups --help for more information.
Please see the Synapse release notes for a complete list of changes in this release.
Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including (in no particular order) Dirk Klimpel, br4nnigan, Philippe Daouadi, Daniël Sonck and AndrewFerr.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.
Work on preparing the release of Matrix v1.2 is currently underway. As of today, the Spec Core Team is aiming for a release of Matrix v1.2 on February 2nd.
If you know of any MSCs which you believe should be included in Matrix v1.2, but haven't started/finished Final Comment Period yet, please bring them up in the #sct-office:matrix.org, and we'll take a look. Thanks!
When I first came across this MSC, I didn't know what "Early Media" as a concept referred to. Even after skimming the MSC... I still didn't really know. But I'll demystify it here in case it peaks your interest and you would like to learn more.
Early media is essentially any media that is exchanged between the moment you start a call, to the moment the other side picks up (a connection is established). For instance, the ringing you hear while waiting for someone to pick up (called "ringback tones"), or the "busy" tone you hear when the line is busy. Those audio bits are known as "early media". Video can also fall into this category (though that's less common).
This MSC would essentially allow Matrix to introduce these concepts in Matrix-only calls (though these days the client just plays sounds while connecting), but more crucially would allow Matrix to interoperate with other protocols (like SIP) that expect to handle these features.
So yay, more interoperability and bridge compatibility!
After a somewhat bumpy process, we released Synapse 1.50 on Tuesday! The big thing you need to be aware of is that we're turning off support for Python 3.6 and PostgreSQL 9.6, including Linux distributions which ship with those versions by default (like Ubuntu 18.04 LTS). Please make sure your infrastructure is up-to-date.
I'm personally very excited that we tracked down a bug which could cause device list updates to get lost when being sent over federation. When device lists fall out of sync, it can cause failures when attempting to decrypt messages, since the keys may not have been sent to all of the user's devices.
We've also made quite a lot of progress in allowing Application Services to support end-to-end encryption via the still-experimental MSC3202.
For MSCs which have been merged into the Matrix Spec, we now implement MSC3419, allowing guests to send state events into rooms, and we now use stable identifiers for cross-signing and fallback keys per MSC1756 and MSC2732.
Looking to the future: We're aiming to release 1.51 next week so it has plenty of time to burn in before we host FOSDEM 2022. This is a pretty quick turnaround from 1.50, but we'll return to our usual fortnightly release cadence for subsequent releases. To that end, 1.51.0rc1 is out today; give it a shot. 😉
Hello again! In the last week we continued to optimize the rocksdb backend and Conduit in general, trying to get it as memory efficient as possible. Using valgrind we could see that memory was not getting released in some cases.
We found out that switching to the jemalloc allocator completely got rid of this problem and memory usage seems to be a lot more stable now!
All of this is currently available on the next branch. We are preparing to make a v0.3.0 release soon!
And back to regular form, my Helm Charts have gotten some upgrades again; element-web got upgraded to 1.9.9, and matrix-synapse got both 1.50.0 and 1.50.1 (the configuration generated by the chart shouldn't be affected by the .0 bug, but always good to upgrade)
To keep your timeline clean and ordered we’ll soon be introducing threaded messages to Element. Currently the team is working hard on the fallback solution, and polishing up the user interface.
Threads is already in Labs on Web, so go ahead and check it out!
For mobile, we’re hoping to release Threads to Labs in the next few weeks.
Polls
Get ready to start asking more questions… Polls are nearly ready to go! You’ll be able to ask folks things like; their favourite superhero, which day you should grab lunch, or even the best way to make tea (milk first, always). ☕️
If you just can’t wait ‘til we’re ready to launch, you can enable Polls from Labs.
Location Sharing
Location sharing is almost here! You’ll soon find a new setting to turn this on, which will give you the location sharing icon in your composer and can be used to tell people exactly where you are!
The setting will be in the next release on iOS and Android, and will start “off” by default. Once the feature is settled in, we’ll turn it on by default.
We’re moving forwards with our PostHog Analytics implementation and are super excited to start to get to know how our users experience Element. Remember; it’s off by default and you have to opt-in to share.
In labs (you can enable labs in settings on develop.element.io or on Nightly)
With the coolest project name by far, the Bubbles team is working hard to bring you message bubbles ASAP! These should land in the next week or so but are available in Labs today.
The integration of analytics tracking has been included in the most recent version of the app. Using PostHog Analytics we’ll be able to make informed product decisions for our app as we’ll have more visibility into the usefulness of each feature.
Remember; Analytics is opt-in and you don’t have to share any info with us. If you choose to opt-in we’ll start to learn how users use Element and how we can simplify your experiences.
We’ve had some trouble with the stability of our releases this week but the team’s been working hard to get it all ironed out. Our latest update to the app store fixes some bugs and includes the option to enable analytics.
Analytics? Yes! Knowing how our users traverse our app, and understanding this cross-platform, will help us to tailor to your needs and make impactful improvements. If you don’t want to send anonymised event info to Element, no problem! Just say no. If you change your mind, there’s a toggle in Settings.
In development:
Message Bubbles? Yes, please! With a week of successful testing internally we’re nearly ready to release message bubbles into the wild. We’re excited to see what you think.
Element’s first time user experience could use a little help, so the team have been working on improvements to our sign up flow that will hopefully reduce confusion for newbies.
Next up: Bulk editing of rooms for updating room power levels or aliases in masses.
If you're a maintainer of a homeserver, space or bridge, please let me know your use cases.
It's a small cli tool that does exactly what it says - exports matrix messages from a room. As example you can check etke.cc/news - that page and all items on it generated by emm from #news:etke.cc room.
The tool gracefully supports room aliases, message edits, custom templates (check the contrib/ dir for example) and 2 export modes - single (all messages exported to a single file) or multi (each message exported to separate file, that's how etke.cc/news works)
Circles is a project to build a privacy-respecting, end-to-end encrypted social network on top of Matrix. It was originally built out of the desire for a safer way to share baby photos with friends and family, but it can be used by anyone who wants easy sharing combined with strong security.
Recent news:
The Circles iOS app is back in beta on TestFlight. Builds 0.99 (6) and 0.99 (8) are rolling out now.
The latest updates fix some bugs in Circles' use of Matrix's encrypted recovery feature to improve the reliability of E2E encryption.
FUTO, the new company behind Circles, is hiring an Android developer to help us bring Circles to Android. Interested candidates should send a resume and cover letter to [email protected].
The (old) Circles homepage: https://kombuchaprivacy.com/circles
The code on Github: https://github.com/KombuchaPrivacy/circles-ios
long time no see! Today I come with a new internal etke.cc tool that publicly available, because open source matters.
Honoroit is a helpdesk matrix bot with end-to-end encryption support, that utilizes MSC3440 (Threading) to act as a proxy between a customer (any matrix user) and your backoffice (users added in special room), each customer's room = thread in a backoffice room, where multiple operators can send messages to the same customer at once.
Pretty bad description, I know - check the source code to see screenshots.
Updates:
the v0.9.2 release brings the fallback reply-to mode, so even on matrix clients that doesn't support threads yet you can use it with good ol' replies.
the v0.9.3 release fixes commands parsing in the reply-to mode and adds prefixes to the thread topics
Matrix Art received some minor changes since the last twim post:
You can now get a rss feed of the posts at https://art.midnightthoughts.space/posts.rss (This is in the same format as devianart does their feeds)
Mobile should be mostly fixed
The About-info isn't hardcoded anymore but now uses an event type.
The page now uses the Roboto Font instead of the default font, which helps with readability.
Matrix Art cmes with a basic Text Logo now
Links to the profile page are now easier to notice
Blurhashes are now supported for images
The repo now has the Apache-2 License applied
Dependencies have been pinned
Various small design improvements were made
Images now have size hints so the page doesnt jump around as much when loading
Lots and lots of metadata was added to each page for SEO
Check it out at https://art.midnightthoughts.space
Check the Code out at https://github.com/MTRNord/matrix-art
Or join the Chat at #matrix-art:nordgedanken.dev
Planned for this week is to add registration and usage of external profiles. As soon as that works I am also going to make uploading images work. So with a bit of luck in the next TWIM anyone is able to post their images :)
Hi TWIM. I published a long-planned, long-in-the-making, long-winded report of how I containerised my matrix-synapse homeserver and its PostgreSQL database in order to get ahead of the application dependency deprecations. This was something I couldn't find for myself, so I put this together to help anyone else who might need it. (And it does contain a TL;DR section!)
I’ve created [blog post|(https://helderferreira.io/matrix-well-known-with-cloudflare/) explaining the way to host the static well-known files using cloudflare workers gaining speed and stability
During the holidays I recorded a podcast about Matrix and Element with AnDaolVras/La Cantine Brestoise, which is a French non-profit organising tech events and operating a coworking space in Brest, France. The episode (in French, sorry!) came out on Monday and can be found on their website, on YouTube as well as on most podcast platforms 🙂
Welcome all for the first Synapse release of 2022: Synapse 1.50!
Note that, as per our platform dependency deprecation policy, Synapse no longer supports Python 3.6 and PostgreSQL 9.6 as of this version. As a result, we have also stopped shipping Debian packages for Ubuntu 18.04 LTS (Bionic Beaver), as it ships with Python 3.6.
As a reminder, please note that Ubuntu 21.04 (Hirsute Hippo) reaches its own end of life on January 20, 2022. Past this date we will stop producing new packages for Ubuntu 21.04.
Application services (sometimes called "appservices"), are privileged processes that can interact with a Matrix homeserver in a way a normal user cannot. This is especially useful for bridges, as it allows them to register and puppet multiple users on the homeserver to replicate activity from other platforms.
One of the main shortcomings of application services currently is that they do not support end-to-end encryption. This means that messages sent through a bridge are never encrypted and always visible by the homeserver.
We've recently started work to tackle this issue in the form of MSC3202. A first part of implementing this MSC (allowing application services to masquerade as specific devices) has landed in this release of Synapse; work is still ongoing towards a full implementation, so watch this space!
While working on this release, we identified a long-standing bug that could prevent Synapse from sending device lists update over federation if the server had a high number of active users and/or users with a lot of devices connected to their account.
This bug was introduced back in Synapse 1.0.0, and meant that the homeserver would miss some device list updates when communicating with other homeservers if the amount of updates to send was too high. In practice, this means users on remote homeservers could see outdated device information for other users (including outdated device verification statuses).
Synapse 1.50 includes a fix to this bug. This should contribute towards making the propagation of device list updates more reliable.
This release introduces support for MSC3419, which allows guest users to send arbitrary state events into a room. This will be especially useful to the ongoing work on group VoIP calls, which involves having users send new state events into the room to signal their participation in a call.
We've also stabilised identifiers for cross-signing and fallback keys now that MSC1756 and MSC2732 have been merged into the Matrix spec.
On the documentation side of things, the page on setting up and configuring a TURN server has been updated to feature instructions on how to deal with NATs. This is a much welcome addition as configuring TURN is something a lot of Synapse admins struggle with!
Please see the Synapse release notes for a complete list of changes in this release.
Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including Dirk Klimpel, Donny Johnson and AndrewFerr.
Note: An issue preventing client logins (#11763) was identified immediately following the release of Synapse 1.50.0. We released Synapse 1.50.1 the same day with a fix for this issue.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.
Hot on the heels (relatively speaking) of v1.1 of the Spec being released in November, v1.2 is now on the horizon! As a reminder, we're working towards quarterly releases of the spec going forwards - no hard dates yet though.
While this is certainly an improvement in speed, when it comes to writing software and updating implementations: quarterly spec updates may actually seem too fast. This is OK; implementations of the spec are not expected to update as soon as a new spec release is published. Rather, it is more realistic to expect that the ecosystem updates gradually over the course of the next few months/year after the release.
Happy Friday everyone! Today the Synapse team has released Synapse 1.50.0rc2. It fixes a particularly nasty federation-breaking regression that crept in during 1.50.0rc1. If you're currently running 1.50.0rc1, we implore you to update to 1.50.0rc2 as soon as possible!
In other news, a reminder that as of Synapse 1.50.0rc1, we have ended support for Python 3.6 and PostgreSQL 9.6 as per our dependency deprecation policy, as upstream has marked them as end-of-life.
We'll let the community test 1.50.0rc2 over the weekend to ensure that no other regressions have emerged (please test if you can)! If not, expect Synapse 1.50 to land sometime early next week.
Support CHGHOST caps (prevents leave+join on host change) ✅
Fix owner auto-registration (regression) 🐛
Upgrade to Mautrix 0.14 ⬆️
Sending RELAYMSGs have been requested for a while so now we do support that if the cap is added to request list. This makes plumbs on networks that support it work nicer. Receiving RELAYMSGs has not been implemented yet so they show up as external like before.
If you prefer keeping your IRC rooms clean without any IRC noise (mode changes etc.) you can now use a new FORWARD command in the network room to make all such events happen in the network room instead. This affects in-room commands as well.
ZNC users can now enjoy messages coming from your own clone in IRC rooms (including PMs!) if you wish so by keeping the znc.in/self-message cap enabled. This isn't extremely well tested yet but feedback welcome.
Not a breaking change but the caps support will cause connections to networks that ignore CAP requests take a few seconds longer unless you remove all the default caps for said network and it will never try requesting them again.
Mautrix 0.14 upgrade bumps the minimum version as well so packages beware.
We rewrote the whole settings page (as a stepping stone to 100% QML). Please test it and complain about everything I broke! We also tried to organize it a bit better so that similar settings are grouped together and tried to use the right controls for the right things. Best case you didn't notice anything. ;-)
And we also now show which profile a notification is for on KDE. I don't think other DEs have such a feature, so those will just show the generic Nheko as always.
We're about to release 0.2.23 after somewhat of a release hiatus. We've been working on two fronts:
Get the SDK out! Hydrogen was always meant to be easy to embedded, reuse in parts, and make customized versions of. But until now, actually doing that was quite challenging. Now that the SDK is out, you can use the hydrogen-view-sdk package in your projects and use parts of Hydrogen in your application. It's still early days, and we're still working out what symbols should be exported (finding a balance between APIs we want to support and utility), etc. Expect updates in the coming weeks. We're also not yet promising API stability for now, some APIs will very likely still change. Once we hit 1.0, we won't change things from underneath you anymore without increasing the major version.
Rich reply previews: as part of providing minimal support for threads (representing them as replies), we're switching from using the embedded reply fallback and actually looking up the replied-to message. One benefit is that reply previews will now updated when they are redacted (and edited, once we support that :).
Also fix some minor other things, like loading images when they are only partially visible, and a very basic location tile.
Issue with thread panel rendering timeline has been fixed
Reviewing plans for backward compatibility with clients that don’t support threaded messages
Polls
iOS and Android starting on the next phase of development, look out for poll editing and other exciting changes! You can enable polls in the Settings, under Labs.
You can now opt in to sending anonymous analytics data in the user settings on all platforms. Please enable it in the “Security & Privacy” menu, under “Analytics” to help us understand better how Element apps are used.
Mobile app users will see an opt-in screen on first startup on Android in 1.3.14 and iOS in 1.6.12
Maximised widgets merged into the release candidate and on track for the next release (see the Beyond Chat section for more info on this feature)
Fix code blocks being wrongly wrapped causing the line numbers to misalign, this was a regression in 1.9.8
Fixed ability to edit horizontal rules in markdown
Fixed some edge cases around Spaces not updating properly causing rooms to show up in the wrong Space
Add ability to cancel an outbound message during its encryption phase
Fix wrongly sending typing indicator when restoring a draft when changing room
Replace kick terminology with remove to be more inclusive
In labs (you can enable labs in Settings on develop.element.io or on Nightly)
Polishing bubbles layout
Upgrade to Electron 16 for Element Nightly - we are expecting this to help resolve some of the issues caused by Electron. Intending to release to production in 1.9.10 (two weeks away)
The maximised widgets feature in Element Web that Timo K. has been working on is now available for everyone on develop, and it's on track for the next web release. 🚀 It's a great match for widget-based collaborative editors, dashboards, games, and more. The widget becomes the focus of the room. You can optionally show chat from the room in a side panel, allowing easy discussion of the document / dashboard / game.
To try it out, add a widget to a room in the usual way, then look for the new "maximise" button in the room info panel's widget section. Please let us know in #beyond-chat:matrix.org or in issues if you have any feedback. 😄
Beeper is a universal chat app built on top of Matrix. We've created 12+ open source Matrix bridges and integrated them into an easy to use all-in-one service which does not require setting up your own homeserver. You can learn more at beeper.com.
Our team is growing! We’re now at 25 people, all remote around the world. Recent additions include hifi (creator of Heisenbridge) and Finn (creator of signald).
We are hiring many full-time remote roles including:
Bridge developers
iOS engineers
Product designers
Learn more here and apply through that site, or message @eric:beeper.com
Beeper Desktop
We recently released a Beeper Desktop update with a new room list and a ton of UI improvements. Check out the video below!
Made it easier to triage your inbox by moving unread dots to left
Made the list of connected bridges more compact
New link previews
Loads of bug fixes including improvements to scrolling
Most importantly, we launched a cloud Mac service that allows you to use iMessage with Beeper if you do not have access to an always-on Mac computer. Included with your Beeper subscription!
Voice messages are bridged in native format in both directions for all Beeper bridges
Signal bridge now supports disappearing messages
All Instagram message types are now bridged
Added support for Telegram message reactions
WhatsApp bridge now backfills 3 months worth of chats
Beeper Android
Coming soon: A recent addition to our Android team has added encrypted search by massaging seshat into the app. Expect an upstream PR for this into Element Android as well!
Trixnity version 1.0.0 is out! I decided to make it 1.0.0 not because it is out of beta, but because it has most features you need to build a usable client.
That means: cross signing has landed into Trixnity! The next big features will be room key backup and push notification.
Here is the changelog:
fully support cross signing
load members of rooms in an async way (you don't need to catch errors anymore)
reactive displayname and avatar url of the logged-in user
reactive avatar url for rooms
reactive is-direct-room state for rooms
better Android support for thumbnails
make part of device keys API public
merge SecureStore into Store (encrypt your database if you want to keep secrets secure)
move trixnity-client-api model-classes into separate module to use them e.g. for a matrix server implementation (thanks to @NicolasJouanin)
fixed long standing bug of wrong room name calculation
account data events are handled as if they have a key (like state events) to bypass inconsistency in the spec (maybe this will lead into a MSC)
Just cut a new release of the Ruby Matrix SDK, with 2.5.0 there's some preliminary support for Matrix 1.1 and the client/v3 API, the information for room knocking is exposed properly, some threading issues have had workarounds applied, and a bunch of fixes have been applied.
Currently running my own ping bot with it, I'm doing the GitHub (and hopefully also GitLab at some point) releasetracker bot, I've got my definitely not suitable for production MatrixFS, there's a notification module for TheForeman which we use at the university. And I've got some MSC hacks as well, like the MSC2108 testbed.
I just open sourced a library called Matrix-CRDT: https://github.com/yousefED/matrix-crdt - feedback very welcome! It allows you to use Matrix as a backend for decentralized, local-first collaborative apps. Above you see a collaborative rich text editor (like Google Docs) powered by Matrix!
You can try the Rich text editor here: https://bup9l.csb.app (see also the links in the repo, e.g. there is also a collaborative todo-list example)
If you have a distributed data structure and an algorithm that ensures all participants end up with the same result when their actions are combined, then that's effectively a CRDT as well. For Matrix, there a few research papers like https://arxiv.org/abs/2011.06488 which examine the CRDT-like properties.
First stable release of the gh-bot which is a simple webhook to Matrix bot made in Python.
As of now, the bot supports webhooks from Github, Gitlab and Gitea (although support for this one is very light).
Some other projects might do that way better than this bot but this is a learning project, to learn step by step how client interact with servers.
The next step would be to add a CRT.sh integration to get notified when a certificate is issued for a certain range of domains (which is something we want since we saw an IRC bot do the same).
Matrix-Art is a new social network prototype on Matrix.
It is a direct Devianart style clone. It currently has a focus on only images but is going to get extended to other media types eventually.
I am doing this as a toy project, so it may sometimes have slow progress. The goal is covering the main functions Devianart provides (Posting, Sharing, Profiles, Following, Collections, Comments) as well as integrations to other social networks on matrix using MSC3639.
A lot of things are currently still missing, but I am trying to get uploading working next :)
An instance is hosted at https://art.midnightthoughts.space/ and the code is at https://github.com/MTRNord/matrix-art
Currently, the interface is limited to viewing basic data from a matrix room. In the future, there are plans to have an open registration as well as login with any account to use it. :)
Note however that I suggest against using your personal account just yet, even though the login works, as it may have unexpected room joins at this time. This is known and expected, but not production ready.
Also, at this time, there are no plans of e2ee support. This may get added after the main features are finished.
I just wrote an article about my experience using Matrix, and the question... is it worth posting in the news? The article is written in Russian and published on Habr... and since it's pretty big I'm not sure about translating it to English 😅
Here is the link: https://habr.com/ru/post/599777/ - it also mentions TWIM.
What a pleasure it is to be back with the community for the new year! This week Erlend is detailing what Commune is, and I'm flabbergasted by how well thought-through the project is.
This week in Matrix should be called Three Weeks in Matrix, since there hasn't been TWIM updates during the holiday season. Nico has published a first and second communitwim while I was away. All the news reports since the last official TWIM still made it to the post you're currently reading!
Your regular spec person, anoa, is out today so you're stuck with me, not-anoa. This time I got the script to work though 😇
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week the last little bit, we've been working on getting MSCs merged into the formal spec itself in preparation for v1.2 this quarter. We expect the release to happen quite soon and to contain Spaces, room versions 8 and 9, and refresh tokens (no spec PR yet) in addition to all the other wonderful stuff which has landed.
The merged MSCs can be seen over at https://spec.matrix.org/unstable/ where they're queued up for the next release. Look out for Added in v1.2 labels throughout the spec, denoting what is new and exciting.
The random MSC this week is MSC2192: Inline widgets (I promise it was random, after a few re-rolls). Inline widgets are a concept that allows for rich functionality in the timeline without needing to necessarily specify an explicit event type. For example, video embeds, minigames, etc could all be represented by inline widgets instead of dedicated event types.
The MSC needs updating to handle MSC1767: Extensible Events (and friends), but once there it could be a very powerful bit of functionality, with free fallback thanks to Extensible Events.
Sliding sync (aka sync v3) is described in MSC3575 - it's still very early days for the proposal which means a lot can change: and YOU can be a part of that. Please take a look at the MSC and provide any and all feedback, be it on the names of keys, format of JSON objects, or the kinds of operations that can be performed. I'm particularly interested in feedback from client developers who have complex room list sort orders or room list filtering requirements and bot/bridge developers who typically don't have a visible room list UI. Any and all feedback at this stage is welcome, visit #sliding-sync:matrix.org to join the discussion.
We're back! New year, new release candidate: we published Synapse 1.50.0rc1 today, marking the end of life for our Python 3.6 and PostgreSQL 9.6 support. We've also extracted some shared utilities into their own package, matrix-common, which is used by Synapse, Sygnal, and Sydent.
We'll talk more about what's in 1.50 next week when it formally releases.
v1.2.9v1.2.10 got released as a maintenance update. With early support for Matrix 1.1, S3 storage classes, and blurhash fixes it's worth the upgrade though there are other goodies - check out the changelog, and report bugs to the issue tracker 🙂
And as a end-of-year update, my Helm Charts have gotten updated yet again. With element-web ending up on 1.9.8, matrix-synapse on 1.49.2 (and .1 before that), and matrix-media-repo on 1.2.10
The first iteration of a Dendrite Helm Chart has been released under the k8s-at-home charts repository. It currently supports a full monolithic deployment and requires minimal configuration to get up and running (just need to generate a matrix key and mount it as per the instructions).
If polylith is your thing then I recommend the chart by S7evinK for now, although it is likely these charts will merge in the future.
An add-on for the matrix-appservice-webhooks bridge. Webhooks are essentially web interfaces for applications to "push" data to. The bridge can receive messages in a certain format, which is nice if the notifying app can be configured. Often it cannot.
Advanced Templating! It is now possible to set format and msgtype based on arbitrary values from the webhook JSON via Jinja2. Shoutout to qg for suggesting this and giving their feedback!
compatibility with matrix-appservice-webhooks forks that read avatarUrl instead of avatar_url
allow mxc:// URL avatars
improvements to the GUI, including automatically resizing text areas and msc:// avatar URL preview
improvements to templates/examples including making use of above features
Howdy folks, it is the time of the year where everyone scarpers! Anyway, perfect time to announce that matrix-hookshot has gotten it's first major release!. 1.0.0 is here!
For those not in the know, the hookshot bridge is used to bridge GitHub, GitLab, JIRA and Generic Webhooks into Matrix rooms. It doesn't just bridge into existing rooms, but can also spawn dynamic rooms based on aliases, send you your notifications in a DM and do lots of other wonderful things!
The notable changes from the 0.1.0 release are:
The bridge has now been renamed from matrix-github to matrix-hookshot.
Now supports JIRA and Generic Webhooks in addition to GitHub and GitLab.
You can get involved and start playing with it by checking out the release here
And that will be my last TWIM entry of the year. Have a good one and stay safe all 🐶
That was in 2021! Half-Shot is back in 2022 with more news
Hey folks! It was only last year we released the 1.0 release of hookshot, after many years of work to get it that far. I'm happy to announce that this week we've got another release. There are a number of buxfixes and improved documentation pieces landing (special shoutout to HarHarLinks for ensuring the docs are competent). The highlights are as follows:
Added support for Figma webhooks. this also means the archival of my old project matrix-figma
Support GitLab wiki page change events.
Added a new script validate-config which allows you to check your config file for simple errors. Handy for people writing ansible roles!
Add support for a html key on generic webhooks to set the HTML content of a Matrix message.
The project can be found over at https://github.com/Half-Shot/matrix-hookshot/, with pretty pretty docs at https://half-shot.github.io/matrix-hookshot. We're also in #hookshot:half-shot.uk if you prefer using Matrix to learn about these things!
UNPLUMB network command to force unplumbing without being in the room
Proper SASL external with CertFP with mechanism override option (see notes)
Disconnect and cleanup from networks that have no rooms open ♻️
Reply (and reject) DM requests to ghosts with QUERY command ↪️
Try to keep IRC users in the room at all costs if they are on the IRC channel
Fix assumption of all IRC replies to have arguments
Prevent accidental namespace changes to cause mayhem
Finally convert from homegrown Matrix API stuff to Mautrix
Bump Mautrix requirement to 0.12=>0.14
Conduit support was broken in 1.8.x but fixed again in 1.9.0, sorry
Finally there's network level spaces support with a new SPACE command. This creates a new bridge controlled space for the network and automatically manages rooms in and out. There's an issue/feature with Element that all rooms that have been converted to DMs with /converttodm will appear in all bridge spaces. The workaround is to convert them back to regular rooms.
CertFP SASL has been updated to do SASL external flow by default. If you are upgrading and have used CertFP with OFTC you need to run SASL --mechanism=none for it to connect again.
Abandoned networks where the user has left all rooms including the network room will now automatically disconnect and cleanup. This is more in line what people would expect and prevents idle connections from hanging around.
This is a small bug fix release. If you reported an issue, there is a 15% chance it is fixed now! This release also supports pinned messages, although those will only show up after someone changed the pinned messages in a room currently (we didn't want to force a full resync just for such a small feature). The spaces list is also now nested, Nheko offers you quick access to your recently used reactions and Nheko will show you your direct chats in the sidebar. Apart from that there are quite a few bugfixes and smaller improvements, you can find the full changelog and downloads here: https://github.com/Nheko-Reborn/nheko/releases/tag/v0.9.1
Thank you everyone, who helped shape this release!
Nheko now is a lot more efficient. We now use one Threadpool instead of 3, got rid of more than 60% of the allocations when scrolling through messages, layout half as much content when scrolling, blurhashes decode in 10% of the time and jdenticons allocate ~10% as much temporary buffers. We also deleted around 1000 lines of unused code. Tooltips also shouldn't steal the mouse focus when scrolling anymore, which could lead to sudden stalls when scrolling.
Additionally edits now replace existing notifications, tastytea added a manpage and fixed blurry or incorrectly sized custom emoji, you can now send custom emotes via the inline completer using ~ and completers now show a scrollable list with more Elements than before. Advanced users can also now opt into an insecure client side secrets storage via a hidden setting.
We're baaaaack! Anyway, last week Drake just translated all of Nheko into Spanish, you can now zoom in and pan in the image viewer, emojis shouldn't split up into their segments anymore and Nheko should always be sending the qualified version of it (according the to the unicode test files, not by just appending FE0F). Blurhashes should be even faster still and we now have support for running the call event loop on macOS and Windows (although call support is still disabled there for now).
We are also now working to restructure our README. It has a lot of outdated stuff in it and you really want to see screenshots pretty early in the README! You can sneak a peak here: https://nheko.im/nheko-reborn/nheko/-/tree/README_updates
This week the Threads team has focussed their efforts on improving some of the smaller details and pesky bugs we’ve found in our first rounds of testing.
If you’d like to help us test Threads we’ll be asking for help in the Community Testing room over the next few weeks. Join#element-community-testing:matrix.org to stay in the loop.
Polls
Polls is available in Labs on all clients! While we know there are some minor improvements to be made, we’re proud of where we are and would love for you to start using it!
Community Testing
Our next session will be on Wednesday, 12th January at 16:00 UTC (17:00 CET). We will be testing the new release candidates on all three platforms! Join us at#element-community-testing:matrix.org
Work continues on the integration of PostHog analytics. With your explicit permissions we’ll receive anonymous usage data that will allow us to understand the areas of Element that are helpful (or not). This info will help fuel things like our Information Architecture project.
In labs (you can enable labs in settings on develop.element.io or on Nightly)
Information Architecture improvements are still being worked on - try it out by enabling things like the new Spotlight search and Breadcrumbs.
Message Bubbles are improving; We’ve been hard at work preparing them for a release in the coming weeks.
During the Christmas break we increase the speed of the app launch by 7x! Once stabilised this update will land with you.
Analytics changes have been merged into the RC and opt-in will be available soon.
In development:
More Spaces improvements are underway; You’ll soon be able to add rooms to Spaces, update room settings, and have room interactions on the ‘long-press’.
Improved onboarding! We’re hoping to make the first few steps in Element easy by simplifying some of the first tasks users take in the app, including signing up.
We’ve started building Message Bubbles on iOS. This will help to distinguish the messages you send from the messages you receive.
We now implement MSC2574! This MSC proposes a standard format for marking up resources (PDFs, other document formats, audiovisual media, websites, geospatial data...) using Matrix.
Room avatars are now displayed in the welcome view for PDF rooms
There's now a UI for using spaces to manage and share PDF collections
Reactions now work like element-web: click to mirror an existing reaction or redact your previous reaction
Next, I'm aiming to implement MSC3592, which proposes a standard format for basic markup on PDFs. Stay tuned! And if you're interested in learning more, come visit #opentower:matrix.org.
Matrix Wrench is a web client to tweak Matrix rooms. After formerly calling it Matrix Navigator or Matrix Screwdriver, I finally settled on the name Matrix Wrench. ¯\_(ツ)_/¯
Version v0.2.0 comes with a Network Log which displays curl equivalents for all network requests. It also allows to easily add and remove room aliases.
https://gitlab.com/jaller94/matrix-wrench/
Cactus Comments is a comment system for the open web, built on Matrix.
We released version v0.11.0 of the client!
The client has been relicensed to LGPL. This means that you can now use Cactus Comments in non-GPL compatible projects.
But mainly, this release brings a bunch of CSS improvements: introducing automatic dark mode, and making it easier to change colors with CSS variables.
Here's the changelog:
Relicense from GPLv3 to LGPLv3.
Rewrite large parts of the stylesheet to use flexbox.
Introduce CSS variables to the stylesheet.
.dark and .light CSS classes with default values for dark/light mode.
Bugfix: "View More" button no longer blinks when auto-refreshing short comment sections.
matrix-bot-sdk has had a v0.6.0-beta.3 release with beta support for crypto! It even includes documentation!
The crypto is considered beta quality at the moment: good enough to use for somewhat unimportant bots, but not fully recommended for production just yet. With that being said, I'm interested in bugs you run into - please use the issue tracker if you run into crypto not working.
Tutorials for the crypto setup are at https://turt2live.github.io/matrix-bot-sdk/tutorial-encryption.html
Note for appservice support to work then you'll need a Synapse with these PRs enabled (may require manual merge too):
https://github.com/matrix-org/synapse/pull/11538
https://github.com/matrix-org/synapse/pull/11617
https://github.com/matrix-org/synapse/pull/11215
For non-linux platforms, the rust-sdk will try to build itself which means you might need a working Rust stack. The Rust SDK repo itself has more information:
Bots and appservices don't automatically support encryption, but adding encryption should be easy. The Rust SDK dependency is required in either case, sorry.
Complement has seem some updates this week in the test output that is produced onto the CLI. Details on how to add this to your CI process are contained in the README. Here's the difference when viewed using Github Actions:
Polyjuice Newt, the newest addition to the Polyjuice project, is an Elixir binding for vodozemac, the new Olm/Megolm implementation in Rust. At the time of writing, Polyjuice Newt supports encryption and decryption using Olm and Megolm, but by the time you read this next year, it may also support the SAS verification functions and pickling/unpickling.
I don't think we've previously had any Nix/NixOS/nixpkgs related entries in TWIM, so I'll start ^^
The current unstable channel has extended its Matrix ecosystem support to also include Heisenbridge and Conduit packages and modules. This makes it super easy to deploy any of those services: For example, my configuration for Heisenbridge is 21 lines long, and Conduit is only 11 lines. You can browse the available configuration options online: services.matrix-conduit, services.heisenbridge (note that some of them are freeform and simply forward to the upstream configuration).
For those that are not into NixOS, a module is the code that turns the declarative configuration files into your running system setup. As an example, if you enable services.heisenbridge the following things are done for you:
Create a new heisenbridge user and group for the service
Create and manage the registration file for the homeserver (i.e. automatically regenerate it after the configuration changed)
Create a systemd service that runs the heisenbridge command with the requested bridge configuration. The unit also sets a few systemd security hardening options.
Jip J. Dekker did the initial work of adding Dendrite support to the playbook back in January 2021. Lots of work (and time) later, Dendrite support is finally ready for testing.
The playbook was previously quite Synapse-centric, but can now accommodate multiple homeserver implementations, with Synapse still remaining the default.
A new Matrix guide has come into town: https://joinmatrix.org
In the hopes to expand Matrix's reach to the non-technical population, this guide is intended to give quick directions on how to use Matrix, as well as clear comparison between Matrix and other dominant platforms.
The pages are available on GitHub. Open to contributions!
If you’re reading this - congratulations; you made it through another year :) Every winter we sit down and review Matrix’s progress over the last twelve months, and look forward to the next - for it’s all too easy to get lost in the day-to-day development and fail to realise how much the overall project is evolving, especially when it’s one as large and ambitious as Matrix!
Looking back at 2021, it’s unbelievable how much stuff has been going on in the core team (as you can tell by the length of this post - sorry!). There’s been a really interesting mix of activity too - between massive improvements to the core functionality and baseline features that Matrix provides, and also major breakthroughs on next generation work. But first, let’s check out what’s been happening in the wider ecosystem…
