This Week in Matrix 2018-09-14

Dimension

Update (this got lost in the original post; sorry Travis!): Dimension received a security update – if you run your own Dimension instance it is strongly recommended you update right away. Telegram bridge support in Dimension is underway, with more updates expected for next week in Matrix.

Clients

Fluffychat

It’s been some months since we checked in with FluffyChat. If you’re a Ubuntu Touch user, or have a device running it, you should see the progress that has been made recently on this Matrix client. Collected changelog 0.5.0 to latest (0.5.4):

  • Search chats
  • Chat avatars
  • Search users in chats
  • Security & Privacy settings:
  1. Disable typing notifications
  2. Auto-accept invitations
  • New message status:
  1. Sending: Activity indicator
  2. Sent: Little cloud
  3. Received: Tick
  4. Seen by someone: Usericon
  • Display stickers
  • Minor UI improvements
  • FluffyChat now automatically opens the link to the matrix.org consens
  • Updated translations

Seaglass

Neil has been keeping up the pace with Seaglass development:

Seaglass has had a substantial rewrite to the room cache to help improve reliability and reduce crashes, better thumbnail behaviour on inline images, various tiny visual tweaks, in-window blending, support for encryption key sharing requests for E2E rooms.

Rendering performance has been massively increased (if you ignore the occasional bug). Resizing the window shouldn’t be so slow anymore and a lot of avatar image operations are no longer repeated unnecessarily

Other than that this week has mostly featured lots and lots of bug fixes, hopefully lots of crashes fixed.

Screenshot below shows the new E2E UI:

Quaternion

When not escaping typhoons, kitsune has found some time to continue work on Quaternion:

Quaternion‘s master branch is alive again – it’s been prone to crashes in the last two weeks, now it shouldn’t. Feel free to try the new room list organised by tag!

SimpleMatrix

MTRNord has been working on SimpleMatrix:

SimpleMatrix now supports Basic messages sending (with Commonmark) and basic receiving of messages.

miniVector

Marcus has re-packaged miniVector for F-Droid:

There’s now a second matrix client available in F-Droid: https://f-droid.org/packages/com.lavadip.miniVector/

This is a fork of Riot Android done by hrjet, f-droid release done by me. It’s removing mostly jitsi group call functionality and some other smaller stuff. In doing so it manages to require far less permissions and is also only 12 MB in size instead of riots 20 MB.

Matrique

Black Hat:

Matrique gained alpha support for multiple accounts

This is thanks to leaning on libqmatrixclient’s native multiple account support!

Riot Web 0.16.4 released

This is pretty much a maintenance release – fixing the DM avatar regression that crept in with 0.16.3, adding better support for warning users when their client hasn’t yet synced with the server, and the final bits of work needed before we can turn on membership Lazy Loading in the upcoming Riot 0.17.

Full changelogs as always are split over the three projects which make up Riot/Web:

SDKs and Libraries

libQMatrixClient ecosystem

As you may know, Matrique, led by Black Hat, and Quaternion, led by kitsune, are both projects build using libQMatrixClient, a Qt5 library from kitsune designed for writing Matrix clients. While kitsune has been working on the library for some time, Black Hat has also now
started making contributions:

libQMatrixClient now has a pkg-config file to further ease clients building on Linux systems, as well as more convenient API to track read markers if all users, not just of the local one.

matrix-js-sdk v0.11.0 released

This release contains support for lazy loading room members, and also some breaking changes relating to startClient().

  • Support for lazy loading members. This should improve performance for users who joined big rooms a lot. Pass to lazyLoadMembers = true option when calling startClient.
  • MatrixClient::startClient now returns a Promise. No method should be called on the client before that promise resolves. Before this method didn’t return anything.
  • A new CATCHUP sync state, emitted by MatrixClient#"sync" and returned by MatrixClient::getSyncState(), when doing initial sync after the ERROR state. See MatrixClient documentation for details.
  • RoomState::maySendEvent('m.room.message', userId) & RoomState::maySendMessage(userId) do not check the membership of the user anymore, only the power level. To check if the syncing user is allowed to write in a room, use Room::maySendMessage() as RoomState is not always aware of the syncing user’s membership anymore, in case lazy loading of members is enabled.

Synapse

Synapse 0.33.4 was released, with a whole host of bug fixes, some enhancements to resource usage management and a bunch of internal changes in readiness for room member state lazy loading and our ongoing port to Python 3.

Meanwhile, Python 3 support for monolithic (non-worker) Synapses has finally landed on the develop branch, thanks to massive work from hawkowl and notafile – if you want to help us test and flush out any remaining byte/utf8 style errors, please create a virtualenv for python 3.6 or 3.5 (twisted doesn’t support 3.7 yet) and point the develop branch of Synapse at it, tail the logs for ERRORs and report them via Github if/when you see them.  In practice it seems pretty stable though, and noticeably reduces RAM and speeds things up thanks to improved GC and general performance work in Python.

We’ve also discovered that jemalloc works *very* well at improving RAM usage on Python 2 under Linux (we haven’t tried it on Python 3 yet) by providing a more fragmentation-resistent malloc implementation; if you are having problems with your Synapse RAM spiking up we recommend giving it a go.  All of the Matrix.org server is using it now.

Also, lots of ops work this week; Erik has prototyped a new storage strategy for state groups which shrinks storage requirements by 10x, we’ll be applying this shortly to Matrix.org otherwise we’re going to run out of disk space.  There was also a regression on Synapse develop on federation, where outbound requests would get stuck and never retry – this impacted the matrix.org server badly over the course of the week, but as of Friday night we have a workaround in place.  We’re not aware of it affecting anyone other than the matrix.org deployment (and we haven’t got to the root cause yet).

Construct homeserver progress

This week:

Added notification counts which show up in Riot now, and expanded support for g++-7 and 8 instead of just g++-6. Construct repository is found at: https://github.com/matrix-construct/construct.

IRC Connection Tracker

Half-Shot is continuing to work on the project to split out IRC connection management from the IRC bridge, letting the bridge be restarted without interrupting IRC connections!

The project is going quite well, and is going to be used on matrix.org once production ready which will really speed up upgrades and give us near zero downtime indifferent to the size of the bridge.

At the moment the project has the ability to spin up and maintain connections, however the connections are not supporting IRC fully yet as there are bits to do on the parsing and maintaining state side. There is also work on a top-like tool to visualise and control the service outside of the bridge so we can quickly handle any oddities if they come up. Finally, it allows you to hot reload the configuration without dropping existing connections!

On the work done to support this on matrix-appservice-bridge, there is basic support for stating connections on the bridge but it’s in early stages at the moment.

Spec

Travis has been tidying up loose bits on the Matrix spec this week:

In practice, finalising the S2S API is now blocked on proving the implementation on Synapse; work on this will resume next week and then we’ll document the end result and ship the r0 at last.  Timings are going to be completely determined by available manpower and what level of ops distractions we face (c.f. the Synapse section above…).  Whilst we’re waiting for the final S2S details to get hashed out, Travis is going to be helping on Riot dev, to try to stop stuff like this, as there’s no point in having the platonic ideal of a perfect spec if actual users are unable to benefit from it.

#matrix-dev

#matrix-dev:matrix.org was reborn as a new room a couple of weeks ago to flush out old corrupted events, but maybe not everyone knows. Come join #matrix-dev:matrix.org, it’s a starting point for all developers looking to build on the platform.  We’re also rebuilding #test:matrix.org and #riot:matrix.org, although once we ship the new state resolution

A sneak peek at Modular…

 

 

Finally, there’s been a massive amount of work on the New Vector side of things to soft-launch Modular – a paid hosting platform for Matrix servers (and, in future, paid integrations).  At this point we’re looking for early adopters who want a dedicated Riot+Synapse for communities or companies of 50 or more users – but don’t want to have to run it themselves.  Modular takes the homeserver hosting we’ve already been providing for Status, TADHack and others, and turns it into a mass-market product.  The pricing for early adopters is over 5x cheaper than Slack, so if you’ve been dying to have a reliable, fast and expertly maintained homeserver without any of the headaches of admining one yourself, please head over to https://modular.im and give it a whirl and let us know how it goes!  This is also a great way to support Matrix development in general, as money from Modular will directly keep the core Matrix team funded to work on Matrix.  Once we’re happy with the soft-launch and have incorporated any feedback we’ll start yelling about it as loud as we can :)

Matrix Live

We’ve had a bit of an accidental hiatus on Matrix Live thanks to getting submerged all the different project endgames happening atm (spec releases, lazy loading, Modular, Riot redesign etc), and for the last few Fridays we’ve got to midnight and beyond with too much still on the todo list to justify recording a video.  But to avoid completely falling behind, here’s a slightly exhausted Saturday morning update instead (warning: Matthew is not a morning person).

Recent matrix.org website improvements

Recently I’ve been working to improve some of the content on the matrix.org website.

Firstly the FAQ now has updated content and a more presentable menu.

We have a Guides Index, which includes a clarified guide list, plus links to off-site contributions from the community. It’s possible to click “recommend” on these items if you’ve had a good experience with them. If you have documentation or guides you’d like to see added to the list, contact me on Matrix or make a pull request on the site repo.

Finally, as part of a programme to improve visibility on projects in the Matrix ecosystem, we are introducing the “Matrix Clients Matrix“. This is a list of some of the most popular current Matrix clients in the ecosystem today, and should shed some light on current feature statuses! The list is not exhaustive, and if you would like to see your client project included, please contact me at the same address, or come chat in the Matrix Client Developers community room. Pretty green Features grid:

This Week in Matrix 2018-08-31

The Spec

As many know, we’ve been aiming for the end of August to cut the first ever stable releases of the remaining APIs in the spec which have up to now been marked unstable – i.e. providing a snapshot of the spec which correctly matches the reference implementations (other than implementation bugs) and which can be used in isolation to develop production grade implementations of clients, servers, etc without need to reference any other implementations. There’s been a massive sprint to pull this together, and as of the time of writing there are still PRs and commits landing every few minutes.  We’ll post a full update on our progress on Monday; meanwhile you can see a sneak peek over at the August 2018 r0 project board.

Spec work has completely precluded any other backend dev this week.

Half-Shot, gone but not really gone

This week we say farewell to Half-Shot, who has been working fulltime on bridges over the summer. He has managed the matrix.org bridges largely single-handedly, with a big focus on the often-volatile IRC bridge(s).

Bridges

matrix-appservice-irc 0.11.0

It’s a big deal, and it’s rolling out to IRC networks this week. Half-Shot released matrix-appservice-irc 0.11.0, with the following included:

New features & improvements:

  • Cache modes internally
  • Replace nicks with user pill mentions
  • Kick users if we fail to create an IRC client for them on join (aka ILINE kicks)
  • SASL support
  • Add err_nononreg so we can announce PMs that failed
  • Formatting of replies

Bug Fixes:

  • Fix invalidchar nick
  • Don’t answer any msgtypes other than text in an admin room.
  • Fix provisoner leaving users on unlink
  • Fixed a bug where content of events the bridge hadn’t cached were not being used in replies.
  • We were calling authedRequest but the request was not mocked out.
  • There was a bug involving intents in m-a-b so it was bumped

Metrics:

  • Metrics for MatrixHandler – Iline Kicks
  • Idle connection metrics
  • QueuePool.waitingItems should use it’s internal queue size

Misc:

  • Section out tests, linting and coverage into seperate stages for Travis

WhatsApp

tulir has been working on mautrix-whatsapp bridge.

now bridges a lot more stuff, such as formatting, media and replies. I’m also almost done with desegregating users so that Matrix users join the same group chat portals rather than everyone having their own portal to the same chat

Zulip chat, bridged by Zulip

Matthew discovered there is a Matrix-Zulip bridge on the Zulip side. So if you’re running a Zulip server (for some reason), and want to bridge with Matrix check out the integration docs here.

IRC Connection Tracker

Half-Shot created a new component to enhance the reliability of IRC-Matrix bridging:

IRC Connection Tracker is a thing now. It’s a project to separate out the IRC connections from the appservice so the two can be run independently, so that restarting the appservice doesn’t affect the IRC connections. It’s hopefully going to allow bridge stuff to run faster when it’s done.

This project is still really early stage. You can take a look at the Proposal document here.

Clients

Nheko 0.5.4

Not technically this week, but Nheko 0.5.4 was recently released

  • The settings page now includes the device id & device fingerprint (thanks @valkum )
  • The Polish translation has been updated (thanks @m4sk1n )

Get the latest builds of Nheko from bintray.

Fractal

Alexandre Franke and the GNOME crew have been working on Fractal, gearing up for their upcoming 3.30 release.

Fractal 3.29.92 got released and we are freezing strings to give GNOME translators a bit of time to complete translations for 3.30 next week. Latest developments include tweaks for the title bar, misc bug fixes, a new presentation for uploaded files (that are not images, those are still displayed inline) with buttons to download or open them.
Development builds are now parallel installable for easier testing and CI has been improved.

Seaglass

neilalexander:

Seaglass now has some early support for inline images and attachments, and supports Quick Look. Also handles emotes and notices better. It also has version numbers now, various other little fixes and Aaron Raimist has been working on auto-update support.

Version numbers! Now when you go to download the tarball from bintray, you can see what’s going on!

There’s also been some work on supporting dark mode on Mojave (which looks particularly sexy) and even Touch Bar support!

Riot Android v0.8.15

Riot Android v0.8.15 is on it’s way to the Play Store.

Changes in Riot Android 0.8.15 (2018-08-30)

MatrixSdk:

  • Upgrade to version 0.9.9.

Improvements:

  • Improve intent to open document (#2544)
  • Avoid useless dialog for permission (#2331)
  • Improve wording when exporting keys (#2289)

Other changes:

  • Upgrade lib libphonenumber from v8.0.1 to 8.9.12
  • Upgrade Google firebase libs

Bugfix:

  • Handle \/ at the beginning of a message to send a message starting with / (#658)
  • Escape nicknames starting with a forward slash / in mentions (#2146)
  • Improve management of Push feature
  • MatrixError mResourceLimitExceededError is now managed in MxDataHandler (vector-im/riot-android#2547 point 2)

Changes in Riot Android 0.8.14 (2018-08-27)

MatrixSdk:

  • Upgrade to version 0.9.8.

Features:

  • Manage server quota notices (#2440)

Improvements:

  • Do not ask permission to write external storage at startup (#2483)
  • Update settings icon and transparent logo for notifications and navigation drawer (#2492)
  • URL previews are no longer requested from the server when displaying URL previews is disabled (PR #2514)
  • Fix some plural and puzzle strings, and remove other unused ones (#2444)
  • Manage System Alerts in a dedicated section

Other changes:

  • Upgrade olm-sdk.aar from version 2.2.2 to version 2.3.0
  • move PieFractionView from the SDK to the client (#2525)

Bugfix:

  • Fix media sharing (#2530)
  • Fix notification sound issue in settings (#2524)
  • Disable app icon badge for “listen for event” notification (#2104)

Riot iOS 0.7.3

Changes in 0.7.3 (2018-08-27)

Improvements:

  • Upgrade MatrixKit version (v0.8.3).

Bug fix:

  • Fix input toolbar reset in RoomViewController on MXSession state change (#2006 and #2008).
  • Fix user interaction disabled in master view of UISplitViewContoller when selecting a room (#2005).

Changes in 0.7.2 (2018-08-24)

Improvements:

  • Upgrade MatrixKit version (v0.8.2).
  • Server Quota Notices in Riot (#1937).

Bug fix:

  • User defaults: the preset application language (if any) is ignored.
  • Recents: Avoid to open a room twice (it crashed on room creation on quick HSes).
  • Riot-bot: Do not try to create a room with it if the user homeserver is not federated.

Riot Web

There’s been lots of work debugging and optimising Lazy Loading, which is edging closer to being turned on by default.  We’ve also been working away at improving E2E UX – starting with finishing key backup, and then improved verification, and then finally cross-signing (at last!)

SDKs

Ruby Matrix SDK

ananace is working on the Ruby Matrix SDK “fixing issues and documenting as I go”.

Matrix Android SDK

Changes to Matrix Android SDK in 0.9.9 (2018-08-30)

Improvements:

  • Clear unreachable Url when clearing media cache (vector-im/riot-android#2479)
  • “In reply to” is not clickable on Riot Android yet. Make it a plain text (vector-im/riot-android#2469)

Bugfix:

  • Removing room from ‘low priority’ or ‘favorite’ does not work (vector-im/riot-android#2526)
  • MatrixError mResourceLimitExceededError is now managed in MxDataHandler (vector-im/riot-android#2547)

API Change:

  • MxSession constructor is now private. Please use MxSession.Builder() to create a MxSession

Changes to Matrix Android SDK in 0.9.8 (2018-08-27)

Features:

  • Manage server_notices tag and server quota notices (vector-im/riot-android#2440)

Bugfix:

  • Room aliases including the ‘@’ and ‘=’ characters are now recognized as valid (vector-im/riot-android#2079, vector-im/riot-android#2542)
  • Room name and topic can be now set back to empty (vector-im/riot-android#2345)

API Change:

  • Remove PieFractionView class from the Matrix SDK. This class is now in Riot sources (#336)
  • MXMediasCache.createTmpMediaFile() methods are renamed to createTmpDecryptedMediaFile()
  • MXMediasCache.clearTmpCache() method is renamed to clearTmpDecryptedMediaCache()
  • Add MXMediasCache.moveToShareFolder() to move a tmp decrypted file to another folder to prevent deletion during sharing. New API MXMediasCache.clearShareDecryptedMediaCache() can be called when the application is resumed. (vector-im/riot-android#2530)

Matrix iOS SDK

Changes in Matrix iOS SDK in 0.11.3 (2018-08-27)

Bug fix:

  • MXJSONModel: Manage m.server_notice empty tag sent due to a bug server side (PR #556).

Changes in Matrix iOS SDK in 0.11.2 (2018-08-24)

Improvements:

  • MXSession: Add the supportedMatrixVersions method getting versions of the specification supported by the homeserver.
  • MXRestClient: Add testUserRegistration to check earlier if a username can be registered.
  • MXSession: Add MXSessionStateSyncError state and MXSession.syncError to manage homeserver resource quota on /sync requests (vector-im/riot-ios/issues/1937).
  • MXError: Add kMXErrCodeStringResourceLimitExceeded to manage homeserver resource quota (vector-im/riot-ios/issues/1937).
  • MXError: Define constant strings for keys and values that can be found in a Matrix JSON dictionary error.
  • Tests: MXHTTPClient_Private.h: Add method to set fake delay in HTTP requests.

Bug fix:

  • People tab is empty in the share extension (vector-im/riot-ios/issues/1988).
  • MXError: MXError lost NSError.userInfo information.

Tools

matrix-to-riot

Half-Shot shared a handy Firefox extension: matrix-to-riot

This is a tiny webextension to forward matrix.to links to an open Riot tab.

Really useful if you often click matrix.to links and find yourself needing to URL-hack to get to where you need to be.

That’s all for now

We’re going to shift this week’s Matrix Live to Monday, alongside the upcoming blog post on the spec release progress. Have a good weekend!

So long Half-Shot, thanks for all the bridges

Thank you to Half-Shot for all your work on Bridges over the last months and beyond. Today is your last day, but I’m sure we’ll see you again before long. Text below is from Half-Shot.


Today marks my last day of my 3 month internship at New Vector (the startup which hires many of the core Matrix team). For those of you who haven’t been reading Ben’s fabulous blog posts, I’ve been working exclusively on bridges; in particular the IRC bridge.

Tasked with the goal of making it crash less and run faster, I hope that the evidence is visible and people are generally having a better experience on it!

Some stats pulled from the matrix-appservice-irc repo:

  • 39 PRs closed (4 remain open)
  • 27 issues closed, 27 issues opened.
  • 334 commits, averaging 7.6 commits a PR.

Commits this year:

Commits this year


But aside from showing off some stats, I wanted to mention all the new features:

  • Replies on Matrix translate well to IRC, or as well as IRC allows.
  • People mentioning your IRC nick now ping your matrix user, finally!
  • So. Many. Metrics. Everything you wanted to know about the internals of the bridge, but were too afraid to --inspect.
  • Not spamming homeservers with join requests on startup (it makes for a happy ops team).
  • No longer are IRC users shackled to a “(IRC)” extension on their displayname, you can be who you want
    with group flairs!
  • Support for node 4 has been dropped, and support for 6,8 and 10 has been assured.

On the matrix-appservice-bridge side, I optimised some calls to cache locally and avoid hitting the homeserver too often, and disabling presence for homeservers that don’t support it.

There are future plans to make bridging more visible to Matrix Clients as a first class citizen. Ideas like speccing a state event (MSC1410) so that bridges can interact with each other properly and clients can create full bridge management views which are still decentralised from an integration manager.

I’d like to give a shoutout to Travis who has reviewed nearly all my changes that have made their way into the bridge, on top of all the other tasks he has on his plate. And of course a thank you to all of the Matrix team who have been very supportive during my time here.

This Week in Matrix 2018-08-24

Synapse 0.33.3

Big week for Synapse: v0.33.3 was released this week. You’ll find preparation for support for Lazy Loading and Room Versioning, lots of bugfixes, and a great contribution from vojeroen: support for sending SNI over federation for vhosted servers. More complete change log here, features below:

  • Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! (#3439)
  • Add /_media/r0/config (#3184)
  • speed up /members API and add at and membership params as per MSC1227 (#3568)
  • implement summary block in /sync response as per MSC688 (#3574)
  • Add lazy-loading support to /messages as per MSC1227 (#3589)
  • Add ability to limit number of monthly active users on the server (#3633)
  • Support more federation endpoints on workers (#3653)
  • Basic support for room versioning (#3654)
  • Ability to disable client/server Synapse via conf toggle (#3655)
  • Ability to whitelist specific threepids against monthly active user limiting (#3662)
  • Add some metrics for the appservice and federation event sending loops (#3664)
  • Where server is disabled, block ability for locked out users to read new messages (#3670)
  • set admin uri via config, to be used in error messages where the user should contact the administrator (#3687)
  • Synapse’s presence functionality can now be disabled with the “use_presence” configuration option. (#3694)
  • For resource limit blocked users, prevent writing into rooms (#3708)

Ananace has upgraded his K8s-optimized Synapse image to 0.33.3

Spec

  • A large chunk of e2e has landed in the spec, largely thanks to Zil0‘s work. There are still some things left, but the main bits related to sending encrypted messages is in there. Attachments and key sharing are being reviewed today
  • TravisR progressed on Push, Application Services, and Client-Server APIs.
  • To follow along with progress, the best place to look is the August 2018 r0 tracker.

E2E

Key backups are being worked on, and we’ve had some new thoughts on key verification.

ru-matrix.org

Well are you?

Alexey and ma1uta have a new Russian-language community wiki available at https://ru-matrix.org. This:

accumulates Russian translates of Matrix FAQ and manuals, and other info about Matrix in Russian. Also we starts a public Matrix Synapse server for Russian users with free open registration

Russian Matrix users: feel free to add and complement articles in our wiki to make Matrix more friendly for Russian users.

Client News

Seaglass

neilalexander has been on a roll the last few weeks working on Seaglass, the macOS-native client:

Seaglass now has support for creating, joining and leaving rooms, and accepting/rejecting room invites, redacting messages, improvements to the room settings pages, timestamps, some visual tweaks, lots of bug fixes, early support for detecting failed message sends, scrolling improvements

Matrique

Black Hat continues work on Matrique, a glossy Qt client:

Typing notification and desktop notification of Matrique are working, plus other improvements

See the latest commits here.

Riot Web

Lazy Loading (LL) is available on develop behind the LABS flag, and is being polished. It will probably not make it to next release, but it’s progressing!

Riot Mobile

New releases are imminent for both Android and iOS:

  • iOS: bug fixes and Lazy Loading in settings
  • Android: bug fixes, better management of permission requests at startup

Shared Secret Auth and Matrix Corporal

Slavi has been working on several operations tools for Synapse. Firstly Shared Secret Auth password provider module for Matrix Synapse.

It allows external systems (which know the shared secret) to easily obtain access tokens for any user hosted on the homeserver. This is incredibly useful for all sorts of automation (no more pre-generated access tokens or plain-text passwords).

With this tool, you can pass a mxid, and get back an access_token for that user. With this, you can do whatever actions would normally be done with a token. For example, you for a new starter at a company, you could join a set of rooms known to be needed by new starters.

Support room for Shared Secret Auth: #matrix-synapse-shared-secret-auth:devture.com

Next, he has released Matrix Corporal, reconciliator and gateway for a managed Matrix server:

It’s a way to take control of Matrix (Synapse) in a corporate (or other such) environment.
Based on a configuration policy (generated by an external system, like your intranet), it can auto-create and auto-disable users, keep their profile and authentication details to date, auto-join/leave users to communities and rooms, etc.
It’s also meant to sit in front of the Matrix Synapse Client API, inspecting and allowing/denying requests, in accordance with the configuration policy (preventing people from leaving certain rooms/communities, preventing them from messing around with their profile details, etc.)

Support room for Matrix Corporal: #matrix-corporal:devture.com

Note that Slavi also maintains matrix-docker-ansible-deploy, which can deploy these projects along with Synapse itself.

Bots

twimbot

I was extremely young when I first read and understood Larry Wall’s explanation about the virtue of laziness, but I did not expect it to lead here. Cadair has created (I think inspired by an idea from TravisR), a “twimbot“. This bot works in conjunction with opsdroid and the opsdroid connector-matrix – it works by storing selected messages from #twim:matrix.org in room state. In this way, it either assists the author by making it easier to produce the blog post you’re reading, or it coldly replaces the author, leading to his sacking and penury. To view updates currently held by the bot, join #twim_updates:cadair.com.

Hashtag bot

When he’s not working away on the Matrix Spec, TravisR continues to work on less [essential|sane] Matrix projects. This week we have @hashtag:t2bot.io, a bot that converts #hashtags to Matrix groups.

Why? Because why not. Source available on github: https://github.com/turt2live/matrix-hashtag-bot

Bridges

matrix-appservice-bridge and matrix-appservice-irc

Half-Shot will only be working full time with the Matrix Core team for another week (!!!), but until then he’s going full-speed with bridge work. First matrix-appservice-irc:

Just pushed out matrix-appservice-irc 0.11.0-rc3 which contains a lot of nice things. The full changelog is there, hoping to get some of this tested and running on the matrix.org servers soon, but initial testing looks good :)

Having worked so much on IRC bridging the last few months, Half-Shot has this week managed to make improvements to matrix-appservice-bridge:

Released matrix-appservice-bridge 1.6.0 which contains mostly things I was writing for the irc bridge and factored out instead. There’s a couple of goodies like automatic caching of profiles/events, and getEvent as a way to fetch events from the homeserver without fiddling with context.

Full changelog: https://github.com/matrix-org/matrix-appservice-bridge/blob/1.6.0/CHANGES.md

Some Intent operations now cache requests that would otherwise fall through
to the homeserver which can be expensive. This is configurable for Intents
via the opts.caching.ttl and size options.
AgeCounters now allow you to set your own time period buckets.
Added a new function Intent.getEvent which will fetch events
from the homeserver without any context information, which should
be quick.
MembershipCache is now exposed to let folks read and write to the cache
while also letting the bridge access it.

a release: Fix issue where roomState would fail.
b release: Fix issue where we were calling this.intent inside intent like fools!
c release: Fix issue where some stole js-sdk code was not checked thoroughly.

WhatsApp

tulir announces that mautrix-whatsapp “is now somewhat usable for basic chatting”.

It uses the go-whatsapp library to talk with the WhatsApp Web API. Using the web API means that you’ll still need WhatsApp on your phone connected to the internet, but it also means you won’t get banned for using the bridge.
The bridge is still very alpha, but basic message bridging and some advanced features already work. There’s a features & roadmap document about that.

To come chat about the WhatsApp bridge: #whatsapp:maunium.net.

mxtoot

mxtoot is a Matrix-Mastodon bot from ma1uta, creator of Jeon and the ecosystem around it.

There were two releases last week:

0.4.6 with additional commands (follow, unfollow, mute, unmute, block, unblock users, show followers and followings of the specified user) and a little bugfix.

0.4.7 is out now, which increases the length of messages to 4000 characters.

GSOC is finished!

GSOC 2018 has been valuable for everyone involved!

  • Thank you zil0 for all your work on the Matrix Python SDK, E2E and and spec work.
  • Thank you APWhiteHat: lots of work on dendrite, including a bunch of stuff on federation, as well as typing notifications!
  • and thank you to Cadair for helping to organising it!

The Construct using asynchronous disk IO

Jason writes:

the Construct has advanced to fully asynchronous disk IO in a single-threaded environment.
RocksDB is a highly configurable and customizable database with a very large callback surface to support many different environments, which is good to connect it with something like AIO.
Normally RocksDB gets its performance from concurrent standard POSIX system calls (blocking IO) with multiple threads. The callback surface and its internals are not specifically suited for deviating from the pthread model. However I hacked RocksDB and tricked it into believing that my userspace contexts, which mimic the std::thread/pthread interface, are threads.
This is an example of a RocksDB callback to read from a file, it expects this callback to return immediately (i.e as soon as the kernel returns from a blocking read(2).) Except, where I call fs::read() that is actually a stack-suspension which makes a request to linux AIO where the stack resumes after AIO has called back with the data (see: https://github.com/jevolk/charybdis/blob/master/ircd/aio.cc)
No modifications to the internals of RocksDB were necessary. The Construct is being developed here: https://github.com/jevolk/charybdis.

Community Guides Index

Some weeks ago I asked for suggestions for content to add to a “Community Guides Index”. There was not a wave of feedback, but we have enough collected content to be able to share a first pass of this Index. Check out the content there, and please please contact me if you have articles, videos or anything else you’d like to see included. Of course, if you’d like to produce some content and have it included, that would be great too! Ideas for missing articles:

  • any content in a language other than English
  • DevOps, how to use k8s or ansible to install Synapse
  • how to use the Application Services API to make a bot

OggCamp 2018

OggCamp last weekend was great – massive thank you to Jon for organising and running the show – we love your work!

Presentations from Ben and Half-Shot seemed to go down well, we all chatted with Open Source fans and generally flew the flag for Matrix.

New Rooms

So long…

Hope you had a good week, hope you enjoyed reading this post! Amandine is back after some needed vacation time, and has this week’s Matrix Live below.