We have a LOT of updates this week, so let's dive straight in!

🔗matrix-docker-ansible-deploy

We've covered the growth of this project several times in TWIM, but I wanted to give a little more attention to the work Slavi has been doing with matrix-docker-ansible-deploy. Synapse is a large Python project with many configurable options, and many optional components, so installing it has sometimes been a challenge. I have seen many reports that using Ansible and Docker, and in particular using these playbooks from Slavi, is a more sane way to install Synapse. The tools get a lot of attention and updates. This week, Slavi reports:

matrix-docker-ansible-deploy now has a self-check command that can help diagnose various configuration problems with the setup (DNS records being misconfigured, firewall ports not being open, etc).

For support and more info, come join the associated room: #matrix-docker-ansible-deploy:devture.com >

Late breaking news from Slavi:

One more matrix-docker-ansible-deploy update: the playbook can now set up Whatsapp-bridging via mautrix-whatsapp. Thanks to @izissise!

🔗Dimension

Dimension is an integration manager for Matrix. It's written and maintained by TravisR, and allows you to an a pre-defined selection of widgets, bots and bridges to improve your self-hosted Matrix experience. Check out: https://dimension.t2bot.io/. This week, TravisR reports:

Dimension has received quite a lot of updates over the last week. Here's what's hot off the press:

• 4 new bridges can be self-hosted and managed in Dimension: Telegram, Webhooks, Slack, and Gitter.
• 3 new widgets are available: Grafana, TradingView, and Spotify.
• Add your own custom bots for people to add to their rooms.
• A dark theme has been added and is automatically applied if you use the dark theme in Riot.
• The overall UI has been updated to be slightly more modern and less bright orange.
• Various bug fixes and improvements (is it even possible to have a changelog without this?)

As per usual, if you find any bugs or have ideas for things to add to Dimension feel free to come by #dimension:t2bot.io

🔗mautrix-telegram

tulir has been working on mautrix-telegram, and has made some massive performance improvements:

mautrix-telegram now uses the non-ORM part of SQLAlchemy for database tables that are used often. That change made the CPU usage on the t2bot.io instance drop from ~100% to ~7%

We have graphs to illustrate the improvements:

TravisR, who hosts the bridge on t2bot.io, reports that the bridge is now effectively instantaneous!

The bottleneck has returned to being synapse instead of the bridge

🔗Discord Bridge 0.3.0-rc1

A tonne of work has happened on the Discord Bridge, and it has all been brought together in 0.3.0-rc1.

From the release notes, this is just a subset of the features:

• #251 Support for Postgresql and a newer SQLite3 backend!
• #182 Replace npmlog with winston, for logging to files and better logging overall.
• #221 Add support for m.sticker.
• #210 Discord-side moderation of matrix users.
• #259 Show Matrix replies as Discord embeds.
• #164 Bot will now mention name, topic and membership changes on Discord.
• #175 Add special discord keys onto m.room.member for ghosts

Go check out the full release notes if you're interested in the Bridge as there are many more changes. Half-Shot also noted:

Shoutout to our new member of the team, @Sorunome who did a lot of the review work behind the scenes for this release. Also, thank you to everyone who submitted a PR or an issue!

🔗Slack Bridge 0.2.0 released!

We covered progress on the Slack Bridge previously, but Half-Shot has now declared it ready for 0.2.0 final! The bridge is reportedly running and very stable - go try it out now!

https://github.com/matrix-org/matrix-appservice-slack/releases/tag/0.2.0

🔗Spectral

We just missed out on this update from Spectral last week. Black Hat says:

Spectral now provides an AppImage along with Flatpak build. Also, thanks to the notification codes from nheko, Spectral can show icons in notifications, and now enters corresponding room when clicking on the notification. It also gains several UI/UX improvements.
P.s. I have resubmitted Spectral to Flathub.

🔗FAQBot

Coffee featuring two weeks in a row in TWIM:

FAQBot has been set free at last! Find its code at https://gitlab.com/Matrixcoffee/FAQBot, and its room at #faqbot:matrix.org.

FAQBot sits in various public rooms and answers common Matrix questions. Very useful for demonstrating the product to newcomers!

Also, if anyone wants to help write answers for FAQBot, https://gitlab.com/Matrixcoffee/matrix-knowledge-base is the place.

Strongly encourage people to go take a look at the knowledge base and see what can be improved.

🔗New communities

swedneck has created a new gaming community on Matrix:

we just bridged the linux-gaming community from discord to matrix, with a matrix community and individually bridged rooms/channels and all main room is #general_linuxgaming:matrix.org community is +linux-gaming:matrix.org

i've set up an instance of matrix-appservice-discord, which is bridging some select rooms from the linux-gaming discord server to +linux-gaming:matrix.org

The Linux Gaming community has gotten a proper matrix community (+linux-gaming:matrix.org) with a fair few rooms in it, all of which are bridged to a discord channel via my matrix-appservice-discord instance.

#prismo:matrix.org and #anfora:matrix.org have also been bridged to discord.

🔗Matrix for Ansible notifications

If you are using ansible, jcgruenhage has a useful addition that will allow you to get notifications from matrix:

After over a month of waiting, the matrix notification module has been merged on to ansible devel which will be released as ansible 2.8 early next year. Src: https://github.com/ansible/ansible/pull/45823

🔗mxisd v1.2.0-beta.3

Max:

mxisd v1.2.0-beta.3 is out with the start of a brand new Identity store based on arbitrary executable, to connect to anything and everything. Authentication is implemented at the moment (see doc). Feedback is very welcome to improve as much as possible for v1.2.0

🔗MSC1695 Message Edits

Discussion of message editing, in particular for how message edits from Bridges are handled has progressed. Nothing is final yet so check out https://github.com/matrix-org/matrix-doc/pull/1695 for the latest.

🔗Quaternion translations: German and Polish now available

Last week we had an update from kitsune to say that there was a new Lokalise project to allow Quaternion translations. This week, we learn that the first translations are now available:

First couple of translations - German and Polish - have made it to the released Quaternion 0.0.9.3 - thanks to krombel and krkk for their contributions! Swedish and Russian translations are in the works.

🔗Fluffychat

We anticipated it last week, but here it is:

The first stable release of the #matrix messenger #fluffychat is out now. ? Get it from: https://www.fluffy.chat
Thanks to all who have helped me. Thanks to regionetz for hosting the ubports.chat homeserver, thanks to @matrix for the awesome work, thanks to @Ubports for this awesome platform and to fabiyamada, advocatux, wayneoutthere, lionelb, Diogo, mithgarthsormr, mark, and all the awesome people from the community!!
With your help, Ubuntu Touch is still alive and has got a new stable messenger!

🔗Informo: new bot for spec changes

Informo is an ambitious project hoping to be a "decentralised news network, making information accessible". The project lives at https://github.com/Informo, but for now you can join #discuss:weu.informo.network to get their latest news.

This week, vabd reports:

We made a Matrix bot that shouts about updates to change submissions to Informo's specifications. It basically processes all changes made to the list of labels for each issue and PR of a GitHub repository's, and generates a notice message that it sends to the configured room(s). We made it because we wanted the people that are interested in Informo to know in real time about any change made to the state of proposals, along with the calls for public reviews. We just set it up in #discuss:weu.informo.network, and published its source code along with a built binary here: https://github.com/Informo/specs-bot
It might also be worth noticing that, although we designed it to shout about updates to Informo's specifications proposals, we also made it compatible with other projects, e.g. the Matrix specs

🔗Push-to-talk functionality with Jitsi on Riot

anoa has been making improvements to Video Calling on Riot:

I've been working on global push-to-talk functionality with Jitsi on Riot. I've got toggle on/off functionality working, but still trying for walkie-talkie mode. To do so, I need to get this library working with Riot: https://github.com/WilixLead/iohook
If anyone has experience with native Node modules and/or Electron, please hit me up! @andrewm:amorgan.xyz

🔗Greetings from Mozfest

Mozfest is a tech-focused event happening this weekend in London. Neil and I have been along tonight and we've been chatting to a lot of people about Matrix, decentralisation and all those things we love! Check out our very short and sweet video below!

This is the start of Season Three of Matrix Live. Matrix Live seasons are variable in length, based on the data available so far. From this season, Matrix Live with change the format slightly, based on feedback. The videos will try to be a bit more interesting, varied, and deep. With this video being the start of a new season, it was meant to be more substantive with us talking to Mozfest-ites, but I lost track of time, so this shorter but still energetic video will hopefully convey the idea.

Olm 3.0.0 has been released, which features several big changes. It can be downloaded from https://git.matrix.org/git/olm/. The npm package for JavaScript can be downloaded from https://matrix.org/packages/npm/olm/olm-3.0.0.tgz

🔗Python

The biggest change is the merge of poljar's improved Python bindings. These bindings should be much easier to use for Python programmers, and are used by Zil0's E2E support in the Matrix Python SDK.

Since the binding API has changed, existing Python code will need to be rewritten in order to work with this release.

poljar has also included comprehensive documentation for the new API.

🔗CMake

mujx contributed support for building olm using CMake. This should allow for easier building on different platforms. Currently the library can be built using either make or CMake. In the future, make support may be removed.

🔗JavaScript

The JavaScript bindings now use WebAssembly by default. WebAssembly is much faster than the previous asm.js build, and is supported by recent versions of the most popular browsers. For compatibility with browsers that do not support WebAssembly, the asm.js version is still provided.

Due to adding support for WebAssembly, the API had to be changed slightly. There is now an init function that must be called before the library can be used. This function will return a promise that will resolve once the library is ready to be used. The matrix-js-sdk has not yet been updated to do this, so users of matrix-js-sdk should continue using olm 2.x until it has been updated.

🔗Key backups

The public key API has been updated to support the proposal for server-side key backups. More details on how to use these functions will be published in the future.

Hi folks,

Today is one of those pivotal days for the Matrix ecosystem: we're incredibly excited to announce that the world's first ever dedicated homeserver hosting service is now fully available over at https://modular.im!  This really is a massive step for Matrix towards being a mature ecosystem, and we look forward to Modular being the first of many hosting providers in the years to come :D

Modular lets anyone spin up a dedicated homeserver and Riot via a super-simple web interface, rather than having to run and admin their own server.  It's built by New Vector (the startup who makes Riot and hires many of the Matrix core team), and comes from taking the various custom homeserver deployments for people like Status and TADHack and turning them into a paid service available to everyone.  You can even point your own DNS at it to get a fully branded dedicated homeserver for your own domain!

Anyway, for full details, check out the announcement over at the Riot blog.  We're particularly excited that Modular helps increase Matrix's decentralisation, and is really forcing us to ensure that the Federation API is getting the attention it deserves.  Hopefully it'll also reduce some load from the Matrix.org homeserver! Modular will also help Matrix by directly funding Matrix development by the folks working at New Vector, which should in turn of course benefit the whole ecosystem.

Many people reading this likely already run their own servers, and obviously they aren't the target audience for Modular.  But for organisations who don't have a sysadmin or don't want to spend the time to run their own server, hopefully Modular gives a very cost-effective way of running your own dedicated reliable Matrix server without having to pay for a sysadmin :)

We're looking forward to see more of these kind of services popping up in the future from everywhere in the ecosystem, and have started a Matrix Hosting page on the Matrix website so that everyone can advertise their own: don't hesitate to get in touch if you have a service to be featured!

If you're interested, please swing by #modular:matrix.org or feel free to shoot questions to [email protected].

🔗Bridges and SDKs

🔗Slack Bridge 0.2.0-rc1

Half-Shot, master of the bridge, has been working with Cadair to produce a new RC for bridging Matrix and Slack.

We've got a Slack RC out at long last! Rejoice! It's absolutely massive and full of features.

Half-Shot even provided this explanation of how it works:

Integrations manager > sign into slack > click your channel > ??? > channel bridged :)

See release notes:

This is the first release of the Slack bridge. 0.1.0 has been the version number for previous efforts but was never an official release.

To test this integration, use Riot on https://riot.im/develop and select the "Event Bridging" when adding the integration.

🔗maubot Python

tulir's maubot bot system:

The maubot Python rewrite I twimmed two weeks ago is now complete. The plugin system seems to work well and I'm pretty sure I'll be able to implement proper plugin reloading now. Next I'll implement plugin config storage and some way to manage maubot and plugins (maybe a plugin to manage plugins?)

🔗Clients

🔗Riot Web 0.17, Riot Android 0.8.18: Lazy Loading

This week saw the launch of Riot Web 0.17, and two bug-release updates (0.17.1 and 0.17.2) to fix issues on the Windows desktop app. This version is substantially faster due to its use of Lazy Loading room members. Read more here.   Meanwhile, Travis continues his foray into 'first impressions' bugs - including an initial implementation of .well-known URI support!

Riot Android 0.8.18 is also available from the Play Store and F-Droid, with Lazy Loading option available in the Labs menu (but still has a few bugs left).

Riot iOS meanwhile is busy implementing incremental server-side E2E key backups, and there's generally been a huge amount of work on E2E encryption UX across the board in preparation for all-new E2E on Web and iOS.  More details will be coming soon!

With this done, Riot is now getting a lot more attention on the impending redesign, with Bruno starting to merge code to the experimental branch.

🔗Quaternion 0.0.9.3

After being in RC for a week, Quaternion 0.0.9.3 is ready and will be released over the weekend. Most importantly, you can now translate it into your language! Just head over to https://lokalise.co/public/730769035bbc328c31e863.62506391/, register (you can optionally reuse your GitHub account), ask in #qmatrixclient:matrix.org to add your language to the list (if it's not there yet) and start translating!

🔗Coffee on Matrix Console for Android

Coffee rolled a natural twenty for bravery this week.

I have decided to take up maintainership of the Matrix Console for Android client. This is still the only multi-account capable Android client, but it is unmaintained and growing long in tooth. In particular, the API endpoints it uses may be removed from Synapse soon. I will not be developing new features for it, but I will integrate reasonable patches if others want to take that on. My own goals are to remove GCM and analytics, and get it added to F-Droid. And of course to switch to the latest API. As part of this work, I have been fighting Gradle and its bugs to get matrix-android-sdk to build together with matrix-android-console as a git submodule, so it's no longer necessary to inject the precompiled sdk into the source code. I did not win yet.

🔗FluffyChat stable release creeps closer

Ubuntu Touch fans will know that FluffyChat has been progressing rapidly, and the project is approaching a first stable release. You can find current features being tested as part of the release here.

🔗nheko

mujx has decided to stop maintaining nheko. Since many people are using it, we hope that someone will step in to continue his work. Thanks to mujx for his work so far.

🔗Featured on Mozilla Hacks blog

If you like blog posts about Matrix (and if you're reading this, you may well do), then you'll be interested to see that Matrix was featured on the Mozilla Hacks blog: Decentralised, Real-Time, Interoperable Communication with Matrix. The article was also picked up by Hacker News.

🔗Raiden Network uses Matrix for transport - this article explains it

The Raiden Network enables fast payments for Ethereum. From their website:

The Raiden Network is an off-chain scaling solution, enabling near-instant, low-fee and scalable payments. It's complementary to the Ethereum blockchain and works with any ERC20 compatible token.

To help explain their use of Matrix in their solution, they have prepared an article: Raiden Transport Explained.

🔗Synapse 0.33.7

Soon, so soon, there will be a full Python 3 release for those running Synapse in worker configuration, but this one is not it. Check out Synapse 0.33.7 changes here.

🔗Synapse on kubernetes update

Ananace:

tracking the latest synapse release on my K8s optimized image. Got no real time to work on things due to deadlines at work, but that should end come November, so expect more odd Ruby stuff after that point.

"Odd Ruby stuff" will hopefully include a return to the Matrix Ruby SDK!

🔗mxisd v1.2.0-beta.2

Max continues work on mxisd:

mxisd v1.2.0-beta.2 is out, fixing bugs found during beta.1

🔗Cadair at GSOC

Cadair was at the GSOC 2018 Mentors Summit in Mountain View last week:

I attended the GSOC mentor summit. I had some great conversations with people who are using matrix and with people about bridging in different chat services. A lot of matrix stickers all vanished off the overloaded sticker table. I have lots of ideas for GSOC next year, and plan to try and get many more community projects involved. Finally, I dont need to eat chocolate for a month.

GSOC chocolate: GCHOC

🔗Until next week

Next Friday /me is going to be at MozFest 2018, promoting Matrix, Open Source, decentralisation etc. (all the stuff we know and love), so I may change the schedule a little next week.  We're also going to reboot Matrix Live, so consider this a hiatus before Season 3 begins next week!!

Hey ho, let's go. Synapse 0.33.7 has arrived.

Regular readers will know how close we are to a full python 3 release. We are not quite there yet but 0.33.7 has support for Synapse under worker mode and we've running it on matrix.org this week. We need more time to conclusively gauge performance improvements but the Synchrotron workers are running with 33% less RAM. Thanks to everyone who has been running their servers under py3, if you do spot anything unusual just let us know. Once we've been running it a bit longer on matrix.org, we'll cut a 0.34.0 release with a recommendation that one and all upgrade to python 3.

Aside from that this release contains support for server side end to end key backups, paving the way for client side support in Riot and Rich continues his long running federation bug squash-a-thon which should help with a whole host of federation snafus.

Up next on the horizon is returning in earnest to getting the server to server r0 spec out starting with shipping our brand new super shiny state resolution algorithm.

As a final point, for those of you that deploy from git checkout or a snapshot url and have email notifications enabled please take a look warning in the change log.

As a final final point #synapse:matrix.org is now an officially supported room, aimed at Synapse admins. If you've not done so already please do drop by and say Hi.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here.

Onwards!

🔗Synapse 0.33.7 Change Log

Warning: This release removes the example email notification templates from res/templates (they are now internal to the python package). This should only affect you if you (a) deploy your Synapse instance from a git checkout or a github snapshot URL, and (b) have email notifications enabled.

If you have email notifications enabled, you should ensure that email.template_dir is either configured to point at a directory where you have installed customised templates, or leave it unset to use the default templates.

The configuration parser will try to detect the situation where email.template_dir is incorrectly set to res/templates and do the right thing, but will warn about this.

🔗Features

• Ship the example email templates as part of the package (#4052)
• Add support for end-to-end key backup (MSC1687) (#4019)

🔗Bugfixes

• Fix bug which made get_missing_events return too few events (#4045)
• Fix bug in event persistence logic which caused 'NoneType is not iterable' (#3995)
• Fix exception in background metrics collection (#3996)
• Fix exception handling in fetching remote profiles (#3997)
• Fix handling of rejected threepid invites (#3999)
• Workers now start on Python 3. (#4027)
• Synapse now starts on Python 3.7. (#4033)

🔗Internal Changes

• Log exceptions in looping calls (#4008)
• Optimisation for serving federation requests (#4017)
• Add metric to count number of non-empty sync responses (#4022)

We have a brand new, exciting guide page offering an introduction to matrix-js-sdk. This guide will live with the documentation at https://matrix.org/docs/guides/usage-of-the-matrix-js-sdk,  but you can find the text below.

---

Matrix allows open real-time communications over the Internet using HTTP and JSON. This makes developing clients to connect to Matrix servers really easy! Because it's open, and uses simple syntax for messages, you can connect Matrix to anything that communicates over a standard HTTP interface - later projects in this series will explore ideas such as building bots, performing machine learning on message content, and connecting IoT devices such as Philips Hue lights.

🔗Making a Matrix Client

Let's explore how we would make a very simple Matrix client, with only the ability to perform an initial sync, and to get member lists and the timeline for rooms of our choice.

This article will explore the Matrix Client-Server API, making use of the matrix-js-sdk. Later articles may discuss making the underlying calls. Specifically we will cover:

• login
• simple syncing
• listening for timeline events
• access the `MatrixInMemoryStore`
• sending messages to rooms
• how to respond to specific messages, as a bot would

We'll use Node.js as our environment, though the matrix-js-sdk can also be used directly in the browser.

Before we start, make sure you have Node.js and NPM installed: follow instructions at nodejs.org for your platform. Then create a new directory to work in:

mkdir my-first-matrix-client
cd my-first-matrix-client

🔗Let's write JavaScript

Once you're ready, the first thing to do is install the matrix-js-sdk from NPM:

npm install matrix-js-sdk

We include the SDK in our source exactly as expected:

import sdk from 'matrix-js-sdk';

🔗Login with an access token

Instantiate a new client object and use an access token to login:

const client = sdk.createClient({
    baseUrl: "https://matrix.org",
    accessToken: "....MDAxM2lkZW50aWZpZXIga2V5CjAwMTBjaWQgZ2Vu....",
    userId: "@USERID:matrix.org"
});
// note that we use the full MXID for the userId value

(jsdoc for createClient)

If you are logged into Riot, you can find an access token for the logged-in user on the Settings page.

If the homeserver you're logging in to supports logging in with a password, you can also retrieve an access token programmatically using the API. To do this, create a new client with no authentication parameters, then call client.login() with "m.login.password":

const client = sdk.createClient("https://matrix.org");
client.login("m.login.password", {"user": "USERID", "password": "hunter2"}).then((response) => {
    console.log(response.access_token);
});

In any case, once logged in either with a password or an access token, the client can get the current access token via:

console.log(client.getAccessToken());

Note: it is essential to keep this access token safe, as it allows complete access to your Matrix account!

🔗Sync and Listen

Next we start the client, this sets up the connection to the server and allows us to begin syncing:

client.startClient();

Perform a first sync, and listen for the response, to get the latest state from the homeserver:

client.once('sync', function(state, prevState, res) {
    console.log(state); // state will be 'PREPARED' when the client is ready to use
});

Once the sync is complete, we can add listeners for events. We could listen to ALL incoming events, but that would be a lot of traffic, and too much for our simple example. If you want to listen to all events, you can add a listen as follows:

client.on("event", function(event){
    console.log(event.getType());
    console.log(event);
})

Instead, let's just listen to events happening on the timeline of rooms for which our user is a member:

client.on("Room.timeline", function(event, room, toStartOfTimeline) {
    console.log(event.event);
});

🔗Access the Store

When we created a new client with sdk.createClient(), an instance of the default store, MatrixInMemoryStore was created and enabled. When we sync, or instruct otherwise our client to fetch data, the data is automatically added to the store.

To access the store, we use accessor methods. For example, to get a list of rooms in which our user is joined:

// client.client.getRooms() returns an array of room objects
var rooms = client.getRooms();
rooms.forEach(room => {
    console.log(room.roomId);
});

(jsdoc for client.getRooms)

More usefully, we could get a list of members for each of these rooms:

var rooms = client.getRooms();
rooms.forEach(room => {
    var members = room.getJoinedMembers();
    members.forEach(member => {
        console.log(member.name);
    });
});

For each room, we can inspect the timeline in the store:

var rooms = client.getRooms();
rooms.forEach(room => {
    room.timeline.forEach(t => {
        console.log(JSON.stringify(t.event.content));
    });
});

🔗Send messages to rooms

To send a message, we create a content object, and specify a room to send to. In this case, I've taken the room ID of #test:matrix.org, and used it as an example:

var testRoomId = "!jhpZBTbckszblMYjMK:matrix.org";

var content = {
    "body": "Hello World",
    "msgtype": "m.text"
};

client.sendEvent(testRoomId, "m.room.message", content, "").then((res) => {
   // message sent successfully
}).catch((err) => {
    console.log(err);
}

(jsdoc for client.sendEvent)

Knowing this, we can put together message listening and message sending, to build a bot which just echos back any message starting with a "!":

var testRoomId = "!jhpZBTbckszblMYjMK:matrix.org";

client.on("Room.timeline", function(event, room, toStartOfTimeline) {
    // we know we only want to respond to messages
    if (event.getType() !== "m.room.message") {
        return;
    }
 
    // we are only interested in messages from the test room, which start with "!"
    if (event.getRoomId() === testRoomId && event.getContent().body[0] === '!') {
        sendNotice(event.event.content.body);
    }
});

function sendNotice(body) {
    var content = {
        "body": body.substring(1),
        "msgtype": "m.notice"
    };
    client.sendEvent(testRoomId, "m.room.message", content, "", (err, res) => {
        console.log(err);
    });
}

Take a look at the msgtype in the object above. In the previous example, we used "m.text" for this field, but now we're using "m.notice". Bots will often use "m.notice" to differentiate their messages. This allows the client to render notices differently, for example Riot, the most popular client, renders notices with a more pale text colour.

🔗Further

There is much, much more to Matrix, the Client-Server API and the matrix-js-sdk, but this guide should give some understanding of simple usage. In subsequent guides we'll cover more detail and also explore projects you can build on top, such as IoT controls and chatbot interfaces. For now you can take a look at other examples in the matrix-js-sdk itself, and also the Matrix Client-Server API which it implements.

🔗Matrix Spec updates

• MSC1695 Half-Shot has a newly released proposal relating to Message Edits: "The key difference to this proposal is that it's only the event schema which is based off the relates stuff. It does NOT do any kind of aggregations but simply is a format to allow bridges/clients to specify edits in the metadata."
• MSC1693 Erik has been thinking about state res v2 in the case of rejected events

🔗libQMatrixClient 0.4.0

We mentioned libQMatrixClient progress last week, but this week kitsune reports:

libQMatrixClient 0.4.0 has just been released: CS API 0.4.0, local echo, caching avatars to disk and more - check the release notes at https://github.com/QMatrixClient/libqmatrixclient/releases/v0.4.0!

Thanks for the work also go to Black Hat (who is using libQMatrixClient to author Spectral) and delijati.

🔗Informo

vadb, back again with news about Informo:

The Informo specifications documentation is out! ? It now lives right here: https://specs.informo.network
As a reminder, we were working on "phase 1" of this documentation, which goal was to outline how Informo was going to work. This is basically the foundation for a more complete technical documentation, which shall come later.
The idea is to enable people to give it a look as soon as possible, and to enable them to contribute to Informo as early in the design process as possible. The spec is entirely open, with a process for contributions documented and all its source files available on GitHub here: https://github.com/Informo/specs
The online version I initially linked is a live version from the repo's master branch, which is updated each time commits are pushed to it (including on merging a PR).
If people have any question or remark regarding the specs, or Informo in a general manner, we'll gladly answer them in #discuss:weu.informo.network (or on the #informo IRC channel on Freenode, which is bridged to the Matrix room)! ?

🔗mxisd v1.2.0-beta.1 (beta release)

Max has continued work on mxisd, an Identity Server:

mxisd got a new beta release v1.2.0-beta.1.
It adds the ability to send email notification about room invites to existing accounts in Identity store that might not have been provisioned in the homeserver yet as users never logged in, or for users' profile that are not auto-populated with 3PIDs. This is especially targeted for onboarding/enrolling times when adopoting Matrix in groups/corporations or for custom setups.
We would love to hear about your experience with it!

🔗matrix-trello-bot

TravisR has been working on matrix-trello-bot:

matrix-trello-bot has received quite the overhaul. It's gone from a small bot that had pre-determined notifications for board activity to a much more capable bot. The bot now supports managing boards (yes, you can create a board from within Matrix!), lists, and cards including creating, moving, archiving, assigning, and querying.

On top of all that, the bot respects that not every room will want to receive every possible notification and offers commands to pick and choose which events it should notify about on a per-board basis. Feel free to give it a spin and check out the massive help menu with !trello help. Please send bugs and suggestions to GitHub :)

The help menu really is massive! Travis provided an image to highlight the scope of functionality provided (click for larger version):

🔗Clients

🔗Riot 0.16.6 Released

Fixes only these two issues:

• Firefox private mode being broken > (https://github.com/matrix-org/matrix-react-sdk/pull/2195)
• Breakage when mixing /app and /develop use (issue #7432)

As both fixes are not relevant for electron, we didn't release 0.16.6 as the electron app, but rest assured 0.17 will of course be released as electron.

🔗Fluffychat

We missed this last week, but FluffyChat (client for Ubuntu Touch) v0.6.0 was released, with LOTS of new features:

• User profiles
• Design improvements
• Audioplayer in chat
• Videoplayer in chat (only audio at the moment)
• Imageviewer
• Edit chat aliases
• Edit chat settings and permissions
• Kick, ban and unban users
• Edit user permissions
• New invite page
• Display and edit chat topics
• Change chat avatar
• Change user avatar
• Edit phone numbers
• Edit email addresses
• Display and edit archived chats
• New add-chat and add-contact pages
• Display contacts and find contacts with their phone number or email address
• Discover public chats on the user's homeserver
• Registration (currently only working with ubports.chat and NOT with matrix.org due captchas)
• Register and login with phone number
• Edit identity-server
• Add in-app viewer for the privacy policy

🔗Construct Docker image

mujx has created a new Docker image for Construct, the C++ homeserver:

Construct has a new docker image (https://github.com/matrix-construct/docker-construct) which will ease the process of deployment

🔗Translations for Quaternion

If it didn't seem that kitsune was busy enough, he also announced translations for Quaternion (the original client based on libQMatrixClient):

Quaternion is now localisable! The web platform for translation will be set up in the nearest days; meanwhile those who can't wait can just take the repo, look how the German translation is made and copy!

🔗That's all we can tell you!

Safe travels to Cadair, who is off to represent Matrix at the GSOC 2018 mentors summit. Check out Matrix Live below:

Guest post today from Giveth: Giveth is re-engineering charitable giving, by creating an entirely free, open-source platform, built on the Ethereum Blockchain. Our system cuts out bureaucracy and enables nonprofits to create a high level of transparency and accountability towards Givers.

---

Giveth's new chatbot in action!

Online or offline, joining a new community always requires some adjustment. Even the most open, inclusive communities have shared knowledge and shared practices which new members learn as they participate.

I recently joined Giveth's Riot community, where the majority of Giveth's communication occurs. Immediately upon joining, I received the message pictured above from the Giveth Bot, kindly encouraging me to download Riot mobile and change my notifications to mention-only. The bot shortened my adjustment period by giving me key tidbits of information that everyone in Giveth's community knows, but that may have taken me time to pick up on my own. This blog post will cover how the Giveth Bot came to be, what it is capable of, and where the project is headed in the future.

The Giveth Bot actually started out as an attempt to solve a completely different problem: helping Giveth efficiently distribute internal reward points. Giveth's system for rewarding people who meaningfully contribute to the project is called RewardDAO. “If someone contributes in a meaningful way, a core contributor from each of the Giveth Campaigns can dish them points to recognize the contribution”, describes Cleo in an article explaining how RewardDAO works. At the end of each month, contributors receive Ether based on how many points they have earned.

The Giveth RewardDAO motto. Photo from https://medium.com/giveth.

However, any time that a core contributor dished points to someone, they had to record who received the points, and how many, on a spreadsheet. In search of a better way, Giveth opened up the project of automating this system to the social coding hub, a community of altruistic developers looking to tackle impactful and interesting projects, offering a 2 eth bounty for a solution.

A lot of great work was submitted, and ultimately Deam's ( @deamlabs) code was chosen to power the bot and the code for the pointsbot itself was further developed and refined by Frederik Bolding. Now, by using a command of the form “!dish [number] [type] points to [contributor] for [contribution]”, Giveth core contributors can distribute points as needed, and the bot will automatically update the spreadsheet accordingly.

The Giveth Bot dishing points like a champion!

Once the bot's framework was established, chatbot features were added. In addition to the welcome message I received, the bot gives custom welcome messages in each of Giveth's different rooms, allows Matrix users to have 1-on-1 chats with it, and listens for keywords and sentences it recognizes in rooms and private chats. Riot is built on top of an open-source protocol called Matrix. Matrix has a javascript standard development kit (SDK), which the bot uses to detect events occurring in each of the Riot rooms and chats that it is a part of.

Giveth began by using Slack, but switched to Riot to support Matrix's decentralized, open-source model, which which aligns far more with Giveth's own business model and values. The Giveth Bot is a great example of how Matrix enables users to build their own solutions to problems. In the future, we hope that the Giveth Bot will be able to interact directly with the Ethereum Blockchain, and that more analytics and measurement tools can be incorporated. And of course, we welcome any and all feedback on the Giveth Bot!

Giveth is an open-source platform for building decentralized altruistic communities. Anyone interested in getting involved should head to join.giveth.io

Interested in checking out the Giveth Bot's inner workings? All code is available at https://github.com/Giveth/giveth-bot .

Interested in learning DApp development or helping out with cool projects like the Giveth Bot? Check out the social_coding Riot channel , tell us what you're interested in, and help build awesome stuff!

🔗Bridges

Half-Shot is supposedly back at university, but he's productive enough to have THREE updates this week (delivered in ascending order of interestingness):

matrix-appservice-irc 0.11.2:

https://github.com/matrix-org/matrix-appservice-irc/releases/tag/0.11.2 is just a hotfix release containing a few things we fixed in develop so yay.

matrix-appservice-node v0.3.5:

https://github.com/matrix-org/matrix-appservice-node/blob/master/CHANGELOG.md#v035 is a long overdue update that just updates dependencies so we are no longer exposing lots of vulnerabilities
Updated body-parser, express, morgan and request packages to fix security vulnerabilities.

matrix-appservice-bridge 1.7.0 released:

matrix-appservice-bridge is out! Bundled in this version are major updates to dependencies (to stop the warnings), a logging component for quick setup of pretty winston logs and a provisoner "validator" to allow bridge admins to curate which bridges can be linked into the same room to stop the dreaded double bridge issue.
More details can be found at https://github.com/matrix-org/matrix-appservice-bridge/releases/tag/1.7.0

🔗Informo news project

Informo is an ambitious project hoping to be a "decentralised news network, making information accessible". The project lives at https://github.com/Informo, but for now you can join #discuss:weu.informo.network to get their latest news.

vadb provided an update:

Hey there, here's a quick update to let you know that we're making great progress towards the completion of phase 1 of Informo's specifications' documentation (i.e. general outline without going to much in depth into the technical specifications)! I've also updated the board to cut the big "Basic spec" task down into smaller tasks to give a more precise insight of our current progress
Once that phase 1 is done we'll publish the whole thing and migrate our work to a public GitHub repository so people can get to know Informo a bit better and contribute towards making the doc better if they want

🔗Riot

riot-web 16.5 released, with matrix-js-sdk 0.11.1:

Go see the latest changes and make sure to start using room member Lazy Loading with the latest matrix-js-sdk.

🔗Libraries

Black Hat is writing an Olm wrapper for Qt:

I am writing a Qt wrapper for Olm. It has APIs similar to python-olm

kitsune and Black Hat are working on libQMatrixClient:

libQMatrixClient is nearing its release 0.4.0, with low-level support of CS API (coincidentally) 0.4.0 added this week. Another very useful addition is caching avatar images to the disk, not only in memory, relieving the network from dozens of extra requests when a client powered by the library is restarted. All thanks for the avatar caching PR go to Black Hat!

🔗SDKs

tulir on mautrix-go and mautrix-python:

My gomatrix fork is now mautrix-go and the basic client API stuff in mautrix-python is starting to be usable enough for me to start the rewrite of maubot in Python.
For a bit of background, maubot is a dynamic plugin-based bot system. It's currently written in Go, but due to many limitations in the Go plugin system, I decided to rewrite it in Python. matrix-python-sdk doesn't have asyncio or a maintainer, so I decided to make a new framework that combines my existing mautrix-appservice-python framework with a more generic client API layer. (tulir)

🔗Homeservers

Max introduces his new server, Gridepo:

mxhsd work has resumed via a new project, Gridepo, which will be a Matrix/Grid dual-stack server. While mxhsd focused on researching the protocol and reverse-engineering the spec, Gridepo will follow whatever is specced in its Matrix compatible mode.

Synapse 0.33.6 released: >

It's literally the previous blog post!

we've been focusing on fixing a whole host of federation bugs to improve reliability and latency. Additionally we've squashed some py3 bugs, improved lazy loading and been working hard in the background to improve our CI infrastructure. Finally, we cleaned up the Docker file, the image is now half the size of 0.33.5.1's standing at 58 MB.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI.

🔗DevOps and Hosting

Slavi on matrix-docker-ansible-deploy:

matrix-docker-ansible-deploy has received some more improvements lately. They're mostly about the ability to tweak things affecting performance: Synapse cache factor configuration, the ability to enable/disable user presence tracking. The logging situation has also been improved for all Docker containers, so that their output would not get logged twice (once by systemd's journal, once by Docker) - something which was previously causing unpredictably-high disk usage for long-running containers.

ananace, Synapse for kubernetes:

Updated the K8s tweaked packaging for Synapse 0.33.6 (And added a preliminary Python 3 version there as well)

swedneck (literally) on IPFS:

swedneck is hosting riot on IPFS: https://riot.swedneck.xyz, see also https://riot-all.swedneck.xyz for all the versions currently hosted there.

🔗Bots

TravisR, always working on bots:

Through use of the https://emoncms.org API, the makerspace I help run now has power monitoring statistics available from within matrix.

🔗Rooms

Coffee, maintaining #synapse:matrix.org:

#synapse:matrix.org became an official Matrix room! It also had a small face lift, changing from "Synapse Community" to "Synapse Admins", hopefully making its purpose and intended audience clearer.

🔗That's all for now

That's all for now. No Matrix Live this week, but we'll be back next week with more Matrix news and another episode of Matrix Live!

Right folks, time for Synapse 0.33.6.

These past few weeks we've been focusing on fixing a whole host of federation bugs to improve reliability and latency. Additionally we've squashed some py3 bugs, improved lazy loading and been working hard in the background to improve our CI infrastructure. Finally, we cleaned up the Docker file, the image is now half the size of 0.33.5.1's standing at 58 MB.

As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here.

🔗Synapse 0.33.6

🔗Features

• Adding the ability to change MAX_UPLOAD_SIZE for the docker container variables. (#3883)
• Report "python_version" in the phone home stats (#3894)
• Always LL ourselves if we're in a room (#3916)
• Include eventid in log lines when processing incoming federation transactions (#3959)
• Remove spurious check which made 'localhost' servers not work (#3964)

🔗Bugfixes

• Fix problem when playing media from Chrome using direct URL (thanks @remjey!) (#3578)
• support registering regular users non-interactively with register_new_matrix_user script (#3836)
• Fix broken invite email links for self hosted riots (#3868)
• Don't ratelimit autojoins (#3879)
• Fix 500 error when deleting unknown room alias (#3889)
• Fix some b'abcd' noise in logs and metrics (#3892, #3895)
• When we join a room, always try the server we used for the alias lookup first, to avoid unresponsive and out-of-date servers. (#3899)
• Fix incorrect server-name indication for outgoing federation requests (#3907)
• Fix adding client IPs to the database failing on Python 3. (#3908)
• Fix bug where things occasionally were not being timed out correctly. (#3910)
• Fix bug where outbound federation would stop talking to some servers when using workers (#3914)
• Fix some instances of ExpiringCache not expiring cache items (#3932, #3980)
• Fix out-of-bounds error when LLing yourself (#3936)
• Sending server notices regarding user consent now works on Python 3. (#3938)
• Fix exceptions from metrics handler (#3956)
• Fix error message for events with m.room.create missing from auth_events (#3960)
• Fix errors due to concurrent monthly_active_user upserts (#3961)
• Fix exceptions when processing incoming events over federation (#3968)
• Replaced all occurrences of e.message with str(e). Contributed by Schnuffle (#3970)
• Fix lazy loaded sync in the presence of rejected state events (#3986)
• Fix error when logging incomplete HTTP requests (#3990)

🔗Internal Changes

• Unit tests can now be run under PostgreSQL in Docker using test_postgresql.sh. (#3699)
• Speed up calculation of typing updates for replication (#3794)
• Remove documentation regarding installation on Cygwin, the use of WSL is recommended instead. (#3873)
• Fix typo in README, synaspse -> synapse (#3897)
• Increase the timeout when filling missing events in federation requests (#3903)
• Improve the logging when handling a federation transaction (#3904, #3966)
• Improve logging of outbound federation requests (#3906, #3909)
• Fix the docker image building on python 3 (#3911)
• Add a regression test for logging failed HTTP requests on Python 3. (#3912)
• Comments and interface cleanup for on_receive_pdu (#3924)
• Fix spurious exceptions when remote http client closes connection (#3925)
• Log exceptions thrown by background tasks (#3927)
• Add a cache to get_destination_retry_timings (#3933, #3991)
• Automate pushes to docker hub (#3946)
• Require attrs 16.0.0 or later (#3947)
• Fix incompatibility with python3 on alpine (#3948)
• Run the test suite on the oldest supported versions of our dependencies in CI. (#3952)
• CircleCI now only runs merged jobs on PRs, and commit jobs on develop, master, and release branches. (#3957)
• Fix docstrings and add tests for state store methods (#3958)
• fix docstring for FederationClient.get_state_for_room (#3963)
• Run notify_app_services as a bg process (#3965)
• Clarifications in FederationHandler (#3967)
• Further reduce the docker image size (#3972)
• Build py3 docker images for docker hub too (#3976)
• Updated the installation instructions to point to the matrix-synapse package on PyPI. (#3985)
• Disable USE_FROZEN_DICTS for unittests by default. (#3987)
• Remove unused Jenkins and development related files from the repo. (#3988)
• Improve stacktraces in certain exceptions in the logs (#3989)
• Pin to prometheus_client<0.4 to avoid renaming all of our metrics (#4002)