We have pushed out a new release of both Synapse, our reference server implementation, and matrix-angular-sdk, our reference webclient implementation!

The major new feature in Synapse is that you can now run Synapse backed by a PostgreSQL database. This increases performance and allows Synapse to scale much better! This, as well as various performance related bug fixes, should make things much snappier than before. Of course, you can still run SQLite; it's up to you what you want to use.

In the webclient you can now change or reset your password - we have had this feature requested a few times (although honestly I'm surprised it hasn't been mentioned even more - maybe people are just better than me at remembering/managing their passwords) so this should be a welcome addition! We also fixed a memory leak in Angular, so again expect better performance!

Finally, we have done some work on improving the Application Service API, making it more reliable and secure. Please see the upgrade notes as well as the full changelog below.

Changes in Synapse v0.9.0:

General:

• Add support for using a PostgreSQL database instead of SQLite. See postgres.rst for details.
• Add password change and reset APIs. See Registration in the spec.
• Fix memory leak due to not releasing stale notifiers - SYN-339.
• Fix race in caches that occasionally caused some presence updates to be dropped - SYN-369.
• Check server name has not changed on restart.
• Add a sample systemd unit file and a logger configuration in contrib/systemd. Contributed Ivan Shapovalov.

Federation:

• Add key distribution mechanisms for fetching public keys of unavailable remote home servers. See Retrieving Server Keys in the spec.

Configuration:

• Add support for multiple config files.
• Add support for dictionaries in config files.
• Remove support for specifying config options on the command line, except for:
  • --daemonize - Daemonize the home server.
  • --manhole - Turn on the twisted telnet manhole service on the given port.
  • --database-path - The path to a sqlite database to use.
  • --verbose - The verbosity level.
  • --log-file - File to log to.
  • --log-config - Python logging config file.
  • --enable-registration - Enable registration for new users.

Application services:

• Reliably retry sending of events from Synapse to application services, as per Application Services spec.
• Application services can no longer register via the /register API, instead their configuration should be saved to a file and listed in the synapse app_service_config_files config option. The AS configuration file has the same format as the old /register request. See application_services.rst for more information.

Changes in Matrix Angular SDK 0.6.6:

Features:

• Add password change and reset feature using v2_alpha APIs.

Bug fixes:

• Fix memory leak caused by not removing a watcher on the root scope.

Matrix.org is happy to be sponsoring and talking at the WebRTC Conference and Expo in Miami, Florida, 12-14 May. Both Amandine and Matthew will be there - please come have a chat by booth #22! This is one of the longest running WebRTC Events, and Matthew is delivering one of the keynotes of the conference on Wednesday 4:00-4:30pm in room K-07.

Matthew will also participate in the "Open Source Options for WebRTC Development" session in room D2-02 at 9:50am on Wednesday (full agenda here).

Finally, Matrix will also be part of the WebRTC World Demos in room X-07 sometime between 4:30 and 7:30pm on Wednesday. Expect a dangerous demo!

Just a quick note to say thanks to everyone who came to talk to us at SMR9 yesterday. SMR is a great way for developers looking for jobs and startups needing engineers to have a chat.

We had a very busy day with plenty of people interested in Matrix and eager to join the team. We received a lot of CVs and will get back to you - but in the meantime please check out our code and come say hi in the Matrix HQ room, using any of these Matrix clients!

If you missed SMR, or just generally is interested in working for Matrix.org - please feel free to send your CV to us - we need all kinds of developers, with skills ranging from backend and frontend to mobile development!

As a follow-on to the previous post, Apple approved version 0.4.0 of the iOS Matrix Console app over the weekend - grab it now from the app store!. Whilst cosmetically it looks almost identical to 0.3.x, the architecture is completely different given the app is now built using MatrixKit - and stability has improved considerably as a result. Please install and let us know how you get on!

Historically we've had two projects for iOS:

• MatrixSDK: a low level library to interact with a Matrix homeserver
• Console: an example Matrix client based on MatrixSDK

The primary intention of Console was to demonstrate how to use MatrixSDK to write a Matrix client app. However, this split isn't helpful for developers who want higher level modules that provides UIViewControllers ready to use in an existing app, with no need to manage low level communications with the Matrix homeserver.

It is where the MatrixKit project started. MatrixKit sits between MatrixSDK and your existing iOS app.

It provides customisable UIViewControllers a developer can integrate in their app.  If you want to add to your app a screen to chat in a room, you just need to use the MXKRoomViewController.

We made MatrixKit so that the components it provides are easy to integrate but also easy to customise. We do not have yet full samples of customisation as we've been focused on the library core, but here are a few examples:

You probably recognise the theme of the first one, as it's what we use in the Console app today. The second one is the iOS7-style look and feel from JSQMessagesViewController. With few lines of code we connected it to MatrixKit data models. Yes, data models provided by MatrixKit are reusable too.

MatrixKit is also highly extensible. If you want to create new table cells to render messages, new views, new view controllers, etc, you will find a place to hook them into the MatrixKit code.

MatrixKit repository is here: https://github.com/matrix-org/matrix-ios-kit and it is available via CocoaPods (the MatrixKit pod).

In parallel of MatrixKit, we did some spring-cleaning - the official Matrix.org iOS offerings are now split into three github repos. One for each deliverable:

• https://github.com/matrix-org/matrix-ios-sdk
• https://github.com/matrix-org/matrix-ios-kit
• https://github.com/matrix-org/matrix-ios-console

Other releases:

Today, we released MatrixSDK 0.4.0 (changes). Update your pods :)

Console 0.4.0 (changes) is in the Apple submission process. This will be the first version of the app using MatrixKit. Aesthetically, there is no change since the previous version. The app is more stable due to all the data abstractions and management improvements provided by MatrixKit.

If you're an iOS developer, please have a go with MatrixKit and let us know on #ios:matrix.org how you get on!

🔗Note: This blog post is outdated, and an up-to-date tutorial is located on the synapse project repo

Synapse has had support for exporting a comprehensive range of metrics via HTTP since 0.8.1 - we added this to help quantify the benefits of all the performance work which is going on currently in advance of Synapse 0.9. If you're interested in monitoring your own synapse and seeing what's going on using something like Prometheus, Leo just wrote a quick tutorial on getting up and running:

🔗How to monitor Synapse metrics using Prometheus

1: Install prometheus:

Follow instructions at http://prometheus.io/docs/introduction/install/

2: Enable synapse metrics:

Simply setting a (local) port number will enable it. Pick a port. prometheus itself defaults to 9090, so starting just above that for locally monitored services seems reasonable. E.g. 9092:

Add to homeserver.yaml

metrics_port: 9092

Restart synapse

3: Check out synapse-prometheus-config

https://github.com/matrix-org/synapse-prometheus-config

4: Add synapse.html and synapse.rules

The .html file needs to appear in prometheus's consoles directory, and the .rules file needs to be invoked somewhere in the main config file. A symlink to each from the git checkout into the prometheus directory might be easiest to ensure git pull keeps it updated.

5: Add a prometheus target for synapse

This is easiest if prometheus runs on the same machine as synapse, as it can then just use localhost:

global: {'{'}
  rule_file: "synapse.rules"
{'}'}
job: {'{'}
name: "synapse"
target_group: {'{'}
target: "http://localhost:9092/"
{'}'}
{'}'}

6: Start prometheus:

./prometheus -config.file=prometheus.conf

7: Wait a few seconds for it to start and perform the first scrape,

then visit the console:

http://server-where-prometheus-runs:9090/consoles/synapse.html

And the end result looks something like...

...amongst many many other system & application metrics.

You can grab the latest version of the tutorial at https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md - thanks to Leo for writing it up. Any questions, let us know!

This week, Matrix is visiting San Francisco for Fluent, a web development conference over three days, with events ranging from 2-day training sessions to 10-min showcase presentations.

I had the opportunity to participate in the latter: Tuesday's Solutions Showcase in the Community Lounge. The presentation was recorded, here is the video and slides.

I also had a 30-min in-depth talk earlier today, where I went through a case study of adding Matrix to your existing app (slides). After evaluating options, we decided to use the flux-chat example by Facebook - it's a basic chat application that uses their internal message dispatcher and showcases how a React/Flux app works.

The code for the original example can be found here, and the complete diff of changes necessary to integrate it with Matrix - using the matrix-js-sdk - can be found here (thanks to Matthew for yet another late-night hack!). I think it's very cool to see how easily their chat example can be turned into a Matrix client, albeit a fairly basic one! Here is an online version if you want to try it out!

The original flux-chat and the Matrix-enabled flux-chat

If you have any questions or comments, we are still at Fluent - you can catch us in the exhibition hall in booth #208 - or virtually, as always, in #matrix:matrix.org!

This post has now been edited into a guide - you can find the source in github, and the formatted guide on the matrix.org website!

---

Like a lot of open source projects, we use IRC a lot. Naturally, we also use Matrix to communicate with each other. For some time now we've had an IRC bot sitting on specific channels to "bridge" together IRC and Matrix. This bot simply sent IRC messages when it received Matrix messages and vice versa. As we started to rely on it more and more though, we realised that there were things that were impossible for simple client-side bots to do by themselves. This spurred the development of Application Services which I introduced in my previous post. In this blog post, I want to outline some of the features and techniques of the IRC application service which we've been working on over the past few weeks.

Features:

• Specific channel-to-matrix room bridging : This is what the original IRC bot did. You can specify specific channels and specific room IDs, and messages will be bridged.
• Dynamic channel-to-matrix room bridging : This allows Matrix users to join any channel on an IRC network, rather than being forced to use one of the specific channels configured.
• Two-way PM support : IRC users can PM the virtual "M-" users and private Matrix rooms will be created. Likewise, Matrix users can invite the virtual "@irc_Nick:domain" user IDs to a room and a PM to the IRC nick will be made.
• IRC nick changing support : Matrix users are no longer forced to use "M-" nicks and can change them by sending "!nick" messages directly to the bridge.
• Ident support : This allows usernames to be authenticated for virtual IRC clients, which means IRC bans can be targeted at the Matrix user rather than the entire application service.

The use of the Application Services API means:

• The bot can reserve user IDs. This prevents humans from registering for @irc_... user IDs which would then clash with the operation of the bot.
• The bot can reserve room aliases. This prevents humans from register for #irc_... aliases which would then clash with the operation of the bot.
• The bot can trivially manage hundreds of users. Events are pushed to the application service directly. If you tried to do this as a client-side bot, you would need one event stream connection per virtual user.
• The bot can lazily create rooms on demand. This means Matrix users can join non-existent room aliases and have the application service quickly track an IRC channel and create a room with that alias, allowing the join request to succeed.

Implementation details:

• Written in Node.js, designed to be run using forever.
• Built on the generic matrix-appservice-node framework.
• Supports sending metrics in statsd format.
• Uses matrix-appservice-node to provide a standardised interface when writing application services, rather than an explicit web framework (though under the hood matrix-appservice-node is using Express).

At present, the IRC application service is in beta, and is being run on #matrix and #matrix-dev. If you want to give it a go, check it out on Github - it is not currently released on npm. N.B. it requires features from the develop branch of synapse; either run your own synapse off the develop branch or wait a few days for us to release Synapse 0.9.0.

Needless to say, we look forward to this being the first of many full network<->network bridges into Matrix - come chat on #matrix:matrix.org if you'd like to write or run your own! Next up is Lync and XMPP...

This week, Matthew and myself went to the WebRTC conference and its related Kranky Geek event in sunny London.

Matrix had a speaker slot in both events; the first talk was "Proposing an open interoperable signalling layer for WebRTC" (slides).

As I was talking to people in the tea-breaks between sessions, I was actually surprised at the amount of people who not only knew about Matrix, but who had been following eagerly since the early days, and had questions about specific features and recent developments!

Later in the day it was time for the Kranky Geek, and the talk then was a bit more technical: "Interoperable HTTP Signalling with Matrix" (slides). The talk included a "dangerous demo" where we made a WebRTC call from our Matrix iOS App to our webclient for the first time - thanks to the OpenWebRTC team for helping us make the demo!

What's great about these kind of events is the feedback and discussion following talks; lots of people have relevant experiences and opinions that they are happy to share, and of course questions on how exactly different features actually work.

It's always great to meet new people and have lots of various discussions. Hopefully we have got a few more people interested in Matrix - we have already seen some new joiners in the #matrix:matrix.org room!

Next up is Fluent in San Francisco next week, where I will be speaking.

Just a quick announcement that Android Console 0.3.0 has been released on the Google Play store.

This release contains a whole lot of fixes, new features, and nicer UI - see the changes file for details, but in summary:

• The UI has been switched to Android's Material Design
• Android Lollipop is now fully supported
• We have added support for contacts
• Various accessibility and usability fixes have been contributed by Nolan Darilek (thanks!)
• We also have clientside GCM support thanks to Leon Handreke! We need to make some changes server-side before GCM can be used, but that's on the current ToDo-list!
• Lots of bugs fixed, as usual, thanks to everyone who reported an issue.