Matrix wins Best of Show at WebRTC World!

WebRTC Best in ShowAmandine and I just got back from WebRTC World 2015 in Miami – the conference was a great success: a fantastic opportunity to meet up with many of the companies who are supporting Matrix and give everyone an update on what Matrix is up to with our Keynote: Defragmenting the Internet for fun and non-profit!.

We also had a little too much fun in the demo shoot-out – hooking up a Parrot Bebop quadrocopter into Matrix using a Matrix-enabled Janus WebRTC Gateway. The problem here is that the Parrot firmware and C SDK provides H.264 video, but doesn’t package it up at all for use with WebRTC – let alone using interoperable signalling like Matrix. So this is a classic use of Matrix to expose a simple open consistent interface to a system which is otherwise is stuck with a proprietary non-web-friendly API. The code hasn’t been tidied up yet, but our hacked Matrixified fork of Janus is up at https://github.com/matrix-org/janus-gateway/tree/ardrone3 if anyone has a drone and is crazy enough to want to experiment with it :)

Meanwhile, we also showed OpenWebRTC-powered VoIP on the latest develop iOS Matrix Console app talking hardware-accelerated H.264 through to Firefox on the desktop. My ancient 2010 MacBook Pro did its best to sabotage the demo (turns out that 1080P AirPlay + Firefox WebRTC is a bridge too far), but it gave a good idea of what’s to come. Many thanks to the OpenWebRTC team for lots of help in getting the demo together in time!

It turns out that all the demo excitement was worth it in the end, as the jury seemed to like what Matrix is up to and was kind enough to award us more points than any of the other 13 demos… meaning that we won Best In Show!!. Huge thanks to the judges for believing in the Matrix vision, and congratulations to all the other demoists too :)

Best in Show at WebRTC World!

Meanwhile, the slides from the demo presentation can be found here: Building bridges between islands of communication, and you can see the full video of our Demo here:

…and the actual video stream that the drone transmitted before I crashed it (recorded on Janus) is at…

Finally, our grand finale was meant to be combining the two demos, and showing OpenWebRTC decoding the H.264 from the Drone in hardware on an iPhone – using Matrix of course to set up the call and control the drone. Alas a TURN-related bug got in the way of this working, but we just fixed it up in the office this morning, and I’m proud to show the first ever Parrot Bebop -> Janus -> Matrix -> OpenWebRTC video stream!! (and very exciting it is too…)

Huge thanks again to Dave for doing the Matrix integration with Janus, Stefan and Rob from OpenWebRTC for all the help on the OWR side, and Manu & Giom for porting the OpenWebRTC pull request to MatrixKit and landing it in iOS Console Develop for the demo!

Announcing Synapse 0.9.0 and Matrix Angular SDK 0.6.6!

We have pushed out a new release of both Synapse, our reference server implementation, and matrix-angular-sdk, our reference webclient implementation!

The major new feature in Synapse is that you can now run Synapse backed by a PostgreSQL database. This increases performance and allows Synapse to scale much better! This, as well as various performance related bug fixes, should make things much snappier than before. Of course, you can still run SQLite; it’s up to you what you want to use.

In the webclient you can now change or reset your password – we have had this feature requested a few times (although honestly I’m surprised it hasn’t been mentioned even more – maybe people are just better than me at remembering/managing their passwords) so this should be a welcome addition! We also fixed a memory leak in Angular, so again expect better performance!

Finally, we have done some work on improving the Application Service API, making it more reliable and secure. Please see the upgrade notes as well as the full changelog below.

Changes in Synapse v0.9.0:

General:

  • Add support for using a PostgreSQL database instead of SQLite. See postgres.rst for details.
  • Add password change and reset APIs. See Registration in the spec.
  • Fix memory leak due to not releasing stale notifiers – SYN-339.
  • Fix race in caches that occasionally caused some presence updates to be dropped – SYN-369.
  • Check server name has not changed on restart.
  • Add a sample systemd unit file and a logger configuration in contrib/systemd. Contributed Ivan Shapovalov.

Federation:

  • Add key distribution mechanisms for fetching public keys of unavailable remote home servers. See Retrieving Server Keys in the spec.

Configuration:

  • Add support for multiple config files.
  • Add support for dictionaries in config files.
  • Remove support for specifying config options on the command line, except for:
    • –daemonize – Daemonize the home server.
    • –manhole – Turn on the twisted telnet manhole service on the given port.
    • –database-path – The path to a sqlite database to use.
    • –verbose – The verbosity level.
    • –log-file – File to log to.
    • –log-config – Python logging config file.
    • –enable-registration – Enable registration for new users.

Application services:

  • Reliably retry sending of events from Synapse to application services, as per Application Services spec.
  • Application services can no longer register via the /register API, instead their configuration should be saved to a file and listed in the synapse app_service_config_files config option. The AS configuration file has the same format as the old /register request. See application_services.rst for more information.

Changes in Matrix Angular SDK 0.6.6:

Features:

  • Add password change and reset feature using v2_alpha APIs.

Bug fixes:

  • Fix memory leak caused by not removing a watcher on the root scope.

iOS: Welcome to MatrixKit

Historically we’ve had two projects for iOS:

  • MatrixSDK: a low level library to interact with a Matrix homeserver
  • Console: an example Matrix client based on MatrixSDK

The primary intention of Console was to demonstrate how to use MatrixSDK to write a Matrix client app.
However, this split isn’t helpful for developers who want higher level modules that provides UIViewControllers ready to use in an existing app, with no need to manage low level communications with the Matrix homeserver.

It is where the MatrixKit project started. MatrixKit sits between MatrixSDK and your existing iOS app.

It provides customisable UIViewControllers a developer can integrate in their app.  If you want to add to your app a screen to chat in a room, you just need to use the MXKRoomViewController.

We made MatrixKit so that the components it provides are easy to integrate but also easy to customise. We do not have yet full samples of customisation as we’ve been focused on the library core, but here are a few examples:

MXKRoomViewController  JSQMessagesViewController

You probably recognise the theme of the first one, as it’s what we use in the Console app today.
The second one is the iOS7-style look and feel from JSQMessagesViewController. With few lines of code we connected it to MatrixKit data models. Yes, data models provided by MatrixKit are reusable too.

MatrixKit is also highly extensible. If you want to create new table cells to render messages, new views, new view controllers, etc, you will find a place to hook them into the MatrixKit code.

MatrixKit repository is here: https://github.com/matrix-org/matrix-ios-kit and it is available via CocoaPods (the MatrixKit pod).

In parallel of MatrixKit, we did some spring-cleaning – the official Matrix.org iOS offerings are now split into three github repos. One for each deliverable:

Other releases:

Today, we released MatrixSDK 0.4.0 (changes). Update your pods :)

Console 0.4.0 (changes) is in the Apple submission process. This will be the first version of the app using MatrixKit. Aesthetically, there is no change since the previous version. The app is more stable due to all the data abstractions and management improvements provided by MatrixKit.

If you’re an iOS developer, please have a go with MatrixKit and let us know on #ios:matrix.org how you get on!

Monitoring Synapse Metrics with Prometheus

Synapse has had support for exporting a comprehensive range of metrics via HTTP since 0.8.1 – we added this to help quantify the benefits of all the performance work which is going on currently in advance of Synapse 0.9. If you’re interested in monitoring your own synapse and seeing what’s going on using something like Prometheus, Leo just wrote a quick tutorial on getting up and running:

How to monitor Synapse metrics using Prometheus

1: Install prometheus:
Follow instructions at http://prometheus.io/docs/introduction/install/
2: Enable synapse metrics:
Simply setting a (local) port number will enable it. Pick a port. prometheus itself defaults to 9090, so starting just above that for locally monitored services seems reasonable. E.g. 9092:

Add to homeserver.yaml

metrics_port: 9092

Restart synapse

3: Check out synapse-prometheus-config
https://github.com/matrix-org/synapse-prometheus-config
4: Add synapse.html and synapse.rules
The .html file needs to appear in prometheus’s consoles directory, and the .rules file needs to be invoked somewhere in the main config file. A symlink to each from the git checkout into the prometheus directory might be easiest to ensure git pull keeps it updated.
5: Add a prometheus target for synapse
This is easiest if prometheus runs on the same machine as synapse, as it can then just use localhost:

global: {
  rule_file: "synapse.rules"
}

job: {
  name: "synapse"

  target_group: {
    target: "http://localhost:9092/"
  }
}
6: Start prometheus:
./prometheus -config.file=prometheus.conf
7: Wait a few seconds for it to start and perform the first scrape,
then visit the console:

http://server-where-prometheus-runs:9090/consoles/synapse.html

And the end result looks something like…

Prometheus screenshot

…amongst many many other system & application metrics.

You can grab the latest version of the tutorial at https://github.com/matrix-org/synapse/blob/develop/docs/metrics-howto.rst – thanks to Leo for writing it up. Any questions, let us know!

Matrix at Fluent

This week, Matrix is visiting San Francisco for Fluent, a web development conference over three days, with events ranging from 2-day training sessions to 10-min showcase presentations.

fluent

I had the opportunity to participate in the latter: Tuesday’s Solutions Showcase in the Community Lounge. The presentation was recorded, here is the video and slides.

I also had a 30-min in-depth talk earlier today, where I went through a case study of adding Matrix to your existing app (slides). After evaluating options, we decided to use the flux-chat example by Facebook – it’s a basic chat application that uses their internal message dispatcher and showcases how a React/Flux app works.

The code for the original example can be found here, and the complete diff of changes necessary to integrate it with Matrix – using the matrix-js-sdk – can be found here (thanks to Matthew for yet another late-night hack!). I think it’s very cool to see how easily their chat example can be turned into a Matrix client, albeit a fairly basic one! Here is an online version if you want to try it out!

flux-chat-org flux-chat-matrix
The original flux-chat and the Matrix-enabled flux-chat

If you have any questions or comments, we are still at Fluent – you can catch us in the exhibition hall in booth #208 – or virtually, as always, in #matrix:matrix.org!