We are back from Kamailio World, where we presented and participated in James Body's "Dangerous Demos". We were racing against the deadline, but managed to join the demos at the very last minute - and even win the award for "Most Entertaining Demo"!

It was great to catch up with old acquaintances - and meet many new ones! There were only around 150 people at Kamailio World, but given the area of expertise is very specialised, you can pretty much start talking to anyone and have a really interesting conversation.

Here are the slides of Matthew's presentation, (also available as .pdf) and a video of the presentation:

A video from the dangerous demo event is available here:

The Parrot Drone we use in the demo has a 14 megapixel fisheye camera with advanced stabilization techniques which means that you can't actually see what happened when everybody went "ooh" - I assure you the "flip" command does exactly what you would expect!

Thanks to everybody who talked to us at Kamailio - and as always, come find us in the #matrix:matrix.org room on Matrix!

In the excitement of KamailioWorld last week we completely forgot to mention that we released Synapse 0.9.1.

This is a pretty important performance and stability update of all the new stuff that landed in 0.9.0, as well as landing a major new feature in the form of 'backfill': we finally have the ability to correctly sync in historical conversation history over federation for newly federated rooms. In other words, if you join a remote room you should be able to navigate its history as intended.

We highly recommend upgrading to 0.9.1 for all the performance improvements and backfill support - get it now from http://github.com/matrix-org/synapse.

Changes in synapse v0.9.1 (2015-05-26) ======================================

General:

• Add support for backfilling when a client paginates. This allows servers to request history for a room from remote servers when a client tries to paginate history the server does not have - SYN-36
• Fix bug where you couldn't disable non-default pushrules - SYN-378
• Fix register_new_user script - SYN-359
• Improve performance of fetching events from the database, this improves both initialSync and sending of events.
• Improve performance of event streams, allowing synapse to handle more simultaneous connected clients.

Federation:

• Fix bug with existing backfill implementation where it returned the wrong selection of events in some circumstances.
• Improve performance of joining remote rooms.

Configuration:

• Add support for changing the bind host of the metrics listener via the metrics_bind_host option.

In our continuous journey around the world to promote Matrix, this week we have come to Kamailio World in Berlin, Germany. During the conference, there will be 5 technical workshops and 28 presentations about SIP, VoIP, WebRTC and other real time communication technologies - and Matthew will talk about Matrix at 11am on Friday.

I'm looking forward to lots of interesting talks (full schedule here), including an open discussion panel with Randy Resnick about real-time communications at 17:10 Thursday evening. Of course there will also be dangerous demos - and hopefully lots of people interested in Matrix! If you are going to the conference, please come and say hello - we will be exhibiting as well as presenting, and we will be there all day Thursday and Friday.

Bis dann!

Amandine and I just got back from WebRTC World 2015 in Miami - the conference was a great success: a fantastic opportunity to meet up with many of the companies who are supporting Matrix and give everyone an update on what Matrix is up to with our Keynote: Defragmenting the Internet for fun and non-profit!.

We also had a little too much fun in the demo shoot-out - hooking up a Parrot Bebop quadrocopter into Matrix using a Matrix-enabled Janus WebRTC Gateway. The problem here is that the Parrot firmware and C SDK provides H.264 video, but doesn't package it up at all for use with WebRTC - let alone using interoperable signalling like Matrix. So this is a classic use of Matrix to expose a simple open consistent interface to a system which is otherwise is stuck with a proprietary non-web-friendly API. The code hasn't been tidied up yet, but our hacked Matrixified fork of Janus is up at https://github.com/matrix-org/janus-gateway/tree/ardrone3 if anyone has a drone and is crazy enough to want to experiment with it :)

Meanwhile, we also showed OpenWebRTC-powered VoIP on the latest develop iOS Matrix Console app talking hardware-accelerated H.264 through to Firefox on the desktop. My ancient 2010 MacBook Pro did its best to sabotage the demo (turns out that 1080P AirPlay + Firefox WebRTC is a bridge too far), but it gave a good idea of what's to come. Many thanks to the OpenWebRTC team for lots of help in getting the demo together in time!

It turns out that all the demo excitement was worth it in the end, as the jury seemed to like what Matrix is up to and was kind enough to award us more points than any of the other 13 demos... meaning that we won Best In Show!!. Huge thanks to the judges for believing in the Matrix vision, and congratulations to all the other demoists too :)

Meanwhile, the slides from the demo presentation can be found here: Building bridges between islands of communication, and you can see the full video of our Demo here:

...and the actual video stream that the drone transmitted before I crashed it (recorded on Janus) is at...

Finally, our grand finale was meant to be combining the two demos, and showing OpenWebRTC decoding the H.264 from the Drone in hardware on an iPhone - using Matrix of course to set up the call and control the drone. Alas a TURN-related bug got in the way of this working, but we just fixed it up in the office this morning, and I'm proud to show the first ever Parrot Bebop -> Janus -> Matrix -> OpenWebRTC video stream!! (and very exciting it is too...)

Huge thanks again to Dave for doing the Matrix integration with Janus, Stefan and Rob from OpenWebRTC for all the help on the OWR side, and Manu & Giom for porting the OpenWebRTC pull request to MatrixKit and landing it in iOS Console Develop for the demo!

We have pushed out a new release of both Synapse, our reference server implementation, and matrix-angular-sdk, our reference webclient implementation!

The major new feature in Synapse is that you can now run Synapse backed by a PostgreSQL database. This increases performance and allows Synapse to scale much better! This, as well as various performance related bug fixes, should make things much snappier than before. Of course, you can still run SQLite; it's up to you what you want to use.

In the webclient you can now change or reset your password - we have had this feature requested a few times (although honestly I'm surprised it hasn't been mentioned even more - maybe people are just better than me at remembering/managing their passwords) so this should be a welcome addition! We also fixed a memory leak in Angular, so again expect better performance!

Finally, we have done some work on improving the Application Service API, making it more reliable and secure. Please see the upgrade notes as well as the full changelog below.

Changes in Synapse v0.9.0:

General:

• Add support for using a PostgreSQL database instead of SQLite. See postgres.rst for details.
• Add password change and reset APIs. See Registration in the spec.
• Fix memory leak due to not releasing stale notifiers - SYN-339.
• Fix race in caches that occasionally caused some presence updates to be dropped - SYN-369.
• Check server name has not changed on restart.
• Add a sample systemd unit file and a logger configuration in contrib/systemd. Contributed Ivan Shapovalov.

Federation:

• Add key distribution mechanisms for fetching public keys of unavailable remote home servers. See Retrieving Server Keys in the spec.

Configuration:

• Add support for multiple config files.
• Add support for dictionaries in config files.
• Remove support for specifying config options on the command line, except for:
  • --daemonize - Daemonize the home server.
  • --manhole - Turn on the twisted telnet manhole service on the given port.
  • --database-path - The path to a sqlite database to use.
  • --verbose - The verbosity level.
  • --log-file - File to log to.
  • --log-config - Python logging config file.
  • --enable-registration - Enable registration for new users.

Application services:

• Reliably retry sending of events from Synapse to application services, as per Application Services spec.
• Application services can no longer register via the /register API, instead their configuration should be saved to a file and listed in the synapse app_service_config_files config option. The AS configuration file has the same format as the old /register request. See application_services.rst for more information.

Changes in Matrix Angular SDK 0.6.6:

Features:

• Add password change and reset feature using v2_alpha APIs.

Bug fixes:

• Fix memory leak caused by not removing a watcher on the root scope.

Matrix.org is happy to be sponsoring and talking at the WebRTC Conference and Expo in Miami, Florida, 12-14 May. Both Amandine and Matthew will be there - please come have a chat by booth #22! This is one of the longest running WebRTC Events, and Matthew is delivering one of the keynotes of the conference on Wednesday 4:00-4:30pm in room K-07.

Matthew will also participate in the "Open Source Options for WebRTC Development" session in room D2-02 at 9:50am on Wednesday (full agenda here).

Finally, Matrix will also be part of the WebRTC World Demos in room X-07 sometime between 4:30 and 7:30pm on Wednesday. Expect a dangerous demo!

Just a quick note to say thanks to everyone who came to talk to us at SMR9 yesterday. SMR is a great way for developers looking for jobs and startups needing engineers to have a chat.

We had a very busy day with plenty of people interested in Matrix and eager to join the team. We received a lot of CVs and will get back to you - but in the meantime please check out our code and come say hi in the Matrix HQ room, using any of these Matrix clients!

If you missed SMR, or just generally is interested in working for Matrix.org - please feel free to send your CV to us - we need all kinds of developers, with skills ranging from backend and frontend to mobile development!

As a follow-on to the previous post, Apple approved version 0.4.0 of the iOS Matrix Console app over the weekend - grab it now from the app store!. Whilst cosmetically it looks almost identical to 0.3.x, the architecture is completely different given the app is now built using MatrixKit - and stability has improved considerably as a result. Please install and let us know how you get on!

Historically we've had two projects for iOS:

• MatrixSDK: a low level library to interact with a Matrix homeserver
• Console: an example Matrix client based on MatrixSDK

The primary intention of Console was to demonstrate how to use MatrixSDK to write a Matrix client app. However, this split isn't helpful for developers who want higher level modules that provides UIViewControllers ready to use in an existing app, with no need to manage low level communications with the Matrix homeserver.

It is where the MatrixKit project started. MatrixKit sits between MatrixSDK and your existing iOS app.

It provides customisable UIViewControllers a developer can integrate in their app.  If you want to add to your app a screen to chat in a room, you just need to use the MXKRoomViewController.

We made MatrixKit so that the components it provides are easy to integrate but also easy to customise. We do not have yet full samples of customisation as we've been focused on the library core, but here are a few examples:

You probably recognise the theme of the first one, as it's what we use in the Console app today. The second one is the iOS7-style look and feel from JSQMessagesViewController. With few lines of code we connected it to MatrixKit data models. Yes, data models provided by MatrixKit are reusable too.

MatrixKit is also highly extensible. If you want to create new table cells to render messages, new views, new view controllers, etc, you will find a place to hook them into the MatrixKit code.

MatrixKit repository is here: https://github.com/matrix-org/matrix-ios-kit and it is available via CocoaPods (the MatrixKit pod).

In parallel of MatrixKit, we did some spring-cleaning - the official Matrix.org iOS offerings are now split into three github repos. One for each deliverable:

• https://github.com/matrix-org/matrix-ios-sdk
• https://github.com/matrix-org/matrix-ios-kit
• https://github.com/matrix-org/matrix-ios-console

Other releases:

Today, we released MatrixSDK 0.4.0 (changes). Update your pods :)

Console 0.4.0 (changes) is in the Apple submission process. This will be the first version of the app using MatrixKit. Aesthetically, there is no change since the previous version. The app is more stable due to all the data abstractions and management improvements provided by MatrixKit.

If you're an iOS developer, please have a go with MatrixKit and let us know on #ios:matrix.org how you get on!

Note: This blog post is outdated, and an up-to-date tutorial is located on the synapse project repo

Synapse has had support for exporting a comprehensive range of metrics via HTTP since 0.8.1 - we added this to help quantify the benefits of all the performance work which is going on currently in advance of Synapse 0.9. If you're interested in monitoring your own synapse and seeing what's going on using something like Prometheus, Leo just wrote a quick tutorial on getting up and running:

How to monitor Synapse metrics using Prometheus

1: Install prometheus:

Follow instructions at http://prometheus.io/docs/introduction/install/

2: Enable synapse metrics:

Simply setting a (local) port number will enable it. Pick a port. prometheus itself defaults to 9090, so starting just above that for locally monitored services seems reasonable. E.g. 9092:

Add to homeserver.yaml

metrics_port: 9092

Restart synapse

3: Check out synapse-prometheus-config

https://github.com/matrix-org/synapse-prometheus-config

4: Add synapse.html and synapse.rules

The .html file needs to appear in prometheus's consoles directory, and the .rules file needs to be invoked somewhere in the main config file. A symlink to each from the git checkout into the prometheus directory might be easiest to ensure git pull keeps it updated.

5: Add a prometheus target for synapse

This is easiest if prometheus runs on the same machine as synapse, as it can then just use localhost:

global: {'{'}
  rule_file: "synapse.rules"
{'}'}
job: {'{'}
name: "synapse"
target_group: {'{'}
target: "http://localhost:9092/"
{'}'}
{'}'}

6: Start prometheus:

./prometheus -config.file=prometheus.conf

7: Wait a few seconds for it to start and perform the first scrape,

then visit the console:

http://server-where-prometheus-runs:9090/consoles/synapse.html

And the end result looks something like...

...amongst many many other system & application metrics.

You can grab the latest version of the tutorial at https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md - thanks to Leo for writing it up. Any questions, let us know!