Android Matrix Console 0.4.0

19.06.2015 00:00 — TechOddvar Lovaas

Matrix Console 0.4.0 for Android has been pushed to the Google Play store and shall be available shortly.

This release contains a whole lot of bugfixes, new features, and nicer UI and UX - see the changes file for details, but in summary:

  • Lots of UI and UX improvements (inc. long click on images)
  • Added GCM support (can be enabled/disabled)
  • Added Google analytics support
  • Added badges management
  • Added "orientation" management for images
  • Now offering image resizing before upload
  • Lots of bugfixes!

Global TADHack Matrix hacks

16.06.2015 00:00 — EventsOddvar Lovaas

Over 800 hackers all over the world spent this weekend working on hacks and prototypes in the Global TADHack hackathon. Some were working from home, and others gathered at one of the 14 official locations. Matrix were physically present at the London and Lisbon locations, and we also handed out two prizes (a massive amount of tessel modules) to the best hacks using Matrix in some way or another.

After a hectic weekend, over 90 different hacks were submitted and presented. There was a live stream available (archive here) so that everyone could watch all the submissions. We were of course particularly interested in Matrix-related hacks, and there were several:

  • Matt Williams continued his Project Clearwater/Matrix integration from TADHack-mini London earlier this year; this time he added support for IM{'<->'}SIP MESSAGEs.
  • Rob Pickering bridged his company's keevio system to Matrix via the AS API.
  • Sacha Nacar and Mike Dauphinais wrote a sign-language interpreter that uses Matrix for conversations.
  • Igor Pavlov made it possible for users of Lytespark to be contacted via Matrix and OpenMarket's SMS AS when they are offline.
  • Phani Pasupula showed a PoC for using Matrix as a platform for starting a Dialogic XMS conference.

In the end, we had to pick only two of these awesome hacks, which wasn't easy. We are really encouraged by seeing the Matrix ecosystem grow bigger - and more bridges to various existing systems being built; this is exactly what Matrix needs!

We awarded our two prizes to Igor for his use of OpenMarket's SMS AS, which adds a new and very wanted feature to Lytespark - and Sacha/Mike for their inventive sign-language -> Matrix translator. Congrats to all of you! We were also happy to see Phani win one of Dialogic's prizes with his hack!

Below you can watch our two winners' presentations:

The TADHack blog has a list of all the winners: local winners and global winners. In total $36k worth of prizes were won!

Thanks to everyone who participated!

Introducing all new matrix-js-sdk v0.1.0

12.06.2015 00:00 — TechMatthew Hodgson

Hi all,

If things seem a bit quiet here at the moment it's because most of the Matrix team is in the 'build' phase across the board on a bunch of major new projects - including the long-awaited reusable UI component library for the Web, end-to-end encryption support, stable VoIP on mobile and a few other surprises. The first wave of all this new work landed today in the form of a new major version release (0.1.0) of matrix-js-sdk: our javascript SDK library.

The history of JS client support on Matrix is that in the original rush to get a PoC webclient out the door back in September, we jumped straight to talking to the Matrix HTTP API from Angular controllers in 'syweb'. This then got split into an Angular SDK (matrix-angular-sdk) containing a bunch of services for handling the Matrix client state with the actual webapp sitting alongside as an example of the SDK. This then in turn got split into a basic matrix-js-sdk (wrapping the Matrix client-server HTTP API) and matrix-angular-sdk (handling the clientside state and exposing it as Angular services) - and this is how the current demo Angular webapp works at http://matrix.org/beta.

However, this poses a major problem if you want a richer client SDK but don't want to use Angular - e.g. if you prefer another framework (or no framework at all), or are using the SDK in an Application Service or similar. So, this has prompted a major upgrade to the matrix-js-sdk in order to extend it to act both as a simple wrapper of the Matrix Client-Server HTTP API, but also expose a full object and event model to describe the state of a Matrix client together with all the higher level functions to drive it. To quote from the README:

This SDK provides a full object model around the Matrix Client-Server API and emits events for incoming data and state changes. Aside from wrapping the HTTP API, it:

  • Handles syncing (via /initialSync and /events)
  • Handles the generation of "friendly" room and member names.
  • Handles historical RoomMember information (e.g. display names).
  • Manages room member state across multiple events (e.g. it handles typing, power levels and membership changes).
  • Exposes high-level objects like Rooms, RoomState, RoomMembers and Users which can be listened to for things like name changes, new messages, membership changes, presence changes, and more.

Later versions of the SDK will:

  • Automatically retry requests to send messages due to network errors.
  • Automatically retry requests to send messages due to rate limiting errors.
  • Mark events' sent status (e.g. 'not sent').
  • Handle "local echo" of messages sent.
  • Handle queueing of messages.
  • Handle pagination.
  • Expose a RoomSummary which would be suitable for a recents page.
  • Provide different pluggable storage layers (e.g. local storage, database-backed)

It also is designed to provide an API that abstracts both version 1 of the client-server API and the upcoming version 2 (which fixes various thinkos we've discovered along the way in v1).

This is incredibly exciting stuff as it now gives a really robust yet lightweight API for client-side and application-service development in JavaScript. Predictably enough we're using it to develop our new reusable web UI components, and I suspect we'll see new Node-based application services making use of in the near future. Meanwhile, it's more than topical as it should run straight on top of the Tessel JavaScript hardware modules we're giving away this weekend as our prize for the best hack built on Matrix at TADHack - letting the little Tessel ARM system-on-a-chip speak a very rich and native Matrix dialect!

The new matrix-js-sdk comes with comprehensive jsdoc detailing the new API which you can find at http://matrix-org.github.io/matrix-js-sdk/global.html. There are also some basic examples available, including a new simple node-powered command-line client designed for testing the SDK. Finally, if you're interested in reading the design process by which we've converged on the current API, all the details may be found in JIRA on SYJS-5.

So: if you've been itching to get your hands on a much more powerful pure-JS Matrix SDK, now's your chance! It's very early days for the new API so we expect to see a few incompatible changes on the horizon, but we encourage you to have a play and tell us what you think!.

To find out more, go check out the project at http://github.com/matrix-org/matrix-js-sdk, or if you're feeling daring just grab the library with npm install matrix-js-sdk and get hacking!

Synapse 0.9.2 released

12.06.2015 00:00 — GeneralErik Johnston

Happy Friday everyone!

Over the past two weeks, we have been hunting down some more performance issues in Synapse, as well as fixing a few potential bugs in the new backfill feature that we introduced in 0.9.1. For those that were having issues, this release should really help speed up when your server joins larger remote rooms.

We have also been busy hacking on end-to-end encryption, which is very exciting. Hopefully we will have more details to share about that soon!

Get v0.9.2 now from http://github.com/matrix-org/synapse.

Changes in synapse v0.9.2 (2015-06-12) ======================================

General:

  • Use ultrajson for json (de)serialisation when a canonical encoding is not required. Ultrajson is significantly faster than simplejson in certain circumstances.
  • Use connection pools for outgoing HTTP connections.
  • Process thumbnails on separate threads.

Configuration:

  • Add option, gzip_responses, to disable HTTP response compression.

Federation:

  • Improve resilience of backfill by ensuring we fetch any missing auth events.
  • Improve performance of backfill and joining remote rooms by removing unnecessary computations. This included handling events we had previously handled as well as attempting to compute the current state for outliers.

Global TADHack hackathon

05.06.2015 00:00 — GeneralOddvar Lovaas

TADHack2015-global-banner-460x860

Next weekend, June 13 and 14, the global TADHack takes place all over the world. You can participate on site or remotely, and there are a lot of different prizes to be won - in total the prize pot is worth $35k!

For the best two hacks using our technology, we will award a whole lot of Tessel modules! Tessel is a new breed of development board that runs entirely on Node.js, and they come with different modules you can plug in - for more information, see: getting started & sample projects.

Both prizes will include several tessel modules, including:

tessel

  • multiple core tessel boards
  • multiple servo modules and many servo motors
  • multiple ambient modules
  • multiple accelerometer modules
  • camera module
  • GPS module with antenna
  • microsd module
  • bluetooth module
  • audio module
  • climate module
  • relay module
  • RFID module
  • DIY module kit

Matrix.org will be present at the London site, Idea London in Shoreditch, where we will help both local and remote participants (via #matrix:matrix.org) using the Matrix APIs as part of their hacks.

So if you have some spare time next weekend - why not have a think about what could be a cool hack and join us for the global TADHack event! See you there!

Matrix wins "Most Entertaining Demo" at Kamailio World!

01.06.2015 00:00 — GeneralOddvar Lovaas

We are back from Kamailio World, where we presented and participated in James Body's "Dangerous Demos". We were racing against the deadline, but managed to join the demos at the very last minute - and even win the award for "Most Entertaining Demo"!

mostentertainingdemo

It was great to catch up with old acquaintances - and meet many new ones! There were only around 150 people at Kamailio World, but given the area of expertise is very specialised, you can pretty much start talking to anyone and have a really interesting conversation.

Here are the slides of Matthew's presentation, (also available as .pdf) and a video of the presentation:

A video from the dangerous demo event is available here:

The Parrot Drone we use in the demo has a 14 megapixel fisheye camera with advanced stabilization techniques which means that you can't actually see what happened when everybody went "ooh" - I assure you the "flip" command does exactly what you would expect!

Thanks to everybody who talked to us at Kamailio - and as always, come find us in the #matrix:matrix.org room on Matrix!

Synapse 0.9.1 released

01.06.2015 00:00 — TechMatthew Hodgson

In the excitement of KamailioWorld last week we completely forgot to mention that we released Synapse 0.9.1.

This is a pretty important performance and stability update of all the new stuff that landed in 0.9.0, as well as landing a major new feature in the form of 'backfill': we finally have the ability to correctly sync in historical conversation history over federation for newly federated rooms. In other words, if you join a remote room you should be able to navigate its history as intended.

We highly recommend upgrading to 0.9.1 for all the performance improvements and backfill support - get it now from http://github.com/matrix-org/synapse.

Changes in synapse v0.9.1 (2015-05-26) ======================================

General:

  • Add support for backfilling when a client paginates. This allows servers to request history for a room from remote servers when a client tries to paginate history the server does not have - SYN-36
  • Fix bug where you couldn't disable non-default pushrules - SYN-378
  • Fix register_new_user script - SYN-359
  • Improve performance of fetching events from the database, this improves both initialSync and sending of events.
  • Improve performance of event streams, allowing synapse to handle more simultaneous connected clients.

Federation:

  • Fix bug with existing backfill implementation where it returned the wrong selection of events in some circumstances.
  • Improve performance of joining remote rooms.

Configuration:

  • Add support for changing the bind host of the metrics listener via the metrics_bind_host option.

Next up... Kamailio World!

26.05.2015 00:00 — EventsOddvar Lovaas

kamailio-world-banner-2014-200x90In our continuous journey around the world to promote Matrix, this week we have come to Kamailio World in Berlin, Germany. During the conference, there will be 5 technical workshops and 28 presentations about SIP, VoIP, WebRTC and other real time communication technologies - and Matthew will talk about Matrix at 11am on Friday.

I'm looking forward to lots of interesting talks (full schedule here), including an open discussion panel with Randy Resnick about real-time communications at 17:10 Thursday evening. Of course there will also be dangerous demos - and hopefully lots of people interested in Matrix! If you are going to the conference, please come and say hello - we will be exhibiting as well as presenting, and we will be there all day Thursday and Friday.

Bis dann!

Matrix wins Best of Show at WebRTC World!

18.05.2015 00:00 — In the NewsMatthew Hodgson

WebRTC Best in Show

Amandine and I just got back from WebRTC World 2015 in Miami - the conference was a great success: a fantastic opportunity to meet up with many of the companies who are supporting Matrix and give everyone an update on what Matrix is up to with our Keynote: Defragmenting the Internet for fun and non-profit!.

We also had a little too much fun in the demo shoot-out - hooking up a Parrot Bebop quadrocopter into Matrix using a Matrix-enabled Janus WebRTC Gateway. The problem here is that the Parrot firmware and C SDK provides H.264 video, but doesn't package it up at all for use with WebRTC - let alone using interoperable signalling like Matrix. So this is a classic use of Matrix to expose a simple open consistent interface to a system which is otherwise is stuck with a proprietary non-web-friendly API. The code hasn't been tidied up yet, but our hacked Matrixified fork of Janus is up at https://github.com/matrix-org/janus-gateway/tree/ardrone3 if anyone has a drone and is crazy enough to want to experiment with it :)

Meanwhile, we also showed OpenWebRTC-powered VoIP on the latest develop iOS Matrix Console app talking hardware-accelerated H.264 through to Firefox on the desktop. My ancient 2010 MacBook Pro did its best to sabotage the demo (turns out that 1080P AirPlay + Firefox WebRTC is a bridge too far), but it gave a good idea of what's to come. Many thanks to the OpenWebRTC team for lots of help in getting the demo together in time!

It turns out that all the demo excitement was worth it in the end, as the jury seemed to like what Matrix is up to and was kind enough to award us more points than any of the other 13 demos... meaning that we won Best In Show!!. Huge thanks to the judges for believing in the Matrix vision, and congratulations to all the other demoists too :)

Best in Show at WebRTC World!

Meanwhile, the slides from the demo presentation can be found here: Building bridges between islands of communication, and you can see the full video of our Demo here:

...and the actual video stream that the drone transmitted before I crashed it (recorded on Janus) is at...

Finally, our grand finale was meant to be combining the two demos, and showing OpenWebRTC decoding the H.264 from the Drone in hardware on an iPhone - using Matrix of course to set up the call and control the drone. Alas a TURN-related bug got in the way of this working, but we just fixed it up in the office this morning, and I'm proud to show the first ever Parrot Bebop -> Janus -> Matrix -> OpenWebRTC video stream!! (and very exciting it is too...)

Huge thanks again to Dave for doing the Matrix integration with Janus, Stefan and Rob from OpenWebRTC for all the help on the OWR side, and Manu & Giom for porting the OpenWebRTC pull request to MatrixKit and landing it in iOS Console Develop for the demo!

Announcing Synapse 0.9.0 and Matrix Angular SDK 0.6.6!

11.05.2015 00:00 — GeneralOddvar Lovaas

We have pushed out a new release of both Synapse, our reference server implementation, and matrix-angular-sdk, our reference webclient implementation!

The major new feature in Synapse is that you can now run Synapse backed by a PostgreSQL database. This increases performance and allows Synapse to scale much better! This, as well as various performance related bug fixes, should make things much snappier than before. Of course, you can still run SQLite; it's up to you what you want to use.

In the webclient you can now change or reset your password - we have had this feature requested a few times (although honestly I'm surprised it hasn't been mentioned even more - maybe people are just better than me at remembering/managing their passwords) so this should be a welcome addition! We also fixed a memory leak in Angular, so again expect better performance!

Finally, we have done some work on improving the Application Service API, making it more reliable and secure. Please see the upgrade notes as well as the full changelog below.

Changes in Synapse v0.9.0:

General:

  • Add support for using a PostgreSQL database instead of SQLite. See postgres.rst for details.
  • Add password change and reset APIs. See Registration in the spec.
  • Fix memory leak due to not releasing stale notifiers - SYN-339.
  • Fix race in caches that occasionally caused some presence updates to be dropped - SYN-369.
  • Check server name has not changed on restart.
  • Add a sample systemd unit file and a logger configuration in contrib/systemd. Contributed Ivan Shapovalov.

Federation:

  • Add key distribution mechanisms for fetching public keys of unavailable remote home servers. See Retrieving Server Keys in the spec.

Configuration:

  • Add support for multiple config files.
  • Add support for dictionaries in config files.
  • Remove support for specifying config options on the command line, except for:
    • --daemonize - Daemonize the home server.
    • --manhole - Turn on the twisted telnet manhole service on the given port.
    • --database-path - The path to a sqlite database to use.
    • --verbose - The verbosity level.
    • --log-file - File to log to.
    • --log-config - Python logging config file.
    • --enable-registration - Enable registration for new users.

Application services:

  • Reliably retry sending of events from Synapse to application services, as per Application Services spec.
  • Application services can no longer register via the /register API, instead their configuration should be saved to a file and listed in the synapse app_service_config_files config option. The AS configuration file has the same format as the old /register request. See application_services.rst for more information.

Changes in Matrix Angular SDK 0.6.6:

Features:

  • Add password change and reset feature using v2_alpha APIs.

Bug fixes:

  • Fix memory leak caused by not removing a watcher on the root scope.