Micropub support as an Application Service!

29.07.2015 00:00 — Tech Kegan Dougal

I was at IndieWebCamp Edinburgh last week and during the hack day I created a Matrix Application Service (AS) which could act as an IndieWeb Micropub client. Any Matrix message sent to the AS (@micropub:domain) would be converted to a request to a Micropub endpoint.

This required the AS to support IndieAuth - which it does by sending !indieauth http://yourdomain.dom to @micropub:domain which then returns an OAuth2 URL to login via. Currently, the AS just supports a 1:1 mapping from m.text to h:entry but in the future, it can be expanded to include categories and potentially the reverse mapping (where Micropub clients can act as Matrix users!).

Overall, it was a great weekend and I look forward to adding more support for IndieWeb protocols in the future.

Synapse 0.9.3 is out! (as of a few weeks ago)

10.07.2015 00:00 — Tech Matthew Hodgson

Hi all,

We seem to have done that thing again where we were too busy writing new stuff (E2E crypto, all-new React-based Web SDK, long-awaited iOS Console update) to remember that we'd released a new version of Synapse - sorry!

Synapse 0.9.3 was released on July 1st (actually on June 23rd as 0.9.3-rc1, but was released without changes). It's a fairly minor release but does provide some performance improvements and bug fixes - see below for details. Get it from http://github.com/matrix-org/synapse if you haven't already.

Changes in synapse v0.9.3 (2015-07-01) ======================================

General:

  • Fix a memory leak in the notifier. (SYN-412)
  • Improve performance of room initial sync. (SYN-418)
  • General improvements to logging.
  • Remove access_token query params from INFO level logging.

Configuration:

  • Add support for specifying and configuring multiple listeners. (SYN-389)

Application services:

  • Fix bug where synapse failed to send user queries to application services.

Android Matrix Console 0.4.2 & 0.4.3

07.07.2015 00:00 — General Oddvar Lovaas

In the spirit of releasing early and often, we have pushed Android Matrix Console 0.4.2 to the Google Play store.

This release comes with a few new features as well as many bugfixes:

  • A notification settings page has been added
  • Added an image slider (access by tapping an image)
  • Improved the management of multiple accounts
  • Retaining filenames on upload
  • Fixed the causes of two crashes that were reported via Google Analytics
  • Several bugfixes - see full changelog here

Update:

We have pushed another update, version 0.4.3, after a bug was reported that affected some users upgrading to 0.4.2:

  • Fixed a bug where updating to 0.4.2 caused the history list to be empty
  • Added presence information on avatars in rooms

Android Matrix Console 0.4.1

30.06.2015 00:00 — General Oddvar Lovaas

Super-quick post just to announce that we have released a new version of the Android Matrix Console. This version fixes a problem where the Playstore wouldn't let some Android devices install the app just because they don't have a SIM card, due to a required permission that wasn't really needed anyway.

Grab the latest version from the play store!

The matrix.org IRC bridge now bridges all of Freenode!

22.06.2015 00:00 — Tech Matthew Hodgson

We've been running an IRC gateway from Matrix to Freenode almost since day 1 - originally it was a simple perl bot written by tm604 using Net::Async::Matrix; later it was rewritten by LeoNerd to be a bit more modular, and nowadays it's a full IRC/Matrix Application Service written in Node.js by Kegan.

Up until the end of last week the bridge was limited to synchronising a fixed number of channels between Freenode and Matrix (#matrix, #matrix-dev, #openwebrtc, #vuc and #hypothes.is, to be precise), as well as any PMs. But as of Friday, with huge thanks to the admins over at Freenode, we can now bridge any channel in Freenode through to Matrix.

Doing this is trivial - you just /join #freenode_#channelname:matrix.org from a typical Matrix client - e.g. /join #freenode_#Node.js:matrix.org will give you access to all of #Node.js via Matrix, effectively using Matrix as a great big distributed IRC bouncer in the sky ;)

There are a few limitations in the current setup:

  • We only incrementally synchronise the membership lists when folks speak in either IRC or Matrix. This is to avoid flooding either IRC or Matrix with lurkers when the bridge initially joins a channel. We have some plans to improve this in future - see BOTS-53 for some of the sordid details.
  • We don't synchronise joins/parts currently to avoid flooding Matrix with lots of IRC join/part spam. Again, this will improve in future.
  • You can't join +k channels.
  • Kicks/bans/invites and other ACLs currently don't propagate between IRC & Matrix. (I.e. a Matrix user can be kicked from IRC, but it'll continue to sit in Matrix unless also kicked for there). +i chanmode supported however.
  • Bridged public IRC rooms are not yet advertised in the public room list on matrix.org.
  • It should be possible to change the nickname of your IRC user by messaging !nick irc.freenode.net MrFlibble to @appservice-irc:matrix.org. (This isn't actually turned on right now, but should be fixed shortly. See https://github.com/matrix-org/matrix-appservice-irc/blob/master/HOWTO.md#changing-nicks for details.

We'll keep chipping away at making the IRC<->Matrix mapping perfect, but in its current state it's still really usable. It's obviously beta still, but please give it a go and let us know in #matrix:matrix.org how you get on with it!

Android Matrix Console 0.4.0

19.06.2015 00:00 — Tech Oddvar Lovaas

Matrix Console 0.4.0 for Android has been pushed to the Google Play store and shall be available shortly.

This release contains a whole lot of bugfixes, new features, and nicer UI and UX - see the changes file for details, but in summary:

  • Lots of UI and UX improvements (inc. long click on images)
  • Added GCM support (can be enabled/disabled)
  • Added Google analytics support
  • Added badges management
  • Added "orientation" management for images
  • Now offering image resizing before upload
  • Lots of bugfixes!

Global TADHack Matrix hacks

16.06.2015 00:00 — Events Oddvar Lovaas

Over 800 hackers all over the world spent this weekend working on hacks and prototypes in the Global TADHack hackathon. Some were working from home, and others gathered at one of the 14 official locations. Matrix were physically present at the London and Lisbon locations, and we also handed out two prizes (a massive amount of tessel modules) to the best hacks using Matrix in some way or another.

After a hectic weekend, over 90 different hacks were submitted and presented. There was a live stream available (archive here) so that everyone could watch all the submissions. We were of course particularly interested in Matrix-related hacks, and there were several:

  • Matt Williams continued his Project Clearwater/Matrix integration from TADHack-mini London earlier this year; this time he added support for IM{'<->'}SIP MESSAGEs.
  • Rob Pickering bridged his company's keevio system to Matrix via the AS API.
  • Sacha Nacar and Mike Dauphinais wrote a sign-language interpreter that uses Matrix for conversations.
  • Igor Pavlov made it possible for users of Lytespark to be contacted via Matrix and OpenMarket's SMS AS when they are offline.
  • Phani Pasupula showed a PoC for using Matrix as a platform for starting a Dialogic XMS conference.

In the end, we had to pick only two of these awesome hacks, which wasn't easy. We are really encouraged by seeing the Matrix ecosystem grow bigger - and more bridges to various existing systems being built; this is exactly what Matrix needs!

We awarded our two prizes to Igor for his use of OpenMarket's SMS AS, which adds a new and very wanted feature to Lytespark - and Sacha/Mike for their inventive sign-language -> Matrix translator. Congrats to all of you! We were also happy to see Phani win one of Dialogic's prizes with his hack!

Below you can watch our two winners' presentations:

The TADHack blog has a list of all the winners: local winners and global winners. In total $36k worth of prizes were won!

Thanks to everyone who participated!

Introducing all new matrix-js-sdk v0.1.0

12.06.2015 00:00 — Tech Matthew Hodgson

Hi all,

If things seem a bit quiet here at the moment it's because most of the Matrix team is in the 'build' phase across the board on a bunch of major new projects - including the long-awaited reusable UI component library for the Web, end-to-end encryption support, stable VoIP on mobile and a few other surprises. The first wave of all this new work landed today in the form of a new major version release (0.1.0) of matrix-js-sdk: our javascript SDK library.

The history of JS client support on Matrix is that in the original rush to get a PoC webclient out the door back in September, we jumped straight to talking to the Matrix HTTP API from Angular controllers in 'syweb'. This then got split into an Angular SDK (matrix-angular-sdk) containing a bunch of services for handling the Matrix client state with the actual webapp sitting alongside as an example of the SDK. This then in turn got split into a basic matrix-js-sdk (wrapping the Matrix client-server HTTP API) and matrix-angular-sdk (handling the clientside state and exposing it as Angular services) - and this is how the current demo Angular webapp works at http://matrix.org/beta.

However, this poses a major problem if you want a richer client SDK but don't want to use Angular - e.g. if you prefer another framework (or no framework at all), or are using the SDK in an Application Service or similar. So, this has prompted a major upgrade to the matrix-js-sdk in order to extend it to act both as a simple wrapper of the Matrix Client-Server HTTP API, but also expose a full object and event model to describe the state of a Matrix client together with all the higher level functions to drive it. To quote from the README:

This SDK provides a full object model around the Matrix Client-Server API and emits events for incoming data and state changes. Aside from wrapping the HTTP API, it:

  • Handles syncing (via /initialSync and /events)
  • Handles the generation of "friendly" room and member names.
  • Handles historical RoomMember information (e.g. display names).
  • Manages room member state across multiple events (e.g. it handles typing, power levels and membership changes).
  • Exposes high-level objects like Rooms, RoomState, RoomMembers and Users which can be listened to for things like name changes, new messages, membership changes, presence changes, and more.

Later versions of the SDK will:

  • Automatically retry requests to send messages due to network errors.
  • Automatically retry requests to send messages due to rate limiting errors.
  • Mark events' sent status (e.g. 'not sent').
  • Handle "local echo" of messages sent.
  • Handle queueing of messages.
  • Handle pagination.
  • Expose a RoomSummary which would be suitable for a recents page.
  • Provide different pluggable storage layers (e.g. local storage, database-backed)

It also is designed to provide an API that abstracts both version 1 of the client-server API and the upcoming version 2 (which fixes various thinkos we've discovered along the way in v1).

This is incredibly exciting stuff as it now gives a really robust yet lightweight API for client-side and application-service development in JavaScript. Predictably enough we're using it to develop our new reusable web UI components, and I suspect we'll see new Node-based application services making use of in the near future. Meanwhile, it's more than topical as it should run straight on top of the Tessel JavaScript hardware modules we're giving away this weekend as our prize for the best hack built on Matrix at TADHack - letting the little Tessel ARM system-on-a-chip speak a very rich and native Matrix dialect!

The new matrix-js-sdk comes with comprehensive jsdoc detailing the new API which you can find at http://matrix-org.github.io/matrix-js-sdk/global.html. There are also some basic examples available, including a new simple node-powered command-line client designed for testing the SDK. Finally, if you're interested in reading the design process by which we've converged on the current API, all the details may be found in JIRA on SYJS-5.

So: if you've been itching to get your hands on a much more powerful pure-JS Matrix SDK, now's your chance! It's very early days for the new API so we expect to see a few incompatible changes on the horizon, but we encourage you to have a play and tell us what you think!.

To find out more, go check out the project at http://github.com/matrix-org/matrix-js-sdk, or if you're feeling daring just grab the library with npm install matrix-js-sdk and get hacking!

Synapse 0.9.2 released

12.06.2015 00:00 — General Erik Johnston

Happy Friday everyone!

Over the past two weeks, we have been hunting down some more performance issues in Synapse, as well as fixing a few potential bugs in the new backfill feature that we introduced in 0.9.1. For those that were having issues, this release should really help speed up when your server joins larger remote rooms.

We have also been busy hacking on end-to-end encryption, which is very exciting. Hopefully we will have more details to share about that soon!

Get v0.9.2 now from http://github.com/matrix-org/synapse.

Changes in synapse v0.9.2 (2015-06-12) ======================================

General:

  • Use ultrajson for json (de)serialisation when a canonical encoding is not required. Ultrajson is significantly faster than simplejson in certain circumstances.
  • Use connection pools for outgoing HTTP connections.
  • Process thumbnails on separate threads.

Configuration:

  • Add option, gzip_responses, to disable HTTP response compression.

Federation:

  • Improve resilience of backfill by ensuring we fetch any missing auth events.
  • Improve performance of backfill and joining remote rooms by removing unnecessary computations. This included handling events we had previously handled as well as attempting to compute the current state for outliers.

Global TADHack hackathon

05.06.2015 00:00 — General Oddvar Lovaas

TADHack2015-global-banner-460x860

Next weekend, June 13 and 14, the global TADHack takes place all over the world. You can participate on site or remotely, and there are a lot of different prizes to be won - in total the prize pot is worth $35k!

For the best two hacks using our technology, we will award a whole lot of Tessel modules! Tessel is a new breed of development board that runs entirely on Node.js, and they come with different modules you can plug in - for more information, see: getting started & sample projects.

Both prizes will include several tessel modules, including:

tessel

  • multiple core tessel boards
  • multiple servo modules and many servo motors
  • multiple ambient modules
  • multiple accelerometer modules
  • camera module
  • GPS module with antenna
  • microsd module
  • bluetooth module
  • audio module
  • climate module
  • relay module
  • RFID module
  • DIY module kit

Matrix.org will be present at the London site, Idea London in Shoreditch, where we will help both local and remote participants (via #matrix:matrix.org) using the Matrix APIs as part of their hacks.

So if you have some spare time next weekend - why not have a think about what could be a cool hack and join us for the global TADHack event! See you there!