by Neil Johnson | Nov 20, 2018 | Uncategorized |
Well here you are then, the 9th episode in the Synapse 0.33.x series.
Features wise, 0.33.9 contains a change to the way that GDPR consent works under the hood. It is now plumbed in to the login flow (rather than following immediately afterwards) such that it does not inadvertently break on-boarding. This is part of a broader set of changes that span Synapse and Riot to improve initial first impressions of using matrix.
Separately we now have support for room version upgrades which is pre-requisite for rolling out the new state resolution algorithm, come and join us in #teststateresv2:jki.re if you would like to help us test.
Finally we’ve spent a bunch of time further improving perf especially in and around reducing device ids federation traffic.
I know I say it every time, but full python 3 support is really really close now, matrix.org is now running entirely on py3 and seeing some amazing perf improvements – the remaining blocker is getting py3 deb packages ready and then we’ll ship an official python 3 release. There will also be a blog post to explain what we’ve been up to and what to expect perf wise.
As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here. Also, check out our new Synapse installation guide page.
Synapse 0.33.9 changelog
Features
- Include flags to optionally add
m.login.terms to the registration flow when consent tracking is enabled. (#4004, #4133, #4142, #4184)
- Support for replacing rooms with new ones (#4091, #4099, #4100, #4101)
Bugfixes
- Fix exceptions when using the email mailer on Python 3. (#4095)
- Fix e2e key backup with more than 9 backup versions (#4113)
- Searches that request profile info now no longer fail with a 500. (#4122)
- fix return code of empty key backups (#4123)
- If the typing stream ID goes backwards (as on a worker when the master restarts), the worker’s typing handler will no longer erroneously report rooms containing new typing events. (#4127)
- Fix table lock of device_lists_remote_cache which could freeze the application (#4132)
- Fix exception when using state res v2 algorithm (#4135)
- Generating the user consent URI no longer fails on Python 3. (#4140, #4163)
- Loading URL previews from the DB cache on Postgres will no longer cause Unicode type errors when responding to the request, and URL previews will no longer fail if the remote server returns a Content-Type header with the chartype in quotes. (#4157)
- The hash_password script now works on Python 3. (#4161)
- Fix noop checks when updating device keys, reducing spurious device list update notifications. (#4164)
Deprecations and Removals
- The disused and un-specced identicon generator has been removed. (#4106)
- The obsolete and non-functional /pull federation endpoint has been removed. (#4118)
- The deprecated v1 key exchange endpoints have been removed. (#4119)
- Synapse will no longer fetch keys using the fallback deprecated v1 key exchange method and will now always use v2. (#4120)
Internal Changes
- Fix build of Docker image with docker-compose (#3778)
- Delete unreferenced state groups during history purge (#4006)
- The “Received rdata” log messages on workers is now logged at DEBUG, not INFO. (#4108)
- Reduce replication traffic for device lists (#4109)
- Fix
synapse_replication_tcp_protocol_*_commands metric label to be full command name, rather than just the first character (#4110)
- Log some bits about room creation (#4121)
- Fix
tox failure on old systems (#4124)
- Add STATE_V2_TEST room version (#4128)
- Clean up event accesses and tests (#4137)
- The default logging config will now set an explicit log file encoding of UTF-8. (#4138)
- Add helpers functions for getting prev and auth events of an event (#4139)
- Add some tests for the HTTP pusher. (#4149)
- add purge_history.sh and purge_remote_media.sh scripts to contrib/ (#4155)
- HTTP tests have been refactored to contain less boilerplate. (#4156)
- Drop incoming events from federation for unknown rooms (#4165)
by Nad Chishtie | Nov 2, 2018 | General |
It’s been a long-standing goal to enable end-to-end encryption by default for private communication in Matrix. The technical effort so far has included our libolm library, an independent cryptographic review and a massive backlog of feature development and bug fixes. Today, instead I’d like to focus on some of the User Experience challenges and goals we’re facing.
I should also introduce myself—I’m Nad Chishtie (@nadonomy:matrix.org) and I recently joined the Matrix core team (at New Vector) as Lead Designer, most recently focusing on end-to-end encryption.
When using encrypted messages, most existing services fall short in one or all of the following:
- They don’t allow you to use multiple devices independently. For example, a web session might be locally tethered to a mobile device.
- They don’t support a way to restore or temporarily access message history. For example, if you don’t have physical access to your main device because it’s broken or has been stolen.
- They don’t allow you to verify that devices are controlled by their owners rather than eavesdroppers, and persist that trust across multiple devices, sessions or rooms.
Modern users, even those we talk to at security and privacy-led organisations, expect these features to ‘just work’ by default out of the box. Before enabling end-to-end encryption by default, we’ve been hard at work figuring out how we can deliver these features without compromising security or usability.
(For some users, restrictions such as limiting the number of places encryption keys reside, and not having a synchronised message history may be desirable security features. We’ll support these cases, but just not as the default behaviour.)
Let’s dive in to some of the fundamental concepts we’ll be putting forward to deliver a default end-to-end encryption experience that makes sense for most modern users. In this post we’ll look at an overview of work-in-progress wireframes, in the spirit of designing in the open and gathering feedback from the wider Matrix community. Please note that these don’t represent the actual interface design.
Cross-signing personal devices
When logging in to a new device, you’ll be able to use an existing device to verify your new one. Verification is done by scanning a QR code on whichever device has the most convenient camera to use, or by comparing a short text string. You only have to complete this process once to mutually verify both devices.
Verifying your new device by cross-signing transfers encryption keys, giving it access to your encrypted messages, and also signals to other users that the new device is trustworthy.
Secure Message Recovery
To the end user, Secure Message Recovery works a lot like setting up disk encryption or a password manager. A user can optionally secure their message history using a recovery passphrase and/or key. If logged out, or using another device, the user can use the recovery passphrase or key to access their encrypted message history.
In practise, this incrementally encrypts and backs up encryption keys to a user’s homeserver, kept secure by the homeserver never having access to the passphrase or key. Like cross-signing, using a recovery passphrase or key will also signal to other users that a device is trustworthy.
We think that in most cases users will cross-sign personal devices, but as a safety net (for example, if a user’s devices are broken or lost) Secure Message Recovery is an invaluable tool for users to minimise the chance of them losing their encrypted message history.
People should trust people
With both cross-signing and Secure Message Recovery in place, we think that people should trust people, instead of individual devices. Now, when you verify a device, it’ll mark all of that users trusted devices as trusted.
Gone are the days of every person you talk to having to independently verify your new device upgrade. Like cross-signing, you can verify a device by scanning a QR code or comparing a short text string.
Sensible and extensible
In Riot, we’re implementing these features with a sensible default experience that strikes a balance between usability and security. We think most people would prefer to trust cross-signed devices, and that user trust shouldn’t block encryption. However, if you aren’t most people, you’ll be free to configure whatever level of security you need.
In Summary
With all of the above in place, and after resolving any remaining technical issues, users will be able to:
- Use end-to-end encryption by default in private rooms.
- Use an existing device or Secure Message Recovery to access their encrypted message history on multiple devices, and to signal device trust to other users.
- Access their encrypted message history using Secure Message Recovery, by storing encrypted message keys on their homeserver.
- Mark a user as trusted by verifying one of their devices, persisting across all rooms and devices.
- Keep their encrypted messages out of the hands of eavesdroppers.
- Opt out, or further configure if they have more specific security requirements.
There’s more nuance to making all this work than we can cover in this overview post; things like recovery key management and immutable security notifications are all important pieces of the puzzle. For further reading, we’re filling up more detail in UX reference documentation, interactive wireframes, GitHub issues and a work-in-progress threat model.
Over the coming days we’re polishing wireframes, nomenclature, iconography and microcopy as we dig deeper into development and implementation, as well as designing these features for the upcoming Riot redesign. Cryptography needn’t be intimidating, and we’re excited to iterate on these advanced features to make them work for everyone.
We’d love to hear your feedback! Let us know your thoughts here or in #e2e-dev:matrix.org.
by Neil Johnson | Nov 1, 2018 | Uncategorized |
Wowzers – our 8th dot release for v0.33!
This time we have a bunch of bug fixes and db performance improvements as well as better support for auto-join rooms and the ability for admins to limit who can create rooms aliases.
v0.33.8 also contains more python 3 fixes: we are running most of matrix.org on python 3 as of right now and seeing some pretty impressive performance improvements. Look out for Hawkowl’s write up coming soon.
For those interested in what we are working on right now, take a look at our task board.
As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here.
Synapse 0.33.8 changelog
Features
- Servers with auto-join rooms will now automatically create those rooms when the first user registers (#3975)
- Add config option to control alias creation (#4051)
- The register_new_matrix_user script is now ported to Python 3. (#4085)
- Configure Docker image to listen on both ipv4 and ipv6. (#4089)
Bugfixes
- Fix HTTP error response codes for federated group requests. (#3969)
- Fix issue where Python 3 users couldn’t paginate /publicRooms (#4046)
- Fix URL previewing to work in Python 3.7 (#4050)
- synctl will use the right python executable to run worker processes (#4057)
- Manhole now works again on Python 3, instead of failing with a “couldn’t match all kex parts” when connecting. (#4060, #4067)
- Fix some metrics being racy and causing exceptions when polled by Prometheus. (#4061)
- Fix bug which prevented email notifications from being sent unless an absolute path was given for
email_templates. (#4068)
- Correctly account for cpu usage by background threads (#4074)
- Fix race condition where config defined reserved users were not being added to
the monthly active user list prior to the homeserver reactor firing up (#4081)
- Fix bug which prevented backslashes being used in event field filters (#4083)
Internal Changes
- Add information about the matrix-docker-ansible-deploy playbook (#3698)
- Add initial implementation of new state resolution algorithm (#3786)
- Reduce database load when fetching state groups (#4011)
- Various cleanups in the federation client code (#4031)
- Run the CircleCI builds in docker containers (#4041)
- Only colourise synctl output when attached to tty (#4049)
- Refactor room alias creation code (#4063)
- Make the Python scripts in the top-level scripts folders meet pep8 and pass flake8. (#4068)
- The README now contains example for the Caddy web server. Contributed by steamp0rt. (#4072)
- Add psutil as an explicit dependency (#4073)
- Clean up threading and logcontexts in pushers (#4075)
- Correctly manage logcontexts during startup to fix some “Unexpected logging context” warnings (#4076)
- Give some more things logcontexts (#4077)
- Clean up some bits of code which were flagged by the linter (#4082)
by Matthew Hodgson | Oct 29, 2018 | General, In the News, Press Release |
Hi all,
Back in June we blogged about the plan of action to establish a formal open governance system for the Matrix protocol: introducing both the idea of the Spec Core Team to act as the neutral technical custodian of the Matrix Spec, as well as confirming the plan to incorporate the Matrix.org Foundation to act as a neutral non-profit legal entity which can act as the legal Guardian for Matrix’s intellectual property, gather donations to fund Matrix work, and be legally responsible for maintaining and evolving the spec in a manner which benefits the whole ecosystem without privileging any individual commercial concerns. We published a draft proposal for the new governance model at MSC1318: a proposal for open governance of the Matrix.org Spec to gather feedback and to trial during the day-to-day development of the spec. Otherwise, we refocused on getting a 1.0 release of the Spec out the door, given there’s not much point in having a fancy legal governance process to safeguard the evolution of the Spec when we don’t even have a stable initial release!
We were originally aiming for end of August to publish a stable release of all Matrix APIs (and thus a so-called 1.0 of the overall standard) – and in the end we managed to publish stable releases of 4 of the 5 APIs (Client-Server, Application Service, Identity Service and Push APIs) as well as a major overhaul of the Server-Server (SS) API. However, the SS API work has run on much longer than expected, as we’ve ended up both redesigning and needing to implement many major changes to to the protocol: the new State Resolution algorithm (State Resolution Reloaded) to fix state resets; versioned rooms (in order to upgrade to the new algorithm); changing event IDs to be hashes; and fixing a myriad federation bugs in Synapse. Now, the remaining work is progressing steadily (you can see the progress over at https://github.com/orgs/matrix-org/projects/2 – although some of the cards are redacted because they refer to non-spec consulting work) – and the end is in sight!
So, in preparation for the upcoming Matrix 1.0 release, we’ve been moving ahead with the rest of the open governance plan – and we’re happy to announce that as of a few hours ago, the initial incarnation of The Matrix.org Foundation exists!
Now, it’s important to understand that this process is not finished – what we’ve done is to set up a solid initial basis for the Foundation in order to finish refining MSC1318 and turning it into the full Articles of Association of the Foundation (i.e. the legal framework which governs the remit of the Foundation), which we’ll be working on over the coming weeks.
In practice, what this means is that in the first phase, today’s Foundation gives us:
- A UK non-profit company – technically incorporated as a private company, limited by guarantee.
- Guardians, whose role is to be legally responsible for ensuring that the Foundation (and by extension the Spec Core Team) keeps on mission and neutrally protects the development of Matrix. Matrix’s Guardians form the Board of Directors of the Foundation, and will provide a ‘checks and balances’ mechanism between each other to ensure that all Guardians act in the best interests of the protocol and ecosystem.
For the purposes of initially setting up the Foundation, the initial Guardians are Matthew & Amandine – but in the coming weeks we’re expecting to appoint at least three independent Guardians in order to ensure that the current team form a minority on the board and ensure the neutrality of the Foundation relative to Matthew & Amandine’s day jobs at New Vector.The profile we’re looking for in Guardians are: folks who are independent of the commercial Matrix ecosystem (and especially independent from New Vector), and may even not be members of today’s Matrix community, but who are deeply aligned with the mission of the project, and who are respected and trusted by the wider community to uphold the guiding principles of the Foundation and keep the other Guardians honest.
- An immutable asset lock, to protect the intellectual property of the Foundation and prevent it from ever being sold or transferred elsewhere.
- An immutable mission lock, which defines the Foundation’s mission as a non-profit neutral guardian of the Matrix standard, with an initial formal goal of finalising the open governance process. To quote article 4 from the initial Articles of Association:
-
4. The objects of the Foundation are for the benefit of the community as a whole to:
4.1.1 empower users to control their communication data and have freedom over their communications infrastructure by creating, maintaining and promoting Matrix as an openly standardised secure decentralised communication protocol and network, open to all, and available to the public for no charge;
4.1.2 build and develop an appropriate governance model for Matrix through the Foundation, in order to drive the adoption of Matrix as a single global federation, an open standard unencumbered from any proprietary intellectual property and/or software patents, minimising fragmentation (whilst encouraging experimentation), maximising speed of development, and prioritising the long-term success and growth of the overall network over the commercial concerns of an individual person or persons.
- You can read the initial Articles of Association here (although all the rest of it is fairly generic legal boilerplate for a non-profit company at this point which hasn’t yet been tuned; the Matrix-specific stuff is Article 4 as quoted above). You can also see the initial details of the Foundation straight from the horse’s mouth over at https://beta.companieshouse.gov.uk/company/11648710.
Then, in the next and final phase, what remains is to:
- Appoint 3+ more Guardians (see above).
- Finalise MSC1318 and incorporate the appropriate bits into the Articles of Associations (AoA). (We might literally edit MSC1318 directly into the final AoA, to incorporate as much input as possible from the full community)
- Tune the boilerplate bits of the AoA to incorporate the conclusions of MSC1318.
- Register the Foundation as a Community Interest Company, to further anchor the Foundation as being for the benefit of the wider community.
- Perform an Asset Transfer of any and all Matrix.org property from New Vector to the Foundation (especially the Matrix.org domain and branding, and donations directed to Matrix.org).
So there you have it! It’s been a long time in coming, and huge thanks to everyone for their patience and support in getting to this point, but finally The Matrix.org Foundation exists. Watch this space over the coming weeks as we announce the Guardians and finish bootstrapping the Foundation into its final long-term form! Meanwhile, any questions: come ask in #matrix-spec-process:matrix.org or in the blog comments here.
thanks,
Matthew, Amandine, and the forthcoming Guardians of [the] Matrix!
by Hubert Chathi | Oct 25, 2018 | Tech |
Olm 3.0.0 has been released, which features several big changes. It can be downloaded from https://git.matrix.org/git/olm/. The npm package for JavaScript can be downloaded from https://matrix.org/packages/npm/olm/olm-3.0.0.tgz
Python
The biggest change is the merge of poljar’s improved Python bindings. These bindings should be much easier to use for Python programmers, and are used by Zil0’s E2E support in the Matrix Python SDK.
Since the binding API has changed, existing Python code will need to be rewritten in order to work with this release.
poljar has also included comprehensive documentation for the new API.
CMake
mujx contributed support for building olm using CMake. This should allow for easier building on different platforms. Currently the library can be built using either make or CMake. In the future, make support may be removed.
JavaScript
The JavaScript bindings now use WebAssembly by default. WebAssembly is much faster than the previous asm.js build, and is supported by recent versions of the most popular browsers. For compatibility with browsers that do not support WebAssembly, the asm.js version is still provided.
Due to adding support for WebAssembly, the API had to be changed slightly.
There is now an init function that must be called before the library can be used. This function will return a promise that will resolve once the library is ready to be used. The matrix-js-sdk has not yet been updated to do this, so users of matrix-js-sdk should continue using olm 2.x until it has been updated.
Key backups
The public key API has been updated to support the proposal for server-side key backups. More details on how to use these functions will be published in the future.
by Matthew Hodgson | Oct 22, 2018 | General, In the News, Uncategorized |
Hi folks,
Today is one of those pivotal days for the Matrix ecosystem: we’re incredibly excited to announce that the world’s first ever dedicated homeserver hosting service is now fully available over at https://modular.im! This really is a massive step for Matrix towards being a mature ecosystem, and we look forward to Modular being the first of many hosting providers in the years to come :D
Modular lets anyone spin up a dedicated homeserver and Riot via a super-simple web interface, rather than having to run and admin their own server. It’s built by New Vector (the startup who makes Riot and hires many of the Matrix core team), and comes from taking the various custom homeserver deployments for people like Status and TADHack and turning them into a paid service available to everyone. You can even point your own DNS at it to get a fully branded dedicated homeserver for your own domain!
Anyway, for full details, check out the announcement over at the Riot blog. We’re particularly excited that Modular helps increase Matrix’s decentralisation, and is really forcing us to ensure that the Federation API is getting the attention it deserves. Hopefully it’ll also reduce some load from the Matrix.org homeserver! Modular will also help Matrix by directly funding Matrix development by the folks working at New Vector, which should in turn of course benefit the whole ecosystem.
Many people reading this likely already run their own servers, and obviously they aren’t the target audience for Modular. But for organisations who don’t have a sysadmin or don’t want to spend the time to run their own server, hopefully Modular gives a very cost-effective way of running your own dedicated reliable Matrix server without having to pay for a sysadmin :)
We’re looking forward to see more of these kind of services popping up in the future from everywhere in the ecosystem, and have started a Matrix Hosting page on the Matrix website so that everyone can advertise their own: don’t hesitate to get in touch if you have a service to be featured!
If you’re interested, please swing by #modular:matrix.org or feel free to shoot questions to [email protected].
by Neil Johnson | Oct 18, 2018 | Uncategorized |
Hey ho, let’s go. Synapse 0.33.7 has arrived.
Regular readers will know how close we are to a full python 3 release. We are not quite there yet but 0.33.7 has support for Synapse under worker mode and we’ve running it on matrix.org this week. We need more time to conclusively gauge performance improvements but the Synchrotron workers are running with 33% less RAM. Thanks to everyone who has been running their servers under py3, if you do spot anything unusual just let us know. Once we’ve been running it a bit longer on matrix.org, we’ll cut a 0.34.0 release with a recommendation that one and all upgrade to python 3.
Aside from that this release contains support for server side end to end key backups, paving the way for client side support in Riot and Rich continues his long running federation bug squash-a-thon which should help with a whole host of federation snafus.
Up next on the horizon is returning in earnest to getting the server to server r0 spec out starting with shipping our brand new super shiny state resolution algorithm.
As a final point, for those of you that deploy from git checkout or a snapshot url and have email notifications enabled please take a look warning in the change log.
As a final final point #synapse:matrix.org is now an officially supported room, aimed at Synapse admins. If you’ve not done so already please do drop by and say Hi.
As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here.
Onwards!
Synapse 0.33.7 Change Log
Warning: This release removes the example email notification templates from res/templates (they are now internal to the python package). This should only affect you if you (a) deploy your Synapse instance from a git checkout or a github snapshot URL, and (b) have email notifications enabled.
If you have email notifications enabled, you should ensure that email.template_dir is either configured to point at a directory where you have installed customised templates, or leave it unset to use the default templates.
The configuration parser will try to detect the situation where email.template_dir is incorrectly set to res/templates and do the right thing, but will warn about this.
Features
- Ship the example email templates as part of the package (#4052)
- Add support for end-to-end key backup (MSC1687) (#4019)
Bugfixes
- Fix bug which made get_missing_events return too few events (#4045)
- Fix bug in event persistence logic which caused ‘NoneType is not iterable’ (#3995)
- Fix exception in background metrics collection (#3996)
- Fix exception handling in fetching remote profiles (#3997)
- Fix handling of rejected threepid invites (#3999)
- Workers now start on Python 3. (#4027)
- Synapse now starts on Python 3.7. (#4033)
Internal Changes
- Log exceptions in looping calls (#4008)
- Optimisation for serving federation requests (#4017)
- Add metric to count number of non-empty sync responses (#4022)
by Ben Parsons | Oct 16, 2018 | Tutorials |
We have a brand new, exciting guide page offering an introduction to matrix-js-sdk. This guide will live with the documentation at https://matrix.org/docs/guides/usage-of-the-matrix-js-sdk, but you can find the text below.
Matrix allows open real-time communications over the Internet using HTTP and JSON. This makes developing clients to connect to Matrix servers really easy! Because it’s open, and uses simple syntax for messages, you can connect Matrix to anything that communicates over a standard HTTP interface – later projects in this series will explore ideas such as building bots, performing machine learning on message content, and connecting IoT devices such as Philips Hue lights.
Making a Matrix Client
Let’s explore how we would make a very simple Matrix client, with only the ability to perform an initial sync, and to get member lists and the timeline for rooms of our choice.
This article will explore the Matrix Client-Server API, making use of the matrix-js-sdk. Later articles may discuss making the underlying calls. Specifically we will cover:
- login
- simple syncing
- listening for timeline events
- access the
MatrixInMemoryStore
- sending messages to rooms
- how to respond to specific messages, as a bot would
We’ll use Node.js as our environment, though the matrix-js-sdk can also be used directly in the browser.
Before we start, make sure you have Node.js and NPM installed: follow instructions at nodejs.org for your platform. Then create a new directory to work in:
mkdir my-first-matrix-client
cd my-first-matrix-client
Let’s write JavaScript
Once you’re ready, the first thing to do is install the matrix-js-sdk from NPM:
npm install matrix-js-sdk
We include the SDK in our source exactly as expected:
import sdk from 'matrix-js-sdk';
Login with an access token
Instantiate a new client object and use an access token to login:
const client = sdk.createClient({
baseUrl: "https://matrix.org",
accessToken: "....MDAxM2lkZW50aWZpZXIga2V5CjAwMTBjaWQgZ2Vu....",
userId: "@USERID:matrix.org"
});
// note that we use the full MXID for the userId value
(jsdoc for createClient)
If you are logged into Riot, you can find an access token for the logged-in user on the Settings page.
If the homeserver you’re logging in to supports logging in with a password, you can also retrieve an access token programmatically using the API. To do this, create a new client with no authentication parameters, then call client.login() with "m.login.password":
const client = sdk.createClient("https://matrix.org");
client.login("m.login.password", {"user": "USERID", "password": "hunter2"}).then((response) => {
console.log(response.access_token);
});
In any case, once logged in either with a password or an access token, the client can get the current access token via:
console.log(client.getAccessToken());
Note: it is essential to keep this access token safe, as it allows complete access to your Matrix account!
Sync and Listen
Next we start the client, this sets up the connection to the server and allows us to begin syncing:
Perform a first sync, and listen for the response, to get the latest state from the homeserver:
client.once('sync', function(state, prevState, res) {
console.log(state); // state will be 'PREPARED' when the client is ready to use
});
Once the sync is complete, we can add listeners for events. We could listen to ALL incoming events, but that would be a lot of traffic, and too much for our simple example. If you want to listen to all events, you can add a listen as follows:
client.on("event", function(event){
console.log(event.getType());
console.log(event);
})
Instead, let’s just listen to events happening on the timeline of rooms for which our user is a member:
client.on("Room.timeline", function(event, room, toStartOfTimeline) {
console.log(event.event);
});
Access the Store
When we created a new client with sdk.createClient(), an instance of the default store, MatrixInMemoryStore was created and enabled. When we sync, or instruct otherwise our client to fetch data, the data is automatically added to the store.
To access the store, we use accessor methods. For example, to get a list of rooms in which our user is joined:
// client.client.getRooms() returns an array of room objects
var rooms = client.getRooms();
rooms.forEach(room => {
console.log(room.roomId);
});
(jsdoc for client.getRooms)
More usefully, we could get a list of members for each of these rooms:
var rooms = client.getRooms();
rooms.forEach(room => {
var members = room.getJoinedMembers();
members.forEach(member => {
console.log(member.name);
});
});
For each room, we can inspect the timeline in the store:
var rooms = client.getRooms();
rooms.forEach(room => {
room.timeline.forEach(t => {
console.log(JSON.stringify(t.event.content));
});
});
Send messages to rooms
To send a message, we create a content object, and specify a room to send to. In this case, I’ve taken the room ID of #test:matrix.org, and used it as an example:
var testRoomId = "!jhpZBTbckszblMYjMK:matrix.org";
var content = {
"body": "Hello World",
"msgtype": "m.text"
};
client.sendEvent(testRoomId, "m.room.message", content, "").then((res) => {
// message sent successfully
}).catch((err) => {
console.log(err);
}
(jsdoc for client.sendEvent)
Knowing this, we can put together message listening and message sending, to build a bot which just echos back any message starting with a “!”:
var testRoomId = "!jhpZBTbckszblMYjMK:matrix.org";
client.on("Room.timeline", function(event, room, toStartOfTimeline) {
// we know we only want to respond to messages
if (event.getType() !== "m.room.message") {
return;
}
// we are only intested in messages from the test room, which start with "!"
if (event.getRoomId() === testRoomId && event.getContent().body[0] === '!') {
sendNotice(event.event.content.body);
}
});
function sendNotice(body) {
var content = {
"body": body.substring(1),
"msgtype": "m.notice"
};
client.sendEvent(testRoomId, "m.room.message", content, "", (err, res) => {
console.log(err);
});
}
Take a look at the msgtype in the object above. In the previous example, we used “m.text” for this field, but now we’re using “m.notice”. Bots will often use “m.notice” to differentiate their messages. This allows the client to render notices differently, for example Riot, the most popular client, renders notices with a more pale text colour.
Further
There is much, much more to Matrix, the Client-Server API and the matrix-js-sdk, but this guide should give some understanding of simple usage. In subsequent guides we’ll cover more detail and also explore projects you can build on top, such as IoT controls and chatbot interfaces. For now you can take a look at other examples in the matrix-js-sdk itself, and also the Matrix Client-Server API which it implements.
by Eli Etzioni | Oct 11, 2018 | General |
Guest post today from Giveth: Giveth is re-engineering charitable giving, by creating an entirely free, open-source platform, built on the Ethereum Blockchain. Our system cuts out bureaucracy and enables nonprofits to create a high level of transparency and accountability towards Givers.
Giveth’s new chatbot in action!
Online or offline, joining a new community always requires some adjustment. Even the most open, inclusive communities have shared knowledge and shared practices which new members learn as they participate.
I recently joined Giveth’s Riot community, where the majority of Giveth’s communication occurs. Immediately upon joining, I received the message pictured above from the Giveth Bot, kindly encouraging me to download Riot mobile and change my notifications to mention-only. The bot shortened my adjustment period by giving me key tidbits of information that everyone in Giveth’s community knows, but that may have taken me time to pick up on my own. This blog post will cover how the Giveth Bot came to be, what it is capable of, and where the project is headed in the future.
The Giveth Bot actually started out as an attempt to solve a completely different problem: helping Giveth efficiently distribute internal reward points. Giveth’s system for rewarding people who meaningfully contribute to the project is called RewardDAO. “If someone contributes in a meaningful way, a core contributor from each of the Giveth Campaigns can dish them points to recognize the contribution”, describes Cleo in an article explaining how RewardDAO works. At the end of each month, contributors receive Ether based on how many points they have earned.
The Giveth RewardDAO motto. Photo from https://medium.com/giveth.
However, any time that a core contributor dished points to someone, they had to record who received the points, and how many, on a spreadsheet. In search of a better way, Giveth opened up the project of automating this system to the social coding hub, a community of altruistic developers looking to tackle impactful and interesting projects, offering a 2 eth bounty for a solution.
A lot of great work was submitted, and ultimately Deam’s (@deamlabs) code was chosen to power the bot and the code for the pointsbot itself was further developed and refined by Frederik Bolding. Now, by using a command of the form “!dish [number] [type] points to [contributor] for [contribution]”, Giveth core contributors can distribute points as needed, and the bot will automatically update the spreadsheet accordingly.
The Giveth Bot dishing points like a champion!
Once the bot’s framework was established, chatbot features were added. In addition to the welcome message I received, the bot gives custom welcome messages in each of Giveth’s different rooms, allows Matrix users to have 1-on-1 chats with it, and listens for keywords and sentences it recognizes in rooms and private chats. Riot is built on top of an open-source protocol called Matrix. Matrix has a javascript standard development kit (SDK), which the bot uses to detect events occurring in each of the Riot rooms and chats that it is a part of.
Giveth began by using Slack, but switched to Riot to support Matrix’s decentralized, open-source model, which which aligns far more with Giveth’s own business model and values. The Giveth Bot is a great example of how Matrix enables users to build their own solutions to problems. In the future, we hope that the Giveth Bot will be able to interact directly with the Ethereum Blockchain, and that more analytics and measurement tools can be incorporated. And of course, we welcome any and all feedback on the Giveth Bot!
Giveth is an open-source platform for building decentralized altruistic communities. Anyone interested in getting involved should head to join.giveth.io
Interested in checking out the Giveth Bot’s inner workings? All code is available at https://github.com/Giveth/giveth-bot.
Interested in learning DApp development or helping out with cool projects like the Giveth Bot? Check out the social_coding Riot channel, tell us what you’re interested in, and help build awesome stuff!
by Neil Johnson | Oct 4, 2018 | Uncategorized |
Right folks, time for Synapse 0.33.6.
These past few weeks we’ve been focusing on fixing a whole host of federation bugs to improve reliability and latency. Additionally we’ve squashed some py3 bugs, improved lazy loading and been working hard in the background to improve our CI infrastructure. Finally, we cleaned up the Docker file, the image is now half the size of 0.33.5.1’s standing at 58 MB.
As ever, you can get the new update here or any of the sources mentioned at https://github.com/matrix-org/synapse. Note, Synapse is now available from PyPI, pick it up here.
Synapse 0.33.6
Features
- Adding the ability to change MAX_UPLOAD_SIZE for the docker container variables. (#3883)
- Report “python_version” in the phone home stats (#3894)
- Always LL ourselves if we’re in a room (#3916)
- Include eventid in log lines when processing incoming federation transactions (#3959)
- Remove spurious check which made ‘localhost’ servers not work (#3964)
Bugfixes
- Fix problem when playing media from Chrome using direct URL (thanks @remjey!) (#3578)
- support registering regular users non-interactively with register_new_matrix_user script (#3836)
- Fix broken invite email links for self hosted riots (#3868)
- Don’t ratelimit autojoins (#3879)
- Fix 500 error when deleting unknown room alias (#3889)
- Fix some b’abcd’ noise in logs and metrics (#3892, #3895)
- When we join a room, always try the server we used for the alias lookup first, to avoid unresponsive and out-of-date servers. (#3899)
- Fix incorrect server-name indication for outgoing federation requests (#3907)
- Fix adding client IPs to the database failing on Python 3. (#3908)
- Fix bug where things occaisonally were not being timed out correctly. (#3910)
- Fix bug where outbound federation would stop talking to some servers when using workers (#3914)
- Fix some instances of ExpiringCache not expiring cache items (#3932, #3980)
- Fix out-of-bounds error when LLing yourself (#3936)
- Sending server notices regarding user consent now works on Python 3. (#3938)
- Fix exceptions from metrics handler (#3956)
- Fix error message for events with m.room.create missing from auth_events (#3960)
- Fix errors due to concurrent monthly_active_user upserts (#3961)
- Fix exceptions when processing incoming events over federation (#3968)
- Replaced all occurences of e.message with str(e). Contributed by Schnuffle (#3970)
- Fix lazy loaded sync in the presence of rejected state events (#3986)
- Fix error when logging incomplete HTTP requests (#3990)
Internal Changes
- Unit tests can now be run under PostgreSQL in Docker using
test_postgresql.sh. (#3699)
- Speed up calculation of typing updates for replication (#3794)
- Remove documentation regarding installation on Cygwin, the use of WSL is recommended instead. (#3873)
- Fix typo in README, synaspse -> synapse (#3897)
- Increase the timeout when filling missing events in federation requests (#3903)
- Improve the logging when handling a federation transaction (#3904, #3966)
- Improve logging of outbound federation requests (#3906, #3909)
- Fix the docker image building on python 3 (#3911)
- Add a regression test for logging failed HTTP requests on Python 3. (#3912)
- Comments and interface cleanup for on_receive_pdu (#3924)
- Fix spurious exceptions when remote http client closes conncetion (#3925)
- Log exceptions thrown by background tasks (#3927)
- Add a cache to get_destination_retry_timings (#3933, #3991)
- Automate pushes to docker hub (#3946)
- Require attrs 16.0.0 or later (#3947)
- Fix incompatibility with python3 on alpine (#3948)
- Run the test suite on the oldest supported versions of our dependencies in CI. (#3952)
- CircleCI now only runs merged jobs on PRs, and commit jobs on develop, master, and release branches. (#3957)
- Fix docstrings and add tests for state store methods (#3958)
- fix docstring for FederationClient.get_state_for_room (#3963)
- Run notify_app_services as a bg process (#3965)
- Clarifications in FederationHandler (#3967)
- Further reduce the docker image size (#3972)
- Build py3 docker images for docker hub too (#3976)
- Updated the installation instructions to point to the matrix-synapse package on PyPI. (#3985)
- Disable USE_FROZEN_DICTS for unittests by default. (#3987)
- Remove unused Jenkins and development related files from the repo. (#3988)
- Improve stacktraces in certain exceptions in the logs (#3989)
- Pin to prometheus_client<0.4 to avoid renaming all of our metrics (#4002)
Recent Comments