Big week for Synapse: v0.33.3 was released this week. You'll find preparation for support for Lazy Loading and Room Versioning, lots of bugfixes, and a great contribution from vojeroen: support for sending SNI over federation for vhosted servers. More complete change log here, features below:
- Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! (#3439)
- Add /_media/r0/config (#3184)
- speed up /members API and add
membershipparams as per MSC1227 (#3568)
summaryblock in /sync response as per MSC688 (#3574)
- Add lazy-loading support to /messages as per MSC1227 (#3589)
- Add ability to limit number of monthly active users on the server (#3633)
- Support more federation endpoints on workers (#3653)
- Basic support for room versioning (#3654)
- Ability to disable client/server Synapse via conf toggle (#3655)
- Ability to whitelist specific threepids against monthly active user limiting (#3662)
- Add some metrics for the appservice and federation event sending loops (#3664)
- Where server is disabled, block ability for locked out users to read new messages (#3670)
- set admin uri via config, to be used in error messages where the user should contact the administrator (#3687)
- Synapse's presence functionality can now be disabled with the "use_presence" configuration option. (#3694)
- For resource limit blocked users, prevent writing into rooms (#3708)
- A large chunk of e2e has landed in the spec, largely thanks to Zil0's work. There are still some things left, but the main bits related to sending encrypted messages is in there. Attachments and key sharing are being reviewed today
- TravisR progressed on Push, Application Services, and Client-Server APIs.
- To follow along with progress, the best place to look is the August 2018 r0 tracker.
Key backups are being worked on, and we've had some new thoughts on key verification.
Well are you?
accumulates Russian translates of Matrix FAQ and manuals, and other info about Matrix in Russian. Also we starts a public Matrix Synapse server for Russian users with free open registration
Russian Matrix users: feel free to add and complement articles in our wiki to make Matrix more friendly for Russian users.
Seaglass now has support for creating, joining and leaving rooms, and accepting/rejecting room invites, redacting messages, improvements to the room settings pages, timestamps, some visual tweaks, lots of bug fixes, early support for detecting failed message sends, scrolling improvements
Typing notification and desktop notification of Matrique are working, plus other improvements
See the latest commits here.
Lazy Loading (LL) is available on develop behind the LABS flag, and is being polished. It will probably not make it to next release, but it's progressing!
New releases are imminent for both Android and iOS:
- iOS: bug fixes and Lazy Loading in settings
- Android: bug fixes, better management of permission requests at startup
Shared Secret Auth and Matrix Corporal
It allows external systems (which know the shared secret) to easily obtain access tokens for any user hosted on the homeserver. This is incredibly useful for all sorts of automation (no more pre-generated access tokens or plain-text passwords).
With this tool, you can pass a mxid, and get back an access_token for that user. With this, you can do whatever actions would normally be done with a token. For example, you for a new starter at a company, you could join a set of rooms known to be needed by new starters.
Support room for Shared Secret Auth: #matrix-synapse-shared-secret-auth:devture.com
Next, he has released Matrix Corporal, reconciliator and gateway for a managed Matrix server:
It's a way to take control of Matrix (Synapse) in a corporate (or other such) environment.
Based on a configuration policy (generated by an external system, like your intranet), it can auto-create and auto-disable users, keep their profile and authentication details to date, auto-join/leave users to communities and rooms, etc.
It's also meant to sit in front of the Matrix Synapse Client API, inspecting and allowing/denying requests, in accordance with the configuration policy (preventing people from leaving certain rooms/communities, preventing them from messing around with their profile details, etc.)
Support room for Matrix Corporal: #matrix-corporal:devture.com
Note that Slavi also maintains matrix-docker-ansible-deploy, which can deploy these projects along with Synapse itself.
I was extremely young when I first read and understood Larry Wall's explanation about the virtue of laziness, but I did not expect it to lead here. Cadair has created (I think inspired by an idea from TravisR), a "twimbot". This bot works in conjunction with opsdroid and the opsdroid connector-matrix - it works by storing selected messages from #twim:matrix.org in room state. In this way, it either assists the author by making it easier to produce the blog post you're reading, or it coldly replaces the author, leading to his sacking and penury. To view updates currently held by the bot, join #twim_updates:cadair.com.
Why? Because why not. Source available on github: https://github.com/turt2live/matrix-hashtag-bot
matrix-appservice-bridge and matrix-appservice-irc
Just pushed out matrix-appservice-irc 0.11.0-rc3 which contains a lot of nice things. The full changelog is there, hoping to get some of this tested and running on the matrix.org servers soon, but initial testing looks good :)
Released matrix-appservice-bridge 1.6.0 which contains mostly things I was writing for the irc bridge and factored out instead. There's a couple of goodies like automatic caching of profiles/events, and getEvent as a way to fetch events from the homeserver without fiddling with context.
Intentoperations now cache requests that would otherwise fall through to the homeserver which can be expensive. This is configurable for
Intents via the
sizeoptions. AgeCounters now allow you to set your own time period buckets. Added a new function
Intent.getEventwhich will fetch events from the homeserver without any context information, which should be quick.
MembershipCacheis now exposed to let folks read and write to the cache while also letting the bridge access it.
arelease: Fix issue where
brelease: Fix issue where we were calling this.intent inside intent like fools!
crelease: Fix issue where some stole js-sdk code was not checked thoroughly.
It uses the go-whatsapp library to talk with the WhatsApp Web API. Using the web API means that you'll still need WhatsApp on your phone connected to the internet, but it also means you won't get banned for using the bridge.
The bridge is still very alpha, but basic message bridging and some advanced features already work. There's a features & roadmap document about that.
To come chat about the WhatsApp bridge: #whatsapp:maunium.net.
There were two releases last week:
0.4.6 with additional commands (follow, unfollow, mute, unmute, block, unblock users, show followers and followings of the specified user) and a little bugfix.
0.4.7 is out now, which increases the length of messages to 4000 characters.
GSOC is finished!
GSOC 2018 has been valuable for everyone involved!
- Thank you zil0 for all your work on the Matrix Python SDK, E2E and and spec work.
- Thank you APWhiteHat: lots of work on dendrite, including a bunch of stuff on federation, as well as typing notifications!
- and thank you to Cadair for helping to organising it!
The Construct using asynchronous disk IO
the Construct has advanced to fully asynchronous disk IO in a single-threaded environment.
RocksDB is a highly configurable and customizable database with a very large callback surface to support many different environments, which is good to connect it with something like AIO.
Normally RocksDB gets its performance from concurrent standard POSIX system calls (blocking IO) with multiple threads. The callback surface and its internals are not specifically suited for deviating from the pthread model. However I hacked RocksDB and tricked it into believing that my userspace contexts, which mimic the
std::thread/pthreadinterface, are threads.
This is an example of a RocksDB callback to read from a file, it expects this callback to return immediately (i.e as soon as the kernel returns from a blocking read(2).) Except, where I call
fs::read()that is actually a stack-suspension which makes a request to linux AIO where the stack resumes after AIO has called back with the data (see: https://github.com/jevolk/charybdis/blob/master/ircd/aio.cc)
No modifications to the internals of RocksDB were necessary. The Construct is being developed here: https://github.com/jevolk/charybdis.
Community Guides Index
Some weeks ago I asked for suggestions for content to add to a "Community Guides Index". There was not a wave of feedback, but we have enough collected content to be able to share a first pass of this Index. Check out the content there, and please please contact me if you have articles, videos or anything else you'd like to see included. Of course, if you'd like to produce some content and have it included, that would be great too! Ideas for missing articles:
- any content in a language other than English
- DevOps, how to use k8s or ansible to install Synapse
- how to use the Application Services API to make a bot
OggCamp last weekend was great - massive thank you to Jon for organising and running the show - we love your work!
Presentations from Ben and Half-Shot seemed to go down well, we all chatted with Open Source fans and generally flew the flag for Matrix.
- tulir has enough projects that he has created #:maunium.net, for any project of his that doesn't have it's own room
- Support room for Shared Secret Auth: #matrix-synapse-shared-secret-auth:devture.com
- Support room for Matrix Corporal: #matrix-corporal:devture.com
- If you prefer the cold output of a machine, #twim_updates:cadair.com shows you the updates as they're posted in #twim:matrix.org.
- WhatsApp bridge: #whatsapp:maunium.net
Hope you had a good week, hope you enjoyed reading this post! Amandine is back after some needed vacation time, and has this week's Matrix Live below.