Category – GSOC
20 posts tagged with "GSOC" (See all categories)

GSOC report: Moving matrix-ircd to async/await and futures 0.3

2020-09-18 — GSOC — Brooks Karlik

This is part of a series of reports on the six projects assigned to Matrix for Google Summer of Code 2020.

View project: Moving matrix-ircd to async/await and futures 0.3


The goal of this project was to port matrix-ircd from the outdated combinators style of futures 0.1 to the new-and-improved style of futures 0.3. As initially proposed, this greatly improves code readability and removes the convoluted compiler errors that were pervasive in futures 0.1

matrix-ircd

The matrix-ircd project functions as a bridge between two chat platforms: Internet relay chat (irc) and Matrix. matrix-ircd lets you use any standard IRC Client to communicate with Matrix chatrooms and direct messages.

Mentors

This project is mentored by Philipp Mandler (@phlmn) and Jonas Platte (@jplatte). I would like to thank them for the helpful advice and code reviews they have given me over the course of GSoC 2020.

Project Results

Since much of the code was written in late 2016, there were many portions of the code that were unidiomatic and produced compiler errors. The first step I made was to remove all compiler errors from cargo and clippy in #64.

Additional tests were then written in various parts of the codebase to ensure that the upgrade would produce the same results. These changes were made in #64 and #66.

Since the project master branch currently used a custom http implementation, I moved it to use hyper, a fast and correct http implementation that already supported async/await. Additionally, I moved the module that most utilized http, matrix, to async-await and the new hyper code. Not only did these changes shrink the code by ~700 lines, they also removed a lot of unnecessary complexity. These changes were made in #67.

The last large module remaining, irc, was then ported to irc, along with its dependencies in stream_fold.rs and its upstream user in the bridge module. The most exciting part of these changes was the removal of the futures 0.1 dependency. All code was now running in futures 0.3! These changes also moved away from tasked-futures which further improved code readability. This PR was in #71.

Since the bulk of the changes were now complete, I moved onto bug fixes. As @jplatte mentioned in #71, the current single threaded approach to the application was not ideal. In #72 I updated all code to be multithreaded-compatible.

Lastly, In #77 I included more tests to functions with heavy changes, added additional logging, removed unnecessary complexity that was introduced to keep code as "1:1" as possible, and added TLS support to hyper so that https would work. This patch also fixed a rather difficult bug regarding the irc TCP streams and the buffer they were reading into.

Remaining Work

Based on my personal testing there is no additional work to be done in the realm of updating to async/await. All tests pass and the IRC server and matrix bridge function as expected. @jplatte kindly announced in the This Week In Matrix blog that we will be conducting public testing of the async_await branch on github. Barring any issues the async/await code should be merged into the master branch in the next few weeks.

Pull request list

GSOC report: Ruma procedural macro refactoring and more

2020-09-16 — GSOC — Devin Ragotzy

This is part of a series of reports on the six projects assigned to Matrix for Google Summer of Code 2020.

View project: Ruma procedural macro refactoring and more


Wow the summer has flown by, it feels like just yesterday I was learning how to rebase and what exactly it is Ruma does. I exaggerate slightly, but it is a big library with lots of public API surface. I have learned more in the last few months than in two years of school. I have been able to observe and participate in a project with a community growing around it, been a part of discussions about design and best practices, given and received numerous code reviews as well as learned the process of addressing the feedback, and working from a specification. In short, this has been an amazing opportunity to gain experience in all the things that are hard to obtain in a classroom.

My project goal was to improve the existing macros in ruma-events-macros and ruma-api-macros. It became clear early on that this would include some major API changes and that improving the macros as they were was pointless without also moving to a new public API. While improving the durability and readability of the macro code I also rewrote entire sections to accommodate the new design.

A quick overview of the Matrix protocol for reference: a client sends content that is interpreted by the server as events. The server distributes those events out to other clients and other servers (the server case is known as federation). Ruma groups these events by kind Message, State, Ephemeral, ToDevice and Basic which are represented as generic structs (StateEvent<C>). Each event kind needs to be able to hold many different content types, for state events there is room creation, room name, and membership events to name a few. Using the macros, enums are generated to represent all state event possibilities, so a variant for membership, room name, etc. These types exist to support the core API request and response types for each endpoint that is defined by the Matrix specification.

GSoC Starts and I start with ruma-events

More work on ruma-events, now a part of the new mono-repo

Work on ruma-client-api

Back to ruma-events

Continuing maintenance

One of my personal goals was to become more familiar with git. With the help of my mentor I now feel more confident using this tool that is so essential to developers. I became fairly adept at merging, rebasing, and navigating all the headaches that come with that. I learned plenty of new commands. A few highlights: cherry-pick and specific uses of reset to avoid copy-pasting fixes and adding more commits. I used the reset command to craft good commits, splitting work into appropriate chunks. I am glad that I had the opportunity to hone my git skills. I feel like I have accomplished my goal and then some!

I am proud of the work that I have done: Being part of moving ruma-events much closer to the 0.22 release and creating macros to generate types specific to the Matrix specification. Working with the community that has grown around Ruma has been rewarding and I plan on sticking around.

GSOC report: HTML Embeddable Matrix Chat Rooms

2020-09-15 — GSOC — Arnav Tiwari

This is part of a series of reports on the six projects assigned to Matrix for Google Summer of Code 2020.

View project: HTML Embeddable Matrix Chat Rooms


My name is Arnav Tiwari and I am a prefinal year undergraduate student from IIT Kharagpur and I wanted to share my amazing journey with Matrix. I am a budding open-source enthusiast and this was my first experience with Google Summer of Code. For the past few months, I had been working on a project to develop an HTML embeddable chat client under the GSoC program for Matrix. Matrix provides a highly versatile SDK for making custom clients that can be leveraged for a variety of applications, one of which is using Matrix to power an embeddable chat client. A chat client itself can have numerous forms, whether it being a live chat to a simple comments section. This project was intended to provide an easy-to-use and yet highly customizable client that can be deployed on a website with minimal effort.

My goal for the project was to have a useable project by the end of the coding period, however, as it turned out, the project was going to be tested in the real world far sooner than that. The need for an embeddable client and the feasibility of the project to fulfill this role was demonstrated during the second month of the coding period itself. The client was deployed on the website of CommCon 2020, a virtual conference on communication technologies (an apt place to be tested, coincidentally). On the days leading up to the conference and during the conference itself, I helped the organizers to set up, integrate, and troubleshoot the client when required. While the process went mostly without any hiccups, there was one small incident when the client broke during production. Since the project was still pretty early in development, I didn’t expect it to be bug-free and had anticipated the possibility of this happening. I was keeping an eye on things, which proved to be a prudent decision as I was able to fix this problem quickly and with minimal downtime. The rest of the conference went smoothly and the client performed quite well even when the number of users was quite large (A testament to Matrix’s scalability). Getting to experience this was a pleasant surprise since I never expected to have real users so soon, much less so many at once. Seeing the client being used out in the wild was a very fulfilling thing to witness. I also gained some very valuable feedback, courtesy of Dan, CommCon’s master of ceremonies.

Over the next month or so, I kept on steadily adding features and building up the client. The next big break for the project came in the form of another conference. KDE Akademy 2020. This was a big surprise as I genuinely didn’t expect to see another large conference using the project so soon. The conference was scheduled to be held after a week or so after the end of the coding period. This time, however, the integration was almost completely handled by the conference organizers themselves since it had to be integrated with their version of BigBlueButton, a web conferencing system. As the conference drew nearer, things seemed to be working out well and there was no sign of trouble. When the day of the conference finally came, however, many things seem to break simultaneously due to an apparent incompatibility with BBB. In the end, despite the numerous attempts by the conference organizers and myself to remedy the issues, they had to roll back to an older version of the chat since the risk would be too great. The organizers were understandably disappointed since they had spent a while working on the integration and had seen the great potential of using this client in place of their old chat system.. Even though It was a sad conclusion to the journey, there were still many lessons to be learned. Most importantly, even though the client might work well in standalone circumstances, ease of integration might have some room for improvement. Open-source development never truly stops and I don’t intend to give up on this project. I look forward to constantly improving it and seeing more people adopt it.

These past few months were a spectacular experience. Even before starting this journey, I knew I would learn a lot but this still managed to exceed all my expectations. I got to learn things I never would have thought I would be able to experience under GSoC. I met some awesome people along the way, I’m extremely grateful to my mentors Ben Parsons and Travis Ralston for being the best mentors anyone could ever ask for. They were always approachable and friendly throughout the entire program and I never hesitated before asking for their help. Without their guidance, all of this would’ve certainly not been possible. Lastly, I want to express my gratitude to Matrix for believing in me and giving me the opportunity to undertake this project. The Matrix community is full of talented people who will go the extra mile if you ask them for help. It has truly been a pleasure working with them and I hope to continue working with them in the future. Cheers and hope to see continue seeing you all!

GSOC report: Enabling E2EE in Opsdroid Matrix Connector

2020-09-10 — GSOC — Tyagraj Desigar

This is part of a series of reports on the six projects assigned to Matrix for Google Summer of Code 2020.

View project: Enabling E2EE in Opsdroid Matrix Connector


I'm Tyagraj Desigar and I worked on Opsdroid with Stuart Mumford and Drew Leonard. Opsdroid is a python framework for creating platform agnostic bots usable with multiple chat services including matrix.

The Plan

The project's focus was on the interaction layer (called a connector) between opsdroid and matrix. As it stood, the connector used the deprecated matrix-python-sdk and had no support for encryption. The aim was to change this by moving over to matrix-nio.

I had planned to work on some other features as well:

  • A database module for opsdroid using matrix state events
  • Support for unused matrix events
  • Homeserver lookup using .well-known API requests
  • Device verification process for the bot

Porting to nio and adding encryption support was the bulk of the project, and the minimum I wanted to accomplish by the end of the project. I wanted to have a connector that worked as it did already with support for encrypted rooms which needed minimal extra configuration.

The Process

The process began with This PR which gave us a head start with the migration to matrix-nio. Through helping to review the PR I got to understand how matrix-nio and opsdroid work.

The implementation of encryption support was a little tricky in that it was tough to figure out the process required to do that in the context of the connector. One problem we faced was the installation of dependencies. Installing libolm, a C library which nio uses for encryption was a less than smooth process. This spawned a couple side projects that dealt with the CI, testing and installation of opsdroid. In the end we found a solution and had a working connector on our hands.

Next we shifted focus to the database module. It was based on this project. I rewrote it with nio and added a few features. The idea was straightforward but the implementation had many catches since we were working under some constraints for the interfacing with opsdroid. It went through several iterations before we settled on the final product.

The encryption and database took longer than I had initially expected which meant we didn't get to work on adding support for more events and homeserver lookups. I had a go at adding device verification steps while working on the encryption support but that turned out to be quite complicated and would introduce some breaking changes, besides cross-signing had just been introduced so we decided to drop that till nio implemented some way to leverage cross-signing.

Along the way some issues cropped up with the testing frameworks and CI that were hard to pinpoint and caused further delays but were solved eventually.

The Conclusion

I am extremely happy with what I've accomplished and hope that I have been able to achieve the standards set by the matrix community. It was a challenging and exhilarating experience. I have learned more in this project than I could've imagined and gained a ton of invaluable experience with software development thanks to my mentors who pushed me forward and guided me throughout. This journey was beyond amazing and I will be sure to contribute to matrix moving forward.

You can find complete details of everything I worked on during GSoC here.

GSOC report: Adding Features in End-to-End encryption for Nheko-Reborn

2020-09-09 — GSOC — CH Chethan Reddy

This is part of a series of reports on the six projects assigned to Matrix for Google Summer of Code 2020.

View project: Adding Features in End-to-End encryption for Nheko-Reborn

This is a re-publishing of Chetan's original blog post.


whoami?

I'm CH Chethan Reddy. I'm currently pursuing Bachelors in Electronics and Communication Engineering at the National Institute of Technology, Trichy in India. In my free time I like to swim, work on projects I find interesting, watch movies and do a lot of other "interesting" stuff. This summer I have interned in Matrix.org for around 3-4 months under GSoC and in this blog I'll be sharing my experience.

Why Matrix.org?

Being passionate about privacy, I wanted to select an organization that actively works on that, and clearly, Matrix.org topped my list. For anyone who doesn't know what Matrix is, in simple words, It is a decentralized communication protocol that supports many awesome features like messaging, end-to-end encryption, voip/video calls support, etc. It uses simple RESTful http APIs which keeps things very simple. The best part of the protocol is, it is decentralized, so if you have a custom domain name, you can setup a synapse server in a jiffy. Knowing all these who wouldn't choose Matrix?

My Work

I worked on Desktop Client called Nheko. It is a light-weight Desktop Client that uses Qt Framework in the frontend and also uses a mtxclient library that implements Matrix-client server API written in C++. The scope of my project involved implementing Device-Verification and Cross-Signing in Nheko.

For people who have not used Matrix before, it is important to note that every device of a particular user is considered as a seperate unit rather than user itself. For end-to-end encrypted chat to be viewable from a device, that particular device should be trusted by the sender's device. Device-verification and Cross-Signing are methods used for verifying.

Coming to the work I have implemented, Device-verification is completely implemented in Nheko. As of now, SAS verification with to-device and room message verification is supported in Nheko. As far as Cross-Signing is concerned, verifying signatures of Cross-Signing keys and showing the verified status of devices have been implemented. The only remaining unfinished parts are SSSS and signing the keys after verification. After that Cross-signing should be feature-complete.

I wished to implement SSSS during my GSoC period, but I couldn't because I did not anticipate the additional things which came with verification while making my proposal. The additional works included : implementing the UI, changes to the userprofile dialog, working on the caching of verified users, and user keys. Moreover, due to the current pandemic, there were a few sudden changes in my academic schedule, which interrupted my work to some extent. While contributing at times I was stuck on some bugs like verifying the signatures, some random crashes in UserProfile and setting up relations in Room-Verification. However, with the help of my mentors, I was able to fix these bugs.

I'm really happy about the work I have done. I am hoping to further work on Nheko in the future, complete Cross-Signing and work on additional features for Nheko.

Community and Mentors

The best part of my experience this summer was the the Matrix community and the learning I had. I am really lucky to be part of the Matrix community which has many passionate people collectively working on really cool projects with clients, bridges, servers, bots and the spec. A special thanks to Uhoreg and Sorunome who have helped me in navigating through the spec.

Last but not the least, a big shout-out to both my mentors, Nico and red_sky, who were always there to help me with any issue in the project inspite of their personal commitments and dealt with my stupid questions with utmost patience and kindness. Without their active help and guidance this experience definitely would not have been so fun and great, they have clearly surpassed the expectations I had from mentors.

GSOC report: E2E encryption for go-neb

2020-09-08 — GSOC — Nikolaos Filippakis

This is part of a series of reports on the six projects assigned to Matrix for Google Summer of Code 2020.

View project: E2E encryption for go-neb


Hello! I am Nikos, an MSc student in Computer Security. My original goal for this GSoC was to implement the basics of end-to-end encryption for Go-NEB, which is a popular and easily extendable bot for the Matrix protocol. I chose this project because it ticked off a lot of the boxes that I look for in a project: The Matrix protocol is very interesting and the e2e specification is fascinating, Go-NEB is written in Go which I wanted to learn better, plus I believe personally that crypto should be easy for everyone to use and adopted more.

My initial plan was to first create bindings in gomatrix for libolm, the C library that implements most of the necessary algorithms for Olm/Megolm, the protocols used in Matrix' end-to-end encryption. Then, I was going to write some nice APIs around those bindings so that most of the work done is hidden from the client that uses the library, and finally I would call these APIs from Go-NEB when needed to set up the encrypted sessions between devices in a room and use them to encrypt/decrypt messages as necessary. What we found out soon after GSoC started was that another library called mautrix-go that was based on gomatrix already did most of these things. My task then became to change the uses of gomatrix in Go-NEB for mautrix-go and make some slight changes to the latter if there were some incompatibilities. A month later, that task was done.

With two months to go in GSoC, my mentors came up with a new task: to implement a service for Go-NEB that allows other client developers to test their e2ee implementations easily. That resulted in a service called "cryptotest" which allows other clients in a specified room to execute some Neb commands which would trigger functions like the forwarding of keys between clients, another feature that had to be implemented in mautrix. Something else that I wanted to add and grabbed my interest was SAS device verification, a multi-step process that involves users comparing a string of numbers or emojis out-of-band to configure they are indeed talking to each other and not to a MITM. These were my main tasks during the second month and I was happy to have achieved them and made a decent verification API for clients to use. They were soon merged into mautrix and Go-NEB.

For my third month I proposed to implement a new Matrix feature called cross-signing, which was similar to SAS verification but dealt with verifying other users instead of their devices who would in turn verify their own devices, creating a graph of trust between users and devices. This functionality was strongly coupled with SSSS, another new feature that allows clients to store their encrypted secrets (in this case the multiple necessary signing keys) on the server, as well as in-room verification which allows verification between users. This task was more challenging as it uses multiple algorithms (some of which not in libolm) for deriving the keys, using them to encrypt other keys which in turn sign other keys. It was also more satisfying when I finally managed to generate the keys, store and retrieve them and then use them to sign another user and when I was done I felt like my understanding of cryptography levelled up.

All in all, I'm very satisfied with the overall experience. The spec was very clear in most cases and when it wasn't the community was always helpful and responsive. They were also happy to discuss the spec with me and explain the more intricate details. For that reason I wasn't asking questions directly to my mentors (besides for defining my tasks) but to the overall community. I'm also very grateful to my mentors, @uhoreg and @Kegan for picking me for this project and helping me with the planning aspect and reviewing my PRs. I'd lastly like to thank the maintainer of mautrix, @tulir, for reviewing and merging a lot of my work into his library.

NextPage 2