GSOC report: Ruma procedural macro refactoring and more

This is part of a series of reports on the six projects assigned to Matrix for Google Summer of Code 2020.

View project: Ruma procedural macro refactoring and more

Wow the summer has flown by, it feels like just yesterday I was learning how to rebase and what exactly it is Ruma does. I exaggerate slightly, but it is a big library with lots of public API surface. I have learned more in the last few months than in two years of school. I have been able to observe and participate in a project with a community growing around it, been a part of discussions about design and best practices, given and received numerous code reviews as well as learned the process of addressing the feedback, and working from a specification. In short, this has been an amazing opportunity to gain experience in all the things that are hard to obtain in a classroom.

My project goal was to improve the existing macros in ruma-events-macros and ruma-api-macros. It became clear early on that this would include some major API changes and that improving the macros as they were was pointless without also moving to a new public API. While improving the durability and readability of the macro code I also rewrote entire sections to accommodate the new design.

A quick overview of the Matrix protocol for reference: a client sends content that is interpreted by the server as events. The server distributes those events out to other clients and other servers (the server case is known as federation). Ruma groups these events by kind Message, State, Ephemeral, ToDevice and Basic which are represented as generic structs (StateEvent<C>). Each event kind needs to be able to hold many different content types, for state events there is room creation, room name, and membership events to name a few. Using the macros, enums are generated to represent all state event possibilities, so a variant for membership, room name, etc. These types exist to support the core API request and response types for each endpoint that is defined by the Matrix specification.

GSoC Starts and I start with ruma-events

• remove old ruma-events macros
• start work for event content derive, move to generic Event structs
• collect all content types into enum
• add more types, making sure the macro works
• work on Event custom macro derive
• remove raw modules and FromRaw trait
• add each events enum
• add trybuild tests for ruma-events

More work on ruma-events, now a part of the new mono-repo

• deserialization optimization for Any*Event, and benchmarking to verify
• reverting the removal of enum_enum macro, learning a lot of git along the way
• it took me awhile but I got the hang of it, only one review issue!
• add accessor methods to the enums, making them much nicer to use

Work on ruma-client-api

• refactoring the Api::to_tokens code helped me get my head around this crate
• notice my commit messages are starting to looking better
• not everything works out...
• but you get it eventually
• deep dive into rust features, I learned a lot about them

Back to ruma-events

• a ruma events and client-api rename
• add code to generate redact method in event_enum!
• adding to existing code without breaking everything is hard
• converting to and from the rusty way
• I love writing proc_macros, this was a fun one! Adding lifetimes in the req/resp macro

Continuing maintenance

• quality of life stuff, the day to day work of a maintainer
• allow the internal macros to be used outside of the ruma crate more easily
• adding small needed features to the macros
• good ol' fashioned closing issues!

One of my personal goals was to become more familiar with git. With the help of my mentor I now feel more confident using this tool that is so essential to developers. I became fairly adept at merging, rebasing, and navigating all the headaches that come with that. I learned plenty of new commands. A few highlights: cherry-pick and specific uses of reset to avoid copy-pasting fixes and adding more commits. I used the reset command to craft good commits, splitting work into appropriate chunks. I am glad that I had the opportunity to hone my git skills. I feel like I have accomplished my goal and then some!

I am proud of the work that I have done: Being part of moving ruma-events much closer to the 0.22 release and creating macros to generate types specific to the Matrix specification. Working with the community that has grown around Ruma has been rewarding and I plan on sticking around.