Hi everyone. I'm Kegan, one of the core developers at matrix.org. This is the first in a series on the matrix.org IRC bridge. The aim of this series is to try to give a behind the scenes look at how the IRC bridge works, what kinds of problems we encountered, and how we plan to scale in the future. This post looks at how the IRC bridge actually works.
Firstly, what is "bridging"? The simple answer is that it is a program which maps between different messaging protocols so that users on different protocols can communicate with each other. Some protocols may have features which are not supported in the other (typing notifications in Matrix, DCC - direct file transfers - in IRC). This means that bridging will always be "inferior" to just using the respective protocol. That being said, where there is common ground a bridge can work well; all messaging protocols support sending and receiving text messages for example. As we'll see however, the devil is in the detail...
A lot of existing IRC bridges for different protocols share one thing in common: they use a single global bot to bridge traffic. This bot listens to all messages from IRC, and sends them to the other network. The bot also listens for messages from users on the other network, and sends messages on their behalf to IRC. This is a lot easier than having to maintain dedicated TCP connections for each user. However, it isn't a great experience for IRC users as they:
- Don't know who is reading messages on a channel as there is just 1 bot in the membership list.
- Cannot PM users on the other network.
- Cannot kick/ban users on the other network without affecting everyone else.
- Cannot bing/mention users on the other network easily (tab completion).
- It involves multiple connections to the IRCd so you need special permission to set up an i:line.
- You need to be able to support identification of individual users (via ident or unique IPv6 addresses).
- With all these connections to the same IRC channels, you need to have some way to identify which incoming messages have already been handled and which have not.
So now that we have a way to send and receive messages, how do we map the rooms/channels between protocols? This isn't as easy as you may think. We can have a single static one-to-one mapping:
- All messages to
- All messages from
- All PMs between
@alice:matrix.organd Bob go to
!wxyz:matrix.organd the respective PM on IRC.
Then you have problem of "ownership" of rooms. Who should be able to kick users in a bridged room? There are two main scenarios to consider:
- The IRC channel has existed for a while and there are existing IRC channel operators.
- The IRC channel does not exist, but there are existing Matrix moderators.
+o), and then downgrade them back to a regular user (e.g.
-o). Instead, the bridge bot itself acts as a custodian for the room, and performs privileged IRC operations (topic changing, kickbans, etc) on the IRC channel operator's behalf.
In the second case, we want to defer ownership to the Matrix moderators. This is what happens when you "provision a room" in Matrix. The bridge will PM a currently online channel operator and ask for their permission to bridge to Matrix. If they accept, the bridge is made and the power levels in the pre-existing Matrix room are left untouched, giving moderators in Matrix control over the room. However, this power doesn't extend completely to IRC. If a Matrix moderator grants moderator powers to another Matrix user, this will not be mapped to IRC. Why? It's not possible for the bridge to give chanops to any random user on any random IRC channel, so it cannot always honour the request. This relies on the humans on either side of the bridge to communicate and map power accordingly. This is done on purpose as there is no 100% perfect mapping between IRC powers and Matrix powers: it's always going to need to compromise which only a human can make.
Finally, there is the problem of one-to-many mappings. It is possible to have two Matrix rooms bridged to the same IRC channel. The problem occurs when a Matrix user in one room speaks. The bridge can easily map that to IRC, but unless it also maps it back to Matrix, the message will never make it to the 2nd Matrix room. The bridge cannot control/puppet the Matrix user who spoke, so instead it creates a virtual Matrix user to represent that real Matrix user and then sends the message into the 2nd Matrix room. Needless to say, this can be quite confusing and we strongly discourage one-to-many mappings for this reason.
Mapping Matrix messages to IRC is rather easy for the most part. Messages are passed from the Homeserver to the bridge via the AS API, and the bridge sends a textual representation of the message to IRC using the IRC connection for that Matrix user. The exact form of the text for images, videos and long text can be quite subjective, and there is inevitably some data loss along the way. For example, you can send big text headings, tables and lists in Matrix, but there is no equivalent on IRC. Thankfully, most Matrix users are sending the corresponding markdown and so the formatting can be reasonably preserved by just sending the plaintext (markdown) body.
Mapping IRC messages to Matrix is more difficult: not because it's hard to represent the message in Matrix, but because of the architecture of the bridge. The bridge maintains separate connections for each Matrix user. This means the bridge might have, for example, 5 users (and hence connections) on the same channel. When an IRC user sends a message, the bridge gets 5 copies of the message. How does the bridge know:
- If the message has already been sent?
- If the message is an intentional duplicate?
Admin rooms are private Matrix rooms between a real Matrix user and the bridge bot. It allows the Matrix user to control their connection to IRC. It allows:
- The IRC nick to be changed.
- The ability to issue /whois commands.
- The ability to bypass the bridge and send raw IRC commands directly down the TCP connection (e.g. MODE commands).
- The ability to save a NickServ password for use when the bridge reconnects you.
- The ability to disconnect from the network entirely.
!whois $ARG. Like all commands, you expect to get a reply once you've issued it. However, IRC makes this extremely difficult to do. There is no request/response pair like there is with HTTP requests. Instead, the IRC server may:
- Ignore the request entirely.
- Send an error you're aware of (in the RFC/most servers)
- Send some information which can be assumed to indicate success.
- Send an error you're unaware of.
- Send some information which sometimes indicates success.
Developing a comprehensive IRC bridge is a very difficult task. This post has outlined a few of the ways in which we've designed our bridge, and some of the general problems in this field. The bridge is constantly improving as we discover new edge cases with the plethora of IRCd implementations out there. The next post will look at some of these edge cases and look back at some previous outages and examine why they occurred.