Federation API

Matrix homeservers use the Federation APIs (also known as server-server APIs) to communicate with each other. Homeservers use these APIs to push messages to each other in real-time, to request historic messages from each other, and to query profile and presence information about users on each other's servers.

The APIs are implemented using HTTPS GETs and PUTs between each of the servers. These HTTPS requests are strongly authenticated using public key signatures at the TLS transport layer and using public key signatures in HTTP Authorization headers at the HTTP layer.

There are three main kinds of communication that occur between homeservers:

Persisted Data Units (PDUs):

These events are broadcast from one homeserver to any others that have joined the same room (identified by Room ID). They are persisted in long-term storage and record the history of messages and state for a room.

Like email, it is the responsibility of the originating server of a PDU to deliver that event to its recipient servers. However PDUs are signed using the originating server's private key so that it is possible to deliver them through third-party servers.

Ephemeral Data Units (EDUs):
These events are pushed between pairs of homeservers. They are not persisted and are not part of the history of a room, nor does the receiving homeserver have to reply to them.
Queries:
These are single request/response interactions between a given pair of servers, initiated by one side sending an HTTPS GET request to obtain some information, and responded by the other. They are not persisted and contain no long-term significant history. They simply request a snapshot state at the instant the query is made.

EDUs and PDUs are further wrapped in an envelope called a Transaction, which is transferred from the origin to the destination homeserver using an HTTPS PUT request.

1   Specification version

This version of the specification is generated from matrix-doc as of Git commit HEAD,f0f91bd.

2   Server Discovery

2.1   Resolving Server Names

Each matrix homeserver is identified by a server name consisting of a DNS name and an optional TLS port.

server_name = dns_name [ ":" tls_port]
dns_name = <host, see [RFC 3986], Section 3.2.2>
tls_port = *DIGIT

If the port is present then the server is discovered by looking up an AAAA or A record for the DNS name and connecting to the specified TLS port. If the port is absent then the server is discovered by looking up a _matrix._tcp SRV record for the DNS name. If this record does not exist then the server is discovered by looking up an AAAA or A record on the DNS name and taking the default fallback port number of 8448. Homeservers may use SRV records to load balance requests between multiple TLS endpoints or to failover to another endpoint if an endpoint fails.

2.2   Server implementation

2.2.1   GET /_matrix/federation/v1/version

Get the implementation name and version of this homeserver.

Request format:

No parameters

Response format:

Parameter Type Description
server Server  

Server

Parameter Type Description
name string Arbitrary name that identify this implementation.
version string Version of this implementation. The version format depends on the implementation.

Example request:

GET /_matrix/federation/v1/version HTTP/1.1

Response:

Status code 200:

The implementation name and version of this homeserver.

Example

{
  "server": {
    "name": "My_Homeserver_Implementation",
    "version": "ArbitraryVersionNumber"
  }
}

2.3   Retrieving Server Keys

2.3.1   Version 2

Each homeserver publishes its public keys under /_matrix/key/v2/server/. Homeservers query for keys by either getting /_matrix/key/v2/server/ directly or by querying an intermediate notary server using a /_matrix/key/v2/query API. Intermediate notary servers query the /_matrix/key/v2/server/ API on behalf of another server and sign the response with their own key. A server may query multiple notary servers to ensure that they all report the same public keys.

This approach is borrowed from the Perspectives Project, but modified to include the NACL keys and to use JSON instead of XML. It has the advantage of avoiding a single trust-root since each server is free to pick which notary servers they trust and can corroborate the keys returned by a given notary server by querying other servers.

2.3.1.1   Publishing Keys

Homeservers publish the allowed TLS fingerprints and signing keys in a JSON object at /_matrix/key/v2/server/{key_id}. The response contains a list of verify_keys that are valid for signing federation requests made by the server and for signing events. It contains a list of old_verify_keys which are only valid for signing events. Finally the response contains a list of TLS certificate fingerprints to validate any connection made to the server.

A server may have multiple keys active at a given time. A server may have any number of old keys. It is recommended that servers return a single JSON response listing all of its keys whenever any key_id is requested to reduce the number of round trips needed to discover the relevant keys for a server. However a server may return a different responses for a different key_id.

The tls_certificates contain a list of hashes of the X.509 TLS certificates currently used by the server. The list must include SHA-256 hashes for every certificate currently in use by the server. These fingerprints are valid until the millisecond POSIX timestamp in valid_until_ts.

The verify_keys can be used to sign requests and events made by the server until the millisecond POSIX timestamp in valid_until_ts. If a homeserver receives an event with a origin_server_ts after the valid_until_ts then it should request that key_id for the originating server to check whether the key has expired.

The old_verify_keys can be used to sign events with an origin_server_ts before the expired_ts. The expired_ts is a millisecond POSIX timestamp of when the originating server stopped using that key.

Intermediate notary servers should cache a response for half of its remaining life time to avoid serving a stale response. Originating servers should avoid returning responses that expire in less than an hour to avoid repeated requests for an about to expire certificate. Requesting servers should limit how frequently they query for certificates to avoid flooding a server with requests.

If a server goes offline intermediate notary servers should continue to return the last response they received from that server so that the signatures of old events sent by that server can still be checked.

Key Type Description
server_name String DNS name of the homeserver.
verify_keys Object Public keys of the homeserver for verifying digital signatures.
old_verify_keys Object The public keys that the server used to use and when it stopped using them.
signatures Object Digital signatures for this object signed using the verify_keys.
tls_fingerprints Array of Objects Hashes of X.509 TLS certificates used by this this server encoded as Unpadded Base64.
valid_until_ts Integer POSIX timestamp when the list of valid keys should be refreshed.
{
    "old_verify_keys": {
        "ed25519:auto1": {
            "expired_ts": 922834800000,
            "key": "Base+64+Encoded+Old+Verify+Key"
        }
    },
    "server_name": "example.org",
    "signatures": {
        "example.org": {
            "ed25519:auto2": "Base+64+Encoded+Signature"
        }
    },
    "tls_fingerprints": [
        {
            "sha256": "Base+64+Encoded+SHA-256-Fingerprint"
        }
    ],
    "valid_until_ts": 1052262000000,
    "verify_keys": {
        "ed25519:auto2": {
            "key": "Base+64+Encoded+Signature+Verification+Key"
        }
    }
}

2.3.1.2   Querying Keys Through Another Server

Servers may offer a query API _matrix/key/v2/query/ for getting the keys for another server. This API can be used to GET at list of JSON objects for a given server or to POST a bulk query for a number of keys from a number of servers. Either way the response is a list of JSON objects containing the JSON published by the server under _matrix/key/v2/server/ signed by both the originating server and by this server.

The minimum_valid_until_ts is a millisecond POSIX timestamp indicating when the returned certificate will need to be valid until to be useful to the requesting server. This can be set using the maximum origin_server_ts of an batch of events that a requesting server is trying to validate. This allows an intermediate notary server to give a prompt cached response even if the originating server is offline.

This API can return keys for servers that are offline be using cached responses taken from when the server was online. Keys can be queried from multiple servers to mitigate against DNS spoofing.

Requests:

GET /_matrix/key/v2/query/${server_name}/${key_id}/?minimum_valid_until_ts=${minimum_valid_until_ts} HTTP/1.1

POST /_matrix/key/v2/query HTTP/1.1
Content-Type: application/json

{
    "server_keys": {
        "$server_name": {
            "$key_id": {
                "minimum_valid_until_ts": $posix_timestamp
            }
        }
    }
}

Response:

HTTP/1.1 200 OK
Content-Type: application/json
{
    "server_keys": [
       # List of responses with same format as /_matrix/key/v2/server
       # signed by both the originating server and this server.
    ]
}

2.3.2   Version 1

Warning

Version 1 of key distribution is obsolete

Homeservers publish their TLS certificates and signing keys in a JSON object at /_matrix/key/v1.

Key Type Description
server_name String DNS name of the homeserver.
verify_keys Object Public keys of the homeserver for verifying digital signatures.
signatures Object Digital signatures for this object signed using the verify_keys.
tls_certificate String The X.509 TLS certificate used by this this server encoded as Unpadded Base64.
{
    "server_name": "example.org",
    "signatures": {
        "example.org": {
            "ed25519:auto": "Base+64+Encoded+Signature"
        }
    },
    "tls_certificate": "Base+64+Encoded+DER+Encoded+X509+TLS+Certificate",
    "verify_keys": {
        "ed25519:auto": "Base+64+Encoded+Signature+Verification+Key"
    }
}

When fetching the keys for a server the client should check that the TLS certificate in the JSON matches the TLS server certificate for the connection and should check that the JSON signatures are correct for the supplied verify_keys

3   Transactions

The transfer of EDUs and PDUs between homeservers is performed by an exchange of Transaction messages, which are encoded as JSON objects, passed over an HTTP PUT request. A Transaction is meaningful only to the pair of homeservers that exchanged it; they are not globally-meaningful.

Each transaction has:
  • An opaque transaction ID, unique among transactions from the same origin.
  • A timestamp (UNIX epoch time in milliseconds) generated by its origin server.
  • An origin and destination server name.
  • A list of PDUs and EDUs - the actual message payload that the Transaction carries.

3.1   Transaction Fields

Key Type Description
origin String Required. server_name of homeserver sending this transaction.
origin_server_ts Integer Required. Timestamp in milliseconds on originating homeserver when this transaction started.
pdus List of Objects Required. List of persistent updates to rooms.
edus List of Objects List of ephemeral messages. May be omitted if there are no ephemeral messages to be sent.

Example:

{
 "origin_server_ts":1404835423000,
 "origin":"matrix.org",
 "pdus":[...],
 "edus":[...]
}

4   PDUs

Each PDU contains a single Room Event which the origin server wants to send to the destination.

4.1   PDU Fields

Key Type Description
room_id String Required. Room identifier.
sender String Required. The ID of the user sending the event.
origin String Required. server_name of the homeserver that created this event.
event_id String Required. Unique identifier for the event being sent.
origin_server_ts Integer Required. Timestamp in milliseconds on origin homeserver when this event was created.
type String Required. Event type
state_key String Optional. If this key is present, the event is a state event, and it will replace previous events with the same type and state_key in the room state.
content Object Required. The content of the event.
prev_events List of (String, {String: String}) pairs Required. Event IDs and hashes of the most recent events in the room that the homeserver was aware of when it made this event
depth Integer Required. The maximum depth of the prev_events, plus one
auth_events List of (String, {String: String}) pairs Required. Event IDs and hashes for the "auth events" of this event.
hashes {String: String} Required. Hashes of the PDU, following the algorithm specified in Signing Events.
signatures {String: {String: String}} Required. Signatures of the redacted PDU, following the algorithm specified in Signing Events.
redacts String Optional. For redaction events, the ID of the event being redacted
unsigned Object Optional. Additional data added by the origin server but not covered by the signatures.

Example:

{
 "room_id": "!UcYsUzyxTGDxLBEvLy:example.org",
 "sender": "@alice:example.com",
 "origin": "example.com",
 "event_id": "$a4ecee13e2accdadf56c1025:example.com",
 "origin_server_ts": 1404838188000,
 "type": "m.room.message",
 "prev_events": [
   ["$af232176:example.org", {"sha256": "abase64encodedsha256hashshouldbe43byteslong"}]
 ],
 "hashes": {"sha256": "thishashcoversallfieldsincasethisisredacted"},
 "signatures": {
   "example.com": {
     "ed25519:key_version:": "these86bytesofbase64signaturecoveressentialfieldsincludinghashessocancheckredactedpdus"
   }
 },
 "content": {...}
}

The prev_events field of a PDU identifies the "parents" of the event, and thus establishes a partial ordering on events within the room by linking them into a Directed Acyclic Graph (DAG). The sending server should populate this field with all of the events in the room for which it has not yet seen a child - thus demonstrating that the event comes after all other known events.

For example, consider a room whose events form the DAG shown below. A server creating a new event in this room should populate the new event's prev_events field with E4 and E5, since neither event yet has a child:

    E1
    ^
    |
+-> E2 <-+
|        |
E3       E5
^
|
E4

The auth_events field of a PDU identifies the set of events which give the sender permission to send the event. The auth_events for the m.room.create event in a room is empty; for other events, it should be the following subset of the room state:

  • The m.room.create event.
  • The current m.room.power_levels event, if any.
  • The current m.room.join_rules event, if any.
  • The sender's current m.room.member event, if any.

4.2   Authorization of PDUs

Whenever a server receives an event from a remote server, the receiving server must check that the event is allowed by the authorization rules. These rules depend on the state of the room at that event.

4.2.1   Definitions

Required Power Level
A given event type has an associated required power level. This is given by the current m.room.power_levels event. The event type is either listed explicitly in the events section or given by either state_default or events_default depending on if the event is a state event or not.
Invite Level, Kick Level, Ban Level, Redact Level
The levels given by the invite, kick, ban, and redact properties in the current m.room.power_levels state. Each defaults to 50 if unspecified.
Target User
For an m.room.member state event, the user given by the state_key of the event.

4.2.2   Rules

The rules governing whether an event is authorized depend solely on the state of the room at the point in the room graph at which the new event is to be inserted. The types of state events that affect authorization are:

  • m.room.create
  • m.room.member
  • m.room.join_rules
  • m.room.power_levels

Servers should not create new events that reference unauthorized events. However, any event that does reference an unauthorized event is not itself automatically considered unauthorized.

Unauthorized events that appear in the event graph do not have any effect on the state of the room.

Note

This is in contrast to redacted events which can still affect the state of the room. For example, a redacted join event will still result in the user being considered joined.

  1. If type is m.room.create, allow if and only if it has no previous events - i.e. it is the first event in the room.
  2. If type is m.room.member:
    1. If membership is join:
      1. If the only previous event is an m.room.create and the state_key is the creator, allow.
      2. If the sender does not match state_key, reject.
      3. If the user's current membership state is invite or join, allow.
      4. If the join_rule is public, allow.
      5. Otherwise, reject.
    2. If membership is invite:
      1. If the sender's current membership state is not joined, reject.
      2. If target user's current membership state is join or ban, reject.
      3. If the sender's power level is greater than or equal to the invite level, allow.
      4. Otherwise, reject.
    3. If membership is leave:
      1. If the sender matches state_key, allow if and only if that user's current membership state is invite or join.
      2. If the sender's current membership state is not joined, reject.
      3. If the target user's current membership state is ban, and the sender's power level is less than the ban level, reject.
      4. If the sender's power level is greater than or equal to the kick level, and the target user's power level is less than the sender's power level, allow.
      5. Otherwise, reject.
    4. If membership is ban:
      1. If the sender's current membership state is not joined, reject.
      2. If the sender's power level is greater than or equal to the ban level, and the target user's power level is less than the sender's power level, allow.
      3. Otherwise, reject.
    5. Otherwise, the membership is unknown. Reject.
  3. If the sender's current membership state is not joined, reject.
  4. If the event type's required power level is greater than the sender's power level, reject.
  5. If type is m.room.power_levels:
    1. If there is no previous m.room.power_levels event in the room, allow.
    2. For each of the keys users_default, events_default, state_default, ban, redact, kick, invite, as well as each entry being changed under the events or users keys:
      1. If the current value is higher than the sender's current power level, reject.
      2. If the new value is higher than the sender's current power level, reject.
    3. For each entry being changed under the users key, other than the sender's own entry:
      1. If the current value is equal to the sender's current power level, reject.
    4. Otherwise, allow.
  6. If type is m.room.redaction:
    1. If the sender's power level is greater than or equal to the redact level, allow.
    2. If the sender of the event being redacted is the same as the sender of the m.room.redaction, allow.
    3. Otherwise, reject.
  7. Otherwise, allow.

Note

Some consequences of these rules:

  • Unless you are a member of the room, the only permitted operations (apart from the intial create/join) are: joining a public room; accepting or rejecting an invitation to a room.
  • To unban somebody, you must have power level greater than or equal to both the kick and ban levels, and greater than the target user's power level.

5   EDUs

Warning

This section may be misleading or inaccurate.

EDUs, by comparison to PDUs, do not have an ID, a room ID, or a list of "previous" IDs. The only mandatory fields for these are the type, origin and destination homeserver names, and the actual nested content.

Key Type Description
edu_type String The type of the ephemeral message.
content Object Content of the ephemeral message.
{
 "edu_type":"m.presence",
 "origin":"blue",
 "destination":"orange",
 "content":{...}
}

6   Room State Resolution

The state of a room is a map of (event_type, state_key) to event_id. Each room starts with an empty state, and each state event which is accepted into the room updates the state of that room.

Where each event has a single prev_event, it is clear what the state of the room after each event should be. However, when two branches in the event graph merge, the state of those branches might differ, so a state resolution algorithm must be used to determine the resultant state.

For example, consider the following event graph (where the oldest event, E0, is at the top):

  E0
  |
  E1
 /  \
E2  E4
|    |
E3   |
 \  /
  E5

Suppose E3 and E4 are both m.room.name events which set the name of the room. What should the name of the room be at E5?

Servers should follow the following recursively-defined algorithm to determine the room state at a given point on the DAG.

6.1   State resolution algorithm

Warning

This section documents the state resolution algorithm as implemented by Synapse as of December 2017 (and therefore the de-facto Matrix protocol). However, this algorithm is known to have some problems.

The room state S’(E) after an event E is defined in terms of the room state S(E) before E, and depends on whether E is a state event or a message event:

  • If E is a message event, then S’(E) = S(E).
  • If E is a state event, then S’(E) is S(E), except that its entry corresponding to E's event_type and state_key is replaced by E's event_id.

The room state S(E) before E is the resolution of the set of states {S’(E’), S’(E’’), …} consisting of the states after each of E's prev_events {E’, E’’, …}.

The resolution of a set of states is defined as follows. The resolved state is built up in a number of passes; here we use R to refer to the results of the resolution so far.

  • Start by setting R to the union of the states to be resolved, excluding any conflicting events.
  • First we resolve conflicts between m.room.power_levels events. If there is no conflict, this step is skipped, otherwise:
    • Assemble all the m.room.power_levels events from the states to be resolved into a list.
    • Sort the list by ascending depth then descending sha1(event_id).
    • Add the first event in the list to R.
    • For each subsequent event in the list, check that the event would be allowed by the authorization rules for a room in state R. If the event would be allowed, then update R with the event and continue with the next event in the list. If it would not be allowed, stop and continue below with m.room.join_rules events.
  • Repeat the above process for conflicts between m.room.join_rules events.
  • Repeat the above process for conflicts between m.room.member events.
  • No other events affect the authorization rules, so for all other conflicts, just pick the event with the highest depth and lowest sha1(event_id) that passes authentication in R and add it to R.

A conflict occurs between states where those states have different event_ids for the same (state_type, state_key). The events thus affected are said to be conflicting events.

7   Protocol URLs

Warning

This section may be misleading or inaccurate.

All these URLs are name-spaced within a prefix of:

/_matrix/federation/v1/...

For active pushing of messages representing live activity "as it happens":

PUT .../send/<transaction_id>/
  Body: JSON encoding of a single Transaction
  Response: TODO-doc

The transaction_id path argument will override any ID given in the JSON body. The destination name will be set to that of the receiving server itself. Each embedded PDU in the transaction body will be processed.

To fetch all the state of a given room:

GET .../state/<room_id>/
  Response: JSON encoding of a single Transaction containing multiple PDUs

Retrieves a snapshot of the entire current state of the given room. The response will contain a single Transaction, inside which will be a list of PDUs that encode the state.

To fetch a particular event:

GET .../event/<event_id>/
  Response: JSON encoding of a partial Transaction containing the event

Retrieves a single event. The response will contain a partial Transaction, having just the origin, origin_server_ts and pdus fields; the event will be encoded as the only PDU in the pdus list.

To backfill events on a given room:

GET .../backfill/<room_id>/
  Query args: v, limit
  Response: JSON encoding of a single Transaction containing multiple PDUs

Retrieves a sliding-window history of previous PDUs that occurred on the given room. Starting from the PDU ID(s) given in the "v" argument, the PDUs that preceded it are retrieved, up to a total number given by the "limit" argument.

To stream events all the events:

GET .../pull/
  Query args: origin, v
  Response: JSON encoding of a single Transaction consisting of multiple PDUs

Retrieves all of the transactions later than any version given by the "v" arguments.

To make a query:

GET .../query/<query_type>
  Query args: as specified by the individual query types
  Response: JSON encoding of a response object

Performs a single query request on the receiving homeserver. The Query Type part of the path specifies the kind of query being made, and its query arguments have a meaning specific to that kind of query. The response is a JSON-encoded object whose meaning also depends on the kind of query.

To join a room:

GET .../make_join/<room_id>/<user_id>
  Response: JSON encoding of a join proto-event

PUT .../send_join/<room_id>/<event_id>
  Response: JSON encoding of the state of the room at the time of the event

Performs the room join handshake. For more information, see "Joining Rooms" below.

8   Joining Rooms

When a new user wishes to join room that the user's homeserver already knows about, the homeserver can immediately determine if this is allowable by inspecting the state of the room, and if it is acceptable, it can generate, sign, and emit a new m.room.member state event adding the user into that room. When the homeserver does not yet know about the room it cannot do this directly. Instead, it must take a longer multi-stage handshaking process by which it first selects a remote homeserver which is already participating in that room, and uses it to assist in the joining process. This is the remote join handshake.

This handshake involves the homeserver of the new member wishing to join (referred to here as the "joining" server), the directory server hosting the room alias the user is requesting to join with, and a homeserver where existing room members are already present (referred to as the "resident" server).

In summary, the remote join handshake consists of the joining server querying the directory server for information about the room alias; receiving a room ID and a list of join candidates. The joining server then requests information about the room from one of the residents. It uses this information to construct a m.room.member event which it finally sends to a resident server.

Conceptually these are three different roles of homeserver. In practice the directory server is likely to be resident in the room, and so may be selected by the joining server to be the assisting resident. Likewise, it is likely that the joining server picks the same candidate resident for both phases of event construction, though in principle any valid candidate may be used at each time. Thus, any join handshake can potentially involve anywhere from two to four homeservers, though most in practice will use just two.

Client         Joining                Directory       Resident
               Server                 Server          Server

join request -->
               |
               directory request ------->
               <---------- directory response
               |
               make_join request ----------------------->
               <------------------------------- make_join response
               |
               send_join request ----------------------->
               <------------------------------- send_join response
               |
<---------- join response

The first part of the handshake usually involves using the directory server to request the room ID and join candidates. This is covered in more detail on the directory server documentation, below. In the case of a new user joining a room as a result of a received invite, the joining user's homeserver could optimise this step away by picking the origin server of that invite message as the join candidate. However, the joining server should be aware that the origin server of the invite might since have left the room, so should be prepared to fall back on the regular join flow if this optimisation fails.

Once the joining server has the room ID and the join candidates, it then needs to obtain enough information about the room to fill in the required fields of the m.room.member event. It obtains this by selecting a resident from the candidate list, and requesting the make_join endpoint using a GET request, specifying the room ID and the user ID of the new member who is attempting to join.

The resident server replies to this request with a JSON-encoded object having a single key called event; within this is an object whose fields contain some of the information that the joining server will need. Despite its name, this object is not a full event; notably it does not need to be hashed or signed by the resident homeserver. The required fields are:

Key Type Description
type String The value m.room.member
auth_events List An event-reference list containing the authorization events that would allow this member to join
content Object The event content
depth Integer (this field must be present but is ignored; it may be 0)
origin String The name of the resident homeserver
origin_server_ts Integer A timestamp added by the resident homeserver
prev_events List An event-reference list containing the immediate predecessor events
room_id String The room ID of the room
sender String The user ID of the joining member
state_key String The user ID of the joining member

The content field itself must be an object, containing:

Key Type Description
membership String The value join

The joining server now has sufficient information to construct the real join event from these protoevent fields. It copies the values of most of them, adding (or replacing) the following fields:

Key Type Description
event_id String A new event ID specified by the joining homeserver
origin String The name of the joining homeserver
origin_server_ts Integer A timestamp added by the joining homeserver

This will be a true event, so the joining server should apply the event-signing algorithm to it, resulting in the addition of the hashes and signatures fields.

To complete the join handshake, the joining server must now submit this new event to an resident homeserver, by using the send_join endpoint. This is invoked using the room ID and the event ID of the new member event.

The resident homeserver then accepts this event into the room's event graph, and responds to the joining server with the full set of state for the newly- joined room. This is returned as a two-element list, whose first element is the integer 200, and whose second element is an object which contains the following keys:

Key Type Description
auth_chain List A list of events giving the authorization chain for this join event
state List A complete list of the prevailing state events at the instant just before accepting the new m.room.member event

9   Backfilling

Once a homeserver has joined a room, it receives all the events emitted by other homeservers in that room, and is thus aware of the entire history of the room from that moment onwards. Since users in that room are able to request the history by the /messages client API endpoint, it's possible that they might step backwards far enough into history before the homeserver itself was a member of that room.

To cover this case, the federation API provides a server-to-server analog of the /messages client API, allowing one homeserver to fetch history from another. This is the /backfill API.

To request more history, the requesting homeserver picks another homeserver that it thinks may have more (most likely this should be a homeserver for some of the existing users in the room at the earliest point in history it has currently), and makes a /backfill request. The parameters of this request give an event ID that the requesting homeserver wishes to obtain, and a number specifying how many more events of history before that one to return at most.

The response to this request is an object with the following keys:

Key Type Description
pdus List A list of events
origin String The name of the resident homeserver
origin_server_ts Integer A timestamp added by the resident homeserver

The list of events given in pdus is returned in reverse chronological order; having the most recent event first (i.e. the event whose event ID is that requested by the requestor in the v parameter).

10   Inviting to a room

When a user wishes to invite an other user to a local room and the other user is on a different server, the inviting server will send a request to the invited server:

PUT .../invite/{roomId}/{eventId}

The required fields in the JSON body are:

Key Type Description
room_id String The room ID of the room. Must be the same as the room ID specified in the path.
event_id String The ID of the event. Must be the same as the event ID specified in the path.
type String The value m.room.member.
auth_events List An event-reference list containing the IDs of the authorization events that would allow this member to be invited in the room.
content Object The content of the event.
depth Integer The depth of the event.
origin String The name of the inviting homeserver.
origin_server_ts Integer A timestamp added by the inviting homeserver.
prev_events List An event-reference list containing the IDs of the immediate predecessor events.
sender String The Matrix ID of the user who sent the original m.room.third_party_invite.
state_key String The Matrix ID of the invited user.
signatures Object The signature of the event from the origin server.
unsigned Object An object containing the properties that aren't part of the signature's computation.

Where the content key contains the content for the m.room.member event specified in the Client-Server API. Note that the membership property of the content must be invite.

Upon receiving this request, the invited homeserver will append its signature to the event and respond to the request with the following JSON body:

[
  200,
  "event": {...}
]

Where event contains the event signed by both homeservers, using the same JSON keys as the initial request on /invite/{roomId}/{eventId}. Note that, except for the signatures object (which now contains an additional signature), all of the event's keys remain the same as in the event initially provided.

This response format is due to a typo in Synapse, the first implementation of Matrix's APIs, and is preserved to maintain compatibility.

Now that the event has been signed by both the inviting homeserver and the invited homeserver, it can be sent to all of the users in the room.

11   Third-party invites

When an user wants to invite another user in a room but doesn't know the Matrix ID to invite, they can do so using a third-party identifier (e.g. an e-mail or a phone number).

This identifier and its bindings to Matrix IDs are verified by an identity server implementing the Identity Service API.

11.1   Cases where an association exists for a third-party identifier

If the third-party identifier is already bound to a Matrix ID, a lookup request on the identity server will return it. The invite is then processed by the inviting homeserver as a standard m.room.member invite event. This is the simplest case.

11.2   Cases where an association doesn't exist for a third-party identifier

If the third-party identifier isn't bound to any Matrix ID, the inviting homeserver will request the identity server to store an invite for this identifier and to deliver it to whoever binds it to its Matrix ID. It will also send a m.room.third_party_invite event in the room to specify a display name, a token and public keys the identity server provided as a response to the invite storage request.

When a third-party identifier with pending invites gets bound to a Matrix ID, the identity server will send a POST request to the ID's homeserver as described in the Invitation Storage section of the Identity Service API.

The following process applies for each invite sent by the identity server:

The invited homeserver will create a m.room.member invite event containing a special third_party_invite section containing the token and a signed object, both provided by the identity server.

If the invited homeserver is in the room the invite came from, it can auth the event and send it.

However, if the invited homeserver isn't in the room the invite came from, it will need to request the room's homeserver to auth the event:

PUT .../exchange_third_party_invite/{roomId}

Where roomId is the ID of the room the invite is for.

The required fields in the JSON body are:

Key Type Description
type String The event type. Must be m.room.member.
room_id String The ID of the room the event is for. Must be the same as the ID specified in the path.
sender String The Matrix ID of the user who sent the original m.room.third_party_invite.
state_key String The Matrix ID of the invited user.
content Object The content of the event.

Where the content key contains the content for the m.room.member event as described in the Client-Server API. Its membership key must be invite and its content must include the third_party_invite object.

The inviting homeserver will then be able to authenticate the event. It will send a fully authenticated event to the invited homeserver as described in the Inviting to a room section above.

Once the invited homeserver responded with the event to which it appended its signature, the inviting homeserver will respond with 200 OK and an empty body ({}) to the initial request on /exchange_third_party_invite/{roomId} and send the now verified m.room.member invite event to the room's members.

11.2.1   Verifying the invite

When a homeserver receives a m.room.member invite event for a room it's in with a third_party_invite object, it must verify that the association between the third-party identifier initially invited to the room and the Matrix ID that claims to be bound to it has been verified without having to rely on a third-party server.

To do so, it will fetch from the room's state events the m.room.third_party_invite event for which the state key matches with the value for the token key in the third_party_invite object from the m.room.member event's content to fetch the public keys initially delivered by the identity server that stored the invite.

It will then use these keys to verify that the signed object (in the third_party_invite object from the m.room.member event's content) was signed by the same identity server.

Since this signed object can only be delivered once in the POST request emitted by the identity server upon binding between the third-party identifier and the Matrix ID, and contains the invited user's Matrix ID and the token delivered when the invite was stored, this verification will prove that the m.room.member invite event comes from the user owning the invited third-party identifier.

12   Authentication

12.1   Request Authentication

Every HTTP request made by a homeserver is authenticated using public key digital signatures. The request method, target and body are signed by wrapping them in a JSON object and signing it using the JSON signing algorithm. The resulting signatures are added as an Authorization header with an auth scheme of X-Matrix. Note that the target field should include the full path starting with /_matrix/..., including the ? and any query parameters if present, but should not include the leading https:, nor the destination server's hostname.

Step 1 sign JSON:

 {
     "method": "GET",
     "uri": "/target",
     "origin": "origin.hs.example.com",
     "destintation": "destination.hs.example.com",
     "content": <request body>,
     "signatures": {
         "origin.hs.example.com": {
             "ed25519:key1": "ABCDEF..."
         }
     }
}

Step 2 add Authorization header:

GET /target HTTP/1.1
Authorization: X-Matrix origin=origin.example.com,key="ed25519:key1",sig="ABCDEF..."
Content-Type: application/json

<JSON-encoded request body>

Example python code:

def authorization_headers(origin_name, origin_signing_key,
                          destination_name, request_method, request_target,
                          content=None):
    request_json = {
         "method": request_method,
         "uri": request_target,
         "origin": origin_name,
         "destination": destination_name,
    }

    if content_json is not None:
        request["content"] = content

    signed_json = sign_json(request_json, origin_name, origin_signing_key)

    authorization_headers = []

    for key, sig in signed_json["signatures"][origin_name].items():
        authorization_headers.append(bytes(
            "X-Matrix origin=%s,key=\"%s\",sig=\"%s\"" % (
                origin_name, key, sig,
            )
        ))

    return ("Authorization", authorization_headers)

12.2   Response Authentication

Responses are authenticated by the TLS server certificate. A homeserver should not send a request until it has authenticated the connected server to avoid leaking messages to eavesdroppers.

12.3   Client TLS Certificates

Requests are authenticated at the HTTP layer rather than at the TLS layer because HTTP services like Matrix are often deployed behind load balancers that handle the TLS and these load balancers make it difficult to check TLS client certificates.

A homeserver may provide a TLS client certificate and the receiving homeserver may check that the client certificate matches the certificate of the origin homeserver.

13   Presence

The server API for presence is based entirely on exchange of the following EDUs. There are no PDUs or Federation Queries involved.

Performing a presence update and poll subscription request:

EDU type: m.presence

Content keys:
  push: (optional): list of push operations.
    Each should be an object with the following keys:
      user_id: string containing a User ID
      presence: "offline"|"unavailable"|"online"|"free_for_chat"
      status_msg: (optional) string of free-form text
      last_active_ago: milliseconds since the last activity by the user

  poll: (optional): list of strings giving User IDs

  unpoll: (optional): list of strings giving User IDs

The presence of this combined message is two-fold: it informs the recipient server of the current status of one or more users on the sending server (by the push key), and it maintains the list of users on the recipient server that the sending server is interested in receiving updates for, by adding (by the poll key) or removing them (by the unpoll key). The poll and unpoll lists apply changes to the implied list of users; any existing IDs that the server sent as poll operations in a previous message are not removed until explicitly requested by a later unpoll.

On receipt of a message containing a non-empty poll list, the receiving server should immediately send the sending server a presence update EDU of its own, containing in a push list the current state of every user that was in the original EDU's poll list.

Sending a presence invite:

EDU type: m.presence_invite

Content keys:
  observed_user: string giving the User ID of the user whose presence is
    requested (i.e. the recipient of the invite)
  observer_user: string giving the User ID of the user who is requesting to
    observe the presence (i.e. the sender of the invite)

Accepting a presence invite:

EDU type: m.presence_accept

Content keys - as for m.presence_invite

Rejecting a presence invite:

EDU type: m.presence_deny

Content keys - as for m.presence_invite

14   Profiles

The server API for profiles is based entirely on the following Federation Queries. There are no additional EDU or PDU types involved, other than the implicit m.presence and m.room.member events (see section below).

Querying profile information:

Query type: profile

Arguments:
  user_id: the ID of the user whose profile to return
  field: (optional) string giving a field name

Returns: JSON object containing the following keys:
  displayname: string of free-form text
  avatar_url: string containing an HTTP-scheme URL

If the query contains the optional field key, it should give the name of a result field. If such is present, then the result should contain only a field of that name, with no others present. If not, the result should contain as much of the user's profile as the homeserver has available and can make public.

15   Directory

The server API for directory queries is also based on Federation Queries.

15.1   GET /_matrix/federation/v1/query/directory

Retrieve the room ID and list of resident homeservers for a Room alias.

Request format:

Parameter Type Description
query parameters
room_alias string Required. Room alias

Response format:

Parameter Type Description
room_id string Required. The room ID mapped to the queried room alias.
servers [string] Required. An array of server names that are likely to hold then given room. This list may or may not include the server answering the query.

Example request:

GET /_matrix/federation/v1/query/directory?room_alias=%23room_alias%3Aexample.org HTTP/1.1

Response:

Status code 200:

The corresponding room ID and list of known resident homeservers for the room.

Example

{
  "room_id": "!roomid1234:example.org",
  "servers": [
    "example.org",
    "example.com",
    "another.example.com:8449"
  ]
}

16   Send-to-device messaging

The server API for send-to-device messaging is based on the following EDU. There are no PDUs or Federation Queries involved.

Each send-to-device message should be sent to the destination server using the following EDU:

EDU type: m.direct_to_device

Content keys:
  sender: user ID of the sender

  type: event type for the message

  message_id: unique id for the message: used for idempotence

  messages: The messages to send. A map from user ID, to a map from device ID
      to message body. The device ID may also be *, meaning all known devices
      for the user.

17   Signing Events

Signing events is complicated by the fact that servers can choose to redact non-essential parts of an event.

Before signing the event, the unsigned and signature members are removed, it is encoded as Canonical JSON, and then hashed using SHA-256. The resulting hash is then stored in the event JSON in a hash object under a sha256 key.

def hash_event(event_json_object):

    # Keys under "unsigned" can be modified by other servers.
    # They are useful for conveying information like the age of an
    # event that will change in transit.
    # Since they can be modifed we need to exclude them from the hash.
    unsigned = event_json_object.pop("unsigned", None)

    # Signatures will depend on the current value of the "hashes" key.
    # We cannot add new hashes without invalidating existing signatures.
    signatures = event_json_object.pop("signatures", None)

    # The "hashes" key might contain multiple algorithms if we decide to
    # migrate away from SHA-2. We don't want to include an existing hash
    # output in our hash so we exclude the "hashes" dict from the hash.
    hashes = event_json_object.pop("hashes", {})

    # Encode the JSON using a canonical encoding so that we get the same
    # bytes on every server for the same JSON object.
    event_json_bytes = encode_canonical_json(event_json_bytes)

    # Add the base64 encoded bytes of the hash to the "hashes" dict.
    hashes["sha256"] = encode_base64(sha256(event_json_bytes).digest())

    # Add the "hashes" dict back the event JSON under a "hashes" key.
    event_json_object["hashes"] = hashes
    if unsigned is not None:
        event_json_object["unsigned"] = unsigned
    return event_json_object

The event is then stripped of all non-essential keys both at the top level and within the content object. Any top-level keys not in the following list MUST be removed:

auth_events
depth
event_id
hashes
membership
origin
origin_server_ts
prev_events
prev_state
room_id
sender
signatures
state_key
type

A new content object is constructed for the resulting event that contains only the essential keys of the original content object. If the original event lacked a content object at all, a new empty JSON object is created for it.

The keys that are considered essential for the content object depend on the the type of the event. These are:

type is "m.room.aliases":
  aliases

type is "m.room.create":
  creator

type is "m.room.history_visibility":
  history_visibility

type is "m.room.join_rules":
  join_rule

type is "m.room.member":
  membership

type is "m.room.power_levels":
  ban
  events
  events_default
  kick
  redact
  state_default
  users
  users_default

The resulting stripped object with the new content object and the original hashes key is then signed using the JSON signing algorithm outlined below:

def sign_event(event_json_object, name, key):

    # Make sure the event has a "hashes" key.
    if "hashes" not in event_json_object:
        event_json_object = hash_event(event_json_object)

    # Strip all the keys that would be removed if the event was redacted.
    # The hashes are not stripped and cover all the keys in the event.
    # This means that we can tell if any of the non-essential keys are
    # modified or removed.
    stripped_json_object = strip_non_essential_keys(event_json_object)

    # Sign the stripped JSON object. The signature only covers the
    # essential keys and the hashes. This means that we can check the
    # signature even if the event is redacted.
    signed_json_object = sign_json(stripped_json_object)

    # Copy the signatures from the stripped event to the original event.
    event_json_object["signatures"] = signed_json_oject["signatures"]
    return event_json_object

Servers can then transmit the entire event or the event with the non-essential keys removed. If the entire event is present, receiving servers can then check the event by computing the SHA-256 of the event, excluding the hash object. If the keys have been redacted, then the hash object is included when calculating the SHA-256 instead.

New hash functions can be introduced by adding additional keys to the hash object. Since the hash object cannot be redacted a server shouldn't allow too many hashes to be listed, otherwise a server might embed illict data within the hash object. For similar reasons a server shouldn't allow hash values that are too long.