1   Server Discovery

server_name = dns_name [ ":" tls_port]
dns_name = <host, see [RFC 3986], Section 3.2.2>
tls_port = *DIGIT

{
    "old_verify_keys": {
        "ed25519:auto1": {
            "expired_ts": 922834800000,
            "key": "Base+64+Encoded+Old+Verify+Key"
        }
    },
    "server_name": "example.org",
    "signatures": {
        "example.org": {
            "ed25519:auto2": "Base+64+Encoded+Signature"
        }
    },
    "tls_fingerprints": [
        {
            "sha256": "Base+64+Encoded+SHA-256-Fingerprint"
        }
    ],
    "valid_until_ts": 1052262000000,
    "verify_keys": {
        "ed25519:auto2": {
            "key": "Base+64+Encoded+Signature+Verification+Key"
        }
    }
}

GET /_matrix/key/v2/query/${server_name}/${key_id}/?minimum_valid_until_ts=${minimum_valid_until_ts} HTTP/1.1

POST /_matrix/key/v2/query HTTP/1.1
Content-Type: application/json

{
    "server_keys": {
        "$server_name": {
            "$key_id": {
                "minimum_valid_until_ts": $posix_timestamp
            }
        }
    }
}

HTTP/1.1 200 OK
Content-Type: application/json
{
    "server_keys": [
       # List of responses with same format as /_matrix/key/v2/server
       # signed by both the originating server and this server.
    ]
}

{
    "server_name": "example.org",
    "signatures": {
        "example.org": {
            "ed25519:auto": "Base+64+Encoded+Signature"
        }
    },
    "tls_certificate": "Base+64+Encoded+DER+Encoded+X509+TLS+Certificate"
    "verify_keys": {
        "ed25519:auto": "Base+64+Encoded+Signature+Verification+Key"
    }
}

2   Transactions

The transfer of EDUs and PDUs between homeservers is performed by an exchange of Transaction messages, which are encoded as JSON objects, passed over an HTTP PUT request. A Transaction is meaningful only to the pair of homeservers that exchanged it; they are not globally-meaningful.

Each transaction has:

• An opaque transaction ID.
• A timestamp (UNIX epoch time in milliseconds) generated by its origin server.
• An origin and destination server name.
• A list of "previous IDs".
• A list of PDUs and EDUs - the actual message payload that the Transaction carries.

{
 "transaction_id":"916d630ea616342b42e98a3be0b74113",
 "ts":1404835423000,
 "origin":"red",
 "prev_ids":["e1da392e61898be4d2009b9fecce5325"],
 "pdus":[...],
 "edus":[...]
}

3   PDUs

All PDUs have:

• An ID to identify the PDU itself
• A room ID that it relates to
• A declaration of their type
• A list of other PDU IDs that have been seen recently in that room (regardless of which origin sent them)

{
 "context":"#example:green.example.com",
 "origin":"green.example.com",
 "pdu_id":"a4ecee13e2accdadf56c1025af232176",
 "origin_server_ts":1404838188000,
 "pdu_type":"m.room.message",
 "prev_pdus":[
   ["blue.example.com","99d16afbc8",
       {"sha256":"abase64encodedsha256hashshouldbe43byteslong"}]
 ],
 "hashes":{"sha256":"thishashcoversallfieldsincasethisisredacted"},
 "signatures":{
   "green.example.com":{
     "ed25519:key_version:":"these86bytesofbase64signaturecoveressentialfieldsincludinghashessocancheckredactedpdus"
   }
 },
 "is_state":false,
 "content": {...}
}

{...,
 "is_state":true,
 "state_key":TODO-doc
 "required_power_level":TODO-doc
 "prev_state_id":TODO-doc
 "prev_state_origin":TODO-doc
}

4   EDUs

EDUs, by comparison to PDUs, do not have an ID, a room ID, or a list of "previous" IDs. The only mandatory fields for these are the type, origin and destination homeserver names, and the actual nested content.

{
 "edu_type":"m.presence",
 "origin":"blue",
 "destination":"orange",
 "content":{...}
}

5   Protocol URLs

All these URLs are name-spaced within a prefix of:

/_matrix/federation/v1/...

For active pushing of messages representing live activity "as it happens":

PUT .../send/<transaction_id>/
  Body: JSON encoding of a single Transaction
  Response: TODO-doc

The transaction_id path argument will override any ID given in the JSON body. The destination name will be set to that of the receiving server itself. Each embedded PDU in the transaction body will be processed.

To fetch all the state of a given room:

GET .../state/<room_id>/
  Response: JSON encoding of a single Transaction containing multiple PDUs

Retrieves a snapshot of the entire current state of the given room. The response will contain a single Transaction, inside which will be a list of PDUs that encode the state.

To fetch a particular event:

GET .../event/<event_id>/
  Response: JSON encoding of a partial Transaction containing the event

Retrieves a single event. The response will contain a partial Transaction, having just the origin, origin_server_ts and pdus fields; the event will be encoded as the only PDU in the pdus list.

To backfill events on a given room:

GET .../backfill/<room_id>/
  Query args: v, limit
  Response: JSON encoding of a single Transaction containing multiple PDUs

Retrieves a sliding-window history of previous PDUs that occurred on the given room. Starting from the PDU ID(s) given in the "v" argument, the PDUs that preceded it are retrieved, up to a total number given by the "limit" argument.

To stream events all the events:

GET .../pull/
  Query args: origin, v
  Response: JSON encoding of a single Transaction consisting of multiple PDUs

Retrieves all of the transactions later than any version given by the "v" arguments.

To make a query:

GET .../query/<query_type>
  Query args: as specified by the individual query types
  Response: JSON encoding of a response object

Performs a single query request on the receiving homeserver. The Query Type part of the path specifies the kind of query being made, and its query arguments have a meaning specific to that kind of query. The response is a JSON-encoded object whose meaning also depends on the kind of query.

To join a room:

GET .../make_join/<room_id>/<user_id>
  Response: JSON encoding of a join proto-event

PUT .../send_join/<room_id>/<event_id>
  Response: JSON encoding of the state of the room at the time of the event

Performs the room join handshake. For more information, see "Joining Rooms" below.

6   Joining Rooms

When a new user wishes to join room that the user's homeserver already knows about, the homeserver can immediately determine if this is allowable by inspecting the state of the room, and if it is acceptable, it can generate, sign, and emit a new m.room.member state event adding the user into that room. When the homeserver does not yet know about the room it cannot do this directly. Instead, it must take a longer multi-stage handshaking process by which it first selects a remote homeserver which is already participating in that room, and uses it to assist in the joining process. This is the remote join handshake.

This handshake involves the homeserver of the new member wishing to join (referred to here as the "joining" server), the directory server hosting the room alias the user is requesting to join with, and a homeserver where existing room members are already present (referred to as the "resident" server).

In summary, the remote join handshake consists of the joining server querying the directory server for information about the room alias; receiving a room ID and a list of join candidates. The joining server then requests information about the room from one of the residents. It uses this information to construct a m.room.member event which it finally sends to a resident server.

Conceptually these are three different roles of homeserver. In practice the directory server is likely to be resident in the room, and so may be selected by the joining server to be the assisting resident. Likewise, it is likely that the joining server picks the same candidate resident for both phases of event construction, though in principle any valid candidate may be used at each time. Thus, any join handshake can potentially involve anywhere from two to four homeservers, though most in practice will use just two.

Client         Joining                Directory       Resident
               Server                 Server          Server

join request -->
               |
               directory request ------->
               <---------- directory response
               |
               make_join request ----------------------->
               <------------------------------- make_join response
               |
               send_join request ----------------------->
               <------------------------------- send_join response
               |
<---------- join response

The first part of the handshake usually involves using the directory server to request the room ID and join candidates. This is covered in more detail on the directory server documentation, below. In the case of a new user joining a room as a result of a received invite, the joining user's homeserver could optimise this step away by picking the origin server of that invite message as the join candidate. However, the joining server should be aware that the origin server of the invite might since have left the room, so should be prepared to fall back on the regular join flow if this optimisation fails.

Once the joining server has the room ID and the join candidates, it then needs to obtain enough information about the room to fill in the required fields of the m.room.member event. It obtains this by selecting a resident from the candidate list, and requesting the make_join endpoint using a GET request, specifying the room ID and the user ID of the new member who is attempting to join.

The resident server replies to this request with a JSON-encoded object having a single key called event; within this is an object whose fields contain some of the information that the joining server will need. Despite its name, this object is not a full event; notably it does not need to be hashed or signed by the resident homeserver. The required fields are:

The content field itself must be an object, containing:

The joining server now has sufficient information to construct the real join event from these protoevent fields. It copies the values of most of them, adding (or replacing) the following fields:

This will be a true event, so the joining server should apply the event-signing algorithm to it, resulting in the addition of the hashes and signatures fields.

To complete the join handshake, the joining server must now submit this new event to an resident homeserver, by using the send_join endpoint. This is invoked using the room ID and the event ID of the new member event.

The resident homeserver then accepts this event into the room's event graph, and responds to the joining server with the full set of state for the newly- joined room. This is returned as a two-element list, whose first element is the integer 200, and whose second element is an object which contains the following keys:

7   Backfilling

Once a homeserver has joined a room, it receives all the events emitted by other homeservers in that room, and is thus aware of the entire history of the room from that moment onwards. Since users in that room are able to request the history by the /messages client API endpoint, it's possible that they might step backwards far enough into history before the homeserver itself was a member of that room.

To cover this case, the federation API provides a server-to-server analog of the /messages client API, allowing one homeserver to fetch history from another. This is the /backfill API.

To request more history, the requesting homeserver picks another homeserver that it thinks may have more (most likely this should be a homeserver for some of the existing users in the room at the earliest point in history it has currently), and makes a /backfill request. The parameters of this request give an event ID that the requesting homeserver wishes to obtain, and a number specifying how many more events of history before that one to return at most.

The response to this request is an object with the following keys:

The list of events given in pdus is returned in reverse chronological order; having the most recent event first (i.e. the event whose event ID is that requested by the requestor in the v parameter).

8   Authentication

 {
     "method": "GET",
     "uri": "/target",
     "origin": "origin.hs.example.com",
     "destintation": "destination.hs.example.com",
     "content": { JSON content ... },
     "signatures": {
         "origin.hs.example.com": {
             "ed25519:key1": "ABCDEF..."
         }
     }
}

GET /target HTTP/1.1
Authorization: X-Matrix origin=origin.example.com,key="ed25519:key1",sig="ABCDEF..."
Content-Type: application/json

{ JSON content ... }

def authorization_headers(origin_name, origin_signing_key,
                          destination_name, request_method, request_target,
                          content_json=None):
    request_json = {
         "method": request_method,
         "uri": request_target,
         "origin": origin_name,
         "destination": destination_name,
    }

    if content_json is not None:
        request["content"] = content_json

    signed_json = sign_json(request_json, origin_name, origin_signing_key)

    authorization_headers = []

    for key, sig in signed_json["signatures"][origin_name].items():
        authorization_headers.append(bytes(
            "X-Matrix origin=%s,key=\"%s\",sig=\"%s\"" % (
                origin_name, key, sig,
            )
        ))

    return ("Authorization", authorization_headers)

9   Server-Server Authorization

10   State Conflict Resolution

11   Presence

The server API for presence is based entirely on exchange of the following EDUs. There are no PDUs or Federation Queries involved.

Performing a presence update and poll subscription request:

EDU type: m.presence

Content keys:
  push: (optional): list of push operations.
    Each should be an object with the following keys:
      user_id: string containing a User ID
      presence: "offline"|"unavailable"|"online"|"free_for_chat"
      status_msg: (optional) string of free-form text
      last_active_ago: milliseconds since the last activity by the user

  poll: (optional): list of strings giving User IDs

  unpoll: (optional): list of strings giving User IDs

The presence of this combined message is two-fold: it informs the recipient server of the current status of one or more users on the sending server (by the push key), and it maintains the list of users on the recipient server that the sending server is interested in receiving updates for, by adding (by the poll key) or removing them (by the unpoll key). The poll and unpoll lists apply changes to the implied list of users; any existing IDs that the server sent as poll operations in a previous message are not removed until explicitly requested by a later unpoll.

On receipt of a message containing a non-empty poll list, the receiving server should immediately send the sending server a presence update EDU of its own, containing in a push list the current state of every user that was in the original EDU's poll list.

Sending a presence invite:

EDU type: m.presence_invite

Content keys:
  observed_user: string giving the User ID of the user whose presence is
    requested (i.e. the recipient of the invite)
  observer_user: string giving the User ID of the user who is requesting to
    observe the presence (i.e. the sender of the invite)

Accepting a presence invite:

EDU type: m.presence_accept

Content keys - as for m.presence_invite

Rejecting a presence invite:

EDU type: m.presence_deny

Content keys - as for m.presence_invite

12   Profiles

The server API for profiles is based entirely on the following Federation Queries. There are no additional EDU or PDU types involved, other than the implicit m.presence and m.room.member events (see section below).

Querying profile information:

Query type: profile

Arguments:
  user_id: the ID of the user whose profile to return
  field: (optional) string giving a field name

Returns: JSON object containing the following keys:
  displayname: string of free-form text
  avatar_url: string containing an HTTP-scheme URL

If the query contains the optional field key, it should give the name of a result field. If such is present, then the result should contain only a field of that name, with no others present. If not, the result should contain as much of the user's profile as the homeserver has available and can make public.

13   Directory

The server API for directory queries is also based on Federation Queries.

Querying directory information:

Query type: directory

Arguments:
  room_alias: the room alias to query

Returns: JSON object containing the following keys:
  room_id: string giving the underlying room ID the alias maps to
  servers: list of strings giving the join candidates

The list of join candidates is a list of server names that are likely to hold the given room; these are servers that the requesting server may wish to use as resident servers as part of the remote join handshake. This list may or may not include the server answering the query.

14   Signing Events

import json

def canonical_json(value):
    return json.dumps(
        value,
        # Encode code-points outside of ASCII as UTF-8 rather than \u escapes
        ensure_ascii=False,
        # Remove unnecessary white space.
        separators=(',',':'),
        # Sort the keys of dictionaries.
        sort_keys=True,
        # Encode the resulting unicode as UTF-8 bytes.
    ).encode("UTF-8")

value     = false / null / true / object / array / number / string
false     = %x66.61.6c.73.65
null      = %x6e.75.6c.6c
true      = %x74.72.75.65
object    = %x7B [ member *( %x2C member ) ] %7D
member    = string %x3A value
array     = %x5B [ value *( %x2C value ) ] %5B
number    = [ %x2D ] int
int       = %x30 / ( %x31-39 *digit )
digit     = %x30-39
string    = %x22 *char %x22
char      = unescaped / %x5C escaped
unescaped = %x20-21 / %x23-5B / %x5D-10FFFF
escaped   = %x22 ; "    quotation mark  U+0022
          / %x5C ; \    reverse solidus U+005C
          / %x62 ; b    backspace       U+0008
          / %x66 ; f    form feed       U+000C
          / %x6E ; n    line feed       U+000A
          / %x72 ; r    carriage return U+000D
          / %x74 ; t    tab             U+0009
          / %x75.30.30.30 (%x30-37 / %x62 / %x65-66) ; u000X
          / %x75.30.30.31 (%x30-39 / %x61-66)        ; u001X

{
   "name": "example.org",
   "signing_keys": {
     "ed25519:1": "XSl0kuyvrXNj6A+7/tkrB9sxSbRi08Of5uRhxOqZtEQ"
   },
   "unsigned": {
      "age_ts": 922834800000
   },
   "signatures": {
      "example.org": {
         "ed25519:1": "s76RUgajp8w172am0zQb/iPTHsRnb4SkrzGoeCOSFfcBY2V/1c8QfrmdXHpvnc2jK5BD1WiJIxiMW95fMjK7Bw"
      }
   }
}

def sign_json(json_object, signing_key, signing_name):
    signatures = json_object.pop("signatures", {})
    unsigned = json_object.pop("unsigned", None)

    signed = signing_key.sign(encode_canonical_json(json_object))
    signature_base64 = encode_base64(signed.signature)

    key_id = "%s:%s" % (signing_key.alg, signing_key.version)
    signatures.setdefault(signing_name, {})[key_id] = signature_base64

    json_object["signatures"] = signatures
    if unsigned is not None:
        json_object["unsigned"] = unsigned

    return json_object

def hash_event(event_json_object):

    # Keys under "unsigned" can be modified by other servers.
    # They are useful for conveying information like the age of an
    # event that will change in transit.
    # Since they can be modifed we need to exclude them from the hash.
    unsigned = event_json_object.pop("unsigned", None)

    # Signatures will depend on the current value of the "hashes" key.
    # We cannot add new hashes without invalidating existing signatures.
    signatures = event_json_object.pop("signatures", None)

    # The "hashes" key might contain multiple algorithms if we decide to
    # migrate away from SHA-2. We don't want to include an existing hash
    # output in our hash so we exclude the "hashes" dict from the hash.
    hashes = event_json_object.pop("hashes", {})

    # Encode the JSON using a canonical encoding so that we get the same
    # bytes on every server for the same JSON object.
    event_json_bytes = encode_canonical_json(event_json_bytes)

    # Add the base64 encoded bytes of the hash to the "hashes" dict.
    hashes["sha256"] = encode_base64(sha256(event_json_bytes).digest())

    # Add the "hashes" dict back the event JSON under a "hashes" key.
    event_json_object["hashes"] = hashes
    if unsigned is not None:
        event_json_object["unsigned"] = unsigned
    return event_json_object

auth_events
depth
event_id
hashes
membership
origin
origin_server_ts
prev_events
prev_state
room_id
sender
signatures
state_key
type

type is "m.room.aliases":
  aliases

type is "m.room.create":
  creator

type is "m.room.history_visibility":
  history_visibility

type is "m.room.join_rules":
  join_rule

type is "m.room.member":
  membership

type is "m.room.power_levels":
  ban
  events
  events_default
  kick
  redact
  state_default
  users
  users_default

def sign_event(event_json_object, name, key):

    # Make sure the event has a "hashes" key.
    if "hashes" not in event_json_object:
        event_json_object = hash_event(event_json_object)

    # Strip all the keys that would be removed if the event was redacted.
    # The hashes are not stripped and cover all the keys in the event.
    # This means that we can tell if any of the non-essential keys are
    # modified or removed.
    stripped_json_object = strip_non_essential_keys(event_json_object)

    # Sign the stripped JSON object. The signature only covers the
    # essential keys and the hashes. This means that we can check the
    # signature even if the event is redacted.
    signed_json_object = sign_json(stripped_json_object)

    # Copy the signatures from the stripped event to the original event.
    event_json_object["signatures"] = signed_json_oject["signatures"]
    return event_json_object