Running a personal Matrix server using docker

2019-12-29
docker>instructions>linux
docker/matrix/selfhosting/server/setup

Matrix

Matrix is a open source communication platform whose main selling point is to be as simple as all modern alternatives, while also being secure and decentralised. While relatively new, and far from being mature, it’s relatively useable and has worked quite well me so far (I’ve been using it for half a year).

In this post I’ll describe how to set up a simple test instance with most of the things you might need using docker. This test instance might not be entirely suitable for a deployment environment, but it will work just as well for everything else. The entire point of this is that since this is a test instance, it won’t be connected to the matrix network, and will work as a separate stand-alone matrix instance. All of the configuration will also be available on my GitHub.

This post was written assuming the reader has some technical background and understands how linux/docker etc works. Basically it goes over the pieces needed to run a Matrix server, and gives configuration examples. It’s more of a starting point for people looking who’re into setting up their own Matrix instances than a detailed in-depth guide. If you’re looking for one of those, then as far as I know, one does not exist… but you might want to try searching for one anyway!

Background

Previously I used Zulip. At first it seems pretty decent and it has threading, which is amazing for productivity oriented communication platform. Unfortunately the server software is really hard to set up (and when I set it up a few years ago, the docker image was considered unofficial and maintained by a 3rd party. It’s only recently that the company behind Zulip took it and started maintaining it themselves) without using docker. The apps are also not that good. I created a fair bit of issues in their issue tracker, and while some did get fixed, some things like the inability to see older messages on the mobile app without an internet connection just slowly made me increasingly frustrated. At some point the decision was made to look for an alternative, and Matrix seemed to fit all our requirements. It has a decent mobile app and an okay desktop app (standard Electron app). It’s decentralised and security focused design might drop convinience for security in some places, but in everyday use I haven’t found a situation where it negatively impacts my experience too much.

Setting up Matrix

To set up Matrix we need 2… no wait, 3 pieces of software.

The first one is the Matrix server (called a homeserver). Even though Matrix is an open standard, and theoretically there can be multiple implementations of it, in practice (at least when this post was written) there’s only one practical option - Synapse. It works well enough, and that’s what we will be using.

The second one is is an identity server. This server handles logins, password management and more. In theory these are optional, and indeed Synapse now implements login management itself. Unfortunately, as of when this post was written, it just does not work as well as it should. Riot (the last piece of software) simply doesn’t work as you would expect without an identity server (no password resets and other niceties). Plus, if you want AD support, as far as I can tell, you absolutely need an identity server. Unlike with homeservers, there are a few practical implementations - the reference implementation Sydent, and mxisd. Unfortunately the later one is discontinued, and while it works, you will want to use the fork which also implements the latest spec.

The last piece of software is the front-end. Just like with the homeserver, there are a few options, but in practice you’ll be using Riot.

In addition to the software mentioned before, this guide will also be using nginx as a proxy before all of the services. I mostly do this to simplify SSL deployment to my subdomains. Synapse does have ACME (LetsEncrypt) support, so that’s worth to keep in mind if you don’t want to do this.

Requirements

You’ll need a

Domain setup

If you want your instance to be available publically, you’ll need 3 (sub)domains. One for each service. Although in theory you should not need to expose the identity server’s API to public, I found that if it’s not pingable Riot will consider that there isn’t one. It’s probably easy to work around that, but I haven’t had the time to set it up, and will assume that it’s exposed to public. In practice this shouldn’t be an issue.

There’s nothing special about these domains, a standard AA or AAAA entries will do, and they must be accessable from whatever environment you’re running Matrix in.

Docker-compose

So, with all the relevant software identified, we can write a simple docker-compose. The docker-compose itself is actually quite simple, and is available here. The only changes you need to make are in the mxisd section - change the domain you’ll be using for your ident server. Sadly it’s not a simple - ‘docker-compose up’, and you’ll need to set it up before using it.

As you can see from the docker-compose, each service will have a volume mounted to it (mxisd-data, riot-data and synapse-data). You might want to create these directories now. If you’re using docker-namespace, these are the UIDs/GIDs used by the container -

Configuration

  1. database

In all of the examples I’ll be using sqlite3 which is a file database in each of the containers, exposed to the host using a volume. Ideally you want to add a configuration in the docker-compose for a database image like postgres. This post will not cover this.

  1. synapse:

First you’ll need to generate a config file. The command that lets you do is docker run -it --rm --mount type=volume,src=synapse-data,dst=/data -e SYNAPSE_SERVER_NAME=matrix.domain -e SYNAPSE_REPORT_STATS=yes matrixdotorg/synapse:latest generate. You can then find the configuration homeserver.yaml file in /var/lib/docker/volumes/synapse-data/_data (if using namespacing, it will be /var/lib/docker/<uid>.<gid>/volumes/synapse-data/_data). For more info you can find the documentation on the docker hub. Take this config file, the log config file and the generated signing key, copy it to the directory synapse_data folder you created in the last step (in the same directory as docker-compose file). My recommendation is to go through this file and understand each setting and configure it according to your needs.

The important settings you’re looking for are:

The rest of the defaults should be fine, but you might want to take a look at them anyway.

Alternatively an example config file is available here. An example logging config file can be found here. You can ctrl+f matrix.domain and CHANGEME to find places you need to change. This config file does work, but I recommend going through the generated config file so that you understand what each option does. It’s also possible that new config options will be added in the future, making this config file outdated. If you’re using this, you might want to change the database used (sqlite is fine for the instance I have set up, but for a real server you probably want to use a different database engine). In addition, you’ll need to clone THIS FOLDER into your synapse_data. For some reason synapse absolutely refuses to start if these files aren’t configured and present. Also, you’ll want to generate a ed25519 signing key.

  1. mxisd (ma1sd):

Mxisd will generate a config file on first run just fine, so what you want to do is to run docker run --rm -e MATRIX_DOMAIN=matrix.domain -v <path-to-mxisd-data>:/etc/mxisd -v <path-to-mxisd-data>:/var/mxisd -p 8090:8090 -t kamax/mxisd (replace the <path-to-mxisd-data> to the location of your mxisd-data folder). You’ll then want to go to the mxisd_data/etc directory and edit mxisd.yaml. The configuration there is straight forward (database, smtp config etc), and you might want to use this example config when configuring it. Just like with synapse, you might want to use a different database backend. Note that the identity server and homeserver use different databases, so don’t set it to use the same one as synapse.

  1. Riot:

Place this config file into your riot-data folder and call it config.json. Open it, and change all references of matrix.domain and matrix-ident.domain to your matrix and your identity server domains. That’s all.

  1. nginx:

The matrix vhost also needs to handle a few of the identity server, but other than that your nginx is going to be a simple app proxy. Example configs (take a look and adapt to your own setup): matrix homeserver; matrix ident server; riot.

Note: my setup also has mobile notifications enabled. This is a useful convinience feature, but because of how mobile apps work, this also means that some data is passed through Riot’s servers in order to push the notification.

Starting / Stopping

Since this is a standard docker-compose setup, you can just use docker-compose to orchestrate deployment (docker-compose up, docker-compose down etc). More info here. Before you daemonize (docker-compose up -d), you want to run it using normal docker-compose up to see if any errors pop up and if you can access everything.

To register a new account you’ll temporarily need to enable global registration in your mxisd (ma1sd) settings. Sadly, as far as I know there’s no way to do it CLI or to preseed any of these services, so you just create an account manually (don’t forget to close down global registration after you make your account). If you want to make your account an admin (allows you to delete rooms, groups etc. without having the required permissions in the group), you’ll need to manually elevate your account. To do this, shut down synapse, access the database (using the client for whichever database you set up in the previous step) and set the admin column value of your account to 1. More information is available here.

After you’re done setting up your account you can invite other people and use it as you wish. If you want bots or to have calling functionality, read on.

Known issues

Because my setup is a bit special (it tries to be as stand-alone as possible), some features are a bit weird at times. For example, even though inviting works, Riot might display an error when sending a notification. Similarly chat room and user discovery is a bit weird at times.

Miscellaneous

Bots

Matrix doesn’t really have ‘bot accounts’. Since the homeserver is basically an API, you can just create a new account and use it as a bot by calling the API functions directly.

First you’ll need to create a new user. Just make an invite in Riot and create the account. Then log into the account using Riot (I recommend doing it using Private/Icognito mode). At this point you’ll want to create or join the channel you want your bot to be a part of (also set up all permissions as required). Then open the setting by clicking on the account’s username on top-left side of the screen and open Settings. There you can personalise the bot (set a name, picture, etc you might need). When done, select the “Help & About” menu in the Settings dialogue. At the bottom there’s an “Advanced” section with a line that says “Access Token”. Click on it to get the access token for the bot. Write it down somewhere and close Riot. DO NOT LOG OUT OF THE ACCOUNT. Doing so will invalidate the token.

For the client app, I’ll be using Nagios as an example. First, in nagios, you want to set up a new contact for Matrix. For that you can use the example configuration file available here. Don’t forget to add the contact to the relevant contact groups. You’ll also need the script that actually makes the request. The one I use is available here. Just open it and fill the required variables at the top, and you’ll be good to go. If you have a different goal, you can use the curl request at the bottom as an example. For more information you can also consult the documentation here.

If everything is set up correctly, your app should now be posting messages to a matrix chat room.

Calls

To enable calls, you’ll need to either set up a TURN server, or use the public matrix server. This is not part of my use case, so my docker setup does not have this set up. For instructions on how to set it up, you can use the official documentation which seems to be fairly well written.