A Local Development Setup for BookWyrm Federation

BookWyrm speaks ActivityPub, but testing its federation capabilities locally can be tricky. Besides running two separate instances, there are a few caveats that complicate things further, namely that both instances need to run on their separate domain and they need to have an SSL certificate signed by a common certificate authority.

Satisfying these caveats without resorting to a service which takes care of tunnelling local ports to a static domain plus SSL support is fairly tricky. I spent way too much time trying to get something purely local working, mainly with mkcert but ultimately failing at making both requests and aiohttp requests work with the custom CA that mkcert creates without patching BookWyrm’s source code.

I ended up using Cloudflare Tunnels given the service is free to use even with static domains. Others, like ngrok, require a paid subscription to get static domains. BookWyrm itself needs to be aware of the domain so you would most likely have to start with empty instances every time you restart the tunnel service and get a new domain assigned, which is quickly getting frustrating. There are other tunnelling services available as well, this is a fairly comprehensive list.

The code for this setup is available on Github at chdorner/bookwyrm-dev.

Structure

The setup is structured into two BookWyrm instances, dev-lit is for literary fiction aficionados and dev-sci for science fiction buffs.

Each of those two local instance runs a few services:

  • gunicorn web server on port 5040 for dev-lit, 5050 for dev-sci
  • celery worker
  • celery beat scheduler
  • celery flower interface on port 5041 for dev-lit, 5051 for dev-sci
  • cloudflared tunnel

Furthermore, it assumes that you are running PostgreSQL and Redis separately. Feel free to either run it natively or in containers, whatever floats your boat.

Set up Tunneling

To use Cloudflare Tunnels you need to have your domain managed by Cloudflare, each tunnel can then be served from a subdomain. To set this up I was mostly just following Cloudflare’s instructions:

  1. Install cloudflared
  2. Authenticate cloudflared with cloudflared tunnel login
  3. Create tunnels and keep a note of the ID for each created tunnel
    • cloudflared tunnel create dev-lit
    • cloudflared tunnel create dev-sci
  4. Route the tunnel via a subdomain
    • cloudflared tunnel route dns dev-lit {dev-lit.your-domain.com}
    • cloudflared tunnel route dns dev-sci {dev-sci.your-domain.com}

Set up the Local Environment

For each instance (dev-lit and dev-sci):

  1. Copy .env.local.example and configure the variables, this file can be used to override any value from .env
  2. Copy and configure cloudflared-tunnel.yaml.example, replace tunnel ID and path to credentials file from the previous section’s output.
  3. Create the two databases (defaults to bookwyrm_lit and bookwyrm_sci)
  4. Use each instance’s manage script to run management commands, for setting up a completely new instance:
    1. ./dev-lit/manage migrate, plus the same command for dev-sci
    2. ./dev-lit/manage initb, plus the same command for dev-sci
    3. ./dev-lit/manage compile_themes, plus the same command for dev-sci
    4. ./dev-lit/manage collectstatic, plus the same command for dev-sci
    5. ./dev-lit/manage admin_code, plus the same command for dev-sci

Running

This setup comes with a Procfile, feel free to use your favourite process manager like foreman or hivemind.

# with foreman
foreman start

# with hivemind
hivemind

Each instance should now be available under the domain you configured when setting up the Cloudflare tunnel. After confirming the instance settings and creating the admin user, you should be able to find the user of the other instance by searching for {username}@{domain of other instance}. Then make sure those two users follow each other and from then on every action is federated to the other instance.

Reply by email