jb-infra/README.md

jb-infra

VPS edge stack for joshbairstow.com: a caddy-docker-proxy edge reverse proxy (TLS + label-based routing) and a self-hosted Forgejo git server with CI and a container registry. Everything else — jb-website and future side projects — is deployed by CI and just attaches to the edge network and declares routing labels.

[home desktop]                                  [Vultr VPS]
 Forgejo runner (host exec)                      docker network: edge (external)
  └ docker build  ───push image──►   ┌────────── edge-caddy   :80/:443  (TLS, routing)
  └ ssh deploy    ───pull+up─────►   ├────────── forgejo      git.joshbairstow.com (+registry)
                                     ├────────── jb-website   internal :80
                                     └────────── <future apps> e.g. shop.joshbairstow.com

Why this shape

• Edge proxy decouples projects. Each project ships its own container with caddy labels; the edge picks them up automatically. Adding a project needs zero edits here — no central config to touch.
• Build off-box. The RAM-hungry Docker build runs on the always-on desktop runner; the VPS only pulls and runs prebuilt images, so it stays small.
• Self-hosted git/CI (Forgejo) — independent of GitHub/GitLab, with a cloud push-mirror kept only as backup.

---

The VPS (provisioned 2026-06-02)

Vultr shared-CPU 2 GB, Debian 13 (trixie), kernel 6.12, ~5.4 GB swap already present (no need to add any). Desktop SSH key installed; Vultr firewall groups applied on both IPv4 and IPv6.

IPv4	45.32.242.27
IPv6	2001:19f0:5801:1313:5400:6ff:fe35:c0a7
SSH	root@45.32.242.27 (port 22)

Pre-flight check

Before and after each bootstrap step, run the read-only checker from your desktop to confirm reachability, ports, the box's facts, and DNS:

./scripts/check-vps.sh --ip6 2001:19f0:5801:1313:5400:6ff:fe35:c0a7

It exits 0 when SSH + port 22 are good; closed app ports and unset DNS show as warnings (expected until DNS is updated and the stack is up).

Bootstrap (one-time, in order)

Order matters — TLS issuance needs DNS + open ports first, and the edge network must exist before any stack references it.

1. DNS — managed at Cloudflare, not VentraIP

VentraIP is only the registrar; the domain's nameservers are delegated to Cloudflare (grace/keenan.ns.cloudflare.com), so create the records in the Cloudflare dashboard. Point them at the VPS and wait for propagation:

• A joshbairstow.com → 45.32.242.27
• A *.joshbairstow.com → 45.32.242.27 (wildcard: covers blog/art/code/git + future)
• AAAA joshbairstow.com → 2001:19f0:5801:1313:5400:6ff:fe35:c0a7
• AAAA *.joshbairstow.com → 2001:19f0:5801:1313:5400:6ff:fe35:c0a7

Set these records to DNS-only (grey cloud), not proxied (orange cloud). The current records point at Cloudflare's proxy. With the proxy ON: Cloudflare terminates TLS with its own edge cert (so Caddy's automatic HTTPS is bypassed), and it only proxies HTTP(S) ports — git-over-SSH on 222 and the registry would break. DNS-only makes clients hit the VPS directly, so Caddy issues real Let's Encrypt certs via HTTP-01 (no DNS-challenge plugin needed) and all ports work. If you later want Cloudflare's proxy/CDN, switch Caddy to the DNS-01 challenge with the Cloudflare plugin + an API token — a separate change.

2. VPS base

Firewall is already handled by the Vultr firewall group (80, 443, 222, 22), so host ufw is optional. Swap already exists. The remaining task is Docker:

# Install Docker Engine + compose plugin
curl -fsSL https://get.docker.com | sh
docker --version && docker compose version   # confirm

(Re-run ./scripts/check-vps.sh — DOCKER should now report a version.)

3. Get the files onto the VPS + bring up the stack

Forgejo doesn't exist yet, so this repo has no server to clone from during bootstrap. The VPS can't pull from your (NAT'd) desktop either — but your desktop can push to the VPS, and docker compose only needs the files, not git. So copy them up with rsync (run from your desktop, in ~/git/jb-infra):

rsync -av --exclude '.git' --exclude 'runner' --exclude 'scripts' \
  ./ root@45.32.242.27:/opt/jb-infra/

(runner/ and scripts/ are desktop-side tools — they don't belong on the VPS.) Then on the VPS:

cd /opt/jb-infra
docker network create edge          # MUST exist before `up`
docker compose up -d
docker compose logs -f caddy        # watch the first cert issuance

Once Forgejo is live (step 4+), create a jb-infra repo inside it and push this there for its permanent, versioned home; future infra edits then deploy like any other project. If you don't want git history on the box, a one-shot git bundle is an alternative to rsync (see project notes).

4. Configure Forgejo

Browse to https://git.joshbairstow.com (cert should be live). The install page is mostly prepopulated from the env vars in compose.yml — confirm these, then complete it:

• Run As Username stays git (the container's OS user, not your login).
• Application URL must be https://git.joshbairstow.com/ (not http://localhost:3000/) — wrong value breaks docker login/push + clone URLs.
• SSH Server Domain git.joshbairstow.com, SSH Server Port 222, Database SQLite3.
• Under server settings: enable Disable Self-Registration (personal instance) and Hide Email Addresses by Default; leave "Require Sign-In to View Pages" off (it would block anonymous clones/pulls).
• Expand Administrator Account Settings and create your admin user with a lowercase username (e.g. josh) — this becomes your registry path git.joshbairstow.com/<owner>/…, which must be lowercase.

Then:

• Confirm Actions is enabled (Site Administration → Actions).
• Create a Personal Access Token with write:package scope (Settings → Applications). The automatic Actions token CANNOT push packages.

5. Register the desktop runner (always-on)

On the home desktop (already has Docker + is in the docker group). Put the binary in place, register in a home-dir working folder (snap Docker needs the build context under $HOME), then install the systemd unit so it survives reboots and auto-claims jobs: Run these from the cloned jb-infra repo:

# 1. Binary — skip if already present (check: command -v forgejo-runner).
#    The unit's ExecStart points at /usr/local/sbin/forgejo-runner.
sudo install -m 0755 forgejo-runner /usr/local/sbin/forgejo-runner

# 2. Working dir + config (under $HOME so snap Docker can read build contexts).
mkdir -p ~/forgejo-runner && cp runner/config.yml ~/forgejo-runner/

# 3. Register IN the working dir so .runner lands there (token: Forgejo →
#    Site Admin → Actions → Runners → Create). The subshell keeps cwd in the repo.
( cd ~/forgejo-runner && forgejo-runner register --no-interactive \
    --instance https://git.joshbairstow.com \
    --token <REGISTRATION_TOKEN> \
    --name desktop --labels desktop:host )     # ':host' => run on host, use host Docker

# 4. Install + enable the always-on service.
sudo cp runner/forgejo-runner.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now forgejo-runner
systemctl status forgejo-runner                # expect active (running); runner shows green in Forgejo

Already registered in the wrong directory? Don't re-register — just mv .runner ~/forgejo-runner/ (it's location-independent credentials). The :host label is what lets the docker build step work (the job runs on the host using its Docker, not in a socket-less container). See runner/config.yml and runner/forgejo-runner.service.

6. Wire the jb-website repo

In the jb-website repo settings on Forgejo (Settings → Actions):

Variable:

• REGISTRY_OWNER — your lowercase Forgejo username/org; the image namespace (git.joshbairstow.com/<owner>/jb-website).

Secrets:

• REGISTRY_USER — your Forgejo login username (the account that owns the PAT); the username for docker login. Usually equals REGISTRY_OWNER.
• REGISTRY_TOKEN — the PAT with write:package scope; the docker login password.
• DEPLOY_HOST — 45.32.242.27.
• DEPLOY_USER — the VPS account the runner SSHes in as: root to start, or a dedicated deploy user (in the docker group) for least privilege.
• DEPLOY_SSH_KEY — the private key of a dedicated deploy keypair (full file incl. BEGIN/END lines; paste multi-line — OpenSSH needs the newlines). NOT your personal key. Generate + install its public half on the VPS:

  ssh-keygen -t ed25519 -f ~/.ssh/jb-deploy -C forgejo-ci-deploy -N ""
  ssh-copy-id -i ~/.ssh/jb-deploy.pub <DEPLOY_USER>@45.32.242.27
  cat ~/.ssh/jb-deploy        # <- paste this into DEPLOY_SSH_KEY
  

On the VPS, put the production compose file in place — rsync it from your desktop (creates the dir + copies; only the prod compose.yml, not the dev one):

rsync -av --rsync-path="mkdir -p /opt/jb-website && rsync" \
  ~/git/jb-website/compose.yml root@45.32.242.27:/opt/jb-website/

For docker compose pull to read the image, the package must be pullable. Note: package visibility ≠ repository visibility — packages are owned by the user/org, so making the repo public does NOT necessarily make the image anonymously pullable, and the package only exists after the first push. After the first pipeline run, verify from the VPS:

docker pull git.joshbairstow.com/<owner>/jb-website:latest
# unauthorized? -> set the PACKAGE public (profile → Packages → jb-website →
# settings), or run `docker login git.joshbairstow.com` once as DEPLOY_USER.

If DEPLOY_USER is not root, it must be in the docker group; for least privilege, restrict its authorized_keys entry with a forced command=.

7. First deploy

# in the jb-website repo (default branch is already `main`).
# Custom SSH port 222 needs the ssh:// URL form (scp-style host:port doesn't work):
git remote add origin ssh://git@git.joshbairstow.com:222/<owner>/jb-website.git
git push -u origin main

The push triggers .forgejo/workflows/deploy.yml: the desktop builds + pushes the image, then SSHes to the VPS to pull && up -d. The edge issues certs for the apex + subdomains on first request.

---

Adding another project later

No changes to this repo. In the new project:

1. Ship a compose.yml on the VPS that joins the external edge network and carries labels, e.g.:

   services:
     app:
       image: git.joshbairstow.com/<owner>/<app>:latest
       networks: [edge]
       labels:
         caddy: shop.joshbairstow.com
         caddy.reverse_proxy: "{{upstreams 3000}}"
   networks:
     edge:
       external: true
   

2. Give it its own .forgejo/workflows/deploy.yml (copy jb-website's).
3. *.joshbairstow.com already resolves, so the edge issues its cert automatically on first hit.

---

Maintenance / hardening

• Backups: the forgejo_data volume is now the source of truth (repos, DB, registry blobs). Run docker compose exec forgejo forgejo dump on a schedule, and set a push-mirror on each repo to a cloud host (GitLab/GitHub) for off-box git backup.
• Registry GC: per-commit :sha tags accumulate. Enable Forgejo's package cleanup rules / run periodic GC so the VPS disk doesn't fill.
• Certs: never delete the caddy_data volume — losing it re-issues every cert and risks Let's Encrypt rate limits.
• Least privilege: restrict the deploy key in authorized_keys with a forced command= that only runs the compose pull/up script.
• Reset the admin password (no email/old password needed): as an admin via Site Administration → User Accounts → (user) → Password, or from the VPS:

  docker compose exec -u git forgejo \
    forgejo admin user change-password --username <user> \
    --password 'NEW' --must-change-password=false