🧩 Deployment Patterns

Audience: DevOps Leads / Backend & Frontend Engineers Purpose: The three ways a project runs on the bare-metal host, and the deploy script that projects without Docker ship so that manual and CI deploys do exactly the same thing.

Pick the pattern, then follow the SOPs

This page decides how a project runs. Provisioning the folder, keys and secrets is in Manual Deployments; the pipeline is in GitLab CI/CD.

1. The Three Patterns on the Host

All three live under /opt/<project>/, owned by app-runner, and are exposed by Caddy (see Server Topology). Docker Compose is the default; the other two are for projects that run directly on the host.

Pattern	Use it for	How it runs	Examples on the host
Static site	SPAs and PWAs (Vite, React)	A built `dist/` that Caddy serves with `file_server`: built on the host by the deploy script (preferred for new projects), or built in CI and copied with `rsync`	Built on the host: `teapoyo.apollyon.lat`. Copied from CI: `chronoteach`
Python service (systemd)	FastAPI backends	A Poetry `.venv` inside the checkout, a systemd unit running uvicorn on `127.0.0.1:<port>`, the host PostgreSQL	`korvyn.ramsus.site` (`ram-korvy-api.service`), `teapoyo-api.apollyon.lat`
Docker Compose (default)	Most services: our own backends and tools, third-party stacks, anything whose runtime the host does not provide	`docker compose up -d --build` from the project folder; data volumes under `/mnt/newhdd/<service>/`	GitLab, Nextcloud, NetBird, Garage, gowa, dawarich, Ollama, Open WebUI, the media stack

Python versions

The host is Debian 13, which ships Python 3.13. A project that needs a newer Python cannot use the systemd pattern with the system interpreter: either target 3.13, install the interpreter for app-runner outside apt, or use Docker Compose (the image brings its own Python).

Static site

Building on the host keeps the server checkout, the deployed commit and the deploy script in one place, and needs no build artifacts passed between jobs. Copying a CI-built dist/ with rsync also works and avoids Node on the host.
Node 22 is installed system-wide (/usr/bin/node).
Vite empties its output folder when it starts, so never build straight into the folder Caddy serves: build into a temporary folder and swap it in at the end (the deploy script below does this with rsync --delay-updates).
dist/ is git-ignored, so it never makes the checkout dirty.

Python service (systemd)

Poetry is installed with pipx and linked at /usr/local/bin/poetry, which is on every PATH, including non-interactive SSH sessions.
Put the virtualenv inside the checkout (POETRY_VIRTUALENVS_IN_PROJECT=true) and point the unit at it (/opt/<project>/.venv/bin/uvicorn).
The host PostgreSQL 17 runs on 127.0.0.1:5432. Give each project its own role and database:

sudo -u postgres psql <<'SQL'
CREATE ROLE <project> LOGIN PASSWORD '<password>';
CREATE DATABASE <project> OWNER <project>;
SQL

Bind uvicorn to 127.0.0.1, never 0.0.0.0: Caddy is the only way in. Production units do not use --reload.
Run migrations in the unit (ExecStartPre=/opt/<project>/.venv/bin/alembic upgrade head), so every restart applies them and a failed migration stops the service from starting with the old schema half-changed.
Let the deploy restart only its own unit with one sudo rule (see Host Administration).

Docker Compose

The standard for most services. Follow Manual Deployments and the volume rules in Server Topology. The CI job runs git pull origin main && docker compose up -d --build in the project folder (GitLab CI/CD); migrations run in the container's entrypoint. A project may wrap that command in a deploy/deploy.sh (section 2) to get the same safeguards as the other patterns, but it is not required.

2. The Deploy Script (`deploy/deploy.sh`)

Projects that run directly on the host (static sites built there, Python services under systemd) ship one script that both an admin and the pipeline run, from the repository folder, as app-runner. Docker Compose projects may adopt it too:

cd /opt/<project> && deploy/deploy.sh            # latest main
cd /opt/<project> && deploy/deploy.sh <sha|tag>  # a specific commit
FORCE=1 deploy/deploy.sh <older sha>             # roll back on purpose

The pipeline calls the same script over SSH with $CI_COMMIT_SHA. A manual deploy and a CI deploy are therefore identical, and the logic is reviewed in the project's merge requests instead of living in a YAML one-liner.

The script must keep these safeguards. TEApoyo's two repositories (apollyon/teapoyoapp/teapoyo and teapoyo-backend) are working references.

Safeguard	Why	How
Path-agnostic	The same script works in any folder and on a developer machine	`REPO="$(git rev-parse --show-toplevel)"`; no `/opt` path written in it
One deploy at a time	A manual and a CI deploy must not interleave	`flock -w 600` on `.deploy/lock` (git-ignored)
Clean checkout only	A hand edit on the server must never be silently deployed or lost	Refuse if `git diff --quiet` fails
Exact commit, detached	`git pull` deploys whatever `main` is now, not the commit that passed CI	`git fetch` then `git checkout --detach <sha>`
Never backwards	Two quick merges can finish out of order and roll the site back	Record the deployed sha in `.deploy/deployed`; skip a commit that is an ancestor of it unless `FORCE=1`. Check this before handing over to the older commit's script
Applies its own fixes	Bash reads a script while running it; checking out a new version mid-run can break it	Wrap the whole script in `main() { …; }; main "$@"; exit`, and after the checkout `exec` the new commit's version (the lock's file descriptor is inherited)
Atomic swap (static)	Visitors must never load half a deploy	Build into `.deploy/dist`, then `rsync -a --delete-after --delay-updates` into `dist/`
Health check	A green job must mean the site works	Static: fetch the page through Caddy with `curl --resolve <host>:443:127.0.0.1` and compare it with the build. Service: poll `127.0.0.1:<port>/health`, then the public URL
Caddyfile parses	A broken site block breaks the shared Caddy for every site at its next reload	`caddy adapt --config deploy/Caddyfile --adapter caddyfile` before deploying

Caddyfiles are imported from the checkout

The main /etc/caddy/Caddyfile imports each project's deploy/Caddyfile directly from its checkout (for example import /opt/teapoyo/teapoyo/deploy/Caddyfile). That means a merged change to that file goes live at the next Caddy reload for any reason, even one done for another site. Review Caddyfile changes in the merge request as carefully as code, keep the syntax check above in the deploy script, and run sudo caddy validate --config /etc/caddy/Caddyfile before every reload.

3. Database Backups (Not in Place Yet)

There is no backup procedure for the host PostgreSQL today. A project must not open sign-ups or store real people's data until it has one:

A scheduled pg_dump per database (for example a systemd timer or a file in /etc/cron.d/), written to /mnt/newhdd/backups/postgres/<database>/, not to the root SSD that holds the database itself.
A copy off the server (the host is a single machine).
A retention rule (for example 7 daily and 4 weekly dumps).
A tested restore: restore a dump into a scratch database at least once, and after every PostgreSQL upgrade.

Document the chosen setup here once it exists.

3. Before Public Launch: Test and Production

A project that deploys every merge to main and also deploys release tags must not publish both to the same folder: the next merge replaces the reviewed release with whatever main holds. Before a public launch, give main its own test host (its own TARGET folder and Caddy site), or stop deploying main to the production site.