Skip to content

fAIr dev environment

Temporary setup. This Compose-on-EC2 deployment will be replaced by a k3s deployment once all the Helm charts are stable.

The dev environment is a single EC2 instance running the whole fAIr stack with Docker Compose, fronted by Caddy (automatic TLS). It tracks the develop branch: CI builds the images on every push, and a redeploy pulls them.

Layout on the box

Everything lives in /opt/fAIr-app (a develop checkout):

File Purpose
docker-compose.yml base stack (api, worker, postgres, minio, stac, mlflow, zenml, frontend)
docker-compose.dev.yml dev override: Caddy ingress, restart policies, the inline Caddyfile
.env all runtime config and secrets (not in git)

.env sets COMPOSE_FILE=docker-compose.yml:docker-compose.dev.yml, so plain docker compose commands pick up both files. The stack is managed by the fAIr-app systemd unit and starts on boot.

Deploy / redeploy

fAIr-autodeploy.timer redeploys develop every 5 minutes; see infra/systemd/Readme.md. To redeploy by hand, pull the latest images and restart. Migrations run automatically on start. The ZenML stack is imported from the compose file only when it does not exist yet, so a later change to the orchestrator's run_args needs zenml orchestrator update docker_compose on the running stack, followed by a worker restart.

ssh fair-dev
cd /opt/fAIr-app
git pull
docker compose pull
sudo systemctl restart fAIr-app

Start / stop / status

Lifecycle is managed by systemd.

sudo systemctl start fAIr-app
sudo systemctl stop fAIr-app
sudo systemctl status fAIr-app

Logs

Check container status and follow a service. Services: api, worker, caddy, stac, zenml, mlflow, postgres, minio.

cd /opt/fAIr-app
sudo docker compose ps
sudo docker compose logs -f api

Editing config (.env)

.env is the single source of truth, grouped into labeled blocks (Core, Database, Auth, CORS, ZenML & STAC, Object storage, Frontend, Ports, Caddy).

Apply changes by restarting (or recreate a single service with docker compose up -d <service>).

sudo systemctl restart fAIr-app

Common changes:

  • Django / API: DEBUG, SECRET_KEY, DATABASE_URL, CORS_ALLOWED_ORIGINS.
  • Auth (hanko): AUTH_PROVIDER, HANKO_API_URL, LOGIN_URL, COOKIE_*, OSM_REDIRECT_URI.
  • Frontend (VITE_*, baked into config.js at container start): after any change restart api too, it serves the SPA and caches config.js at boot.
  • Domain: PUBLIC_DOMAIN, FRONTEND_URL, API_BASE_URL, ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS, plus the domains in the Caddyfile.

Where to change what

  • Service/image/port wiring: docker-compose.yml (base) and docker-compose.dev.yml (dev-only overrides).
  • Ingress, TLS, domains: the configs.caddyfile block in docker-compose.dev.yml. mlflow/zenml/minio subdomains are commented out (kept internal); uncomment to expose them once DNS + auth are in place.
  • Runtime config / secrets: .env.
  • Lifecycle: infra/systemd/fAIr-app.service.

Access the database

Postgres is bound to 127.0.0.1:5434 on the box. Open an SSH tunnel from your machine.

ssh -L 5434:localhost:5434 fair-dev

While the tunnel is open, connect locally with a client. Use the credentials from .env (POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB).

psql "postgresql://<POSTGRES_USER>:<POSTGRES_PASSWORD>@localhost:5434/<POSTGRES_DB>"

Internal services (mlflow, zenml, minio)

Not exposed publicly. Tunnel to them: mlflow on localhost:5000, zenml on localhost:8080, minio console on localhost:9001.

ssh -L 5000:localhost:5000 -L 8080:localhost:8080 -L 9001:localhost:9001 fair-dev

Notes

  • Training runs on the instance GPU (Docker default runtime is nvidia; ZenML spawns the training container on the same Docker network).
  • Dev data lives on the instance disk in Docker volumes; the shared EFS is not used by dev.