Kordant/tasks/kordant-unified-restructure/42-deployment-config.md

# 42. Deployment — Update Docker, CI/CD, and Environment Configuration

meta:
  id: kordant-unified-restructure-42
  feature: kordant-unified-restructure
  priority: P0
  depends_on: [kordant-unified-restructure-41]
  tags: [deployment, docker, ci-cd, infrastructure]

objective:
- Create production-ready deployment configuration for the unified monolith. Update Docker containers, CI/CD pipelines, and environment configuration to support the new single-app architecture plus native mobile builds.

deliverables:
- `web/Dockerfile` — Production container for the web app:
  - Multi-stage build: Node.js builder → runtime
  - Installs pnpm, dependencies, builds app
  - Exposes port 3000
  - Health check endpoint
  - Non-root user for security
- `docker-compose.yml` — Local development orchestration:
  - `web` service: builds from `web/Dockerfile`, ports `3000:3000`
  - `postgres` service: PostgreSQL 16 with volume for data persistence
  - `redis` service: Redis 7 for job queues and caching
  - `nginx` service: reverse proxy with SSL termination (optional)
  - Environment variables from `.env` file
- `docker-compose.prod.yml` — Production orchestration:
  - Similar to dev but with production-optimized settings
  - Volume mounts for uploads/logs
  - Restart policies
  - Resource limits
- `.github/workflows/ci.yml` — GitHub Actions CI pipeline:
  - Lint and type check (TypeScript)
  - Unit tests for web app
  - Build verification
  - Dependency audit (`pnpm audit`)
- `.github/workflows/deploy.yml` — GitHub Actions CD pipeline:
  - Build Docker image on release tag
  - Push to container registry (GitHub Packages, Docker Hub, or ECR)
  - Deploy to staging on push to `main`
  - Deploy to production on release tag
  - Run database migrations before deployment
  - Health check after deployment
- `web/.env.example` — Complete environment variable documentation:
  - Database: `DATABASE_URL`
  - Redis: `REDIS_URL`
  - Auth: `JWT_SECRET`, `NEXTAUTH_SECRET`
  - Stripe: `STRIPE_SECRET_KEY`, `STRIPE_WEBHOOK_SECRET`, `STRIPE_PRICE_*`
  - Notifications: `RESEND_API_KEY`, `FIREBASE_SERVICE_ACCOUNT_PATH`, `TWILIO_*`
  - External APIs: `HIBP_API_KEY`, `SECURITYTRAILS_API_KEY`, etc.
  - Monitoring: `SENTRY_DSN`, `DATADOG_API_KEY` (optional)
  - App: `NODE_ENV`, `PORT`, `API_URL`, `APP_URL`
- `scripts/deploy.sh` — Deployment helper script:
  - Database backup before migration
  - Migration runner
  - Health check verification
  - Rollback on failure
- `scripts/backup.sh` — Database backup script:
  - `pg_dump` to timestamped file
  - Upload to S3 or similar storage

steps:
1. Create `web/Dockerfile`:
   ```dockerfile
   # Build stage
   FROM node:20-alpine AS builder
   WORKDIR /app
   RUN npm install -g pnpm
   COPY web/package.json web/pnpm-lock.yaml ./
   RUN pnpm install --frozen-lockfile
   COPY web/ .
   RUN pnpm build

   # Runtime stage
   FROM node:20-alpine
   WORKDIR /app
   RUN npm install -g pnpm
   COPY --from=builder /app/package.json /app/pnpm-lock.yaml ./
   COPY --from=builder /app/node_modules ./node_modules
   COPY --from=builder /app/.output ./.output
   COPY --from=builder /app/drizzle ./drizzle
   ENV NODE_ENV=production
   EXPOSE 3000
   HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
     CMD curl -f http://localhost:3000/health || exit 1
   USER node
   CMD ["node", ".output/server/index.mjs"]
   ```
2. Create `docker-compose.yml`:
   - Define services with appropriate environment variables
   - PostgreSQL with `volumes: postgres_data:/var/lib/postgresql/data`
   - Redis with `volumes: redis_data:/data`
   - Network configuration
3. Create `docker-compose.prod.yml`:
   - Add restart policies: `unless-stopped`
   - Add resource limits: `mem_limit`, `cpus`
   - Add logging driver configuration
   - Remove port bindings for internal services (postgres, redis)
4. Create `.github/workflows/ci.yml`:
   - Trigger: push to any branch, pull requests
   - Jobs:
     - `lint`: `pnpm lint`
     - `typecheck`: `tsc --noEmit`
     - `test`: `pnpm test`
     - `build`: `pnpm build`
     - `audit`: `pnpm audit --audit-level=high`
5. Create `.github/workflows/deploy.yml`:
   - Trigger: release tags (`v*`), manual dispatch
   - Jobs:
     - `build`: build Docker image, tag with version
     - `push`: push to registry
     - `migrate`: SSH into server, run `pnpm db:migrate`
     - `deploy`: update Docker Compose stack
     - `healthcheck`: verify `/health` endpoint responds 200
   - Use GitHub secrets for SSH keys, registry credentials
6. Update `web/.env.example`:
   - Document every environment variable used by the app
   - Include description, example value, and whether required
   - Group by category (Database, Auth, Payments, APIs, etc.)
7. Create `scripts/deploy.sh`:
   - `#!/bin/bash` with error handling (`set -euo pipefail`)
   - Backup database: `docker exec postgres pg_dump ...`
   - Run migrations: `docker compose exec web pnpm db:migrate`
   - Deploy: `docker compose -f docker-compose.prod.yml up -d`
   - Health check: `curl -f http://localhost:3000/health`
   - Rollback on failure: `docker compose rollback` or restore backup
8. Create `scripts/backup.sh`:
   - Generate timestamped dump
   - Compress with gzip
   - Upload to S3 using AWS CLI or rclone
   - Retain last 30 backups
9. Test deployment locally:
   - `docker compose up --build`
   - Verify app accessible at `http://localhost:3000`
   - Verify database migrations run
   - Verify health check passes
10. Document deployment process in `docs/DEPLOYMENT.md`.

steps:
- Integration: `docker compose up --build` starts all services successfully
- Integration: App is accessible and functional inside Docker
- Integration: CI pipeline passes on GitHub Actions
- Integration: Deployment script completes without errors
- Security: Dockerfile uses non-root user
- Security: No secrets committed to repository

acceptance_criteria:
- [ ] `web/Dockerfile` builds a production-ready container
- [ ] `docker-compose.yml` orchestrates web, postgres, and redis for local dev
- [ ] `docker-compose.prod.yml` is optimized for production with restart policies and resource limits
- [ ] CI pipeline runs lint, type check, tests, build, and audit on every PR
- [ ] CD pipeline builds and deploys on release tags
- [ ] Database migrations run automatically before deployment
- [ ] Health check endpoint verifies app is ready
- [ ] Rollback script restores previous version on deployment failure
- [ ] Backup script creates and stores database dumps
- [ ] `.env.example` documents all required environment variables
- [ ] No secrets are present in repository (verified by `git-secrets` or manual audit)

validation:
- `docker compose up --build -d` → verify `docker ps` shows all containers running
- `curl http://localhost:3000/health` → returns `{"status":"ok"}`
- `docker logs kordant-web` → no startup errors
- Push to a test branch and verify GitHub Actions CI pipeline runs
- Create a test release tag and verify CD pipeline triggers
- Run `scripts/backup.sh` and verify dump file created
- Run `scripts/deploy.sh` and verify deployment succeeds

notes:
- The unified monolith simplifies deployment significantly: one container instead of 5+ microservices.
- For high availability, run multiple web container instances behind a load balancer (nginx, AWS ALB, etc.).
- Consider using a managed database (RDS, Supabase, Neon) instead of self-hosted PostgreSQL for production.
- For Redis, consider Upstash or ElastiCache for managed service.
- The web app uses SolidStart with Nitro, which can run as a standalone server. Ensure the `.output/server/index.mjs` entry point is correct.
- For SSL, use Let's Encrypt with nginx or a managed load balancer. Document certificate renewal.
- Monitor disk space for logs and uploads. Configure log rotation in Docker.
- For mobile app deployment (iOS/Android), set up separate CI workflows for building and signing apps. This is outside the scope of the web deployment but should be documented.
- Consider using Terraform or Pulumi for infrastructure as code if managing cloud resources.