Kordant/tasks/shieldai-unified-restructure/42-deployment-config.md

42. Deployment — Update Docker, CI/CD, and Environment Configuration

meta: id: shieldai-unified-restructure-42 feature: shieldai-unified-restructure priority: P0 depends_on: [shieldai-unified-restructure-41] tags: [deployment, docker, ci-cd, infrastructure]

objective:

• Create production-ready deployment configuration for the unified monolith. Update Docker containers, CI/CD pipelines, and environment configuration to support the new single-app architecture plus native mobile builds.

deliverables:

• web/Dockerfile — Production container for the web app:
  • Multi-stage build: Node.js builder → runtime
  • Installs pnpm, dependencies, builds app
  • Exposes port 3000
  • Health check endpoint
  • Non-root user for security
• docker-compose.yml — Local development orchestration:
  • web service: builds from web/Dockerfile, ports 3000:3000
  • postgres service: PostgreSQL 16 with volume for data persistence
  • redis service: Redis 7 for job queues and caching
  • nginx service: reverse proxy with SSL termination (optional)
  • Environment variables from .env file
• docker-compose.prod.yml — Production orchestration:
  • Similar to dev but with production-optimized settings
  • Volume mounts for uploads/logs
  • Restart policies
  • Resource limits
• .github/workflows/ci.yml — GitHub Actions CI pipeline:
  • Lint and type check (TypeScript)
  • Unit tests for web app
  • Build verification
  • Dependency audit (pnpm audit)
• .github/workflows/deploy.yml — GitHub Actions CD pipeline:
  • Build Docker image on release tag
  • Push to container registry (GitHub Packages, Docker Hub, or ECR)
  • Deploy to staging on push to main
  • Deploy to production on release tag
  • Run database migrations before deployment
  • Health check after deployment
• web/.env.example — Complete environment variable documentation:
  • Database: DATABASE_URL
  • Redis: REDIS_URL
  • Auth: JWT_SECRET, NEXTAUTH_SECRET
  • Stripe: STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET, STRIPE_PRICE_*
  • Notifications: RESEND_API_KEY, FIREBASE_SERVICE_ACCOUNT_PATH, TWILIO_*
  • External APIs: HIBP_API_KEY, SECURITYTRAILS_API_KEY, etc.
  • Monitoring: SENTRY_DSN, DATADOG_API_KEY (optional)
  • App: NODE_ENV, PORT, API_URL, APP_URL
• scripts/deploy.sh — Deployment helper script:
  • Database backup before migration
  • Migration runner
  • Health check verification
  • Rollback on failure
• scripts/backup.sh — Database backup script:
  • pg_dump to timestamped file
  • Upload to S3 or similar storage

steps:

1. Create web/Dockerfile:

   # Build stage
   FROM node:20-alpine AS builder
   WORKDIR /app
   RUN npm install -g pnpm
   COPY web/package.json web/pnpm-lock.yaml ./
   RUN pnpm install --frozen-lockfile
   COPY web/ .
   RUN pnpm build
   
   # Runtime stage
   FROM node:20-alpine
   WORKDIR /app
   RUN npm install -g pnpm
   COPY --from=builder /app/package.json /app/pnpm-lock.yaml ./
   COPY --from=builder /app/node_modules ./node_modules
   COPY --from=builder /app/.output ./.output
   COPY --from=builder /app/drizzle ./drizzle
   ENV NODE_ENV=production
   EXPOSE 3000
   HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
     CMD curl -f http://localhost:3000/health || exit 1
   USER node
   CMD ["node", ".output/server/index.mjs"]
   

2. Create docker-compose.yml:
   • Define services with appropriate environment variables
   • PostgreSQL with volumes: postgres_data:/var/lib/postgresql/data
   • Redis with volumes: redis_data:/data
   • Network configuration
3. Create docker-compose.prod.yml:
   • Add restart policies: unless-stopped
   • Add resource limits: mem_limit, cpus
   • Add logging driver configuration
   • Remove port bindings for internal services (postgres, redis)
4. Create .github/workflows/ci.yml:
   • Trigger: push to any branch, pull requests
   • Jobs:
     • lint: pnpm lint
     • typecheck: tsc --noEmit
     • test: pnpm test
     • build: pnpm build
     • audit: pnpm audit --audit-level=high
5. Create .github/workflows/deploy.yml:
   • Trigger: release tags (v*), manual dispatch
   • Jobs:
     • build: build Docker image, tag with version
     • push: push to registry
     • migrate: SSH into server, run pnpm db:migrate
     • deploy: update Docker Compose stack
     • healthcheck: verify /health endpoint responds 200
   • Use GitHub secrets for SSH keys, registry credentials
6. Update web/.env.example:
   • Document every environment variable used by the app
   • Include description, example value, and whether required
   • Group by category (Database, Auth, Payments, APIs, etc.)
7. Create scripts/deploy.sh:
   • #!/bin/bash with error handling (set -euo pipefail)
   • Backup database: docker exec postgres pg_dump ...
   • Run migrations: docker compose exec web pnpm db:migrate
   • Deploy: docker compose -f docker-compose.prod.yml up -d
   • Health check: curl -f http://localhost:3000/health
   • Rollback on failure: docker compose rollback or restore backup
8. Create scripts/backup.sh:
   • Generate timestamped dump
   • Compress with gzip
   • Upload to S3 using AWS CLI or rclone
   • Retain last 30 backups
9. Test deployment locally:
   • docker compose up --build
   • Verify app accessible at http://localhost:3000
   • Verify database migrations run
   • Verify health check passes
10. Document deployment process in docs/DEPLOYMENT.md.

steps:

• Integration: docker compose up --build starts all services successfully
• Integration: App is accessible and functional inside Docker
• Integration: CI pipeline passes on GitHub Actions
• Integration: Deployment script completes without errors
• Security: Dockerfile uses non-root user
• Security: No secrets committed to repository

acceptance_criteria:

• web/Dockerfile builds a production-ready container
• docker-compose.yml orchestrates web, postgres, and redis for local dev
• docker-compose.prod.yml is optimized for production with restart policies and resource limits
• CI pipeline runs lint, type check, tests, build, and audit on every PR
• CD pipeline builds and deploys on release tags
• Database migrations run automatically before deployment
• Health check endpoint verifies app is ready
• Rollback script restores previous version on deployment failure
• Backup script creates and stores database dumps
• .env.example documents all required environment variables
• No secrets are present in repository (verified by git-secrets or manual audit)

validation:

• docker compose up --build -d → verify docker ps shows all containers running
• curl http://localhost:3000/health → returns {"status":"ok"}
• docker logs shieldai-web → no startup errors
• Push to a test branch and verify GitHub Actions CI pipeline runs
• Create a test release tag and verify CD pipeline triggers
• Run scripts/backup.sh and verify dump file created
• Run scripts/deploy.sh and verify deployment succeeds

notes:

• The unified monolith simplifies deployment significantly: one container instead of 5+ microservices.
• For high availability, run multiple web container instances behind a load balancer (nginx, AWS ALB, etc.).
• Consider using a managed database (RDS, Supabase, Neon) instead of self-hosted PostgreSQL for production.
• For Redis, consider Upstash or ElastiCache for managed service.
• The web app uses SolidStart with Nitro, which can run as a standalone server. Ensure the .output/server/index.mjs entry point is correct.
• For SSL, use Let's Encrypt with nginx or a managed load balancer. Document certificate renewal.
• Monitor disk space for logs and uploads. Configure log rotation in Docker.
• For mobile app deployment (iOS/Android), set up separate CI workflows for building and signing apps. This is outside the scope of the web deployment but should be documented.
• Consider using Terraform or Pulumi for infrastructure as code if managing cloud resources.