restructure tasks

tasks/shieldai-unified-restructure/42-deployment-config.md

@@ -0,0 +1,177 @@
+# 42. Deployment — Update Docker, CI/CD, and Environment Configuration
+
+meta:
+  id: shieldai-unified-restructure-42
+  feature: shieldai-unified-restructure
+  priority: P0
+  depends_on: [shieldai-unified-restructure-41]
+  tags: [deployment, docker, ci-cd, infrastructure]
+
+objective:
+- Create production-ready deployment configuration for the unified monolith. Update Docker containers, CI/CD pipelines, and environment configuration to support the new single-app architecture plus native mobile builds.
+
+deliverables:
+- `web/Dockerfile` — Production container for the web app:
+  - Multi-stage build: Node.js builder → runtime
+  - Installs pnpm, dependencies, builds app
+  - Exposes port 3000
+  - Health check endpoint
+  - Non-root user for security
+- `docker-compose.yml` — Local development orchestration:
+  - `web` service: builds from `web/Dockerfile`, ports `3000:3000`
+  - `postgres` service: PostgreSQL 16 with volume for data persistence
+  - `redis` service: Redis 7 for job queues and caching
+  - `nginx` service: reverse proxy with SSL termination (optional)
+  - Environment variables from `.env` file
+- `docker-compose.prod.yml` — Production orchestration:
+  - Similar to dev but with production-optimized settings
+  - Volume mounts for uploads/logs
+  - Restart policies
+  - Resource limits
+- `.github/workflows/ci.yml` — GitHub Actions CI pipeline:
+  - Lint and type check (TypeScript)
+  - Unit tests for web app
+  - Build verification
+  - Dependency audit (`pnpm audit`)
+- `.github/workflows/deploy.yml` — GitHub Actions CD pipeline:
+  - Build Docker image on release tag
+  - Push to container registry (GitHub Packages, Docker Hub, or ECR)
+  - Deploy to staging on push to `main`
+  - Deploy to production on release tag
+  - Run database migrations before deployment
+  - Health check after deployment
+- `web/.env.example` — Complete environment variable documentation:
+  - Database: `DATABASE_URL`
+  - Redis: `REDIS_URL`
+  - Auth: `JWT_SECRET`, `NEXTAUTH_SECRET`
+  - Stripe: `STRIPE_SECRET_KEY`, `STRIPE_WEBHOOK_SECRET`, `STRIPE_PRICE_*`
+  - Notifications: `RESEND_API_KEY`, `FIREBASE_SERVICE_ACCOUNT_PATH`, `TWILIO_*`
+  - External APIs: `HIBP_API_KEY`, `SECURITYTRAILS_API_KEY`, etc.
+  - Monitoring: `SENTRY_DSN`, `DATADOG_API_KEY` (optional)
+  - App: `NODE_ENV`, `PORT`, `API_URL`, `APP_URL`
+- `scripts/deploy.sh` — Deployment helper script:
+  - Database backup before migration
+  - Migration runner
+  - Health check verification
+  - Rollback on failure
+- `scripts/backup.sh` — Database backup script:
+  - `pg_dump` to timestamped file
+  - Upload to S3 or similar storage
+
+steps:
+1. Create `web/Dockerfile`:
+   ```dockerfile
+   # Build stage
+   FROM node:20-alpine AS builder
+   WORKDIR /app
+   RUN npm install -g pnpm
+   COPY web/package.json web/pnpm-lock.yaml ./
+   RUN pnpm install --frozen-lockfile
+   COPY web/ .
+   RUN pnpm build
+
+   # Runtime stage
+   FROM node:20-alpine
+   WORKDIR /app
+   RUN npm install -g pnpm
+   COPY --from=builder /app/package.json /app/pnpm-lock.yaml ./
+   COPY --from=builder /app/node_modules ./node_modules
+   COPY --from=builder /app/.output ./.output
+   COPY --from=builder /app/drizzle ./drizzle
+   ENV NODE_ENV=production
+   EXPOSE 3000
+   HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
+     CMD curl -f http://localhost:3000/health || exit 1
+   USER node
+   CMD ["node", ".output/server/index.mjs"]
+   ```
+2. Create `docker-compose.yml`:
+   - Define services with appropriate environment variables
+   - PostgreSQL with `volumes: postgres_data:/var/lib/postgresql/data`
+   - Redis with `volumes: redis_data:/data`
+   - Network configuration
+3. Create `docker-compose.prod.yml`:
+   - Add restart policies: `unless-stopped`
+   - Add resource limits: `mem_limit`, `cpus`
+   - Add logging driver configuration
+   - Remove port bindings for internal services (postgres, redis)
+4. Create `.github/workflows/ci.yml`:
+   - Trigger: push to any branch, pull requests
+   - Jobs:
+     - `lint`: `pnpm lint`
+     - `typecheck`: `tsc --noEmit`
+     - `test`: `pnpm test`
+     - `build`: `pnpm build`
+     - `audit`: `pnpm audit --audit-level=high`
+5. Create `.github/workflows/deploy.yml`:
+   - Trigger: release tags (`v*`), manual dispatch
+   - Jobs:
+     - `build`: build Docker image, tag with version
+     - `push`: push to registry
+     - `migrate`: SSH into server, run `pnpm db:migrate`
+     - `deploy`: update Docker Compose stack
+     - `healthcheck`: verify `/health` endpoint responds 200
+   - Use GitHub secrets for SSH keys, registry credentials
+6. Update `web/.env.example`:
+   - Document every environment variable used by the app
+   - Include description, example value, and whether required
+   - Group by category (Database, Auth, Payments, APIs, etc.)
+7. Create `scripts/deploy.sh`:
+   - `#!/bin/bash` with error handling (`set -euo pipefail`)
+   - Backup database: `docker exec postgres pg_dump ...`
+   - Run migrations: `docker compose exec web pnpm db:migrate`
+   - Deploy: `docker compose -f docker-compose.prod.yml up -d`
+   - Health check: `curl -f http://localhost:3000/health`
+   - Rollback on failure: `docker compose rollback` or restore backup
+8. Create `scripts/backup.sh`:
+   - Generate timestamped dump
+   - Compress with gzip
+   - Upload to S3 using AWS CLI or rclone
+   - Retain last 30 backups
+9. Test deployment locally:
+   - `docker compose up --build`
+   - Verify app accessible at `http://localhost:3000`
+   - Verify database migrations run
+   - Verify health check passes
+10. Document deployment process in `docs/DEPLOYMENT.md`.
+
+steps:
+- Integration: `docker compose up --build` starts all services successfully
+- Integration: App is accessible and functional inside Docker
+- Integration: CI pipeline passes on GitHub Actions
+- Integration: Deployment script completes without errors
+- Security: Dockerfile uses non-root user
+- Security: No secrets committed to repository
+
+acceptance_criteria:
+- [ ] `web/Dockerfile` builds a production-ready container
+- [ ] `docker-compose.yml` orchestrates web, postgres, and redis for local dev
+- [ ] `docker-compose.prod.yml` is optimized for production with restart policies and resource limits
+- [ ] CI pipeline runs lint, type check, tests, build, and audit on every PR
+- [ ] CD pipeline builds and deploys on release tags
+- [ ] Database migrations run automatically before deployment
+- [ ] Health check endpoint verifies app is ready
+- [ ] Rollback script restores previous version on deployment failure
+- [ ] Backup script creates and stores database dumps
+- [ ] `.env.example` documents all required environment variables
+- [ ] No secrets are present in repository (verified by `git-secrets` or manual audit)
+
+validation:
+- `docker compose up --build -d` → verify `docker ps` shows all containers running
+- `curl http://localhost:3000/health` → returns `{"status":"ok"}`
+- `docker logs shieldai-web` → no startup errors
+- Push to a test branch and verify GitHub Actions CI pipeline runs
+- Create a test release tag and verify CD pipeline triggers
+- Run `scripts/backup.sh` and verify dump file created
+- Run `scripts/deploy.sh` and verify deployment succeeds
+
+notes:
+- The unified monolith simplifies deployment significantly: one container instead of 5+ microservices.
+- For high availability, run multiple web container instances behind a load balancer (nginx, AWS ALB, etc.).
+- Consider using a managed database (RDS, Supabase, Neon) instead of self-hosted PostgreSQL for production.
+- For Redis, consider Upstash or ElastiCache for managed service.
+- The web app uses SolidStart with Nitro, which can run as a standalone server. Ensure the `.output/server/index.mjs` entry point is correct.
+- For SSL, use Let's Encrypt with nginx or a managed load balancer. Document certificate renewal.
+- Monitor disk space for logs and uploads. Configure log rotation in Docker.
+- For mobile app deployment (iOS/Android), set up separate CI workflows for building and signing apps. This is outside the scope of the web deployment but should be documented.
+- Consider using Terraform or Pulumi for infrastructure as code if managing cloud resources.