Kordant/tasks/shieldai-unified-restructure/15-darkwatch-router.md

15. Backend Router — DarkWatch (Dark Web Monitoring)

meta: id: shieldai-unified-restructure-15 feature: shieldai-unified-restructure priority: P1 depends_on: [shieldai-unified-restructure-12, shieldai-unified-restructure-13, shieldai-unified-restructure-14] tags: [backend, trpc, darkwatch, security, api]

objective:

• Build the tRPC router for DarkWatch, the dark web monitoring service. Port all logic from services/darkwatch/ and packages/api/src/routes/darkwatch.routes.ts into a unified darkwatch router and service layer.

deliverables:

• web/src/server/api/routers/darkwatch.ts — DarkWatch router:
  • darkwatch.getWatchlist — protectedProcedure returning user's watchlist items
  • darkwatch.addWatchlistItem — protectedProcedure adding email/phone/SSN/domain to watchlist
  • darkwatch.removeWatchlistItem — protectedProcedure removing item
  • darkwatch.getExposures — protectedProcedure returning detected exposures
  • darkwatch.getExposureDetails — protectedProcedure returning single exposure with metadata
  • darkwatch.runScan — protectedProcedure triggering manual scan (respects tier limits)
  • darkwatch.getScanStatus — protectedProcedure checking scan progress
  • darkwatch.getReports — protectedProcedure returning generated PDF reports
• web/src/server/services/darkwatch.service.ts — Core business logic:
  • addWatchlistItem(userId, type, value) — hash value, deduplicate, save to DB
  • removeWatchlistItem(userId, itemId) — delete and cascade
  • getExposures(userId, filters?) — query exposures with pagination, sorting
  • runScan(userId) — orchestrate multi-source scan:
    • HIBP breach check
    • SecurityTrails lookup
    • Censys/Shodan queries
    • Dark web forum scraping (where applicable)
  • generateReport(userId, periodStart, periodEnd) — compile exposures into PDF report
  • checkTierLimits(userId) — verify scan frequency against subscription tier
• web/src/server/services/darkwatch/scan.engine.ts — Scan orchestration:
  • scanHIBP(email) — query Have I Been Pwned API
  • scanSecurityTrails(identifier) — query SecurityTrails API
  • scanCensys(query) — query Censys API
  • scanShodan(query) — query Shodan API
  • scanForums(identifier) — placeholder for forum scraping logic
• web/src/server/services/darkwatch/alert.pipeline.ts — Exposure-to-alert pipeline:
  • processExposure(exposure) — severity scoring, deduplication, alert creation
  • severityScore(exposure) — calculate severity based on source, data type, recurrence

steps:

1. Create web/src/server/api/routers/darkwatch.ts.
2. Define Zod schemas:
   • addWatchlistItemSchema: type: z.enum(['email', 'phoneNumber', 'ssn', 'address', 'domain']), value: z.string()
   • exposureFilterSchema: severity: z.enum(['info', 'warning', 'critical']).optional(), source: z.enum([...]).optional(), page: z.number().default(1), limit: z.number().default(20)
3. Implement router procedures:
   • Watchlist CRUD with user ownership checks
   • Exposure queries with filtering and pagination
   • Manual scan with tier limit enforcement
4. Create web/src/server/services/darkwatch.service.ts:
   • Port logic from services/darkwatch/src/watchlist.service.ts
   • Port logic from services/darkwatch/src/scan.service.ts
   • Port logic from services/darkwatch/src/alert.pipeline.ts
5. Create scan engine modules:
   • Each scanner is a function that takes an identifier and returns raw results
   • Use environment variables for API keys (HIBP_API_KEY, SECURITYTRAILS_API_KEY, etc.)
   • Implement circuit breaker pattern for external APIs (reference services/spamshield/test/circuit-breaker.test.ts)
6. Create alert pipeline:
   • processExposure creates or updates Exposure record
   • If new or severity increased, create Alert record and trigger notification (via task 14 service)
   • Deduplicate based on identifierHash and source
7. Implement tier limit checks:
   • Basic: 1 manual scan/month
   • Plus: 1 manual scan/week
   • Premium: unlimited + real-time monitoring
8. Wire router into web/src/server/api/root.ts.
9. Write unit tests for service functions with mocked external APIs.

steps:

• Unit: addWatchlistItem hashes and deduplicates values
• Unit: runScan calls all scan engines and aggregates results
• Unit: severityScore returns correct severity for different exposure types
• Unit: checkTierLimits enforces scan frequency correctly
• Unit: Alert pipeline creates alert only for new/high-severity exposures
• Integration: tRPC procedures enforce user ownership

acceptance_criteria:

• Watchlist items can be added, listed, and removed per user
• Exposures are queryable with filtering and pagination
• Manual scan orchestrates all data sources and creates exposure records
• Tier limits prevent excessive scanning based on subscription level
• Alert pipeline creates notifications for new/high-severity exposures
• External API failures are handled gracefully (circuit breaker, retries)
• All user data is properly scoped (users cannot see other users' exposures)

validation:

• Add a test watchlist item, run manual scan, verify exposure records created
• Check that exceeding tier limit returns appropriate error
• Simulate HIBP API failure and verify circuit breaker opens
• Run cd web && pnpm test for DarkWatch unit tests

notes:

• Reference legacy: services/darkwatch/src/, packages/api/src/routes/darkwatch.routes.ts
• The HIBP API requires an API key for breach lookups. Store in HIBP_API_KEY.
• SecurityTrails, Censys, and Shodan also require API keys. Document all required env vars.
• SSN values should be hashed before storage (already in schema). Never log raw SSNs.
• The scan engine should be designed to run both synchronously (manual scan) and asynchronously (scheduled scans via task 22).
• Consider rate-limiting the runScan endpoint independently to prevent abuse.