restructure tasks

tasks/shieldai-unified-restructure/15-darkwatch-router.md

@@ -0,0 +1,101 @@
+# 15. Backend Router — DarkWatch (Dark Web Monitoring)
+
+meta:
+  id: shieldai-unified-restructure-15
+  feature: shieldai-unified-restructure
+  priority: P1
+  depends_on: [shieldai-unified-restructure-12, shieldai-unified-restructure-13, shieldai-unified-restructure-14]
+  tags: [backend, trpc, darkwatch, security, api]
+
+objective:
+- Build the tRPC router for DarkWatch, the dark web monitoring service. Port all logic from `services/darkwatch/` and `packages/api/src/routes/darkwatch.routes.ts` into a unified `darkwatch` router and service layer.
+
+deliverables:
+- `web/src/server/api/routers/darkwatch.ts` — DarkWatch router:
+  - `darkwatch.getWatchlist` — `protectedProcedure` returning user's watchlist items
+  - `darkwatch.addWatchlistItem` — `protectedProcedure` adding email/phone/SSN/domain to watchlist
+  - `darkwatch.removeWatchlistItem` — `protectedProcedure` removing item
+  - `darkwatch.getExposures` — `protectedProcedure` returning detected exposures
+  - `darkwatch.getExposureDetails` — `protectedProcedure` returning single exposure with metadata
+  - `darkwatch.runScan` — `protectedProcedure` triggering manual scan (respects tier limits)
+  - `darkwatch.getScanStatus` — `protectedProcedure` checking scan progress
+  - `darkwatch.getReports` — `protectedProcedure` returning generated PDF reports
+- `web/src/server/services/darkwatch.service.ts` — Core business logic:
+  - `addWatchlistItem(userId, type, value)` — hash value, deduplicate, save to DB
+  - `removeWatchlistItem(userId, itemId)` — delete and cascade
+  - `getExposures(userId, filters?)` — query exposures with pagination, sorting
+  - `runScan(userId)` — orchestrate multi-source scan:
+    - HIBP breach check
+    - SecurityTrails lookup
+    - Censys/Shodan queries
+    - Dark web forum scraping (where applicable)
+  - `generateReport(userId, periodStart, periodEnd)` — compile exposures into PDF report
+  - `checkTierLimits(userId)` — verify scan frequency against subscription tier
+- `web/src/server/services/darkwatch/scan.engine.ts` — Scan orchestration:
+  - `scanHIBP(email)` — query Have I Been Pwned API
+  - `scanSecurityTrails(identifier)` — query SecurityTrails API
+  - `scanCensys(query)` — query Censys API
+  - `scanShodan(query)` — query Shodan API
+  - `scanForums(identifier)` — placeholder for forum scraping logic
+- `web/src/server/services/darkwatch/alert.pipeline.ts` — Exposure-to-alert pipeline:
+  - `processExposure(exposure)` — severity scoring, deduplication, alert creation
+  - `severityScore(exposure)` — calculate severity based on source, data type, recurrence
+
+steps:
+1. Create `web/src/server/api/routers/darkwatch.ts`.
+2. Define Zod schemas:
+   - `addWatchlistItemSchema`: `type: z.enum(['email', 'phoneNumber', 'ssn', 'address', 'domain'])`, `value: z.string()`
+   - `exposureFilterSchema`: `severity: z.enum(['info', 'warning', 'critical']).optional()`, `source: z.enum([...]).optional()`, `page: z.number().default(1)`, `limit: z.number().default(20)`
+3. Implement router procedures:
+   - Watchlist CRUD with user ownership checks
+   - Exposure queries with filtering and pagination
+   - Manual scan with tier limit enforcement
+4. Create `web/src/server/services/darkwatch.service.ts`:
+   - Port logic from `services/darkwatch/src/watchlist.service.ts`
+   - Port logic from `services/darkwatch/src/scan.service.ts`
+   - Port logic from `services/darkwatch/src/alert.pipeline.ts`
+5. Create scan engine modules:
+   - Each scanner is a function that takes an identifier and returns raw results
+   - Use environment variables for API keys (`HIBP_API_KEY`, `SECURITYTRAILS_API_KEY`, etc.)
+   - Implement circuit breaker pattern for external APIs (reference `services/spamshield/test/circuit-breaker.test.ts`)
+6. Create alert pipeline:
+   - `processExposure` creates or updates `Exposure` record
+   - If new or severity increased, create `Alert` record and trigger notification (via task 14 service)
+   - Deduplicate based on `identifierHash` and `source`
+7. Implement tier limit checks:
+   - Basic: 1 manual scan/month
+   - Plus: 1 manual scan/week
+   - Premium: unlimited + real-time monitoring
+8. Wire router into `web/src/server/api/root.ts`.
+9. Write unit tests for service functions with mocked external APIs.
+
+steps:
+- Unit: `addWatchlistItem` hashes and deduplicates values
+- Unit: `runScan` calls all scan engines and aggregates results
+- Unit: `severityScore` returns correct severity for different exposure types
+- Unit: `checkTierLimits` enforces scan frequency correctly
+- Unit: Alert pipeline creates alert only for new/high-severity exposures
+- Integration: tRPC procedures enforce user ownership
+
+acceptance_criteria:
+- [ ] Watchlist items can be added, listed, and removed per user
+- [ ] Exposures are queryable with filtering and pagination
+- [ ] Manual scan orchestrates all data sources and creates exposure records
+- [ ] Tier limits prevent excessive scanning based on subscription level
+- [ ] Alert pipeline creates notifications for new/high-severity exposures
+- [ ] External API failures are handled gracefully (circuit breaker, retries)
+- [ ] All user data is properly scoped (users cannot see other users' exposures)
+
+validation:
+- Add a test watchlist item, run manual scan, verify exposure records created
+- Check that exceeding tier limit returns appropriate error
+- Simulate HIBP API failure and verify circuit breaker opens
+- Run `cd web && pnpm test` for DarkWatch unit tests
+
+notes:
+- Reference legacy: `services/darkwatch/src/`, `packages/api/src/routes/darkwatch.routes.ts`
+- The HIBP API requires an API key for breach lookups. Store in `HIBP_API_KEY`.
+- SecurityTrails, Censys, and Shodan also require API keys. Document all required env vars.
+- SSN values should be hashed before storage (already in schema). Never log raw SSNs.
+- The scan engine should be designed to run both synchronously (manual scan) and asynchronously (scheduled scans via task 22).
+- Consider rate-limiting the `runScan` endpoint independently to prevent abuse.