Kordant/tasks/shieldai-unified-restructure/19-removebrokers-router.md

19. Backend Router — RemoveBrokers (Data Broker Removal)

meta: id: shieldai-unified-restructure-19 feature: shieldai-unified-restructure priority: P1 depends_on: [shieldai-unified-restructure-12, shieldai-unified-restructure-13, shieldai-unified-restructure-14] tags: [backend, trpc, removebrokers, privacy, api]

objective:

• Build the tRPC router for RemoveBrokers, the data broker removal service. Port all logic from services/removebrokers/ and packages/api/src/routes/removebrokers.routes.ts into a unified removebrokers router and service layer.

deliverables:

• web/src/server/api/routers/removebrokers.ts — RemoveBrokers router:
  • removebrokers.getBrokerRegistry — protectedProcedure returning list of supported brokers
  • removebrokers.getRemovalRequests — protectedProcedure returning user's removal requests
  • removebrokers.createRemovalRequest — protectedProcedure initiating removal for a broker
  • removebrokers.getRequestStatus — protectedProcedure checking removal progress
  • removebrokers.getBrokerListings — protectedProcedure returning found listings
  • removebrokers.scanForListings — protectedProcedure scanning brokers for user's data
  • removebrokers.getStats — protectedProcedure returning removal statistics
• web/src/server/services/removebrokers.service.ts — Core business logic:
  • getBrokerRegistry() — return all active brokers with metadata
  • createRemovalRequest(subscriptionId, brokerId, personalInfo) — validate, create request, initiate removal
  • getRequestStatus(requestId) — return current status and history
  • scanForListings(subscriptionId, brokerId) — search broker site for user's data
  • processRemovals() — batch processor for pending removals
  • updateRequestStatus(requestId, status, metadata) — update after broker response
• web/src/server/services/removebrokers/broker.registry.ts — Broker definitions:
  • Static registry of supported data brokers
  • Each entry: name, domain, category, removalMethod, removalUrl, requiresAccount, estimatedDays
  • Methods: AUTOMATED, MANUAL_FORM, EMAIL, PHONE, MAIL
• web/src/server/services/removebrokers/removal.engine.ts — Removal automation:
  • submitAutomatedRemoval(broker, personalInfo) — API-based removal where available
  • generateFormPayload(broker, personalInfo) — prepare form data for manual submission
  • sendRemovalEmail(broker, personalInfo) — email-based removal request
  • trackRemovalStatus(broker, requestId) — poll broker for status updates

steps:

1. Create web/src/server/api/routers/removebrokers.ts.
2. Define Zod schemas:
   • createRequestSchema: brokerId: z.string().uuid(), personalInfo: z.object({ fullName: z.string(), email: z.string().optional(), phone: z.string().optional(), address: z.string().optional(), dob: z.string().optional() })
   • scanSchema: brokerId: z.string().uuid().optional() (if omitted, scan all)
3. Implement router procedures:
   • Broker registry listing (public or protected)
   • Removal request CRUD
   • Listing scan and results
   • Stats aggregation
4. Create web/src/server/services/removebrokers.service.ts:
   • Port from services/removebrokers/src/
   • Implement request lifecycle: PENDING → SUBMITTED → IN_PROGRESS → COMPLETED/FAILED
5. Create broker registry:
   • Hardcode initial list of 20-50 major data brokers
   • Include removal instructions and URLs
   • Allow admin updates via API (future enhancement)
6. Create removal engine:
   • submitAutomatedRemoval: call broker API if available (rare)
   • generateFormPayload: create structured data for form filling
   • sendRemovalEmail: use notification service (task 14) to send removal request email
   • trackRemovalStatus: placeholder for polling logic
7. Implement listing scanner:
   • Search broker websites for user's name, email, phone
   • Use web scraping (Cheerio, Playwright) where APIs are unavailable
   • Store found listings in BrokerListing table
8. Implement scheduler integration:
   • Pending removals should be picked up by background job (task 22)
   • Retries for failed removals with exponential backoff
9. Wire router into web/src/server/api/root.ts.
10. Write unit tests with mocked broker interactions.

steps:

• Unit: createRemovalRequest creates record with PENDING status
• Unit: processRemovals advances eligible requests to SUBMITTED
• Unit: scanForListings creates BrokerListing records for found data
• Unit: Broker registry returns correct metadata for known brokers
• Integration: tRPC procedures enforce subscription scoping

acceptance_criteria:

• Broker registry lists all supported data brokers with removal methods
• Removal requests can be created per broker with personal info
• Request status tracks lifecycle from PENDING to COMPLETED/FAILED
• Listings scanner finds user's data on broker sites
• Automated removals use APIs where available; manual methods generate instructions
• Failed removals are retried with exponential backoff
• Stats endpoint shows removal progress (total, completed, pending, failed)

validation:

• List brokers and verify registry data
• Create a removal request and verify DB record
• Simulate a scan and verify listing records created
• Run cd web && pnpm test for RemoveBrokers unit tests

notes:

• Reference legacy: services/removebrokers/src/, packages/api/src/routes/removebrokers.routes.ts
• Data broker removal is often a manual process. The automation layer should handle the easy cases and provide clear instructions for manual cases.
• Web scraping broker sites may violate Terms of Service. Use public APIs where available and document legal considerations.
• Personal info submitted for removal should be handled carefully. Do not log raw PII.
• The removal engine should be designed as a plugin system: each broker can have its own adapter for API, form, or email removal.
• Consider integrating with a third-party service (e.g., DeleteMe, Optery) for broader broker coverage if building individual adapters is impractical.