shortcommings

tasks/core-services-implementation/06-spamshield-reputation.md

@@ -0,0 +1,70 @@
+# 06. Twilio Lookup and Phone Reputation API Integration
+
+meta:
+  id: core-services-06
+  feature: core-services-implementation
+  priority: P1
+  depends_on: [core-services-01]
+  tags: [spamshield, reputation, twilio, caller-id, api-integration, table-stakes]
+
+objective:
+- Replace the stub Hiya/Truecaller lookup functions that return `{ score: 0, isSpam: false }` with real phone reputation API integrations (Twilio Lookup) and integrate results into the spam classification pipeline.
+
+deliverables:
+- Twilio Lookup API client for caller name, line type, and carrier info
+- Phone reputation scoring system with caching
+- Integration with existing rule engine (reputation score augments rule-based decisions)
+- STIR/SHAKEN attestation verification (if carrier partnership available)
+- Rate-limited, cost-aware API usage
+
+steps:
+1. Sign up for Twilio account and enable Lookup API at https://www.twilio.com/lookup
+2. Add `TWILIO_ACCOUNT_SID`, `TWILIO_AUTH_TOKEN` to `.env.example`
+3. Create `spamshield/twilio.client.ts`:
+   - `lookupPhone(phoneNumber, type?)` — caller name, line type (mobile/landline/VoIP), carrier
+   - `lookupReputation(phoneNumber)` — spam risk score, call volume, report counts
+   - `verifyStirShaken(phoneNumber)` — attestation level (A/B/C) if available
+4. Replace stub `lookupHiya()` and `lookupTruecaller()` in `reputation.api.ts` with real Twilio calls
+5. Implement reputation scoring algorithm:
+   - Twilio spam risk score (0–100) mapped to internal confidence (0.0–1.0)
+   - Line type weighting: VoIP = higher risk, landline = lower risk
+   - Carrier reputation: known spam carriers = +20 risk
+   - STIR/SHAKEN attestation: Full attestation (A) = -30 risk, None (C) = +20 risk
+6. Cache results in Redis with 24h TTL (phone numbers don't change reputation rapidly)
+7. Wire into `spamshield.service.ts`:
+   - Before rule engine, check reputation
+   - If reputation confidence > 0.7, block immediately
+   - If reputation confidence 0.4–0.7, flag for review
+   - If reputation confidence < 0.4, proceed to rule engine + ML classifier
+8. Add cost tracking: $0.004–$0.03 per lookup, track monthly usage per user
+9. Implement fallback: if Twilio API fails, use internal rule engine only (graceful degradation)
+
+tests:
+- Unit: Mock Twilio API responses, verify reputation scoring algorithm
+- Integration: Test with real Twilio Lookup API using known spam number
+- E2E: Submit spam check for phone number → verify reputation lookup → get classification result
+
+acceptance_criteria:
+- [ ] `lookupPhone()` makes real HTTP request to Twilio Lookup API
+- [ ] Reputation scores are calculated from real Twilio data (not hardcoded zeros)
+- [ ] High-reputation numbers (confidence > 0.7) trigger automatic block without rule/ML processing
+- [ ] Cache stores reputation results for 24 hours, reducing API costs
+- [ ] Twilio API failures gracefully fall back to rule engine (no crashes)
+- [ ] Cost tracking records each lookup for billing analytics
+- [ ] STIR/SHAKEN attestation is checked and factored into score when available
+- [ ] VoIP lines get +20 risk weighting compared to landline
+- [ ] Internal DB cache (`lookupInternalDB`) is checked before Twilio API call
+- [ ] Rate limits: max 100 lookups/minute per user to prevent abuse
+
+validation:
+- Run `vitest run spamshield.service.test.ts` — all tests pass
+- Manual: Check reputation for known spam number (e.g., reported robocall number), verify high score
+- Check cache: Redis `GET spamshield:reputation:+15551234567` returns cached result
+- Monitor cost: Database shows lookup usage per user per month
+
+notes:
+- Twilio Lookup costs $0.004 per basic lookup, $0.03 per advanced lookup (reputation, caller name)
+- At 100 lookups/user/month, cost is $0.40–$3.00 per user — manageable at $12+/mo ARPU
+- Hiya and Truecaller have proprietary APIs but require carrier partnerships — Twilio is the best consumer-accessible option
+- STIR/SHAKEN requires telecom partner for full attestation data — implement if/when partnership exists
+- The existing rule engine (`ruleEngine()`) is functional — reputation augments it, doesn't replace it