New service for helping clients remove personal listings from data broker sites.
Service features:
- BrokerRegistry: Catalog of 20+ data brokers with removal methods
- RemoveBrokersService: Core service for scanning, creating removal requests,
submitting removals, and verifying completions
- RemoveBrokersScheduler: Automated processing of pending removals and
verification of completed removals
- BrokerAlertPipeline: Alert integration for listing discoveries and removal status
API endpoints (/removebrokers):
- GET /brokers - List available data brokers
- GET /status - Get removal request status and stats
- POST /scan - Scan for personal listings across brokers
- POST /request - Create a new removal request
- GET /request/:id - Get specific removal request details
- DELETE /request/:id - Cancel a removal request
- POST /process - Trigger processing of pending removals
- POST /verify/:id - Manually verify a removal completion
DB models: InfoBroker, RemovalRequest, BrokerListing
Types: BrokerStatus, RemovalStatus, RemovalMethod, and related interfaces
- Make verifyCustomerOwnership public in BillingService
- Add ownership verification before fetching invoice history
- Returns 403 if customerId does not belong to authenticated user
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- ATTOM Property API integration for structured property data
- USPS address standardization via API
- County clerk/recorder feed scraping for deed changes and liens
- Rate limiting, caching, and retry logic
- Unit tests for each data source adapter
- PropertyRecord, CountyDeedRecord, DataSourceType types in types.ts
- Consolidated type exports in index.ts
Co-Authored-By: Paperclip <noreply@paperclip.ing>
WebhookService.constructEvent now reads from config.stripe.webhookSecret
instead of process.env.STRIPE_WEBHOOK_SECRET, matching BillingService.handleWebhook.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Replace in-memory Map<string, number> with Redis-based idempotency
using setIfNotExists (NX) for distributed multi-instance deployments.
Removes cleanupOldEvents (no longer needed with Redis TTL).
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Add MIXPANEL_TOKEN, MIXPANEL_API_SECRET, ANALYTICS_ENV to .env.example
- Add packages/web/.env.example with VITE_MIXPANEL_TOKEN and other analytics vars
- Update docs/MIXPANEL_ANALYTICS.md with complete setup instructions
- Document event taxonomy (30+ events across User, Subscription, DarkWatch, VoicePrint, SpamShield)
- Add KPI definitions (MAU, MRR, conversion, churn, CAC, LTV, NPS, viral coefficient)
- Include integration examples for backend and frontend usage
- Document alert thresholds for monitoring
Implementation was already complete in packages/shared-analytics and packages/web.
This completes the configuration and documentation for Mixpanel setup.
FRE-5281
- Added @shieldai/shared-notifications, bullmq, ioredis deps to API
- POST /api/waitlist/signup now sends waitlist_confirmation email via EmailService
- Schedules welcome sequence (day1 intro, day3 features, day7 launch teaser) via BullMQ delayed jobs
- Added waitlist email worker in @shieldai/jobs to process delayed welcome sequence emails
- Templates already in place: waitlist_confirmation, waitlist_intro, waitlist_features, waitlist_launch_teaser with dark-themed HTML layouts
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Add SHA256 verification for k6 binary download (supply chain integrity)
- Remove literal 'test-token' fallback for API_TOKEN in CI workflow;
add validation step that fails if LOAD_TEST_API_TOKEN secret is missing
- Replace 'test-token' fallback with empty string + warning in run-all.sh
- Replace 'test-token' fallback with empty string in all 4 service scripts
- P1#1: Document constant-arrival-rate limitation (no setup() data to scenarios)
- P1#2: Capture EXIT_CODE inside each case branch to avoid set -e truncation
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Add DD_API_KEY and DD_SITE to Zod validation schema (config.ts)
- Truncate API key before storing in user.id to prevent Sentry leak (auth.middleware.ts)
Import datadog-init as first module to ensure dd-trace .init()
runs before any other imports, fixing P1 auto-instrumentation issue.
Removed redundant manual initDatadog/initSentry calls since
datadog-init.ts already invokes all three init functions.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
P1: Load dd-trace before other modules via datadog-init.ts entry point
P1: Batch all CloudWatch metrics into single PutMetricDataCommand per request
P2: Deduplicate warning logs with else-if for high latency vs error
P3: Add response.ok check to Datadog log forwarding fetch
P3: Update getSentryHub() to use getCurrentScope() for Sentry SDK 8.x
Co-Authored-By: Paperclip <noreply@paperclip.ing>
P1 fixes:
- Fix import paths in background/index.ts (./ -> ../lib/)
- Fix Promise-in-string bug in api-client.ts authenticate()
- Add missing background/service_worker key to manifest
- Copy HTML to public/ so Vite places them in dist
P2 fixes:
- Add notifications permission to manifest
- Make showWarningNotification async with proper await
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- P2: Remove dead heredoc from run.sh mixed scenario
- P2: Add setup() warmup to seed real tokens for standalone scenarios
- P3: Replace handleSummary file output with --summary-export in run.sh
- P3: Add .gitignore for k6 results and .env
- Fix stray closing brace in scripts/load-test/lib/common.js
Co-Authored-By: Paperclip <noreply@paperclip.ing>
P1-1: Replace non-deterministic Math.random() with buffer-variance score
P1-7: Fix findSimilar result ordering by using Map instead of index zip
P2-2: Replace weak hashes with SHA-256 for both embedding and audio
Co-Authored-By: Paperclip <noreply@paperclip.ing>
P3-1: Replace envSchema.parse() with safeParse() + default fallback to
avoid module-level crash when env vars are missing.
P3-3: Add fs.existsSync check on ECAPA_TDNN_MODEL_PATH at startup
with warning log when model path is missing.
P3-4: Add Zod strict() mode to env schema to catch typos in env
var names (extra keys now produce validation errors).
P1-6: Confirmed resolved - voiceprint.service.ts already imports
VoiceEnrollment/VoiceAnalysis from @shieldai/db (consolidated package).
P1-2: Add onRequest auth hook to reject anonymous requests on all 7
VoicePrint endpoints. Previously, the auth middleware always attached
a placeholder user (id='anonymous'), so per-route userId checks passed
for unauthenticated clients.
P1-3: Replace JSON body parsing with @fastify/multipart for POST
/endpoints (/enroll, /analyze, /batch). Fastify JSON parser cannot
produce Buffer from request.body; multipart/form-data is required
for audio file uploads. Added 50MB file size limit.
- Report service: data collection from all three engines, HTML rendering (Handlebars), PDF generation (pdfkit)
- REST API: /reports endpoints for generate, history, view, PDF download, scheduling
- BullMQ workers: queued report generation with retry, monthly/annual scheduler triggers
- DB: SecurityReport model with Prisma schema and type exports
- Email: report_ready template in shared-notifications
- All dependencies wired through existing packages
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Add ALLOWED_ORIGINS env var with comma-separated origin list
- Validate origins at startup in production: reject wildcards, empty values,
and malformed URLs (non-http/https protocol)
- Update both server entry points (server.ts, index.ts) to use getCorsOrigins()
- Development mode retains existing localhost fallback behavior
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Rewrote spamshield.js to test real endpoints: POST /sms/classify,
POST /number/reputation, POST /call/analyze, POST /feedback,
GET /history, GET /statistics
- Added proper P99 latency thresholds per classification type:
SMS classify < 150ms, number reputation < 300ms, call analyze < 400ms
- Previous version tested non-existent endpoints (/classify, /health, /blocklist/check)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Add load-test job to ci.yml that runs after docker-build on push to main
- Create combined load test runner (scripts/load-test/run-all.sh) for all services
- Create k6 load test scripts for api, darkwatch, spamshield, and voiceprint
- Add shared k6 utilities (lib/common.js)
- Update load-test.yml to support all services and report artifacts
- Configure k6 cloud output and P99 threshold validation
- Generate load test report as CI artifact
Co-Authored-By: Paperclip <noreply@paperclip.ing>
