98b01bf48f
Add k6 load test scripts for Darkwatch authentication endpoints (FRE-4928)
...
- darkwatch-auth.js: k6 script testing POST /auth/login, /auth/logout, /auth/refresh
- P99 thresholds: login <200ms, logout <100ms, refresh <150ms
- Config: 500 req/s sustained for 5 minutes
- Mixed workload scenario + individual endpoint scenarios
- .env.example and run.sh for execution
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-09 08:08:10 -04:00
Senior Engineer
cb5851ec8c
Add k6 load test scripts for Voiceprint verification endpoints (FRE-4930)
...
- k6 script with P99 latency thresholds (enrollment <500ms, verification <250ms, model retrieval <100ms)
- Configurable 500 req/s sustained throughput for 5 minutes
- Mixed workload scenario + individual endpoint scenarios
- GitHub Actions workflow for automated load testing
- Runner script with environment configuration
- JSON result export for CI artifact collection
- .gitignore entry for load test results
2026-05-09 07:50:29 -04:00
bce4787802
Add rollback procedure documentation and testing scripts (FRE-4808)
...
- infra/ROLLBACK.md: comprehensive rollback runbook with ECS, Docker Compose,
database migration, blue-green, and emergency rollback procedures
- infra/scripts/rollback.sh: enhanced ECS rollback with validation, logging,
health verification, and per-service rollback support
- infra/scripts/rollback-compose.sh: Docker Compose rollback for local/staging
- infra/scripts/rollback-migration.sh: Drizzle migration rollback with
AWS Secrets Manager integration
- infra/scripts/test-rollback.sh: automated test suite (51 tests)
- Updated infra/README.md to reference ROLLBACK.md
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-09 06:27:31 -04:00
540ca5ebad
Add k6 load testing infrastructure for Darkwatch service
...
- Create load test directory structure (infra/load-tests/)
- Implement k6 script for Darkwatch endpoints (darkwatch.js)
- Tests watchlist, scan, exposure, and alert operations
- Configured for 500 req/s sustained load with P99 < 200ms
- Includes error rate metrics and threshold validation
- Add documentation and usage guide (README.md)
Related: [FRE-4807](/FRE/issues/FRE-4807)
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-09 06:18:47 -04:00
Senior Engineer
a0799c0647
Add Terraform AWS infrastructure and enhanced CI/CD pipeline (FRE-4574)
...
- Terraform modules: VPC, ECS Fargate, RDS PostgreSQL, ElastiCache Redis, S3, Secrets Manager, CloudWatch
- Multi-environment support: staging and production configs
- ECS auto-scaling: CPU-based scaling with configurable min/max
- CI/CD: pnpm caching, Docker Buildx, Trivy security scanning, Terraform plan on PR
- Deploy: ECS service updates with automatic rollback on health check failure
- Backup: automated RDS snapshots, S3 versioning, ElastiCache snapshots
- Monitoring: CloudWatch dashboards, CPU/memory/5xx alarms
- Rollback script for manual service rollback
- Infrastructure documentation with architecture overview
2026-05-08 02:54:39 -04:00
baa216d62c
turbo
2026-05-03 22:45:03 -04:00
f2593c1e67
use crypto package instead
2026-05-03 22:44:48 -04:00
a4684e9121
Fix SMS classifier test mock: add defaultScores and metadataLimits exports (FRE-4509)
...
The test mock for spamshield.config was missing defaultScores and
metadataLimits exports that are imported by spamshield.service.ts,
causing 8 tests to fail with 'No defaultScores export is defined'.
2026-05-02 20:23:29 -04:00
Senior Engineer
91e4985a8e
FRE-4474 Phase 5: Verify and resolve security review findings for SpamShield and Cross-Service Correlation
...
- FRE-4499 (SpamShield): Verified 6 security fixes (2 High, 4 Medium)
- S01: Pre-compiled regex in RuleEngine (ReDoS fix)
- S02: SmsClassifier accepts senderPhoneNumber context
- S03: AlertServer JWT auth + origin validation
- S04: SHA-256 phone hashing (PII protection)
- S05: DecisionEngine timeout enforcement via Promise.race
- S06: CarrierFactory.getAllCarriers properly async/await
- FRE-4500 (Correlation): Verified 7 security fixes (2 Critical, 2 High, 2 Medium, 1 Low)
- C1: Ingest endpoints auth via request.user.id
- C2: IDOR protection on group endpoints (userId filter)
- H3: JWT middleware registered in server.ts
- H4: Fastify schema validation on all routes
- M6: Payload sanitization with depth limit and circular ref detection
- L7: CORS origin restricted to env var
- Resolved liveness incidents FRE-4652 and FRE-4654
- All Phase 5 child issues now complete
2026-05-02 18:36:29 -04:00
0afdf8b6e8
FRE-4500: Fix security review findings (Critical/High/Medium/Low)
...
- Critical #1 : Add auth check to ingest endpoints (use request.user.id)
- Critical #2 : Add IDOR protection on group endpoints (userId ownership)
- High #3 : Register auth middleware in server.ts (populates request.user)
- High #4 : Add Fastify schema validation to all route handlers
- Medium #5 : Add NormalizedAlert/CorrelationGroup models to Prisma schema
- Medium #6 : Sanitize payload storage in normalizer (depth limit, circular ref)
- Low #7 : Restrict CORS origins (use CORS_ORIGIN env var)
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-02 16:40:01 -04:00
274afa6335
FRE-4499: Fix security review findings (S01-S06)
...
- S01 (High): Pre-compile regex patterns in RuleEngine.loadActiveRules() and
cache them; eliminate per-evaluation RegExp construction in rule-engine.ts
and spamshield.service.ts (ReDoS mitigation)
- S02 (High): SMS classifier now accepts optional senderPhoneNumber via
SmsClassificationContext; reputation check uses actual sender instead of
hardcoded 'placeholder'
- S03 (Medium): AlertServer (services/spamshield) now enforces JWT auth,
origin allowlist, and max client limit on WebSocket connections
- S04 (Medium): hashPhoneNumber() uses SHA-256 (crypto.createHash) instead
of reversible hex encoding (Buffer.toString('hex'))
- S05 (Medium): DecisionEngine.evaluate() wraps evaluation in Promise.race
with configurable evaluationTimeout; returns fallback decision on timeout
- S06 (Medium): CarrierFactory.getAllCarriers() is now async and properly
awaits isHealthy() promises instead of returning raw Promise objects
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-02 15:58:49 -04:00
24bc9c235f
Consolidate @shieldai/db and @shieldsai/shared-db packages (FRE-4603)
...
- Merged singleton pattern + type exports from shared-db
- Kept FieldEncryptionService from original db package
- Upgraded to Prisma v6.2.0 (newer version)
- Adopted shared-db's complete schema for multi-service platform
- Updated 17 consumer imports across darkwatch, voiceprint, jobs, api
- Standardized on @shieldai/db namespace
Files changed:
- packages/db/package.json (v0.1.0 → v0.2.0)
- packages/db/src/index.ts (consolidated exports)
- packages/db/prisma/schema.prisma (merged schema)
- packages/db/prisma/seed.ts (updated for new schema)
- 17 consumer files updated
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-02 15:06:02 -04:00
93ff4885ee
Add integration tests README documentation (FRE-4522)
...
Documentation for integration test suite including:
- Test file descriptions and coverage
- External provider mock configuration
- Running tests commands
- CI integration requirements
- Environment variables needed
- Test strategy and error scenarios
2026-05-02 13:23:12 -04:00
67622a2f11
Add integration tests for notification services (FRE-4522)
...
Comprehensive integration test suite for notification services:
- EmailService integration tests (Resend provider)
- SMSService integration tests (Twilio provider)
- PushService integration tests (FCM/APNs providers)
- NotificationService integration tests (orchestration layer)
Test coverage includes:
- Successful notification delivery
- Error handling (API errors, network timeouts, invalid inputs)
- Rate limiting enforcement
- Batch operations with partial failures
- Notification preferences and deduplication
- Template-based email sending
- Metadata and attachment handling
Total: ~1400 lines across 4 test files
2026-05-02 13:22:41 -04:00
bdf8ad30b6
Apply security remediations for FRE-4498 (FRE-4612)
...
Security findings from April 30 review were claimed fixed but never committed.
Applied all remediations:
HIGH:
- WebhookHandler: fail fast when DARKWATCH_WEBHOOK_SECRET missing instead of defaulting to hardcoded secret
- field-encryption.service: require PII_ENCRYPTION_KEY at startup instead of defaulting
MEDIUM:
- WebhookHandler: make signature required (was optional, accepted unsigned events)
- WebhookHandler: reject unknown event types instead of silently defaulting to SCAN_TRIGGER
- scheduler.routes + webhook.routes: add ownership checks on /:userId endpoints (IDOR)
LOW:
- webhook.routes: generic error responses, full error logged server-side
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-02 13:03:28 -04:00
f34adc5e82
Add null checks in feedback processing pipeline (FRE-4514)
...
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-02 13:01:02 -04:00
e704a9074a
FRE-4533: Merge apps/{api,web,mobile} and shared-db into ShieldAI repo
...
Merge FrenoCorp apps into ShieldAI packages/:
- packages/api: merged routes (notifications), middleware (auth, rate-limit, error, logging), config, services (darkwatch, spamshield, voiceprint), tests
- packages/web: new SolidJS web app stub
- packages/mobile: new SolidJS mobile app stub
- packages/shared-db: new Prisma DB package (separate from existing packages/db)
- pnpm-workspace.yaml: restored (apps/* removed, already covered by packages/*)
Next: reconcile packages/shared-db with packages/db, and fix server.ts correlationRoutes import
2026-05-02 10:19:11 -04:00
1197fe48f7
FRE-4533: Merge apps/{api,web,mobile} and shared-db into ShieldAI repo
...
- Copy apps/api (Fastify server with spamshield/voiceprint/darkwatch services)
- Copy apps/web (SolidJS web app)
- Copy apps/mobile (SolidJS mobile app)
- Copy packages/shared-db (Prisma schema/models)
- Add apps/* to pnpm-workspace.yaml
2026-05-02 10:16:18 -04:00
1e42c4a5c2
FRE-4529: Transfer ShieldAI code from FrenoCorp repo
...
Transferred ShieldAI-related files mistakenly placed in ~/code/FrenoCorp:
- Services: spamshield (feature-flags, audit-logger, error-handler), voiceprint (config, service, feature-flags), darkwatch (pipeline, scan, scheduler, watchlist, webhook)
- Packages: shared-analytics, shared-auth, shared-ui, shared-utils (new); shared-billing, jobs supplemented with unique FC files
- Server: alerts (FC version newer), routes (spamshield, darkwatch, voiceprint)
- Config: turbo.json, tsconfig.base.json, vite/vitest configs, drizzle, Dockerfile
- VoicePrint ML service
- Examples
Pending: apps/{api,web,mobile}/ structured merge, shared-db/db mapping
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-02 10:13:13 -04:00
8687868632
Add request ID validation and CSPRNG fallback (FRE-4516)
...
- Max-length guard (256 chars) on incoming request IDs to prevent log bloat
- Format whitelist (alphanumeric, hyphen, underscore) to prevent log injection
- Replace Math.random() with crypto.randomBytes in fallback for CSPRNG
2026-05-02 09:43:13 -04:00
fe754761d9
Auto-commit 2026-05-02 09:37
2026-05-02 09:37:30 -04:00
b6b0f86d73
Add MixpanelService with hashed phoneNumber in spamBlocked() (FRE-4519)
...
Create MixpanelService that uses FieldEncryptionService.hashPhoneNumber()
to SHA-256 hash phone numbers before sending to Mixpanel analytics.
- Implement spamBlocked() method with phone number hashing
- Add 16 unit tests verifying hash correctness and API behavior
- Export service from package index
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-02 09:21:42 -04:00
b01b79d02a
Add ReDoS validation for SpamRule.pattern field (FRE-4512)
...
- Create regex-validation utility with ReDoS detection (nested quantifiers,
overlapping alternations, complexity limits)
- Add @db.VarChar(500) constraint on pattern field in Prisma schema
- Integrate validation in rule-engine at load time and evaluation time
- Add 46 unit tests covering syntax, ReDoS patterns, complexity, edge cases
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-02 07:23:39 -04:00
e580a693c7
FRE-4510: Implement feature flag checks for spam classification
...
- Add runtime flag evaluation from FLAG_<KEY> environment variables
- Add enableCallAnalysis flag check to analyzeCall() and interceptCall()
- Add enableFeedbackLoop flag check to recordFeedback()
- Add 19 tests for feature flag behavior (checkFeatureFlag, getters, service integration)
- Add vitest config and test script to spamshield package
2026-05-02 01:53:59 -04:00
90fbbc4465
FRE-4493: Complete API gateway review
...
✅ Approved Fastify API gateway implementation with:
- Request ID correlation middleware
- Multi-service routing (DarkWatch, VoicePrint, Correlation)
- CORS, Helmet security, health checks
- Docker containerization
Production gaps: rate limiting registration, JWT middleware, CORS whitelist
Artifacts:
- Review doc: packages/api/docs/FRE-4493-review.md
- Daily notes: memory/2026-05-02.md
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-02 01:51:23 -04:00
Senior Engineer
03276dde2d
Add cross-service alert correlation system FRE-4500
...
- Unified alert types (AlertSource, AlertCategory, CorrelationStatus, EntityType)
- NormalizedAlert and CorrelationGroup Prisma models
- AlertNormalizer for all 4 services (DarkWatch, SpamShield, VoicePrint, CallAnalysis)
- CorrelationEngine with temporal + entity-based correlation detection
- CorrelationService orchestrator with dashboard API
- Correlation API routes (/api/v1/correlation/*)
- Service emitters wired to DarkWatch, SpamShield, VoicePrint
- pnpm workspace config for monorepo
2026-05-02 01:10:44 -04:00
685fb57e53
Update daily notes with FRE-4520 Code Reviewer handoff
...
- Document reassignment to Code Reviewer (f274248f-c47e-4f79-98ad-45919d951aa0)
- Note completion timestamp and comment posted
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-01 20:07:00 -04:00
3663e5b80a
FRE-4517, FRE-4499: Complete SpamShield implementation and billing updates
...
- SpamFeedback table migration with timestamp index
- Real-time interception engine completion
- Billing service enhancements
- Classifier and rule engine updates
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-01 19:53:19 -04:00
3955b56e8d
Update daily notes with FRE-4520 security remediation status
...
- Document all 4 Medium and 2 Low severity fixes
- Note that issue is now in_review for Code Reviewer
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-01 19:45:28 -04:00
c490735ba2
FRE-4520: Fix security vulnerabilities in notification template system
...
- Fix HTML injection vulnerability with proper entity encoding
- Fix rate limit cleanup bug (count vs timestamp confusion)
- Add URL validation to prevent open redirect attacks
- Add expiration to in-memory deduplication entries
- Use Zod schema for config validation
- Add email format validation
All 29 tests passing. Ready for Code Reviewer final review.
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-01 19:35:22 -04:00
2a5c6f49a7
Add SpamFeedback table migration with timestamp index (FRE-4517)
...
Create migration to add SpamFeedback table with indexes on:
- userId (user relationship queries)
- phoneNumberHash (anonymized lookup)
- createdAt (time-based queries, requested in FRE-4517)
2026-05-01 18:43:39 -04:00
2241b97c81
FRE-4518: Replace hardcoded default score values with constants
...
- Created decision-engine.constants.ts with all scoring weights, thresholds, and behavioral scores
- Updated decision-engine.ts to import and use constants instead of inline values
- All 12 hardcoded values now have named, documented constants
- Pre-existing type errors are unrelated to this change
2026-05-01 18:02:28 -04:00
Senior Engineer
574bcf2264
FRE-4521 Implement Redis integration for rate limiting and deduplication
...
- Add ioredis dependency for Redis connection pooling
- Create RedisService singleton with connection management
- Add Redis config (url, dedupWindowSeconds) to notification.config.ts
- Implement NotificationService.checkRateLimit using Redis INCR+EXPIRE
- Implement NotificationService.deduplicateNotification using Redis SET/NX
- Add configurable rate limit windows and thresholds via env vars
- Add 29 unit tests covering Redis operations, rate limiting, and dedup
- All tests pass, TypeScript compiles cleanly for new files
2026-05-01 16:13:17 -04:00
7aed2d8b2b
FRE-4520: Add unit tests for notification template system
...
- 25 tests covering template resolution, localization fallback, variable substitution, caching, custom template registration, and edge cases
- Update package.json to use vitest for test execution
- All 25 tests passing
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-01 10:08:48 -04:00
8b30cad462
FRE-4499: Implement real-time SpamShield interception engine
...
Phase 1 & 2 complete: Carrier API integration, decision engine, and WebSocket alerts
## Carrier API Integration
- Carrier types interface for Twilio/Plivo/SIP
- Twilio carrier implementation with block/flag/allow operations
- Plivo carrier implementation with custom action headers
- Carrier factory for carrier management and health checks
## Decision Engine
- Multi-layer scoring: Reputation (40%), Rules (30%), Behavioral (20%), User History (10%)
- Thresholds: BLOCK >= 0.85, FLAG >= 0.60, ALLOW < 0.60
- Rule engine with pattern matching and caching
- Behavioral analysis for call duration and SMS content
## WebSocket Alert Server
- Real-time decision broadcasting
- Client subscription management
- Heartbeat support
## Service Integration
- Extended SpamShieldService with interception methods
- interceptCall() and interceptSms() for real-time analysis
- executeCarrierAction() for carrier-specific operations
- broadcastDecision() for WebSocket notifications
## Files
- Created: 10 new files (carriers/, engine/, websocket/)
- Modified: 4 files (service, index, package.json, plan)
TypeScript typecheck shows 27 errors (type-safety improvements only)
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-01 10:04:25 -04:00
3192d1a779
Fix JWT security issues in signaling and alert servers (FRE-4497)
...
- Replace custom JWT parser with jsonwebtoken library (timing-safe HMAC)
- Prefer Authorization header over URL query for token extraction
- Add jsonwebtoken + @types/jsonwebtoken to server dependencies
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-05-01 09:04:28 -04:00
ec4565f44c
Implement WebRTC real-time call analysis with security hardening (FRE-4497)
...
- signaling-server.ts: JWT auth, origin validation, JSON schema validation,
crypto.randomBytes peer IDs, message size limits, idle timeout, graceful shutdown
- alert-server.ts: JWT auth enabled by default, non-empty jwtSecret from env,
origin allowlist, per-subscriber callId filtering, bounded alert history with TTL,
alert cooldown, graceful shutdown with timeout
- call-analysis-engine.ts: Bounded eventBuffer/anomalyBuffer with FIFO eviction,
real quality metrics from signal properties, configurable buffer sizes
- audio-stream-capture.ts: Proper destroy() lifecycle with awaited stop(),
AudioWorklet support with ScriptProcessorNode fallback, bounded frame buffers
- Added ws dependency and server tsconfig
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-04-30 16:49:53 -04:00
19c5a951fe
billing
2026-04-30 11:07:38 -04:00
9fb5379b7a
Add tier-based scan scheduler and webhook triggers (FRE-4498)
...
- ScanScheduler: tier-based scheduling (BASIC=24h, PLUS=6h, PREMIUM=1h)
- WebhookHandler: HMAC-verified webhook ingestion with SCAN_TRIGGER support
- API routes: /scheduler and /webhooks endpoints under /api/v1/darkwatch
- Jobs: scheduled scan checker + webhook retry processor via BullMQ
- Schema: ScanSchedule, WebhookEvent models; ScanJob.scheduledBy field
- Types: ScheduleStatus, WebhookEventType, WebhookTriggerInput
- Tests: scheduler lifecycle + webhook signature/processing tests
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-04-30 10:57:56 -04:00
Senior Engineer
76d431e1ec
Add E.164 input validation for phone numbers (FRE-4506)
...
- Extract phone validation to reusable utility (src/utils/phone-validation.ts)
- Use libphonenumber-js for strict E.164 format validation
- Normalize accepted numbers to canonical E.164 format
- Add 22 comprehensive validation tests covering valid/invalid formats
- Update existing tests to use valid E.164 test numbers
- All 56 tests passing
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-04-29 21:22:23 -04:00
3ad030a412
Add circuit breaker for Hiya/Truecaller external APIs (FRE-4508)
...
- Implement CircuitBreaker class with CLOSED/OPEN/HALF_OPEN states
- Configurable failure threshold, success threshold, and timeout
- Fallback behavior when circuit opens (returns neutral 0.5 score)
- State change callbacks for monitoring and logging
- Comprehensive metrics tracking (executions, failures, successes, timestamps)
- Update SpamShieldService to use circuit breakers for both Hiya and Truecaller
- Add parallel API calls with graceful degradation
- Export circuit breaker types and service interfaces
- 32 unit tests covering circuit transitions, fallback, and service integration
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-04-29 19:07:54 -04:00
509259bcf2
for first push
2026-04-29 16:29:03 -04:00
Senior Engineer
218de3b03b
FRE-4471: Scaffold DarkWatch MVP — monorepo, schema, services, API routes, tests
...
- Turborepo monorepo structure (packages: api, db, types, jobs; services: darkwatch)
- Prisma schema: User, WatchListItem, Exposure, Alert, ScanJob models
- WatchListService: CRUD with normalization, dedup, tier-based limits
- HIBPService: API integration with severity scoring
- MatchingEngine: exact-match with content hash dedup
- AlertPipeline: dedup window, email notifications
- ScanService: orchestrates watch list -> HIBP -> match -> alert flow
- BullMQ job workers for scan and alert processing
- Fastify API routes: watchlist, exposures, alerts, scan
- Docker Compose: PostgreSQL 16 + Redis 7
- 15 unit tests passing
- Implementation plan document uploaded
2026-04-29 09:47:45 -04:00
f8f90502fa
Add ShieldAI technical architecture and implementation plan (FRE-4459)
...
- System overview with 3 core services: VoicePrint, DarkWatch, SpamShield
- Tech stack: TypeScript, Fastify, Next.js, PostgreSQL, Redis, Python/FastAPI
- Build vs buy decisions for each component
- 6-phase implementation timeline (24 weeks)
- Infrastructure, security, risk mitigation, and team estimates
- API surface definitions for all services
- Child issues created: FRE-4470 through FRE-4475
Co-Authored-By: Paperclip <noreply@paperclip.ing >
2026-04-28 13:58:27 -04:00
4f39829627
FRE-4414: Create ShieldAI product plan
...
- Defined 3-tier product structure (Basic/Plus/Premium)
- Product name: ShieldAI
- Features: Voice cloning protection, dark web scans, spam blocking, home title protection
- Created child issues FRE-4459 (CTO) and FRE-4460 (CMO) for execution
2026-04-28 12:56:43 -04:00
