Commit Graph

44 Commits

Author SHA1 Message Date
56016a6124 Fix P1 security findings for FRE-4806
- Add DD_API_KEY and DD_SITE to Zod validation schema (config.ts)
- Truncate API key before storing in user.id to prevent Sentry leak (auth.middleware.ts)
2026-05-12 12:42:42 -04:00
0f997b639f Fix P2/P3 review findings: DNR redirect format, runtime type guard, cache test setup 2026-05-11 13:54:51 -04:00
726aafef74 Fix dd-trace init timing in index.ts (FRE-4806)
Import datadog-init as first module to ensure dd-trace .init()
runs before any other imports, fixing P1 auto-instrumentation issue.
Removed redundant manual initDatadog/initSentry calls since
datadog-init.ts already invokes all three init functions.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-11 02:58:51 -04:00
31e0b39794 fix: address Code Reviewer findings for Datadog/Sentry integration FRE-4806
P1: Load dd-trace before other modules via datadog-init.ts entry point
P1: Batch all CloudWatch metrics into single PutMetricDataCommand per request
P2: Deduplicate warning logs with else-if for high latency vs error
P3: Add response.ok check to Datadog log forwarding fetch
P3: Update getSentryHub() to use getCurrentScope() for Sentry SDK 8.x

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-10 16:02:18 -04:00
a653c77959 FRE-5006: VoicePrint quality improvements
- P2-1: Consolidate mock ML logic to Python canonical source
- P2-2: Fix weak hashes with SHA-256
- P2-3: Parallelize batch processing with Promise.allSettled()
- P2-4: Add DI pattern support to services
- P2-5: Add structured logging utility
- P3-2: Persist batch jobId for result retrieval

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-10 12:06:16 -04:00
35e9f7e812 Fix 4 P1 and 2 P2 code review findings for FRE-4576
P1 fixes:
- Fix import paths in background/index.ts (./ -> ../lib/)
- Fix Promise-in-string bug in api-client.ts authenticate()
- Add missing background/service_worker key to manifest
- Copy HTML to public/ so Vite places them in dist

P2 fixes:
- Add notifications permission to manifest
- Make showWarningNotification async with proper await

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-10 11:53:25 -04:00
c1e4e8e404 Fix 3 P1 code review findings in VoicePrint job worker layer (FRE-5004)
- P1-4: Replace fragile relative import with dynamic import within job handler
- P1-5: Move worker creation to lazy createAnalysisWorker() function
- P1-8: Add maxRetryAttempts cap to Redis retryStrategy

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-10 11:38:09 -04:00
bc72a5b1cb Fix VoicePrint service-layer correctness bugs P1-1, P1-7, P2-2 (FRE-5002)
P1-1: Replace non-deterministic Math.random() with buffer-variance score
P1-7: Fix findSimilar result ordering by using Map instead of index zip
P2-2: Replace weak hashes with SHA-256 for both embedding and audio

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-10 11:17:23 -04:00
7b925c89bd Fix 3 Code Review findings on FRE-4574
- P2: Replace wget with curl for ECS health check (Alpine lacks wget)
- P2: Add AWS credentials step to CI terraform-plan job for S3 backend auth
- P3: Remove unused GitHub provider from infra/main.tf

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-10 07:09:39 -04:00
2d0611c2c9 Fix VoicePrint config validation & env safety (FRE-5005)
P3-1: Replace envSchema.parse() with safeParse() + default fallback to
avoid module-level crash when env vars are missing.

P3-3: Add fs.existsSync check on ECAPA_TDNN_MODEL_PATH at startup
with warning log when model path is missing.

P3-4: Add Zod strict() mode to env schema to catch typos in env
var names (extra keys now produce validation errors).

P1-6: Confirmed resolved - voiceprint.service.ts already imports
VoiceEnrollment/VoiceAnalysis from @shieldai/db (consolidated package).
2026-05-10 03:26:26 -04:00
Security Reviewer
4d30bacc53 Fix VoicePrint auth bypass & audio upload (FRE-5003)
P1-2: Add onRequest auth hook to reject anonymous requests on all 7
VoicePrint endpoints. Previously, the auth middleware always attached
a placeholder user (id='anonymous'), so per-route userId checks passed
for unauthenticated clients.

P1-3: Replace JSON body parsing with @fastify/multipart for POST
/endpoints (/enroll, /analyze, /batch). Fastify JSON parser cannot
produce Buffer from request.body; multipart/form-data is required
for audio file uploads. Added 50MB file size limit.
2026-05-10 03:20:31 -04:00
Senior Engineer
fb82dc68d7 Fix CORS origin trimming, unused import, and fragile error handling (FRE-4749)
- P2: Add .map(s => s.trim()) to trim whitespace from comma-separated ALLOWED_ORIGINS
- P3: Remove unused setSentryUser import from @shieldai/monitoring
- P3: Replace fragile string prefix matching with boolean isValidProtocol sentinel

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-10 02:58:02 -04:00
c7df40ac26 feat: integrate Datadog APM + Sentry error tracking with CloudWatch metrics FRE-4806
- Add CloudWatch metrics emitter (api_latency, api_requests, api_errors)
- Add request monitoring middleware for API (latency, error rate, throughput)
- Register error-handling, logging, and monitoring middleware in server.ts
- Add Datadog log forwarding via HTTP intake API
- Add application-level CloudWatch alarms for P99 latency, error rate, throughput
- Inject Datadog/Sentry env vars and secrets into ECS task definitions
- Add DD_API_KEY and SENTRY_DSN to ECS secrets
- Create CloudWatch log groups for datadog and sentry services
- Update .env.example with AWS_REGION and monitoring variables
- Add @aws-sdk/client-cloudwatch dependency to monitoring package

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-10 02:15:11 -04:00
57a206d7b3 Fix type errors in report routes (redundant parseInt, JsonValue cast) (FRE-4575)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-09 22:57:03 -04:00
2521c4e998 Add Protection Report Generator with HTML/PDF output and scheduled delivery (FRE-4575)
- Report service: data collection from all three engines, HTML rendering (Handlebars), PDF generation (pdfkit)
- REST API: /reports endpoints for generate, history, view, PDF download, scheduling
- BullMQ workers: queued report generation with retry, monthly/annual scheduler triggers
- DB: SecurityReport model with Prisma schema and type exports
- Email: report_ready template in shared-notifications
- All dependencies wired through existing packages

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-09 22:54:46 -04:00
de0ddac65d Add ShieldAI browser extension with phishing & spam detection (FRE-4576)
- Extension package: Manifest V3, background service worker, content scripts
- Phishing detection engine with heuristic analysis (typosquatting, entropy, TLD, brand impersonation)
- Local URL caching layer (Storage API) for <100ms cached lookups
- Popup UI with protection status, stats, and phishing report button
- Options page for settings management (blocked/allowed domains, feature toggles)
- Server-side extension routes: URL check, phishing report, auth, stats, exposure check
- Tier-aware feature gating (Basic/Plus/Premium)
- 25 passing tests for phishing detection heuristics
- Declarative net request rules for known phishing patterns
- DarkWatch integration for credential exposure checks
- Firefox compatibility layer via build modes

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-09 21:53:29 -04:00
Senior Engineer
a10ef7eb70 Harden CORS origin validation in production (FRE-4749)
- Add ALLOWED_ORIGINS env var with comma-separated origin list
- Validate origins at startup in production: reject wildcards, empty values,
  and malformed URLs (non-http/https protocol)
- Update both server entry points (server.ts, index.ts) to use getCorsOrigins()
- Development mode retains existing localhost fallback behavior

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-09 11:48:33 -04:00
f2593c1e67 use crypto package instead 2026-05-03 22:44:48 -04:00
a4684e9121 Fix SMS classifier test mock: add defaultScores and metadataLimits exports (FRE-4509)
The test mock for spamshield.config was missing defaultScores and
metadataLimits exports that are imported by spamshield.service.ts,
causing 8 tests to fail with 'No defaultScores export is defined'.
2026-05-02 20:23:29 -04:00
Senior Engineer
91e4985a8e FRE-4474 Phase 5: Verify and resolve security review findings for SpamShield and Cross-Service Correlation
- FRE-4499 (SpamShield): Verified 6 security fixes (2 High, 4 Medium)
  - S01: Pre-compiled regex in RuleEngine (ReDoS fix)
  - S02: SmsClassifier accepts senderPhoneNumber context
  - S03: AlertServer JWT auth + origin validation
  - S04: SHA-256 phone hashing (PII protection)
  - S05: DecisionEngine timeout enforcement via Promise.race
  - S06: CarrierFactory.getAllCarriers properly async/await

- FRE-4500 (Correlation): Verified 7 security fixes (2 Critical, 2 High, 2 Medium, 1 Low)
  - C1: Ingest endpoints auth via request.user.id
  - C2: IDOR protection on group endpoints (userId filter)
  - H3: JWT middleware registered in server.ts
  - H4: Fastify schema validation on all routes
  - M6: Payload sanitization with depth limit and circular ref detection
  - L7: CORS origin restricted to env var

- Resolved liveness incidents FRE-4652 and FRE-4654
- All Phase 5 child issues now complete
2026-05-02 18:36:29 -04:00
0afdf8b6e8 FRE-4500: Fix security review findings (Critical/High/Medium/Low)
- Critical #1: Add auth check to ingest endpoints (use request.user.id)
- Critical #2: Add IDOR protection on group endpoints (userId ownership)
- High #3: Register auth middleware in server.ts (populates request.user)
- High #4: Add Fastify schema validation to all route handlers
- Medium #5: Add NormalizedAlert/CorrelationGroup models to Prisma schema
- Medium #6: Sanitize payload storage in normalizer (depth limit, circular ref)
- Low #7: Restrict CORS origins (use CORS_ORIGIN env var)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-02 16:40:01 -04:00
24bc9c235f Consolidate @shieldai/db and @shieldsai/shared-db packages (FRE-4603)
- Merged singleton pattern + type exports from shared-db
- Kept FieldEncryptionService from original db package
- Upgraded to Prisma v6.2.0 (newer version)
- Adopted shared-db's complete schema for multi-service platform
- Updated 17 consumer imports across darkwatch, voiceprint, jobs, api
- Standardized on @shieldai/db namespace

Files changed:
- packages/db/package.json (v0.1.0 → v0.2.0)
- packages/db/src/index.ts (consolidated exports)
- packages/db/prisma/schema.prisma (merged schema)
- packages/db/prisma/seed.ts (updated for new schema)
- 17 consumer files updated

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-02 15:06:02 -04:00
93ff4885ee Add integration tests README documentation (FRE-4522)
Documentation for integration test suite including:
- Test file descriptions and coverage
- External provider mock configuration
- Running tests commands
- CI integration requirements
- Environment variables needed
- Test strategy and error scenarios
2026-05-02 13:23:12 -04:00
67622a2f11 Add integration tests for notification services (FRE-4522)
Comprehensive integration test suite for notification services:
- EmailService integration tests (Resend provider)
- SMSService integration tests (Twilio provider)
- PushService integration tests (FCM/APNs providers)
- NotificationService integration tests (orchestration layer)

Test coverage includes:
- Successful notification delivery
- Error handling (API errors, network timeouts, invalid inputs)
- Rate limiting enforcement
- Batch operations with partial failures
- Notification preferences and deduplication
- Template-based email sending
- Metadata and attachment handling

Total: ~1400 lines across 4 test files
2026-05-02 13:22:41 -04:00
bdf8ad30b6 Apply security remediations for FRE-4498 (FRE-4612)
Security findings from April 30 review were claimed fixed but never committed.
Applied all remediations:

HIGH:
- WebhookHandler: fail fast when DARKWATCH_WEBHOOK_SECRET missing instead of defaulting to hardcoded secret
- field-encryption.service: require PII_ENCRYPTION_KEY at startup instead of defaulting

MEDIUM:
- WebhookHandler: make signature required (was optional, accepted unsigned events)
- WebhookHandler: reject unknown event types instead of silently defaulting to SCAN_TRIGGER
- scheduler.routes + webhook.routes: add ownership checks on /:userId endpoints (IDOR)

LOW:
- webhook.routes: generic error responses, full error logged server-side

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-02 13:03:28 -04:00
f34adc5e82 Add null checks in feedback processing pipeline (FRE-4514)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-02 13:01:02 -04:00
e704a9074a FRE-4533: Merge apps/{api,web,mobile} and shared-db into ShieldAI repo
Merge FrenoCorp apps into ShieldAI packages/:
- packages/api: merged routes (notifications), middleware (auth, rate-limit, error, logging), config, services (darkwatch, spamshield, voiceprint), tests
- packages/web: new SolidJS web app stub
- packages/mobile: new SolidJS mobile app stub
- packages/shared-db: new Prisma DB package (separate from existing packages/db)
- pnpm-workspace.yaml: restored (apps/* removed, already covered by packages/*)

Next: reconcile packages/shared-db with packages/db, and fix server.ts correlationRoutes import
2026-05-02 10:19:11 -04:00
1197fe48f7 FRE-4533: Merge apps/{api,web,mobile} and shared-db into ShieldAI repo
- Copy apps/api (Fastify server with spamshield/voiceprint/darkwatch services)
- Copy apps/web (SolidJS web app)
- Copy apps/mobile (SolidJS mobile app)
- Copy packages/shared-db (Prisma schema/models)
- Add apps/* to pnpm-workspace.yaml
2026-05-02 10:16:18 -04:00
1e42c4a5c2 FRE-4529: Transfer ShieldAI code from FrenoCorp repo
Transferred ShieldAI-related files mistakenly placed in ~/code/FrenoCorp:
- Services: spamshield (feature-flags, audit-logger, error-handler), voiceprint (config, service, feature-flags), darkwatch (pipeline, scan, scheduler, watchlist, webhook)
- Packages: shared-analytics, shared-auth, shared-ui, shared-utils (new); shared-billing, jobs supplemented with unique FC files
- Server: alerts (FC version newer), routes (spamshield, darkwatch, voiceprint)
- Config: turbo.json, tsconfig.base.json, vite/vitest configs, drizzle, Dockerfile
- VoicePrint ML service
- Examples

Pending: apps/{api,web,mobile}/ structured merge, shared-db/db mapping

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-02 10:13:13 -04:00
8687868632 Add request ID validation and CSPRNG fallback (FRE-4516)
- Max-length guard (256 chars) on incoming request IDs to prevent log bloat
- Format whitelist (alphanumeric, hyphen, underscore) to prevent log injection
- Replace Math.random() with crypto.randomBytes in fallback for CSPRNG
2026-05-02 09:43:13 -04:00
b01b79d02a Add ReDoS validation for SpamRule.pattern field (FRE-4512)
- Create regex-validation utility with ReDoS detection (nested quantifiers,
  overlapping alternations, complexity limits)
- Add @db.VarChar(500) constraint on pattern field in Prisma schema
- Integrate validation in rule-engine at load time and evaluation time
- Add 46 unit tests covering syntax, ReDoS patterns, complexity, edge cases

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-02 07:23:39 -04:00
90fbbc4465 FRE-4493: Complete API gateway review
 Approved Fastify API gateway implementation with:
- Request ID correlation middleware
- Multi-service routing (DarkWatch, VoicePrint, Correlation)
- CORS, Helmet security, health checks
- Docker containerization

Production gaps: rate limiting registration, JWT middleware, CORS whitelist

Artifacts:
- Review doc: packages/api/docs/FRE-4493-review.md
- Daily notes: memory/2026-05-02.md

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-02 01:51:23 -04:00
Senior Engineer
03276dde2d Add cross-service alert correlation system FRE-4500
- Unified alert types (AlertSource, AlertCategory, CorrelationStatus, EntityType)
- NormalizedAlert and CorrelationGroup Prisma models
- AlertNormalizer for all 4 services (DarkWatch, SpamShield, VoicePrint, CallAnalysis)
- CorrelationEngine with temporal + entity-based correlation detection
- CorrelationService orchestrator with dashboard API
- Correlation API routes (/api/v1/correlation/*)
- Service emitters wired to DarkWatch, SpamShield, VoicePrint
- pnpm workspace config for monorepo
2026-05-02 01:10:44 -04:00
3663e5b80a FRE-4517, FRE-4499: Complete SpamShield implementation and billing updates
- SpamFeedback table migration with timestamp index
- Real-time interception engine completion
- Billing service enhancements
- Classifier and rule engine updates

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-01 19:53:19 -04:00
c490735ba2 FRE-4520: Fix security vulnerabilities in notification template system
- Fix HTML injection vulnerability with proper entity encoding
- Fix rate limit cleanup bug (count vs timestamp confusion)
- Add URL validation to prevent open redirect attacks
- Add expiration to in-memory deduplication entries
- Use Zod schema for config validation
- Add email format validation

All 29 tests passing. Ready for Code Reviewer final review.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-01 19:35:22 -04:00
2a5c6f49a7 Add SpamFeedback table migration with timestamp index (FRE-4517)
Create migration to add SpamFeedback table with indexes on:
- userId (user relationship queries)
- phoneNumberHash (anonymized lookup)
- createdAt (time-based queries, requested in FRE-4517)
2026-05-01 18:43:39 -04:00
Senior Engineer
574bcf2264 FRE-4521 Implement Redis integration for rate limiting and deduplication
- Add ioredis dependency for Redis connection pooling
- Create RedisService singleton with connection management
- Add Redis config (url, dedupWindowSeconds) to notification.config.ts
- Implement NotificationService.checkRateLimit using Redis INCR+EXPIRE
- Implement NotificationService.deduplicateNotification using Redis SET/NX
- Add configurable rate limit windows and thresholds via env vars
- Add 29 unit tests covering Redis operations, rate limiting, and dedup
- All tests pass, TypeScript compiles cleanly for new files
2026-05-01 16:13:17 -04:00
7aed2d8b2b FRE-4520: Add unit tests for notification template system
- 25 tests covering template resolution, localization fallback, variable substitution, caching, custom template registration, and edge cases
- Update package.json to use vitest for test execution
- All 25 tests passing

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-01 10:08:48 -04:00
8b30cad462 FRE-4499: Implement real-time SpamShield interception engine
Phase 1 & 2 complete: Carrier API integration, decision engine, and WebSocket alerts

## Carrier API Integration
- Carrier types interface for Twilio/Plivo/SIP
- Twilio carrier implementation with block/flag/allow operations
- Plivo carrier implementation with custom action headers
- Carrier factory for carrier management and health checks

## Decision Engine
- Multi-layer scoring: Reputation (40%), Rules (30%), Behavioral (20%), User History (10%)
- Thresholds: BLOCK >= 0.85, FLAG >= 0.60, ALLOW < 0.60
- Rule engine with pattern matching and caching
- Behavioral analysis for call duration and SMS content

## WebSocket Alert Server
- Real-time decision broadcasting
- Client subscription management
- Heartbeat support

## Service Integration
- Extended SpamShieldService with interception methods
- interceptCall() and interceptSms() for real-time analysis
- executeCarrierAction() for carrier-specific operations
- broadcastDecision() for WebSocket notifications

## Files
- Created: 10 new files (carriers/, engine/, websocket/)
- Modified: 4 files (service, index, package.json, plan)

TypeScript typecheck shows 27 errors (type-safety improvements only)

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-01 10:04:25 -04:00
ec4565f44c Implement WebRTC real-time call analysis with security hardening (FRE-4497)
- signaling-server.ts: JWT auth, origin validation, JSON schema validation,
  crypto.randomBytes peer IDs, message size limits, idle timeout, graceful shutdown
- alert-server.ts: JWT auth enabled by default, non-empty jwtSecret from env,
  origin allowlist, per-subscriber callId filtering, bounded alert history with TTL,
  alert cooldown, graceful shutdown with timeout
- call-analysis-engine.ts: Bounded eventBuffer/anomalyBuffer with FIFO eviction,
  real quality metrics from signal properties, configurable buffer sizes
- audio-stream-capture.ts: Proper destroy() lifecycle with awaited stop(),
  AudioWorklet support with ScriptProcessorNode fallback, bounded frame buffers
- Added ws dependency and server tsconfig

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-30 16:49:53 -04:00
19c5a951fe billing 2026-04-30 11:07:38 -04:00
9fb5379b7a Add tier-based scan scheduler and webhook triggers (FRE-4498)
- ScanScheduler: tier-based scheduling (BASIC=24h, PLUS=6h, PREMIUM=1h)
- WebhookHandler: HMAC-verified webhook ingestion with SCAN_TRIGGER support
- API routes: /scheduler and /webhooks endpoints under /api/v1/darkwatch
- Jobs: scheduled scan checker + webhook retry processor via BullMQ
- Schema: ScanSchedule, WebhookEvent models; ScanJob.scheduledBy field
- Types: ScheduleStatus, WebhookEventType, WebhookTriggerInput
- Tests: scheduler lifecycle + webhook signature/processing tests

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-30 10:57:56 -04:00
509259bcf2 for first push 2026-04-29 16:29:03 -04:00
Senior Engineer
218de3b03b FRE-4471: Scaffold DarkWatch MVP — monorepo, schema, services, API routes, tests
- Turborepo monorepo structure (packages: api, db, types, jobs; services: darkwatch)
- Prisma schema: User, WatchListItem, Exposure, Alert, ScanJob models
- WatchListService: CRUD with normalization, dedup, tier-based limits
- HIBPService: API integration with severity scoring
- MatchingEngine: exact-match with content hash dedup
- AlertPipeline: dedup window, email notifications
- ScanService: orchestrates watch list -> HIBP -> match -> alert flow
- BullMQ job workers for scan and alert processing
- Fastify API routes: watchlist, exposures, alerts, scan
- Docker Compose: PostgreSQL 16 + Redis 7
- 15 unit tests passing
- Implementation plan document uploaded
2026-04-29 09:47:45 -04:00