Mike/Kordant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

clear old assets, new ci/cd flow

Browse Source
This commit is contained in:
Michael Freno
2026-05-26 11:54:41 -04:00
parent 82815009c9
commit 72609755f8
87 changed files with 4132 additions and 7158 deletions
Download Patch File Download Diff File Expand all files Collapse all files

348
README.md
View File

@@ -2,7 +2,7 @@
**Multi-layered consumer identity protection against predatory AI-driven scams.**
Kordant combines three detection engines — voice cloning detection, dark web monitoring, and real-time spam classification — to give consumers proactive defense against modern identity fraud.
Kordant combines five service domains — voice cloning detection, dark web monitoring, spam classification, property monitoring, and data broker removal — into a unified platform with web, iOS, and Android apps.
---
@@ -15,97 +15,74 @@ Kordant flips the model. We detect the scam _as it happens_:
- **VoicePrint** analyzes inbound calls in real time to flag synthetic AI-generated voices before you're socially engineered.
- **DarkWatch** continuously monitors dark web forums, breach databases, and data broker caches — notifying you the moment your credentials, phone, or SSN surface.
- **SpamShield** intercepts and classifies spam calls and SMS at the network level, blocking threats before they reach your phone.
Backed by ML models (ECAPA-TDNN, BERT) and a real-time alert pipeline, Kordant gives consumers enterprise-grade threat detection for their personal life.
- **HomeTitle** monitors county deed records for unauthorized ownership changes, liens, and fraud.
- **RemoveBrokers** automates data broker opt-out requests to remove your personal info from people-search sites.
---
## Architecture Overview
## Architecture
Unified SolidStart monolith with tRPC, Drizzle ORM, and native mobile apps.
```
┌─────────────────────────────────────────────────────────┐
│ Clients │
│ Mobile (Expo/RN) │ Web (SolidJS) │ Browser Extension │
└──────────┬──────────────────────────────┬───────────────┘
│ REST + WebSocket │
▼ ▼
┌──────────────────────┐ ┌──────────────────────────────┐
│ API Gateway │ │ WebSocket Alert Server │
│ (Fastify 5) │ │ (Real-time push) │
└──────┬───────┬───────┘ └──────────────────────────────┘
│ │
▼ ▼
┌──────────┐ ┌─────────────────────────────────────────────┐
│ Auth │ │ Microservices │
│(NextAuth)│ │ VoicePrint │ DarkWatch │ SpamShield │
└──────────┘ │ HomeTitle │ RemoveBrokers │
└──────────┬──────────┬───────────────────────┘
│ │
┌────────▼──────────▼────────┐
│ Background Workers │
│ (BullMQ + Redis) │
└────────┬───────────────────┘
│
┌────────▼────────┐
│ PostgreSQL │
│ + Redis │
└─────────────────┘
┌──────────────────────────────────────────────────────────────┐
│ Clients │
│ Web (SolidStart) │ iOS (SwiftUI) │ Android (Compose) │ Ext │
└────────────────────┬─────────────────────────────────────────┘
│ tRPC (HTTP/WS)
▼
┌──────────────────────────────────────────────────────────────┐
│ web/ (SolidStart) │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Frontend (SolidStart + Tailwind) │ │
│ │ Landing │ Auth │ Dashboard │ Service Pages │ │
│ └─────────────────────┬───────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────▼───────────────────────────────────┐ │
│ │ Backend (tRPC routers) │ │
│ │ auth │ user │ family │ billing │ darkwatch │ │ │
│ │ voiceprint │ spamshield │ hometitle │ removebrokers │ │ │
│ │ alerts │ reports │ notifications │ correlation │ │
│ └─────────────────────┬───────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────▼───────────────────────────────────┐ │
│ │ Background Jobs (scheduler + workers) │ │
│ └─────────────────────┬───────────────────────────────────┘ │
└────────────────────────┼──────────────────────────────────────┘
│
┌────────▼────────┐
│ PostgreSQL │
│ + Redis │
└─────────────────┘
```
---
## Features & Implementation Status
## Directory Structure
| Feature | Service |Status | Notes |
|---------|---------|--------|-------|
| Voice enrollment & profile management | VoicePrint | ✅ Done | Register family voice profiles |
| Audio preprocessing (VAD, noise reduction) | VoicePrint | ✅ Done | WebRTC VAD + RNNoise |
| Synthetic voice detection (ECAPA-TDNN) | VoicePrint | ✅ Done | FAISS vector index for matching |
| Real-time streaming audio analysis | VoicePrint | ✅ Done | WebSocket-based |
| Batch audio analysis | VoicePrint | ✅ Done | Configurable confidence thresholds per tier |
| HIBP breach checking | DarkWatch | ✅ Done | Email + password breach lookup |
| Dark web multi-source scanning | DarkWatch | ✅ Done | HIBP, SecurityTrails, Censys, Shodan, forums |
| Watch list management | DarkWatch | ✅ Done | Emails, phones, SSN (hashed) |
| Scheduled + real-time scanning | DarkWatch | ✅ Done | Tier-based frequency |
| Fuzzy matching engine | DarkWatch | ✅ Done | Levenshtein + exact matching |
| Severity-scored alert pipeline | DarkWatch | ✅ Done | Dedup pipeline |
| PDF report generation | DarkWatch | ✅ Done | Handlebars + PDFKit |
| Number reputation (Hiya/Truecaller) | SpamShield | ✅ Done | Circuit breaker pattern |
| SMS classification (BERT) | SpamShield | ✅ Done | ML-based spam detection |
| Call analysis rule engine | SpamShield | ✅ Done | Multi-layer scoring |
| Real-time carrier interception | SpamShield | ⏳ In Progress | Twilio/Plivo integration |
| Real-time WebSocket alerts | SpamShield | ✅ Done | Alert broadcasting |
| User feedback loop (FP/FN) | SpamShield | ✅ Done | Metadata validation |
| Phone validation (E.164) | SpamShield | ✅ Done | Normalization |
| Audit logging | SpamShield | ✅ Done | All decisions logged |
| SpamShield rate limiting | SpamShield | ⏳ In Progress | Per-endpoint + global |
| SpamShield route optimization | SpamShield | ⏳ In Progress | Route consolidation |
| Feature flags | All | ✅ Done | Env-variable toggles |
| Property record matching | HomeTitle | ✅ Done | Fuzzy string matching |
| Change detection (ownership, liens) | HomeTitle | ✅ Done | County deed scanning |
| Watchlist management | HomeTitle | ✅ Done | |
| Scheduled county deed scanning | HomeTitle | ✅ Done | |
| Alert pipeline | HomeTitle | ✅ Done | Severity classification |
| Data broker removal requests | RemoveBrokers | ✅ Done | |
| Broker API integration | RemoveBrokers | ✅ Done | With caching |
| User auth (JWT, RBAC) | Shared | ✅ Done | NextAuth.js |
| Family group management | Shared | ✅ Done | |
| Stripe subscriptions & billing | Shared | ✅ Done | Tier-based feature gating |
| Email (Resend) | Shared | ✅ Done | Transactional + marketing |
| Push notifications (FCM/APNs) | Shared | ✅ Done | Android + iOS |
| SMS (Twilio) | Shared | ✅ Done | |
| Mixpanel analytics (30+ events) | Shared | ✅ Done | KPI tracking |
| Datadog APM + Sentry | Shared | ✅ Done | Full observability |
| Cross-service event correlation | Shared | ✅ Done | Alert correlation engine |
| Browser extension (MV3) | Extension | ✅ Done | Phishing detection |
| Mobile app (Expo RN) | Mobile | ✅ Done | iOS + Android |
| Shared UI component library | Shared UI | ✅ Done | SolidJS |
| CI/CD pipelines | DevOps | ✅ Done | GitHub Actions |
| Terraform infrastructure | DevOps | ✅ Done | AWS ECS, RDS, ElastiCache |
| Load testing (k6) | DevOps | ✅ Done | VoicePrint + DarkWatch |
| Docker + Compose | DevOps | ✅ Done | Dev + prod configs |
| Integration tests | QA | ⏳ In Progress | Coverage expanding |
| Rate limit tests | QA | ⏳ In Progress | |
```
kordant/
├── web/ # SolidStart monolith (frontend + tRPC backend)
│ ├── src/
│ │ ├── routes/ # Page routes (landing, auth, dashboard)
│ │ ├── components/ # UI components (primitives, layouts, widgets)
│ │ ├── server/ # tRPC routers, services, database, jobs
│ │ ├── hooks/ # Solid hooks
│ │ ├── lib/ # Shared utilities
│ │ └── theme/ # Generated design tokens
│ └── Dockerfile
├── browser-ext/ # Chrome Manifest V3 extension
├── iOS/Kordant/ # SwiftUI native iOS app
├── android/ # Jetpack Compose native Android app
├── design-tokens/ # Brand tokens (single source of truth)
├── docs/ # Brand guidelines, runbooks
├── scripts/ # Build and deployment scripts
├── tasks/ # Project task tracking
├── docker-compose.yml # Local dev (web + postgres + redis)
├── docker-compose.prod.yml # Production deployment
└── .github/workflows/ # CI/CD pipelines
```
---
@@ -113,25 +90,24 @@ Backed by ML models (ECAPA-TDNN, BERT) and a real-time alert pipeline, Kordant g
| Layer | Technology |
|-------|-----------|
| **Language** | TypeScript (Node.js ≥20) |
| **API** | Fastify 5 (CORS, Helmet, rate-limit, Swagger, multipart) |
| **Frontend** | SolidJS + Vite |
| **Mobile** | React Native / Expo SDK 51 |
| **Language** | TypeScript (Node.js ≥22) |
| **Framework** | SolidStart (SSR + API server) |
| **API** | tRPC (type-safe RPC) |
| **Database** | PostgreSQL 16 (Drizzle ORM) |
| **Cache / Queue** | Redis 7 |
| **Styling** | Tailwind CSS + CSS custom properties |
| **Mobile iOS** | SwiftUI (native) |
| **Mobile Android** | Jetpack Compose (native) |
| **Extension** | Chrome Manifest V3 |
| **Databases** | PostgreSQL 15/16 (Prisma ORM) + Turso/SQLite (Drizzle) |
| **Cache / Queue** | Redis + BullMQ |
| **Auth** | NextAuth.js + JWT |
| **Auth** | JWT + session cookies |
| **Billing** | Stripe |
| **Email** | Resend |
| **Push** | Firebase Cloud Messaging + APNs |
| **SMS** | Twilio |
| **Analytics** | Mixpanel / Segment |
| **Monitoring** | Datadog APM + Sentry |
| **ML Models** | ECAPA-TDNN (voice), BERT (SMS), FAISS (vector index) |
| **Infrastructure** | Terraform (AWS ECS Fargate, RDS, ElastiCache, S3, ALB) |
| **CI/CD** | GitHub Actions |
| **Monorepo** | pnpm workspaces + Turborepo |
| **Testing** | Vitest, Jest, k6 |
| **Design Tokens** | JSON → generated TS/Swift/XML |
| **CI/CD** | Vercel (web) + Docker (scheduler) |
| **Monorepo** | pnpm workspaces |
| **Testing** | Vitest |
---
@@ -139,160 +115,96 @@ Backed by ML models (ECAPA-TDNN, BERT) and a real-time alert pipeline, Kordant g
### Prerequisites
- Node.js >= 20.0.0
- Node.js >= 22.0.0
- pnpm >= 9.0.0
- Docker & Docker Compose
### Setup
```bash
# Install all dependencies
# Install dependencies
pnpm install
# Start local infrastructure (Postgres, Redis, Mailhog)
docker compose up -d
# Copy environment variables
cp .env.example .env
# Edit .env with your Turso credentials
# DATABASE_URL=libsql://your-db.turso.io
# DATABASE_AUTH_TOKEN=your-token
# Run database migrations
pnpm db:migrate
# Start all development servers
# Start development server
pnpm dev
```
This launches the API server, all microservices, and the web frontend concurrently via Turborepo.
The web app runs at `http://localhost:3000`.
---
## Building
## Design Tokens
All platforms (web, iOS, Android) share the same design tokens defined in `design-tokens/`:
```
design-tokens/
├── colors.json # Brand, semantic, background, text, border colors
├── typography.json # Font family, scale, weights
├── spacing.json # 4px-based spacing scale
├── shadows.json # Elevation definitions
└── radius.json # Border radius scale
```
Generate platform-specific code:
```bash
# Build all packages and services
pnpm build
# Build individual Docker images
docker build -f packages/api/Dockerfile -t kordant-api .
docker build -f services/spamshield/Dockerfile -t kordant-spamshield .
docker build -f services/darkwatch/Dockerfile -t kordant-darkwatch .
docker build -f services/voiceprint/Dockerfile -t kordant-voiceprint .
node scripts/generate-tokens.mjs
```
This produces:
- `web/src/theme/tokens.ts` — TypeScript constants
- `iOS/Kordant/Theme/GeneratedTokens.swift` — SwiftUI colors + spacing
- `android/.../res/values/generated_tokens.xml` — Android resources
See `docs/BRAND_GUIDELINES.md` for full brand guidelines.
---
## Testing
## Deployment
| Component | Platform | Notes |
|-----------|----------|-------|
| Web app | Vercel | git push auto-deploys |
| Database | Turso (managed) | run `pnpm db:migrate` to apply schema changes |
| Background jobs | Docker on `pan` | scheduler + Redis containers |
### Setting up the Scheduler (pan server)
The background job scheduler (dark web scans, reports, etc.) runs as Docker containers on your `pan` server. Run the setup script from anywhere:
```bash
# Run all tests
pnpm test
# From dev machine (SSHs into pan):
bash scripts/setup-pan.sh
# With coverage
pnpm test:coverage
# Individual service tests
pnpm test --filter @kordant/spamshield
pnpm test --filter @kordant/darkwatch
pnpm test --filter @kordant/voiceprint
pnpm test --filter @kordant/hometitle
# Integration & E2E
cd packages/integration-tests && pnpm test
cd packages/integration-tests && pnpm test:e2e
# Load tests (requires k6)
cd scripts/load-test && ./run-all.sh
# Or directly on pan:
sudo bash scripts/setup-pan.sh
```
---
This installs Docker + Compose, clones the repo to `/opt/kordant`, creates a systemd service, and starts the scheduler. See the script for details and the optional Gitea post-receive hook for auto-deploy on push.
## Production
### Scripts
### Docker Compose
```bash
docker compose -f docker-compose.prod.yml up -d
```
### CI/CD Pipeline (GitHub Actions)
| Event | Deploy To |
|-------|-----------|
| Push to `main` | Staging |
| GitHub Release created | Production |
Pipeline stages: `lint` → `typecheck` → `test` → `Docker build` → `push to GHCR` → `Terraform apply` → `ECS deploy` → `health check` → auto-rollback on failure.
### Infrastructure
All infrastructure lives in `infra/` and is managed with Terraform:
- **Compute**: AWS ECS Fargate (API + services + workers)
- **Database**: RDS PostgreSQL 15/16
- **Cache**: ElastiCache Redis
- **Storage**: S3 (reports, audio samples)
- **Networking**: VPC, ALB, security groups
- **Observability**: CloudWatch + Datadog
- **Secrets**: AWS Secrets Manager
See `infra/README.md` and `infra/ROLLBACK.md` for detailed operational runbooks.
---
## Project Structure
```
kordant/
├── packages/ # Shared libraries (20 packages)
│ ├── api/ # Fastify API server
│ ├── core/ # Core shared logic
│ ├── db/ # Prisma schemas (v6)
│ ├── shared-db/ # Prisma schemas (v5)
│ ├── shared-auth/ # NextAuth.js
│ ├── shared-billing/ # Stripe subscriptions
│ ├── shared-notifications/ # Email, Push, SMS
│ ├── shared-analytics/ # Mixpanel/Segment
│ ├── shared-ui/ # SolidJS components
│ ├── shared-utils/ # Utilities
│ ├── types/ # Shared TypeScript types
│ ├── mobile/ # React Native / Expo app
│ ├── extension/ # Browser extension (MV3)
│ ├── jobs/ # BullMQ workers
│ ├── monitoring/ # Datadog + Sentry
│ ├── report/ # PDF generation
│ ├── correlation/ # Event correlation
│ ├── mobile-api-client/ # RN API client
│ └── integration-tests/ # E2E tests
├── services/ # Microservices (5)
│ ├── voiceprint/ # Voice cloning detection
│ ├── darkwatch/ # Dark web monitoring
│ ├── spamshield/ # Spam call/SMS blocking
│ ├── hometitle/ # Home title fraud
│ └── removebrokers/ # Data broker removal
├── infra/ # Terraform (AWS)
├── docs/ # Documentation
├── plans/ # Product & technical plans
├── scripts/ # Utility scripts
├── load-tests/ # k6 load test scripts
├── assets/ # Ad creative assets
└── server/ # Legacy WebSocket server
```
---
## Documentation
| Document | Location |
|----------|----------|
| Product Plan | `plans/Kordant-product-plan.md` |
| Technical Architecture | `plans/Kordant-technical-architecture.md` |
| Infrastructure | `infra/README.md` |
| Rollback Runbook | `infra/ROLLBACK.md` |
| Stripe Integration | `docs/STRIPE_INTEGRATION.md` |
| Push Notifications | `docs/PUSH_NOTIFICATIONS_SETUP.md` |
| Mobile Push Integration | `docs/MOBILE_PUSH_INTEGRATION.md` |
| Mixpanel Analytics | `docs/MIXPANEL_ANALYTICS.md` |
| Code Review Workflow | `kordant-workflow.md` |
| Command | Description |
|---------|-------------|
| `pnpm dev` | Start web dev server |
| `pnpm build` | Build web app for production |
| `pnpm test` | Run web tests |
| `pnpm lint` | Lint web app |
| `pnpm db:migrate` | Run database migrations |
| `pnpm db:seed` | Seed database with test data |
| `pnpm build:ext` | Build browser extension |
| `node scripts/generate-tokens.mjs` | Generate platform design tokens |
| `bash scripts/setup-pan.sh` | Deploy scheduler to pan server |
---