plant-disease-id/tasks/hyper-specific-plant-disease-id/04-ml-model-integration.md

# 04. ML Model Loading, Inference Pipeline, and Confidence Scoring

meta:
  id: hyper-specific-plant-disease-id-04
  feature: hyper-specific-plant-disease-id
  priority: P1
  depends_on: [hyper-specific-plant-disease-id-02, hyper-specific-plant-disease-id-03]
  tags: [ml, inference, backend]

objective:
- Integrate a custom TensorFlow.js or ONNX-compatible plant disease classifier model into the Next.js API layer — handle model loading, batched inference, confidence scoring, and result ranking against the knowledge base.

deliverables:
- `lib/ml/model-loader.ts` — singleton model loader that lazy-loads the TF.js/ONNX model and caches it in memory
- `lib/ml/inference.ts` — `runInference(imageTensor: Float32Array): Promise<RawPrediction[]>` returning top-K class probabilities
- `lib/ml/labels.ts` — class label mapping (model output index → disease ID / "healthy" / "unknown")
- `lib/ml/confidence.ts` — softmax + confidence calibration, threshold logic (high ≥0.8, medium ≥0.5, low <0.5)
- `app/api/identify/route.ts` — `POST /api/identify` accepting `{ imageId }`, running full pipeline, returning ranked results with knowledge base enrichment
- `lib/api/identify.ts` — client helper to call the identify endpoint

steps:
1. Set up model storage and loading:
   - Place compiled model files (`model.json` + weight shards) in `public/models/plant-disease-classifier/`.
   - Implement `lib/ml/model-loader.ts` with lazy singleton pattern — loads model on first call, keeps in `globalThis` cache for subsequent calls.
   - Support both TensorFlow.js (`@tensorflow/tfjs-node` for server, `@tensorflow/tfjs` for client fallback) and ONNX Runtime (`onnxruntime-node`).
   - Graceful fallback: if no model file found, use a deterministic mock returning "model not loaded" with explanatory message.
2. Build `lib/ml/inference.ts`:
   - Accept normalized Float32Array of shape `[1, 3, 224, 224]`.
   - Run model forward pass.
   - Apply softmax to logits.
   - Return top-5 predictions with class indices and raw probabilities.
   - Measure inference time and attach to result.
3. Implement `lib/ml/labels.ts`:
   - Map model output index → disease ID string (e.g., `0 → "tomato-early-blight"`, `1 → "tomato-late-blight"`, …).
   - Include `"healthy"` class for each plant.
   - Include `"unknown"` as final catch-all class.
4. Implement `lib/ml/confidence.ts`:
   - `calibrateConfidence(rawProb: number): { adjusted: number, label: "high" | "medium" | "low" }`.
   - Apply threshold logic: only return predictions above `minConfidence` (configurable, default 0.15).
5. Build `app/api/identify/route.ts`:
   - Accept `{ imageId }` in request body.
   - Load image from `public/uploads/{imageId}` and preprocess (reuse pipeline from task 03).
   - Run inference.
   - Look up each top-K disease ID in knowledge base (from task 02) to enrich with name, description, symptoms, treatment.
   - Enrich with lookalike disease cross-references.
   - Return:
     ```json
     {
       "predictions": [
         {
           "diseaseId": "tomato-early-blight",
           "disease": { /* enriched from knowledge base */ },
           "confidence": { "raw": 0.87, "adjusted": 0.91, "label": "high" },
           "lookalikes": ["tomato-septoria-leaf-spot"]
         }
       ],
       "metadata": { "model": "plant-classifier-v1", "inferenceTimeMs": 320, "imageId": "..." }
     }
     ```
6. Add `lib/api/identify.ts` — a typed client-side function that `POST`s to `/api/identify` with the imageId and returns the typed response.
7. If no model file is present at build/runtime, return a deterministic mock response with a `"demo_mode": true` flag so the UI still works for development.

tests:
- **Unit:** `softmax([1, 2, 3])` sums to ~1.0.
- **Unit:** `calibrateConfidence(0.9)` returns label `"high"`.
- **Unit:** Top-5 extraction returns exactly 5 entries sorted descending.
- **Integration:** `POST /api/identify` with valid imageId returns 200 with predictions array.
- **Integration:** `POST /api/identify` with invalid imageId returns 404.
- **Integration:** Each prediction's `diseaseId` exists in knowledge base (cross-reference).
- **Load:** Inference completes under 3 seconds (Vercel serverless timeout).
  - Potential issue: serverless functions may have higher GPU latency.
  - Mitigation: consider using Vercel Serverless GPU or a Node.js function with ONNX Runtime CPU.
  - For initial deployment, CPU inference with MobileNet-derived model under 5MB is acceptable (<1s on V8).

acceptance_criteria:
- Model loads once and caches for subsequent requests.
- Inference returns top-5 predictions with confidence scores.
- Each prediction is enriched with full knowledge base data.
- Predictions include lookalike cross-references.
- Response includes inference timing metadata.
- Mock mode works when model file is absent.

validation:
```bash
# First upload an image
UPLOAD_RESP=$(curl -X POST -F "image=@test-assets/tomato-leaf.jpg" http://localhost:3000/api/upload)
IMAGE_ID=$(echo $UPLOAD_RESP | jq -r '.imageId')

# Then identify
curl -X POST -H "Content-Type: application/json" \
  -d "{\"imageId\": \"$IMAGE_ID\"}" \
  http://localhost:3000/api/identify | jq '.predictions[0].disease.name'
# → "Early Blight"
```

notes:
- A pre-trained MobileNetV2 fine-tuned on PlantVillage + augmented custom data is recommended — it's small (<10 MB), fast on CPU, and reasonably accurate.
- The actual model training process is OUT OF SCOPE for this task. This task assumes a trained model file is provided. Seed a placeholder warning if missing.
- If TF.js Node binding has issues, fall back to ONNX Runtime which is pure C++ and more stable on Lambda/Vercel.
- Consider Vercel's maximum serverless function duration (60s on Pro, 10s on Hobby) — keep model <10 MB and inference <3s.