81 lines
3.5 KiB
Markdown
81 lines
3.5 KiB
Markdown
# 04. Enhanced API Route for Multi-Image, Species-Aware Identification
|
|
|
|
meta:
|
|
id: multi-image-user-feedback-04
|
|
feature: multi-image-user-feedback
|
|
priority: P1
|
|
depends_on: [multi-image-user-feedback-01, multi-image-user-feedback-02, multi-image-user-feedback-03]
|
|
tags: [api, backend]
|
|
|
|
objective:
|
|
|
|
- Update the `/api/identify` route to accept optional `options` (secondImageId, userSpecies), run ensemble inference when multiple images provided, apply species-constrained softmax when species is known, and return top-5 species+disease combo predictions with confidence metadata.
|
|
|
|
deliverables:
|
|
|
|
- `src/app/api/identify/route.ts` — updated route handler
|
|
|
|
steps:
|
|
|
|
1. Update the request parsing to accept `options?: IdentifyOptions` alongside `imageId`:
|
|
|
|
```typescript
|
|
const { imageId, options } = body;
|
|
const secondImageId = options?.secondImageId;
|
|
const userSpecies = options?.userSpecies;
|
|
```
|
|
|
|
2. Update image loading to support optional second image:
|
|
- Load first image tensor (existing logic)
|
|
- If `secondImageId` provided, load and preprocess that image too
|
|
- Validate both images exist before inference
|
|
|
|
3. Update inference logic to use the new pipeline:
|
|
- If 2 images provided → call `runEnsembleInference(tensors, topK=10)`
|
|
- If `userSpecies` provided → get species class range, call `runSpeciesConstrainedInference` or `runEnsembleSpeciesConstrained`
|
|
- If 2 images + species → use full ensemble+constrained
|
|
- If 1 image + no species → use existing single-inference path (backward compatible)
|
|
- Pass the `infoProvided` list to the response (which optional inputs were used)
|
|
|
|
4. Generate top-5 species+disease combo predictions:
|
|
- After enrichment, construct `TopPrediction[]` from the top enriched predictions
|
|
- Each entry: `{ speciesName, diseaseName, diseaseId, confidence, rank }`
|
|
- Include species confidence when `userSpecies` was provided
|
|
|
|
5. Add `speciesConfidence` to response:
|
|
- When user provides species, compute how much the constraint improves confidence vs unconstrained
|
|
- Return both constrained and unconstrained confidence for comparison
|
|
|
|
6. Handle demo/mock mode:
|
|
- When no real model loaded, return mock top-5 combos with appropriate demo_mode flag
|
|
- Mock combos should be realistic (use knowledge base to generate plausible species/disease pairs)
|
|
|
|
tests:
|
|
|
|
- Integration: single image + no options → existing behavior unchanged
|
|
- Integration: single image + species → constrained results, all predictions belong to that species
|
|
- Integration: 2 images → ensemble results, confidence should differ from single image
|
|
- Integration: 2 images + species → fully constrained ensemble
|
|
- Integration: missing secondImageId returns 400 error
|
|
- Integration: demo mode returns mock data
|
|
|
|
acceptance_criteria:
|
|
|
|
- Existing single-image identify flow works unchanged when `options` omitted
|
|
- When secondImageId is provided, inference runs on both images
|
|
- When userSpecies is provided, only diseases of that species are returned
|
|
- Top-5 species/disease combos are included in response
|
|
- Confidence reflects whether 1 or 2 images were used
|
|
- 400 error for invalid/missing second image
|
|
|
|
validation:
|
|
|
|
- `npx tsc --noEmit` passes
|
|
- Existing identify API tests pass: `npx vitest run src/app/api/identify/`
|
|
- Manual test with curl sending multi-image request
|
|
|
|
notes:
|
|
|
|
- The `infoProvided` array helps the UI show what data was used for the diagnosis
|
|
- When userSpecies is given but no model can restrict to that species (e.g., mock mode), fall back to filtering results client-side by the plant name
|