initial import: @mikefreno/omp-deepi-research (omp port)

2026-08-10 09:46:09 -04:00
commit 88abadfabf
14 changed files with 3721 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,115 @@
+# Deep Research
+
+Multi-round deep web research powered by Firecrawl with iterative query refinement.
+
+Deep Research is a local omp extension under `~/.omp/agent/extensions/deepi-research/`.
+Omp loads it via `omp.extensions` in `package.json` (entry `./index.ts`).
+
+## Features
+
+- **Multi-round iteration**: Each round generates follow-up queries based on previous findings (depth 1-3)
+- **Parallel query expansion**: Multiple diverse search queries per round (breadth 1-5) covering technical, practical, comparative, critical, and forward-looking angles
+- **Sub-question decomposition**: Broad questions are broken into focused sub-topics before query generation (depth > 1)
+- **Round-robin parallel execution**: Searches and analyses run concurrently within each round using bounded-concurrency worker pools, dramatically reducing total research time
+- **LLM-driven analysis**: Each query's results are analyzed by its own agent session (per-query provenance) to extract structured findings with confidence ratings
+- **Source authority scoring**: Every source is scored by domain authority; low-quality SEO domains are penalized with a hard floor; findings are ranked by authority × confidence before synthesis
+- **Cross-query corroboration**: A finding is corroborated only when its sources were independently surfaced by multiple different search queries
+- **Citation integrity**: References are rebuilt from the authoritative bibliography (never the LLM's), and hallucinated inline citation numbers are stripped
+- **Near-duplicate detection**: Syndicated copies of the same article are removed by title similarity, and duplicate findings across rounds are merged
+- **Automatic deduplication**: Search results are deduplicated by URL across all queries
+- **Robust LLM output parsing**: JSON output with code fences, prose prefixes, or trailing commas is parsed reliably
+- **Graceful degradation**: Individual search or analysis failures don't crash the full research — partial results are preserved, with retry-with-backoff for transient Firecrawl errors
+- **Progress streaming**: Real-time progress widget with spinner, phase indicators, and progress bar
+- **Abort support**: Research can be cancelled mid-flight via `AbortSignal`
+- **Rich TUI rendering**: Compact collapsed view and detailed expanded view in the terminal UI
+- **Fallback resilience**: Built-in fallback query generation and report synthesis when LLM calls fail
+
+## Usage
+
+### Tool (LLM-callable)
+
+Registers the `deep_research` tool for AI agent use:
+
+```
+deep_research — multi-round deep web research via Firecrawl with iterative query refinement
+```
+
+Parameters:
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `question` | string | — | The research question to investigate |
+| `depth` | integer (1-3) | 2 | Number of research rounds |
+| `breadth` | integer (1-5) | 3 | Search queries per round |
+| `format` | "markdown" \| "structured" | "markdown" | Output format for the report |
+| `audience` | "general" \| "expert" \| "executive" | "general" | Tone and depth for the report audience |
+| `details.showRoundDetails` | boolean | false | Include per-round search metadata (incl. failed searches) in output |
+
+### Command (interactive)
+
+```
+/deepi-research <your research question>
+```
+
+Prompts for depth (1-3 rounds) and breadth (1-5 queries) interactively, then runs the research and sends the final report as a user message.
+
+### Recommended usage
+
+- Use `deep_research` for complex, multi-faceted questions that benefit from multiple search angles and iterative refinement.
+- The tool handles query generation, web search, result analysis, and report synthesis automatically.
+- For simple fact-finding questions, use `firecrawl_search` directly instead.
+
+## Architecture
+
+```
+Research Flow:
+
+  Question
+     ↓
+  ┌─ Round 1 ───────────────────────────┐
+  │  LLM → generate queries (N angles)   │
+  │  Firecrawl → search each query       │
+  │  LLM → analyze results → findings    │
+  └──────────────┬───────────────────────┘
+                 ↓ (follow-up queries)
+  ┌─ Round 2 ───────────────────────────┐
+  │  LLM → identify knowledge gaps       │
+  │  Firecrawl → search follow-ups       │
+  │  LLM → analyze → new findings        │
+  └──────────────┬───────────────────────┘
+                 ↓ (iterate depth times)
+  ┌─ Synthesis ─────────────────────────┐
+  │  LLM → synthesize all findings       │
+  │  → comprehensive research report     │
+  └─────────────────────────────────────┘
+```
+
+## Configuration
+
+Deep Research reads Firecrawl configuration from omp's config.yml files, with the following resolution order (later wins):
+
+1. Environment variables (`FIRECRAWL_BASE_URL`, `FIRECRAWL_API_KEY`)
+2. Global config (`$agentDir/config.yml`) → `firecrawl.*`
+3. Project config (`.omp/config.yml`) → `firecrawl.*`
+4. Default `http://localhost:3002` (if nothing else sets baseUrl)
+
+The agent directory (`$agentDir`) defaults to `~/.omp/agent`.
+
+**Global config** (`~/.omp/agent/config.yml`):
+
+```yaml
+firecrawl:
+  baseUrl: http://localhost:3002
+```
+
+**Project config** (`.omp/config.yml` — overrides global):
+
+```yaml
+firecrawl:
+  baseUrl: https://firecrawl.team.internal
+  apiKey: your-api-key
+```
+
+### Session startup check
+
+On `session_start`, the extension checks whether the Firecrawl endpoint is reachable. If not, it shows a warning notification so you know searches will fail before you try to use it.