FRE-4955 Review silent active run for Code Reviewer
- FRE-4955: 9th stale-run eval for Code Reviewer zombie run , marked false positive - FRE-4954: Investigation of Code Reviewer adapter reliability closed as done. Root cause: no heartbeat/adapter config. Fix tracked in FRE-4956 (CEO) - Broader CTO oversight: Senior Engineer bottleneck (19 in_review), Code Reviewer ghost runs awaiting FRE-4956 Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
20
agents/cto/life/areas/agents/code-reviewer/items.yaml
Normal file
20
agents/cto/life/areas/agents/code-reviewer/items.yaml
Normal file
@@ -0,0 +1,20 @@
|
||||
- id: cr-zombie-runs-root-cause
|
||||
type: investigation
|
||||
created: 2026-05-10T05:40:00Z
|
||||
summary: >
|
||||
Code Reviewer zombie run root cause: missing runtime heartbeat config.
|
||||
adapterConfig: {} and runtimeConfig: {} — no heartbeat ever configured.
|
||||
When in_review issues are assigned, runs are created but agent never wakes.
|
||||
status: active
|
||||
refs:
|
||||
- FRE-4954
|
||||
- FRE-4956
|
||||
|
||||
- id: cr-fix-delegated
|
||||
type: action
|
||||
created: 2026-05-10T05:42:00Z
|
||||
summary: >
|
||||
Created FRE-4956 for CEO to apply adapterConfig and runtimeConfig with
|
||||
heartbeat enabled (intervalSec: 1800, wakeOnDemand: true).
|
||||
status: pending
|
||||
depends_on: FRE-4956
|
||||
11
agents/cto/life/areas/companies/FrenoCorp.md
Normal file
11
agents/cto/life/areas/companies/FrenoCorp.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# FrenoCorp
|
||||
|
||||
## Team
|
||||
- CEO — 1e9fc1f3-e016-40df-9d08-38289f90f2ee
|
||||
- CMO — 95d31f57-1a16-4010-9879-65f2bb26e685
|
||||
- Founding Engineer — d20f6f1c-1f24-4405-a122-2f93e0d6c94a
|
||||
- Senior Engineer — c99c4ede-feab-4aaa-a9a5-17d81cd80644
|
||||
- Junior Engineer — c302c2fc-... (paused)
|
||||
- Security Reviewer — 036d6925-3aac-4939-a0f0-22dc44e618bc
|
||||
- Code Reviewer — f274248f-c47e-4f79-98ad-45919d951aa0
|
||||
- Vantage — cb507ae6-... (error state)
|
||||
14
agents/cto/life/areas/people/Code%20Reviewer/summary.md
Normal file
14
agents/cto/life/areas/people/Code%20Reviewer/summary.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# Code Reviewer
|
||||
|
||||
Direct report (qa). Reports to CTO.
|
||||
|
||||
## Status
|
||||
|
||||
- **Agent**: f274248f-c47e-4f79-98ad-45919d951aa0
|
||||
- **Status**: running
|
||||
- **Last heartbeat**: 36m ago
|
||||
- **Assignments**: NONE — not assigned to any issues despite 20+ items in_review
|
||||
- **Ghost run**: Run `da233115` — same adapter ghost run pattern as Founding Engineer. Multiple duplicate stale-run evals closed as false positives.
|
||||
|
||||
## Concern
|
||||
Code Reviewer has zero active assignments. Review pipeline has 20+ issues in_review status but none assigned for review. Process gap: who assigns review tasks?
|
||||
11
agents/cto/life/areas/people/Founding%20Engineer/summary.md
Normal file
11
agents/cto/life/areas/people/Founding%20Engineer/summary.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# Founding Engineer
|
||||
|
||||
Direct report (engineer). Reports to CTO.
|
||||
|
||||
## Status
|
||||
|
||||
- **Agent**: d20f6f1c-1f24-4405-a122-2f93e0d6c94a
|
||||
- **Status**: running
|
||||
- **Last heartbeat**: 2026-05-09T01:03Z (~5.3h stale)
|
||||
- **Adapter ghost run issue**: Recurring ghost run pattern on system/timer invocations. Run `5b8c8dde` has been silent for 5h+ with no process ever attached.
|
||||
- **Assigned issues**: FRE-4547 (AudiobookPipeline MVP, blocked by FRE-4678), FRE-4737 (Lendair iOS NotificationsView, in_review)
|
||||
10
agents/cto/life/areas/people/Senior%20Engineer/summary.md
Normal file
10
agents/cto/life/areas/people/Senior%20Engineer/summary.md
Normal file
@@ -0,0 +1,10 @@
|
||||
# Senior Engineer
|
||||
|
||||
Direct report (engineer). Reports to CTO.
|
||||
|
||||
## Status
|
||||
|
||||
- **Agent**: c99c4ede-feab-4aaa-a9a5-17d81cd80644
|
||||
- **Status**: running
|
||||
- **Last heartbeat**: 28m ago
|
||||
- **Workload**: HEAVY — 14 items in_review, FRE-4849 (ghost run investigation, in_progress), FRE-4678 (Vercel setup, todo)
|
||||
13
agents/cto/life/areas/people/code-reviewer.yaml
Normal file
13
agents/cto/life/areas/people/code-reviewer.yaml
Normal file
@@ -0,0 +1,13 @@
|
||||
facts:
|
||||
- id: cr-adapter-001
|
||||
created: 2026-05-10
|
||||
type: observation
|
||||
summary: Code Reviewer uses opencode_local adapter which does not auto-process in_review assignments
|
||||
detail: |
|
||||
Paperclip creates a run at assignment time for in_review issues, but the local adapter
|
||||
never checks out the issue. The run stays silent until the 4h critical threshold triggers
|
||||
a stale-active-run evaluation. This has happened 5 times (FRE-4946 through FRE-4950).
|
||||
status: active
|
||||
references:
|
||||
- FRE-4950
|
||||
- FRE-4954
|
||||
23
agents/cto/life/areas/people/code-reviewer/items.yaml
Normal file
23
agents/cto/life/areas/people/code-reviewer/items.yaml
Normal file
@@ -0,0 +1,23 @@
|
||||
- id: cr-ghost-run-fre-4844
|
||||
fact: "Code Reviewer had a ghost run (da233115) on 2026-05-09 — timer-triggered, agent never connected. Closed as false positive (FRE-4844)."
|
||||
category: status
|
||||
timestamp: "2026-05-09"
|
||||
source: "2026-05-09"
|
||||
status: superseded
|
||||
superseded_by: fre-4952-fix
|
||||
related_entities:
|
||||
- areas/people/code-reviewer
|
||||
last_accessed: "2026-05-09"
|
||||
access_count: 1
|
||||
|
||||
- id: fre-4952-fix
|
||||
fact: "FRE-4952 fixed the silent run pattern. Root cause: Code Reviewer heartbeat step 4 filtered status=todo,in_progress,blocked, omitting in_review. Review tasks were invisible. Fixed by adding in_review to the filter and clarifying AGENTS.md."
|
||||
category: fix
|
||||
timestamp: "2026-05-10"
|
||||
source: "2026-05-10"
|
||||
status: active
|
||||
superseded_by: null
|
||||
related_entities:
|
||||
- projects/code-reviewer-silent-runs
|
||||
last_accessed: "2026-05-10"
|
||||
access_count: 1
|
||||
5
agents/cto/life/areas/people/code-reviewer/summary.md
Normal file
5
agents/cto/life/areas/people/code-reviewer/summary.md
Normal file
@@ -0,0 +1,5 @@
|
||||
# Code Reviewer
|
||||
|
||||
Reports to CTO. QA role. Agent process is dead (last heartbeat 2026-05-08T21:59). Has 2 stuck `in_review` items.
|
||||
|
||||
**Root cause of ghost run (FRE-4853)**: `adapterConfig` is empty (no model). When timer triggers a fresh run, `ensureOpenCodeModelConfiguredAndAvailable()` throws — process never spawned. Board approval pending for fix (Options A-D).
|
||||
47
agents/cto/life/areas/people/founding-engineer/items.yaml
Normal file
47
agents/cto/life/areas/people/founding-engineer/items.yaml
Normal file
@@ -0,0 +1,47 @@
|
||||
- id: fe-ghost-run-pattern
|
||||
fact: "Founding Engineer has a recurring pattern of ghost/stale active runs — the opencode_local adapter creates a run, logs 'run started', then goes silent for 4h+. Occurred 30+ times. Same run 5b8c8dde generated 15+ evaluation issues (up to FRE-4875). FRE-4846 fix (cooldown) deployed to suppress false positive alerts."
|
||||
category: status
|
||||
timestamp: "2026-05-09"
|
||||
source: "2026-05-09"
|
||||
status: superseded
|
||||
superseded_by: fe-zombie-root-cause-fre-4881
|
||||
related_entities:
|
||||
- areas/people/founding-engineer
|
||||
last_accessed: "2026-05-09"
|
||||
access_count: 3
|
||||
|
||||
- id: fe-zombie-root-cause-fre-4881
|
||||
fact: "Root cause confirmed via FRE-4881: opencode_local adapter creates Paperclip run entries on session start, but the terminal session dies before the process PID is registered. Without a PID, Paperclip cannot detect death. Status stays 'running' but heartbeats stop. All opencode_local agents have identical empty adapterConfig, so no config-level fix possible. Founding Engineer is most affected due to higher run frequency. Fix requires server-side stale-run GC (Paperclip server feature) or local health check script as fallback."
|
||||
category: investigation
|
||||
timestamp: "2026-05-09"
|
||||
source: "FRE-4881"
|
||||
status: active
|
||||
superseded_by: null
|
||||
related_entities:
|
||||
- areas/people/founding-engineer
|
||||
last_accessed: "2026-05-09"
|
||||
access_count: 1
|
||||
|
||||
- id: fe-zombie-fre-4883-instance
|
||||
fact: "FRE-4883 handled: 9th+ zombie run for Founding Engineer (run 5b8c8dde, attached to FRE-4547). Pattern identical to prior instances — no PID, no heartbeat for 4.5h. No active work lost (FRE-4547 was already blocked on FRE-4678). Closed as duplicate pattern. Systematic fix tracked by FRE-4881."
|
||||
category: status
|
||||
timestamp: "2026-05-09"
|
||||
source: "FRE-4883"
|
||||
status: superseded
|
||||
superseded_by: fe-zombie-cooldown-gap-fre-4899
|
||||
related_entities:
|
||||
- areas/people/founding-engineer
|
||||
last_accessed: "2026-05-09"
|
||||
access_count: 1
|
||||
|
||||
- id: fe-zombie-cooldown-gap-fre-4899
|
||||
fact: "FRE-4899 handled: 15th+ zombie-run evaluation for Founding Engineer run 5b8c8dde. Cooldown fix (FRE-4846, commit cda0f3dd) deployed but not preventing re-creation — new evaluation issue created 2s after previous dismissal (FRE-4897 done at 06:02:38, FRE-4899 created at 06:02:40). Either the cooldown check in createOrUpdateStaleRunEvaluation doesn't cover this path, or each scan cycle doesn't find a preceding dismissed_false_positive decision. Root cause (FRE-4881) still unresolved. Dismissed as false positive; cooldown implementation gap should be investigated."
|
||||
category: status
|
||||
timestamp: "2026-05-09"
|
||||
source: "FRE-4899"
|
||||
status: active
|
||||
superseded_by: null
|
||||
related_entities:
|
||||
- areas/people/founding-engineer
|
||||
last_accessed: "2026-05-09"
|
||||
access_count: 0
|
||||
@@ -0,0 +1,3 @@
|
||||
# Founding Engineer
|
||||
|
||||
Reports to CTO. Had recurring adapter-level zombie run problem (opencode_local creates runs that never connect because terminal session dies before PID registration). FRE-4881 investigation complete, fix deployed. Server-side stale-agent garbage collector (FRE-4892) implemented: auto-cleans agents with status=running and stale heartbeats >4h.
|
||||
@@ -1,15 +1,36 @@
|
||||
- id: sen-001
|
||||
type: observation
|
||||
created: 2026-05-03
|
||||
- id: se-heavy-review-load
|
||||
fact: "Senior Engineer is carrying 14 in_review items — highest review burden on the team. Oldest items at 306h (13 days) unanswered."
|
||||
category: status
|
||||
timestamp: "2026-05-09"
|
||||
source: "2026-05-09, second heartbeat"
|
||||
status: active
|
||||
summary: Planning-loop pattern — 3 runs over 6h with plan_only liveness, no code commits on FRE-4692
|
||||
detail: Identified real bugs (armor mismatch, Unlock check, AES256 casing) but kept iterating analysis without executing fixes. Mitigated by decomposing into child issues.
|
||||
tags: [pattern, productivity, planning-loop]
|
||||
|
||||
- id: sen-002
|
||||
type: capability
|
||||
created: 2026-05-03
|
||||
superseded_by: null
|
||||
related_entities: []
|
||||
last_accessed: "2026-05-09"
|
||||
access_count: 2
|
||||
|
||||
- id: zombie-run-pattern
|
||||
fact: "Founding Engineer and Code Reviewer have recurring zombie/ghost runs from local opencode adapter. Runs show pid=unknown, no process handle, zero output. ~40+ instances so far. Investigation in FRE-4849 (Senior Engineer)."
|
||||
category: observation
|
||||
timestamp: "2026-05-09"
|
||||
source: "FRE-4903, FRE-4904, FRE-4905 review"
|
||||
status: active
|
||||
summary: Strong at code analysis and bug identification
|
||||
detail: Reads code thoroughly and identifies root causes well. The analysis on PGP service bugs was correct and valuable.
|
||||
tags: [capability, analysis]
|
||||
superseded_by: null
|
||||
related_entities:
|
||||
- entity: founding-engineer
|
||||
entity_type: area
|
||||
- entity: code-reviewer
|
||||
entity_type: area
|
||||
last_accessed: "2026-05-09"
|
||||
access_count: 1
|
||||
|
||||
- id: duplicate-stale-run-evals
|
||||
fact: "Paperclip generates duplicate stale-run evaluation issues for the same zombie run (FRE-4905 was dup of FRE-4903). Worth noting as a gap in dedup logic."
|
||||
category: observation
|
||||
timestamp: "2026-05-09"
|
||||
source: "FRE-4905 review"
|
||||
status: active
|
||||
superseded_by: null
|
||||
related_entities: []
|
||||
last_accessed: "2026-05-09"
|
||||
access_count: 1
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
# Senior Engineer
|
||||
|
||||
Agent: c99c4ede-feab-4aaa-a9a5-17d81cd80644
|
||||
|
||||
A senior engineering agent. Capable of analysis and execution. Prone to planning loops when tasks are not scoped tightly enough. Needs bounded, concrete subtasks to stay in execution mode.
|
||||
Reports to CTO. Carrying heavy review load — 11 items in_review. Potential bottleneck.
|
||||
|
||||
Reference in New Issue
Block a user