fix: review fires for all tasks — commit then review committed diff, merge on pass

Previous flow reviewed uncommitted changes, so when a task agent
self-committed its work the review was silently skipped (no uncommitted
changes → review loop body never entered). This caused reviews to fire
inconsistently across tasks.

New review-gated flow (when autoReview is on):
1. Execute task
2. Ensure committed — commit session fallback when the agent didn't
   self-commit (handles both self-commit and no-commit agents)
3. Review the COMPLETE task diff (baseRef..HEAD) captured before
   execution, so the reviewer sees all commits not just the latest
4. On fail → re-execute with feedback → commit → re-review (same
   baseRef, so reviewer sees complete state including fixes)
5. On pass → merge worktree (all changes already committed)

Commit is now mandated when autoReview is on (autoCommit forced true,
not asked at startup). autoCommit only asked when autoReview is off.

Add captureGitHead + getCommitRangeDiff helpers to utils.ts. Switch
review prompt from buildReviewPromptUncommitted to buildReviewPrompt
(reviewing committed changes, not uncommitted).
This commit is contained in:
2026-07-22 18:29:29 -04:00
parent 0ef540ed47
commit 74c9ead7af
5 changed files with 200 additions and 98 deletions

View File

@@ -120,8 +120,8 @@ execution:
models: # round-robin in <provider>/<model> format models: # round-robin in <provider>/<model> format
- google/gemini-3.5-flash # 1st and 3rd task in parallel - google/gemini-3.5-flash # 1st and 3rd task in parallel
- openai/gpt-5.5 # 2nd task in parallel - openai/gpt-5.5 # 2nd task in parallel
autoCommit: true # commit changes after each task passes review (or when autoReview is off) autoCommit: true # commit after each task (mandated when autoReview is on; standalone toggle when off)
autoReview: false # review changes BEFORE commit; loop on fail, commit on pass autoReview: false # commit → review → loop on fail → merge on pass
implModel: "" # model for task impl (sequential mode, empty = inherit parent) implModel: "" # model for task impl (sequential mode, empty = inherit parent)
commitModel: "" # model for commit sessions (empty = inherit task model) commitModel: "" # model for commit sessions (empty = inherit task model)
reviewModel: "" # model for review sessions (empty = inherit task model) reviewModel: "" # model for review sessions (empty = inherit task model)
@@ -145,15 +145,17 @@ prompts:
#### Auto-review and Auto-commit #### Auto-review and Auto-commit
At loop startup the review question is asked FIRST, then the commit At loop startup the review question is asked FIRST. When `autoReview` is
question. When `autoReview` is enabled, a review gate runs BEFORE the enabled, commit is **mandated** — after task execution, changes are
commit: each task's uncommitted changes are reviewed against the task committed (via a commit agent session when the task agent didn't
description, and on a `fail` verdict the task is re-executed with the self-commit), then the complete task diff (`baseRef..HEAD`) is reviewed
review feedback injected into the prompt (looping until the review against the task description. On a `fail` verdict the task is
passes or `maxReviewRetries` is exhausted). When `autoCommit` is also re-executed with the review feedback injected into the prompt (looping
enabled, a follow-up agent stages and commits the changes once the until the review passes or `maxReviewRetries` is exhausted). After
review passes (or after retries exhaust); otherwise the changes are re-execution, changes are committed again and the full diff is
left uncommitted for manual inspection. re-reviewed with the same base ref so the reviewer sees the complete
state — original work plus fixes. On pass, the changes are already
committed and the worktree merges.
When `autoReview` is disabled, `autoCommit` runs a follow-up commit When `autoReview` is disabled, `autoCommit` runs a follow-up commit
agent after each task with no review. Both options can be overridden at agent after each task with no review. Both options can be overridden at

View File

@@ -123,10 +123,10 @@ function buildPlanByMode(
/** /**
* Prompt the user to select auto-review and auto-commit options for this loop. * Prompt the user to select auto-review and auto-commit options for this loop.
* Reviews are asked about FIRST, then commits — so the user can opt into a * Reviews are asked about FIRST. When autoReview is on, commit is always
* review gate (review changes, loop on fail) and decide whether a passing * mandated (it happens before review) — so autoCommit is forced true and not
* review should commit. Fields explicitly set in the config YAML are * asked about. When autoReview is off, autoCommit is asked as a stand-alone
* skipped (no prompt). * toggle. Fields explicitly set in the config YAML are skipped (no prompt).
* Returns the selected options (or config defaults if cancelled). * Returns the selected options (or config defaults if cancelled).
*/ */
async function selectLoopOptions( async function selectLoopOptions(
@@ -136,17 +136,16 @@ async function selectLoopOptions(
const explicit = config.execution.explicitKeys; const explicit = config.execution.explicitKeys;
// ── 1. Auto-review (asked FIRST) ── // ── 1. Auto-review (asked FIRST) ──
// When enabled, a review gate runs BEFORE the commit: uncommitted changes // When enabled, a commit is mandated before review (the task agent's
// are reviewed, and on a "fail" verdict the task is re-executed with the // changes are committed, then the complete diff is reviewed). On 'fail'
// review feedback injected (looping until pass or maxReviewRetries is // the task is re-executed with review feedback (looping until pass or
// exhausted). Only then does the commit session run (when autoCommit is // maxReviewRetries exhausted). On pass the worktree merges.
// also enabled).
let autoReview: boolean; let autoReview: boolean;
if (explicit?.has("autoReview")) { if (explicit?.has("autoReview")) {
autoReview = config.execution.autoReview; autoReview = config.execution.autoReview;
} else { } else {
const reviewChoice = await ctx.ui.select("Auto-review after each task?", [ const reviewChoice = await ctx.ui.select("Auto-review after each task?", [
"Yes — review changes and loop on failures (re-execute until pass)", "Yes — review the task commit and loop on failures (re-execute until pass)",
"No — skip review", "No — skip review",
]); ]);
autoReview = reviewChoice autoReview = reviewChoice
@@ -173,24 +172,15 @@ async function selectLoopOptions(
} }
} }
// ── 3. Auto-commit (asked AFTER review) ── // ── 3. Auto-commit ──
// When review is enabled, this gates whether the changes are committed // When autoReview is on, commit is always mandated (it happens before the
// after a passing review (or after retries exhaust). When review is // review). autoCommit is forced true and not asked about. When review is
// disabled, this is a stand-alone "commit per task" toggle. // disabled, autoCommit is asked as a stand-alone "commit per task" toggle.
let autoCommit: boolean; let autoCommit: boolean;
if (explicit?.has("autoCommit")) { if (autoReview) {
autoCommit = true; // mandated by the review-gated flow
} else if (explicit?.has("autoCommit")) {
autoCommit = config.execution.autoCommit; autoCommit = config.execution.autoCommit;
} else if (autoReview) {
const commitChoice = await ctx.ui.select(
"Auto-commit after a passing review?",
[
"Yes — commit changes once the review passes",
"No — leave changes uncommitted after review",
],
);
autoCommit = commitChoice
? commitChoice.startsWith("Yes")
: config.execution.autoCommit;
} else { } else {
const commitChoice = await ctx.ui.select("Auto-commit after each task?", [ const commitChoice = await ctx.ui.select("Auto-commit after each task?", [
"Yes — stage and commit changes automatically", "Yes — stage and commit changes automatically",

View File

@@ -15,7 +15,7 @@ import type {
} from "@earendil-works/pi-coding-agent"; } from "@earendil-works/pi-coding-agent";
import { import {
buildTaskPrompt, buildTaskPrompt,
buildReviewPromptUncommitted, buildReviewPrompt,
buildConflictResolutionPrompt, buildConflictResolutionPrompt,
MAX_DIFF_BYTES, MAX_DIFF_BYTES,
} from "./prompts"; } from "./prompts";
@@ -42,6 +42,8 @@ import {
writeFileSafe, writeFileSafe,
ensureDir, ensureDir,
captureGitCommits, captureGitCommits,
captureGitHead,
getCommitRangeDiff,
hasUncommittedChanges, hasUncommittedChanges,
getGitStatusPorcelain, getGitStatusPorcelain,
getGitDiff, getGitDiff,
@@ -805,6 +807,12 @@ async function executeTask(
task.dependencies || [], task.dependencies || [],
); );
// Capture base HEAD before execution so the review-gated loop can diff
// the complete task output (baseRef..HEAD) across execution + fix attempts.
const baseRef = config.execution.autoReview
? captureGitHead(worktreeDir)
: undefined;
// Run the task // Run the task
const result = await runTask( const result = await runTask(
task, task,
@@ -826,26 +834,57 @@ async function executeTask(
let reviewRetries = 0; let reviewRetries = 0;
if (config.execution.autoReview) { if (config.execution.autoReview) {
// ── Review-gated loop: review FIRST, re-execute on reject, commit on pass ── // ── Review-gated loop: commit → review re-execute on fail → merge on pass ──
// The review examines uncommitted changes. If the verdict is "fail", // The commit is mandated — when the task agent didn't self-commit, a
// the task is re-executed with the review feedback injected into the // commit session handles it. Then the COMPLETE task diff (baseRef..HEAD)
// prompt (up to maxReviewRetries). On pass (or after retries exhaust) // is reviewed. On 'fail' the task is re-executed with the review feedback
// the commit session runs when autoCommit is also enabled — otherwise // injected (up to maxReviewRetries); after re-execution changes are
// the changes are left uncommitted for manual inspection. // committed again and the full diff is re-reviewed with the SAME baseRef
// so the reviewer sees the complete state, not just incremental fixes.
// On pass the changes are already committed — the worktree merges next.
const maxRetries = config.execution.maxReviewRetries; const maxRetries = config.execution.maxReviewRetries;
let attempt = 0; let attempt = 0;
try { try {
while (hasUncommittedChanges(worktreeDir)) { // ── Ensure committed (commit session fallback) ──
const status = getGitStatusPorcelain(worktreeDir); // If the task agent didn't self-commit, a commit session handles it.
const reviewDiff = getGitDiff(worktreeDir); if (hasUncommittedChanges(worktreeDir)) {
if (!reviewDiff && !status) break; const commitResult = await runCommitSession(
ctx,
config,
task,
worktreeDir,
currentModel,
roundRobin,
sendChatMessage,
);
if (commitResult.success) {
finalCommitMessages = [
...finalCommitMessages,
...commitResult.commitMessages,
];
finalCommitSummary = finalCommitSummary
? `${finalCommitSummary}; ${commitResult.commitSummary}`
: commitResult.commitSummary;
}
}
const reviewPrompt = buildReviewPromptUncommitted( // ── Review loop ──
// baseRef was captured before runTask (above). Each review iteration
// diffs the range baseRef..HEAD — the complete task output including
// all fix attempts. On re-execution the same baseRef is reused.
while (true) {
const reviewInfo = baseRef
? getCommitRangeDiff(worktreeDir, baseRef)
: null;
if (!reviewInfo || !reviewInfo.diff) break; // nothing to review
const reviewPrompt = buildReviewPrompt(
task, task,
project, project,
status, reviewInfo.hash,
reviewDiff, reviewInfo.subject,
reviewInfo.diff,
config.prompts.projectContext, config.prompts.projectContext,
); );
@@ -875,11 +914,15 @@ async function executeTask(
`~ review for ${task.id} · ${task.title} — review session failed: ${reviewResult.error}`, `~ review for ${task.id} · ${task.title} — review session failed: ${reviewResult.error}`,
{ toolCalls: reviewToolCalls }, { toolCalls: reviewToolCalls },
); );
break; // commit on autoCommit, otherwise leave uncommitted break; // proceed with what we have (changes already committed)
} }
const reviewText = reviewResult.text.trim(); const reviewText = reviewResult.text.trim();
const review = extractReview(reviewText, task.id, "uncommitted"); const review = extractReview(
reviewText,
task.id,
reviewInfo.hash,
);
finalReview = review ?? undefined; finalReview = review ?? undefined;
// Persist structured review JSON when opted in. // Persist structured review JSON when opted in.
@@ -897,7 +940,7 @@ async function executeTask(
review && review &&
(review.verdict === "pass" || review.verdict === "warn") (review.verdict === "pass" || review.verdict === "warn")
) { ) {
// Review passed — proceed to commit. // Review passed — all changes are committed; merge will follow.
const label = `${verdictGlyph(review.verdict)} ${verdictSummary(review)}`; const label = `${verdictGlyph(review.verdict)} ${verdictSummary(review)}`;
const savedHint = reviewPath ? ` · saved to ${reviewPath}` : ""; const savedHint = reviewPath ? ` · saved to ${reviewPath}` : "";
sendChatMessage?.( sendChatMessage?.(
@@ -909,7 +952,7 @@ async function executeTask(
reviewResult: review, reviewResult: review,
}, },
); );
break; // good to commit break; // good to merge
} }
// Review rejected (fail) or verdict not parsed. // Review rejected (fail) or verdict not parsed.
@@ -934,7 +977,7 @@ async function executeTask(
} }
if (attempt >= maxRetries) { if (attempt >= maxRetries) {
// Retries exhausted. // Retries exhausted — changes are already committed; proceed.
if (config.execution.reviewBlockOnFail) { if (config.execution.reviewBlockOnFail) {
sendChatMessage?.( sendChatMessage?.(
`${task.id} · ${task.title} — review rejected after ${maxRetries} retr${maxRetries === 1 ? "y" : "ies"} (reviewBlockOnFail)`, `${task.id} · ${task.title} — review rejected after ${maxRetries} retr${maxRetries === 1 ? "y" : "ies"} (reviewBlockOnFail)`,
@@ -952,11 +995,9 @@ async function executeTask(
return; return;
} }
sendChatMessage?.( sendChatMessage?.(
config.execution.autoCommit `~ review for ${task.id} · ${task.title} — max retries (${maxRetries}) exhausted, proceeding with current state`,
? `~ review for ${task.id} · ${task.title} — max retries (${maxRetries}) exhausted, committing anyway`
: `~ review for ${task.id} · ${task.title} — max retries (${maxRetries}) exhausted, leaving changes as-is`,
); );
break; // commit on autoCommit, otherwise leave uncommitted break; // changes already committed — merge proceeds
} }
attempt++; attempt++;
@@ -984,7 +1025,7 @@ async function executeTask(
sendChatMessage?.( sendChatMessage?.(
`~ re-execution for ${task.id} · ${task.title} failed: ${fixResult.error}`, `~ re-execution for ${task.id} · ${task.title} failed: ${fixResult.error}`,
); );
break; // commit what we have break; // proceed with what we have
} }
// Merge commit messages from the fix attempt. // Merge commit messages from the fix attempt.
@@ -995,34 +1036,30 @@ async function executeTask(
finalCommitSummary = finalCommitSummary finalCommitSummary = finalCommitSummary
? `${finalCommitSummary}; ${fixResult.commitSummary ?? ""}` ? `${finalCommitSummary}; ${fixResult.commitSummary ?? ""}`
: (fixResult.commitSummary ?? ""); : (fixResult.commitSummary ?? "");
// Loop back to review the updated changes.
}
// ── Commit (after review passes or retries exhausted) ── // Ensure committed after re-execution (same commit fallback).
// Only commit when autoCommit is enabled; review-only mode leaves if (hasUncommittedChanges(worktreeDir)) {
// changes uncommitted so the user can inspect them manually. const commitResult = await runCommitSession(
if ( ctx,
config.execution.autoCommit && config,
hasUncommittedChanges(worktreeDir) task,
) { worktreeDir,
const commitResult = await runCommitSession( currentModel,
ctx, roundRobin,
config, sendChatMessage,
task, );
worktreeDir, if (commitResult.success) {
currentModel, finalCommitMessages = [
roundRobin, ...finalCommitMessages,
sendChatMessage, ...commitResult.commitMessages,
); ];
if (commitResult.success) { finalCommitSummary = finalCommitSummary
finalCommitMessages = [ ? `${finalCommitSummary}; ${commitResult.commitSummary}`
...finalCommitMessages, : commitResult.commitSummary;
...commitResult.commitMessages, }
];
finalCommitSummary = finalCommitSummary
? `${finalCommitSummary}; ${commitResult.commitSummary}`
: commitResult.commitSummary;
} }
// Loop back to review with the same baseRef — the reviewer sees the
// complete diff (original work + fixes), not just incremental changes.
} }
} catch (error) { } catch (error) {
sendChatMessage?.( sendChatMessage?.(

View File

@@ -211,12 +211,13 @@ export interface RalpiConfig {
models: string[]; models: string[];
/** Spawn a follow-up agent to commit changes after each task completes */ /** Spawn a follow-up agent to commit changes after each task completes */
autoCommit: boolean; autoCommit: boolean;
/** Spawn a review agent to review uncommitted changes against the /** Spawn a review agent to review the task's committed changes against
* task description BEFORE committing. On a 'fail' verdict the task is * the task description. When autoReview is on, commit is mandated:
* re-executed with the review feedback injected (loops until pass or * changes are committed (via commit session fallback when the agent
* maxReviewRetries is exhausted). When autoCommit is also enabled, the * didn't self-commit), then the COMPLETE diff (baseRef..HEAD) is
* commit runs after the review passes (or retries exhaust); otherwise * reviewed. On 'fail' the task is re-executed with feedback (loops
* changes are left uncommitted. */ * until pass or maxReviewRetries). On pass the worktree merges.
* When autoReview is off, autoCommit controls standalone commit. */
autoReview: boolean; autoReview: boolean;
/** Persist the full review output to `.ralpi/reviews/<task-id>.md`. /** Persist the full review output to `.ralpi/reviews/<task-id>.md`.
* Only active when autoReview is true and the user opts in at loop start. */ * Only active when autoReview is true and the user opts in at loop start. */
@@ -236,13 +237,12 @@ export interface RalpiConfig {
reviewTimeoutMs: number; reviewTimeoutMs: number;
/** Max review-fix re-execution attempts before giving up (0 = no retries; /** Max review-fix re-execution attempts before giving up (0 = no retries;
* review runs once, reject = stop). Active whenever autoReview is * review runs once, reject = stop). Active whenever autoReview is
* enabled. On exhaustion: if autoCommit is enabled, the changes are * enabled. On exhaustion the task proceeds with its committed changes
* committed anyway; if reviewBlockOnFail is set, the task is marked * (the worktree merges) unless reviewBlockOnFail is set. */
* failed instead of committing. */
maxReviewRetries: number; maxReviewRetries: number;
/** When true, a 'fail' review verdict after exhausting maxReviewRetries /** When true, a 'fail' review verdict after exhausting maxReviewRetries
* marks the task as failed instead of leaving changes (committing when * marks the task as failed instead of proceeding with its committed
* autoCommit, or leaving uncommitted otherwise). */ * changes (the worktree does not merge). */
reviewBlockOnFail: boolean; reviewBlockOnFail: boolean;
/** Maximum total duration for the entire loop execution in milliseconds (0 = no limit). Checked between batches — in-progress tasks finish naturally. */ /** Maximum total duration for the entire loop execution in milliseconds (0 = no limit). Checked between batches — in-progress tasks finish naturally. */
loopTimeoutMs: number; loopTimeoutMs: number;

View File

@@ -807,3 +807,76 @@ export function getLatestCommitDiff(
return null; return null;
} }
} }
/**
* Capture the current HEAD commit SHA. Returns the full 40-char SHA, or
* undefined when not a git repo / git unavailable. Used to snapshot the
* worktree HEAD before a task runs so the review can diff the complete task
* output (baseRef..HEAD) — including any commits the task agent makes.
*/
export function captureGitHead(projectDir: string): string | undefined {
const { execSync } = require("node:child_process");
try {
const sha = execSync("git rev-parse HEAD", {
cwd: projectDir,
encoding: "utf-8",
stdio: ["pipe", "pipe", "pipe"],
}).trim();
// Guard against injection — only accept hex SHAs.
return /^[0-9a-f]{7,40}$/i.test(sha) ? sha : undefined;
} catch {
return undefined;
}
}
/**
* Get the diff from `baseRef` to HEAD — the complete set of committed changes
* made since the base reference. Used by the review-gated loop so the reviewer
* sees the full task diff (all commits, not just the latest) across execution
* attempts and re-execution fixes. `baseRef` must be a validated hex SHA from
* captureGitHead(). Returns the short HEAD hash, HEAD subject, and range diff,
* or null when git is unavailable / baseRef is invalid / no changes exist.
*/
export function getCommitRangeDiff(
projectDir: string,
baseRef: string,
): { hash: string; subject: string; diff: string } | null {
const { execSync } = require("node:child_process");
// Only pass validated hex SHAs to the shell.
if (!/^[0-9a-f]{7,40}$/i.test(baseRef)) return null;
try {
execSync("git rev-parse --git-dir", {
cwd: projectDir,
stdio: "pipe",
});
} catch {
return null;
}
try {
const hash = execSync("git rev-parse --short HEAD", {
cwd: projectDir,
encoding: "utf-8",
}).trim();
const subject = execSync("git log -1 --format=%s", {
cwd: projectDir,
encoding: "utf-8",
}).trim();
// Diff from baseRef to HEAD — shows all committed changes made since
// the snapshot. Includes stat overview + full patch.
const diff = execSync(`git diff ${baseRef} HEAD --stat --patch`, {
cwd: projectDir,
encoding: "utf-8",
maxBuffer: 1024 * 1024,
}).trim();
if (!diff) return null; // no changes since baseRef
return { hash, subject, diff };
} catch {
return null;
}
}