commit 1c83bd73e2db74b1a29ce8fae55df2f36192c9b0 Author: Michael Freno Date: Wed Oct 21 15:20:12 2026 -0400 pi-image-paths: attach dropped image paths and draw them in the chat A screenshot is normally sent to pi as a path, so the model gets its text and the reader gets nothing: pi renders image blocks in tool results but never on user messages. This extension closes both halves — the path collapses to "[Image #N]" as you type or on send, and the picture is drawn below the sent message. The rewrite is a terminal-input listener that re-reads a settled editor, joins word by word for paths with unescaped spaces, and reports a path that does not exist instead of eating the prompt. The drawing appends a custom session entry carrying the message's image blocks, one macrotask after message_end so it lands below the message pi persists afterwards, and renders it with pi-tui's inline image component under terminal.showImages and terminal.imageWidthCells. Entries stay out of the model context, so each image is still sent to the model once. Tests: npm test (21, node:test over a stand-in for pi: rewrite, send-time attachment, missing paths, append gate, renderer gating) and npm run typecheck. Co-Authored-By: nw-flash-flex diff --git a/.gitea/workflows/publish.yml b/.gitea/workflows/publish.yml new file mode 100644 index 0000000..0145830 --- /dev/null +++ b/.gitea/workflows/publish.yml @@ -0,0 +1,37 @@ +name: publish + +# Publish this package to the npm registry on version tags. +# +# Prerequisites on npmjs.com / git.freno.me: +# - an npm automation token (publish scope) available to this repo as the +# secret NPM_TOKEN (consumed via actions/setup-node registry auth) +# - the scoped package name claimed on npm (@mikefreno/pi-image-paths) +# +# The package ships TypeScript sources — nothing is built — so lifecycle +# scripts are skipped: typecheck and test run against the global pi SDK that +# CI does not install. +# +# Manual publish: Actions tab → Run workflow (workflow_dispatch). + +on: + push: + tags: ['v*'] + workflow_dispatch: + +jobs: + publish: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup node (npm registry auth) + uses: actions/setup-node@v4 + with: + node-version: '22' + registry-url: 'https://registry.npmjs.org' + + - name: Publish pi package + env: + NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} + run: npm publish --access public --ignore-scripts diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..3542bc1 --- /dev/null +++ b/.gitignore @@ -0,0 +1,7 @@ +.DS_Store +node_modules/ +dist/ +.pi-lens/ +.ralpi/ +package-lock.json +.worktrees/ diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..8fa6dd9 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Michael Freno + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..0081595 --- /dev/null +++ b/README.md @@ -0,0 +1,61 @@ +# pi-image-paths + +Drop a screenshot into [pi](https://pi.dev)'s chat input — by path, by drag and drop, or pasted +whole from anywhere — and the picture becomes a real attachment instead of a long line of text. + +```text +pi install npm:@mikefreno/pi-image-paths +``` + +```text +you › look at ~/Desktop/Screenshot\ 2026-10-21\ at\ 10.42.11.png — why is the header clipped? + ↓ +you › look at [Image #1] — why is the header clipped? + ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄ + █ (the screenshot, drawn in the chat) █ + ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ +``` + +## What it does + +| Piece | Mechanism | +|---|---| +| Collapses the path as you type | A terminal-input listener re-reads the editor when it is idle: every token run that resolves to an image file is rewritten in place to `[Image #N]`, the way pi collapses big pastes into `[paste #N]`. Dropping three files at once flips all three. | +| Attaches on send | The `input` event resolves the markers it wrote, then looks for any image path still in the prompt (pasted whole, RPC mode, rewritten too late), attaches every file it finds, and leaves `[Image #N]` in the text. | +| Draws what was sent | Pi renders image blocks on tool results but never on user messages, so the sent images ride in a custom session entry that a registered entry renderer draws with pi-tui's inline-image component. The entry is appended one macrotask after `message_end` — pi persists the user message after extension handlers return — which puts the picture directly below the message in the live chat and in the session log. | + +Supported: `png`, `jpg`/`jpeg`, `gif`, `webp`, `bmp`; `~` expansion; backslash-escaped spaces, +surrounding quotes, `file://` drops; bare filenames relative to the working directory; and paths +with unescaped spaces, which are joined word by word and disambiguated against the filesystem. +URLs are left alone, and a path that does not exist is reported as a warning instead of eating the +prompt. + +## Settings + +The drawing honours pi's own settings, live: + +| Setting | Effect | +|---|---| +| `terminal.showImages` (default `true`) | Off: no entry is appended and nothing is drawn — the images still reach the model. | +| `terminal.imageWidthCells` (default `60`) | Width the picture is drawn at. | +| `terminal.images` (default `auto`) | No inline-image protocol: the renderer stays out of the way and the transcript keeps only the marker. | + +Custom entries stay out of the model context, so the model still sees each image exactly once. +The image data is stored in the entry as well as in the message, which roughly doubles a session's +image bytes on disk. + +Modes other than the interactive TUI attach images but draw nothing, and append no entries. + +## Off the terminal: The Nook + +[The Nook](https://nook.freno.me) puts every coding agent you run in your Mac's notch, on your +iPhone and on your Apple Watch. It renders the same image blocks, so a screenshot dropped into pi +shows up there as a picture too. + +## Develop + +```bash +npm test # node:test, driving the extension through a stand-in for pi +npm run typecheck +scripts/release-tag.sh # bump, tag and push; the v* tag publishes to npm +``` diff --git a/index.test.ts b/index.test.ts new file mode 100644 index 0000000..b8522af --- /dev/null +++ b/index.test.ts @@ -0,0 +1,383 @@ +// Drives the extension the way pi does: a stand-in for pi records the handlers it registers, a +// temp directory holds real image files, the editor rewrite runs off a fake input box, and the +// entry the extension appends is rendered back through the renderer it registered. +// +// The load-bearing test is "the sent image is drawn below the message": pi renders no image +// blocks on user messages, so without the appended entry the transcript carries only the marker. +// Run with `npm test`. + +import assert from "node:assert/strict"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; + +import { setCapabilities } from "@earendil-works/pi-tui"; + +import imagePaths, { extractImages, resolveImageMarkers } from "./index"; +import type { ImageContent } from "./index"; + +/** 1x1 transparent PNG. */ +const PNG = + "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8DwHwAF" + + "AAH/q842iQAAAABJRU5ErkJggg=="; + +type Handler = (event: any, ctx: any) => any; +type Renderer = (entry: any, options: any, theme: any) => any; + +const THEME = { fg: (_color: string, text: string) => text, bold: (text: string) => text }; + +const tick = (ms = 20) => new Promise((resolve) => setTimeout(resolve, ms)); + +/** A directory with a screenshot, a screenshot whose name has a space, and a text file. */ +function fixture() { + const dir = mkdtempSync(join(tmpdir(), "pi-image-paths-")); + const shot = join(dir, "shot.png"); + const spaced = join(dir, "two words.png"); + writeFileSync(shot, Buffer.from(PNG, "base64")); + writeFileSync(spaced, Buffer.from(PNG, "base64")); + writeFileSync(join(dir, "notes.txt"), "not an image"); + return { dir, shot, spaced }; +} + +/** A stand-in for pi's extension API, plus the state a test needs to inspect. */ +function load(options: { showImages?: boolean; imageWidthCells?: number } = {}) { + const handlers = new Map(); + const entries: { type: string; data: any }[] = []; + const notices: string[] = []; + let entryType: string | undefined; + let renderer: Renderer | undefined; + let editorText = ""; + let terminalInput: ((data: string) => unknown) | undefined; + + const pi = { + on: (name: string, handler: Handler) => { + handlers.set(name, handler); + return () => handlers.delete(name); + }, + registerEntryRenderer: (type: string, fn: Renderer) => { + entryType = type; + renderer = fn; + }, + registerCommand: () => {}, + registerShortcut: () => {}, + getSettings: () => ({ + terminal: { + showImages: options.showImages ?? true, + imageWidthCells: options.imageWidthCells, + }, + }), + appendEntry: (type: string, data: unknown) => entries.push({ type, data }), + }; + + imagePaths(pi as unknown as Parameters[0]); + + return { + handlers, + entries, + notices, + entryType: () => entryType, + renderer: () => renderer as Renderer, + terminalInput: () => terminalInput, + editorText: () => editorText, + setEditorText: (text: string) => { + editorText = text; + }, + ctx: (cwd: string, mode = "tui") => ({ + mode, + hasUI: true, + cwd, + ui: { + onTerminalInput: (fn: (data: string) => unknown) => { + terminalInput = fn; + return () => { + terminalInput = undefined; + }; + }, + getEditorText: () => editorText, + setEditorText: (text: string) => { + editorText = text; + }, + notify: (message: string) => notices.push(message), + }, + }), + }; +} + +/** A registered handler, asserted present so tests fail loudly instead of silently skipping. */ +function handler(loaded: ReturnType, name: string): Handler { + const found = loaded.handlers.get(name); + assert.ok(found, `no ${name} handler registered`); + return found; +} + +/** The terminal-input listener the session_start handler subscribed. */ +function terminalInput(loaded: ReturnType): (data: string) => unknown { + const found = loaded.terminalInput(); + assert.ok(found, "no terminal input listener registered"); + return found; +} + +function userMessage(images = true) { + return { + role: "user", + content: images + ? [{ type: "text", text: "[Image #1] why is the header clipped?" }, { type: "image", data: PNG, mimeType: "image/png" }] + : [{ type: "text", text: "no pictures here" }], + }; +} + +// ---------------------------------------------------------------- paths in the prompt + +test("a bare absolute path becomes a marker and an attachment", () => { + const { dir, shot } = fixture(); + const extracted = extractImages(`look at ${shot} please`, dir); + assert.equal(extracted.text, "look at [Image #1] please"); + assert.deepEqual(extracted.files, [{ file: shot, ext: "png" }]); + assert.deepEqual(extracted.missing, []); +}); + +test("several paths number in order of appearance", () => { + const { dir, shot, spaced } = fixture(); + const extracted = extractImages(`${shot} and "${spaced}"`, dir); + // Quoting is the prompt's, so the marker lands inside it. + assert.equal(extracted.text, '[Image #1] and "[Image #2]"'); + assert.deepEqual( + extracted.files.map((f) => f.file), + [shot, spaced], + ); +}); + +test("a path with escaped spaces resolves as one file", () => { + const { dir, spaced } = fixture(); + const extracted = extractImages(`see ${spaced.replaceAll(" ", "\\ ")}`, dir); + assert.equal(extracted.text, "see [Image #1]"); + assert.deepEqual(extracted.files, [{ file: spaced, ext: "png" }]); +}); + +test("a path with unescaped spaces is joined and disambiguated against the filesystem", () => { + const { dir, spaced } = fixture(); + const extracted = extractImages(`see ${spaced} now`, dir); + assert.equal(extracted.text, "see [Image #1] now"); + assert.deepEqual(extracted.files, [{ file: spaced, ext: "png" }]); +}); + +test("startNumber offsets the first marker", () => { + const { dir, shot } = fixture(); + assert.equal(extractImages(shot, dir, 4).text, "[Image #4]"); +}); + +test("a missing image is reported, not swallowed", () => { + const { dir } = fixture(); + const missing = join(dir, "gone.png"); + const extracted = extractImages(`look at ${missing}`, dir); + assert.equal(extracted.text, `look at ${missing}`); + assert.deepEqual(extracted.files, []); + assert.deepEqual(extracted.missing, [missing]); +}); + +test("URLs and non-image extensions are left alone", () => { + const { dir } = fixture(); + const text = `https://example.com/a.png ${join(dir, "notes.txt")}`; + const extracted = extractImages(text, dir); + assert.equal(extracted.text, text); + assert.deepEqual(extracted.files, []); + assert.deepEqual(extracted.missing, []); +}); + +test("markers resolve through the registry, unknown ones stay literal", () => { + const registry = new Map([ + [1, { file: "/tmp/one.png", ext: "png" }], + [2, { file: "/tmp/two.png", ext: "png" }], + ]); + const resolved = resolveImageMarkers("[Image #2] then [Image #1] and [Image #7]", registry); + assert.equal(resolved.text, "[Image #1] then [Image #2] and [Image #7]"); + assert.deepEqual( + resolved.files.map((f) => f.file), + ["/tmp/two.png", "/tmp/one.png"], + ); +}); + +test("text without a marker short-circuits", () => { + assert.deepEqual(resolveImageMarkers("nothing here", new Map()), { + text: "nothing here", + files: [], + missing: [], + }); +}); + +// ---------------------------------------------------------------- the editor rewrite + +test("the live rewrite collapses a dropped path to a marker", async () => { + const { dir, shot } = fixture(); + const loaded = load(); + const ctx = loaded.ctx(dir); + handler(loaded, "session_start")({ type: "session_start" }, ctx); + + loaded.setEditorText(`look at ${shot}`); + terminalInput(loaded)("x"); + await tick(250); + + assert.equal(loaded.editorText(), "look at [Image #1]"); +}); + +test("submitting with Enter never triggers a rewrite", async () => { + const { dir, shot } = fixture(); + const loaded = load(); + const ctx = loaded.ctx(dir); + handler(loaded, "session_start")({ type: "session_start" }, ctx); + + loaded.setEditorText(`look at ${shot}`); + terminalInput(loaded)("\r"); + await tick(250); + + assert.equal(loaded.editorText(), `look at ${shot}`); +}); + +test("paste placeholders are left for pi to expand", async () => { + const { dir, shot } = fixture(); + const loaded = load(); + const ctx = loaded.ctx(dir); + handler(loaded, "session_start")({ type: "session_start" }, ctx); + + loaded.setEditorText(`[paste #1 +20 lines] ${shot}`); + terminalInput(loaded)("x"); + await tick(250); + + assert.equal(loaded.editorText(), `[paste #1 +20 lines] ${shot}`); +}); + +test("a rewritten marker still attaches on send", async () => { + const { dir, shot, spaced } = fixture(); + const loaded = load(); + const ctx = loaded.ctx(dir); + handler(loaded, "session_start")({ type: "session_start" }, ctx); + + loaded.setEditorText(`${shot} and ${spaced}`); + terminalInput(loaded)("x"); + await tick(250); + + const result = await handler(loaded, "input")( + { type: "input", text: loaded.editorText(), source: "interactive" }, + ctx, + ); + assert.equal(result.action, "transform"); + assert.equal(result.text, "[Image #1] and [Image #2]"); + assert.deepEqual( + result.images.map((image: ImageContent) => image.mimeType), + ["image/png", "image/png"], + ); +}); + +// ---------------------------------------------------------------- attaching on send + +test("the input handler attaches the file behind a path left in the prompt", async () => { + const { dir, shot } = fixture(); + const loaded = load(); + const result = await handler(loaded, "input")( + { type: "input", text: `look at ${shot}`, source: "interactive" }, + loaded.ctx(dir), + ); + assert.equal(result.action, "transform"); + assert.equal(result.text, "look at [Image #1]"); + assert.deepEqual(result.images, [{ type: "image", data: PNG, mimeType: "image/png" }]); +}); + +test("a path that does not exist warns and leaves the prompt alone", async () => { + const { dir } = fixture(); + const loaded = load(); + const missing = join(dir, "gone.png"); + const result = await handler(loaded, "input")( + { type: "input", text: `look at ${missing}`, source: "interactive" }, + loaded.ctx(dir), + ); + assert.equal(result.action, "continue"); + assert.equal(loaded.notices.length, 1); + assert.match(loaded.notices[0], /gone\.png/); +}); + +test("extension-injected input is never rewritten", async () => { + const { dir, shot } = fixture(); + const loaded = load(); + const result = await handler(loaded, "input")({ type: "input", text: shot, source: "extension" }, loaded.ctx(dir)); + assert.deepEqual(result, { action: "continue" }); +}); + +// ---------------------------------------------------------------- drawing the sent image + +test("the sent image is drawn below the message", async () => { + const { dir } = fixture(); + const loaded = load(); + const messageEnd = handler(loaded, "message_end"); + + messageEnd({ type: "message_end", message: userMessage() }, loaded.ctx(dir)); + assert.equal(loaded.entries.length, 0, "appended before the session stored the message"); + await tick(); + + assert.equal(loaded.entries.length, 1); + assert.equal(loaded.entries[0].type, loaded.entryType()); + assert.deepEqual(loaded.entries[0].data.images, [{ data: PNG, mimeType: "image/png" }]); +}); + +test("showImages off appends nothing while the image still reaches the model", async () => { + const { dir } = fixture(); + const loaded = load({ showImages: false }); + handler(loaded, "message_end")({ type: "message_end", message: userMessage() }, loaded.ctx(dir)); + await tick(); + assert.deepEqual(loaded.entries, []); +}); + +test("only user messages with images get an entry", async () => { + const { dir } = fixture(); + const loaded = load(); + const messageEnd = handler(loaded, "message_end"); + + messageEnd({ type: "message_end", message: userMessage(false) }, loaded.ctx(dir)); + messageEnd({ type: "message_end", message: { role: "assistant", content: [{ type: "text", text: "hi" }] } }, loaded.ctx(dir)); + messageEnd({ type: "message_end", message: userMessage() }, loaded.ctx(dir, "rpc")); + await tick(); + + assert.deepEqual(loaded.entries, []); +}); + +test("the renderer draws the image in the transcript", () => { + setCapabilities({ images: "kitty", trueColor: true, hyperlinks: false }); + const loaded = load({ imageWidthCells: 40 }); + const entry = { + type: "custom", + customType: loaded.entryType(), + data: { images: [{ data: PNG, mimeType: "image/png" }] }, + }; + + const component = loaded.renderer()(entry, { expanded: false }, THEME); + assert.ok(component, "no component for a drawable entry"); + const lines = component.render(40); + assert.ok(lines.length > 0); + assert.ok( + lines.some((line: string) => line.includes("\x1b_G")), + "no kitty image escape in the rendered lines", + ); +}); + +test("the renderer stays out of the way when drawing is off or impossible", () => { + const entry = { + type: "custom", + customType: "image-paths:attached", + data: { images: [{ data: PNG, mimeType: "image/png" }] }, + }; + + setCapabilities({ images: "kitty", trueColor: true, hyperlinks: false }); + const off = load({ showImages: false }); + assert.equal(off.renderer()(entry, { expanded: false }, THEME), undefined); + + setCapabilities({ images: null, trueColor: true, hyperlinks: false }); + const terminal = load(); + assert.equal(terminal.renderer()(entry, { expanded: false }, THEME), undefined); + + setCapabilities({ images: "kitty", trueColor: true, hyperlinks: false }); + const empty = load(); + assert.equal( + empty.renderer()({ ...entry, data: {} }, { expanded: false }, THEME), + undefined, + ); +}); diff --git a/index.ts b/index.ts new file mode 100644 index 0000000..825e947 --- /dev/null +++ b/index.ts @@ -0,0 +1,474 @@ +/** + * image-paths — turn image file paths in chat input into attached images. + * + * 1. Live editor rewrite (TUI): the moment the input box ends with a path + * that resolves to an existing image file, the path is rewritten in + * place to "[Image #N]" — the same way pi collapses big pastes into + * "[paste #N +M lines]". + * + * 2. Send-time fallback: any image path still present in the prompt when + * it is submitted (un-rewritten, pasted whole, RPC mode, ...) is + * replaced with "[Image #N]" and the image is attached to the message. + * + * 3. Transcript render: once the message is sent, the attached images are + * drawn below it in the chat, honouring the `terminal.showImages` and + * `terminal.imageWidthCells` settings. Pi does not render image blocks on + * user messages itself, so the images ride in a custom session entry that + * stays out of the model context. + * + * Handles backslash-escaped spaces, quoted paths, ~ expansion, bare + * filenames relative to cwd, and paths containing unescaped spaces + * (disambiguated against the filesystem). + */ + +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as pathMod from "node:path"; +import type { + ExtensionAPI, + ExtensionContext, +} from "@earendil-works/pi-coding-agent"; +import { Container, getCapabilities, Image, Spacer } from "@earendil-works/pi-tui"; + +export interface ImageContent { + type: "image"; + data: string; + mimeType: string; +} + +const IMAGE_EXT = /\.(png|jpe?g|gif|webp|bmp)$/i; + +const MIME_TYPES: Record = { + png: "image/png", + jpg: "image/jpeg", + jpeg: "image/jpeg", + gif: "image/gif", + webp: "image/webp", + bmp: "image/bmp", +}; + +/** How many preceding whitespace-separated words may be joined into one path. */ +const MAX_JOIN = 6; + +/** Debounce for the live editor rewrite. */ +const REWRITE_DEBOUNCE_MS = 120; + +/** Custom entry that displays a sent message's images; never enters model context. */ +const DISPLAY_ENTRY_TYPE = "image-paths:attached"; + +/** `terminal.imageWidthCells` default, used when the setting is unset. */ +const DEFAULT_IMAGE_WIDTH_CELLS = 60; + +interface AttachedImages { + images: { data: string; mimeType: string }[]; +} + +/** Image blocks carried by a message, in order. */ +function attachedImages(message: { content: unknown }): AttachedImages["images"] { + if (!Array.isArray(message.content)) return []; + const images: AttachedImages["images"] = []; + for (const block of message.content) { + if (!block || typeof block !== "object") continue; + const { type, data, mimeType } = block as Partial; + if (type === "image" && data && mimeType) images.push({ data, mimeType }); + } + return images; +} + +interface Token { + text: string; + start: number; + end: number; +} + +/** + * Split text into whitespace-separated tokens, treating a whitespace char + * that is preceded by a backslash as part of the token (shell-style escaping). + * All whitespace splits — including newlines — so multi-file drops and + * multi-line prompts resolve per line. + */ +function tokenize(text: string): Token[] { + const tokens: Token[] = []; + let start = -1; + for (let i = 0; i <= text.length; i++) { + const ch = i < text.length ? text[i] : " "; + const escapedSpace = /\s/.test(ch) && text[i - 1] === "\\"; + const boundary = i === text.length || (/\s/.test(ch) && !escapedSpace); + if (!boundary && start === -1) start = i; + if (boundary && start !== -1) { + tokens.push({ text: text.slice(start, i), start, end: i }); + start = -1; + } + } + return tokens; +} + +/** Strip surrounding quotes and unescape shell backslash escapes. */ +function cleanToken(raw: string): string { + let s = raw.trim(); + if (s.length > 1 && (s.startsWith('"') || s.startsWith("'"))) s = s.slice(1); + if (s.length > 1 && (s.endsWith('"') || s.endsWith("'"))) s = s.slice(0, -1); + return s.replace(/\\([ '"()&;[\]])/g, "$1"); +} + +function expandTilde(p: string): string { + if (p === "~") return os.homedir(); + if (p.startsWith("~/")) return pathMod.join(os.homedir(), p.slice(2)); + return p; +} + +function looksPathish(p: string): boolean { + return ( + p.startsWith("/") || + p.startsWith("~") || + p.startsWith("./") || + p.startsWith("../") || + /^[A-Za-z]:[\\/]/.test(p) + ); +} + +function toAbs(p: string, cwd: string): string { + // Some terminals drop files as file:// URIs. + if (p.startsWith("file://")) { + try { + p = decodeURIComponent(p.slice(7)); + } catch { + // Malformed escape: fall through with the raw remainder. + } + if (!p.startsWith("/")) p = "/" + p; + } + const expanded = expandTilde(p); + return pathMod.isAbsolute(expanded) + ? expanded + : pathMod.resolve(cwd, expanded); +} + +type Resolution = + | { kind: "found"; start: number; end: number; file: string; ext: string } + | { kind: "missing"; path: string } + | { kind: "ignore" }; + +/** + * Try to resolve the token at `index` (which ends with an image extension) + * into an existing image file, optionally by joining up to MAX_JOIN + * preceding tokens (for paths with unescaped spaces). + */ +function resolveImagePath( + tokens: Token[], + index: number, + text: string, + cwd: string, +): Resolution { + // Skip URLs and tokens that don't actually end in an image extension + // once trailing punctuation is stripped. + const base = cleanToken(tokens[index].text).replace(/[.,;:!?)\]]+$/, ""); + if (!IMAGE_EXT.test(base) || /^(https?|ftp):/i.test(base)) + return { kind: "ignore" }; + + let lastPathish: string | undefined; + for (let k = 0; k <= MAX_JOIN && index - k >= 0; k++) { + const first = tokens[index - k]; + const last = tokens[index]; + const candidate = cleanToken(text.slice(first.start, last.end)).replace( + /[.,;:!?)\]]+$/, + "", + ); + if (candidate.includes("\n")) break; // never join across lines + if (looksPathish(candidate)) lastPathish = candidate; + + const full = toAbs(candidate, cwd); + try { + if (fs.statSync(full).isFile()) { + return { + kind: "found", + start: first.start, + end: last.end, + file: full, + ext: IMAGE_EXT.exec(candidate)![1].toLowerCase(), + }; + } + } catch { + // Not an existing file at this join width; try joining more words. + } + } + + if (lastPathish) return { kind: "missing", path: lastPathish }; + return { kind: "ignore" }; +} + +interface PathMatch { + start: number; + end: number; + file: string; + ext: string; +} + +function findImagePaths( + text: string, + cwd: string, +): { found: PathMatch[]; missing: string[] } { + const tokens = tokenize(text); + const found: PathMatch[] = []; + const missing: string[] = []; + let lastEnd = -1; + + for (let i = 0; i < tokens.length; i++) { + if (tokens[i].start < lastEnd) continue; // already consumed by an earlier join + const base = cleanToken(tokens[i].text).replace(/[.,;:!?)\]]+$/, ""); + if (!IMAGE_EXT.test(base)) continue; + + const resolved = resolveImagePath(tokens, i, text, cwd); + if (resolved.kind === "ignore") continue; + if (resolved.kind === "missing") { + missing.push(resolved.path); + continue; + } + lastEnd = resolved.end; + // Leave balanced surrounding quotes outside the match range. + let start = resolved.start; + let end = resolved.end; + const raw = text.slice(start, end).trim(); + if ( + raw.length > 1 && + ((raw.startsWith('"') && raw.endsWith('"')) || + (raw.startsWith("'") && raw.endsWith("'"))) + ) { + start += 1; + end -= 1; + } + found.push({ start, end, file: resolved.file, ext: resolved.ext }); + } + + return { found, missing }; +} + +export interface ExtractedImages { + /** Prompt text with each resolved path replaced by "[Image #N]". */ + text: string; + /** Resolved files in order of appearance. */ + files: { file: string; ext: string }[]; + /** Path-looking candidates that did not exist on disk. */ + missing: string[]; +} + +/** Pure logic: find image paths in text and compute the transformed prompt. */ +export function extractImages( + text: string, + cwd: string, + startNumber = 1, +): ExtractedImages { + const { found, missing } = findImagePaths(text, cwd); + + let out = ""; + let pos = 0; + found.forEach((f, i) => { + out += text.slice(pos, f.start) + `[Image #${startNumber + i}]`; + pos = f.end; + }); + out += text.slice(pos); + + return { + text: out, + files: found.map((f) => ({ file: f.file, ext: f.ext })), + missing, + }; +} + +/** + * Pure logic: resolve trailing "[Image #N]" markers written by the live + * editor rewrite into files, renumbering sequentially from 1. Unknown + * markers (never registered by this session) are left as literal text. + */ +export function resolveImageMarkers( + text: string, + registry: Map, +): ExtractedImages { + if (!text.includes("[Image #")) return { text, files: [], missing: [] }; + + const files: { file: string; ext: string }[] = []; + let out = ""; + let pos = 0; + for (const m of text.matchAll(/\[Image #(\d+)\]/g)) { + const entry = registry.get(Number(m[1])); + if (!entry) continue; + files.push(entry); + out += text.slice(pos, m.index) + `[Image #${files.length}]`; + pos = m.index + m[0].length; + } + out += text.slice(pos); + return { text: out, files, missing: [] }; +} + +export default function (pi: ExtensionAPI) { + // Session-scoped registry of live-rewritten markers: id -> file. + const registry = new Map(); + let nextMarkerId = 0; + let unsubscribeInput: (() => void) | undefined; + let rewriteTimer: ReturnType | undefined; + const displayTimers = new Set>(); + + pi.registerEntryRenderer( + DISPLAY_ENTRY_TYPE, + (entry, _options, theme) => { + const images = entry.data?.images ?? []; + if (images.length === 0) return undefined; + // Settings are live, so turning images off hides entries already in + // the transcript the next time they are rebuilt. + const settings = pi.getSettings(); + if (!(settings.terminal?.showImages ?? true)) return undefined; + if (!getCapabilities().images) return undefined; + const maxWidthCells = + settings.terminal?.imageWidthCells ?? DEFAULT_IMAGE_WIDTH_CELLS; + + const container = new Container(); + for (const image of images) { + container.addChild(new Spacer(1)); + container.addChild( + new Image( + image.data, + image.mimeType, + { fallbackColor: (text) => theme.fg("dim", text) }, + { maxWidthCells }, + ), + ); + } + return container; + }, + ); + + pi.on("session_start", (_event, ctx) => { + nextMarkerId = 0; + registry.clear(); + if (unsubscribeInput) { + unsubscribeInput(); + unsubscribeInput = undefined; + } + // Live rewrite only makes sense with a real input box. + if (ctx.mode !== "tui") return; + + unsubscribeInput = ctx.ui.onTerminalInput((data) => { + // Enter submits the message — never schedule a rewrite for it. + if (data !== "\r" && data !== "\n") scheduleRewrite(ctx); + return undefined; + }); + }); + + pi.on("session_shutdown", () => { + if (rewriteTimer) { + clearTimeout(rewriteTimer); + rewriteTimer = undefined; + } + for (const timer of displayTimers) clearTimeout(timer); + displayTimers.clear(); + if (unsubscribeInput) { + unsubscribeInput(); + unsubscribeInput = undefined; + } + }); + + function scheduleRewrite(ctx: ExtensionContext) { + if (rewriteTimer) return; + rewriteTimer = setTimeout(() => { + rewriteTimer = undefined; + // Re-read the editor at fire time: pastes (drag & drop included) + // land in the editor *after* this listener saw the raw sequence. + const current = ctx.ui.getEditorText(); + if (!current || current.includes("[paste #")) return; + + // Rewrite EVERY resolvable path in one pass, so dropping several + // files at once flips them all to markers immediately. + const { found } = findImagePaths(current, ctx.cwd); + if (found.length === 0) return; + + let out = ""; + let pos = 0; + for (const f of found) { + const id = ++nextMarkerId; + registry.set(id, { file: f.file, ext: f.ext }); + out += current.slice(pos, f.start) + `[Image #${id}]`; + pos = f.end; + } + out += current.slice(pos); + ctx.ui.setEditorText(out); + }, REWRITE_DEBOUNCE_MS); + } + + // Pi renders image blocks on tool results but not on user messages, so the + // attached images are re-drawn from a session entry appended right after the + // message lands. Entries are excluded from the model context. + pi.on("message_end", (event, ctx) => { + if (ctx.mode !== "tui") return; + if (event.message.role !== "user") return; + const images = attachedImages(event.message); + if (images.length === 0) return; + if (!(pi.getSettings().terminal?.showImages ?? true)) return; + + // The session appends the user message after this handler returns, so + // wait a macrotask to place the images directly below it. + const timer = setTimeout(() => { + displayTimers.delete(timer); + pi.appendEntry(DISPLAY_ENTRY_TYPE, { images }); + }, 0); + displayTimers.add(timer); + }); + + pi.on("input", async (event, ctx) => { + // Never touch extension-injected messages. + if (event.source === "extension") return { action: "continue" }; + if (!event.text.includes(".") && !event.text.includes("[Image #")) { + return { action: "continue" }; + } + + // 1. Resolve markers written by the live editor rewrite. + const markers = resolveImageMarkers(event.text, registry); + + // 2. Fall back to path detection for anything still un-rewritten. + const extracted = extractImages( + markers.text, + ctx.cwd, + markers.files.length + 1, + ); + + const allFiles = [...markers.files, ...extracted.files]; + if (allFiles.length === 0) { + if (extracted.missing.length > 0 && ctx.hasUI) { + ctx.ui.notify( + `Image file not found: ${extracted.missing.join(", ")}`, + "warning", + ); + } + return { action: "continue" }; + } + + const images: ImageContent[] = []; + try { + for (const f of allFiles) { + const data = await fs.promises.readFile(f.file); + images.push({ + type: "image", + data: data.toString("base64"), + mimeType: MIME_TYPES[f.ext] ?? "application/octet-stream", + }); + } + } catch (err) { + // File vanished or became unreadable between stat and read: send the + // original prompt untouched rather than a half-attached message. + if (ctx.hasUI) { + ctx.ui.notify( + `Failed to read image, sending path as text: ${err}`, + "warning", + ); + } + return { action: "continue" }; + } + + if (extracted.missing.length > 0 && ctx.hasUI) { + ctx.ui.notify( + `Image file not found: ${extracted.missing.join(", ")}`, + "warning", + ); + } + + return { action: "transform", text: extracted.text, images }; + }); +} diff --git a/package.json b/package.json new file mode 100644 index 0000000..6f23fda --- /dev/null +++ b/package.json @@ -0,0 +1,44 @@ +{ + "name": "@mikefreno/pi-image-paths", + "version": "0.1.0", + "description": "Image file paths in pi's chat input become attached images — the path collapses to a marker as you type, and the picture is drawn below the sent message.", + "keywords": [ + "pi-package", + "pi-extension", + "images", + "screenshot", + "drag-and-drop", + "tui" + ], + "author": "Michael Freno", + "license": "MIT", + "homepage": "https://git.freno.me/Mike/pi-image-paths", + "repository": { + "type": "git", + "url": "git+https://git.freno.me/Mike/pi-image-paths.git" + }, + "bugs": { + "url": "https://git.freno.me/Mike/pi-image-paths/issues" + }, + "files": [ + "index.ts", + "README.md", + "LICENSE" + ], + "scripts": { + "test": "JITI_ALIAS='{\"@earendil-works/pi-tui\":\"/opt/homebrew/lib/node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-tui/dist/index.js\"}' node /opt/homebrew/lib/node_modules/@earendil-works/pi-coding-agent/node_modules/jiti/lib/jiti-cli.mjs index.test.ts", + "typecheck": "tsc -p ." + }, + "pi": { + "extensions": [ + "./index.ts" + ] + }, + "peerDependencies": { + "@earendil-works/pi-coding-agent": "*", + "@earendil-works/pi-tui": "*" + }, + "publishConfig": { + "access": "public" + } +} diff --git a/scripts/release-tag.sh b/scripts/release-tag.sh new file mode 100755 index 0000000..5c353f9 --- /dev/null +++ b/scripts/release-tag.sh @@ -0,0 +1,184 @@ +#!/bin/bash + +# release-tag.sh — version bump, commit, tag, and push for this pi extension. +# +# Mirrors the release flow from PodTui's scripts/release-tag.sh, adapted for +# the pi extension repos: the version lives in package.json. +# +# Usage: +# scripts/release-tag.sh interactive release +# scripts/release-tag.sh --dry-run plan the bump/tag/pushes without doing +# +# After this script runs, pushing the v* tag triggers the publish workflow, +# which npm-publishes this package. + +set -e + +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +NC='\033[0m' # No Color + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +cd "$PROJECT_ROOT" + +DRY_RUN=0 +for arg in "$@"; do + case "$arg" in + --dry-run) DRY_RUN=1 ;; + -h | --help) + echo "usage: $0 [--dry-run]" + exit 0 + ;; + esac +done + +if [ ! -d .git ] && [ ! -f .git ]; then + echo -e "${RED}✗ not a git repository: $PROJECT_ROOT${NC}" + exit 1 +fi + +if ! git diff-index --quiet HEAD --; then + echo -e "${RED}✗ working tree is dirty — commit or stash before releasing${NC}" + git status --short + exit 1 +fi + +NAME=$(python3 -c "import json;print(json.load(open('package.json'))['name'])") +CURRENT_VERSION=$(python3 -c "import json;print(json.load(open('package.json'))['version'])") + +echo -e "${CYAN}Package:${NC} ${GREEN}${NAME}${NC}" +echo -e "${CYAN}Current version:${NC} ${GREEN}v${CURRENT_VERSION}${NC}" +echo "" + +IFS='.' read -r MAJOR MINOR PATCH <<<"$CURRENT_VERSION" +MAJOR=$(echo "$MAJOR" | sed 's/[^0-9].*//') +MINOR=$(echo "$MINOR" | sed 's/[^0-9].*//') +PATCH=$(echo "$PATCH" | sed 's/[^0-9].*//') + +echo -e "${CYAN}Select version bump type:${NC}" +echo " 1) Major (breaking changes) ${MAJOR}.${MINOR}.${PATCH} → $((MAJOR + 1)).0.0" +echo " 2) Minor (new features) ${MAJOR}.${MINOR}.${PATCH} → ${MAJOR}.$((MINOR + 1)).0" +echo " 3) Patch (bug fixes) ${MAJOR}.${MINOR}.${PATCH} → ${MAJOR}.${MINOR}.$((PATCH + 1))" +echo " 4) Custom version" +echo " 5) Cancel" +echo "" +read -p "Enter choice (1-5): " -n 1 -r CHOICE +echo "" +echo "" + +case $CHOICE in +1) NEW_VERSION="$((MAJOR + 1)).0.0" ;; +2) NEW_VERSION="${MAJOR}.$((MINOR + 1)).0" ;; +3) NEW_VERSION="${MAJOR}.${MINOR}.$((PATCH + 1))" ;; +4) + read -p "Enter new version (e.g. 0.6.1): " -r NEW_VERSION + echo "" + ;; +5) + echo -e "${YELLOW}Cancelled.${NC}" + exit 0 + ;; +*) + echo -e "${RED}✗ invalid choice${NC}" + exit 1 + ;; +esac + +if ! echo "$NEW_VERSION" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+$'; then + echo -e "${RED}✗ version must be MAJOR.MINOR.PATCH (got: ${NEW_VERSION})${NC}" + exit 1 +fi + +if git rev-parse -q --verify "refs/tags/v${NEW_VERSION}" >/dev/null; then + echo -e "${RED}✗ tag v${NEW_VERSION} already exists${NC}" + exit 1 +fi + +echo -e "${CYAN}New version:${NC} ${GREEN}v${NEW_VERSION}${NC}" +echo "" +echo -e "${YELLOW}This will:${NC}" +echo " 1. Set package.json → \"version\": \"${NEW_VERSION}\"" +echo " 2. Commit the bump" +echo " 3. Create annotated tag v${NEW_VERSION}" +echo " 4. Push master and the tag to every remote" +REMOTES=$(git remote) +for r in $REMOTES; do + echo " → $r" +done +echo "" +echo -e "${YELLOW}Note: the v* tag push triggers the npm publish workflow.${NC}" +echo "" +read -p "Proceed? (y/n) " -n 1 -r +echo "" +if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo -e "${YELLOW}Aborted.${NC}" + exit 0 +fi + +if [ "$DRY_RUN" -eq 1 ]; then + echo -e "${BLUE}── dry run: no changes made ──${NC}" + exit 0 +fi + +# ── Apply the bump ─────────────────────────────────────────────────────────── +echo "" +echo -e "${CYAN}[1/4]${NC} Updating package.json..." +sed -i.bak "s/\"version\": \"[^\"]*\"/\"version\": \"${NEW_VERSION}\"/" package.json +rm -f package.json.bak +echo -e "${GREEN}✓ package.json updated${NC}" + +if git diff --quiet -- package.json; then + echo -e "${RED}✗ package.json did not change${NC}" + exit 1 +fi + +echo -e "${CYAN}[2/4]${NC} Committing..." +git add package.json +git commit -m "chore: bump version to v${NEW_VERSION}" +echo -e "${GREEN}✓ committed${NC}" + +echo -e "${CYAN}[3/4]${NC} Tagging..." +git tag -a "v${NEW_VERSION}" -m "${NAME} v${NEW_VERSION}" +echo -e "${GREEN}✓ tagged v${NEW_VERSION}${NC}" +echo "" + +echo -e "${CYAN}[4/4]${NC} Pushing..." +FAILED="" +for r in $REMOTES; do + echo -e " → ${BLUE}${r}${NC} master..." + if git push "$r" master >/dev/null 2>&1; then + echo -e " ${GREEN}✓ master pushed${NC}" + else + echo -e " ${RED}✗ master push failed${NC}" + FAILED="$FAILED $r" + continue + fi + echo -e " → ${BLUE}${r}${NC} v${NEW_VERSION}..." + if git push "$r" "v${NEW_VERSION}" >/dev/null 2>&1; then + echo -e " ${GREEN}✓ tag pushed${NC}" + else + echo -e " ${RED}✗ tag push failed${NC}" + FAILED="$FAILED $r" + fi +done + +echo "" +if [ -n "$FAILED" ]; then + echo -e "${RED}✗ push failed for remote(s):${FAILED}${NC}" + echo -e "${YELLOW}Re-run: git push master && git push v${NEW_VERSION}${NC}" + exit 1 +fi + +echo -e "${GREEN}═══════════════════════════════════════${NC}" +echo -e "${GREEN}✓ ${NAME} v${NEW_VERSION} released${NC}" +echo -e "${GREEN}═══════════════════════════════════════${NC}" +echo "" +echo -e "${CYAN}Version:${NC} ${CURRENT_VERSION} → ${GREEN}${NEW_VERSION}${NC}" +echo -e "${CYAN}Tag:${NC} v${NEW_VERSION}" +echo "" +echo -e "${BLUE}Next step (automatic, nothing to do):${NC}" +echo " Gitea Actions publish npm-publishes ${NAME} to the npm registry" diff --git a/tsconfig.json b/tsconfig.json new file mode 100644 index 0000000..6bcfee8 --- /dev/null +++ b/tsconfig.json @@ -0,0 +1,29 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "bundler", + "strict": true, + "noEmit": true, + "skipLibCheck": true, + "paths": { + "@earendil-works/pi-coding-agent": [ + "/opt/homebrew/lib/node_modules/@earendil-works/pi-coding-agent/dist/index.d.ts" + ], + "@earendil-works/pi-agent-core": [ + "/opt/homebrew/lib/node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-agent-core/dist/index.d.ts" + ], + "@earendil-works/pi-ai": [ + "/opt/homebrew/lib/node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-ai/dist/index.d.ts" + ], + "@earendil-works/pi-tui": [ + "/opt/homebrew/lib/node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-tui/dist/index.d.ts" + ], + "typebox": [ + "/opt/homebrew/lib/node_modules/@earendil-works/pi-coding-agent/node_modules/typebox" + ] + }, + "baseUrl": "." + }, + "include": ["*.ts"] +}