diff --git a/packages/coding-agent/UNRELEASED_OLD.md b/packages/coding-agent/UNRELEASED_OLD.md deleted file mode 100644 index 3ac1728a..00000000 --- a/packages/coding-agent/UNRELEASED_OLD.md +++ /dev/null @@ -1,64 +0,0 @@ - -### Breaking Changes - -- **Session tree structure (v2)**: Sessions now store entries as a tree with `id`/`parentId` fields, enabling in-place branching without creating new files. Existing v1 sessions are auto-migrated on load. -- **SessionManager API**: - - `saveXXX()` renamed to `appendXXX()` (e.g., `appendMessage`, `appendCompaction`) - - `branchInPlace()` renamed to `branch()` - - `reset()` renamed to `newSession()` - - `createBranchedSessionFromEntries(entries, index)` replaced with `createBranchedSession(leafId)` - - `saveCompaction(entry)` replaced with `appendCompaction(summary, firstKeptEntryId, tokensBefore)` - - `getEntries()` now excludes the session header (use `getHeader()` separately) - - New methods: `getTree()`, `getPath()`, `getLeafUuid()`, `getLeafEntry()`, `getEntry()`, `branchWithSummary()` - - New `appendCustomEntry(customType, data)` for hooks to store custom data (not in LLM context) - - New `appendCustomMessageEntry(customType, content, display, details?)` for hooks to inject messages into LLM context -- **Compaction API**: - - `CompactionEntry` and `CompactionResult` are now generic with optional `details?: T` for hook-specific data - - `compact()` now returns `CompactionResult` (`{ summary, firstKeptEntryId, tokensBefore, details? }`) instead of `CompactionEntry` - - `appendCompaction()` now accepts optional `details` parameter - - `CompactionEntry.firstKeptEntryIndex` replaced with `firstKeptEntryId` - - `prepareCompaction()` now returns `firstKeptEntryId` in its result -- **Hook types**: - - `SessionEventBase` no longer has `sessionManager`/`modelRegistry` - access them via `HookEventContext` instead - - `HookEventContext` now has `sessionManager` and `modelRegistry` (moved from events) - - `HookEventContext` no longer has `exec()` - use `pi.exec()` instead - - `HookCommandContext` no longer has `exec()` - use `pi.exec()` instead - - `before_compact` event passes `preparation: CompactionPreparation` and `previousCompactions: CompactionEntry[]` (newest first) - - `before_switch` event now has `targetSessionFile`, `switch` event has `previousSessionFile` - - Removed `resolveApiKey` (use `modelRegistry.getApiKey(model)`) - - Hooks can return `compaction.details` to store custom data (e.g., ArtifactIndex for structured compaction) -- **Hook API**: - - `pi.send(text, attachments?)` replaced with `pi.sendMessage(message, triggerTurn?)` which creates `CustomMessageEntry` instead of user messages - - New `pi.appendEntry(customType, data?)` to persist hook state (does NOT participate in LLM context) - - New `pi.registerCommand(name, options)` to register custom slash commands - - New `pi.registerMessageRenderer(customType, renderer)` to register custom renderers for hook messages - - New `pi.exec(command, args, options?)` to execute shell commands (moved from `HookEventContext`/`HookCommandContext`) - - `HookMessageRenderer` type: `(message: HookMessage, options, theme) => Component | null` - - Renderers return inner content; the TUI wraps it in a styled Box - - New types: `HookMessage`, `RegisteredCommand`, `HookCommandContext` - - Handler types renamed: `SendHandler` → `SendMessageHandler`, new `AppendEntryHandler` -- **SessionManager**: - - `getSessionFile()` now returns `string | undefined` (undefined for in-memory sessions) -- **Themes**: Custom themes must add `customMessageBg`, `customMessageText`, `customMessageLabel` color tokens - -### Added - -- **`enabledModels` setting**: Configure whitelisted models in `settings.json` (same format as `--models` CLI flag). CLI `--models` takes precedence over the setting. - -### Changed - -- **Entry IDs**: Session entries now use short 8-character hex IDs instead of full UUIDs -- **API key priority**: `ANTHROPIC_OAUTH_TOKEN` now takes precedence over `ANTHROPIC_API_KEY` -- **New entry types**: `BranchSummaryEntry` for branch context, `CustomEntry` for hook state persistence, `CustomMessageEntry` for hook-injected context messages, `LabelEntry` for user-defined bookmarks -- **Entry labels**: New `getLabel(id)` and `appendLabelChange(targetId, label)` methods for labeling entries. Labels are included in `SessionTreeNode` for UI/export. -- **TUI**: `CustomMessageEntry` renders with purple styling (customMessageBg, customMessageText, customMessageLabel theme colors). Entries with `display: false` are hidden. -- **AgentSession**: New `sendHookMessage(message, triggerTurn?)` method for hooks to inject messages. Handles queuing during streaming, direct append when idle, and optional turn triggering. -- **HookMessage**: New message type with `role: "hookMessage"` for hook-injected messages in agent events. Use `isHookMessage(msg)` type guard to identify them. These are converted to user messages for LLM context via `messageTransformer`. -- **Agent.prompt()**: Now accepts `AppMessage` directly (in addition to `string, attachments?`) for custom message types like `HookMessage`. - -### Fixed - -- **Edit tool fails on Windows due to CRLF line endings**: Files with CRLF line endings now match correctly when LLMs send LF-only text. Line endings are normalized before matching and restored to original style on write. ([#355](https://github.com/badlogic/pi-mono/issues/355)) -- **Session file validation**: `findMostRecentSession()` now validates session headers before returning, preventing non-session JSONL files from being loaded -- **Compaction error handling**: `generateSummary()` and `generateTurnPrefixSummary()` now throw on LLM errors instead of returning empty strings - diff --git a/packages/coding-agent/docs/hooks-v2.md b/packages/coding-agent/docs/hooks-v2.md deleted file mode 100644 index 287584b6..00000000 --- a/packages/coding-agent/docs/hooks-v2.md +++ /dev/null @@ -1,385 +0,0 @@ -# Hooks v2: Context Control + Commands - -Issue: #289 - -## Motivation - -Enable features like session stacking (`/pop`) as hooks, not core code. Core provides primitives, hooks implement features. - -## Primitives - -| Primitive | Purpose | -|-----------|---------| -| `ctx.saveEntry({type, ...})` | Persist custom entry to session | -| `pi.on("context", handler)` | Transform messages before LLM | -| `ctx.rebuildContext()` | Trigger context rebuild | -| `pi.command(name, opts)` | Register slash command | - -## Extended HookEventContext - -```typescript -interface HookEventContext { - // Existing - exec, ui, hasUI, cwd, sessionFile - - // State (read-only) - model: Model | null; - thinkingLevel: ThinkingLevel; - entries: readonly SessionEntry[]; - - // Utilities - findModel(provider: string, id: string): Model | null; - availableModels(): Promise[]>; - resolveApiKey(model: Model): Promise; - - // Mutation - saveEntry(entry: { type: string; [k: string]: unknown }): Promise; - rebuildContext(): Promise; -} - -interface ContextMessage { - message: AppMessage; - entryIndex: number | null; // null = synthetic -} - -interface ContextEvent { - type: "context"; - entries: readonly SessionEntry[]; - messages: ContextMessage[]; -} -``` - -Commands also get: `args`, `argsRaw`, `signal`, `setModel()`, `setThinkingLevel()`. - -## Stacking: Design - -### Entry Format - -```typescript -interface StackPopEntry { - type: "stack_pop"; - backToIndex: number; - summary: string; - prePopSummary?: string; // when crossing compaction - timestamp: number; -} -``` - -### Crossing Compaction - -Entries are never deleted. Raw data always available. - -When `backToIndex < compaction.firstKeptEntryIndex`: -1. Read raw entries `[0, backToIndex)` → summarize → `prePopSummary` -2. Read raw entries `[backToIndex, now)` → summarize → `summary` - -### Context Algorithm: Later Wins - -Assign sequential IDs to ranges. On overlap, highest ID wins. - -``` -Compaction at 40: range [0, 30) id=0 -StackPop at 50, backTo=20, prePopSummary: ranges [0, 20) id=1, [20, 50) id=2 - -Index 0-19: id=0 and id=1 cover → id=1 wins (prePopSummary) -Index 20-29: id=0 and id=2 cover → id=2 wins (popSummary) -Index 30-49: id=2 covers → id=2 (already emitted at 20) -Index 50+: no coverage → include as messages -``` - -## Complex Scenario Trace - -``` -Initial: [msg1, msg2, msg3, msg4, msg5] - idx: 1, 2, 3, 4, 5 - -Compaction triggers: - [msg1-5, compaction{firstKept:4, summary:C1}] - idx: 1-5, 6 - Context: [C1, msg4, msg5] - -User continues: - [..., compaction, msg4, msg5, msg6, msg7] - idx: 6, 4*, 5*, 7, 8 (* kept from before) - -User does /pop to msg2 (index 2): - - backTo=2 < firstKept=4 → crossing! - - prePopSummary: summarize raw [0,2) → P1 - - summary: summarize raw [2,8) → S1 - - save: stack_pop{backTo:2, summary:S1, prePopSummary:P1} at index 9 - - Ranges: - compaction [0,4) id=0 - prePopSummary [0,2) id=1 - popSummary [2,9) id=2 - - Context build: - idx 0: covered by id=0,1 → id=1 wins, emit P1 - idx 1: covered by id=0,1 → id=1 (already emitted) - idx 2: covered by id=0,2 → id=2 wins, emit S1 - idx 3-8: covered by id=0 or id=2 → id=2 (already emitted) - idx 9: stack_pop entry, skip - idx 10+: not covered, include as messages - - Result: [P1, S1, msg10+] - -User continues, another compaction: - [..., stack_pop, msg10, msg11, msg12, compaction{firstKept:11, summary:C2}] - idx: 9, 10, 11, 12, 13 - - Ranges: - compaction@6 [0,4) id=0 - prePopSummary [0,2) id=1 - popSummary [2,9) id=2 - compaction@13 [0,11) id=3 ← this now covers previous ranges! - - Context build: - idx 0-10: covered by multiple, id=3 wins → emit C2 at idx 0 - idx 11+: include as messages - - Result: [C2, msg11, msg12] - - C2's summary text includes info from P1 and S1 (they were in context when C2 was generated). -``` - -The "later wins" rule naturally handles all cases. - -## Core Changes - -| File | Change | -|------|--------| -| `session-manager.ts` | `saveEntry()`, `buildSessionContext()` returns `ContextMessage[]` | -| `hooks/types.ts` | `ContextEvent`, `ContextMessage`, extended context, command types | -| `hooks/loader.ts` | Track commands | -| `hooks/runner.ts` | `setStateCallbacks()`, `emitContext()`, command methods | -| `agent-session.ts` | `saveEntry()`, `rebuildContext()`, state callbacks | -| `interactive-mode.ts` | Command handling, autocomplete | - -## Stacking Hook: Complete Implementation - -```typescript -import { complete } from "@mariozechner/pi-ai"; -import type { HookAPI, AppMessage, SessionEntry, ContextMessage } from "@mariozechner/pi-coding-agent/hooks"; - -export default function(pi: HookAPI) { - pi.command("pop", { - description: "Pop to previous turn, summarizing work", - handler: async (ctx) => { - const entries = ctx.entries as SessionEntry[]; - - // Get user turns - const turns = entries - .map((e, i) => ({ e, i })) - .filter(({ e }) => e.type === "message" && (e as any).message.role === "user") - .map(({ e, i }) => ({ idx: i, text: preview((e as any).message) })); - - if (turns.length < 2) return { status: "Need at least 2 turns" }; - - // Select target (skip last turn - that's current) - const options = turns.slice(0, -1).map(t => `[${t.idx}] ${t.text}`); - const selected = ctx.args[0] - ? options.find(o => o.startsWith(`[${ctx.args[0]}]`)) - : await ctx.ui.select("Pop to:", options); - - if (!selected) return; - const backTo = parseInt(selected.match(/\[(\d+)\]/)![1]); - - // Check compaction crossing - const compactions = entries.filter(e => e.type === "compaction") as any[]; - const latestCompaction = compactions[compactions.length - 1]; - const crossing = latestCompaction && backTo < latestCompaction.firstKeptEntryIndex; - - // Generate summaries - let prePopSummary: string | undefined; - if (crossing) { - ctx.ui.notify("Crossing compaction, generating pre-pop summary...", "info"); - const preMsgs = getMessages(entries.slice(0, backTo)); - prePopSummary = await summarize(preMsgs, ctx, "context before this work"); - } - - const popMsgs = getMessages(entries.slice(backTo)); - const summary = await summarize(popMsgs, ctx, "completed work"); - - // Save and rebuild - await ctx.saveEntry({ - type: "stack_pop", - backToIndex: backTo, - summary, - prePopSummary, - }); - - await ctx.rebuildContext(); - return { status: `Popped to turn ${backTo}` }; - } - }); - - pi.on("context", (event, ctx) => { - const hasPops = event.entries.some(e => e.type === "stack_pop"); - if (!hasPops) return; - - // Collect ranges with IDs - let rangeId = 0; - const ranges: Array<{from: number; to: number; summary: string; id: number}> = []; - - for (let i = 0; i < event.entries.length; i++) { - const e = event.entries[i] as any; - if (e.type === "compaction") { - ranges.push({ from: 0, to: e.firstKeptEntryIndex, summary: e.summary, id: rangeId++ }); - } - if (e.type === "stack_pop") { - if (e.prePopSummary) { - ranges.push({ from: 0, to: e.backToIndex, summary: e.prePopSummary, id: rangeId++ }); - } - ranges.push({ from: e.backToIndex, to: i, summary: e.summary, id: rangeId++ }); - } - } - - // Build messages - const messages: ContextMessage[] = []; - const emitted = new Set(); - - for (let i = 0; i < event.entries.length; i++) { - const covering = ranges.filter(r => r.from <= i && i < r.to); - - if (covering.length) { - const winner = covering.reduce((a, b) => a.id > b.id ? a : b); - if (i === winner.from && !emitted.has(winner.id)) { - messages.push({ - message: { role: "user", content: `[Summary]\n\n${winner.summary}`, timestamp: Date.now() } as AppMessage, - entryIndex: null - }); - emitted.add(winner.id); - } - continue; - } - - const e = event.entries[i]; - if (e.type === "message") { - messages.push({ message: (e as any).message, entryIndex: i }); - } - } - - return { messages }; - }); -} - -function getMessages(entries: SessionEntry[]): AppMessage[] { - return entries.filter(e => e.type === "message").map(e => (e as any).message); -} - -function preview(msg: AppMessage): string { - const text = typeof msg.content === "string" ? msg.content - : (msg.content as any[]).filter(c => c.type === "text").map(c => c.text).join(" "); - return text.slice(0, 40) + (text.length > 40 ? "..." : ""); -} - -async function summarize(msgs: AppMessage[], ctx: any, purpose: string): Promise { - const apiKey = await ctx.resolveApiKey(ctx.model); - const resp = await complete(ctx.model, { - messages: [...msgs, { role: "user", content: `Summarize as "${purpose}". Be concise.`, timestamp: Date.now() }] - }, { apiKey, maxTokens: 2000, signal: ctx.signal }); - return resp.content.filter((c: any) => c.type === "text").map((c: any) => c.text).join("\n"); -} -``` - -## Edge Cases - -### Session Resumed Without Hook - -User has stacking hook, does `/pop`, saves `stack_pop` entry. Later removes hook and resumes session. - -**What happens:** -1. Core loads all entries (including `stack_pop`) -2. Core's `buildSessionContext()` ignores unknown types, returns compaction + message entries -3. `context` event fires, but no handler processes `stack_pop` -4. Core's messages pass through unchanged - -**Result:** Messages that were "popped" return to context. The pop is effectively undone. - -**Why this is OK:** -- Session file is intact, no data lost -- If compaction happened after pop, the compaction summary captured the popped state -- User removed the hook, so hook's behavior (hiding messages) is gone -- User can re-add hook to restore stacking behavior - -**Mitigation:** Could warn on session load if unknown entry types found: -```typescript -// In session load -const unknownTypes = entries - .map(e => e.type) - .filter(t => !knownTypes.has(t)); -if (unknownTypes.length) { - console.warn(`Session has entries of unknown types: ${unknownTypes.join(", ")}`); -} -``` - -### Hook Added to Existing Session - -User has old session without stacking. Adds stacking hook, does `/pop`. - -**What happens:** -1. Hook saves `stack_pop` entry -2. `context` event fires, hook processes it -3. Works normally - -No issue. Hook processes entries it recognizes, ignores others. - -### Multiple Hooks with Different Entry Types - -Hook A handles `type_a` entries, Hook B handles `type_b` entries. - -**What happens:** -1. `context` event chains through both hooks -2. Each hook checks for its entry types, passes through if none found -3. Each hook's transforms are applied in order - -**Best practice:** Hooks should: -- Only process their own entry types -- Return `undefined` (pass through) if no relevant entries -- Use prefixed type names: `myhook_pop`, `myhook_prune` - -### Conflicting Hooks - -Two hooks both try to handle the same entry type (e.g., both handle `compaction`). - -**What happens:** -- Later hook (project > global) wins in the chain -- Earlier hook's transform is overwritten - -**Mitigation:** -- Core entry types (`compaction`, `message`, etc.) should not be overridden by hooks -- Hooks should use unique prefixed type names -- Document which types are "reserved" - -### Session with Future Entry Types - -User downgrades pi version, session has entry types from newer version. - -**What happens:** -- Same as "hook removed" - unknown types ignored -- Core handles what it knows, hooks handle what they know - -**Session file is forward-compatible:** Unknown entries are preserved in file, just not processed. - -## Implementation Phases - -| Phase | Scope | LOC | -|-------|-------|-----| -| v2.0 | `saveEntry`, `context` event, `rebuildContext`, extended context | ~150 | -| v2.1 | `pi.command()`, TUI integration, autocomplete | ~200 | -| v2.2 | Example hooks, documentation | ~300 | - -## Implementation Order - -1. `ContextMessage` type, update `buildSessionContext()` return type -2. `saveEntry()` in session-manager -3. `context` event in runner with chaining -4. State callbacks interface and wiring -5. `rebuildContext()` in agent-session -6. Manual test with simple hook -7. Command registration in loader -8. Command invocation in runner -9. TUI command handling + autocomplete -10. Stacking example hook -11. Pruning example hook -12. Update hooks.md diff --git a/packages/coding-agent/docs/rpc.md b/packages/coding-agent/docs/rpc.md index d5f25bb5..491049c6 100644 --- a/packages/coding-agent/docs/rpc.md +++ b/packages/coding-agent/docs/rpc.md @@ -36,9 +36,9 @@ Send a user prompt to the agent. Returns immediately; events stream asynchronous {"id": "req-1", "type": "prompt", "message": "Hello, world!"} ``` -With attachments: +With images: ```json -{"type": "prompt", "message": "What's in this image?", "attachments": [...]} +{"type": "prompt", "message": "What's in this image?", "images": [{"type": "image", "source": {"type": "base64", "mediaType": "image/png", "data": "..."}}]} ``` Response: @@ -46,7 +46,7 @@ Response: {"id": "req-1", "type": "response", "command": "prompt", "success": true} ``` -The `attachments` field is optional. See [Attachments](#attachments) for the schema. +The `images` field is optional. Each image uses `ImageContent` format with base64 or URL source. #### queue_message diff --git a/packages/coding-agent/docs/sdk.md b/packages/coding-agent/docs/sdk.md index 1ae0548c..0a9eccdc 100644 --- a/packages/coding-agent/docs/sdk.md +++ b/packages/coding-agent/docs/sdk.md @@ -76,12 +76,13 @@ The session manages the agent lifecycle, message history, and event streaming. interface AgentSession { // Send a prompt and wait for completion prompt(text: string, options?: PromptOptions): Promise; + prompt(message: AppMessage): Promise; // For HookMessage, etc. // Subscribe to events (returns unsubscribe function) subscribe(listener: (event: AgentSessionEvent) => void): () => void; // Session info - sessionFile: string | null; + sessionFile: string | undefined; // undefined for in-memory sessionId: string; // Model control @@ -98,9 +99,14 @@ interface AgentSession { isStreaming: boolean; // Session management - reset(): Promise; - branch(entryIndex: number): Promise<{ selectedText: string; skipped: boolean }>; - switchSession(sessionPath: string): Promise; + newSession(): Promise; // Returns false if cancelled by hook + switchSession(sessionPath: string): Promise; + + // Branching (tree-based) + branch(entryId: string): Promise<{ cancelled: boolean }>; + + // Hook message injection + sendHookMessage(message: HookMessage, triggerTurn?: boolean): void; // Compaction compact(customInstructions?: string): Promise; @@ -436,18 +442,38 @@ import { createAgentSession, discoverHooks, type HookFactory } from "@mariozechn // Inline hook const loggingHook: HookFactory = (api) => { + // Log tool calls api.on("tool_call", async (event) => { console.log(`Tool: ${event.toolName}`); return undefined; // Don't block }); + // Block dangerous commands api.on("tool_call", async (event) => { - // Block dangerous commands if (event.toolName === "bash" && event.input.command?.includes("rm -rf")) { return { block: true, reason: "Dangerous command" }; } return undefined; }); + + // Register custom slash command + api.registerCommand("stats", { + description: "Show session stats", + handler: async (ctx) => { + const entries = ctx.sessionManager.getEntries(); + ctx.ui.notify(`${entries.length} entries`, "info"); + }, + }); + + // Inject messages + api.sendMessage({ + customType: "my-hook", + content: "Hook initialized", + display: false, // Hidden from TUI + }, false); // Don't trigger agent turn + + // Persist hook state + api.appendEntry("my-hook", { initialized: true }); }; // Replace discovery @@ -472,7 +498,15 @@ const { session } = await createAgentSession({ }); ``` -> See [examples/sdk/06-hooks.ts](../examples/sdk/06-hooks.ts) +Hook API methods: +- `api.on(event, handler)` - Subscribe to events +- `api.sendMessage(message, triggerTurn?)` - Inject message (creates `CustomMessageEntry`) +- `api.appendEntry(customType, data?)` - Persist hook state (not in LLM context) +- `api.registerCommand(name, options)` - Register custom slash command +- `api.registerMessageRenderer(customType, renderer)` - Custom TUI rendering +- `api.exec(command, args, options?)` - Execute shell commands + +> See [examples/sdk/06-hooks.ts](../examples/sdk/06-hooks.ts) and [docs/hooks.md](hooks.md) ### Skills @@ -560,6 +594,8 @@ const { session } = await createAgentSession({ ### Session Management +Sessions use a tree structure with `id`/`parentId` linking, enabling in-place branching. + ```typescript import { createAgentSession, SessionManager } from "@mariozechner/pi-coding-agent"; @@ -597,12 +633,32 @@ const customDir = "/path/to/my-sessions"; const { session } = await createAgentSession({ sessionManager: SessionManager.create(process.cwd(), customDir), }); -// Also works with list and continueRecent: -// SessionManager.list(process.cwd(), customDir); -// SessionManager.continueRecent(process.cwd(), customDir); ``` -> See [examples/sdk/11-sessions.ts](../examples/sdk/11-sessions.ts) +**SessionManager tree API:** + +```typescript +const sm = SessionManager.open("/path/to/session.jsonl"); + +// Tree traversal +const entries = sm.getEntries(); // All entries (excludes header) +const tree = sm.getTree(); // Full tree structure +const path = sm.getPath(); // Path from root to current leaf +const leaf = sm.getLeafEntry(); // Current leaf entry +const entry = sm.getEntry(id); // Get entry by ID +const children = sm.getChildren(id); // Direct children of entry + +// Labels +const label = sm.getLabel(id); // Get label for entry +sm.appendLabelChange(id, "checkpoint"); // Set label + +// Branching +sm.branch(entryId); // Move leaf to earlier entry +sm.branchWithSummary(id, "Summary..."); // Branch with context summary +sm.createBranchedSession(leafId); // Extract path to new file +``` + +> See [examples/sdk/11-sessions.ts](../examples/sdk/11-sessions.ts) and [docs/session.md](session.md) ### Settings Management @@ -888,7 +944,21 @@ type Tool For hook types, import from the hooks subpath: ```typescript -import type { HookAPI, HookEvent, ToolCallEvent } from "@mariozechner/pi-coding-agent/hooks"; +import type { + HookAPI, + HookMessage, + HookFactory, + HookEventContext, + HookCommandContext, + ToolCallEvent, + ToolResultEvent, +} from "@mariozechner/pi-coding-agent/hooks"; +``` + +For message utilities: + +```typescript +import { isHookMessage, createHookMessage } from "@mariozechner/pi-coding-agent"; ``` For config utilities: diff --git a/packages/coding-agent/docs/session-tree.md b/packages/coding-agent/docs/session-tree.md deleted file mode 100644 index 839b23ca..00000000 --- a/packages/coding-agent/docs/session-tree.md +++ /dev/null @@ -1,452 +0,0 @@ -# Session Tree Format - -Analysis of switching from linear JSONL to tree-based session storage. - -## Current Format (Linear) - -```jsonl -{"type":"session","id":"...","timestamp":"...","cwd":"..."} -{"type":"message","timestamp":"...","message":{"role":"user",...}} -{"type":"message","timestamp":"...","message":{"role":"assistant",...}} -{"type":"compaction","timestamp":"...","summary":"...","firstKeptEntryIndex":2,"tokensBefore":50000} -{"type":"message","timestamp":"...","message":{"role":"user",...}} -``` - -Context is built by scanning linearly, applying compaction ranges. - -## Proposed Format (Tree) - -Each entry has a `uuid` and `parentUuid` field (null for root). Session header includes `version` for future migrations: - -```jsonl -{"type":"session","version":2,"uuid":"a1b2c3","parentUuid":null,"id":"...","cwd":"..."} -{"type":"message","uuid":"d4e5f6","parentUuid":"a1b2c3","message":{"role":"user",...}} -{"type":"message","uuid":"g7h8i9","parentUuid":"d4e5f6","message":{"role":"assistant",...}} -{"type":"message","uuid":"j0k1l2","parentUuid":"g7h8i9","message":{"role":"user",...}} -{"type":"message","uuid":"m3n4o5","parentUuid":"j0k1l2","message":{"role":"assistant",...}} -``` - -Version history: -- **v1** (implicit): Linear format, no uuid/parentUuid -- **v2**: Tree format with uuid/parentUuid - -The **last entry** is always the current leaf. Context = walk from leaf to root via `parentUuid`. - -Using UUIDs (like Claude Code does) instead of indices because: -- No remapping needed when branching to new file -- Robust to entry deletion/reordering -- Orphan references are detectable -- ~30 extra bytes per entry is negligible for text-heavy sessions - -### Branching - -Branch from entry `g7h8i9` (after first assistant response): - -```jsonl -... entries unchanged ... -{"type":"message","uuid":"p6q7r8","parentUuid":"g7h8i9","message":{"role":"user",...}} -{"type":"message","uuid":"s9t0u1","parentUuid":"p6q7r8","message":{"role":"assistant",...}} -``` - -Walking s9t0u1→p6q7r8→g7h8i9→d4e5f6→a1b2c3 gives the branched context. - -The old path (j0k1l2, m3n4o5) remains in the file but is not in the current context. - -### Visual - -``` - [a1b2:session] - │ - [d4e5:user "hello"] - │ - [g7h8:assistant "hi"] - │ - ┌────┴────┐ - │ │ -[j0k1:user A] [p6q7:user B] ← branch point - │ │ -[m3n4:asst A] [s9t0:asst B] ← current leaf - │ - (old path) -``` - -## Context Building - -```typescript -function buildContext(entries: SessionEntry[]): AppMessage[] { - // Build UUID -> entry map - const byUuid = new Map(entries.map(e => [e.uuid, e])); - - // Start from last entry (current leaf) - let current: SessionEntry | undefined = entries[entries.length - 1]; - - // Walk to root, collecting messages - const path: SessionEntry[] = []; - while (current) { - path.unshift(current); - current = current.parentUuid ? byUuid.get(current.parentUuid) : undefined; - } - - // Extract messages, apply compaction summaries - return pathToMessages(path); -} -``` - -Complexity: O(n) to build map, O(depth) to walk. Total O(n), but walk is fast. - -## Consequences for Stacking - -### Current Approach (hooks-v2.md) - -Stacking uses `stack_pop` entries with complex range overlap rules: - -```typescript -interface StackPopEntry { - type: "stack_pop"; - backToIndex: number; - summary: string; - prePopSummary?: string; -} -``` - -Context building requires tracking ranges, IDs, "later wins" logic. - -### Tree Approach - -Stacking becomes trivial branching: - -```jsonl -... conversation entries ... -{"type":"stack_summary","uuid":"x1y2z3","parentUuid":"g7h8i9","summary":"Work done after this point"} -``` - -To "pop" to entry `g7h8i9`: -1. Generate summary of entries after `g7h8i9` -2. Append summary entry with `parentUuid: "g7h8i9"` - -Context walk follows parentUuid chain. Abandoned entries are not traversed. - -**No range tracking. No overlap rules. No "later wins" logic.** - -### Multiple Pops - -``` -[a]─[b]─[c]─[d]─[e]─[f]─[g]─[h] - │ - └─[i:summary]─[j]─[k]─[l] - │ - └─[m:summary]─[n:current] -``` - -Each pop just creates a new branch. Context: n→m→k→j→i→c→b→a. - -## Consequences for Compaction - -### Current Approach - -Compaction stores `firstKeptEntryIndex` (an index) and requires careful handling when stacking crosses compaction boundaries. - -### Tree Approach - -Compaction is just another entry in the linear chain, not a branch. Only change: `firstKeptEntryIndex` → `firstKeptEntryUuid`. - -``` -root → m1 → m2 → m3 → m4 → m5 → m6 → m7 → m8 → m9 → m10 → compaction -``` - -```jsonl -{"type":"compaction","uuid":"c1","parentUuid":"m10","summary":"...","firstKeptEntryUuid":"m6","tokensBefore":50000} -``` - -Context building: -1. Walk from leaf (compaction) to root -2. See compaction entry → note `firstKeptEntryUuid: "m6"` -3. Continue walking: m10, m9, m8, m7, m6 ← stop here -4. Everything before m6 is replaced by summary -5. Result: `[summary, m6, m7, m8, m9, m10]` - -**Tree is for branching (stacking, alternative paths). Compaction is just a marker in the linear chain.** - -### Compaction + Stacking - -Stacking creates a branch, compaction is inline on each branch: - -``` -[root]─[m1]─[m2]─[m3]─[m4]─[m5]─[compaction1]─[m6]─[m7]─[m8] - │ - └─[stack_summary]─[m9]─[m10]─[compaction2]─[m11:current] -``` - -Each branch has its own compaction history. Context walks the current branch only. - -## Consequences for API - -### SessionManager Changes - -```typescript -interface SessionEntry { - type: string; - uuid: string; // NEW: unique identifier - parentUuid: string | null; // NEW: null for root - timestamp?: string; - // ... type-specific fields -} - -class SessionManager { - // NEW: Get current leaf entry - getCurrentLeaf(): SessionEntry; - - // NEW: Walk from entry to root - getPath(fromUuid?: string): SessionEntry[]; - - // NEW: Get entry by UUID - getEntry(uuid: string): SessionEntry | undefined; - - // CHANGED: Uses tree walk instead of linear scan - buildSessionContext(): SessionContext; - - // NEW: Create branch point - branch(parentUuid: string): string; // returns new entry's uuid - - // NEW: Create branch with summary of abandoned subtree - branchWithSummary(parentUuid: string, summary: string): string; - - // CHANGED: Simpler, just creates summary node - saveCompaction(entry: CompactionEntry): void; - - // CHANGED: Now requires parentUuid (uses current leaf if omitted) - saveMessage(message: AppMessage, parentUuid?: string): void; - saveEntry(entry: SessionEntry): void; -} -``` - -### AgentSession Changes - -```typescript -class AgentSession { - // CHANGED: Uses tree-based branching - async branch(entryUuid: string): Promise; - - // NEW: Branch in current session (no new file) - async branchInPlace(entryUuid: string, options?: { - summarize?: boolean; // Generate summary of abandoned subtree - }): Promise; - - // NEW: Get tree structure for visualization - getSessionTree(): SessionTree; - - // CHANGED: Simpler implementation - async compact(): Promise; -} - -interface BranchResult { - selectedText: string; - cancelled: boolean; - newSessionFile?: string; // If branching to new file - inPlace: boolean; // If branched in current file -} -``` - -### Hook API Changes - -```typescript -interface HookEventContext { - // NEW: Tree-aware entry access - entries: readonly SessionEntry[]; - currentPath: readonly SessionEntry[]; // Entries from root to current leaf - - // NEW: Branch without creating new file - branchInPlace(parentUuid: string, summary?: string): Promise; - - // Existing - saveEntry(entry: SessionEntry): Promise; - rebuildContext(): Promise; -} -``` - -## New Features Enabled - -### 1. In-Place Branching - -Currently, `/branch` always creates a new session file. With tree format: - -``` -/branch → Create new session file (current behavior) -/branch-here → Branch in current file, optionally with summary -``` - -Use case: Quick "let me try something else" without file proliferation. - -### 2. Branch History Navigation - -``` -/branches → List all branches in current session -/switch → Switch to branch at entry -``` - -The session file contains full history. UI can visualize the tree. - -### 3. Simpler Stacking - -No hooks needed for basic stacking: - -``` -/pop → Branch to previous user message with auto-summary -/pop → Branch to specific entry with auto-summary -``` - -Core functionality, not hook-dependent. - -### 4. Subtree Export - -``` -/export-branch → Export just the subtree from entry -``` - -Useful for sharing specific conversation paths. No index remapping needed since UUIDs are stable. - -### 5. Merge/Cherry-pick (Future) - -With tree structure, could support: - -``` -/cherry-pick → Copy entry's message to current branch -/merge → Merge branch into current -``` - -## Migration - -### Strategy: Migrate on Load + Rewrite - -When loading a session, check if migration is needed. If so, migrate in memory and rewrite the file. This is transparent to users and only happens once per session file. - -```typescript -const CURRENT_VERSION = 2; - -function loadSession(path: string): SessionEntry[] { - const content = readFileSync(path, 'utf8'); - const entries = parseEntries(content); - - const header = entries.find(e => e.type === 'session'); - const version = header?.version ?? 1; - - if (version < CURRENT_VERSION) { - migrateEntries(entries, version); - writeFileSync(path, entries.map(e => JSON.stringify(e)).join('\n') + '\n'); - } - - return entries; -} - -function migrateEntries(entries: SessionEntry[], fromVersion: number): void { - if (fromVersion < 2) { - // v1 → v2: Add uuid/parentUuid, convert firstKeptEntryIndex - const uuids: string[] = []; - - for (let i = 0; i < entries.length; i++) { - const entry = entries[i]; - const uuid = generateUuid(); - uuids.push(uuid); - - entry.uuid = uuid; - entry.parentUuid = i === 0 ? null : uuids[i - 1]; - - // Update session header version - if (entry.type === 'session') { - entry.version = CURRENT_VERSION; - } - - // Convert compaction index to UUID - if (entry.type === 'compaction' && 'firstKeptEntryIndex' in entry) { - entry.firstKeptEntryUuid = uuids[entry.firstKeptEntryIndex]; - delete entry.firstKeptEntryIndex; - } - } - } - - // Future migrations: if (fromVersion < 3) { ... } -} -``` - -### What Gets Migrated - -| v1 Field | v2 Field | -|----------|----------| -| (none) | `uuid` (generated) | -| (none) | `parentUuid` (previous entry's uuid, null for root) | -| (none on session) | `version: 2` | -| `firstKeptEntryIndex` | `firstKeptEntryUuid` | - -Migrated sessions work exactly as before (linear path). Tree features become available. - -### API Compatibility - -- `buildSessionContext()` returns same structure -- `branch()` still works, just uses UUIDs -- Existing hooks continue to work -- Old sessions auto-migrate on first load - -## Complexity Analysis - -| Operation | Linear | Tree | -|-----------|--------|------| -| Append message | O(1) | O(1) | -| Build context | O(n) | O(n) map + O(depth) walk | -| Branch to new file | O(n) copy | O(path) copy, no remapping | -| Find entry by UUID | O(n) | O(1) with map | -| Compaction | O(n) | O(depth) | - -Tree with UUIDs is comparable or better. The UUID map can be cached. - -## File Size - -Tree format adds ~50 bytes per entry (`"uuid":"...","parentUuid":"..."`, 36 chars each). For 1000-entry session: ~50KB overhead. Negligible for text-heavy sessions. - -Abandoned branches remain in file but don't affect context building performance. - -## Example: Full Session with Branching - -```jsonl -{"type":"session","version":2,"uuid":"ses1","parentUuid":null,"id":"abc","cwd":"/project"} -{"type":"message","uuid":"m1","parentUuid":"ses1","message":{"role":"user","content":"Build a CLI"}} -{"type":"message","uuid":"m2","parentUuid":"m1","message":{"role":"assistant","content":"I'll create..."}} -{"type":"message","uuid":"m3","parentUuid":"m2","message":{"role":"user","content":"Add --verbose flag"}} -{"type":"message","uuid":"m4","parentUuid":"m3","message":{"role":"assistant","content":"Here's the flag..."}} -{"type":"message","uuid":"m5","parentUuid":"m4","message":{"role":"user","content":"Actually use Python"}} -{"type":"message","uuid":"m6","parentUuid":"m5","message":{"role":"assistant","content":"Converting to Python..."}} -{"type":"branch_summary","uuid":"bs1","parentUuid":"m2","summary":"Attempted Node.js CLI with --verbose flag"} -{"type":"message","uuid":"m7","parentUuid":"bs1","message":{"role":"user","content":"Use Rust instead"}} -{"type":"message","uuid":"m8","parentUuid":"m7","message":{"role":"assistant","content":"Creating Rust CLI..."}} -``` - -Context path: m8→m7→bs1→m2→m1→ses1 - -Result: -1. User: "Build a CLI" -2. Assistant: "I'll create..." -3. Summary: "Attempted Node.js CLI with --verbose flag" -4. User: "Use Rust instead" -5. Assistant: "Creating Rust CLI..." - -Entries m3-m6 (the Node.js/Python path) are preserved but not in context. - -## Prior Art - -Claude Code uses the same approach: -- `uuid` field on each entry -- `parentUuid` links to parent (null for root) -- `leafUuid` in summary entries to track conversation endpoints -- Separate files for sidechains (`isSidechain: true`) - -## Recommendation - -The tree format with UUIDs: -- Simplifies stacking (no range overlap logic) -- Simplifies compaction (no boundary crossing) -- Enables in-place branching -- Enables branch visualization/navigation -- No index remapping on branch-to-file -- Maintains backward compatibility -- Validated by Claude Code's implementation - -**Recommend implementing for v2 of hooks/session system.**