fix(tui): Fix garbled output when content exceeds viewport

- Implemented new renderLineBased method that properly handles scrollback boundaries - Fixed ANSI code preservation in MarkdownComponent line wrapping - Added comprehensive test to reproduce and verify the fix - Root cause: PARTIAL rendering strategy couldn't position cursor in scrollback - Solution: Component-agnostic line comparison with proper viewport boundary handling
2026-04-21 20:04:55 +00:00 · 2025-08-11 14:17:46 +02:00 · 2025-08-11 14:17:46 +02:00 · 6e40c5d761
commit 6e40c5d761
parent 1d9b77298c
6 changed files with 485 additions and 106 deletions
--- a/todos/todos.md
+++ b/todos/todos.md
@ -1,102 +1,3 @@
- agent/tui: interrupting requests to model  via ESC doesn't work. Interrupting bash tool works.
-
- agent/tui: "read all README.md files except in node_modules", gpt-5. wait for completion, then send a new message. Getting garbled output. this happens for both of the renderDifferential and renderDifferentialSurgical methods. We need to emiulate this in a test and get to the bottom of it.
-    ```markdown
-    >> pi interactive chat <<<
-    Press Escape to interrupt while processing
-    Press CTRL+C to clear the text editorM deployments and building AI agents.peScript API)
-    Press CTRL+C twice quickly to exitdeterministic programs (via JSON mode in any language or the
-    ## PackagesAPI)
-    [user]iding your own system prompts and tools
-    read all README.md files except in node_modulesrminal UI library with differential rendering
-    - **[@mariozechner/pi-agent](packages/agent)** - General-purpose agent with tool calling and session
-    [assistant]re lines)
-    [tool] glob({"pattern":"**/README.md"}) - CLI for managing vLLM deployments on GPU pods
-    README.mdad({"path": "packages/pods/README.md"})
-    node_modules/@biomejs/biome/README.md
-    node_modules/@esbuild/darwin-arm64/README.md
-    node_modules/@types/node/README.mdnt/README.md"}) vLLM configuration for agentic workloads.
-    node_modules/@xterm/headless/README.md
-    node_modules/@xterm/xterm/README.md
-    node_modules/chalk/readme.md tool calling and session persistence, modeled after Claude Code but
-    node_modules/esbuild/README.md. It comes with a built-in TUI (also modeled after Claude Code) for
-    node_modules/fsevents/README.md
-    node_modules/get-tsconfig/README.md
-    ... (59 more lines)ned to be easy:
-    - Writing custom UIs on top of it (via JSON mode in any language or the TypeScript API)
-    [tool] read({"path": "README.md"})deterministic programs (via JSON mode in any language or the
-    # Pi MonorepoI)ath": "packages/tui/README.md"})
-    - Providing your own system prompts and tools
-    A collection of tools for managing LLM deployments and building AI agents.
-    Terminal UI framework with surgical differential rendering for building flicker-free interactive CLI
-    ## Packagesre lines)
-
-    - **[@mariozechner/pi-tui](packages/tui)** - Terminal UI library with differential rendering
-    - **[@mariozechner/pi-agent](packages/agent)** - General-purpose agent with tool calling and session
-    persistencel Differential Rendering**: Three-strategy system that minimizes redraws to 1-2 lines for
-    - **[@mariozechner/pi](packages/pods)** - CLI for managing vLLM deployments on GPU podsads.
-    - **Scrollback Buffer Preservation**: Correctly maintains terminal history when content exceeds
-    ... (76 more lines)
-    - **Zero Flicker**: Components like text editors remain perfectly still while other parts update
-    [tool] read({"path": "packages/agent/README.md"})tocomplete, selection lists, markdown rendering
-    # pi-agentl -g @mariozechner/pi
-    ```
-    A general-purpose agent with tool calling and session persistence, modeled after Claude Code but
-    extremely hackable and minimal. It comes with a built-in TUI (also modeled after Claude Code) for────╮
-    interactive use.                                                                                     │
-    [tool] read({"path": "packages/tui/README.md"})──────────────────────────────────────────────────────╯
-    Everything is designed to be easy:
-    - Writing custom UIs on top of it (via JSON mode in any language or the TypeScript API)
-    - Using it for inference steps in deterministic programs (via JSON mode in any language or theve CLI
-    TypeScript API)
-    - Providing your own system prompts and tools
-    - Working with various LLM providers or self-hosted LLMs
-
-    ... (422 more lines)ntial Rendering**: Three-strategy system that minimizes redraws to 1-2 lines for
-    typical updates
-    [tool] read({"path": "packages/pods/README.md"})maintains terminal history when content exceeds
-    # piport
-    - **Zero Flicker**: Components like text editors remain perfectly still while other parts update
-    Deploy and manage LLMs on GPU pods with automatic vLLM configuration for agentic workloads.ering
-    ... (570 more lines)
-    ## Installation
-    [user]
-    ```bash
-    npm install -g @mariozechner/pi
-    ```sistant]
-    ⠴ Thinking...
-    ... (501 more lines)
-
-    [tool] read({"path": "packages/tui/README.md"})──────────────────────────────────────────────────────╮
-    # @mariozechner/pi-tui                                                                               │
-    ╰────────────────────────────────────────────────────────────────────────────────────────────────────╯
-    Terminal UI framework with surgical differential rendering for building flicker-free interactive CLI
-    applications.
-
-    ## Features
-
-    - **Surgical Differential Rendering**: Three-strategy system that minimizes redraws to 1-2 lines for
-    typical updates
-    - **Scrollback Buffer Preservation**: Correctly maintains terminal history when content exceeds
-    viewport
-    - **Zero Flicker**: Components like text editors remain perfectly still while other parts update
-    - **Interactive Components**: Text editor with autocomplete, selection lists, markdown rendering
-    ... (570 more lines)
-
-    [user]
-    l
-
-    [assistant]
-    Do you want me to list the current directory contents? If yes, should I include hidden files and subdir
-    ectories?
-
-
-    ╭────────────────────────────────────────────────────────────────────────────────────────────────────╮
-    │ >                                                                                                  │
-    ╰────────────────────────────────────────────────────────────────────────────────────────────────────╯
-    ↑14,783 ↓160 ⚡128 ⚒ 5
-    ```
-
 - pods: pi start outputs all models that can be run on the pod. however, it doesn't check the vllm version. e.g. gpt-oss can only run via vllm+gpt-oss. glm4.5 can only run on vllm nightly.

 - agent: improve reasoning section in README.md
--- a/todos/work/20250811-122302-tui-garbled-output-fix/analysis.md
+++ b/todos/work/20250811-122302-tui-garbled-output-fix/analysis.md
@ -0,0 +1,93 @@
+# TUI Garbled Output Analysis
+
+## Problem Description
+When reading multiple README.md files and then sending a new message, the TUI displays garbled output. This happens for both renderDifferential and renderDifferentialSurgical methods, affecting any model (not just gpt-5).
+
+## Rendering System Overview
+
+### Three Rendering Strategies
+1. **SURGICAL Updates** - Updates only changed lines (1-2 lines typical)
+2. **PARTIAL Re-render** - Clears from first change to end, re-renders tail
+3. **FULL Re-render** - Clears scrollback and screen, renders everything
+
+### Key Components
+- **TUI Class** (`packages/tui/src/tui.ts`): Main rendering engine
+- **Container Class**: Manages child components, auto-triggers re-renders
+- **TuiRenderer** (`packages/agent/src/renderers/tui-renderer.ts`): Agent's TUI integration
+- **Event System**: Event-driven updates through AgentEvent
+
+## Root Causes Identified
+
+### 1. Complex ANSI Code Handling
+- MarkdownComponent line wrapping has issues with ANSI escape sequences
+- Code comment at line 203: "Need to wrap - this is complex with ANSI codes"
+- ANSI codes can be split across render operations, causing corruption
+
+### 2. Race Conditions in Rapid Updates
+When processing multiple tool calls:
+- Multiple containers change simultaneously
+- Content added both above and within viewport
+- Surgical renderer handles structural changes while maintaining cursor position
+- Heavy ANSI content (colored tool output, markdown) increases complexity
+
+### 3. Cursor Position Miscalculation
+- Rapid updates can cause cursor positioning logic errors
+- Content shifts due to previous renders not properly accounted for
+- Viewport vs scrollback buffer calculations can become incorrect
+
+### 4. Container Change Detection Timing
+- Recent fix (192d8d2) addressed container clear detection
+- But rapid component addition/removal still may leave artifacts
+- Multiple render requests debounced but may miss intermediate states
+
+## Specific Scenario Analysis
+
+### Sequence When Issue Occurs:
+1. User sends "read all README.md files"
+2. Multiple tool calls execute rapidly:
+   - glob() finds files
+   - Multiple read() calls for each README
+3. Long file contents displayed with markdown formatting
+4. User sends new message while output still rendering
+5. New components added while previous render incomplete
+
+### Visual Artifacts Observed:
+- Text overlapping from different messages
+- Partial ANSI codes causing color bleeding
+- Editor borders duplicated or misaligned
+- Content from previous render persisting
+- Line wrapping breaking mid-word with styling
+
+## Related Fixes
+- Commit 1d9b772: Fixed ESC interrupt handling race conditions
+- Commit 192d8d2: Fixed container change detection for clear operations
+- Commit 2ec8a27: Added instructional header to chat demo
+
+## Test Coverage Gaps
+- No tests for rapid multi-tool execution scenarios
+- Missing tests for ANSI code handling across line wraps
+- No stress tests for viewport overflow with rapid updates
+- Layout shift artifacts test exists but limited scope
+
+## Recommended Solutions
+
+### 1. Improve ANSI Handling
+- Fix MarkdownComponent line wrapping to preserve ANSI codes
+- Ensure escape sequences never split across operations
+- Add ANSI-aware string measurement utilities
+
+### 2. Add Render Queuing
+- Implement render operation queue to prevent overlaps
+- Ensure each render completes before next begins
+- Add render state tracking
+
+### 3. Enhanced Change Detection
+- Track render generation/version numbers
+- Validate cursor position before surgical updates
+- Add checksums for rendered content verification
+
+### 4. Comprehensive Testing
+- Create test simulating exact failure scenario
+- Add stress tests with rapid multi-component updates
+- Test ANSI-heavy content with line wrapping
+- Verify viewport calculations under load
--- a/todos/work/20250811-122302-tui-garbled-output-fix/task.md
+++ b/todos/work/20250811-122302-tui-garbled-output-fix/task.md
@ -0,0 +1,35 @@
+# Fix TUI Garbled Output When Sending Multiple Messages
+
+**Status:** InProgress
+**Agent PID:** 54802
+
+## Original Todo
+agent/tui: "read all README.md files except in node_modules". wait for completion, then send a new message. Getting garbled output. this happens for both of the renderDifferential and renderDifferentialSurgical methods. We need to emulate this in a test and get to the bottom of it.
+
+## Description
+Fix the TUI rendering corruption that occurs when sending multiple messages in rapid succession, particularly after tool calls that produce large outputs. The issue manifests as garbled/overlapping text when new messages are sent while previous output is still being displayed.
+
+*Read [analysis.md](./analysis.md) in full for detailed codebase research and context*
+
+## Implementation Plan
+[how we are building it]
+- [x] Create test to reproduce the issue: Simulate rapid tool calls with large outputs followed by new message
+- [x] Fix ANSI code handling in MarkdownComponent line wrapping (packages/tui/src/components/markdown-component.ts:203-276)
+- [x] Implement new line-based rendering strategy that properly handles scrollback and viewport boundaries
+- [x] Add comprehensive test coverage for multi-message scenarios
+- [ ] User test: Run agent, execute "read all README.md files", wait for completion, send new message, verify no garbled output
+
+## Notes
+- Successfully reproduced the issue with test showing garbled text overlay
+- Fixed ANSI code handling in MarkdownComponent line wrapping  
+- Root cause: PARTIAL rendering strategy incorrectly calculated cursor position when content exceeded viewport
+- When content is in scrollback, cursor can't reach it (can only move within viewport)
+- Old PARTIAL strategy tried to move cursor 33 lines up when only 30 were possible
+- This caused cursor to land at wrong position (top of viewport instead of target line in scrollback)
+- Solution: Implemented new `renderLineBased` method that:
+  - Compares old and new lines directly (component-agnostic)
+  - Detects if changes are in scrollback (unreachable) or viewport
+  - For scrollback changes: does full clear and re-render
+  - For viewport changes: moves cursor correctly within viewport bounds and updates efficiently
+  - Handles surgical line-by-line updates when possible for minimal redraws
+- Test now passes - no more garbled output when messages exceed viewport!