diff --git a/packages/web-ui/README.md b/packages/web-ui/README.md index 64006c10..f085b8ac 100644 --- a/packages/web-ui/README.md +++ b/packages/web-ui/README.md @@ -6,13 +6,13 @@ Built with [mini-lit](https://github.com/badlogic/mini-lit) web components and T ## Features -- Modern Chat Interface: Complete chat UI with message history, streaming responses, and tool execution -- Tool Support: Built-in renderers for common tools plus custom tool rendering -- Attachments: PDF, Office documents, images with preview and text extraction -- Artifacts: HTML, SVG, Markdown, and text artifact rendering with sandboxed execution -- CORS Proxy Support: Automatic proxy handling for browser environments -- Platform Agnostic: Works in browser extensions, web apps, VS Code extensions, Electron apps -- TypeScript: Full type safety +- **Chat UI**: Complete interface with message history, streaming, and tool execution +- **Tools**: JavaScript REPL, document extraction, and artifacts (HTML, SVG, Markdown, etc.) +- **Attachments**: PDF, DOCX, XLSX, PPTX, images with preview and text extraction +- **Artifacts**: Interactive HTML, SVG, Markdown with sandboxed execution +- **Storage**: IndexedDB-backed storage for sessions, API keys, and settings +- **CORS Proxy**: Automatic proxy handling for browser environments +- **Custom Providers**: Support for Ollama, LM Studio, vLLM, and OpenAI-compatible APIs ## Installation @@ -36,6 +36,7 @@ import { SettingsStore, setAppStorage, defaultConvertToLlm, + ApiKeyPromptDialog, } from '@mariozechner/pi-web-ui'; import '@mariozechner/pi-web-ui/app.css'; @@ -47,7 +48,12 @@ const sessions = new SessionsStore(); const backend = new IndexedDBStorageBackend({ dbName: 'my-app', version: 1, - stores: [settings.getConfig(), providerKeys.getConfig(), sessions.getConfig()], + stores: [ + settings.getConfig(), + providerKeys.getConfig(), + sessions.getConfig(), + SessionsStore.getMetadataConfig(), + ], }); settings.setBackend(backend); @@ -69,84 +75,101 @@ const agent = new Agent({ convertToLlm: defaultConvertToLlm, }); -// Create chat panel and attach agent +// Create chat panel const chatPanel = new ChatPanel(); await chatPanel.setAgent(agent, { - onApiKeyRequired: async (provider) => { - // Prompt user for API key - return await ApiKeyPromptDialog.prompt(provider); - }, + onApiKeyRequired: (provider) => ApiKeyPromptDialog.prompt(provider), }); document.body.appendChild(chatPanel); ``` -**Run the example:** - -```bash -cd example -npm install -npm run dev -``` - ## Architecture -The web-ui package provides UI components that work with the `Agent` class from `@mariozechner/pi-agent-core`. The Agent handles: +``` +┌─────────────────────────────────────────────────────┐ +│ ChatPanel │ +│ ┌─────────────────────┐ ┌─────────────────────┐ │ +│ │ AgentInterface │ │ ArtifactsPanel │ │ +│ │ (messages, input) │ │ (HTML, SVG, MD) │ │ +│ └─────────────────────┘ └─────────────────────┘ │ +└─────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────┐ +│ Agent (from pi-agent-core) │ +│ - State management (messages, model, tools) │ +│ - Event emission (agent_start, message_update, ...) │ +│ - Tool execution │ +└─────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────┐ +│ AppStorage │ +│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ +│ │ Settings │ │ Provider │ │ Sessions │ │ +│ │ Store │ │Keys Store│ │ Store │ │ +│ └──────────┘ └──────────┘ └──────────┘ │ +│ │ │ +│ IndexedDBStorageBackend │ +└─────────────────────────────────────────────────────┘ +``` -- Conversation state management -- LLM streaming via `streamFn` -- Tool execution -- Event emission - -The web-ui provides: - -- `ChatPanel` / `AgentInterface`: UI components that subscribe to Agent events -- `defaultConvertToLlm`: Message transformer for web-ui custom message types -- Storage backends for API keys, sessions, and settings -- CORS proxy utilities for browser environments - -## Core Components +## Components ### ChatPanel -High-level chat interface with artifacts panel support. +High-level chat interface with built-in artifacts panel. ```typescript -import { ChatPanel, ApiKeyPromptDialog } from '@mariozechner/pi-web-ui'; - const chatPanel = new ChatPanel(); await chatPanel.setAgent(agent, { + // Prompt for API key when needed onApiKeyRequired: async (provider) => ApiKeyPromptDialog.prompt(provider), - onBeforeSend: async () => { /* pre-send hook */ }, - onCostClick: () => { /* cost display clicked */ }, + + // Hook before sending messages + onBeforeSend: async () => { /* save draft, etc. */ }, + + // Handle cost display click + onCostClick: () => { /* show cost breakdown */ }, + + // Custom sandbox URL for browser extensions + sandboxUrlProvider: () => chrome.runtime.getURL('sandbox.html'), + + // Add custom tools toolsFactory: (agent, agentInterface, artifactsPanel, runtimeProvidersFactory) => { - // Return additional tools - return [createJavaScriptReplTool()]; + const replTool = createJavaScriptReplTool(); + replTool.runtimeProvidersFactory = runtimeProvidersFactory; + return [replTool]; }, }); ``` ### AgentInterface -Lower-level chat interface for custom layouts (used internally by ChatPanel). +Lower-level chat interface for custom layouts. ```typescript -import { AgentInterface } from '@mariozechner/pi-web-ui'; - const chat = document.createElement('agent-interface') as AgentInterface; chat.session = agent; chat.enableAttachments = true; chat.enableModelSelector = true; +chat.enableThinkingSelector = true; chat.onApiKeyRequired = async (provider) => { /* ... */ }; +chat.onBeforeSend = async () => { /* ... */ }; ``` +Properties: +- `session`: Agent instance +- `enableAttachments`: Show attachment button (default: true) +- `enableModelSelector`: Show model selector (default: true) +- `enableThinkingSelector`: Show thinking level selector (default: true) +- `showThemeToggle`: Show theme toggle (default: false) + ### Agent (from pi-agent-core) -The Agent class is imported from `@mariozechner/pi-agent-core`: - ```typescript import { Agent } from '@mariozechner/pi-agent-core'; -import { defaultConvertToLlm } from '@mariozechner/pi-web-ui'; const agent = new Agent({ initialState: { @@ -159,93 +182,95 @@ const agent = new Agent({ convertToLlm: defaultConvertToLlm, }); -// Subscribe to events +// Events agent.subscribe((event) => { switch (event.type) { - case 'agent_start': - case 'agent_end': + case 'agent_start': // Agent loop started + case 'agent_end': // Agent loop finished + case 'turn_start': // LLM call started + case 'turn_end': // LLM call finished case 'message_start': - case 'message_update': + case 'message_update': // Streaming update case 'message_end': - case 'turn_start': - case 'turn_end': - // Handle events break; } }); -// Send a message +// Send message await agent.prompt('Hello!'); +await agent.prompt({ role: 'user-with-attachments', content: 'Check this', attachments, timestamp: Date.now() }); -// Or with custom message type -await agent.prompt({ - role: 'user-with-attachments', - content: 'Check this image', - attachments: [imageAttachment], - timestamp: Date.now(), -}); +// Control +agent.abort(); +agent.setModel(newModel); +agent.setThinkingLevel('medium'); +agent.setTools([...]); +agent.queueMessage(customMessage); ``` ## Message Types ### UserMessageWithAttachments -Custom message type for user messages with file attachments: +User message with file attachments: ```typescript -import { isUserMessageWithAttachments, type UserMessageWithAttachments } from '@mariozechner/pi-web-ui'; - const message: UserMessageWithAttachments = { role: 'user-with-attachments', content: 'Analyze this document', - attachments: [pdfAttachment, imageAttachment], + attachments: [pdfAttachment], timestamp: Date.now(), }; + +// Type guard +if (isUserMessageWithAttachments(msg)) { + console.log(msg.attachments); +} ``` ### ArtifactMessage -For session persistence of created artifacts: +For session persistence of artifacts: ```typescript -import { isArtifactMessage, type ArtifactMessage } from '@mariozechner/pi-web-ui'; - const artifact: ArtifactMessage = { role: 'artifact', - artifactId: 'chart-1', - type: 'html', - title: 'Sales Chart', + action: 'create', // or 'update', 'delete' + filename: 'chart.html', content: '
...
', timestamp: new Date().toISOString(), }; + +// Type guard +if (isArtifactMessage(msg)) { + console.log(msg.filename); +} ``` ### Custom Message Types -Extend `CustomAgentMessages` from pi-agent-core: +Extend via declaration merging: ```typescript -// Define your custom message -interface SystemNotificationMessage { +interface SystemNotification { role: 'system-notification'; message: string; level: 'info' | 'warning' | 'error'; timestamp: string; } -// Register with pi-agent-core's type system declare module '@mariozechner/pi-agent-core' { interface CustomAgentMessages { - 'system-notification': SystemNotificationMessage; + 'system-notification': SystemNotification; } } -// Register a renderer +// Register renderer registerMessageRenderer('system-notification', { - render: (msg) => html`
${msg.message}
`, + render: (msg) => html`
${msg.message}
`, }); -// Extend convertToLlm to handle your type +// Extend convertToLlm function myConvertToLlm(messages: AgentMessage[]): Message[] { const processed = messages.map((m) => { if (m.role === 'system-notification') { @@ -259,66 +284,73 @@ function myConvertToLlm(messages: AgentMessage[]): Message[] { ## Message Transformer -The `convertToLlm` function transforms app messages to LLM-compatible format: +`convertToLlm` transforms app messages to LLM-compatible format: ```typescript import { defaultConvertToLlm, convertAttachments } from '@mariozechner/pi-web-ui'; // defaultConvertToLlm handles: -// - UserMessageWithAttachments → user message with content blocks +// - UserMessageWithAttachments → user message with image/text content blocks // - ArtifactMessage → filtered out (UI-only) // - Standard messages (user, assistant, toolResult) → passed through - -// For custom types, wrap defaultConvertToLlm: -const agent = new Agent({ - convertToLlm: (messages) => { - const processed = messages.map(m => { - // Handle your custom types - return m; - }); - return defaultConvertToLlm(processed); - }, -}); ``` -## CORS Proxy +## Tools -Browser environments may need a CORS proxy for certain providers: +### JavaScript REPL + +Execute JavaScript in a sandboxed browser environment: ```typescript -import { - createStreamFn, - shouldUseProxyForProvider, - applyProxyIfNeeded, - isCorsError, -} from '@mariozechner/pi-web-ui'; +import { createJavaScriptReplTool } from '@mariozechner/pi-web-ui'; -// AgentInterface automatically sets up proxy support if using AppStorage -// For manual setup: -agent.streamFn = createStreamFn(async () => { - const enabled = await storage.settings.get('proxy.enabled'); - return enabled ? await storage.settings.get('proxy.url') : undefined; -}); +const replTool = createJavaScriptReplTool(); + +// Configure runtime providers for artifact/attachment access +replTool.runtimeProvidersFactory = () => [ + new AttachmentsRuntimeProvider(attachments), + new ArtifactsRuntimeProvider(artifactsPanel, agent, true), // read-write +]; + +agent.setTools([replTool]); ``` -Providers requiring proxy: -- `zai`: Always requires proxy -- `anthropic`: Only OAuth tokens (`sk-ant-oat-*`) require proxy +### Extract Document -## Tool Renderers +Extract text from documents at URLs: -Customize how tool calls are displayed: +```typescript +import { createExtractDocumentTool } from '@mariozechner/pi-web-ui'; + +const extractTool = createExtractDocumentTool(); +extractTool.corsProxyUrl = 'https://corsproxy.io/?'; + +agent.setTools([extractTool]); +``` + +### Artifacts Tool + +Built into ArtifactsPanel, supports: HTML, SVG, Markdown, text, JSON, images, PDF, DOCX, XLSX. + +```typescript +const artifactsPanel = new ArtifactsPanel(); +artifactsPanel.agent = agent; + +// The tool is available as artifactsPanel.tool +agent.setTools([artifactsPanel.tool]); +``` + +### Custom Tool Renderers ```typescript import { registerToolRenderer, type ToolRenderer } from '@mariozechner/pi-web-ui'; -import { html } from 'lit'; const myRenderer: ToolRenderer = { - renderParams(params, isStreaming) { - return html`
Calling with: ${JSON.stringify(params)}
`; - }, - renderResult(params, result) { - return html`
Result: ${result.output}
`; + render(params, result, isStreaming) { + return { + content: html`
...
`, + isCustom: false, // true = no card wrapper + }; }, }; @@ -327,17 +359,15 @@ registerToolRenderer('my_tool', myRenderer); ## Storage -### AppStorage - -Central storage configuration: +### Setup ```typescript import { AppStorage, IndexedDBStorageBackend, + SettingsStore, ProviderKeysStore, SessionsStore, - SettingsStore, CustomProvidersStore, setAppStorage, getAppStorage, @@ -349,7 +379,7 @@ const providerKeys = new ProviderKeysStore(); const sessions = new SessionsStore(); const customProviders = new CustomProvidersStore(); -// Create backend +// Create backend with all store configs const backend = new IndexedDBStorageBackend({ dbName: 'my-app', version: 1, @@ -357,6 +387,7 @@ const backend = new IndexedDBStorageBackend({ settings.getConfig(), providerKeys.getConfig(), sessions.getConfig(), + SessionsStore.getMetadataConfig(), customProviders.getConfig(), ], }); @@ -367,14 +398,118 @@ providerKeys.setBackend(backend); sessions.setBackend(backend); customProviders.setBackend(backend); -// Create and set app storage +// Create and set global storage const storage = new AppStorage(settings, providerKeys, sessions, customProviders, backend); setAppStorage(storage); +``` -// Access anywhere -const storage = getAppStorage(); -await storage.providerKeys.set('anthropic', 'sk-...'); +### SettingsStore + +Key-value settings: + +```typescript +await storage.settings.set('proxy.enabled', true); +await storage.settings.set('proxy.url', 'https://proxy.example.com'); +const enabled = await storage.settings.get('proxy.enabled'); +``` + +### ProviderKeysStore + +API keys by provider: + +```typescript +await storage.providerKeys.set('anthropic', 'sk-ant-...'); +const key = await storage.providerKeys.get('anthropic'); +const providers = await storage.providerKeys.list(); +``` + +### SessionsStore + +Chat sessions with metadata: + +```typescript +// Save session await storage.sessions.save(sessionData, metadata); + +// Load session +const data = await storage.sessions.get(sessionId); +const metadata = await storage.sessions.getMetadata(sessionId); + +// List sessions (sorted by lastModified) +const allMetadata = await storage.sessions.getAllMetadata(); + +// Update title +await storage.sessions.updateTitle(sessionId, 'New Title'); + +// Delete +await storage.sessions.delete(sessionId); +``` + +### CustomProvidersStore + +Custom LLM providers: + +```typescript +const provider: CustomProvider = { + id: crypto.randomUUID(), + name: 'My Ollama', + type: 'ollama', + baseUrl: 'http://localhost:11434', +}; + +await storage.customProviders.set(provider); +const all = await storage.customProviders.getAll(); +``` + +## Attachments + +Load and process files: + +```typescript +import { loadAttachment, type Attachment } from '@mariozechner/pi-web-ui'; + +// From File input +const file = inputElement.files[0]; +const attachment = await loadAttachment(file); + +// From URL +const attachment = await loadAttachment('https://example.com/doc.pdf'); + +// From ArrayBuffer +const attachment = await loadAttachment(arrayBuffer, 'document.pdf'); + +// Attachment structure +interface Attachment { + id: string; + type: 'image' | 'document'; + fileName: string; + mimeType: string; + size: number; + content: string; // base64 encoded + extractedText?: string; // For documents + preview?: string; // base64 preview image +} +``` + +Supported formats: PDF, DOCX, XLSX, PPTX, images, text files. + +## CORS Proxy + +For browser environments with CORS restrictions: + +```typescript +import { createStreamFn, shouldUseProxyForProvider, isCorsError } from '@mariozechner/pi-web-ui'; + +// AgentInterface auto-configures proxy from settings +// For manual setup: +agent.streamFn = createStreamFn(async () => { + const enabled = await storage.settings.get('proxy.enabled'); + return enabled ? await storage.settings.get('proxy.url') : undefined; +}); + +// Providers requiring proxy: +// - zai: always +// - anthropic: only OAuth tokens (sk-ant-oat-*) ``` ## Dialogs @@ -382,9 +517,13 @@ await storage.sessions.save(sessionData, metadata); ### SettingsDialog ```typescript -import { SettingsDialog, ProvidersModelsTab, ProxyTab } from '@mariozechner/pi-web-ui'; +import { SettingsDialog, ProvidersModelsTab, ProxyTab, ApiKeysTab } from '@mariozechner/pi-web-ui'; -SettingsDialog.open([new ProvidersModelsTab(), new ProxyTab()]); +SettingsDialog.open([ + new ProvidersModelsTab(), // Custom providers + model list + new ProxyTab(), // CORS proxy settings + new ApiKeysTab(), // API keys per provider +]); ``` ### SessionListDialog @@ -406,6 +545,16 @@ import { ApiKeyPromptDialog } from '@mariozechner/pi-web-ui'; const success = await ApiKeyPromptDialog.prompt('anthropic'); ``` +### ModelSelector + +```typescript +import { ModelSelector } from '@mariozechner/pi-web-ui'; + +ModelSelector.open(currentModel, (selectedModel) => { + agent.setModel(selectedModel); +}); +``` + ## Styling Import the pre-built CSS: @@ -414,7 +563,7 @@ Import the pre-built CSS: import '@mariozechner/pi-web-ui/app.css'; ``` -Or customize with your own Tailwind config: +Or use Tailwind with custom config: ```css @import '@mariozechner/mini-lit/themes/claude.css'; @@ -423,14 +572,29 @@ Or customize with your own Tailwind config: @tailwind utilities; ``` +## Internationalization + +```typescript +import { i18n, setLanguage, translations } from '@mariozechner/pi-web-ui'; + +// Add translations +translations.de = { + 'Loading...': 'Laden...', + 'No sessions yet': 'Noch keine Sitzungen', +}; + +setLanguage('de'); +console.log(i18n('Loading...')); // "Laden..." +``` + ## Examples -- [example/](./example) - Complete web application with sessions, artifacts, and custom messages +- [example/](./example) - Complete web app with sessions, artifacts, custom messages - [sitegeist](https://github.com/badlogic/sitegeist) - Browser extension using pi-web-ui -## Known Bugs +## Known Issues -- **PersistentStorageDialog**: Currently broken and commented out in examples +- **PersistentStorageDialog**: Currently broken ## License