Complete web-ui README rewrite with full API documentation

- Architecture diagram
- All components (ChatPanel, AgentInterface)
- Message types and custom message extension pattern
- Tools (JavaScript REPL, Extract Document, Artifacts)
- Storage system (all stores, backends)
- Attachments processing
- CORS proxy configuration
- Dialogs
- Internationalization
This commit is contained in:
Mario Zechner 2025-12-31 00:01:55 +01:00
parent 3e11b3e68b
commit a65a313a9a

View file

@ -6,13 +6,13 @@ Built with [mini-lit](https://github.com/badlogic/mini-lit) web components and T
## Features
- Modern Chat Interface: Complete chat UI with message history, streaming responses, and tool execution
- Tool Support: Built-in renderers for common tools plus custom tool rendering
- Attachments: PDF, Office documents, images with preview and text extraction
- Artifacts: HTML, SVG, Markdown, and text artifact rendering with sandboxed execution
- CORS Proxy Support: Automatic proxy handling for browser environments
- Platform Agnostic: Works in browser extensions, web apps, VS Code extensions, Electron apps
- TypeScript: Full type safety
- **Chat UI**: Complete interface with message history, streaming, and tool execution
- **Tools**: JavaScript REPL, document extraction, and artifacts (HTML, SVG, Markdown, etc.)
- **Attachments**: PDF, DOCX, XLSX, PPTX, images with preview and text extraction
- **Artifacts**: Interactive HTML, SVG, Markdown with sandboxed execution
- **Storage**: IndexedDB-backed storage for sessions, API keys, and settings
- **CORS Proxy**: Automatic proxy handling for browser environments
- **Custom Providers**: Support for Ollama, LM Studio, vLLM, and OpenAI-compatible APIs
## Installation
@ -36,6 +36,7 @@ import {
SettingsStore,
setAppStorage,
defaultConvertToLlm,
ApiKeyPromptDialog,
} from '@mariozechner/pi-web-ui';
import '@mariozechner/pi-web-ui/app.css';
@ -47,7 +48,12 @@ const sessions = new SessionsStore();
const backend = new IndexedDBStorageBackend({
dbName: 'my-app',
version: 1,
stores: [settings.getConfig(), providerKeys.getConfig(), sessions.getConfig()],
stores: [
settings.getConfig(),
providerKeys.getConfig(),
sessions.getConfig(),
SessionsStore.getMetadataConfig(),
],
});
settings.setBackend(backend);
@ -69,84 +75,101 @@ const agent = new Agent({
convertToLlm: defaultConvertToLlm,
});
// Create chat panel and attach agent
// Create chat panel
const chatPanel = new ChatPanel();
await chatPanel.setAgent(agent, {
onApiKeyRequired: async (provider) => {
// Prompt user for API key
return await ApiKeyPromptDialog.prompt(provider);
},
onApiKeyRequired: (provider) => ApiKeyPromptDialog.prompt(provider),
});
document.body.appendChild(chatPanel);
```
**Run the example:**
```bash
cd example
npm install
npm run dev
```
## Architecture
The web-ui package provides UI components that work with the `Agent` class from `@mariozechner/pi-agent-core`. The Agent handles:
```
┌─────────────────────────────────────────────────────┐
│ ChatPanel │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ AgentInterface │ │ ArtifactsPanel │ │
│ │ (messages, input) │ │ (HTML, SVG, MD) │ │
│ └─────────────────────┘ └─────────────────────┘ │
└─────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────┐
│ Agent (from pi-agent-core) │
│ - State management (messages, model, tools) │
│ - Event emission (agent_start, message_update, ...) │
│ - Tool execution │
└─────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────┐
│ AppStorage │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Settings │ │ Provider │ │ Sessions │ │
│ │ Store │ │Keys Store│ │ Store │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │ │
│ IndexedDBStorageBackend │
└─────────────────────────────────────────────────────┘
```
- Conversation state management
- LLM streaming via `streamFn`
- Tool execution
- Event emission
The web-ui provides:
- `ChatPanel` / `AgentInterface`: UI components that subscribe to Agent events
- `defaultConvertToLlm`: Message transformer for web-ui custom message types
- Storage backends for API keys, sessions, and settings
- CORS proxy utilities for browser environments
## Core Components
## Components
### ChatPanel
High-level chat interface with artifacts panel support.
High-level chat interface with built-in artifacts panel.
```typescript
import { ChatPanel, ApiKeyPromptDialog } from '@mariozechner/pi-web-ui';
const chatPanel = new ChatPanel();
await chatPanel.setAgent(agent, {
// Prompt for API key when needed
onApiKeyRequired: async (provider) => ApiKeyPromptDialog.prompt(provider),
onBeforeSend: async () => { /* pre-send hook */ },
onCostClick: () => { /* cost display clicked */ },
// Hook before sending messages
onBeforeSend: async () => { /* save draft, etc. */ },
// Handle cost display click
onCostClick: () => { /* show cost breakdown */ },
// Custom sandbox URL for browser extensions
sandboxUrlProvider: () => chrome.runtime.getURL('sandbox.html'),
// Add custom tools
toolsFactory: (agent, agentInterface, artifactsPanel, runtimeProvidersFactory) => {
// Return additional tools
return [createJavaScriptReplTool()];
const replTool = createJavaScriptReplTool();
replTool.runtimeProvidersFactory = runtimeProvidersFactory;
return [replTool];
},
});
```
### AgentInterface
Lower-level chat interface for custom layouts (used internally by ChatPanel).
Lower-level chat interface for custom layouts.
```typescript
import { AgentInterface } from '@mariozechner/pi-web-ui';
const chat = document.createElement('agent-interface') as AgentInterface;
chat.session = agent;
chat.enableAttachments = true;
chat.enableModelSelector = true;
chat.enableThinkingSelector = true;
chat.onApiKeyRequired = async (provider) => { /* ... */ };
chat.onBeforeSend = async () => { /* ... */ };
```
Properties:
- `session`: Agent instance
- `enableAttachments`: Show attachment button (default: true)
- `enableModelSelector`: Show model selector (default: true)
- `enableThinkingSelector`: Show thinking level selector (default: true)
- `showThemeToggle`: Show theme toggle (default: false)
### Agent (from pi-agent-core)
The Agent class is imported from `@mariozechner/pi-agent-core`:
```typescript
import { Agent } from '@mariozechner/pi-agent-core';
import { defaultConvertToLlm } from '@mariozechner/pi-web-ui';
const agent = new Agent({
initialState: {
@ -159,93 +182,95 @@ const agent = new Agent({
convertToLlm: defaultConvertToLlm,
});
// Subscribe to events
// Events
agent.subscribe((event) => {
switch (event.type) {
case 'agent_start':
case 'agent_end':
case 'agent_start': // Agent loop started
case 'agent_end': // Agent loop finished
case 'turn_start': // LLM call started
case 'turn_end': // LLM call finished
case 'message_start':
case 'message_update':
case 'message_update': // Streaming update
case 'message_end':
case 'turn_start':
case 'turn_end':
// Handle events
break;
}
});
// Send a message
// Send message
await agent.prompt('Hello!');
await agent.prompt({ role: 'user-with-attachments', content: 'Check this', attachments, timestamp: Date.now() });
// Or with custom message type
await agent.prompt({
role: 'user-with-attachments',
content: 'Check this image',
attachments: [imageAttachment],
timestamp: Date.now(),
});
// Control
agent.abort();
agent.setModel(newModel);
agent.setThinkingLevel('medium');
agent.setTools([...]);
agent.queueMessage(customMessage);
```
## Message Types
### UserMessageWithAttachments
Custom message type for user messages with file attachments:
User message with file attachments:
```typescript
import { isUserMessageWithAttachments, type UserMessageWithAttachments } from '@mariozechner/pi-web-ui';
const message: UserMessageWithAttachments = {
role: 'user-with-attachments',
content: 'Analyze this document',
attachments: [pdfAttachment, imageAttachment],
attachments: [pdfAttachment],
timestamp: Date.now(),
};
// Type guard
if (isUserMessageWithAttachments(msg)) {
console.log(msg.attachments);
}
```
### ArtifactMessage
For session persistence of created artifacts:
For session persistence of artifacts:
```typescript
import { isArtifactMessage, type ArtifactMessage } from '@mariozechner/pi-web-ui';
const artifact: ArtifactMessage = {
role: 'artifact',
artifactId: 'chart-1',
type: 'html',
title: 'Sales Chart',
action: 'create', // or 'update', 'delete'
filename: 'chart.html',
content: '<div>...</div>',
timestamp: new Date().toISOString(),
};
// Type guard
if (isArtifactMessage(msg)) {
console.log(msg.filename);
}
```
### Custom Message Types
Extend `CustomAgentMessages` from pi-agent-core:
Extend via declaration merging:
```typescript
// Define your custom message
interface SystemNotificationMessage {
interface SystemNotification {
role: 'system-notification';
message: string;
level: 'info' | 'warning' | 'error';
timestamp: string;
}
// Register with pi-agent-core's type system
declare module '@mariozechner/pi-agent-core' {
interface CustomAgentMessages {
'system-notification': SystemNotificationMessage;
'system-notification': SystemNotification;
}
}
// Register a renderer
// Register renderer
registerMessageRenderer('system-notification', {
render: (msg) => html`<div class="notification">${msg.message}</div>`,
render: (msg) => html`<div class="alert">${msg.message}</div>`,
});
// Extend convertToLlm to handle your type
// Extend convertToLlm
function myConvertToLlm(messages: AgentMessage[]): Message[] {
const processed = messages.map((m) => {
if (m.role === 'system-notification') {
@ -259,66 +284,73 @@ function myConvertToLlm(messages: AgentMessage[]): Message[] {
## Message Transformer
The `convertToLlm` function transforms app messages to LLM-compatible format:
`convertToLlm` transforms app messages to LLM-compatible format:
```typescript
import { defaultConvertToLlm, convertAttachments } from '@mariozechner/pi-web-ui';
// defaultConvertToLlm handles:
// - UserMessageWithAttachments → user message with content blocks
// - UserMessageWithAttachments → user message with image/text content blocks
// - ArtifactMessage → filtered out (UI-only)
// - Standard messages (user, assistant, toolResult) → passed through
// For custom types, wrap defaultConvertToLlm:
const agent = new Agent({
convertToLlm: (messages) => {
const processed = messages.map(m => {
// Handle your custom types
return m;
});
return defaultConvertToLlm(processed);
},
});
```
## CORS Proxy
## Tools
Browser environments may need a CORS proxy for certain providers:
### JavaScript REPL
Execute JavaScript in a sandboxed browser environment:
```typescript
import {
createStreamFn,
shouldUseProxyForProvider,
applyProxyIfNeeded,
isCorsError,
} from '@mariozechner/pi-web-ui';
import { createJavaScriptReplTool } from '@mariozechner/pi-web-ui';
// AgentInterface automatically sets up proxy support if using AppStorage
// For manual setup:
agent.streamFn = createStreamFn(async () => {
const enabled = await storage.settings.get<boolean>('proxy.enabled');
return enabled ? await storage.settings.get<string>('proxy.url') : undefined;
});
const replTool = createJavaScriptReplTool();
// Configure runtime providers for artifact/attachment access
replTool.runtimeProvidersFactory = () => [
new AttachmentsRuntimeProvider(attachments),
new ArtifactsRuntimeProvider(artifactsPanel, agent, true), // read-write
];
agent.setTools([replTool]);
```
Providers requiring proxy:
- `zai`: Always requires proxy
- `anthropic`: Only OAuth tokens (`sk-ant-oat-*`) require proxy
### Extract Document
## Tool Renderers
Extract text from documents at URLs:
Customize how tool calls are displayed:
```typescript
import { createExtractDocumentTool } from '@mariozechner/pi-web-ui';
const extractTool = createExtractDocumentTool();
extractTool.corsProxyUrl = 'https://corsproxy.io/?';
agent.setTools([extractTool]);
```
### Artifacts Tool
Built into ArtifactsPanel, supports: HTML, SVG, Markdown, text, JSON, images, PDF, DOCX, XLSX.
```typescript
const artifactsPanel = new ArtifactsPanel();
artifactsPanel.agent = agent;
// The tool is available as artifactsPanel.tool
agent.setTools([artifactsPanel.tool]);
```
### Custom Tool Renderers
```typescript
import { registerToolRenderer, type ToolRenderer } from '@mariozechner/pi-web-ui';
import { html } from 'lit';
const myRenderer: ToolRenderer = {
renderParams(params, isStreaming) {
return html`<div>Calling with: ${JSON.stringify(params)}</div>`;
},
renderResult(params, result) {
return html`<div>Result: ${result.output}</div>`;
render(params, result, isStreaming) {
return {
content: html`<div>...</div>`,
isCustom: false, // true = no card wrapper
};
},
};
@ -327,17 +359,15 @@ registerToolRenderer('my_tool', myRenderer);
## Storage
### AppStorage
Central storage configuration:
### Setup
```typescript
import {
AppStorage,
IndexedDBStorageBackend,
SettingsStore,
ProviderKeysStore,
SessionsStore,
SettingsStore,
CustomProvidersStore,
setAppStorage,
getAppStorage,
@ -349,7 +379,7 @@ const providerKeys = new ProviderKeysStore();
const sessions = new SessionsStore();
const customProviders = new CustomProvidersStore();
// Create backend
// Create backend with all store configs
const backend = new IndexedDBStorageBackend({
dbName: 'my-app',
version: 1,
@ -357,6 +387,7 @@ const backend = new IndexedDBStorageBackend({
settings.getConfig(),
providerKeys.getConfig(),
sessions.getConfig(),
SessionsStore.getMetadataConfig(),
customProviders.getConfig(),
],
});
@ -367,14 +398,118 @@ providerKeys.setBackend(backend);
sessions.setBackend(backend);
customProviders.setBackend(backend);
// Create and set app storage
// Create and set global storage
const storage = new AppStorage(settings, providerKeys, sessions, customProviders, backend);
setAppStorage(storage);
```
// Access anywhere
const storage = getAppStorage();
await storage.providerKeys.set('anthropic', 'sk-...');
### SettingsStore
Key-value settings:
```typescript
await storage.settings.set('proxy.enabled', true);
await storage.settings.set('proxy.url', 'https://proxy.example.com');
const enabled = await storage.settings.get<boolean>('proxy.enabled');
```
### ProviderKeysStore
API keys by provider:
```typescript
await storage.providerKeys.set('anthropic', 'sk-ant-...');
const key = await storage.providerKeys.get('anthropic');
const providers = await storage.providerKeys.list();
```
### SessionsStore
Chat sessions with metadata:
```typescript
// Save session
await storage.sessions.save(sessionData, metadata);
// Load session
const data = await storage.sessions.get(sessionId);
const metadata = await storage.sessions.getMetadata(sessionId);
// List sessions (sorted by lastModified)
const allMetadata = await storage.sessions.getAllMetadata();
// Update title
await storage.sessions.updateTitle(sessionId, 'New Title');
// Delete
await storage.sessions.delete(sessionId);
```
### CustomProvidersStore
Custom LLM providers:
```typescript
const provider: CustomProvider = {
id: crypto.randomUUID(),
name: 'My Ollama',
type: 'ollama',
baseUrl: 'http://localhost:11434',
};
await storage.customProviders.set(provider);
const all = await storage.customProviders.getAll();
```
## Attachments
Load and process files:
```typescript
import { loadAttachment, type Attachment } from '@mariozechner/pi-web-ui';
// From File input
const file = inputElement.files[0];
const attachment = await loadAttachment(file);
// From URL
const attachment = await loadAttachment('https://example.com/doc.pdf');
// From ArrayBuffer
const attachment = await loadAttachment(arrayBuffer, 'document.pdf');
// Attachment structure
interface Attachment {
id: string;
type: 'image' | 'document';
fileName: string;
mimeType: string;
size: number;
content: string; // base64 encoded
extractedText?: string; // For documents
preview?: string; // base64 preview image
}
```
Supported formats: PDF, DOCX, XLSX, PPTX, images, text files.
## CORS Proxy
For browser environments with CORS restrictions:
```typescript
import { createStreamFn, shouldUseProxyForProvider, isCorsError } from '@mariozechner/pi-web-ui';
// AgentInterface auto-configures proxy from settings
// For manual setup:
agent.streamFn = createStreamFn(async () => {
const enabled = await storage.settings.get<boolean>('proxy.enabled');
return enabled ? await storage.settings.get<string>('proxy.url') : undefined;
});
// Providers requiring proxy:
// - zai: always
// - anthropic: only OAuth tokens (sk-ant-oat-*)
```
## Dialogs
@ -382,9 +517,13 @@ await storage.sessions.save(sessionData, metadata);
### SettingsDialog
```typescript
import { SettingsDialog, ProvidersModelsTab, ProxyTab } from '@mariozechner/pi-web-ui';
import { SettingsDialog, ProvidersModelsTab, ProxyTab, ApiKeysTab } from '@mariozechner/pi-web-ui';
SettingsDialog.open([new ProvidersModelsTab(), new ProxyTab()]);
SettingsDialog.open([
new ProvidersModelsTab(), // Custom providers + model list
new ProxyTab(), // CORS proxy settings
new ApiKeysTab(), // API keys per provider
]);
```
### SessionListDialog
@ -406,6 +545,16 @@ import { ApiKeyPromptDialog } from '@mariozechner/pi-web-ui';
const success = await ApiKeyPromptDialog.prompt('anthropic');
```
### ModelSelector
```typescript
import { ModelSelector } from '@mariozechner/pi-web-ui';
ModelSelector.open(currentModel, (selectedModel) => {
agent.setModel(selectedModel);
});
```
## Styling
Import the pre-built CSS:
@ -414,7 +563,7 @@ Import the pre-built CSS:
import '@mariozechner/pi-web-ui/app.css';
```
Or customize with your own Tailwind config:
Or use Tailwind with custom config:
```css
@import '@mariozechner/mini-lit/themes/claude.css';
@ -423,14 +572,29 @@ Or customize with your own Tailwind config:
@tailwind utilities;
```
## Internationalization
```typescript
import { i18n, setLanguage, translations } from '@mariozechner/pi-web-ui';
// Add translations
translations.de = {
'Loading...': 'Laden...',
'No sessions yet': 'Noch keine Sitzungen',
};
setLanguage('de');
console.log(i18n('Loading...')); // "Laden..."
```
## Examples
- [example/](./example) - Complete web application with sessions, artifacts, and custom messages
- [example/](./example) - Complete web app with sessions, artifacts, custom messages
- [sitegeist](https://github.com/badlogic/sitegeist) - Browser extension using pi-web-ui
## Known Bugs
## Known Issues
- **PersistentStorageDialog**: Currently broken and commented out in examples
- **PersistentStorageDialog**: Currently broken
## License