getcompanion-ai/co-mono
1
0
Fork 0
mirror of https://github.com/getcompanion-ai/co-mono.git synced 2026-04-21 03:04:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Complete web-ui README rewrite with full API documentation

Browse source 
- Architecture diagram
- All components (ChatPanel, AgentInterface)
- Message types and custom message extension pattern
- Tools (JavaScript REPL, Extract Document, Artifacts)
- Storage system (all stores, backends)
- Attachments processing
- CORS proxy configuration
- Dialogs
- Internationalization
This commit is contained in:
Mario Zechner 2025-12-31 00:01:55 +01:00
parent 3e11b3e68b
commit a65a313a9a
1 changed files with 298 additions and 134 deletions
Download patch file Download diff file Expand all files Collapse all files

432
packages/web-ui/README.md
View file

@ -6,13 +6,13 @@ Built with [mini-lit](https://github.com/badlogic/mini-lit) web components and T
## Features ## Features
- Modern Chat Interface: Complete chat UI with message history, streaming responses, and tool execution - **Chat UI**: Complete interface with message history, streaming, and tool execution
- Tool Support: Built-in renderers for common tools plus custom tool rendering - **Tools**: JavaScript REPL, document extraction, and artifacts (HTML, SVG, Markdown, etc.)
- Attachments: PDF, Office documents, images with preview and text extraction - **Attachments**: PDF, DOCX, XLSX, PPTX, images with preview and text extraction
- Artifacts: HTML, SVG, Markdown, and text artifact rendering with sandboxed execution - **Artifacts**: Interactive HTML, SVG, Markdown with sandboxed execution
- CORS Proxy Support: Automatic proxy handling for browser environments - **Storage**: IndexedDB-backed storage for sessions, API keys, and settings
- Platform Agnostic: Works in browser extensions, web apps, VS Code extensions, Electron apps - **CORS Proxy**: Automatic proxy handling for browser environments
- TypeScript: Full type safety - **Custom Providers**: Support for Ollama, LM Studio, vLLM, and OpenAI-compatible APIs
## Installation ## Installation
@ -36,6 +36,7 @@ import {
SettingsStore, SettingsStore,
setAppStorage, setAppStorage,
defaultConvertToLlm, defaultConvertToLlm,
ApiKeyPromptDialog,
} from '@mariozechner/pi-web-ui'; } from '@mariozechner/pi-web-ui';
import '@mariozechner/pi-web-ui/app.css'; import '@mariozechner/pi-web-ui/app.css';
@ -47,7 +48,12 @@ const sessions = new SessionsStore();
const backend = new IndexedDBStorageBackend({ const backend = new IndexedDBStorageBackend({
dbName: 'my-app', dbName: 'my-app',
version: 1, version: 1,
stores: [settings.getConfig(), providerKeys.getConfig(), sessions.getConfig()], stores: [
settings.getConfig(),
providerKeys.getConfig(),
sessions.getConfig(),
SessionsStore.getMetadataConfig(),
],
}); });
settings.setBackend(backend); settings.setBackend(backend);
@ -69,84 +75,101 @@ const agent = new Agent({
convertToLlm: defaultConvertToLlm, convertToLlm: defaultConvertToLlm,
}); });
// Create chat panel and attach agent // Create chat panel
const chatPanel = new ChatPanel(); const chatPanel = new ChatPanel();
await chatPanel.setAgent(agent, { await chatPanel.setAgent(agent, {
onApiKeyRequired: async (provider) => { onApiKeyRequired: (provider) => ApiKeyPromptDialog.prompt(provider),
// Prompt user for API key
return await ApiKeyPromptDialog.prompt(provider);
},
}); });
document.body.appendChild(chatPanel); document.body.appendChild(chatPanel);
``` ```
**Run the example:**
```bash
cd example
npm install
npm run dev
```
## Architecture ## Architecture
The web-ui package provides UI components that work with the `Agent` class from `@mariozechner/pi-agent-core`. The Agent handles: ```
┌─────────────────────────────────────────────────────┐
│ ChatPanel │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ AgentInterface │ │ ArtifactsPanel │ │
│ │ (messages, input) │ │ (HTML, SVG, MD) │ │
│ └─────────────────────┘ └─────────────────────┘ │
└─────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────┐
│ Agent (from pi-agent-core) │
│ - State management (messages, model, tools) │
│ - Event emission (agent_start, message_update, ...) │
│ - Tool execution │
└─────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────┐
│ AppStorage │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Settings │ │ Provider │ │ Sessions │ │
│ │ Store │ │Keys Store│ │ Store │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │ │
│ IndexedDBStorageBackend │
└─────────────────────────────────────────────────────┘
```
- Conversation state management ## Components
- LLM streaming via `streamFn`
- Tool execution
- Event emission
The web-ui provides:
- `ChatPanel` / `AgentInterface`: UI components that subscribe to Agent events
- `defaultConvertToLlm`: Message transformer for web-ui custom message types
- Storage backends for API keys, sessions, and settings
- CORS proxy utilities for browser environments
## Core Components
### ChatPanel ### ChatPanel
High-level chat interface with artifacts panel support. High-level chat interface with built-in artifacts panel.
```typescript ```typescript
import { ChatPanel, ApiKeyPromptDialog } from '@mariozechner/pi-web-ui';
const chatPanel = new ChatPanel(); const chatPanel = new ChatPanel();
await chatPanel.setAgent(agent, { await chatPanel.setAgent(agent, {
// Prompt for API key when needed
onApiKeyRequired: async (provider) => ApiKeyPromptDialog.prompt(provider), onApiKeyRequired: async (provider) => ApiKeyPromptDialog.prompt(provider),
onBeforeSend: async () => { /* pre-send hook */ },
onCostClick: () => { /* cost display clicked */ }, // Hook before sending messages
onBeforeSend: async () => { /* save draft, etc. */ },
// Handle cost display click
onCostClick: () => { /* show cost breakdown */ },
// Custom sandbox URL for browser extensions
sandboxUrlProvider: () => chrome.runtime.getURL('sandbox.html'),
// Add custom tools
toolsFactory: (agent, agentInterface, artifactsPanel, runtimeProvidersFactory) => { toolsFactory: (agent, agentInterface, artifactsPanel, runtimeProvidersFactory) => {
// Return additional tools const replTool = createJavaScriptReplTool();
return [createJavaScriptReplTool()]; replTool.runtimeProvidersFactory = runtimeProvidersFactory;
return [replTool];
}, },
}); });
``` ```
### AgentInterface ### AgentInterface
Lower-level chat interface for custom layouts (used internally by ChatPanel). Lower-level chat interface for custom layouts.
```typescript ```typescript
import { AgentInterface } from '@mariozechner/pi-web-ui';
const chat = document.createElement('agent-interface') as AgentInterface; const chat = document.createElement('agent-interface') as AgentInterface;
chat.session = agent; chat.session = agent;
chat.enableAttachments = true; chat.enableAttachments = true;
chat.enableModelSelector = true; chat.enableModelSelector = true;
chat.enableThinkingSelector = true;
chat.onApiKeyRequired = async (provider) => { /* ... */ }; chat.onApiKeyRequired = async (provider) => { /* ... */ };
chat.onBeforeSend = async () => { /* ... */ };
``` ```
Properties:
- `session`: Agent instance
- `enableAttachments`: Show attachment button (default: true)
- `enableModelSelector`: Show model selector (default: true)
- `enableThinkingSelector`: Show thinking level selector (default: true)
- `showThemeToggle`: Show theme toggle (default: false)
### Agent (from pi-agent-core) ### Agent (from pi-agent-core)
The Agent class is imported from `@mariozechner/pi-agent-core`:
```typescript ```typescript
import { Agent } from '@mariozechner/pi-agent-core'; import { Agent } from '@mariozechner/pi-agent-core';
import { defaultConvertToLlm } from '@mariozechner/pi-web-ui';
const agent = new Agent({ const agent = new Agent({
initialState: { initialState: {
@ -159,93 +182,95 @@ const agent = new Agent({
convertToLlm: defaultConvertToLlm, convertToLlm: defaultConvertToLlm,
}); });
// Subscribe to events // Events
agent.subscribe((event) => { agent.subscribe((event) => {
switch (event.type) { switch (event.type) {
case 'agent_start': case 'agent_start': // Agent loop started
case 'agent_end': case 'agent_end': // Agent loop finished
case 'turn_start': // LLM call started
case 'turn_end': // LLM call finished
case 'message_start': case 'message_start':
case 'message_update': case 'message_update': // Streaming update
case 'message_end': case 'message_end':
case 'turn_start':
case 'turn_end':
// Handle events
break; break;
} }
}); });
// Send a message // Send message
await agent.prompt('Hello!'); await agent.prompt('Hello!');
await agent.prompt({ role: 'user-with-attachments', content: 'Check this', attachments, timestamp: Date.now() });
// Or with custom message type // Control
await agent.prompt({ agent.abort();
role: 'user-with-attachments', agent.setModel(newModel);
content: 'Check this image', agent.setThinkingLevel('medium');
attachments: [imageAttachment], agent.setTools([...]);
timestamp: Date.now(), agent.queueMessage(customMessage);
});
``` ```
## Message Types ## Message Types
### UserMessageWithAttachments ### UserMessageWithAttachments
Custom message type for user messages with file attachments: User message with file attachments:
```typescript ```typescript
import { isUserMessageWithAttachments, type UserMessageWithAttachments } from '@mariozechner/pi-web-ui';
const message: UserMessageWithAttachments = { const message: UserMessageWithAttachments = {
role: 'user-with-attachments', role: 'user-with-attachments',
content: 'Analyze this document', content: 'Analyze this document',
attachments: [pdfAttachment, imageAttachment], attachments: [pdfAttachment],
timestamp: Date.now(), timestamp: Date.now(),
}; };
// Type guard
if (isUserMessageWithAttachments(msg)) {
console.log(msg.attachments);
}
``` ```
### ArtifactMessage ### ArtifactMessage
For session persistence of created artifacts: For session persistence of artifacts:
```typescript ```typescript
import { isArtifactMessage, type ArtifactMessage } from '@mariozechner/pi-web-ui';
const artifact: ArtifactMessage = { const artifact: ArtifactMessage = {
role: 'artifact', role: 'artifact',
artifactId: 'chart-1', action: 'create', // or 'update', 'delete'
type: 'html', filename: 'chart.html',
title: 'Sales Chart',
content: '<div>...</div>', content: '<div>...</div>',
timestamp: new Date().toISOString(), timestamp: new Date().toISOString(),
}; };
// Type guard
if (isArtifactMessage(msg)) {
console.log(msg.filename);
}
``` ```
### Custom Message Types ### Custom Message Types
Extend `CustomAgentMessages` from pi-agent-core: Extend via declaration merging:
```typescript ```typescript
// Define your custom message interface SystemNotification {
interface SystemNotificationMessage {
role: 'system-notification'; role: 'system-notification';
message: string; message: string;
level: 'info' | 'warning' | 'error'; level: 'info' | 'warning' | 'error';
timestamp: string; timestamp: string;
} }
// Register with pi-agent-core's type system
declare module '@mariozechner/pi-agent-core' { declare module '@mariozechner/pi-agent-core' {
interface CustomAgentMessages { interface CustomAgentMessages {
'system-notification': SystemNotificationMessage; 'system-notification': SystemNotification;
} }
} }
// Register a renderer // Register renderer
registerMessageRenderer('system-notification', { registerMessageRenderer('system-notification', {
render: (msg) => html`<div class="notification">${msg.message}</div>`, render: (msg) => html`<div class="alert">${msg.message}</div>`,
}); });
// Extend convertToLlm to handle your type // Extend convertToLlm
function myConvertToLlm(messages: AgentMessage[]): Message[] { function myConvertToLlm(messages: AgentMessage[]): Message[] {
const processed = messages.map((m) => { const processed = messages.map((m) => {
if (m.role === 'system-notification') { if (m.role === 'system-notification') {
@ -259,66 +284,73 @@ function myConvertToLlm(messages: AgentMessage[]): Message[] {
## Message Transformer ## Message Transformer
The `convertToLlm` function transforms app messages to LLM-compatible format: `convertToLlm` transforms app messages to LLM-compatible format:
```typescript ```typescript
import { defaultConvertToLlm, convertAttachments } from '@mariozechner/pi-web-ui'; import { defaultConvertToLlm, convertAttachments } from '@mariozechner/pi-web-ui';
// defaultConvertToLlm handles: // defaultConvertToLlm handles:
// - UserMessageWithAttachments → user message with content blocks // - UserMessageWithAttachments → user message with image/text content blocks
// - ArtifactMessage → filtered out (UI-only) // - ArtifactMessage → filtered out (UI-only)
// - Standard messages (user, assistant, toolResult) → passed through // - Standard messages (user, assistant, toolResult) → passed through
// For custom types, wrap defaultConvertToLlm:
const agent = new Agent({
convertToLlm: (messages) => {
const processed = messages.map(m => {
// Handle your custom types
return m;
});
return defaultConvertToLlm(processed);
},
});
``` ```
## CORS Proxy ## Tools
Browser environments may need a CORS proxy for certain providers: ### JavaScript REPL
Execute JavaScript in a sandboxed browser environment:
```typescript ```typescript
import { import { createJavaScriptReplTool } from '@mariozechner/pi-web-ui';
createStreamFn,
shouldUseProxyForProvider,
applyProxyIfNeeded,
isCorsError,
} from '@mariozechner/pi-web-ui';
// AgentInterface automatically sets up proxy support if using AppStorage const replTool = createJavaScriptReplTool();
// For manual setup:
agent.streamFn = createStreamFn(async () => { // Configure runtime providers for artifact/attachment access
const enabled = await storage.settings.get<boolean>('proxy.enabled'); replTool.runtimeProvidersFactory = () => [
return enabled ? await storage.settings.get<string>('proxy.url') : undefined; new AttachmentsRuntimeProvider(attachments),
}); new ArtifactsRuntimeProvider(artifactsPanel, agent, true), // read-write
];
agent.setTools([replTool]);
``` ```
Providers requiring proxy: ### Extract Document
- `zai`: Always requires proxy
- `anthropic`: Only OAuth tokens (`sk-ant-oat-*`) require proxy
## Tool Renderers Extract text from documents at URLs:
Customize how tool calls are displayed: ```typescript
import { createExtractDocumentTool } from '@mariozechner/pi-web-ui';
const extractTool = createExtractDocumentTool();
extractTool.corsProxyUrl = 'https://corsproxy.io/?';
agent.setTools([extractTool]);
```
### Artifacts Tool
Built into ArtifactsPanel, supports: HTML, SVG, Markdown, text, JSON, images, PDF, DOCX, XLSX.
```typescript
const artifactsPanel = new ArtifactsPanel();
artifactsPanel.agent = agent;
// The tool is available as artifactsPanel.tool
agent.setTools([artifactsPanel.tool]);
```
### Custom Tool Renderers
```typescript ```typescript
import { registerToolRenderer, type ToolRenderer } from '@mariozechner/pi-web-ui'; import { registerToolRenderer, type ToolRenderer } from '@mariozechner/pi-web-ui';
import { html } from 'lit';
const myRenderer: ToolRenderer = { const myRenderer: ToolRenderer = {
renderParams(params, isStreaming) { render(params, result, isStreaming) {
return html`<div>Calling with: ${JSON.stringify(params)}</div>`; return {
}, content: html`<div>...</div>`,
renderResult(params, result) { isCustom: false, // true = no card wrapper
return html`<div>Result: ${result.output}</div>`; };
}, },
}; };
@ -327,17 +359,15 @@ registerToolRenderer('my_tool', myRenderer);
## Storage ## Storage
### AppStorage ### Setup
Central storage configuration:
```typescript ```typescript
import { import {
AppStorage, AppStorage,
IndexedDBStorageBackend, IndexedDBStorageBackend,
SettingsStore,
ProviderKeysStore, ProviderKeysStore,
SessionsStore, SessionsStore,
SettingsStore,
CustomProvidersStore, CustomProvidersStore,
setAppStorage, setAppStorage,
getAppStorage, getAppStorage,
@ -349,7 +379,7 @@ const providerKeys = new ProviderKeysStore();
const sessions = new SessionsStore(); const sessions = new SessionsStore();
const customProviders = new CustomProvidersStore(); const customProviders = new CustomProvidersStore();
// Create backend // Create backend with all store configs
const backend = new IndexedDBStorageBackend({ const backend = new IndexedDBStorageBackend({
dbName: 'my-app', dbName: 'my-app',
version: 1, version: 1,
@ -357,6 +387,7 @@ const backend = new IndexedDBStorageBackend({
settings.getConfig(), settings.getConfig(),
providerKeys.getConfig(), providerKeys.getConfig(),
sessions.getConfig(), sessions.getConfig(),
SessionsStore.getMetadataConfig(),
customProviders.getConfig(), customProviders.getConfig(),
], ],
}); });
@ -367,14 +398,118 @@ providerKeys.setBackend(backend);
sessions.setBackend(backend); sessions.setBackend(backend);
customProviders.setBackend(backend); customProviders.setBackend(backend);
// Create and set app storage // Create and set global storage
const storage = new AppStorage(settings, providerKeys, sessions, customProviders, backend); const storage = new AppStorage(settings, providerKeys, sessions, customProviders, backend);
setAppStorage(storage); setAppStorage(storage);
```
// Access anywhere ### SettingsStore
const storage = getAppStorage();
await storage.providerKeys.set('anthropic', 'sk-...'); Key-value settings:
```typescript
await storage.settings.set('proxy.enabled', true);
await storage.settings.set('proxy.url', 'https://proxy.example.com');
const enabled = await storage.settings.get<boolean>('proxy.enabled');
```
### ProviderKeysStore
API keys by provider:
```typescript
await storage.providerKeys.set('anthropic', 'sk-ant-...');
const key = await storage.providerKeys.get('anthropic');
const providers = await storage.providerKeys.list();
```
### SessionsStore
Chat sessions with metadata:
```typescript
// Save session
await storage.sessions.save(sessionData, metadata); await storage.sessions.save(sessionData, metadata);
// Load session
const data = await storage.sessions.get(sessionId);
const metadata = await storage.sessions.getMetadata(sessionId);
// List sessions (sorted by lastModified)
const allMetadata = await storage.sessions.getAllMetadata();
// Update title
await storage.sessions.updateTitle(sessionId, 'New Title');
// Delete
await storage.sessions.delete(sessionId);
```
### CustomProvidersStore
Custom LLM providers:
```typescript
const provider: CustomProvider = {
id: crypto.randomUUID(),
name: 'My Ollama',
type: 'ollama',
baseUrl: 'http://localhost:11434',
};
await storage.customProviders.set(provider);
const all = await storage.customProviders.getAll();
```
## Attachments
Load and process files:
```typescript
import { loadAttachment, type Attachment } from '@mariozechner/pi-web-ui';
// From File input
const file = inputElement.files[0];
const attachment = await loadAttachment(file);
// From URL
const attachment = await loadAttachment('https://example.com/doc.pdf');
// From ArrayBuffer
const attachment = await loadAttachment(arrayBuffer, 'document.pdf');
// Attachment structure
interface Attachment {
id: string;
type: 'image' | 'document';
fileName: string;
mimeType: string;
size: number;
content: string; // base64 encoded
extractedText?: string; // For documents
preview?: string; // base64 preview image
}
```
Supported formats: PDF, DOCX, XLSX, PPTX, images, text files.
## CORS Proxy
For browser environments with CORS restrictions:
```typescript
import { createStreamFn, shouldUseProxyForProvider, isCorsError } from '@mariozechner/pi-web-ui';
// AgentInterface auto-configures proxy from settings
// For manual setup:
agent.streamFn = createStreamFn(async () => {
const enabled = await storage.settings.get<boolean>('proxy.enabled');
return enabled ? await storage.settings.get<string>('proxy.url') : undefined;
});
// Providers requiring proxy:
// - zai: always
// - anthropic: only OAuth tokens (sk-ant-oat-*)
``` ```
## Dialogs ## Dialogs
@ -382,9 +517,13 @@ await storage.sessions.save(sessionData, metadata);
### SettingsDialog ### SettingsDialog
```typescript ```typescript
import { SettingsDialog, ProvidersModelsTab, ProxyTab } from '@mariozechner/pi-web-ui'; import { SettingsDialog, ProvidersModelsTab, ProxyTab, ApiKeysTab } from '@mariozechner/pi-web-ui';
SettingsDialog.open([new ProvidersModelsTab(), new ProxyTab()]); SettingsDialog.open([
new ProvidersModelsTab(), // Custom providers + model list
new ProxyTab(), // CORS proxy settings
new ApiKeysTab(), // API keys per provider
]);
``` ```
### SessionListDialog ### SessionListDialog
@ -406,6 +545,16 @@ import { ApiKeyPromptDialog } from '@mariozechner/pi-web-ui';
const success = await ApiKeyPromptDialog.prompt('anthropic'); const success = await ApiKeyPromptDialog.prompt('anthropic');
``` ```
### ModelSelector
```typescript
import { ModelSelector } from '@mariozechner/pi-web-ui';
ModelSelector.open(currentModel, (selectedModel) => {
agent.setModel(selectedModel);
});
```
## Styling ## Styling
Import the pre-built CSS: Import the pre-built CSS:
@ -414,7 +563,7 @@ Import the pre-built CSS:
import '@mariozechner/pi-web-ui/app.css'; import '@mariozechner/pi-web-ui/app.css';
``` ```
Or customize with your own Tailwind config: Or use Tailwind with custom config:
```css ```css
@import '@mariozechner/mini-lit/themes/claude.css'; @import '@mariozechner/mini-lit/themes/claude.css';
@ -423,14 +572,29 @@ Or customize with your own Tailwind config:
@tailwind utilities; @tailwind utilities;
``` ```
## Internationalization
```typescript
import { i18n, setLanguage, translations } from '@mariozechner/pi-web-ui';
// Add translations
translations.de = {
'Loading...': 'Laden...',
'No sessions yet': 'Noch keine Sitzungen',
};
setLanguage('de');
console.log(i18n('Loading...')); // "Laden..."
```
## Examples ## Examples
- [example/](./example) - Complete web application with sessions, artifacts, and custom messages - [example/](./example) - Complete web app with sessions, artifacts, custom messages
- [sitegeist](https://github.com/badlogic/sitegeist) - Browser extension using pi-web-ui - [sitegeist](https://github.com/badlogic/sitegeist) - Browser extension using pi-web-ui
## Known Bugs ## Known Issues
- **PersistentStorageDialog**: Currently broken and commented out in examples - **PersistentStorageDialog**: Currently broken
## License ## License