Add Anthropic prompt caching, pluggable storage, and CORS proxy support

Storage Architecture:
- New pluggable storage system with backends (LocalStorage, ChromeStorage, IndexedDB)
- SettingsRepository for app settings (proxy config, etc.)
- ProviderKeysRepository for API key management
- AppStorage with global accessors (getAppStorage, setAppStorage, initAppStorage)

Transport Refactoring:
- Renamed DirectTransport → ProviderTransport (calls LLM providers with optional CORS proxy)
- Renamed ProxyTransport → AppTransport (uses app server with user auth)
- Updated TransportMode: "direct" → "provider", "proxy" → "app"

CORS Proxy Integration:
- ProviderTransport checks proxy.enabled/proxy.url from storage
- When enabled, modifies model baseUrl to route through proxy: {proxyUrl}/?url={originalBaseUrl}
- ProviderKeyInput test function also honors proxy settings
- Settings dialog with Proxy tab (Switch toggle, URL input, explanatory description)

Anthropic Prompt Caching:
- System prompt cached with cache_control markers (both OAuth and regular API keys)
- Last user message cached to cache conversation history
- Saves 90% on input tokens for cached content (10x cost reduction)

Settings Dialog Improvements:
- Configurable tab system with SettingsTab base class
- ApiKeysTab and ProxyTab as custom elements
- Switch toggle for proxy enable (instead of Checkbox)
- Explanatory paragraphs for each tab
- ApiKeyPromptDialog reuses ProviderKeyInput component

Removed:
- Deprecated ApiKeysDialog (replaced by ProviderKeyInput in SettingsDialog)
- Old storage-adapter and key-store (replaced by new storage architecture)
This commit is contained in:
Mario Zechner 2025-10-05 23:00:36 +02:00
parent 66f092c0c6
commit 0496651308
31 changed files with 1141 additions and 488 deletions

View file

@ -291,7 +291,16 @@ function buildParams(
});
}
} else if (context.systemPrompt) {
params.system = context.systemPrompt;
// Add cache control to system prompt for non-OAuth tokens
params.system = [
{
type: "text",
text: context.systemPrompt,
cache_control: {
type: "ephemeral",
},
},
];
}
if (options?.temperature !== undefined) {
@ -440,6 +449,24 @@ function convertMessages(messages: Message[], model: Model<"anthropic-messages">
});
}
}
// Add cache_control to the last user message to cache conversation history
if (params.length > 0) {
const lastMessage = params[params.length - 1];
if (lastMessage.role === "user") {
// Add cache control to the last content block
if (Array.isArray(lastMessage.content)) {
const lastBlock = lastMessage.content[lastMessage.content.length - 1];
if (
lastBlock &&
(lastBlock.type === "text" || lastBlock.type === "image" || lastBlock.type === "tool_result")
) {
(lastBlock as any).cache_control = { type: "ephemeral" };
}
}
}
}
return params;
}