mirror of
https://github.com/getcompanion-ai/co-mono.git
synced 2026-04-17 00:04:50 +00:00
Add Anthropic prompt caching, pluggable storage, and CORS proxy support
Storage Architecture:
- New pluggable storage system with backends (LocalStorage, ChromeStorage, IndexedDB)
- SettingsRepository for app settings (proxy config, etc.)
- ProviderKeysRepository for API key management
- AppStorage with global accessors (getAppStorage, setAppStorage, initAppStorage)
Transport Refactoring:
- Renamed DirectTransport → ProviderTransport (calls LLM providers with optional CORS proxy)
- Renamed ProxyTransport → AppTransport (uses app server with user auth)
- Updated TransportMode: "direct" → "provider", "proxy" → "app"
CORS Proxy Integration:
- ProviderTransport checks proxy.enabled/proxy.url from storage
- When enabled, modifies model baseUrl to route through proxy: {proxyUrl}/?url={originalBaseUrl}
- ProviderKeyInput test function also honors proxy settings
- Settings dialog with Proxy tab (Switch toggle, URL input, explanatory description)
Anthropic Prompt Caching:
- System prompt cached with cache_control markers (both OAuth and regular API keys)
- Last user message cached to cache conversation history
- Saves 90% on input tokens for cached content (10x cost reduction)
Settings Dialog Improvements:
- Configurable tab system with SettingsTab base class
- ApiKeysTab and ProxyTab as custom elements
- Switch toggle for proxy enable (instead of Checkbox)
- Explanatory paragraphs for each tab
- ApiKeyPromptDialog reuses ProviderKeyInput component
Removed:
- Deprecated ApiKeysDialog (replaced by ProviderKeyInput in SettingsDialog)
- Old storage-adapter and key-store (replaced by new storage architecture)
This commit is contained in:
parent
66f092c0c6
commit
0496651308
31 changed files with 1141 additions and 488 deletions
|
|
@ -291,7 +291,16 @@ function buildParams(
|
|||
});
|
||||
}
|
||||
} else if (context.systemPrompt) {
|
||||
params.system = context.systemPrompt;
|
||||
// Add cache control to system prompt for non-OAuth tokens
|
||||
params.system = [
|
||||
{
|
||||
type: "text",
|
||||
text: context.systemPrompt,
|
||||
cache_control: {
|
||||
type: "ephemeral",
|
||||
},
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
if (options?.temperature !== undefined) {
|
||||
|
|
@ -440,6 +449,24 @@ function convertMessages(messages: Message[], model: Model<"anthropic-messages">
|
|||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Add cache_control to the last user message to cache conversation history
|
||||
if (params.length > 0) {
|
||||
const lastMessage = params[params.length - 1];
|
||||
if (lastMessage.role === "user") {
|
||||
// Add cache control to the last content block
|
||||
if (Array.isArray(lastMessage.content)) {
|
||||
const lastBlock = lastMessage.content[lastMessage.content.length - 1];
|
||||
if (
|
||||
lastBlock &&
|
||||
(lastBlock.type === "text" || lastBlock.type === "image" || lastBlock.type === "tool_result")
|
||||
) {
|
||||
(lastBlock as any).cache_control = { type: "ephemeral" };
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return params;
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue