getcompanion-ai/co-mono
1
0
Fork 0
mirror of https://github.com/getcompanion-ai/co-mono.git synced 2026-04-22 00:00:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat(ai): add PI_CACHE_RETENTION env var for extended prompt caching

Browse source 
Adds support for extended cache retention via PI_CACHE_RETENTION=long:
- Anthropic: 5m -> 1h TTL
- OpenAI: in-memory -> 24h retention

Only applies to direct API calls (api.anthropic.com, api.openai.com).
Proxies and other providers are unaffected.

fixes #967
This commit is contained in:
Mario Zechner 2026-01-29 02:22:06 +01:00
parent 605f6f494b
commit 1b6a147579
6 changed files with 256 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

13
packages/ai/README.md
View file

@ -909,6 +909,19 @@ const response = await complete(model, context, {
});
```
#### Cache Retention
Set `PI_CACHE_RETENTION=long` to extend prompt cache retention:
| Provider | Default | With `PI_CACHE_RETENTION=long` |
|----------|---------|-------------------------------|
| Anthropic | 5 minutes | 1 hour |
| OpenAI | in-memory | 24 hours |
This only affects direct API calls to `api.anthropic.com` and `api.openai.com`. Proxies and other providers are unaffected.
> **Note**: Extended cache retention may increase costs for Anthropic (cache writes are charged at a higher rate). OpenAI's 24h retention has no additional cost.
### Checking Environment Variables
```typescript