Add OpenAICompat for openai-completions provider quirks

Fixes #133
This commit is contained in:
Mario Zechner 2025-12-08 19:00:57 +01:00
parent e34e0c503f
commit 87a1a9ded4
7 changed files with 165 additions and 24 deletions

View file

@ -611,6 +611,23 @@ const ollamaModel: Model<'openai-completions'> = {
maxTokens: 32000
};
// Example: LiteLLM proxy with explicit compat settings
const litellmModel: Model<'openai-completions'> = {
id: 'gpt-4o',
name: 'GPT-4o (via LiteLLM)',
api: 'openai-completions',
provider: 'litellm',
baseUrl: 'http://localhost:4000/v1',
reasoning: false,
input: ['text', 'image'],
cost: { input: 2.5, output: 10, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128000,
maxTokens: 16384,
compat: {
supportsStore: false, // LiteLLM doesn't support the store field
}
};
// Example: Custom endpoint with headers (bypassing Cloudflare bot detection)
const proxyModel: Model<'anthropic-messages'> = {
id: 'claude-sonnet-4',
@ -635,6 +652,25 @@ const response = await stream(ollamaModel, context, {
});
```
### OpenAI Compatibility Settings
The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for known providers (Cerebras, xAI, Mistral, Chutes, etc.). For custom proxies or unknown endpoints, you can override these settings via the `compat` field:
```typescript
interface OpenAICompat {
supportsStore?: boolean; // Whether provider supports the `store` field (default: true)
supportsDeveloperRole?: boolean; // Whether provider supports `developer` role vs `system` (default: true)
supportsReasoningEffort?: boolean; // Whether provider supports `reasoning_effort` (default: true)
maxTokensField?: 'max_completion_tokens' | 'max_tokens'; // Which field name to use (default: max_completion_tokens)
}
```
If `compat` is not set, the library falls back to URL-based detection. If `compat` is partially set, unspecified fields use the detected defaults. This is useful for:
- **LiteLLM proxies**: May not support `store` field
- **Custom inference servers**: May use non-standard field names
- **Self-hosted endpoints**: May have different feature support
### Type Safety
Models are typed by their API, ensuring type-safe options: