getcompanion-ai/co-mono
1
0
Fork 0
mirror of https://github.com/getcompanion-ai/co-mono.git synced 2026-04-22 06:00:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
co-mono/todos/work/2025-08-09-23-33-17-missing-thinking-tokens/task.md
Mario Zechner 62d9eefc2a agent: Add reasoning token support for OpenAI reasoning models 
- Extract and display reasoning tokens from both Chat Completions and Responses APIs
- Add smart preflight detection to check reasoning support per model/API (cached per agent)
- Support both reasoning_text (o1/o3) and summary_text (gpt-5) formats
- Display reasoning tokens with ⚡ symbol in console and TUI renderers
- Only send reasoning parameters to models that support them
- Fix event type from "thinking" to "reasoning" for consistency

Note: Chat Completions API only returns reasoning token counts, not content (by design).
Only Responses API exposes actual thinking/reasoning events.
2025-08-10 00:32:30 +02:00

3.1 KiB
Raw Blame History

Fix Missing Thinking Tokens for GPT-5 and Anthropic Models

Status: AwaitingCommit Agent PID: 27674

Original Todo

agent: we do not get thinking tokens for gpt-5. possibly also not for anthropic models?

Description

The agent doesn't extract or report reasoning/thinking tokens from OpenAI's reasoning models (gpt-5, o1, o3) when using the Chat Completions API. While the codebase has full thinking token support for the Responses API, the Chat Completions API implementation is missing the extraction of reasoning_tokens from the usage.completion_tokens_details object. This means users don't see how many tokens were used for reasoning, which can be significant (thousands of tokens) for these models.

Read analysis.md in full for detailed codebase research and context

Implementation Plan

  • Extend AgentEvent token_usage type to include reasoningTokens field (packages/agent/src/agent.ts:16-23)
  • Update Chat Completions API token extraction to include reasoning tokens from usage.completion_tokens_details (packages/agent/src/agent.ts:210-220)
  • Update console renderer to display reasoning tokens in usage metrics (packages/agent/src/renderers/console-renderer.ts:117-121)
  • Update TUI renderer to display reasoning tokens in usage metrics (packages/agent/src/renderers/tui-renderer.ts:219-227)
  • Update JSON renderer to include reasoning tokens in output (packages/agent/src/renderers/json-renderer.ts:20)
  • User test: Run agent with gpt-4o-mini model (or other reasoning model) and verify reasoning token count appears in metrics display
  • Debug: Fix missing reasoningTokens field in JSON output even when value is 0
  • Debug: Investigate why o3 model doesn't report reasoning tokens in responses API
  • Fix: Parse reasoning summaries from gpt-5 models (summary_text vs reasoning_text)
  • Fix: Only send reasoning parameter for models that support it (o3, gpt-5, etc)
  • Fix: Better detection of reasoning support - preflight test instead of hardcoded model names
  • Fix: Add reasoning support detection for Chat Completions API
  • Fix: Add correct summary parameter value and increase max_output_tokens for preflight check
  • Investigate: Chat Completions API has reasoning tokens but no thinking events

Notes

User reported that o3 model with responses API doesn't show reasoning tokens or thinking events. Fixed by:

  1. Adding reasoningTokens field to AgentEvent type
  2. Extracting reasoning tokens from both Chat Completions and Responses APIs
  3. Smart preflight detection of reasoning support for both APIs (cached per agent instance)
  4. Only sending reasoning parameter for supported models
  5. Parsing both reasoning_text (o1/o3) and summary_text (gpt-5) formats
  6. Displaying reasoning tokens in console and TUI renderers with ⚡ symbol
  7. Properly handling reasoning_effort for Chat Completions API

Important finding: Chat Completions API by design only returns reasoning token counts but not the actual thinking/reasoning content for o1 models. This is expected behavior - only the Responses API exposes thinking events.