add agent schemas

This commit is contained in:
Nathan Flurry 2026-01-24 22:37:22 -08:00
commit c4153c5335
20 changed files with 2735 additions and 0 deletions

40
.gitignore vendored Normal file
View file

@ -0,0 +1,40 @@
# Dependencies
node_modules/
# Build outputs
dist/
build/
target/
# Package manager
pnpm-lock.yaml
package-lock.json
yarn.lock
# Cache
.cache/
*.tsbuildinfo
# Environment
.env
.env.local
.env.*
# IDE
.idea/
.vscode/
*.swp
*.swo
*~
# OS
.DS_Store
Thumbs.db
# Logs
*.log
npm-debug.log*
# Rust
Cargo.lock
**/*.rs.bk

View file

@ -0,0 +1,16 @@
[package]
name = "agent-schema"
version = "0.1.0"
edition = "2021"
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
regress = "0.10"
[build-dependencies]
typify = "0.4"
serde_json = "1.0"
schemars = "0.8"
prettyplease = "0.2"
syn = "2.0"

View file

@ -0,0 +1,54 @@
use std::fs;
use std::path::Path;
fn main() {
let out_dir = std::env::var("OUT_DIR").unwrap();
let schema_dir = Path::new("../../dist");
let schemas = [
("opencode", "opencode.json"),
("claude", "claude.json"),
("codex", "codex.json"),
("amp", "amp.json"),
];
for (name, file) in schemas {
let schema_path = schema_dir.join(file);
// Tell cargo to rerun if schema changes
println!("cargo:rerun-if-changed={}", schema_path.display());
if !schema_path.exists() {
eprintln!("Warning: Schema file not found: {}", schema_path.display());
// Write empty module
let out_path = Path::new(&out_dir).join(format!("{}.rs", name));
fs::write(&out_path, "// Schema not found\n").unwrap();
continue;
}
let schema_content = fs::read_to_string(&schema_path)
.unwrap_or_else(|e| panic!("Failed to read {}: {}", schema_path.display(), e));
let schema: schemars::schema::RootSchema = serde_json::from_str(&schema_content)
.unwrap_or_else(|e| panic!("Failed to parse {}: {}", schema_path.display(), e));
let mut type_space = typify::TypeSpace::default();
type_space
.add_root_schema(schema)
.unwrap_or_else(|e| panic!("Failed to process {}: {}", schema_path.display(), e));
let contents = type_space.to_stream();
// Format the generated code
let formatted = prettyplease::unparse(&syn::parse2(contents.clone()).unwrap_or_else(|e| {
panic!("Failed to parse generated code for {}: {}", name, e)
}));
let out_path = Path::new(&out_dir).join(format!("{}.rs", name));
fs::write(&out_path, formatted)
.unwrap_or_else(|e| panic!("Failed to write {}: {}", out_path.display(), e));
println!("cargo:warning=Generated {} types from {}", name, file);
}
}

View file

@ -0,0 +1,76 @@
//! Generated types from AI coding agent JSON schemas.
//!
//! This crate provides Rust types for:
//! - OpenCode SDK
//! - Claude Code SDK
//! - Codex SDK
//! - AMP Code SDK
pub mod opencode {
//! OpenCode SDK types extracted from OpenAPI 3.1.1 spec.
include!(concat!(env!("OUT_DIR"), "/opencode.rs"));
}
pub mod claude {
//! Claude Code SDK types extracted from TypeScript definitions.
include!(concat!(env!("OUT_DIR"), "/claude.rs"));
}
pub mod codex {
//! Codex SDK types.
include!(concat!(env!("OUT_DIR"), "/codex.rs"));
}
pub mod amp {
//! AMP Code SDK types.
include!(concat!(env!("OUT_DIR"), "/amp.rs"));
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_claude_bash_input() {
let input = claude::BashInput {
command: "ls -la".to_string(),
timeout: Some(5000.0),
description: Some("List files".to_string()),
run_in_background: None,
simulated_sed_edit: None,
dangerously_disable_sandbox: None,
};
let json = serde_json::to_string(&input).unwrap();
assert!(json.contains("ls -la"));
let parsed: claude::BashInput = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.command, "ls -la");
}
#[test]
fn test_codex_thread_event() {
let event = codex::ThreadEvent {
type_: codex::ThreadEventType::ThreadCreated,
thread_id: Some("thread-123".to_string()),
item: None,
error: serde_json::Map::new(),
};
let json = serde_json::to_string(&event).unwrap();
assert!(json.contains("thread.created"));
}
#[test]
fn test_amp_message() {
let msg = amp::Message {
role: amp::MessageRole::User,
content: "Hello".to_string(),
tool_calls: vec![],
};
let json = serde_json::to_string(&msg).unwrap();
assert!(json.contains("user"));
assert!(json.contains("Hello"));
}
}

View file

@ -0,0 +1,305 @@
# Type definitions for coding agent CLIs in SDK mode
Four major coding agent CLIs now offer programmatic access through TypeScript SDKs with well-defined type systems. **OpenCode provides the most complete formal specification** with a published OpenAPI 3.1.1 schema, while Claude Code and Codex offer comprehensive TypeScript types through npm packages. All four tools use similar patterns: streaming events via JSON lines, discriminated union types for messages, and structured configuration schemas.
## Codex CLI has TypeScript SDK but no formal schema
OpenAI's Codex CLI provides programmatic control through the **`@openai/codex-sdk`** package, which wraps the bundled binary and exchanges JSONL events over stdin/stdout. Types are well-defined in source code but not published as JSON Schema or OpenAPI specifications.
**Core SDK types from `@openai/codex-sdk`:**
```typescript
interface CodexOptions {
codexPathOverride?: string;
baseURL?: string;
apiKey?: string;
env?: Record<string, string>;
}
interface ThreadOptions {
model?: string;
sandboxMode?: "read-only" | "workspace-write" | "danger-full-access";
workingDirectory?: string;
skipGitRepoCheck?: boolean;
}
interface TurnOptions {
outputSchema?: Record<string, unknown>; // JSON Schema for structured output
}
type Input = string | UserInput[];
interface UserInput {
type: "text" | "local_image";
text?: string;
path?: string;
}
```
**Event types for streaming (`runStreamed()`):**
```typescript
type EventType = "thread.started" | "turn.started" | "turn.completed" |
"turn.failed" | "item.started" | "item.updated" |
"item.completed" | "error";
interface ThreadEvent {
type: EventType;
thread_id?: string;
usage?: { input_tokens: number; cached_input_tokens: number; output_tokens: number };
error?: { message: string };
item?: ThreadItem;
}
type ItemType = "agent_message" | "reasoning" | "command_execution" |
"file_change" | "mcp_tool_call" | "web_search" |
"todo_list" | "error" | "unknown";
```
**Key source files:** `sdk/typescript/src/` contains `items.ts`, `events.ts`, `options.ts`, `input.ts`, and `thread.ts`. Documentation at https://developers.openai.com/codex/sdk/ and https://developers.openai.com/codex/noninteractive/.
---
## Claude Code SDK offers the most comprehensive TypeScript types
Now rebranded as the **Claude Agent SDK**, the official package `@anthropic-ai/claude-agent-sdk` provides production-ready types with full coverage of tools, hooks, permissions, and streaming events.
**Core query function and options:**
```typescript
import { query } from "@anthropic-ai/claude-agent-sdk";
interface Options {
cwd?: string;
model?: string;
systemPrompt?: string | { type: 'preset'; preset: 'claude_code'; append?: string };
tools?: string[] | { type: 'preset'; preset: 'claude_code' };
allowedTools?: string[];
disallowedTools?: string[];
permissionMode?: 'default' | 'acceptEdits' | 'bypassPermissions' | 'plan';
maxTurns?: number;
maxBudgetUsd?: number;
outputFormat?: { type: 'json_schema', schema: JSONSchema };
mcpServers?: Record<string, McpServerConfig>;
hooks?: Partial<Record<HookEvent, HookCallbackMatcher[]>>;
sandbox?: SandboxSettings;
// ... 30+ additional options
}
```
**Message types (SDK output):**
```typescript
type SDKMessage = SDKAssistantMessage | SDKUserMessage | SDKResultMessage |
SDKSystemMessage | SDKPartialAssistantMessage | SDKCompactBoundaryMessage;
type SDKResultMessage = {
type: 'result';
subtype: 'success' | 'error_max_turns' | 'error_during_execution' |
'error_max_budget_usd' | 'error_max_structured_output_retries';
uuid: UUID;
session_id: string;
duration_ms: number;
duration_api_ms: number;
is_error: boolean;
num_turns: number;
result?: string;
total_cost_usd: number;
usage: { input_tokens: number; output_tokens: number; /* ... */ };
modelUsage: { [modelName: string]: ModelUsage };
structured_output?: unknown;
errors?: string[];
};
```
**Built-in tool input types:**
```typescript
type ToolInput = AgentInput | BashInput | FileEditInput | FileReadInput |
FileWriteInput | GlobInput | GrepInput | WebFetchInput |
WebSearchInput | /* ... 10+ more */;
interface BashInput {
command: string;
timeout?: number; // Max 600000ms
description?: string;
run_in_background?: boolean;
}
interface FileEditInput {
file_path: string; // Absolute path
old_string: string;
new_string: string;
replace_all?: boolean;
}
```
**Source:** https://github.com/anthropics/claude-agent-sdk-typescript and https://platform.claude.com/docs/en/agent-sdk/typescript
---
## OpenCode provides a full OpenAPI specification
The sst/opencode project is the **only CLI with a published OpenAPI 3.1.1 specification**, enabling automatic client generation. Types are auto-generated from the spec using `@hey-api/openapi-ts`.
**Key resources:**
- **OpenAPI spec:** `packages/sdk/openapi.json` at https://github.com/sst/opencode/blob/dev/packages/sdk/openapi.json
- **Generated types:** `packages/sdk/js/src/gen/types.gen.ts`
- **npm packages:** `@opencode-ai/sdk` and `@opencode-ai/plugin`
**Core types from the SDK:**
```typescript
interface Session {
id: string; // Pattern: ^ses.*
version: string;
projectID: string;
directory: string;
parentID?: string;
title: string;
summary?: { additions: number; deletions: number; files: number; diffs?: FileDiff[] };
time: { created: number; updated: number; compacting?: number; archived?: number };
}
type Message = UserMessage | AssistantMessage;
interface AssistantMessage {
id: string;
role: "assistant";
sessionID: string;
error?: NamedError;
tokens: { input: number; output: number; cache: { read: number; write: number } };
cost: { input: number; output: number };
modelID: string;
providerID: string;
}
type Part = TextPart | ReasoningPart | ToolPart | FilePart | AgentPart |
StepStartPart | StepFinishPart | SnapshotPart | PatchPart | RetryPart;
```
**SSE event types:**
```typescript
type Event =
| { type: "session.created"; properties: { session: Session } }
| { type: "session.updated"; properties: { session: Session } }
| { type: "message.updated"; properties: { message: Message } }
| { type: "message.part.updated"; properties: { part: Part } }
| { type: "permission.asked"; properties: { permission: Permission } }
| { type: "pty.created"; properties: { pty: Pty } }
// 30+ total event types
```
**Plugin system types (`@opencode-ai/plugin`):**
```typescript
type Plugin = (ctx: PluginInput) => Promise<Hooks>;
interface Hooks {
event?: (input: { event: Event }) => Promise<void>;
tool?: { [key: string]: ToolDefinition };
"tool.execute.before"?: (input, output) => Promise<void>;
"tool.execute.after"?: (input, output) => Promise<void>;
stop?: (input: { sessionID: string }) => Promise<{ continue?: boolean }>;
}
```
---
## Amp CLI documents stream JSON schema in manual
Sourcegraph's Amp provides `@sourcegraph/amp-sdk` with documented TypeScript types, though the package source is closed. The **stream JSON schema is fully documented** at https://ampcode.com/manual/appendix.
**Complete StreamJSONMessage type:**
```typescript
type StreamJSONMessage =
| {
type: "assistant";
message: {
type: "message";
role: "assistant";
content: Array<
| { type: "text"; text: string }
| { type: "tool_use"; id: string; name: string; input: Record<string, unknown> }
| { type: "thinking"; thinking: string }
| { type: "redacted_thinking"; data: string }
>;
stop_reason: "end_turn" | "tool_use" | "max_tokens" | null;
usage?: { input_tokens: number; output_tokens: number; /* ... */ };
};
parent_tool_use_id: string | null;
session_id: string;
}
| {
type: "user";
message: {
role: "user";
content: Array<{ type: "tool_result"; tool_use_id: string; content: string; is_error: boolean }>;
};
parent_tool_use_id: string | null;
session_id: string;
}
| {
type: "result";
subtype: "success";
duration_ms: number;
is_error: false;
num_turns: number;
result: string;
session_id: string;
}
| {
type: "system";
subtype: "init";
cwd: string;
session_id: string;
tools: string[];
mcp_servers: { name: string; status: "connected" | "connecting" | "connection-failed" | "disabled" }[];
};
```
**SDK execute function:**
```typescript
import { execute, type AmpOptions, type MCPConfig } from '@sourcegraph/amp-sdk';
interface AmpOptions {
cwd?: string;
dangerouslyAllowAll?: boolean;
toolbox?: string;
mcpConfig?: MCPConfig;
permissions?: PermissionRule[];
continue?: boolean | string;
}
interface PermissionRule {
tool: string; // Glob pattern like "Bash", "mcp__playwright__*"
matches?: { [argumentName: string]: string | string[] | boolean };
action: "allow" | "reject" | "ask" | "delegate";
context?: "thread" | "subagent";
}
```
Documentation at https://ampcode.com/manual/sdk and npm package at https://www.npmjs.com/package/@sourcegraph/amp-sdk.
---
## Comparing schema availability across all four CLIs
| CLI | TypeScript Types | OpenAPI/JSON Schema | Source Available |
|-----|------------------|---------------------|------------------|
| **Codex** | ✅ Full SDK types | ❌ None published | ✅ GitHub |
| **Claude Code** | ✅ Comprehensive | ❌ None published | ✅ GitHub |
| **OpenCode** | ✅ Auto-generated | ✅ **OpenAPI 3.1.1** | ✅ GitHub |
| **Amp** | ✅ Documented types | ❌ None published | ❌ Closed |
**Common patterns across all SDKs:**
All four tools share remarkably similar architectural patterns. They use discriminated union types for messages (user, assistant, system, result), JSONL streaming for real-time output, session/thread-based conversation management, MCP server integration with identical config shapes, and permission systems with allow/deny/ask actions.
**For maximum type safety**, OpenCode is the only option with a formal OpenAPI specification that enables automatic client generation in any language. Claude Code provides the most comprehensive TypeScript-native types with **35+ option fields** and full hook/plugin typing. Codex types are well-structured but require importing from source. Amp types are documented but not directly inspectable in source code.
## Conclusion
Developers building integrations should prioritize **OpenCode for formal schema needs** (OpenAPI enables code generation) and **Claude Code for TypeScript-first development** (most complete type exports). All four SDKs are actively maintained and converging on similar patterns, suggesting a de facto standard for coding agent interfaces is emerging around discriminated message unions, streaming events, and permission-based tool control.

171
research/claude.md Normal file
View file

@ -0,0 +1,171 @@
# Claude Code Research
Research notes on Claude Code's configuration, credential discovery, and runtime behavior based on agent-jj implementation.
## Overview
- **Provider**: Anthropic
- **Execution Method**: CLI subprocess (`claude` command)
- **Session Persistence**: Session ID (string)
- **SDK**: None (spawns CLI directly)
## Credential Discovery
### Priority Order
1. User-configured credentials (passed as `ANTHROPIC_API_KEY` env var)
2. Environment variables: `ANTHROPIC_API_KEY` or `CLAUDE_API_KEY`
3. Bootstrap extraction from config files
4. OAuth fallback (Claude CLI handles internally)
### Config File Locations
| Path | Description |
|------|-------------|
| `~/.claude.json.api` | API key config (highest priority) |
| `~/.claude.json` | General config |
| `~/.claude.json.nathan` | User-specific backup (custom) |
| `~/.claude/.credentials.json` | OAuth credentials |
| `~/.claude-oauth-credentials.json` | Docker mount alternative for OAuth |
### API Key Field Names (checked in order)
```json
{
"primaryApiKey": "sk-ant-...",
"apiKey": "sk-ant-...",
"anthropicApiKey": "sk-ant-...",
"customApiKey": "sk-ant-..."
}
```
Keys must start with `sk-ant-` prefix to be valid.
### OAuth Structure
```json
// ~/.claude/.credentials.json
{
"claudeAiOauth": {
"accessToken": "...",
"expiresAt": "2024-01-01T00:00:00Z"
}
}
```
OAuth tokens are validated for expiry before use.
## CLI Invocation
### Command Structure
```bash
claude \
--print \
--output-format stream-json \
--verbose \
--dangerously-skip-permissions \
[--resume SESSION_ID] \
[--model MODEL_ID] \
[--permission-mode plan] \
"PROMPT"
```
### Arguments
| Flag | Description |
|------|-------------|
| `--print` | Output mode |
| `--output-format stream-json` | Newline-delimited JSON streaming |
| `--verbose` | Verbose output |
| `--dangerously-skip-permissions` | Skip permission prompts |
| `--resume SESSION_ID` | Resume existing session |
| `--model MODEL_ID` | Specify model (e.g., `claude-sonnet-4-20250514`) |
| `--permission-mode plan` | Plan mode (read-only exploration) |
### Environment Variables
Only `ANTHROPIC_API_KEY` is passed if an API key is found. If no key is found, Claude CLI uses its built-in OAuth flow from `~/.claude/.credentials.json`.
## Streaming Response Format
Claude CLI outputs newline-delimited JSON events:
```json
{"type": "assistant", "message": {"content": [{"type": "text", "text": "..."}]}}
{"type": "tool_use", "tool_use": {"name": "Read", "input": {...}}}
{"type": "result", "result": "Final response text", "session_id": "abc123"}
```
### Event Types
| Type | Description |
|------|-------------|
| `assistant` | Assistant message with content blocks |
| `tool_use` | Tool invocation |
| `tool_result` | Tool result (may include `is_error`) |
| `result` | Final result with session ID |
### Content Block Types
```typescript
{
type: "text" | "tool_use";
text?: string;
name?: string; // tool name
input?: object; // tool input
}
```
## Response Schema
```typescript
// ClaudeCliResponseSchema
{
result?: string; // Final response text
session_id?: string; // Session ID for resumption
structured_output?: unknown; // Optional structured output
error?: unknown; // Error information
}
```
## Session Management
- Session ID is captured from streaming events (`event.session_id`)
- Use `--resume SESSION_ID` to continue a session
- Sessions are stored internally by Claude CLI
## Timeout
- Default timeout: 5 minutes (300,000 ms)
- Process is killed with `SIGTERM` on timeout
## Agent Modes
| Mode | Behavior |
|------|----------|
| `build` | Default execution mode |
| `plan` | Adds `--permission-mode plan` flag |
| `chat` | Available but no special handling |
## Error Handling
- Non-zero exit codes result in errors
- stderr is captured and included in error messages
- Spawn errors are caught separately
## Conversion to Universal Format
Claude output is converted via `convertClaudeOutput()`:
1. If response is a string, wrap as assistant message
2. If response is object with `result` field, extract content
3. Parse with `ClaudeCliResponseSchema` as fallback
4. Extract `structured_output` as metadata if present
## Notes
- Claude CLI manages its own OAuth refresh internally
- No SDK dependency - direct CLI subprocess
- stdin is closed immediately after spawn
- Working directory is set via `cwd` option on spawn

253
research/codex.md Normal file
View file

@ -0,0 +1,253 @@
# Codex Research
Research notes on OpenAI Codex's configuration, credential discovery, and runtime behavior based on agent-jj implementation.
## Overview
- **Provider**: OpenAI
- **Execution Method**: SDK (`@openai/codex-sdk`) or CLI binary
- **Session Persistence**: Thread ID (string)
- **Import**: Dynamic import to avoid bundling issues
- **Binary Location**: `~/.nvm/versions/node/v24.3.0/bin/codex` (npm global install)
## SDK Architecture
**The SDK wraps a bundled binary** - it does NOT make direct API calls.
- The TypeScript SDK includes a pre-compiled Codex binary
- When you use the SDK, it spawns this binary as a child process
- Communication happens via stdin/stdout using JSONL (JSON Lines) format
- The binary itself handles the actual communication with OpenAI's backend services
Sources: [Codex SDK docs](https://developers.openai.com/codex/sdk/), [GitHub](https://github.com/openai/codex)
## CLI Usage (Alternative to SDK)
You can use the `codex` binary directly instead of the SDK:
### Interactive Mode
```bash
codex "your prompt here"
codex --model o3 "your prompt"
```
### Non-Interactive Mode (`codex exec`)
```bash
codex exec "your prompt here"
codex exec --json "your prompt" # JSONL output
codex exec -m o3 "your prompt"
codex exec --dangerously-bypass-approvals-and-sandbox "prompt"
codex exec resume --last # Resume previous session
```
### Key CLI Flags
| Flag | Description |
|------|-------------|
| `--json` | Print events to stdout as JSONL |
| `-m, --model MODEL` | Model to use |
| `-s, --sandbox MODE` | `read-only`, `workspace-write`, `danger-full-access` |
| `--full-auto` | Auto-approve with workspace-write sandbox |
| `--dangerously-bypass-approvals-and-sandbox` | Skip all prompts (dangerous) |
| `-C, --cd DIR` | Working directory |
| `-o, --output-last-message FILE` | Write final response to file |
| `--output-schema FILE` | JSON Schema for structured output |
### Session Management
```bash
codex resume # Pick from previous sessions
codex resume --last # Resume most recent
codex fork --last # Fork most recent session
```
## Credential Discovery
### Priority Order
1. User-configured credentials (from `credentials` array)
2. Environment variable: `CODEX_API_KEY`
3. Environment variable: `OPENAI_API_KEY`
4. Bootstrap extraction from config files
### Config File Location
| Path | Description |
|------|-------------|
| `~/.codex/auth.json` | Primary auth config |
### Auth File Structure
```json
// API Key authentication
{
"OPENAI_API_KEY": "sk-..."
}
// OAuth authentication
{
"tokens": {
"access_token": "..."
}
}
```
## SDK Usage
### Client Initialization
```typescript
import { Codex } from "@openai/codex-sdk";
// With API key
const codex = new Codex({ apiKey: "sk-..." });
// Without API key (uses default auth)
const codex = new Codex();
```
Dynamic import is used to avoid bundling the SDK:
```typescript
const { Codex } = await import("@openai/codex-sdk");
```
### Thread Management
```typescript
// Start new thread
const thread = codex.startThread();
// Resume existing thread
const thread = codex.resumeThread(threadId);
```
### Running Prompts
```typescript
const { events } = await thread.runStreamed(prompt);
for await (const event of events) {
// Process events
}
```
## Event Types
| Event Type | Description |
|------------|-------------|
| `thread.started` | Thread initialized, contains `thread_id` |
| `item.completed` | Item finished, check for `agent_message` type |
| `turn.failed` | Turn failed with error message |
### Event Structure
```typescript
// thread.started
{
type: "thread.started",
thread_id: "thread_abc123"
}
// item.completed (agent message)
{
type: "item.completed",
item: {
type: "agent_message",
text: "Response text"
}
}
// turn.failed
{
type: "turn.failed",
error: {
message: "Error description"
}
}
```
## Response Schema
```typescript
// CodexRunResultSchema
type CodexRunResult = string | {
result?: string;
output?: string;
message?: string;
// ...additional fields via passthrough
};
```
Content is extracted in priority order: `result` > `output` > `message`
## Thread ID Retrieval
Thread ID can be obtained from multiple sources:
1. `thread.started` event's `thread_id` property
2. Thread object's `id` getter (after first turn)
3. Thread object's `threadId` or `_id` properties (fallbacks)
```typescript
function getThreadId(thread: unknown): string | null {
const value = thread as { id?: string; threadId?: string; _id?: string };
return value.id ?? value.threadId ?? value._id ?? null;
}
```
## Agent Modes
Modes are implemented via prompt prefixing:
| Mode | Prompt Prefix |
|------|---------------|
| `build` | No prefix (default) |
| `plan` | `"Make a plan before acting.\n\n"` |
| `chat` | `"Answer conversationally.\n\n"` |
```typescript
function withModePrefix(prompt: string, mode: AgentMode): string {
if (mode === "plan") {
return `Make a plan before acting.\n\n${prompt}`;
}
if (mode === "chat") {
return `Answer conversationally.\n\n${prompt}`;
}
return prompt;
}
```
## Error Handling
- `turn.failed` events are captured but don't throw
- Thread ID is still returned on error for potential resumption
- Events iterator may throw after errors - caught and logged
```typescript
interface CodexPromptResult {
result: unknown;
threadId?: string | null;
error?: string; // Set if turn failed
}
```
## Conversion to Universal Format
Codex output is converted via `convertCodexOutput()`:
1. Parse with `CodexRunResultSchema`
2. If result is string, use directly
3. Otherwise extract from `result`, `output`, or `message` fields
4. Wrap as assistant message entry
## Session Continuity
- Thread ID persists across prompts
- Use `resumeThread(threadId)` to continue conversation
- Thread ID is captured from `thread.started` event or thread object
## Notes
- SDK is dynamically imported to reduce bundle size
- No explicit timeout (relies on SDK defaults)
- Thread ID may not be available until first event
- Error messages are preserved for debugging
- Working directory is not explicitly set (SDK handles internally)

475
research/opencode.md Normal file
View file

@ -0,0 +1,475 @@
# OpenCode Research
Research notes on OpenCode's configuration, credential discovery, and runtime behavior based on agent-jj implementation.
## Overview
- **Provider**: Multi-provider (OpenAI, Anthropic, others)
- **Execution Method**: Embedded server via SDK, or CLI binary
- **Session Persistence**: Session ID (string)
- **SDK**: `@opencode-ai/sdk` (server + client)
- **Binary Location**: `~/.opencode/bin/opencode`
- **Written in**: Go (with Bubble Tea TUI)
## CLI Usage (Alternative to SDK)
OpenCode can be used as a standalone binary instead of embedding the SDK:
### Interactive TUI Mode
```bash
opencode # Start TUI in current directory
opencode /path/to/project # Start in specific directory
opencode -c # Continue last session
opencode -s SESSION_ID # Continue specific session
```
### Non-Interactive Mode (`opencode run`)
```bash
opencode run "your prompt here"
opencode run --format json "prompt" # Raw JSON events output
opencode run -m anthropic/claude-sonnet-4-20250514 "prompt"
opencode run --agent plan "analyze this code"
opencode run -c "follow up question" # Continue last session
opencode run -s SESSION_ID "prompt" # Continue specific session
opencode run -f file1.ts -f file2.ts "review these files"
```
### Key CLI Flags
| Flag | Description |
|------|-------------|
| `--format json` | Output raw JSON events (for parsing) |
| `-m, --model PROVIDER/MODEL` | Model in format `provider/model` |
| `--agent AGENT` | Agent to use (`build`, `plan`) |
| `-c, --continue` | Continue last session |
| `-s, --session ID` | Continue specific session |
| `-f, --file FILE` | Attach file(s) to message |
| `--attach URL` | Attach to running server |
| `--port PORT` | Local server port |
| `--variant VARIANT` | Reasoning effort (e.g., `high`, `max`) |
### Headless Server Mode
```bash
opencode serve # Start headless server
opencode serve --port 4096 # Specific port
opencode attach http://localhost:4096 # Attach to running server
```
### Other Commands
```bash
opencode models # List available models
opencode models anthropic # List models for provider
opencode auth # Manage credentials
opencode session # Manage sessions
opencode export SESSION_ID # Export session as JSON
opencode stats # Token usage statistics
```
Sources: [OpenCode GitHub](https://github.com/opencode-ai/opencode), [OpenCode Docs](https://opencode.ai/docs/cli/)
## Architecture
OpenCode runs as an embedded HTTP server per workspace/change:
```
┌─────────────────────┐
│ agent-jj backend │
│ │
│ ┌───────────────┐ │
│ │ OpenCode │ │
│ │ Server │◄─┼── HTTP API
│ │ (per change) │ │
│ └───────────────┘ │
└─────────────────────┘
```
- One server per `changeId` (workspace+repo+change combination)
- Multiple sessions can share a server
- Server runs on dynamic port (4200-4300 range)
## Credential Discovery
### Priority Order
1. Environment variables: `ANTHROPIC_API_KEY`, `CLAUDE_API_KEY`
2. Environment variables: `OPENAI_API_KEY`, `CODEX_API_KEY`
3. Claude Code config files
4. Codex config files
5. OpenCode config files
### Config File Location
| Path | Description |
|------|-------------|
| `~/.local/share/opencode/auth.json` | Primary auth config |
### Auth File Structure
```json
{
"anthropic": {
"type": "api",
"key": "sk-ant-..."
},
"openai": {
"type": "api",
"key": "sk-..."
},
"custom-provider": {
"type": "oauth",
"access": "token...",
"refresh": "refresh-token...",
"expires": 1704067200000
}
}
```
### Provider Config Types
```typescript
interface OpenCodeProviderConfig {
type: "api" | "oauth";
key?: string; // For API type
access?: string; // For OAuth type
refresh?: string; // For OAuth type
expires?: number; // Unix timestamp (ms)
}
```
OAuth tokens are validated for expiry before use.
## Server Management
### Starting a Server
```typescript
import { createOpencodeServer } from "@opencode-ai/sdk/server";
import { createOpencodeClient } from "@opencode-ai/sdk";
const server = await createOpencodeServer({
hostname: "127.0.0.1",
port: 4200,
config: { logLevel: "DEBUG" }
});
const client = createOpencodeClient({
baseUrl: `http://127.0.0.1:${port}`
});
```
### Server Configuration
```typescript
// From config.json
{
"opencode": {
"host": "127.0.0.1", // Bind address
"advertisedHost": "127.0.0.1" // External address (for tunnels)
}
}
```
### Port Selection
Uses `get-port` package to find available port in range 4200-4300.
## Client API
### Session Management
```typescript
// Create session
const response = await client.session.create({});
const sessionId = response.data.id;
// Get session info
const session = await client.session.get({ path: { id: sessionId } });
// Get session messages
const messages = await client.session.messages({ path: { id: sessionId } });
// Get session todos
const todos = await client.session.todo({ path: { id: sessionId } });
```
### Sending Prompts
#### Synchronous
```typescript
const response = await client.session.prompt({
path: { id: sessionId },
body: {
model: { providerID: "openai", modelID: "gpt-4o" },
agent: "build",
parts: [{ type: "text", text: "prompt text" }]
}
});
```
#### Asynchronous (Streaming)
```typescript
// Start prompt asynchronously
await client.session.promptAsync({
path: { id: sessionId },
body: {
model: { providerID: "openai", modelID: "gpt-4o" },
agent: "build",
parts: [{ type: "text", text: "prompt text" }]
}
});
// Subscribe to events
const eventStream = await client.event.subscribe({});
for await (const event of eventStream.stream) {
// Process events
}
```
## Event Types
| Event Type | Description |
|------------|-------------|
| `message.part.updated` | Message part streamed/updated |
| `session.status` | Session status changed |
| `session.idle` | Session finished processing |
| `session.error` | Session error occurred |
| `question.asked` | AI asking user question |
| `permission.asked` | AI requesting permission |
### Event Structure
```typescript
interface SDKEvent {
type: string;
properties: {
part?: SDKPart & { sessionID?: string };
delta?: string; // Text delta for streaming
status?: { type?: string };
sessionID?: string;
error?: { data?: { message?: string } };
id?: string;
questions?: QuestionInfo[];
permission?: string;
patterns?: string[];
metadata?: Record<string, unknown>;
always?: string[];
tool?: { messageID?: string; callID?: string };
};
}
```
## Message Parts
OpenCode has rich message part types:
| Type | Description |
|------|-------------|
| `text` | Plain text content |
| `reasoning` | Model reasoning (chain-of-thought) |
| `tool` | Tool invocation with status |
| `file` | File reference |
| `step-start` | Step boundary start |
| `step-finish` | Step boundary end with reason |
| `subtask` | Delegated subtask |
### Part Structure
```typescript
interface MessagePart {
type: "text" | "reasoning" | "tool" | "file" | "step-start" | "step-finish" | "subtask" | "other";
id: string;
content: string;
// Tool-specific
toolName?: string;
toolStatus?: "pending" | "running" | "completed" | "error";
toolInput?: Record<string, unknown>;
toolOutput?: string;
// File-specific
filename?: string;
mimeType?: string;
// Step-specific
stepReason?: string;
// Subtask-specific
subtaskAgent?: string;
subtaskDescription?: string;
}
```
## Questions and Permissions
### Question Request
```typescript
interface QuestionRequest {
id: string;
sessionID: string;
questions: Array<{
header?: string;
question: string;
options: Array<{ label: string; description?: string }>;
multiSelect?: boolean;
}>;
tool?: { messageID: string; callID: string };
}
```
### Responding to Questions
```typescript
// V2 client for question/permission APIs
const clientV2 = createOpencodeClientV2({
baseUrl: `http://127.0.0.1:${port}`
});
// Reply with answers
await clientV2.question.reply({
requestID: requestId,
answers: [["selected option"]] // Array of selected labels per question
});
// Reject question
await clientV2.question.reject({ requestID: requestId });
```
### Permission Request
```typescript
interface PermissionRequest {
id: string;
sessionID: string;
permission: string; // Permission type (e.g., "file:write")
patterns: string[]; // Affected paths/patterns
metadata: Record<string, unknown>;
always: string[]; // Options for "always allow"
tool?: { messageID: string; callID: string };
}
```
### Responding to Permissions
```typescript
await clientV2.permission.reply({
requestID: requestId,
reply: "once" | "always" | "reject"
});
```
## Provider/Model Discovery
```typescript
// Get available providers and models
const providerResponse = await client.provider.list({});
const agentResponse = await client.app.agents({});
interface ProviderInfo {
id: string;
name: string;
models: Array<{
id: string;
name: string;
reasoning: boolean;
toolCall: boolean;
}>;
}
interface AgentInfo {
id: string;
name: string;
primary: boolean; // "build" and "plan" are primary
}
```
### Internal Agents (Hidden from UI)
- `compaction`
- `title`
- `summary`
## Token Usage
```typescript
interface TokenUsage {
input: number;
output: number;
reasoning?: number;
cache?: {
read: number;
write: number;
};
}
```
Available in message `info` field for assistant messages.
## Agent Modes
| Mode | Agent ID |
|------|----------|
| `build` | `"build"` |
| `plan` | `"plan"` |
Modes map directly to OpenCode agent IDs.
## Defaults
```typescript
const DEFAULT_OPENCODE_MODEL_ID = "gpt-4o";
const DEFAULT_OPENCODE_PROVIDER_ID = "openai";
```
## Concurrency Control
Server startup uses a lock to prevent race conditions:
```typescript
async function withStartLock<T>(fn: () => Promise<T>): Promise<T> {
const prior = startLock;
let release: () => void;
startLock = new Promise((resolve) => { release = resolve; });
await prior;
try {
return await fn();
} finally {
release();
}
}
```
## Working Directory
Server must be started in the correct working directory:
```typescript
async function withWorkingDir<T>(workingDir: string, fn: () => Promise<T>): Promise<T> {
const previous = process.cwd();
process.chdir(workingDir);
try {
return await fn();
} finally {
process.chdir(previous);
}
}
```
## Polling Fallback
A polling mechanism checks session status every 2 seconds in case SSE events don't arrive:
```typescript
const pollInterval = setInterval(async () => {
const session = await client.session.get({ path: { id: sessionId } });
if (session.data?.status?.type === "idle") {
abortController.abort();
}
}, 2000);
```
## Notes
- OpenCode is the most feature-rich runtime (streaming, questions, permissions)
- Server persists for the lifetime of a change (workspace+repo+change)
- Parts are streamed incrementally with delta updates
- V2 client is needed for question/permission APIs
- Working directory affects credential discovery and file operations

3
resources/agent-schemas/.gitignore vendored Normal file
View file

@ -0,0 +1,3 @@
node_modules/
.cache/
pnpm-lock.yaml

View file

@ -0,0 +1,3 @@
[workspace]
resolver = "2"
members = ["engine/packages/*"]

View file

@ -0,0 +1,23 @@
{
"name": "agent-schemas",
"version": "1.0.0",
"type": "module",
"scripts": {
"extract": "tsx src/index.ts",
"extract:opencode": "tsx src/index.ts --agent=opencode",
"extract:claude": "tsx src/index.ts --agent=claude",
"extract:codex": "tsx src/index.ts --agent=codex",
"extract:amp": "tsx src/index.ts --agent=amp"
},
"dependencies": {
"ts-json-schema-generator": "^2.4.0",
"cheerio": "^1.0.0",
"typescript": "^5.7.0",
"@anthropic-ai/claude-code": "latest",
"@openai/codex": "latest"
},
"devDependencies": {
"tsx": "^4.19.0",
"@types/node": "^22.0.0"
}
}

View file

@ -0,0 +1,271 @@
import * as cheerio from "cheerio";
import { fetchWithCache } from "./cache.js";
import { createNormalizedSchema, type NormalizedSchema } from "./normalize.js";
import type { JSONSchema7 } from "json-schema";
const AMP_DOCS_URL = "https://ampcode.com/manual/appendix";
// Key types we want to extract
const TARGET_TYPES = ["StreamJSONMessage", "AmpOptions", "PermissionRule", "Message", "ToolCall"];
export async function extractAmpSchema(): Promise<NormalizedSchema> {
console.log("Extracting AMP schema from documentation...");
try {
const html = await fetchWithCache(AMP_DOCS_URL);
const $ = cheerio.load(html);
// Find TypeScript code blocks
const codeBlocks: string[] = [];
$("pre code").each((_, el) => {
const code = $(el).text();
// Look for TypeScript interface/type definitions
if (
code.includes("interface ") ||
code.includes("type ") ||
code.includes(": {") ||
code.includes("export ")
) {
codeBlocks.push(code);
}
});
if (codeBlocks.length === 0) {
console.log(" [warn] No TypeScript code blocks found, using fallback schema");
return createFallbackSchema();
}
console.log(` [found] ${codeBlocks.length} code blocks`);
// Parse TypeScript definitions into schemas
const definitions = parseTypeScriptToSchema(codeBlocks.join("\n"));
// Verify target types exist
const found = TARGET_TYPES.filter((name) => definitions[name]);
const missing = TARGET_TYPES.filter((name) => !definitions[name]);
if (missing.length > 0) {
console.log(` [warn] Missing expected types: ${missing.join(", ")}`);
}
if (Object.keys(definitions).length === 0) {
console.log(" [warn] No types extracted, using fallback schema");
return createFallbackSchema();
}
console.log(` [ok] Extracted ${Object.keys(definitions).length} types (${found.length} target types)`);
return createNormalizedSchema("amp", "AMP Code SDK Schema", definitions);
} catch (error) {
console.log(` [error] Failed to fetch docs: ${error}`);
console.log(" [fallback] Using embedded schema definitions");
return createFallbackSchema();
}
}
function parseTypeScriptToSchema(code: string): Record<string, JSONSchema7> {
const definitions: Record<string, JSONSchema7> = {};
// Match interface definitions
const interfaceRegex = /(?:export\s+)?interface\s+(\w+)\s*(?:extends\s+[\w,\s]+)?\s*\{([^}]+)\}/g;
let match;
while ((match = interfaceRegex.exec(code)) !== null) {
const [, name, body] = match;
definitions[name] = parseInterfaceBody(body);
}
// Match type definitions (simple object types)
const typeRegex = /(?:export\s+)?type\s+(\w+)\s*=\s*\{([^}]+)\}/g;
while ((match = typeRegex.exec(code)) !== null) {
const [, name, body] = match;
definitions[name] = parseInterfaceBody(body);
}
// Match union type definitions
const unionRegex = /(?:export\s+)?type\s+(\w+)\s*=\s*([^;{]+);/g;
while ((match = unionRegex.exec(code)) !== null) {
const [, name, body] = match;
if (body.includes("|")) {
definitions[name] = parseUnionType(body);
}
}
return definitions;
}
function parseInterfaceBody(body: string): JSONSchema7 {
const properties: Record<string, JSONSchema7> = {};
const required: string[] = [];
// Match property definitions
const propRegex = /(\w+)(\?)?:\s*([^;]+);/g;
let match;
while ((match = propRegex.exec(body)) !== null) {
const [, propName, optional, propType] = match;
properties[propName] = typeToSchema(propType.trim());
if (!optional) {
required.push(propName);
}
}
const schema: JSONSchema7 = {
type: "object",
properties,
};
if (required.length > 0) {
schema.required = required;
}
return schema;
}
function typeToSchema(tsType: string): JSONSchema7 {
// Handle union types
if (tsType.includes("|")) {
return parseUnionType(tsType);
}
// Handle array types
if (tsType.endsWith("[]")) {
const itemType = tsType.slice(0, -2);
return {
type: "array",
items: typeToSchema(itemType),
};
}
// Handle Array<T>
const arrayMatch = tsType.match(/^Array<(.+)>$/);
if (arrayMatch) {
return {
type: "array",
items: typeToSchema(arrayMatch[1]),
};
}
// Handle basic types
switch (tsType) {
case "string":
return { type: "string" };
case "number":
return { type: "number" };
case "boolean":
return { type: "boolean" };
case "null":
return { type: "null" };
case "any":
case "unknown":
return {};
case "object":
return { type: "object" };
default:
// Could be a reference to another type
if (/^[A-Z]/.test(tsType)) {
return { $ref: `#/definitions/${tsType}` };
}
// String literal
if (tsType.startsWith('"') || tsType.startsWith("'")) {
return { type: "string", const: tsType.slice(1, -1) };
}
return { type: "string" };
}
}
function parseUnionType(unionStr: string): JSONSchema7 {
const parts = unionStr.split("|").map((p) => p.trim());
// Check if it's a string literal union
const allStringLiterals = parts.every((p) => p.startsWith('"') || p.startsWith("'"));
if (allStringLiterals) {
return {
type: "string",
enum: parts.map((p) => p.slice(1, -1)),
};
}
// General union
return {
oneOf: parts.map((p) => typeToSchema(p)),
};
}
function createFallbackSchema(): NormalizedSchema {
// Fallback schema based on AMP documentation structure
const definitions: Record<string, JSONSchema7> = {
StreamJSONMessage: {
type: "object",
properties: {
type: {
type: "string",
enum: ["message", "tool_call", "tool_result", "error", "done"],
},
id: { type: "string" },
content: { type: "string" },
tool_call: { $ref: "#/definitions/ToolCall" },
error: { type: "string" },
},
required: ["type"],
},
AmpOptions: {
type: "object",
properties: {
model: { type: "string" },
apiKey: { type: "string" },
baseURL: { type: "string" },
maxTokens: { type: "number" },
temperature: { type: "number" },
systemPrompt: { type: "string" },
tools: { type: "array", items: { type: "object" } },
workingDirectory: { type: "string" },
permissionRules: {
type: "array",
items: { $ref: "#/definitions/PermissionRule" },
},
},
},
PermissionRule: {
type: "object",
properties: {
tool: { type: "string" },
action: { type: "string", enum: ["allow", "deny", "ask"] },
pattern: { type: "string" },
description: { type: "string" },
},
required: ["tool", "action"],
},
Message: {
type: "object",
properties: {
role: { type: "string", enum: ["user", "assistant", "system"] },
content: { type: "string" },
tool_calls: {
type: "array",
items: { $ref: "#/definitions/ToolCall" },
},
},
required: ["role", "content"],
},
ToolCall: {
type: "object",
properties: {
id: { type: "string" },
name: { type: "string" },
arguments: {
oneOf: [{ type: "string" }, { type: "object" }],
},
},
required: ["id", "name", "arguments"],
},
};
console.log(` [ok] Using fallback schema with ${Object.keys(definitions).length} definitions`);
return createNormalizedSchema("amp", "AMP Code SDK Schema", definitions);
}

View file

@ -0,0 +1,94 @@
import { createHash } from "crypto";
import { existsSync, mkdirSync, readFileSync, writeFileSync, statSync } from "fs";
import { join } from "path";
const CACHE_DIR = join(import.meta.dirname, "..", ".cache");
const DEFAULT_TTL_MS = 24 * 60 * 60 * 1000; // 24 hours
interface CacheEntry<T> {
data: T;
timestamp: number;
ttl: number;
}
function ensureCacheDir(): void {
if (!existsSync(CACHE_DIR)) {
mkdirSync(CACHE_DIR, { recursive: true });
}
}
function hashKey(key: string): string {
return createHash("sha256").update(key).digest("hex");
}
function getCachePath(key: string): string {
return join(CACHE_DIR, `${hashKey(key)}.json`);
}
export function getCached<T>(key: string): T | null {
const path = getCachePath(key);
if (!existsSync(path)) {
return null;
}
try {
const content = readFileSync(path, "utf-8");
const entry: CacheEntry<T> = JSON.parse(content);
const now = Date.now();
if (now - entry.timestamp > entry.ttl) {
// Cache expired
return null;
}
return entry.data;
} catch {
return null;
}
}
export function setCache<T>(key: string, data: T, ttl: number = DEFAULT_TTL_MS): void {
ensureCacheDir();
const entry: CacheEntry<T> = {
data,
timestamp: Date.now(),
ttl,
};
const path = getCachePath(key);
writeFileSync(path, JSON.stringify(entry, null, 2));
}
export async function fetchWithCache(url: string, ttl?: number): Promise<string> {
const cached = getCached<string>(url);
if (cached !== null) {
console.log(` [cache hit] ${url}`);
return cached;
}
console.log(` [fetching] ${url}`);
let lastError: Error | null = null;
for (let attempt = 0; attempt < 3; attempt++) {
try {
const response = await fetch(url);
if (!response.ok) {
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
}
const text = await response.text();
setCache(url, text, ttl);
return text;
} catch (error) {
lastError = error as Error;
if (attempt < 2) {
const delay = Math.pow(2, attempt) * 1000;
console.log(` [retry ${attempt + 1}] waiting ${delay}ms...`);
await new Promise((resolve) => setTimeout(resolve, delay));
}
}
}
throw lastError;
}

View file

@ -0,0 +1,186 @@
import { createGenerator, type Config } from "ts-json-schema-generator";
import { existsSync, readFileSync } from "fs";
import { join, dirname } from "path";
import { createNormalizedSchema, type NormalizedSchema } from "./normalize.js";
import type { JSONSchema7 } from "json-schema";
// Try multiple possible paths for the SDK types
const POSSIBLE_PATHS = [
"node_modules/@anthropic-ai/claude-code/sdk-tools.d.ts",
"node_modules/@anthropic-ai/claude-code/dist/index.d.ts",
"node_modules/@anthropic-ai/claude-code/dist/types.d.ts",
"node_modules/@anthropic-ai/claude-code/index.d.ts",
];
// Key types we want to extract
const TARGET_TYPES = [
"ToolInputSchemas",
"AgentInput",
"BashInput",
"FileEditInput",
"FileReadInput",
"FileWriteInput",
"GlobInput",
"GrepInput",
"WebFetchInput",
"WebSearchInput",
"AskUserQuestionInput",
];
function findTypesPath(): string | null {
const baseDir = join(import.meta.dirname, "..");
for (const relativePath of POSSIBLE_PATHS) {
const fullPath = join(baseDir, relativePath);
if (existsSync(fullPath)) {
return fullPath;
}
}
return null;
}
export async function extractClaudeSchema(): Promise<NormalizedSchema> {
console.log("Extracting Claude Code SDK schema...");
const typesPath = findTypesPath();
if (!typesPath) {
console.log(" [warn] Claude Code SDK types not found, using fallback schema");
return createFallbackSchema();
}
console.log(` [found] ${typesPath}`);
const config: Config = {
path: typesPath,
tsconfig: join(import.meta.dirname, "..", "tsconfig.json"),
type: "*",
skipTypeCheck: true,
topRef: false,
expose: "export",
jsDoc: "extended",
};
try {
const generator = createGenerator(config);
const schema = generator.createSchema(config.type);
const definitions: Record<string, JSONSchema7> = {};
if (schema.definitions) {
for (const [name, def] of Object.entries(schema.definitions)) {
definitions[name] = def as JSONSchema7;
}
}
// Verify target types exist
const found = TARGET_TYPES.filter((name) => definitions[name]);
const missing = TARGET_TYPES.filter((name) => !definitions[name]);
if (missing.length > 0) {
console.log(` [warn] Missing expected types: ${missing.join(", ")}`);
}
console.log(` [ok] Extracted ${Object.keys(definitions).length} types (${found.length} target types)`);
return createNormalizedSchema("claude", "Claude Code SDK Schema", definitions);
} catch (error) {
console.log(` [error] Schema generation failed: ${error}`);
console.log(" [fallback] Using embedded schema definitions");
return createFallbackSchema();
}
}
function createFallbackSchema(): NormalizedSchema {
// Fallback schema based on known SDK structure
const definitions: Record<string, JSONSchema7> = {
SDKMessage: {
type: "object",
properties: {
type: { type: "string", enum: ["user", "assistant", "result"] },
content: { type: "string" },
timestamp: { type: "string", format: "date-time" },
},
required: ["type"],
},
SDKResultMessage: {
type: "object",
properties: {
type: { type: "string", const: "result" },
result: { type: "object" },
error: { type: "string" },
duration_ms: { type: "number" },
},
required: ["type"],
},
Options: {
type: "object",
properties: {
model: { type: "string" },
maxTokens: { type: "number" },
temperature: { type: "number" },
systemPrompt: { type: "string" },
tools: { type: "array", items: { type: "string" } },
allowedTools: { type: "array", items: { type: "string" } },
workingDirectory: { type: "string" },
},
},
BashInput: {
type: "object",
properties: {
command: { type: "string" },
timeout: { type: "number" },
workingDirectory: { type: "string" },
},
required: ["command"],
},
FileEditInput: {
type: "object",
properties: {
path: { type: "string" },
oldText: { type: "string" },
newText: { type: "string" },
},
required: ["path", "oldText", "newText"],
},
FileReadInput: {
type: "object",
properties: {
path: { type: "string" },
startLine: { type: "number" },
endLine: { type: "number" },
},
required: ["path"],
},
FileWriteInput: {
type: "object",
properties: {
path: { type: "string" },
content: { type: "string" },
},
required: ["path", "content"],
},
GlobInput: {
type: "object",
properties: {
pattern: { type: "string" },
path: { type: "string" },
},
required: ["pattern"],
},
GrepInput: {
type: "object",
properties: {
pattern: { type: "string" },
path: { type: "string" },
include: { type: "string" },
},
required: ["pattern"],
},
};
console.log(` [ok] Using fallback schema with ${Object.keys(definitions).length} definitions`);
return createNormalizedSchema("claude", "Claude Code SDK Schema", definitions);
}

View file

@ -0,0 +1,180 @@
import { createGenerator, type Config } from "ts-json-schema-generator";
import { existsSync } from "fs";
import { join } from "path";
import { createNormalizedSchema, type NormalizedSchema } from "./normalize.js";
import type { JSONSchema7 } from "json-schema";
// Try multiple possible paths for the SDK types
const POSSIBLE_PATHS = [
"node_modules/@openai/codex/dist/index.d.ts",
"node_modules/@openai/codex/dist/types.d.ts",
"node_modules/@openai/codex/index.d.ts",
];
// Key types we want to extract
const TARGET_TYPES = [
"ThreadEvent",
"ThreadItem",
"CodexOptions",
"ThreadOptions",
"Input",
"ResponseItem",
"FunctionCall",
"Message",
];
function findTypesPath(): string | null {
const baseDir = join(import.meta.dirname, "..");
for (const relativePath of POSSIBLE_PATHS) {
const fullPath = join(baseDir, relativePath);
if (existsSync(fullPath)) {
return fullPath;
}
}
return null;
}
export async function extractCodexSchema(): Promise<NormalizedSchema> {
console.log("Extracting Codex SDK schema...");
const typesPath = findTypesPath();
if (!typesPath) {
console.log(" [warn] Codex SDK types not found, using fallback schema");
return createFallbackSchema();
}
console.log(` [found] ${typesPath}`);
const config: Config = {
path: typesPath,
tsconfig: join(import.meta.dirname, "..", "tsconfig.json"),
type: "*",
skipTypeCheck: true,
topRef: false,
expose: "export",
jsDoc: "extended",
};
try {
const generator = createGenerator(config);
const schema = generator.createSchema(config.type);
const definitions: Record<string, JSONSchema7> = {};
if (schema.definitions) {
for (const [name, def] of Object.entries(schema.definitions)) {
definitions[name] = def as JSONSchema7;
}
}
// Verify target types exist
const found = TARGET_TYPES.filter((name) => definitions[name]);
const missing = TARGET_TYPES.filter((name) => !definitions[name]);
if (missing.length > 0) {
console.log(` [warn] Missing expected types: ${missing.join(", ")}`);
}
console.log(` [ok] Extracted ${Object.keys(definitions).length} types (${found.length} target types)`);
return createNormalizedSchema("codex", "Codex SDK Schema", definitions);
} catch (error) {
console.log(` [error] Schema generation failed: ${error}`);
console.log(" [fallback] Using embedded schema definitions");
return createFallbackSchema();
}
}
function createFallbackSchema(): NormalizedSchema {
// Fallback schema based on known SDK structure
const definitions: Record<string, JSONSchema7> = {
ThreadEvent: {
type: "object",
properties: {
type: {
type: "string",
enum: ["thread.created", "thread.updated", "item.created", "item.updated", "error"],
},
thread_id: { type: "string" },
item: { $ref: "#/definitions/ThreadItem" },
error: { type: "object" },
},
required: ["type"],
},
ThreadItem: {
type: "object",
properties: {
id: { type: "string" },
type: { type: "string", enum: ["message", "function_call", "function_result"] },
role: { type: "string", enum: ["user", "assistant", "system"] },
content: {
oneOf: [{ type: "string" }, { type: "array", items: { type: "object" } }],
},
status: { type: "string", enum: ["pending", "in_progress", "completed", "failed"] },
},
required: ["id", "type"],
},
CodexOptions: {
type: "object",
properties: {
apiKey: { type: "string" },
model: { type: "string" },
baseURL: { type: "string" },
maxTokens: { type: "number" },
temperature: { type: "number" },
},
},
ThreadOptions: {
type: "object",
properties: {
instructions: { type: "string" },
tools: { type: "array", items: { type: "object" } },
model: { type: "string" },
workingDirectory: { type: "string" },
},
},
Input: {
type: "object",
properties: {
type: { type: "string", enum: ["text", "file", "image"] },
content: { type: "string" },
path: { type: "string" },
mimeType: { type: "string" },
},
required: ["type"],
},
ResponseItem: {
type: "object",
properties: {
type: { type: "string" },
id: { type: "string" },
content: { type: "string" },
function_call: { $ref: "#/definitions/FunctionCall" },
},
},
FunctionCall: {
type: "object",
properties: {
name: { type: "string" },
arguments: { type: "string" },
call_id: { type: "string" },
},
required: ["name", "arguments"],
},
Message: {
type: "object",
properties: {
role: { type: "string", enum: ["user", "assistant", "system"] },
content: { type: "string" },
},
required: ["role", "content"],
},
};
console.log(` [ok] Using fallback schema with ${Object.keys(definitions).length} definitions`);
return createNormalizedSchema("codex", "Codex SDK Schema", definitions);
}

View file

@ -0,0 +1,109 @@
import { writeFileSync, existsSync, mkdirSync } from "fs";
import { join } from "path";
import { extractOpenCodeSchema } from "./opencode.js";
import { extractClaudeSchema } from "./claude.js";
import { extractCodexSchema } from "./codex.js";
import { extractAmpSchema } from "./amp.js";
import { validateSchema, type NormalizedSchema } from "./normalize.js";
const DIST_DIR = join(import.meta.dirname, "..", "dist");
type AgentName = "opencode" | "claude" | "codex" | "amp";
const EXTRACTORS: Record<AgentName, () => Promise<NormalizedSchema>> = {
opencode: extractOpenCodeSchema,
claude: extractClaudeSchema,
codex: extractCodexSchema,
amp: extractAmpSchema,
};
function parseArgs(): { agents: AgentName[] } {
const args = process.argv.slice(2);
const agentArg = args.find((arg) => arg.startsWith("--agent="));
if (agentArg) {
const agent = agentArg.split("=")[1] as AgentName;
if (!EXTRACTORS[agent]) {
console.error(`Unknown agent: ${agent}`);
console.error(`Valid agents: ${Object.keys(EXTRACTORS).join(", ")}`);
process.exit(1);
}
return { agents: [agent] };
}
return { agents: Object.keys(EXTRACTORS) as AgentName[] };
}
function ensureDistDir(): void {
if (!existsSync(DIST_DIR)) {
mkdirSync(DIST_DIR, { recursive: true });
}
}
async function extractAndWrite(agent: AgentName): Promise<boolean> {
try {
const extractor = EXTRACTORS[agent];
const schema = await extractor();
// Validate schema
const validation = validateSchema(schema);
if (!validation.valid) {
console.error(` [error] Schema validation failed for ${agent}:`);
validation.errors.forEach((err) => console.error(` - ${err}`));
return false;
}
// Write to file
const outputPath = join(DIST_DIR, `${agent}.json`);
writeFileSync(outputPath, JSON.stringify(schema, null, 2));
console.log(` [wrote] ${outputPath}`);
return true;
} catch (error) {
console.error(` [error] Failed to extract ${agent}: ${error}`);
return false;
}
}
async function main(): Promise<void> {
console.log("Agent Schema Extractor");
console.log("======================\n");
const { agents } = parseArgs();
ensureDistDir();
console.log(`Extracting schemas for: ${agents.join(", ")}\n`);
const results: Record<string, boolean> = {};
for (const agent of agents) {
results[agent] = await extractAndWrite(agent);
console.log();
}
// Summary
console.log("Summary");
console.log("-------");
const successful = Object.entries(results)
.filter(([, success]) => success)
.map(([name]) => name);
const failed = Object.entries(results)
.filter(([, success]) => !success)
.map(([name]) => name);
if (successful.length > 0) {
console.log(`Successful: ${successful.join(", ")}`);
}
if (failed.length > 0) {
console.log(`Failed: ${failed.join(", ")}`);
process.exit(1);
}
console.log("\nDone!");
}
main().catch((error) => {
console.error("Fatal error:", error);
process.exit(1);
});

View file

@ -0,0 +1,129 @@
import type { JSONSchema7 } from "json-schema";
export interface NormalizedSchema {
$schema: string;
$id: string;
title: string;
definitions: Record<string, JSONSchema7>;
}
/**
* Converts OpenAPI 3.1 schema to JSON Schema draft-07.
* OpenAPI 3.1 is largely compatible with JSON Schema draft 2020-12,
* but we want draft-07 for broader tool compatibility.
*/
export function openApiToJsonSchema(schema: Record<string, unknown>): JSONSchema7 {
const result: Record<string, unknown> = {};
for (const [key, value] of Object.entries(schema)) {
// Skip OpenAPI-specific fields
if (key === "discriminator" || key === "xml" || key === "externalDocs") {
continue;
}
// Handle nullable (OpenAPI 3.0 style)
if (key === "nullable" && value === true) {
continue; // Will be handled by type conversion
}
// Recursively convert nested schemas
if (key === "properties" && typeof value === "object" && value !== null) {
result[key] = {};
for (const [propName, propSchema] of Object.entries(value as Record<string, unknown>)) {
(result[key] as Record<string, unknown>)[propName] = openApiToJsonSchema(
propSchema as Record<string, unknown>
);
}
continue;
}
if (key === "items" && typeof value === "object" && value !== null) {
result[key] = openApiToJsonSchema(value as Record<string, unknown>);
continue;
}
if (key === "additionalProperties" && typeof value === "object" && value !== null) {
result[key] = openApiToJsonSchema(value as Record<string, unknown>);
continue;
}
if ((key === "oneOf" || key === "anyOf" || key === "allOf") && Array.isArray(value)) {
result[key] = value.map((item) =>
typeof item === "object" && item !== null
? openApiToJsonSchema(item as Record<string, unknown>)
: item
);
continue;
}
// Convert $ref paths from OpenAPI to local definitions
if (key === "$ref" && typeof value === "string") {
result[key] = value.replace("#/components/schemas/", "#/definitions/");
continue;
}
result[key] = value;
}
// Handle nullable by adding null to type array
if (schema["nullable"] === true && result["type"]) {
const currentType = result["type"];
if (Array.isArray(currentType)) {
if (!currentType.includes("null")) {
result["type"] = [...currentType, "null"];
}
} else {
result["type"] = [currentType as string, "null"];
}
}
return result as JSONSchema7;
}
/**
* Creates a normalized schema with consistent metadata.
*/
export function createNormalizedSchema(
id: string,
title: string,
definitions: Record<string, JSONSchema7>
): NormalizedSchema {
return {
$schema: "http://json-schema.org/draft-07/schema#",
$id: `https://sandbox-daemon/schemas/${id}.json`,
title,
definitions,
};
}
/**
* Validates a schema against JSON Schema draft-07 meta-schema.
* Basic validation - checks required fields and structure.
*/
export function validateSchema(schema: unknown): { valid: boolean; errors: string[] } {
const errors: string[] = [];
if (typeof schema !== "object" || schema === null) {
return { valid: false, errors: ["Schema must be an object"] };
}
const s = schema as Record<string, unknown>;
if (s.$schema && typeof s.$schema !== "string") {
errors.push("$schema must be a string");
}
if (s.definitions && typeof s.definitions !== "object") {
errors.push("definitions must be an object");
}
if (s.definitions && typeof s.definitions === "object") {
for (const [name, def] of Object.entries(s.definitions as Record<string, unknown>)) {
if (typeof def !== "object" || def === null) {
errors.push(`Definition "${name}" must be an object`);
}
}
}
return { valid: errors.length === 0, errors };
}

View file

@ -0,0 +1,55 @@
import { fetchWithCache } from "./cache.js";
import { createNormalizedSchema, openApiToJsonSchema, type NormalizedSchema } from "./normalize.js";
import type { JSONSchema7 } from "json-schema";
const OPENAPI_URL =
"https://raw.githubusercontent.com/sst/opencode/dev/packages/sdk/openapi.json";
// Key schemas we want to extract
const TARGET_SCHEMAS = [
"Session",
"Message",
"Part",
"Event",
"PermissionRequest",
"QuestionRequest",
"TextPart",
"ToolCallPart",
"ToolResultPart",
"ErrorPart",
];
interface OpenAPISpec {
components?: {
schemas?: Record<string, unknown>;
};
}
export async function extractOpenCodeSchema(): Promise<NormalizedSchema> {
console.log("Extracting OpenCode schema from OpenAPI spec...");
const specText = await fetchWithCache(OPENAPI_URL);
const spec: OpenAPISpec = JSON.parse(specText);
if (!spec.components?.schemas) {
throw new Error("OpenAPI spec missing components.schemas");
}
const definitions: Record<string, JSONSchema7> = {};
// Extract all schemas, not just target ones, to preserve references
for (const [name, schema] of Object.entries(spec.components.schemas)) {
definitions[name] = openApiToJsonSchema(schema as Record<string, unknown>);
}
// Verify target schemas exist
const missing = TARGET_SCHEMAS.filter((name) => !definitions[name]);
if (missing.length > 0) {
console.warn(` [warn] Missing expected schemas: ${missing.join(", ")}`);
}
const found = TARGET_SCHEMAS.filter((name) => definitions[name]);
console.log(` [ok] Extracted ${Object.keys(definitions).length} schemas (${found.length} target schemas)`);
return createNormalizedSchema("opencode", "OpenCode SDK Schema", definitions);
}

View file

@ -0,0 +1,16 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"esModuleInterop": true,
"strict": true,
"skipLibCheck": true,
"outDir": "dist",
"rootDir": "src",
"resolveJsonModule": true,
"declaration": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}

276
spec.md Normal file
View file

@ -0,0 +1,276 @@
i need to build a library that is a universal api to work with agents
## glossary
- agent = claude code, codex, and opencode -> the acutal binary/sdk that runs the coding agent
- agent mode = what the agent does, for example build/plan agent mode
- model = claude, codex, gemni, etc -> the model that's use din the agent
- variant = variant on the model if exists, eg low, mid, high, xhigh for codex
## concepts
### universal api types
we need to define a universal base type for input & output from agents that is a common denominator for all agent schemas
this also needs to support quesitons (ie human in the loop)
### working with the agents
these agents all have differnet ways of working with them.
- claude code uses headless mode
- codex uses a typescript sdk
- opencode uses a server
## component: daemon
this is what runs inside the sandbox to manage everything
this is a rust component that exposes an http server
**router**
use axum for routing and utoipa for the json schema and schemars for generating json schemas. see how this is done in:
- ~/rivet
- engine/packages/config-schema-gen/build.rs
- ~/rivet/engine/packages/api-public/src/router.rs (but use thiserror instead of anyhow)
we need a standard thiserror for error responses. return errors as RFC 7807 Problem Details
### cli
it's ran with a token like this using clap:
sandbox-daemon --token <token> --host xxxx --port xxxx
(you can specify --no-token too)
also expose a CLI endpoint for every http endpoint we have (specify this in claude.md to keep this to date) so we can do:
sandbox-daemon sessions get-messages --endpoint xxxx --token xxxx
### http api
POST /agents/{}/install (this will install the agent)
{}
POST /sessions/{} (will install agent if not already installed)
>
{
agent:"claud"|"codex"|"opencode",
model?:string,
variant?:string,
token?: string,
validateToken?: boolean
healthy: boolean,
error?: AgentError
}
POST /sessions/{}/messages
{
message: string
}
GET /sessions/{}/events?offset=x&limit=x
<
{
events: UniversalEvent[],
hasMore: bool
}
GET /sessions/{}/events/sse?offset=x
- same as bove but using sse
types:
type UniversalEvent = { message: UniversalMessage } | { started: Started } | { error: CrashInfo };
type AgentError = { tokenError: ... } | { processExisted: ... } | { installFailed: ... } | etc
### schema converters
we need to have a 2 way conversion for both:
- universal agent input message <-> agent input message
- universal agent event <-> agent event
for messages, we need to have a sepcial universal message type for failed to parse with the raw json that we attempted to parse
### managing agents
> **Note:** We do NOT use JS SDKs for agent communication. All agents are spawned as subprocesses or accessed via a shared server. This keeps the daemon language-agnostic (Rust) and avoids Node.js dependencies.
#### agent comparison
| Agent | Provider | Binary | Install Method | Session ID | Streaming Format |
|-------|----------|--------|----------------|------------|------------------|
| Claude Code | Anthropic | `claude` | curl installer (native binary) | `session_id` (string) | JSONL via stdout |
| Codex | OpenAI | `codex` | GitHub releases / Homebrew (Rust binary) | `thread_id` (string) | JSONL via stdout |
| OpenCode | Multi-provider | `opencode` | curl installer (Go binary) | `session_id` (string) | SSE or JSONL |
| Amp | Sourcegraph | `amp` | curl installer (bundled Bun) | `session_id` (string) | JSONL via stdout |
#### spawning approaches
There are two ways to spawn agents:
##### 1. subprocess per session
Each session spawns a dedicated agent subprocess that lives for the duration of the session.
**How it works:**
- On session create, spawn the agent binary with appropriate flags
- Communicate via stdin/stdout using JSONL
- Process terminates when session ends or times out
**Agents that support this:**
- **Claude Code**: `claude --print --output-format stream-json --verbose --dangerously-skip-permissions [--resume SESSION_ID] "PROMPT"`
- **Codex**: `codex exec --json --dangerously-bypass-approvals-and-sandbox "PROMPT"` or `codex exec resume --last`
- **Amp**: `amp --print --output-format stream-json --dangerously-skip-permissions "PROMPT"`
**Pros:**
- Simple implementation
- Process isolation per session
- No shared state to manage
**Cons:**
- Higher latency (process startup per message)
- More resource usage (one process per active session)
- No connection reuse
##### 2. shared server (preferred for OpenCode)
A single long-running server handles multiple sessions. The daemon connects to this server via HTTP/SSE.
**How it works:**
- On daemon startup (or first session for an agent), start the server if not running
- Server listens on a port (e.g., 4200-4300 range for OpenCode)
- Sessions are created/managed via HTTP API
- Events streamed via SSE
**Agents that support this:**
- **OpenCode**: `opencode serve --port PORT` starts the server, then use HTTP API:
- `POST /session` - create session
- `POST /session/{id}/prompt` - send message
- `GET /event/subscribe` - SSE event stream
- Supports questions/permissions via `/question/reply`, `/permission/reply`
**Pros:**
- Lower latency (no process startup per message)
- Shared resources across sessions
- Better for high-throughput scenarios
- Native support for SSE streaming
**Cons:**
- More complex lifecycle management
- Need to handle server crashes/restarts
- Shared state between sessions
#### which approach to use
| Agent | Recommended Approach | Reason |
|-------|---------------------|--------|
| Claude Code | Subprocess per session | No server mode available |
| Codex | Subprocess per session | No server mode available |
| OpenCode | Shared server | Native server support, lower latency |
| Amp | Subprocess per session | No server mode available |
#### installation
Before spawning, agents must be installed. **Prefer native installers over npm** - they have no Node.js dependency and are simpler to manage.
| Agent | Native Install (preferred) | Fallback (npm) | Verify |
|-------|---------------------------|----------------|--------|
| Claude Code | `curl -fsSL https://claude.ai/install.sh \| bash` | `npm i -g @anthropic-ai/claude-code` | `claude --version` |
| Codex | `brew install --cask codex` or [GitHub Releases](https://github.com/openai/codex/releases) | `npm i -g @openai/codex` | `codex --version` |
| OpenCode | `curl -fsSL https://opencode.ai/install \| bash` | `npm i -g opencode-ai` | `opencode --version` |
| Amp | `curl -fsSL https://ampcode.com/install.sh \| bash` | `npm i -g @sourcegraph/amp` | `amp --version` |
**Notes:**
- Claude Code native installer: signed by Anthropic, notarized by Apple on macOS
- Codex: Rust binary, download from GitHub releases and rename to `codex`
- OpenCode: Go binary, also available via Homebrew (`brew install anomalyco/tap/opencode`), Scoop, Nix
- Amp: bundles its own Bun runtime, no prerequisites needed
#### communication
**Subprocess mode (Claude Code, Codex, Amp):**
1. Spawn process with appropriate flags
2. Close stdin immediately after sending prompt (for single-turn) or keep open (for multi-turn)
3. Read JSONL events from stdout line-by-line
4. Parse each line as JSON and convert to `UniversalEvent`
5. Capture session/thread ID from events for resumption
6. Handle process exit/timeout
**Server mode (OpenCode):**
1. Ensure server is running (`opencode serve --port PORT`)
2. Create session via `POST /session`
3. Send prompts via `POST /session/{id}/prompt` (async version for streaming)
4. Subscribe to events via `GET /event/subscribe` (SSE)
5. Handle questions/permissions via dedicated endpoints
6. Session persists across multiple prompts
#### credential passing
| Agent | Env Var | Config File |
|-------|---------|-------------|
| Claude Code | `ANTHROPIC_API_KEY` | `~/.claude.json`, `~/.claude/.credentials.json` |
| Codex | `OPENAI_API_KEY` or `CODEX_API_KEY` | `~/.codex/auth.json` |
| OpenCode | `ANTHROPIC_API_KEY`, `OPENAI_API_KEY` | `~/.local/share/opencode/auth.json` |
| Amp | `ANTHROPIC_API_KEY` | Uses Claude Code credentials |
When spawning subprocesses, pass the API key via environment variable. For OpenCode server mode, the server reads credentials from its config on startup.
### testing
TODO
## component: sdks
we need to auto-generate types from our json schema for these languages
- typescript sdk
- also need to support standard schema
- can run in inline mode that doesn't require this
- python sdk
## spec todo
- generate common denominator with conversion functions
- how do we handle HIL
- how do you run each of these agents
- what else do we need, like todo, etc?
- how can we dump the spec for all of the agents somehow
## future problems to visit
- api features
- list agent modes available
- list models available
- handle planning mode
- api key gateway
- configuring mcp/skills/etc
- process management inside container
- otel
- better authentication systems
- s3-based file system
- ai sdk compatability for their ecosystem (useChat, etc)
- resumable messages
- todo lists
- all other features
- misc
- bootstrap tool that extracts tokens from the current system
- management ui
- skill
- pre-package these as bun binaries instead of npm installations
## future work
- provide a pty to access the agent data
- other agent features like file system
## misc
comparison to agentapi:
- it does not use the pty since we need to get more information from the agent