mirror of
https://github.com/harivansh-afk/sandbox-agent.git
synced 2026-04-15 09:01:17 +00:00
Add desktop runtime management (Xvfb, openbox, dbus), screen capture, mouse/keyboard input, and video streaming via neko binary extracted from the m1k1o/neko container. Includes Docker test rig, TypeScript SDK desktop support, and inspector Desktop tab. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
276 lines
7 KiB
Text
276 lines
7 KiB
Text
---
|
|
title: "SDK Overview"
|
|
description: "Use the TypeScript SDK to manage Sandbox Agent sessions and APIs."
|
|
icon: "compass"
|
|
---
|
|
|
|
The TypeScript SDK is centered on `sandbox-agent` and its `SandboxAgent` class.
|
|
|
|
## Install
|
|
|
|
<Tabs>
|
|
<Tab title="npm">
|
|
```bash
|
|
npm install sandbox-agent@0.3.x
|
|
```
|
|
</Tab>
|
|
<Tab title="bun">
|
|
```bash
|
|
bun add sandbox-agent@0.3.x
|
|
# Allow Bun to run postinstall scripts for native binaries (required for SandboxAgent.start()).
|
|
bun pm trust @sandbox-agent/cli-linux-x64 @sandbox-agent/cli-linux-arm64 @sandbox-agent/cli-darwin-arm64 @sandbox-agent/cli-darwin-x64 @sandbox-agent/cli-win32-x64
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
## Optional React components
|
|
|
|
```bash
|
|
npm install @sandbox-agent/react@0.3.x
|
|
```
|
|
|
|
## Create a client
|
|
|
|
```ts
|
|
import { SandboxAgent } from "sandbox-agent";
|
|
|
|
const sdk = await SandboxAgent.connect({
|
|
baseUrl: "http://127.0.0.1:2468",
|
|
});
|
|
```
|
|
|
|
`SandboxAgent.connect(...)` now waits for `/v1/health` by default before other SDK requests proceed. To disable that gate, pass `waitForHealth: false`. To keep the default gate but fail after a bounded wait, pass `waitForHealth: { timeoutMs: 120_000 }`. To cancel the startup wait early, pass `signal: abortController.signal`.
|
|
|
|
With a custom fetch handler (for example, proxying requests inside Workers):
|
|
|
|
```ts
|
|
const sdk = await SandboxAgent.connect({
|
|
fetch: (input, init) => customFetch(input, init),
|
|
});
|
|
```
|
|
|
|
With an abort signal for the startup health gate:
|
|
|
|
```ts
|
|
const controller = new AbortController();
|
|
|
|
const sdk = await SandboxAgent.connect({
|
|
baseUrl: "http://127.0.0.1:2468",
|
|
signal: controller.signal,
|
|
});
|
|
|
|
controller.abort();
|
|
```
|
|
|
|
With persistence (see [Persisting Sessions](/session-persistence) for driver options):
|
|
|
|
```ts
|
|
import { SandboxAgent, InMemorySessionPersistDriver } from "sandbox-agent";
|
|
|
|
const persist = new InMemorySessionPersistDriver();
|
|
|
|
const sdk = await SandboxAgent.connect({
|
|
baseUrl: "http://127.0.0.1:2468",
|
|
persist,
|
|
});
|
|
```
|
|
|
|
Local spawn with a sandbox provider:
|
|
|
|
```ts
|
|
import { SandboxAgent } from "sandbox-agent";
|
|
import { local } from "sandbox-agent/local";
|
|
|
|
const sdk = await SandboxAgent.start({
|
|
sandbox: local(),
|
|
});
|
|
|
|
// sdk.sandboxId — prefixed provider ID (e.g. "local/127.0.0.1:2468")
|
|
|
|
await sdk.destroySandbox(); // provider-defined cleanup + disposes client
|
|
```
|
|
|
|
`SandboxAgent.start(...)` requires a `sandbox` provider. Built-in providers:
|
|
|
|
| Import | Provider |
|
|
|--------|----------|
|
|
| `sandbox-agent/local` | Local subprocess |
|
|
| `sandbox-agent/docker` | Docker container |
|
|
| `sandbox-agent/e2b` | E2B sandbox |
|
|
| `sandbox-agent/daytona` | Daytona workspace |
|
|
| `sandbox-agent/vercel` | Vercel Sandbox |
|
|
| `sandbox-agent/cloudflare` | Cloudflare Sandbox |
|
|
|
|
Use `sdk.dispose()` to disconnect without changing sandbox state, `sdk.pauseSandbox()` for graceful suspension when supported, or `sdk.killSandbox()` for permanent deletion.
|
|
|
|
## Session flow
|
|
|
|
```ts
|
|
const session = await sdk.createSession({
|
|
agent: "mock",
|
|
cwd: "/",
|
|
});
|
|
|
|
const prompt = await session.prompt([
|
|
{ type: "text", text: "Summarize this repository." },
|
|
]);
|
|
|
|
console.log(prompt.stopReason);
|
|
```
|
|
|
|
Load and destroy:
|
|
|
|
```ts
|
|
const restored = await sdk.resumeSession(session.id);
|
|
await restored.prompt([{ type: "text", text: "Continue from previous context." }]);
|
|
|
|
await sdk.destroySession(restored.id);
|
|
```
|
|
|
|
## Session configuration
|
|
|
|
Set model, mode, or thought level at creation or on an existing session:
|
|
|
|
```ts
|
|
const session = await sdk.createSession({
|
|
agent: "codex",
|
|
model: "gpt-5.3-codex",
|
|
});
|
|
|
|
await session.setModel("gpt-5.2-codex");
|
|
await session.setMode("auto");
|
|
|
|
const options = await session.getConfigOptions();
|
|
const modes = await session.getModes();
|
|
```
|
|
|
|
Handle permission requests from agents that ask before executing tools:
|
|
|
|
```ts
|
|
const claude = await sdk.createSession({
|
|
agent: "claude",
|
|
mode: "default",
|
|
});
|
|
|
|
claude.onPermissionRequest((request) => {
|
|
void claude.respondPermission(request.id, "once");
|
|
});
|
|
```
|
|
|
|
See [Agent Sessions](/agent-sessions) for full details on config options and error handling.
|
|
|
|
## Events
|
|
|
|
Subscribe to live events:
|
|
|
|
```ts
|
|
const unsubscribe = session.onEvent((event) => {
|
|
console.log(event.eventIndex, event.sender, event.payload);
|
|
});
|
|
|
|
await session.prompt([{ type: "text", text: "Give me a short summary." }]);
|
|
unsubscribe();
|
|
```
|
|
|
|
Fetch persisted events:
|
|
|
|
```ts
|
|
const page = await sdk.getEvents({
|
|
sessionId: session.id,
|
|
limit: 100,
|
|
});
|
|
|
|
console.log(page.items.length);
|
|
```
|
|
|
|
## Control-plane and HTTP helpers
|
|
|
|
```ts
|
|
const health = await sdk.getHealth();
|
|
const agents = await sdk.listAgents();
|
|
await sdk.installAgent("codex", { reinstall: true });
|
|
|
|
const entries = await sdk.listFsEntries({ path: "." });
|
|
const writeResult = await sdk.writeFsFile({ path: "./hello.txt" }, "hello");
|
|
|
|
console.log(health.status, agents.agents.length, entries.length, writeResult.path);
|
|
```
|
|
|
|
## Desktop API
|
|
|
|
The SDK also wraps the desktop host/runtime HTTP API.
|
|
|
|
Install desktop dependencies first on Linux hosts:
|
|
|
|
```bash
|
|
sandbox-agent install desktop --yes
|
|
```
|
|
|
|
Then query status, surface remediation if needed, and start the runtime:
|
|
|
|
```ts
|
|
const status = await sdk.getDesktopStatus();
|
|
|
|
if (status.state === "install_required") {
|
|
console.log(status.installCommand);
|
|
}
|
|
|
|
const started = await sdk.startDesktop({
|
|
width: 1440,
|
|
height: 900,
|
|
dpi: 96,
|
|
});
|
|
|
|
const screenshot = await sdk.takeDesktopScreenshot();
|
|
const displayInfo = await sdk.getDesktopDisplayInfo();
|
|
|
|
await sdk.moveDesktopMouse({ x: 400, y: 300 });
|
|
await sdk.clickDesktop({ x: 400, y: 300, button: "left", clickCount: 1 });
|
|
await sdk.typeDesktopText({ text: "hello world", delayMs: 10 });
|
|
await sdk.pressDesktopKey({ key: "ctrl+l" });
|
|
|
|
await sdk.stopDesktop();
|
|
```
|
|
|
|
Screenshot helpers return `Uint8Array` PNG bytes. The SDK does not attempt to install OS packages remotely; callers should surface `missingDependencies` and `installCommand` from `getDesktopStatus()`.
|
|
|
|
## Error handling
|
|
|
|
```ts
|
|
import { SandboxAgentError } from "sandbox-agent";
|
|
|
|
try {
|
|
await sdk.listAgents();
|
|
} catch (error) {
|
|
if (error instanceof SandboxAgentError) {
|
|
console.error(error.status, error.problem);
|
|
}
|
|
}
|
|
```
|
|
|
|
## Inspector URL
|
|
|
|
```ts
|
|
import { buildInspectorUrl } from "sandbox-agent";
|
|
|
|
const url = buildInspectorUrl({
|
|
baseUrl: "https://your-sandbox-agent.example.com",
|
|
headers: { "X-Custom-Header": "value" },
|
|
});
|
|
|
|
console.log(url);
|
|
```
|
|
|
|
Parameters:
|
|
|
|
- `baseUrl` (required unless `fetch` is provided): Sandbox Agent server URL
|
|
- `token` (optional): Bearer token for authenticated servers
|
|
- `headers` (optional): Additional request headers
|
|
- `fetch` (optional): Custom fetch implementation used by SDK HTTP and session calls
|
|
- `skipHealthCheck` (optional): set `true` to skip the startup `/v1/health` wait
|
|
- `waitForHealth` (optional, defaults to enabled): waits for `/v1/health` before HTTP helpers and session setup proceed; pass `false` to disable or `{ timeoutMs }` to bound the wait
|
|
- `signal` (optional): aborts the startup `/v1/health` wait used by `connect()`
|
|
|
|
## LLM credentials
|
|
|
|
Sandbox Agent supports personal API keys, shared organization keys, and per-tenant gateway keys with budget enforcement. See [LLM Credentials](/llm-credentials) for setup details.
|