sandbox-agent/.context/proposal-rivetkit-sandbox-resilience.md

Proposal: RivetKit Sandbox Actor Resilience

Context

The rivetkit sandbox actor (src/sandbox/actor.ts) does not handle the case where the underlying cloud sandbox (e.g. E2B VM) is destroyed while the actor is still alive. This causes cascading 500 errors when the actor tries to call the dead sandbox. Additionally, a UNIQUE constraint bug in event persistence crashes the host process.

The sandbox-agent repo (which defines the E2B provider) will be updated separately to use autoPause and expose pause()/typed errors. This proposal covers the rivetkit-side changes needed to handle those signals.

Changes

1. Fix persistObservedEnvelope UNIQUE constraint crash

File: insertEvent in the sandbox actor's SQLite persistence layer

The sandbox_agent_events table has a UNIQUE constraint on (session_id, event_index). When the same event is observed twice (reconnection, replay, duplicate WebSocket delivery), the insert throws and crashes the host process as an unhandled rejection.

Fix: Change the INSERT to INSERT OR IGNORE / ON CONFLICT DO NOTHING. Duplicate events are expected and harmless — they should be silently deduplicated at the persistence layer.

2. Handle destroyed sandbox in ensureAgent()

File: src/sandbox/actor.ts — ensureAgent() function

When the provider's start() is called with an existing sandboxId and the sandbox no longer exists, the provider throws a typed SandboxDestroyedError (defined in the sandbox-agent provider contract).

ensureAgent() should catch this error and check the onSandboxExpired config option:

// New config option on sandboxActor()
onSandboxExpired?: "destroy" | "recreate"; // default: "destroy"

"destroy" (default):

• Set state.sandboxDestroyed = true
• Emit sandboxExpired event to all connected clients
• All subsequent action calls (runProcess, createSession, etc.) return a clear error: "Sandbox has expired. Create a new task to continue."
• The sandbox actor stays alive (preserves session history, audit log) but rejects new work

"recreate":

• Call provider create() to provision a fresh sandbox
• Store new sandboxId in state
• Emit sandboxRecreated event to connected clients with a notice that sessions are lost (new VM, no prior state)
• Resume normal operation with the new sandbox

3. Expose pause action

File: src/sandbox/actor.ts — actions

Add a pause action that delegates to the provider's pause() method. This is user-initiated only (e.g. user clicks "Pause sandbox" in UI to save credits). The sandbox actor should never auto-pause.

async pause(c) {
  await c.provider.pause();
  state.sandboxPaused = true;
  c.broadcast("sandboxPaused", {});
}

4. Expose resume action

File: src/sandbox/actor.ts — actions

Add a resume action for explicit recovery. Calls provider.start({ sandboxId: state.sandboxId }) which auto-resumes if paused.

async resume(c) {
  await ensureAgent(c); // handles reconnect internally
  state.sandboxPaused = false;
  c.broadcast("sandboxResumed", {});
}

5. Keep-alive while sessions are active

File: src/sandbox/actor.ts

While the sandbox actor has connected WebSocket clients, periodically extend the underlying sandbox TTL to prevent it from being garbage collected mid-session.

• On first client connect: start a keep-alive interval (e.g. every 2 minutes)
• Each tick: call provider.extendTimeout(extensionMs) (the provider maps this to sandbox.setTimeout() for E2B)
• On last client disconnect: clear the interval, let the sandbox idle toward its natural timeout

This prevents the common case where a user is actively working but the sandbox expires because the E2B default timeout (5 min) is too short. The timeoutMs in create options is the initial TTL; keep-alive extends it dynamically.

Key invariant

Never silently fail. Every destroyed/expired/error state must be surfaced to connected clients via events. The actor must always tell the UI what happened so the user can act on it. See CLAUDE.md "never silently catch errors" rule.

Dependencies

These changes depend on the sandbox-agent provider contract exposing:

• pause() method
• extendTimeout(ms) method
• Typed SandboxDestroyedError thrown from start() when sandbox is gone
• start() auto-resuming paused sandboxes via Sandbox.connect(sandboxId)