mirror of
https://github.com/harivansh-afk/sandbox-agent.git
synced 2026-04-18 03:00:42 +00:00
feat: desktop computer-use APIs with neko-based streaming
Add desktop runtime management (Xvfb, openbox, dbus), screen capture, mouse/keyboard input, and video streaming via neko binary extracted from the m1k1o/neko container. Includes Docker test rig, TypeScript SDK desktop support, and inspector Desktop tab. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
3895e34bdb
commit
33821d8660
66 changed files with 13190 additions and 1135 deletions
|
|
@ -35,6 +35,7 @@ console.log(url);
|
|||
- Prompt testing
|
||||
- Request/response debugging
|
||||
- Interactive permission prompts (approve, always-allow, or reject tool-use requests)
|
||||
- Desktop panel for status, remediation, start/stop, and screenshot refresh
|
||||
- Process management (create, stop, kill, delete, view logs)
|
||||
- Interactive PTY terminal for tty processes
|
||||
- One-shot command execution
|
||||
|
|
@ -50,3 +51,16 @@ console.log(url);
|
|||
The Inspector includes an embedded Ghostty-based terminal for interactive tty
|
||||
processes. The UI uses the SDK's high-level `connectProcessTerminal(...)`
|
||||
wrapper via the shared `@sandbox-agent/react` `ProcessTerminal` component.
|
||||
|
||||
## Desktop panel
|
||||
|
||||
The `Desktop` panel shows the current desktop runtime state, missing dependencies,
|
||||
the suggested install command, last error details, process/log paths, and the
|
||||
latest captured screenshot.
|
||||
|
||||
Use it to:
|
||||
|
||||
- Check whether desktop dependencies are installed
|
||||
- Start or stop the managed desktop runtime
|
||||
- Refresh desktop status
|
||||
- Capture a fresh screenshot on demand
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue