sandbox-agent/docs
Nathan Flurry 944ad1ba55 feat: replace neko with native GStreamer WebRTC for desktop streaming
Replace the neko binary dependency with a native GStreamer pipeline
(ximagesrc -> vp8enc -> webrtcbin) for desktop video streaming. This
removes the external neko process and integrates screen capture directly
via gstreamer-rs crate bindings behind a `desktop-gstreamer` feature flag.

Key changes:
- Add desktop_gstreamer.rs with GStreamer WebRTC pipeline management
- Rewrite signaling protocol (ready/offer/answer/candidate over WS)
- Add leaky queues and videorate for low-latency streaming
- Rewrite ICE candidates to 127.0.0.1 for Docker connectivity
- Constrain UDP port range (30000-30100) via libnice agent
- Update TypeScript SDK desktop-stream.ts for new signaling
- Update inspector DesktopTab with WebRTC Live View
- Update Dockerfiles to install GStreamer dev packages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 17:54:39 -07:00
..
ai fix host for docs 2026-02-13 04:53:53 +05:00
deploy Add desktop runtime API and SDK support 2026-03-15 17:54:16 -07:00
images feat(inspector): improve contrast and add collapsible debug panel (#182) 2026-02-13 07:15:40 +00:00
logo chore(site): links and image updates 2026-01-28 02:16:37 -08:00
agent-capabilities.mdx feat: add configuration for model, mode, and thought level (#205) 2026-03-06 00:24:32 -08:00
agent-sessions.mdx Add ACP permission mode support to the SDK (#224) 2026-03-10 21:52:43 -07:00
architecture.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
attachments.mdx Add ACP permission mode support to the SDK (#224) 2026-03-10 21:52:43 -07:00
cli.mdx Add desktop runtime API and SDK support 2026-03-15 17:54:16 -07:00
cors.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
custom-tools.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
daemon.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
docs.json wip (#258) 2026-03-15 12:37:42 -07:00
favicon.svg chore(site): diagram mobile and favicon 2026-01-31 22:49:01 -08:00
file-system.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
gigacode.mdx chore: update readme (#98) 2026-02-06 03:03:24 -08:00
inspector.mdx Add desktop runtime API and SDK support 2026-03-15 17:54:16 -07:00
llm-credentials.mdx wip (#258) 2026-03-15 12:37:42 -07:00
manage-sessions.mdx Add ACP permission mode support to the SDK (#224) 2026-03-10 21:52:43 -07:00
mcp-config.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
multiplayer.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
observability.mdx feat: add process management support (#207) 2026-03-07 17:58:31 -08:00
openapi.json feat: replace neko with native GStreamer WebRTC for desktop streaming 2026-03-16 17:54:39 -07:00
opencode-compatibility.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
pi-support-plan.md wip: pi working 2026-02-06 16:54:53 -05:00
processes.mdx Add ACP permission mode support to the SDK (#224) 2026-03-10 21:52:43 -07:00
quickstart.mdx Add desktop runtime API and SDK support 2026-03-15 17:54:16 -07:00
react-components.mdx Add transcript virtualization to Foundry UI (#255) 2026-03-14 17:55:05 -07:00
sdk-overview.mdx Add desktop runtime API and SDK support 2026-03-15 17:54:16 -07:00
security.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
session-persistence.mdx chore(release): update version to 0.3.0 2026-03-07 18:54:35 -08:00
session-restoration.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
session-transcript-schema.mdx Merge branch 'main' into feat/support-pi 2026-02-10 22:27:03 -08:00
skills-config.mdx chore: fix bad merge 2026-02-11 07:57:02 -08:00
telemetry.mdx docs: documentation overhaul and universal schema reference (#10) 2026-01-28 05:07:15 -08:00
theme.css docs: documentation overhaul and universal schema reference (#10) 2026-01-28 05:07:15 -08:00
troubleshooting.mdx fix: detect musl/glibc at runtime for correct Claude binary download 2026-01-28 04:19:35 -08:00