harivansh-afk/sandbox-agent
1
0
Fork 0
mirror of https://github.com/harivansh-afk/sandbox-agent.git synced 2026-04-20 22:02:42 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: replace neko with native GStreamer WebRTC for desktop streaming

Browse source 
Replace the neko binary dependency with a native GStreamer pipeline
(ximagesrc -> vp8enc -> webrtcbin) for desktop video streaming. This
removes the external neko process and integrates screen capture directly
via gstreamer-rs crate bindings behind a `desktop-gstreamer` feature flag.

Key changes:
- Add desktop_gstreamer.rs with GStreamer WebRTC pipeline management
- Rewrite signaling protocol (ready/offer/answer/candidate over WS)
- Add leaky queues and videorate for low-latency streaming
- Rewrite ICE candidates to 127.0.0.1 for Docker connectivity
- Constrain UDP port range (30000-30100) via libnice agent
- Update TypeScript SDK desktop-stream.ts for new signaling
- Update inspector DesktopTab with WebRTC Live View
- Update Dockerfiles to install GStreamer dev packages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Nathan Flurry 2026-03-16 17:54:39 -07:00
parent e638148345
commit 944ad1ba55
22 changed files with 1848 additions and 1170 deletions
Download patch file Download diff file Expand all files Collapse all files

4
server/packages/sandbox-agent/Cargo.toml
View file

@ -42,6 +42,9 @@ toml_edit.workspace = true
tar.workspace = true
zip.workspace = true
tempfile = { workspace = true, optional = true }
gstreamer = { version = "0.23", optional = true }
gstreamer-sdp = { version = "0.23", optional = true }
gstreamer-webrtc = { version = "0.23", optional = true }
[target.'cfg(unix)'.dependencies]
libc = "0.2"
@ -59,3 +62,4 @@ tokio-tungstenite = "0.24"
[features]
test-utils = ["tempfile"]
desktop-gstreamer = ["gstreamer", "gstreamer-sdp", "gstreamer-webrtc"]