mirror of
https://github.com/harivansh-afk/sandbox-agent.git
synced 2026-04-18 08:02:55 +00:00
feat: replace neko with native GStreamer WebRTC for desktop streaming
Replace the neko binary dependency with a native GStreamer pipeline (ximagesrc -> vp8enc -> webrtcbin) for desktop video streaming. This removes the external neko process and integrates screen capture directly via gstreamer-rs crate bindings behind a `desktop-gstreamer` feature flag. Key changes: - Add desktop_gstreamer.rs with GStreamer WebRTC pipeline management - Rewrite signaling protocol (ready/offer/answer/candidate over WS) - Add leaky queues and videorate for low-latency streaming - Rewrite ICE candidates to 127.0.0.1 for Docker connectivity - Constrain UDP port range (30000-30100) via libnice agent - Update TypeScript SDK desktop-stream.ts for new signaling - Update inspector DesktopTab with WebRTC Live View - Update Dockerfiles to install GStreamer dev packages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
e638148345
commit
944ad1ba55
22 changed files with 1848 additions and 1170 deletions
|
|
@ -1,6 +1,16 @@
|
|||
FROM rust:1.88.0-bookworm AS builder
|
||||
WORKDIR /build
|
||||
|
||||
# Install GStreamer dev packages for the desktop-gstreamer feature.
|
||||
RUN apt-get update -qq && \
|
||||
apt-get install -y -qq --no-install-recommends \
|
||||
libgstreamer1.0-dev \
|
||||
libgstreamer-plugins-base1.0-dev \
|
||||
libgstreamer-plugins-bad1.0-dev \
|
||||
libnice-dev \
|
||||
> /dev/null 2>&1 && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY Cargo.toml Cargo.lock ./
|
||||
COPY server/ ./server/
|
||||
COPY gigacode/ ./gigacode/
|
||||
|
|
@ -12,7 +22,7 @@ ENV SANDBOX_AGENT_SKIP_INSPECTOR=1
|
|||
RUN --mount=type=cache,target=/usr/local/cargo/registry \
|
||||
--mount=type=cache,target=/usr/local/cargo/git \
|
||||
--mount=type=cache,target=/build/target \
|
||||
cargo build -p sandbox-agent --release && \
|
||||
cargo build -p sandbox-agent --release --features desktop-gstreamer && \
|
||||
cp target/release/sandbox-agent /sandbox-agent
|
||||
|
||||
FROM node:22-bookworm-slim
|
||||
|
|
@ -26,6 +36,15 @@ RUN apt-get update -qq && \
|
|||
xdotool \
|
||||
imagemagick \
|
||||
ffmpeg \
|
||||
gstreamer1.0-tools \
|
||||
gstreamer1.0-plugins-base \
|
||||
gstreamer1.0-plugins-good \
|
||||
gstreamer1.0-plugins-bad \
|
||||
gstreamer1.0-plugins-ugly \
|
||||
gstreamer1.0-nice \
|
||||
gstreamer1.0-x \
|
||||
gstreamer1.0-pulseaudio \
|
||||
libxcvt0 \
|
||||
x11-xserver-utils \
|
||||
dbus-x11 \
|
||||
xauth \
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue