harivansh-afk/sandbox-agent
1
0
Fork 0
mirror of https://github.com/harivansh-afk/sandbox-agent.git synced 2026-04-19 09:04:48 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: desktop computer-use APIs with neko-based streaming

Browse source 
Add desktop runtime management (Xvfb, openbox, dbus), screen capture,
mouse/keyboard input, and video streaming via neko binary extracted
from the m1k1o/neko container. Includes Docker test rig, TypeScript SDK
desktop support, and inspector Desktop tab.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Nathan Flurry 2026-03-16 17:56:39 -07:00
parent 3895e34bdb
commit 33821d8660
66 changed files with 13190 additions and 1135 deletions
Download patch file Download diff file Expand all files Collapse all files

93
frontend/packages/inspector/index.html
View file

@ -2889,6 +2889,94 @@
gap: 20px;
}
.desktop-panel {
display: flex;
flex-direction: column;
gap: 16px;
}
.desktop-state-grid {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 12px;
margin-bottom: 12px;
}
.desktop-start-controls {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 10px;
}
.desktop-input-group {
display: flex;
flex-direction: column;
gap: 4px;
}
.desktop-chip-list {
display: flex;
flex-wrap: wrap;
gap: 8px;
}
.desktop-command {
margin-top: 6px;
padding: 8px 10px;
border-radius: var(--radius);
border: 1px solid var(--border);
background: var(--surface);
overflow-x: auto;
}
.desktop-diagnostic-block + .desktop-diagnostic-block {
margin-top: 14px;
}
.desktop-process-list {
display: flex;
flex-direction: column;
gap: 10px;
margin-top: 8px;
}
.desktop-process-item {
padding: 10px;
border-radius: var(--radius);
border: 1px solid var(--border);
background: var(--surface);
display: flex;
flex-direction: column;
gap: 4px;
}
.desktop-screenshot-empty {
padding: 18px;
border: 1px dashed var(--border);
border-radius: var(--radius);
color: var(--muted);
background: var(--surface);
text-align: center;
}
.desktop-screenshot-frame {
border-radius: calc(var(--radius) + 2px);
overflow: hidden;
border: 1px solid var(--border);
background:
linear-gradient(135deg, rgba(15, 23, 42, 0.9), rgba(30, 41, 59, 0.92)),
radial-gradient(circle at top right, rgba(56, 189, 248, 0.12), transparent 40%);
padding: 10px;
}
.desktop-screenshot-image {
display: block;
width: 100%;
height: auto;
border-radius: var(--radius);
background: rgba(0, 0, 0, 0.24);
}
.processes-section {
display: flex;
flex-direction: column;
@ -3551,6 +3639,11 @@
grid-template-columns: 1fr;
}
.desktop-state-grid,
.desktop-start-controls {
grid-template-columns: 1fr;
}
.session-sidebar {
display: none;
}