diff --git a/packages/mom/docs/v86.md b/packages/mom/docs/v86.md new file mode 100644 index 00000000..817e101f --- /dev/null +++ b/packages/mom/docs/v86.md @@ -0,0 +1,319 @@ +# v86 Sandbox Evaluation + +v86 is an x86 emulator written in JavaScript/WebAssembly that can run Linux in the browser or Node.js. This document details our evaluation for using it as a sandboxed execution environment. + +## Overview + +- **What it is**: x86 PC emulator (32-bit, Pentium 4 level) +- **How it works**: Translates machine code to WebAssembly at runtime +- **Guest OS**: Alpine Linux 3.21 (32-bit x86) +- **Available packages**: Node.js 22, Python 3.12, git, curl, etc. (full Alpine repos) + +## Key Findings + +### What Works + +| Feature | Status | Notes | +|---------|--------|-------| +| Outbound TCP | ✅ | HTTP, HTTPS, TLS all work | +| Outbound UDP | ✅ | DNS queries work | +| WebSocket client | ✅ | Can connect to external WebSocket servers | +| File I/O | ✅ | 9p filesystem for host<->guest file exchange | +| State save/restore | ✅ | ~80-100MB state files, instant resume | +| Package persistence | ✅ | Installed packages persist in saved state | +| npm install | ✅ | Works (outbound HTTPS) | +| git clone | ✅ | Works (outbound HTTPS) | + +### What Doesn't Work + +| Feature | Status | Notes | +|---------|--------|-------| +| Inbound connections | ❌ | VM is behind NAT (10.0.2.x), needs port forwarding | +| ICMP ping | ❌ | Userspace network stack limitation | +| 64-bit | ❌ | v86 only emulates 32-bit x86 | + +## Architecture + +``` +┌─────────────────────────────────────────────────────────┐ +│ Host (Node.js) │ +│ │ +│ ┌──────────────┐ ┌─────────────────────────────┐ │ +│ │ rootlessRelay│◄───►│ v86 │ │ +│ │ (WebSocket) │ │ ┌─────────────────────┐ │ │ +│ │ │ │ │ Alpine Linux │ │ │ +│ │ - DHCP │ │ │ - Node.js 22 │ │ │ +│ │ - DNS proxy │ │ │ - Python 3.12 │ │ │ +│ │ - NAT │ │ │ - etc. │ │ │ +│ └──────────────┘ │ └─────────────────────┘ │ │ +│ │ │ │ │ │ +│ │ │ 9p filesystem │ │ +│ ▼ │ │ │ │ +│ Internet │ ▼ │ │ +│ │ Host filesystem │ │ +│ └─────────────────────────────┘ │ +└─────────────────────────────────────────────────────────┘ +``` + +## Components & Sizes + +| Component | Size | Purpose | +|-----------|------|---------| +| v86.wasm | ~2 MB | x86 emulator | +| libv86.mjs | ~330 KB | JavaScript runtime | +| seabios.bin | ~128 KB | BIOS | +| vgabios.bin | ~36 KB | VGA BIOS | +| Alpine rootfs | ~57 MB | Compressed filesystem (loaded on-demand) | +| alpine-fs.json | ~160 KB | Filesystem index | +| rootlessRelay | ~75 KB | Network relay | +| **Total** | **~60 MB** | Without saved state | +| Saved state | ~80-100 MB | Optional, for instant resume | + +## Installation + +```bash +npm install v86 ws +``` + +## Building the Alpine Image + +v86 provides Docker tooling to build the Alpine image: + +```bash +git clone https://github.com/copy/v86.git +cd v86/tools/docker/alpine + +# Edit Dockerfile to add packages: +# ENV ADDPKGS=nodejs,npm,python3,git,curl + +./build.sh +``` + +This creates: +- `images/alpine-fs.json` - Filesystem index +- `images/alpine-rootfs-flat/` - Compressed file chunks + +## Network Relay Setup + +v86 needs a network relay for TCP/UDP connectivity. We use rootlessRelay: + +```bash +git clone https://github.com/obegron/rootlessRelay.git +cd rootlessRelay +npm install +``` + +### Required Patches for Host Access + +To allow the VM to connect to host services via the gateway IP (10.0.2.2), apply these patches to `relay.js`: + +**Patch 1: Disable reverse TCP handling for gateway (line ~684)** +```javascript +// Change: +if (protocol === 6 && dstIP === GATEWAY_IP) { + this.handleReverseTCP(ipPacket); + return; +} + +// To: +if (false && protocol === 6 && dstIP === GATEWAY_IP) { // PATCHED + this.handleReverseTCP(ipPacket); + return; +} +``` + +**Patch 2: Redirect gateway TCP to localhost (line ~792)** +```javascript +// Change: +const socket = net.connect(dstPort, dstIP, () => { + +// To: +const actualDstIP = dstIP === GATEWAY_IP ? "127.0.0.1" : dstIP; +const socket = net.connect(dstPort, actualDstIP, () => { +``` + +**Patch 3: Redirect gateway UDP to localhost (lines ~1431 and ~1449)** +```javascript +// Change: +this.udpSocket.send(payload, dstPort, dstIP, (err) => { + +// To: +const actualUdpDstIP = dstIP === GATEWAY_IP ? "127.0.0.1" : dstIP; +this.udpSocket.send(payload, dstPort, actualUdpDstIP, (err) => { +``` + +### Starting the Relay + +```bash +ENABLE_WSS=false LOG_LEVEL=1 node relay.js +# Listens on ws://127.0.0.1:8086/ +``` + +## Basic Usage + +```javascript +import { V86 } from "v86"; +import path from "node:path"; + +const emulator = new V86({ + wasm_path: path.join(__dirname, "node_modules/v86/build/v86.wasm"), + bios: { url: path.join(__dirname, "bios/seabios.bin") }, + vga_bios: { url: path.join(__dirname, "bios/vgabios.bin") }, + filesystem: { + basefs: path.join(__dirname, "images/alpine-fs.json"), + baseurl: path.join(__dirname, "images/alpine-rootfs-flat/"), + }, + autostart: true, + memory_size: 512 * 1024 * 1024, + bzimage_initrd_from_filesystem: true, + cmdline: "rw root=host9p rootfstype=9p rootflags=trans=virtio,cache=loose modules=virtio_pci tsc=reliable console=ttyS0", + net_device: { + type: "virtio", + relay_url: "ws://127.0.0.1:8086/", + }, +}); + +// Capture output +emulator.add_listener("serial0-output-byte", (byte) => { + process.stdout.write(String.fromCharCode(byte)); +}); + +// Send commands +emulator.serial0_send("echo hello\n"); +``` + +## Communication Methods + +### 1. Serial Console (stdin/stdout) + +```javascript +// Send command +emulator.serial0_send("ls -la\n"); + +// Receive output +let output = ""; +emulator.add_listener("serial0-output-byte", (byte) => { + output += String.fromCharCode(byte); +}); +``` + +### 2. 9p Filesystem (file I/O) + +```javascript +// Write file to VM +const data = new TextEncoder().encode("#!/bin/sh\necho hello\n"); +await emulator.create_file("/tmp/script.sh", data); + +// Read file from VM +const result = await emulator.read_file("/tmp/output.txt"); +console.log(new TextDecoder().decode(result)); +``` + +### 3. Network (TCP to host services) + +From inside the VM, connect to `10.0.2.2:PORT` to reach `localhost:PORT` on the host (requires patched relay). + +```bash +# Inside VM +wget http://10.0.2.2:8080/ # Connects to host's localhost:8080 +``` + +## State Save/Restore + +```javascript +// Save state (includes all installed packages, files, etc.) +const state = await emulator.save_state(); +fs.writeFileSync("vm-state.bin", Buffer.from(state)); + +// Restore state (instant resume, ~2 seconds) +const stateBuffer = fs.readFileSync("vm-state.bin"); +await emulator.restore_state(stateBuffer.buffer); +``` + +## Network Setup Inside VM + +After boot, run these commands to enable networking: + +```bash +modprobe virtio-net +ip link set eth0 up +udhcpc -i eth0 +``` + +Or as a one-liner: +```bash +modprobe virtio-net && ip link set eth0 up && udhcpc -i eth0 +``` + +The VM will get IP `10.0.2.15` (or similar) via DHCP from the relay. + +## Performance + +| Metric | Value | +|--------|-------| +| Cold boot | ~20-25 seconds | +| State restore | ~2-3 seconds | +| Memory usage | ~512 MB (configurable) | + +## Typical Workflow for Mom + +1. **First run**: + - Start rootlessRelay + - Boot v86 with Alpine (~25s) + - Setup network + - Install needed packages (`apk add nodejs npm python3 git`) + - Save state + +2. **Subsequent runs**: + - Start rootlessRelay + - Restore saved state (~2s) + - Ready to execute commands + +3. **Command execution**: + - Send commands via `serial0_send()` + - Capture output via `serial0-output-byte` listener + - Exchange files via 9p filesystem + +## Alternative: fetch Backend (No Relay Needed) + +For HTTP-only networking, v86 has a built-in `fetch` backend: + +```javascript +net_device: { + type: "virtio", + relay_url: "fetch", +} +``` + +This uses the browser/Node.js `fetch()` API for HTTP requests. Limitations: +- Only HTTP/HTTPS (no raw TCP/UDP) +- No WebSocket +- Host access via `http://.external` (e.g., `http://8080.external`) + +## Files Reference + +After building, you need these files: + +``` +project/ +├── node_modules/v86/build/ +│ ├── v86.wasm +│ └── libv86.mjs +├── bios/ +│ ├── seabios.bin +│ └── vgabios.bin +├── images/ +│ ├── alpine-fs.json +│ └── alpine-rootfs-flat/ +│ └── *.bin.zst (many files) +└── rootlessRelay/ + └── relay.js (patched) +``` + +## Resources + +- [v86 GitHub](https://github.com/copy/v86) +- [v86 Networking Docs](https://github.com/copy/v86/blob/master/docs/networking.md) +- [v86 Alpine Setup](https://github.com/copy/v86/tree/master/tools/docker/alpine) +- [rootlessRelay](https://github.com/obegron/rootlessRelay) +- [v86 npm package](https://www.npmjs.com/package/v86)