co-mono/packages/mom/docs/v86.md

# v86 Sandbox Evaluation

v86 is an x86 emulator written in JavaScript/WebAssembly that can run Linux in the browser or Node.js. This document details our evaluation for using it as a sandboxed execution environment.

## Overview

- **What it is**: x86 PC emulator (32-bit, Pentium 4 level)
- **How it works**: Translates machine code to WebAssembly at runtime
- **Guest OS**: Alpine Linux 3.21 (32-bit x86)
- **Available packages**: Node.js 22, Python 3.12, git, curl, etc. (full Alpine repos)

## Key Findings

### What Works

| Feature | Status | Notes |
|---------|--------|-------|
| Outbound TCP | ✅ | HTTP, HTTPS, TLS all work |
| Outbound UDP | ✅ | DNS queries work |
| WebSocket client | ✅ | Can connect to external WebSocket servers |
| File I/O | ✅ | 9p filesystem for host<->guest file exchange |
| State save/restore | ✅ | ~80-100MB state files, instant resume |
| Package persistence | ✅ | Installed packages persist in saved state |
| npm install | ✅ | Works (outbound HTTPS) |
| git clone | ✅ | Works (outbound HTTPS) |

### What Doesn't Work

| Feature | Status | Notes |
|---------|--------|-------|
| Inbound connections | ❌ | VM is behind NAT (10.0.2.x), needs port forwarding |
| ICMP ping | ❌ | Userspace network stack limitation |
| 64-bit | ❌ | v86 only emulates 32-bit x86 |

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│                      Host (Node.js)                      │
│                                                          │
│  ┌──────────────┐     ┌─────────────────────────────┐   │
│  │ rootlessRelay│◄───►│           v86               │   │
│  │  (WebSocket) │     │  ┌─────────────────────┐    │   │
│  │              │     │  │   Alpine Linux      │    │   │
│  │  - DHCP      │     │  │   - Node.js 22      │    │   │
│  │  - DNS proxy │     │  │   - Python 3.12     │    │   │
│  │  - NAT       │     │  │   - etc.            │    │   │
│  └──────────────┘     │  └─────────────────────┘    │   │
│         │             │            │                 │   │
│         │             │     9p filesystem            │   │
│         ▼             │            │                 │   │
│    Internet           │            ▼                 │   │
│                       │     Host filesystem          │   │
│                       └─────────────────────────────┘   │
└─────────────────────────────────────────────────────────┘
```

## Components & Sizes

| Component | Size | Purpose |
|-----------|------|---------|
| v86.wasm | ~2 MB | x86 emulator |
| libv86.mjs | ~330 KB | JavaScript runtime |
| seabios.bin | ~128 KB | BIOS |
| vgabios.bin | ~36 KB | VGA BIOS |
| Alpine rootfs | ~57 MB | Compressed filesystem (loaded on-demand) |
| alpine-fs.json | ~160 KB | Filesystem index |
| rootlessRelay | ~75 KB | Network relay |
| **Total** | **~60 MB** | Without saved state |
| Saved state | ~80-100 MB | Optional, for instant resume |

## Installation

```bash
npm install v86 ws
```

## Building the Alpine Image

v86 provides Docker tooling to build the Alpine image:

```bash
git clone https://github.com/copy/v86.git
cd v86/tools/docker/alpine

# Edit Dockerfile to add packages:
# ENV ADDPKGS=nodejs,npm,python3,git,curl

./build.sh
```

This creates:
- `images/alpine-fs.json` - Filesystem index
- `images/alpine-rootfs-flat/` - Compressed file chunks

## Network Relay Setup

v86 needs a network relay for TCP/UDP connectivity. We use rootlessRelay:

```bash
git clone https://github.com/obegron/rootlessRelay.git
cd rootlessRelay
npm install
```

### Required Patches for Host Access

To allow the VM to connect to host services via the gateway IP (10.0.2.2), apply these patches to `relay.js`:

**Patch 1: Disable reverse TCP handling for gateway (line ~684)**
```javascript
// Change:
if (protocol === 6 && dstIP === GATEWAY_IP) {
  this.handleReverseTCP(ipPacket);
  return;
}

// To:
if (false && protocol === 6 && dstIP === GATEWAY_IP) { // PATCHED
  this.handleReverseTCP(ipPacket);
  return;
}
```

**Patch 2: Redirect gateway TCP to localhost (line ~792)**
```javascript
// Change:
const socket = net.connect(dstPort, dstIP, () => {

// To:
const actualDstIP = dstIP === GATEWAY_IP ? "127.0.0.1" : dstIP;
const socket = net.connect(dstPort, actualDstIP, () => {
```

**Patch 3: Redirect gateway UDP to localhost (lines ~1431 and ~1449)**
```javascript
// Change:
this.udpSocket.send(payload, dstPort, dstIP, (err) => {

// To:
const actualUdpDstIP = dstIP === GATEWAY_IP ? "127.0.0.1" : dstIP;
this.udpSocket.send(payload, dstPort, actualUdpDstIP, (err) => {
```

### Starting the Relay

```bash
ENABLE_WSS=false LOG_LEVEL=1 node relay.js
# Listens on ws://127.0.0.1:8086/
```

## Basic Usage

```javascript
import { V86 } from "v86";
import path from "node:path";

const emulator = new V86({
    wasm_path: path.join(__dirname, "node_modules/v86/build/v86.wasm"),
    bios: { url: path.join(__dirname, "bios/seabios.bin") },
    vga_bios: { url: path.join(__dirname, "bios/vgabios.bin") },
    filesystem: {
        basefs: path.join(__dirname, "images/alpine-fs.json"),
        baseurl: path.join(__dirname, "images/alpine-rootfs-flat/"),
    },
    autostart: true,
    memory_size: 512 * 1024 * 1024,
    bzimage_initrd_from_filesystem: true,
    cmdline: "rw root=host9p rootfstype=9p rootflags=trans=virtio,cache=loose modules=virtio_pci tsc=reliable console=ttyS0",
    net_device: {
        type: "virtio",
        relay_url: "ws://127.0.0.1:8086/",
    },
});

// Capture output
emulator.add_listener("serial0-output-byte", (byte) => {
    process.stdout.write(String.fromCharCode(byte));
});

// Send commands
emulator.serial0_send("echo hello\n");
```

## Communication Methods

### 1. Serial Console (stdin/stdout)

```javascript
// Send command
emulator.serial0_send("ls -la\n");

// Receive output
let output = "";
emulator.add_listener("serial0-output-byte", (byte) => {
    output += String.fromCharCode(byte);
});
```

### 2. 9p Filesystem (file I/O)

```javascript
// Write file to VM
const data = new TextEncoder().encode("#!/bin/sh\necho hello\n");
await emulator.create_file("/tmp/script.sh", data);

// Read file from VM
const result = await emulator.read_file("/tmp/output.txt");
console.log(new TextDecoder().decode(result));
```

### 3. Network (TCP to host services)

From inside the VM, connect to `10.0.2.2:PORT` to reach `localhost:PORT` on the host (requires patched relay).

```bash
# Inside VM
wget http://10.0.2.2:8080/  # Connects to host's localhost:8080
```

## State Save/Restore

```javascript
// Save state (includes all installed packages, files, etc.)
const state = await emulator.save_state();
fs.writeFileSync("vm-state.bin", Buffer.from(state));

// Restore state (instant resume, ~2 seconds)
const stateBuffer = fs.readFileSync("vm-state.bin");
await emulator.restore_state(stateBuffer.buffer);
```

## Network Setup Inside VM

After boot, run these commands to enable networking:

```bash
modprobe virtio-net
ip link set eth0 up
udhcpc -i eth0
```

Or as a one-liner:
```bash
modprobe virtio-net && ip link set eth0 up && udhcpc -i eth0
```

The VM will get IP `10.0.2.15` (or similar) via DHCP from the relay.

## Performance

| Metric | Value |
|--------|-------|
| Cold boot | ~20-25 seconds |
| State restore | ~2-3 seconds |
| Memory usage | ~512 MB (configurable) |

## Typical Workflow for Mom

1. **First run**:
   - Start rootlessRelay
   - Boot v86 with Alpine (~25s)
   - Setup network
   - Install needed packages (`apk add nodejs npm python3 git`)
   - Save state

2. **Subsequent runs**:
   - Start rootlessRelay
   - Restore saved state (~2s)
   - Ready to execute commands

3. **Command execution**:
   - Send commands via `serial0_send()`
   - Capture output via `serial0-output-byte` listener
   - Exchange files via 9p filesystem

## Alternative: fetch Backend (No Relay Needed)

For HTTP-only networking, v86 has a built-in `fetch` backend:

```javascript
net_device: {
    type: "virtio",
    relay_url: "fetch",
}
```

This uses the browser/Node.js `fetch()` API for HTTP requests. Limitations:
- Only HTTP/HTTPS (no raw TCP/UDP)
- No WebSocket
- Host access via `http://<port>.external` (e.g., `http://8080.external`)

## Files Reference

After building, you need these files:

```
project/
├── node_modules/v86/build/
│   ├── v86.wasm
│   └── libv86.mjs
├── bios/
│   ├── seabios.bin
│   └── vgabios.bin
├── images/
│   ├── alpine-fs.json
│   └── alpine-rootfs-flat/
│       └── *.bin.zst (many files)
└── rootlessRelay/
    └── relay.js (patched)
```

## Resources

- [v86 GitHub](https://github.com/copy/v86)
- [v86 Networking Docs](https://github.com/copy/v86/blob/master/docs/networking.md)
- [v86 Alpine Setup](https://github.com/copy/v86/tree/master/tools/docker/alpine)
- [rootlessRelay](https://github.com/obegron/rootlessRelay)
- [v86 npm package](https://www.npmjs.com/package/v86)