diff --git a/.env.agent.example b/.env.agent.example new file mode 100644 index 0000000..b00a369 --- /dev/null +++ b/.env.agent.example @@ -0,0 +1,12 @@ +BETTERNAS_CLONE_NAME=betternas-main +COMPOSE_PROJECT_NAME=betternas-main +BETTERNAS_CONTROL_PLANE_PORT=3001 +BETTERNAS_NODE_AGENT_PORT=3090 +BETTERNAS_NEXTCLOUD_PORT=8080 +BETTERNAS_EXPORT_PATH=.state/betternas-main/export +BETTERNAS_VERSION=local-dev +BETTERNAS_NODE_DIRECT_ADDRESS=http://localhost:${BETTERNAS_NODE_AGENT_PORT} +BETTERNAS_EXAMPLE_MOUNT_URL=http://localhost:${BETTERNAS_NODE_AGENT_PORT}/dav/ +NEXTCLOUD_BASE_URL=http://localhost:${BETTERNAS_NEXTCLOUD_PORT} +NEXTCLOUD_ADMIN_USER=admin +NEXTCLOUD_ADMIN_PASSWORD=admin diff --git a/README.md b/README.md index b306e03..b3fb072 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ - `packages/ui`: shared React UI - `infra/docker`: local Docker runtime -The root planning and delegation guide lives in [skeleton.md](/home/rathi/Documents/GitHub/betterNAS/skeleton.md). +The root planning and delegation guide lives in [skeleton.md](./skeleton.md). ## Verify @@ -24,3 +24,23 @@ Run the repo acceptance loop with: ```bash pnpm verify ``` + +## Agent loop + +Bootstrap a clone-local environment with: + +```bash +pnpm agent:bootstrap +``` + +Run the full static and integration loop with: + +```bash +pnpm agent:verify +``` + +Create or refresh the sibling agent clones with: + +```bash +pnpm clones:setup +``` diff --git a/apps/control-plane/README.md b/apps/control-plane/README.md index 34af4ce..4ded815 100644 --- a/apps/control-plane/README.md +++ b/apps/control-plane/README.md @@ -12,4 +12,4 @@ It is intentionally small for now: - `POST /api/v1/cloud-profiles/issue` The request and response shapes must follow the contracts in -[`packages/contracts`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts). +[`packages/contracts`](../../packages/contracts). diff --git a/apps/node-agent/Dockerfile b/apps/node-agent/Dockerfile new file mode 100644 index 0000000..30d2e97 --- /dev/null +++ b/apps/node-agent/Dockerfile @@ -0,0 +1,15 @@ +FROM golang:1.26-alpine AS build + +WORKDIR /src +COPY apps/node-agent ./apps/node-agent + +WORKDIR /src/apps/node-agent +RUN CGO_ENABLED=0 GOOS=linux go build -o /out/node-agent ./cmd/node-agent + +FROM alpine:3.21 + +WORKDIR /app +COPY --from=build /out/node-agent /usr/local/bin/node-agent + +EXPOSE 8090 +CMD ["node-agent"] diff --git a/control.md b/control.md new file mode 100644 index 0000000..72e9a08 --- /dev/null +++ b/control.md @@ -0,0 +1,45 @@ +# Control + +This clone is the main repo. + +Use it for: + +- shared contracts +- repo guardrails +- runtime scripts +- integration verification +- architecture and coordination + +Planned clone layout: + +```text +/home/rathi/Documents/GitHub/betterNAS/ + betterNAS + betterNAS-runtime + betterNAS-control + betterNAS-node +``` + +Clone roles: + +- `betterNAS` + - main coordination repo + - owns contracts, scripts, and shared verification rules +- `betterNAS-runtime` + - owns Docker Compose, stack env, readiness checks, and end-to-end runtime verification +- `betterNAS-control` + - owns the Go control plane and contract-backed API behavior +- `betterNAS-node` + - owns the node agent, WebDAV serving, and NAS-side registration/export behavior + +Rules: + +- shared interface changes land in `packages/contracts` first +- runtime verification must stay green in the main repo +- feature agents should stay inside their assigned clone unless a contract change is required + +Agent command surface: + +- main repo creates or refreshes sibling clones with `pnpm clones:setup` +- each clone bootstraps itself with `pnpm agent:bootstrap` +- each clone runs the full loop with `pnpm agent:verify` diff --git a/docs/architecture.md b/docs/architecture.md index 724c0c3..74edc8f 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -3,7 +3,7 @@ This file is the canonical contract for the repository. If the planning docs, scaffold code, or future tasks disagree, this file and -[`packages/contracts`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts) +[`packages/contracts`](../packages/contracts) win. ## The single first task @@ -63,26 +63,26 @@ parallelized without interface drift. Use these in this order: -1. [`docs/architecture.md`](/home/rathi/Documents/GitHub/betterNAS/docs/architecture.md) +1. [`docs/architecture.md`](./architecture.md) for boundaries, ownership, and delivery rules -2. [`packages/contracts`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts) +2. [`packages/contracts`](../packages/contracts) for machine-readable types, schemas, and route constants 3. the part docs for local detail: - - [`docs/01-nas-node.md`](/home/rathi/Documents/GitHub/betterNAS/docs/01-nas-node.md) - - [`docs/02-control-plane.md`](/home/rathi/Documents/GitHub/betterNAS/docs/02-control-plane.md) - - [`docs/03-local-device.md`](/home/rathi/Documents/GitHub/betterNAS/docs/03-local-device.md) - - [`docs/04-cloud-web-layer.md`](/home/rathi/Documents/GitHub/betterNAS/docs/04-cloud-web-layer.md) - - [`docs/05-build-plan.md`](/home/rathi/Documents/GitHub/betterNAS/docs/05-build-plan.md) + - [`docs/01-nas-node.md`](./01-nas-node.md) + - [`docs/02-control-plane.md`](./02-control-plane.md) + - [`docs/03-local-device.md`](./03-local-device.md) + - [`docs/04-cloud-web-layer.md`](./04-cloud-web-layer.md) + - [`docs/05-build-plan.md`](./05-build-plan.md) ## Repo lanes The monorepo is split into these primary implementation lanes: -- [`apps/node-agent`](/home/rathi/Documents/GitHub/betterNAS/apps/node-agent) -- [`apps/control-plane`](/home/rathi/Documents/GitHub/betterNAS/apps/control-plane) -- [`apps/web`](/home/rathi/Documents/GitHub/betterNAS/apps/web) -- [`apps/nextcloud-app`](/home/rathi/Documents/GitHub/betterNAS/apps/nextcloud-app) -- [`packages/contracts`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts) +- [`apps/node-agent`](../apps/node-agent) +- [`apps/control-plane`](../apps/control-plane) +- [`apps/web`](../apps/web) +- [`apps/nextcloud-app`](../apps/nextcloud-app) +- [`packages/contracts`](../packages/contracts) Every parallel task should primarily stay inside one of those lanes unless it is an explicit contract task. @@ -132,7 +132,7 @@ Each area gets an owner and a narrow write surface. The only shared write surface across teams should be: -- [`packages/contracts`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts) +- [`packages/contracts`](../packages/contracts) - this file when the architecture contract changes ## Verification loop @@ -165,7 +165,7 @@ The initial scaffold is complete when: 1. No part may invent private request or response shapes for shared flows. 2. Contract changes must update - [`packages/contracts`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts) + [`packages/contracts`](../packages/contracts) first. 3. Architecture changes must update this file in the same change. 4. Additive contract changes are preferred over breaking ones. diff --git a/package.json b/package.json index 4c31a07..8dbac9b 100644 --- a/package.json +++ b/package.json @@ -12,6 +12,8 @@ "stack:up": "./scripts/dev-up", "stack:down": "./scripts/dev-down", "stack:verify": "./scripts/integration/verify-stack", + "agent:bootstrap": "./scripts/agent-bootstrap", + "agent:verify": "./scripts/agent-verify", "clones:setup": "./scripts/setup-clones", "test": "turbo run test", "verify": "pnpm run guardrails && pnpm run format:check && turbo run lint check-types test build" diff --git a/packages/contracts/README.md b/packages/contracts/README.md index b1ee271..55eb2fe 100644 --- a/packages/contracts/README.md +++ b/packages/contracts/README.md @@ -25,20 +25,20 @@ Use it to keep the four product parts aligned: ## Current contract layers -- [`src/control-plane.ts`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts/src/control-plane.ts) +- [`src/control-plane.ts`](./src/control-plane.ts) - current runtime scaffold for health and version -- [`src/foundation.ts`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts/src/foundation.ts) +- [`src/foundation.ts`](./src/foundation.ts) - first product-level entities and route constants for node, mount, and cloud flows -- [`openapi/`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts/openapi) +- [`openapi/`](./openapi) - language-neutral source documents for future SDK generation -- [`schemas/`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts/schemas) +- [`schemas/`](./schemas) - JSON schema mirrors for the first shared entities ## Change rules 1. Shared API shape changes happen here first. 2. If the boundary changes, also update - [`docs/architecture.md`](/home/rathi/Documents/GitHub/betterNAS/docs/architecture.md). + [`docs/architecture.md`](../../docs/architecture.md). 3. Prefer additive changes until all four parts are live. 4. Do not put Nextcloud-only assumptions into the core contracts unless the field is explicitly part of the cloud adapter. diff --git a/scripts/agent-bootstrap b/scripts/agent-bootstrap new file mode 100755 index 0000000..67cb815 --- /dev/null +++ b/scripts/agent-bootstrap @@ -0,0 +1,43 @@ +#!/usr/bin/env bash + +set -euo pipefail + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +env_file="$repo_root/.env.agent" +example_env_file="$repo_root/.env.agent.example" + +if [[ ! -f "$env_file" ]]; then + cp "$example_env_file" "$env_file" +fi + +# shellcheck disable=SC1091 +source "$repo_root/scripts/lib/runtime-env.sh" + +mkdir -p "$BETTERNAS_EXPORT_PATH" + +if [[ ! -f "$BETTERNAS_EXPORT_PATH/README.txt" ]]; then + cat >"$BETTERNAS_EXPORT_PATH/README.txt" </dev/null + +register_response="$( + curl -fsS \ + -X POST \ + -H 'Content-Type: application/json' \ + -d @- \ + "http://localhost:${BETTERNAS_CONTROL_PLANE_PORT}/api/v1/nodes/register" </dev/null + +mount_profile="$( + curl -fsS \ + -X POST \ + -H 'Content-Type: application/json' \ + -d '{"userId":"integration-user","deviceId":"integration-device","exportId":"dev-export"}' \ + "http://localhost:${BETTERNAS_CONTROL_PLANE_PORT}/api/v1/mount-profiles/issue" +)" +echo "$mount_profile" | jq -e --arg expected "$BETTERNAS_EXAMPLE_MOUNT_URL" '.protocol == "webdav" and .mountUrl == $expected' >/dev/null + +cloud_profile="$( + curl -fsS \ + -X POST \ + -H 'Content-Type: application/json' \ + -d '{"userId":"integration-user","exportId":"dev-export","provider":"nextcloud"}' \ + "http://localhost:${BETTERNAS_CONTROL_PLANE_PORT}/api/v1/cloud-profiles/issue" +)" +echo "$cloud_profile" | jq -e --arg expected "$NEXTCLOUD_BASE_URL" '.provider == "nextcloud" and .baseUrl == $expected' >/dev/null + +nextcloud_status="$(curl -fsS "${NEXTCLOUD_BASE_URL}/status.php")" +echo "$nextcloud_status" | jq -e '.installed == true' >/dev/null + +nextcloud_app_status="$( + curl -fsS \ + -u "${NEXTCLOUD_ADMIN_USER}:${NEXTCLOUD_ADMIN_PASSWORD}" \ + -H 'OCS-APIRequest: true' \ + "${NEXTCLOUD_BASE_URL}/ocs/v2.php/apps/betternascontrolplane/api/status" +)" +echo "$nextcloud_app_status" | jq -e '.ocs.meta.statuscode == 100' >/dev/null + +echo "Stack verified for ${BETTERNAS_CLONE_NAME}." diff --git a/scripts/integration/verify-webdav b/scripts/integration/verify-webdav new file mode 100755 index 0000000..70b8355 --- /dev/null +++ b/scripts/integration/verify-webdav @@ -0,0 +1,19 @@ +#!/usr/bin/env bash + +set -euo pipefail + +# shellcheck disable=SC1091 +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/../lib/runtime-env.sh" + +headers="$(mktemp)" +trap 'rm -f "$headers"' EXIT + +curl -fsS -D "$headers" -o /dev/null -X PROPFIND -H 'Depth: 0' "$BETTERNAS_EXAMPLE_MOUNT_URL" + +if ! grep -Eq '^HTTP/[0-9.]+ 207' "$headers"; then + echo "WebDAV PROPFIND did not return 207 for $BETTERNAS_EXAMPLE_MOUNT_URL" >&2 + cat "$headers" >&2 + exit 1 +fi + +echo "WebDAV verified: $BETTERNAS_EXAMPLE_MOUNT_URL" diff --git a/scripts/integration/wait-stack b/scripts/integration/wait-stack new file mode 100755 index 0000000..8376ada --- /dev/null +++ b/scripts/integration/wait-stack @@ -0,0 +1,27 @@ +#!/usr/bin/env bash + +set -euo pipefail + +# shellcheck disable=SC1091 +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/../lib/runtime-env.sh" + +wait_for_url() { + local url="$1" + local label="$2" + local max_attempts="${3:-60}" + + for _ in $(seq 1 "$max_attempts"); do + if curl -fsS "$url" >/dev/null 2>&1; then + echo "$label ready: $url" + return 0 + fi + sleep 2 + done + + echo "$label did not become ready: $url" >&2 + return 1 +} + +wait_for_url "http://localhost:${BETTERNAS_CONTROL_PLANE_PORT}/health" "control plane" +wait_for_url "http://localhost:${BETTERNAS_NODE_AGENT_PORT}/health" "node agent" +wait_for_url "${NEXTCLOUD_BASE_URL}/status.php" "nextcloud" diff --git a/scripts/lib/runtime-env.sh b/scripts/lib/runtime-env.sh new file mode 100755 index 0000000..9b6284b --- /dev/null +++ b/scripts/lib/runtime-env.sh @@ -0,0 +1,58 @@ +#!/usr/bin/env bash + +set -euo pipefail + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +compose_file="$repo_root/infra/docker/compose.dev.yml" +default_env_file="$repo_root/.env.agent" +env_file="${BETTERNAS_ENV_FILE:-$default_env_file}" + +if [[ -f "$env_file" ]]; then + set -a + # shellcheck disable=SC1090 + source "$env_file" + set +a +fi + +: "${BETTERNAS_CLONE_NAME:=betternas-main}" +: "${COMPOSE_PROJECT_NAME:=betternas-${BETTERNAS_CLONE_NAME}}" +: "${BETTERNAS_CONTROL_PLANE_PORT:=3001}" +: "${BETTERNAS_NODE_AGENT_PORT:=3090}" +: "${BETTERNAS_NEXTCLOUD_PORT:=8080}" +: "${BETTERNAS_VERSION:=local-dev}" +: "${NEXTCLOUD_ADMIN_USER:=admin}" +: "${NEXTCLOUD_ADMIN_PASSWORD:=admin}" + +if [[ -z "${BETTERNAS_EXPORT_PATH:-}" ]]; then + BETTERNAS_EXPORT_PATH="$repo_root/.state/$BETTERNAS_CLONE_NAME/export" +fi + +if [[ "$BETTERNAS_EXPORT_PATH" != /* ]]; then + BETTERNAS_EXPORT_PATH="$repo_root/$BETTERNAS_EXPORT_PATH" +fi + +: "${BETTERNAS_NODE_DIRECT_ADDRESS:=http://localhost:${BETTERNAS_NODE_AGENT_PORT}}" +: "${BETTERNAS_EXAMPLE_MOUNT_URL:=http://localhost:${BETTERNAS_NODE_AGENT_PORT}/dav/}" +: "${NEXTCLOUD_BASE_URL:=http://localhost:${BETTERNAS_NEXTCLOUD_PORT}}" + +export repo_root +export compose_file +export env_file +export BETTERNAS_CLONE_NAME +export COMPOSE_PROJECT_NAME +export BETTERNAS_CONTROL_PLANE_PORT +export BETTERNAS_NODE_AGENT_PORT +export BETTERNAS_NEXTCLOUD_PORT +export BETTERNAS_EXPORT_PATH +export BETTERNAS_VERSION +export NEXTCLOUD_ADMIN_USER +export NEXTCLOUD_ADMIN_PASSWORD +export BETTERNAS_NODE_DIRECT_ADDRESS +export BETTERNAS_EXAMPLE_MOUNT_URL +export NEXTCLOUD_BASE_URL + +mkdir -p "$BETTERNAS_EXPORT_PATH" + +compose() { + docker compose -f "$compose_file" "$@" +} diff --git a/scripts/setup-clones b/scripts/setup-clones new file mode 100755 index 0000000..7b04e76 --- /dev/null +++ b/scripts/setup-clones @@ -0,0 +1,51 @@ +#!/usr/bin/env bash + +set -euo pipefail + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +parent_dir="$(cd "$repo_root/.." && pwd)" +repo_name="$(basename "$repo_root")" +sync_clone_script="$repo_root/scripts/sync-clone" + +declare -A clone_ports=( + ["betterNAS-runtime"]="41080 41090 41001" + ["betterNAS-control"]="42080 42090 42001" + ["betterNAS-node"]="43080 43090 43001" +) +clone_names=("betterNAS-runtime" "betterNAS-control" "betterNAS-node") + +for clone_name in "${clone_names[@]}"; do + clone_dir="$parent_dir/$clone_name" + + "$sync_clone_script" "$clone_dir" + + read -r nextcloud_port node_agent_port control_plane_port <<<"${clone_ports[$clone_name]}" + + cat >"$clone_dir/.env.agent" <&2 + exit 1 +fi + +clone_dir="$1" + +if [[ "$clone_dir" != /* ]]; then + echo "clone path must be absolute: $clone_dir" >&2 + exit 1 +fi + +if [[ ! -d "$clone_dir/.git" ]]; then + git clone --local --no-hardlinks "$repo_root" "$clone_dir" +fi + +find "$clone_dir" -mindepth 1 -maxdepth 1 \ + ! -name .git \ + ! -name .env.agent \ + -exec rm -rf {} + + +tar \ + --exclude=.git \ + --exclude=.state \ + --exclude=.turbo \ + --exclude=node_modules \ + --exclude=dist \ + --exclude='apps/web/.next' \ + -C "$repo_root" \ + -cf - \ + . | tar -C "$clone_dir" -xf - + +echo "Synced working tree into $clone_dir" diff --git a/skeleton.md b/skeleton.md index a141c94..35a2e58 100644 --- a/skeleton.md +++ b/skeleton.md @@ -35,8 +35,8 @@ betterNAS/ ## Runtime and language choices -| Part | Language | Why | -| ------------------ | ---------------------------------- | -------------------------------------------------------------------- | +| Part | Language | Why | +| -------------------- | ---------------------------------- | -------------------------------------------------------------------- | | `apps/web` | TypeScript + Next.js | best UI velocity, best admin/control-plane UX | | `apps/control-plane` | Go | strong concurrency, static binaries, operationally simple | | `apps/node-agent` | Go | best fit for host runtime, WebDAV service, and future Nix deployment | @@ -47,10 +47,10 @@ betterNAS/ The source of truth for shared interfaces is: -1. [`docs/architecture.md`](/home/rathi/Documents/GitHub/betterNAS/docs/architecture.md) -2. [`packages/contracts/openapi/betternas.v1.yaml`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts/openapi/betternas.v1.yaml) -3. [`packages/contracts/schemas`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts/schemas) -4. [`packages/contracts/src`](/home/rathi/Documents/GitHub/betterNAS/packages/contracts/src) +1. [`docs/architecture.md`](./docs/architecture.md) +2. [`packages/contracts/openapi/betternas.v1.yaml`](./packages/contracts/openapi/betternas.v1.yaml) +3. [`packages/contracts/schemas`](./packages/contracts/schemas) +4. [`packages/contracts/src`](./packages/contracts/src) Agents must not invent private shared request or response shapes outside those locations.