From b65796c727289f861dce6123629ac25108abe9ad Mon Sep 17 00:00:00 2001 From: pcnofelt Date: Sun, 26 Jul 2026 21:52:17 -0700 Subject: [PATCH 1/5] docs: rewrite README around a Linux-first setup path The README assumed contributors would figure out the Linux requirement on their own, which left macOS and Windows users guessing at a workflow that only works in Linux. Restructure it around that constraint: - Lead with what Intermesh is before jumping into contributor setup - State the Linux requirement up front and give a container path for macOS/Windows (mount the repo and the Docker socket) - Add a prompt-driven setup section for assistant-guided installs, covering the non-obvious bits: BuildKit for the e2e image build, docker CLI and compose inside the dev container, and a UTF-8 locale for the Ruby lints - Set time expectations on the first cargo local run so long silent stretches are not mistaken for a hang - Collect the recurring setup failures into Common Gotchas --- README.md | 241 +++++++++++++++++++++++++++++++++--------------------- 1 file changed, 149 insertions(+), 92 deletions(-) diff --git a/README.md b/README.md index c77ee7d..9933b60 100644 --- a/README.md +++ b/README.md @@ -1,136 +1,193 @@ # intermesh -## Contributing +Intermesh is a trust-based identity mesh written in Rust. Nodes +exchange signed control statements about names, addresses, and policy. Each +node evaluates that state locally to build its own view of the mesh. It is a +different control model from centrally managed meshes, and in practice it gives +you explicit, auditable trust decisions at each node. + +Intermesh runs on Linux. Build and test on Linux to match production and CI. + +## Getting Started + +### Platform + +- Linux is required for Intermesh build and test workflows. +- Use a Linux host directly (bare metal, VM, or cloud instance), or use + Docker Desktop/Colima and run your workflow in a Linux container (mount the + repo and the Docker socket). + +### Prompt-Driven Setup (especially useful on macOS/Windows) + +If you are on macOS or Windows and want an assistant to drive setup, paste this +into Claude Code (or similar). + +Budget 10-20 minutes (sometimes a touch longer) for a first run. Most of that +is unattended downloads and a cold Rust build. Several steps print nothing for +minutes at a time; that is normal, not a hang. + +```text +I need a Linux-based Intermesh dev environment on this machine. +Create the smallest reliable setup and explain each step briefly. + +Requirements: +- Intermesh tooling must run in Linux +- Docker daemon must be available +- Read this Intermesh README (where this prompt came from) first and treat it + as the source of truth +- Repo is already cloned at (ask user if no path was provided) +- Prefer manual tool install in Linux (rustup + protoc + cmake + ruby + + pkg-config + musl-tools + git + build-essential) +- If using a container: mount the repo and the Docker socket, and set + DOCKER_BUILDKIT=1 (the e2e image build requires BuildKit) +- If using a container: ensure docker CLI, buildx, and docker compose are + installed in that container +- Set LANG=C.UTF-8 and LC_ALL=C.UTF-8 before running cargo local +- Nix is optional, not required + +Please: +1) Detect host OS and choose a Linux execution path (native Linux, or Linux container/VM). +2) Give exact commands to set up dependencies. +3) Run verification commands and show pass/fail: + - docker info + - cargo --version + - cargo local +4) Warn me before any step that takes more than a few minutes, and say how + long it should take. The first cargo local runs 10-20 minutes and goes + silent for long stretches; do not report it as hung or kill it. +5) If a step fails, show likely cause and next command to run. +6) End with a short checklist of what is done vs what is still needed. +``` ### Docker -You need Docker installed and running. On Linux, install it via your -distribution's package manager. On macOS, you can use -[Docker Desktop](https://www.docker.com/products/docker-desktop/) or -[Colima](https://github.com/abiosoft/colima). Development on macOS works -but is not as heavily tested as Linux — if you hit mac-specific issues, -ask in Slack. +Docker is required for e2e tests. Install it with your distro package manager +on Linux, then confirm: -The e2e tests use a shared Nix-built Docker base image. xtask builds a static -musl intermesh binary for the host architecture and copies it into a thin image -layered on that base, so Rust code changes do not invalidate the slower tools -layer. +```bash +docker info +``` -### Build Dependencies +## Setup -#### Option 1: Nix + direnv (Recommended) +### Option 1: Manual Setup (recommended) -[Nix](https://nixos.org/) is a package manager focused on reproducible -builds that works on Linux and macOS. You don't need to understand it — -it will just make sure you get all the right dependencies at the right -versions, guaranteed to be compatible with each other. If you're curious, -[Zero to Nix](https://zero-to-nix.com/) is a good introduction. +If you want a straightforward, roll-your-own setup, install: -- Install Nix — the [Determinate Systems installer](https://github.com/DeterminateSystems/nix-installer) - is the easiest option, though some prefer the - [official installer](https://nixos.org/download/) -- Install [direnv](https://direnv.net/) and hook it into your shell - (bonus points if you use `nix profile install nixpkgs#direnv`) +- `rustup` (then run `rustup toolchain install` in this repo) +- `protoc` +- `cmake` +- `ruby` +- `pkg-config` +- `musl-tools` +- `git` +- a C toolchain (`build-essential` on Debian/Ubuntu) +- `flock` (usually provided by `util-linux`) -Once both are installed, `cd` into the repo and run `direnv allow`. From -that point on, every time you enter the directory the dev shell provides -system build tools plus the Rust toolchain selected by -`rust-toolchain.toml`, including clippy, rustfmt, rust-analyzer, and the -musl target used by e2e tests. If the Rust version changes, re-enter the -directory or run `direnv reload`. +`rust-toolchain.toml` pins Rust, clippy, rustfmt, rust-analyzer, and targets. -#### Option 2: Manual Installation +### Option 2: Nix + direnv -If you'd rather manage dependencies yourself, you'll need: +Nix gives a reproducible dev shell with the exact toolchain and versions: -- [rustup](https://rustup.rs/) - - After installing it, run `rustup toolchain install` from the repo root - - Rustup reads `rust-toolchain.toml`, which selects Rust, clippy, - rustfmt, rust-analyzer, and the musl target -- protobuf compiler (`protoc`) -- cmake -- Ruby (for the custom linter) -- pkg-config +- Install [Nix](https://nixos.org/) (the + [Determinate installer](https://github.com/DeterminateSystems/nix-installer) + is the easiest path) +- Install [direnv](https://direnv.net/) and hook it into your shell +- From repo root: `direnv allow` -This is mostly standard Rust ecosystem tooling. You're on your own for -version management, but it shouldn't be too hard. +If you'd rather not manage tool versions yourself, use this. -### Verify Your Setup +## Verify Your Setup -A good first goal is to get `cargo local` to pass: +Run the full local check: ```bash cargo local ``` -This runs formatting, linting, a full build, unit tests, and e2e tests. -If it passes, you're mostly good to go. +This runs formatting, lint, build, unit tests, and e2e tests. -## Development Commands +Expect 10-20 minutes the first time. The cold Rust build and the e2e image +build take most of the time, and both go quiet for minutes at a time. Later +runs are much faster once the build cache and the e2e base image are warm. -- `cargo build` — Build the project -- `cargo test [filter]` — Run unit tests (use the filter to run a specific - test while iterating, e.g. `cargo test my_test_name`) -- `cargo lint` — Run format check, clippy, and custom lints -- `cargo local` — Lint, build, unit tests, and e2e tests -- `cargo all` — Everything in `local` plus VM tests when Incus is available -- `cargo deps` — Check for outdated/unused/vulnerable dependencies +## Core Commands + +- `cargo build` - build the project +- `cargo test [filter]` - run unit tests +- `cargo lint` - run format check, clippy, and custom lints +- `cargo local` - lint + build + unit tests + e2e tests +- `cargo all` - `local` plus perf tests and VM tests when Incus is available +- `cargo deps` - dependency freshness and vulnerability checks ## E2E Tests ```bash -cargo test-e2e # run all e2e tests -cargo test-e2e test_name # run a specific test +cargo test-e2e +cargo test-e2e test_name ``` -The e2e tests spin up Docker containers on a local bridge network. Each -container uses the shared Nix-based test image plus the intermesh binary -built from your working tree. The only requirement beyond the build -dependencies is a running Docker daemon. - -## VM Tests +E2E tests run Docker containers on a local bridge network with binaries built +from your working tree. -The VM tests exercise intermesh across isolated local Incus guests. +If you run inside a dev container, install `docker` CLI + buildx + compose in +that container so commands like `cargo test-e2e` and `docker compose` work. -### Setup +## VM Tests -You do not need Incus for normal local development. The VM integration tests -run in CI, and most contributors are not expected to have Incus set up locally. -You only need a working local Incus installation if you want to run those tests -before pushing. The Nix dev shell includes the `incus` CLI, but it does not -install or configure the host Incus daemon for you. +VM tests run intermesh across isolated local [Incus](https://linuxcontainers.org/incus/) +VMs. They run in CI, so most contributors never need a local Incus setup; +reach for them when you want guest-level network realism before pushing. -The harness does not require a NixOS-specific Incus setup. It needs a local -Incus daemon that your user can access, VM support enabled, and a default -profile with a root disk and network. The host must be able to launch the -Ubuntu 24.04 cloud VM image and the guest must have outbound network access -while the cached image is built: +Quick commands: ```bash -incus image info images:ubuntu/24.04/cloud --vm -incus launch images:ubuntu/24.04/cloud intermesh-incus-check --vm --ephemeral -incus delete --force intermesh-incus-check +cargo xtask vm-image +cargo test-vm +cargo test-vm test_name +cargo xtask clean-vm-images ``` -The VM image build and tests currently request 4 CPUs and 4 GiB of memory per -VM. No specific storage driver is required; our NixOS machines use a ZFS-backed -default pool, but any Incus storage pool that can run VMs should work. +`cargo all` attempts VM tests and skips them when Incus is unavailable. -`cargo test-vm` runs the VM integration tests in ephemeral Incus VMs, building -a reusable local test image on first use. +## Local Mesh Demo -### Running +The repo ships a small local mesh demo in `sandbox/`: ```bash -cargo xtask vm-image # ensure the cached VM image exists and print its alias -cargo test-vm # run all VM tests; fails if Incus is unavailable -cargo test-vm test_name # run a specific test -cargo all # run everything; skips VM tests if Incus is unavailable -cargo xtask clean-vm-images # drop cached local Intermesh VM images +cd sandbox && docker compose up -d ``` -Unlike e2e tests, the smoke test runs `intermesh` directly on the guest host -so it can operate on the VM's real network interfaces. Test workloads run in -that guest's local Docker daemon. `cargo all` attempts the VM suite but skips -it when Incus is unavailable; use `cargo test-vm` when VM coverage must be -enforced. +See `sandbox/README.md` for what to try once it's up. + +## Day-to-Day Workflow + +A tight loop that works well: + +1. `cargo test [filter]` +2. `cargo lint` +3. `cargo test-e2e [filter]` +4. `cargo local` before push + +## Common Gotchas + +- **Docker daemon not reachable:** start Docker and re-run `docker info` +- **Lint fails with `invalid byte sequence in US-ASCII`:** set + `LANG=C.UTF-8 LC_ALL=C.UTF-8` +- **First e2e run is slow:** expected; the e2e base image and build cache have + to warm up. Long silent stretches are normal, so let it finish +- **VM tests skipped:** expected without a local Incus setup +- **Using a Linux dev container on macOS/Windows:** set + `CARGO_TARGET_DIR` to a container-local path to avoid host/guest target dir + conflicts (for example `export CARGO_TARGET_DIR=/tmp/intermesh-target`) + +Stuck on something platform-specific? File a GitHub issue: + + +## Next Steps + +1. Get `cargo local` passing +2. Run `cargo test-e2e` once to warm the environment +3. Bring up the `sandbox/` demo mesh +4. Pick a small issue and follow the day-to-day loop above From 78c89220a555e34f70b7a6a0e104ffdc41b700f9 Mon Sep 17 00:00:00 2001 From: pcnofelt Date: Sun, 26 Jul 2026 21:59:37 -0700 Subject: [PATCH 2/5] docs: annotate README command blocks Restore the inline comments the old README had on the e2e and VM command blocks, and spell out that cargo test takes a name filter. --- README.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 9933b60..53e41b7 100644 --- a/README.md +++ b/README.md @@ -115,7 +115,8 @@ runs are much faster once the build cache and the e2e base image are warm. ## Core Commands - `cargo build` - build the project -- `cargo test [filter]` - run unit tests +- `cargo test [filter]` - run unit tests; pass a filter to run just the tests + whose name matches it, e.g. `cargo test my_test_name` - `cargo lint` - run format check, clippy, and custom lints - `cargo local` - lint + build + unit tests + e2e tests - `cargo all` - `local` plus perf tests and VM tests when Incus is available @@ -124,8 +125,8 @@ runs are much faster once the build cache and the e2e base image are warm. ## E2E Tests ```bash -cargo test-e2e -cargo test-e2e test_name +cargo test-e2e # run all e2e tests +cargo test-e2e test_name # run a specific test ``` E2E tests run Docker containers on a local bridge network with binaries built @@ -143,10 +144,10 @@ reach for them when you want guest-level network realism before pushing. Quick commands: ```bash -cargo xtask vm-image -cargo test-vm -cargo test-vm test_name -cargo xtask clean-vm-images +cargo xtask vm-image # ensure the cached VM image exists, print its alias +cargo test-vm # run all VM tests; fails if Incus is unavailable +cargo test-vm test_name # run a specific test +cargo xtask clean-vm-images # drop cached local Intermesh VM images ``` `cargo all` attempts VM tests and skips them when Incus is unavailable. From 3415a4d3b5f3790b735c15f1c61938c8521dd29c Mon Sep 17 00:00:00 2001 From: pcnofelt Date: Sun, 26 Jul 2026 22:04:43 -0700 Subject: [PATCH 3/5] docs: clarify distro package names and sandbox first run Note that package names vary by distro, flag that the first sandbox bring-up is not instant, and reword the opening paragraph. --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 53e41b7..790ef16 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ Intermesh is a trust-based identity mesh written in Rust. Nodes exchange signed control statements about names, addresses, and policy. Each -node evaluates that state locally to build its own view of the mesh. It is a +node evaluates that state locally to build its own view of the mesh. This uses a different control model from centrally managed meshes, and in practice it gives you explicit, auditable trust decisions at each node. @@ -74,6 +74,8 @@ docker info If you want a straightforward, roll-your-own setup, install: +Package names vary by distro; `build-essential` is the Debian/Ubuntu name. + - `rustup` (then run `rustup toolchain install` in this repo) - `protoc` - `cmake` @@ -157,6 +159,7 @@ cargo xtask clean-vm-images # drop cached local Intermesh VM images The repo ships a small local mesh demo in `sandbox/`: ```bash +# first run takes time cd sandbox && docker compose up -d ``` From 2927a16d2525cd8ba6cbdbfd463e77e54ae3bca5 Mon Sep 17 00:00:00 2001 From: pcnofelt Date: Sun, 26 Jul 2026 22:07:23 -0700 Subject: [PATCH 4/5] docs: drop redundant distro package note --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index 790ef16..ebd537b 100644 --- a/README.md +++ b/README.md @@ -74,8 +74,6 @@ docker info If you want a straightforward, roll-your-own setup, install: -Package names vary by distro; `build-essential` is the Debian/Ubuntu name. - - `rustup` (then run `rustup toolchain install` in this repo) - `protoc` - `cmake` From d22917aa3a32313f5aa239788d729dacfd560f82 Mon Sep 17 00:00:00 2001 From: pcnofelt Date: Sun, 26 Jul 2026 22:57:26 -0700 Subject: [PATCH 5/5] docs: restore Incus preflight checks and tighten intro Bring back the host Incus sanity commands from the old README so contributors can confirm their daemon can launch VMs before running the suite. --- README.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index ebd537b..5a15ba6 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,9 @@ # intermesh -Intermesh is a trust-based identity mesh written in Rust. Nodes -exchange signed control statements about names, addresses, and policy. Each -node evaluates that state locally to build its own view of the mesh. This uses a -different control model from centrally managed meshes, and in practice it gives -you explicit, auditable trust decisions at each node. +Intermesh is a trust-based identity mesh written in Rust. Nodes exchange signed +control statements about names, addresses, and policy. Each node evaluates that +state locally to build its own view of the mesh. By bringing control to the +node, Intermesh distributes responsibility for trust across the network. Intermesh runs on Linux. Build and test on Linux to match production and CI. @@ -141,6 +140,16 @@ VM tests run intermesh across isolated local [Incus](https://linuxcontainers.org VMs. They run in CI, so most contributors never need a local Incus setup; reach for them when you want guest-level network realism before pushing. +You only need this if you plan to run VM tests locally. The Nix shell includes +the `incus` CLI, but it does not install or configure the host Incus daemon. +Your host should be able to launch the Ubuntu 24.04 cloud VM image: + +```bash +incus image info images:ubuntu/24.04/cloud --vm +incus launch images:ubuntu/24.04/cloud intermesh-incus-check --vm --ephemeral +incus delete --force intermesh-incus-check +``` + Quick commands: ```bash @@ -161,7 +170,7 @@ The repo ships a small local mesh demo in `sandbox/`: cd sandbox && docker compose up -d ``` -See `sandbox/README.md` for what to try once it's up. +See `sandbox/README.md` for what to try once it's up and running. ## Day-to-Day Workflow