diff --git a/specs/contract-deployment-registry/contracts/registry-api.md b/specs/contract-deployment-registry/contracts/registry-api.md new file mode 100644 index 0000000..b17d2c0 --- /dev/null +++ b/specs/contract-deployment-registry/contracts/registry-api.md @@ -0,0 +1,148 @@ +# Contract: the registry package's exported surface + +Phase 1 output. These are the signatures the generator and the harness call. They +come before implementation, per the constitution's second principle. + +## Loading + +```go +// Load returns the registry the binary carries, with any supplied files layered +// over it. A supplied file naming a chain the binary carries replaces that +// chain's entry, and Load records which source won (CDR-018, CDR-019, CDR-020). +func Load(paths ...string) (*Registry, error) + +// Sources reports which file supplied each chain, for the run to log. +func (r *Registry) Sources() map[string]string +``` + +`Load()` with no paths returns the embedded registry alone. That is the long-lived +chain case, and it touches no disk and no network. + +## Lookup + +```go +// Chain returns the entry matching both the chain id and the genesis hash +// (CDR-015). It reports false when no entry matches either. +func (r *Registry) Chain(chainID int64, genesisHash string) (Chain, bool) + +// Contract returns one named contract. It reports false when the chain carries +// no entry for that name, which is the deploy case (CDR-003). +func (c Chain) Contract(name string) (Contract, bool) +``` + +Two returns rather than an error. A missing chain and a missing contract are both +ordinary, and both lead to a deployment rather than a failure. + +## Verification + +```go +// CodeReader is the one chain call this package makes. *ethclient.Client and +// bind.ContractBackend both satisfy it, and a test supplies a fake. +type CodeReader interface { + CodeAt(ctx context.Context, account common.Address, block *big.Int) ([]byte, error) +} + +// Verify checks that the code at the recorded address hashes to the recorded +// hash (CDR-006, CDR-007). It returns a *MismatchError naming the chain, the +// contract, the address, and both hashes (CDR-008). It never deploys (CDR-009). +func Verify(ctx context.Context, code CodeReader, chain Chain, name string) error + +// MismatchError is the failure CDR-008 describes. It is a distinct type so a +// caller can tell a stale registry from a dial failure. +type MismatchError struct { + ChainName string + ChainID int64 + ContractName string + Address common.Address + Want common.Hash // recorded at deployment + Got common.Hash // observed now; the zero hash means no code +} + +func (e *MismatchError) Error() string +``` + +`Got` holds the zero hash for the absent-code case. A caller therefore needs one +error type, not two, and the message tells the two apart. + +## Recording + +```go +// Record reads the code at a freshly deployed address and returns the entry to +// write (CDR-010). It does not write anything. +func Record(ctx context.Context, code CodeReader, name string, + addr common.Address) (Contract, error) + +// WriteChain writes a chain file for an operator to review and commit. It fails +// if path names a file inside the embedded chains directory (CDR-011). +func WriteChain(path string, chain Chain) error +``` + +`WriteChain` refusing to write into `registry/chains/` is what makes CDR-011 +enforceable rather than a convention. Pointing a run at the committed registry +still does not let it edit that registry. + +## What the generator calls + +`Resolve` answers one question: does this contract already exist on this chain? + +```go +// Resolve returns the address for a named contract, and reports whether the +// caller needs to deploy. It verifies first (CDR-006, CDR-007). +func Resolve(ctx context.Context, code CodeReader, r *Registry, + chainID int64, genesisHash, name string) (common.Address, bool, error) +``` + +It returns `(addr, false, nil)` to bind, `(zero, true, nil)` to deploy, and an +error when a recorded address failed verification. A profile forcing a deployment +(CDR-016) does not call it. + +`Resolve` is the registry's whole contribution. Binding is the caller's, and the +caller does it once for every contract the profile needs, before load generation +starts. + +## The seam load generation sees + +A scenario receives a bound contract and never an address (CDR-021). The +hand-off already exists on `ContractDeployer[T]`: + +```go +GetBindFunc() ContractBindFunc[T] // the caller uses this to bind +SetContract(contract *T) // the caller hands the instance over +``` + +The preparation step lives outside `registry` and outside the scenario. It: + +1. reads the profile and collects the contracts its scenarios need (CDR-022), +2. calls `Resolve` for each, and deploys where the registry has nothing, +3. binds each result with one client (CDR-023), which is not the client that + sends load, +4. calls `SetContract` on each scenario. + +`AttachScenario` goes away. It takes an address, dials its own client, binds, and +panics on failure. This step now owns those three concerns, and turns that +panic into an error. + +A scenario is then a struct holding its bound contracts and its config. It shapes +transactions and knows nothing about where they came from. + +The binding client is not on the load path. `CreateTransactionOpts` sets +`auth.NoSend`, so a bound contract hands the transaction back rather than sending +it, and the sender's own per-endpoint clients do the sending. One client for +binding and code reads is therefore not a throughput concern. + +## The boundary + +```go +// registry imports, in full: +// context, encoding/json, embed, fmt, math/big, os, path/filepath +// github.com/ethereum/go-ethereum/common +// github.com/ethereum/go-ethereum/crypto +// github.com/sei-protocol/sei-load/generator/bindings +``` + +CDR-017 asserts this with `go list -deps`. `generator/bindings` is the one +sei-load import, and it is itself a leaf. + +Note what is absent: no `config`, no `types`, no `generator`, no `sender`, no +`stats`, and no `ethclient`. `Resolve` takes a `CodeReader`, so the caller owns +the client. diff --git a/specs/contract-deployment-registry/data-model.md b/specs/contract-deployment-registry/data-model.md new file mode 100644 index 0000000..4fae0da --- /dev/null +++ b/specs/contract-deployment-registry/data-model.md @@ -0,0 +1,123 @@ +# Data model: the chain file and the registry types + +Phase 1 output. The file format is the one-way door in this feature: once a +committed file exists, changing its shape is a migration. It comes first, and it +needs sign-off before code depends on it. + +## The chain file + +One file per chain. Files the binary knows live in `registry/chains/` and compile +in. A deployment supplies an extra file by path. + +```json +{ + "chainId": 713715, + "chainName": "arctic-1", + "genesisHash": "3f1a…64 hex chars…9c", + "genesisS3URI": "s3://prod-sei-k8s-genesis-artifacts/arctic-1/genesis.json", + "contracts": [ + { + "name": "storagerw", + "address": "0x1234567890123456789012345678901234567890", + "codeHash": "0xabcd…" + } + ] +} +``` + +### Field by field + +| Field | Type | Required | Why it is here | +| -- | -- | -- | -- | +| `chainId` | number | yes | The EVM chain id. Half the identity, and what a transaction signs against. | +| `chainName` | string | yes | For a human and for an error message. Never matched on. | +| `genesisHash` | string | yes | The other half of the identity. Bare hex, no algorithm prefix, matching `SeiNetwork.Status.GenesisHash`. | +| `genesisS3URI` | string | no | Provenance. Where the genesis came from. Never matched on. | +| `contracts` | array | yes | The contracts on this chain, each named. An empty array means the chain carries none yet. | +| `contracts[].name` | string | yes | The contract's name, which a scenario asks for. | +| `contracts[].address` | string | yes | The deployed address. | +| `contracts[].codeHash` | string | yes | Keccak-256 of the runtime code, observed at deployment. | + +### Three decisions inside the format + +**`chainName` and `genesisS3URI` are never matched on.** CDR-015 matches on +`chainId` and `genesisHash`. Both other fields exist for a human reading a +failure. Recording that here stops a later reader from treating either as a key. + +**`codeHash` is Keccak-256, not SHA-256.** The EVM already defines an account's +code hash as Keccak-256 of its runtime code. A reader can therefore check this +value against chain state, not only against `eth_getCode`. `genesisHash` stays +SHA-256: the controller defines it, and this file does not redefine it. Two +hashes in one file use two algorithms, which is worth stating rather than +discovering. + +**`contracts` is a list, not a map keyed by scenario.** CDR-005. A scenario is a +list of contracts, and most files hold a list of one today. The shape does not +change when TokenOps needs three. + +## Go types + +In package `registry`. + +```go +// Contract is one named contract on one chain. A scenario is a list of these, +// and a list of one is ordinary. +type Contract struct { + Name string `json:"name"` + Address common.Address `json:"address"` + CodeHash common.Hash `json:"codeHash"` +} + +// Chain is one chain and the contracts deployed on it. +type Chain struct { + ChainID int64 `json:"chainId"` + ChainName string `json:"chainName"` + GenesisHash string `json:"genesisHash"` + GenesisS3URI string `json:"genesisS3URI,omitempty"` + Contracts []Contract `json:"contracts"` +} + +// Registry holds every chain the binary carries, plus any the deployment +// supplied. +type Registry struct { + chains map[chainKey]Chain +} + +// chainKey is what CDR-015 matches on, and nothing else. +type chainKey struct { + chainID int64 + genesisHash string +} +``` + +`common.Address` and `common.Hash` already marshal to and from hex strings, so +the JSON shape above needs no custom marshaller. + +## What the types deliberately do not hold + +**No ABI, and no bytecode.** Those live in `generator/bindings`, which the +registry imports. Duplicating them in a chain file would let the file and the +binding disagree, and CDR-007 exists to detect exactly that disagreement. + +**No deployer key, and no endpoint.** A chain file describes a chain, not how to +reach it. The run already has endpoints from its profile. + +**No timestamps, and no provenance beyond the S3 URI.** Git holds when an entry +landed and who committed it. + +## Where deployment stays + +The registry looks up, verifies, and records. It does not deploy. + +`ContractScenarioBase.Deploy` already deploys, and moving it would mean moving +`ContractDeployer[T]` and its four implementors into the registry package. That +is a refactor this feature does not need: CDR-003 deploys when no entry exists, +and the scenario already knows how. + +The flow: a preparation step asks the registry for each contract the profile needs. +It deploys where the registry has nothing. It binds every result with one client, +and hands each scenario its bound contracts. The scenario never sees an address +(CDR-021). + +Un-defer when an operator needs to deploy without starting a run. The spec's +Deferred section already names that as the deploy subcommand. diff --git a/specs/contract-deployment-registry/plan.md b/specs/contract-deployment-registry/plan.md new file mode 100644 index 0000000..bd6f0d3 --- /dev/null +++ b/specs/contract-deployment-registry/plan.md @@ -0,0 +1,157 @@ +# Implementation Plan: Contract deployment and the contract registry + +**Branch**: `contract-deployment-registry` | **Date**: 2026-08-22 | **Spec**: `specs/contract-deployment-registry/spec.md` + +**Input**: Feature specification from `specs/contract-deployment-registry/spec.md` + +## Summary + +A run gets a contract's address from a registry compiled into the binary, or from +a chain file the deployment supplies. It deploys only when neither holds one. It +verifies the code at a recorded address before it sends load, and fails rather +than redeploying on a mismatch. + +The registry lives in its own package. It imports no sei-load package other than +the contract bindings. That boundary is the plan's spine. The nightly harness +that creates a chain and deploys contracts cares nothing for offered rate, +scenarios, or metrics. + +## Technical Context + +**Language/Version**: Go 1.25.1, per `go.mod` and the CI matrix. + +**Primary Dependencies**: `go-ethereum`, for `abi`, `bind`, `ethclient` and +`common.Address`. `generator/bindings`, for bytecode and bind functions. No new +module. + +**Storage**: JSON files in the repo, compiled in with `embed`. One file per +chain. A deployment supplies an extra file by path. + +**Testing**: `go test`, `testify/require`, and the repo's mock chain. A package +boundary test using `go list -deps`. + +**Target Platform**: Linux container, run as a Kubernetes Deployment or Job, and +run locally. + +**Project Type**: Single Go module, command plus packages. + +**Performance Goals**: Startup only. Verification costs one `eth_getCode` per +contract, not per account or transaction. + +**Constraints**: The registry package imports no sei-load package other than the +bindings. No Kubernetes API. No S3 fetch. No network call for chain identity. + +**Scale/Scope**: Six contracts exist. Four long-lived chains, plus one ephemeral +chain per nightly run. A scenario holds one contract today. + +## Constitution Check + +*GATE: passed before Phase 0. Re-checked after Phase 1.* + +| Principle | Assessment | +|---|---| +| I. Verify before claiming done | Every requirement has a named test in the spec's Verification section. The boundary in CDR-017 is machine-checkable with `go list -deps`. Each new assertion gets a deliberate break before it counts. | +| II. Interfaces first, two-way doors | Phase 1 delivers the exact types and signatures before any task. **One one-way door: the registry file format.** Once a committed file exists, changing its shape is a migration. It needs sign-off at Phase 1, not at implementation. | +| III. Idiomatic Go, package pattern wins | The registry is a leaf package, which `config` and `generator/bindings` already are. It reuses `ContractBindFunc[T]`, the repo's existing shape, rather than inventing a loader. | +| IV. Smallest slice | The spec's Deferred section names four cuts with un-defer conditions. Phase 1 adds one more: this plan does not change `ContractDeployer[T]`. | +| V. Simplified Technical English | `vale specs/` gates it, and reports no errors and no warnings. | + +**No violations to justify.** Complexity Tracking stays empty. + +### The gate that needed a decision + +`ContractDeployer[T]` is generic over one contract type, so a scenario holds one +address. CDR-005 keys the registry by contract role, which implies more than one. + +This plan does **not** change that interface. The registry stores a role-keyed +map because changing the file format later is the one-way door, and the interface +is not. A scenario with one contract reads the single role. When TokenOps needs +three, it changes `ContractDeployer[T]` and finds the registry already shaped for +it. + +Recorded here so a reader sees the choice rather than inferring it from the code. + +## Project Structure + +### Documentation (this feature) + +```text +specs/contract-deployment-registry/ +├── spec.md # 20 requirements, CDR-001 to CDR-020 +├── plan.md # This file +├── research.md # Phase 0: the four unknowns and how they resolved +├── data-model.md # Phase 1: the file format and the Go types +├── contracts/ # Phase 1: the package's exported surface +├── quickstart.md # Phase 1: adding a chain, and running against one +└── tasks.md # Phase 2, from /speckit-tasks +``` + +### Source code + +```text +registry/ # new leaf package, CDR-017 +├── registry.go # Registry, Lookup, the embedded files +├── chains/ # one JSON file per known chain, embedded +│ ├── arctic-1.json +│ ├── atlantic-2.json +│ └── pacific-1.json +├── deploy.go # deploying a contract and reporting the entry +├── verify.go # the code check, CDR-006 to CDR-008 +├── registry_test.go +└── boundary_test.go # CDR-017, via go list -deps + +config/config.go # Scenario gains the force-deploy flag, CDR-016 +generator/generator.go # deployAll consults the registry before deploying +main.go # a flag for the supplied chain file, CDR-019 +``` + +**Structure Decision**: a new leaf package `registry/`, beside `config/` rather +than under `generator/`. Placing it under `generator/` would put it inside the +tree it is forbidden to depend on, and would hide the boundary. The +chain files sit inside the package so `embed` reaches them. + +## Phase 0: research + +Four unknowns. `research.md` records each, with what settled it. + +1. What a chain file holds, and how a run identifies its chain. Settled by the + clarification session: chain id and genesis hash, with the S3 URI as + provenance. +2. How to get a runtime code hash that a later run reproduces. `Bin` is + creation bytecode, so the registry records what `eth_getCode` returned at + deployment. +3. Whether `embed` reaches a subdirectory of the package. It does, with + `//go:embed chains/*.json`. +4. What the harness fills a chain file from. `SeiNetwork.Status.GenesisHash` and + `GenesisS3URI`, both already published by the controller. + +None remains open. No `NEEDS CLARIFICATION` markers survive into Phase 1. + +## Phase 1: design + +`data-model.md` fixes the file format and the Go types. `contracts/` fixes the +package's exported surface, which is what the generator and the harness both +call. `quickstart.md` states how an operator adds a chain, and how a deployment +supplies one. + +The order matters. The file format is the one-way door, so it gets written and +signed off before any code depends on it. + +## Post-design Constitution re-check + +Run at the end of Phase 1. One gate does not clear on its own. + +| Principle | Post-design | +|---|---| +| I. Verify before claiming done | Holds. `contracts/registry-api.md` gives `MismatchError` a named type, so a test asserts the failure rather than matching a string. | +| II. Interfaces first, two-way doors | **Signatures delivered. The one-way door is now concrete and needs sign-off.** The chain file format in `data-model.md` becomes a migration the moment a committed file exists. Tasks MUST NOT start before an operator approves it. | +| III. Idiomatic Go, package pattern wins | Holds. `Chain`/`Contract` return two values because a miss is ordinary, matching the repo's map-lookup shape. `CodeReader` is one method, so `*ethclient.Client` satisfies it without an adapter. | +| IV. Smallest slice | Holds, and tightened. Phase 1 keeps deployment in `ContractScenarioBase` rather than moving `ContractDeployer[T]` and four implementors. The registry looks up, verifies, and records. | +| V. Simplified Technical English | Holds. `vale` reports no prose findings across all Phase 1 artifacts. | + +**Gate result: proceed to tasks once an operator approves the file format.** +Nothing else blocks. No Complexity Tracking entries. + +## Complexity Tracking + +No Constitution Check violations. This section stays empty. diff --git a/specs/contract-deployment-registry/quickstart.md b/specs/contract-deployment-registry/quickstart.md new file mode 100644 index 0000000..67182ad --- /dev/null +++ b/specs/contract-deployment-registry/quickstart.md @@ -0,0 +1,80 @@ +# Quickstart + +Phase 1 output. Three tasks an operator does, and what each one costs. + +## Run against a chain the binary knows + +Nothing to do. Name the chain in the profile as today. The run finds the entry, +verifies the code, attaches, and sends no deployment. + +The run logs which source supplied each chain, so a reader confirms it used the +compiled registry rather than a file someone left behind. + +## Add a chain the binary does not know + +Two steps, and a human between them. + +**Deploy.** Point a run at the new chain, supplying its chain file: + +```sh +seiload --config profile.json --chain-file ./newchain.json +``` + +The file needs the chain's identity and an empty contract set: + +```json +{ + "chainId": 713714, + "chainName": "devnet-nightly", + "genesisHash": "…from SeiNetwork.Status.GenesisHash…", + "genesisS3URI": "…from SeiNetwork.Status.GenesisS3URI…", + "contracts": {} +} +``` + +The run finds no entry for each role, deploys, and writes a completed file +alongside its report. + +**Commit.** Review the written file. Move it into `registry/chains/` if the binary needs to +carry that chain. A nightly chain is not: it disappears after +the run, and committing its addresses would leave an entry that can never verify. + +## Add a chain from the nightly harness + +The harness holds the `SeiNetwork` it created, so it fills the chain file from +the resource's status rather than by hand: + +```go +chain := registry.Chain{ + ChainID: network.Spec.EVMChainID, + ChainName: network.Name, + GenesisHash: network.Status.GenesisHash, + GenesisS3URI: network.Status.GenesisS3URI, + Contracts: map[registry.Role]registry.Deployment{}, +} +``` + +It writes that file, mounts it into the sei-load pods, and passes `--chain-file`. +The pods deploy on first run. + +This is the case the separation exists for. The harness imports `registry` and +gets no scenario, no sender, and no metrics with it. + +## When a run fails at startup + +A `MismatchError` names the chain, the role, the address, and both hashes. + +**Got is the zero hash.** The address holds no code. Either the chain was +re-created and kept its id, or the entry names an address from a different chain. +Check the genesis hash in the entry against the chain's own. + +**Got differs from Want.** The address holds different code. Either someone +deployed over it, or the entry predates a change to the contract. + +Neither case redeploys, and that is deliberate. A run that repaired itself would +turn a stale entry into a second deployment nobody recorded. + +## What this does not do + +It does not fetch the genesis from S3, so it needs no credentials. It does not +read a Kubernetes API. It does not discover a chain it was not told about. diff --git a/specs/contract-deployment-registry/spec.md b/specs/contract-deployment-registry/spec.md new file mode 100644 index 0000000..89d3d28 --- /dev/null +++ b/specs/contract-deployment-registry/spec.md @@ -0,0 +1,633 @@ +# Contract deployment and the contract registry + +Status: DRAFT +Owner: platform +Linear: TBD + +## Anchors + +EARS · RFC 2119 · Spec Kit · idempotence · fail-closed · one-way door · +content addressing · CREATE2 + +## Why this needs a specification + +A contract scenario deploys its contract at the start of every run. That is +correct on a fresh chain and wrong everywhere else. + +Long-lived chains make that wrong: arctic-1, atlantic-2, pacific-1, and any +standing benchmark environment. Each run leaves another copy of the contract +behind. +Every restart of the prod canary moves its contract to a new address. A metric +series spanning two restarts describes two different contracts. Nothing in the +run says the address changed. + +sei-load already has the mechanism to avoid this. `TxGenerator.Attach` binds a +scenario to an address that already holds the contract, and every contract +scenario implements it. Only `mockDeployAll` calls it, which is the unit-test +path. `generatorBuilder.deployAll` always deploys, and `config.Scenario` carries +no address field. The capability exists, and no profile can reach it. + +The gap is small. The hazards it opens are not, and they are the reason this is a +specification rather than a config field. + +## Clarifications + +### Session 2026-08-22 + +- Q: Where does the registry live, and in what format? → A: A JSON file in the + sei-load repo, beside the bindings that encode against it. +- Q: What identifies a chain in a registry entry? → A: The EVM chain id and the + genesis hash. The genesis S3 URI rides alongside as provenance, not as a key. +- Q: Can a profile force a fresh deployment when the registry has an entry? → + A: Yes, through an explicit per-scenario opt-out. +- Q: How does a new entry reach the repo? → A: The run writes a file of its own. + An operator reviews it and commits it. Refined on 2026-08-23, below. + +### Session 2026-08-23 + +Three review comments on the published spec. + +- Q: Is the type a role or a contract? → A: A contract. A scenario is a list of + contracts, and a list of one is ordinary. `Role` and `Deployment` fold into one + `Contract`, and a chain entry holds a list rather than a map. +- Q: One shared client, or one per scenario? → A: One, for resolution and + binding. It is not on the load path: `auth.NoSend` keeps the transaction in + hand, and the sender's own per-endpoint clients send. +- Q: What does a run write, and do concurrent runs share contracts? → A: It writes + a chain file locally, which an operator commits to make the binary carry the + chain. Concurrent runs on an ephemeral chain each deploy their own contracts, + because sharing one contract would put contention in neither profile. + +## The workflow this serves + +This is one step of a chain, and the chain explains why the pieces divide the way +they do. + +A Go harness creates a `SeiNetwork` through the controller's API. The controller +runs the genesis ceremony and reports the chain id and the genesis hash on the +resource's status. The harness then deploys sei-load against that chain, as a +Helm release of load-generating pods. It runs nightly. + +Two deployment modes fall out, and they need different things. + +**A chain the deployment already knows.** pacific-1, atlantic-2, arctic-1, and any +standing benchmark environment. Their contracts exist. Whoever prepares the +deployment knows which chain it targets before it starts, so the chain's identity +and its contract addresses travel with the binary. The run discovers nothing. + +**A chain the harness just created.** No contracts exist. The harness holds the +chain's identity from the resource it created, and hands it to the run. The run +deploys, and reports what it deployed. + +Neither mode needs sei-load to query a Kubernetes API, and neither needs it to +discover a chain it was not told about. + +## Separation + +The registry and the deployment machinery MUST stay separable from load +generation. They serve the harness above, which has no interest in offered rate, +scenarios, or metrics. Keeping the seam clean now costs a package boundary. +Recovering it later costs a refactor of whatever grew across it. + +CDR-017 states the boundary in a form a test can check. + +## Scope + +This specification covers how a run obtains the address of the contract a +scenario drives: by deploying it, or by reading a recorded deployment. It covers +what the run MUST verify before it trusts a recorded address. + +Out of scope: what the contracts do, how a scenario builds its calls, and how the +run funds the deployer. `funder.Deployer` already resolves the deploying account. + +## Current behaviour + +- `generatorBuilder.deployAll` calls `Scenario.Deploy` for every instance. No + branch reads an address from anywhere. +- `generatorBuilder.mockDeployAll` calls `Scenario.Attach` with a generated + address. `config.LoadConfig.MockDeploy` selects it, and only tests set it. +- `config.Scenario` has no address field. +- `ContractDeployer[T]` is generic over one contract type. A scenario holds one + address, so a scenario needing more than one contract does not fit today. +- A binding's `MetaData.Bin` holds **creation** bytecode. `eth_getCode` returns + **runtime** bytecode. The two differ, so a guard cannot compare them directly. +- `StorageRWv1` exposes a `VERSION` method. No other contract in the repo does, + so an on-chain identity call is not a general mechanism. +- `config.LoadConfig` carries `ChainID`, the EVM chain id, and `SeiChainID`, a + textual label. + +## User stories + +Each story states how to test it on its own, then the scenarios that say what +holds afterwards. + +**Story 1. An operator runs a workload on pacific-1 and deploys nothing.** + +The contracts are already there. The run reads their addresses from the registry +and starts driving load. + +**Independent Test**: run a profile naming a chain the registry covers, against a +mock chain holding the recorded code. The run sends no deployment transaction. + +*Acceptance scenarios* + +**AS-1.1** — the run binds a recorded address. +- GIVEN the registry holds an entry for the run's chain and contract +- AND the code at that address matches the recorded hash +- WHEN the run starts +- THEN it binds that address, and sends no deployment + +**AS-1.2** — the run resolves only what its profile drives. +- GIVEN a profile driving one scenario +- AND a registry entry holding more than one contract +- WHEN the run starts +- THEN it reads the code of that scenario's contract only + +**AS-1.3** — a scenario holds no address. +- GIVEN the run has started +- WHEN a scenario builds a transaction +- THEN it holds a bound contract, and no field of it holds an address + +**Story 2. An operator brings up a new chain and records what the run deployed.** + +No registry entry exists. The run deploys, and it emits an entry an operator can +commit. + +**Independent Test**: run against a chain absent from the registry. The run +deploys, and it writes an entry naming the chain, the contract, the address, and +the code hash it observed. + +*Acceptance scenarios* + +**AS-2.1** — an unknown chain deploys. +- GIVEN no registry entry for the run's chain +- WHEN the run starts +- THEN it deploys each contract its profile needs, and binds what it deployed + +**AS-2.2** — the run records what it deployed. +- GIVEN the run deployed a contract +- WHEN it finishes startup +- THEN it writes a chain file holding the chain id, the genesis hash, the + contract name, the address, and the code hash it read back + +**AS-2.3** — the compiled-in registry stays read-only. +- GIVEN a path inside the compiled-in chains directory +- WHEN the run tries to write there +- THEN it fails, and writes nothing + +**AS-2.4** — a supplied file wins. +- GIVEN a supplied chain file naming a chain the binary also carries +- WHEN the run loads the registry +- THEN the supplied file wins, and the run logs which source it used + +**Story 3. A stale entry fails the run instead of corrupting it.** + +A devnet keeps its chain id across a re-genesis. The recorded address now holds +nothing, or holds different code. + +**Independent Test**: point a run at a chain whose recorded address holds no code. +The run fails at startup and names the chain, the contract, and the address. + +*Acceptance scenarios* + +**AS-3.1** — an empty address fails the run. +- GIVEN a registry entry whose address holds no code +- WHEN the run starts +- THEN it fails, naming the chain, the contract, the address, and both hashes + +**AS-3.2** — wrong code fails the run, distinguishably. +- GIVEN a registry entry whose address holds different code +- WHEN the run starts +- THEN it fails the same way, and the message tells this case from absent code + +**AS-3.3** — a mismatch repairs nothing. +- GIVEN either mismatch +- WHEN the run fails +- THEN it has deployed nothing, and sent no transaction + +## Requirements + +Every requirement carries an ID. A test names the ID it covers. + +### Selecting an address + +**CDR-001.** WHERE a profile names a scenario that needs a contract, the run MUST +get that contract's address from one source only. That source is the registry, or +a deployment the run performs. + +**CDR-002.** IF the registry holds an entry for the run's chain and that +contract, THEN the run MUST use the recorded address. It MUST NOT deploy. + +**CDR-003.** IF the registry holds no entry for the run's chain and that +contract, THEN the run MUST deploy the contract. + +**CDR-004.** A profile MUST be able to override the registry with an explicit +address, for a contract deployed outside this repo. + +**CDR-016.** A profile MUST be able to force a deployment for a scenario, even +where the registry holds an entry. The profile MUST name that choice explicitly. + +Rationale for CDR-016: a run measuring contention from an empty keyspace needs +virgin storage. Registry-first is the default, so a reader can see from the +profile why a run redeploys. + +**CDR-005.** A chain entry MUST hold a list of contracts, each named. A scenario +is a list of contracts, and a list of one is ordinary. + +Rationale for CDR-005: the token and DeFi workloads need three and seven +contracts. A registry shaped around one address per scenario would need a second +breaking change to hold them. + +### Trusting a recorded address + +**CDR-006.** WHEN a run reads an address from the registry, the run MUST verify +that the address holds contract code before it sends any load. + +**CDR-007.** The registry MUST record a hash of the runtime code observed at +deployment. WHEN a run reads an address, it MUST compare the code at that address +against the recorded hash. + +**CDR-008.** IF the code at a recorded address is absent, or its hash differs +from the recorded hash, THEN the run MUST fail at startup. It MUST name the +chain, the contract role, the address, and both hashes. + +**CDR-014.** A registry entry MUST identify its chain by the EVM chain id and the +genesis hash together. An entry MUST also record the genesis S3 URI. + +### Where the values come from + +**CDR-017.** The registry package MUST NOT import any sei-load package other than +the contract bindings. A test MUST assert this. + +Rationale for CDR-017: the harness that creates a chain and deploys contracts has +no interest in offered rate, scenarios, or metrics. `config` and +`generator/bindings` already import no other sei-load package, so a leaf is the +established shape here. `generator/utils` is not a leaf, so the registry builds +its own deployment options instead of borrowing that one. + +**CDR-018.** The binary MUST carry the registry for every chain it already knows, +compiled in. A run against one of those chains MUST NOT read the registry from +disk. It MUST NOT reach a network to find it. + +**CDR-019.** WHERE a deployment targets a chain the binary does not carry, the +deployment MUST supply that chain's file. The run MUST read the chain id, the +genesis hash, and any known contract addresses from that file. + +**CDR-020.** IF a supplied file names a chain the binary also carries, THEN the +supplied file MUST win. The run MUST log which source it used. + +### What load generation receives + +**CDR-021.** Load generation MUST receive bound contracts. A scenario MUST NOT +receive a contract address, and MUST NOT bind a contract itself. + +**CDR-022.** A run MUST resolve only the contracts its profile needs. It MUST NOT +verify or deploy a contract no scenario in the profile drives. + +**CDR-023.** One client MUST serve the whole resolution. A scenario MUST NOT open +its own connection to bind. + +Rationale for CDR-021 to CDR-023: an address is a fact about deployment, and a +scenario's job is to shape a transaction. Today `AttachScenario` takes an +address, dials its own client, binds, and panics if either fails. Every scenario +therefore carries three concerns that belong to one earlier step. + +Moving them makes the boundary in CDR-017 real rather than nominal. A scenario +that never sees an address cannot depend on the registry, so extracting the +registry stays a move rather than a refactor. + +The hand-off already exists. `ContractDeployer[T]` declares `GetBindFunc` and +`SetContract`, so the resolution binds and hands the instance over. What goes +away is `AttachScenario`, which is the part that dials and panics. + +Rationale for CDR-018 to CDR-020: whoever prepares a deployment knows the chain +before it starts. A long-lived chain travels with the binary. A chain the harness +created travels with the deployment. Neither mode discovers a chain, so neither +can point at one nobody chose. + +**CDR-015.** WHEN a run reads an entry, the run MUST match on the chain id and +the genesis hash. It MUST NOT match on the genesis S3 URI. + +Rationale for CDR-014 and CDR-015: `SeiNetwork.Status` already carries both +values, and the controller already pairs them this way. A node booting from the +S3 fallback verifies its download against `GenesisHash`. The hash is the +identity, and the URI is where the artifact lives. A re-genesis changes the +hash. Whether it changes the URI depends on how the controller names the object, +which this specification does not control. + +Rationale for CDR-006 to CDR-008: an EVM chain id does not identify a chain +instance. A devnet keeps its id across a re-genesis, so a recorded address +survives the state that made it meaningful. A registry entry is a cache, and this +is the check that stops a stale cache entry from becoming a silent wrong run. + +The failure is otherwise invisible. Calls to an address holding no code succeed +at the EVM layer and do nothing. Calls to an address holding different code +encode against the wrong ABI. sei-load reads inclusion rather than execution +status, so neither shows up as a failure. See +`specs/transaction-outcome-tracking/spec.md`. + +**CDR-009.** The run MUST NOT repair a mismatch by deploying. It MUST fail. + +Rationale for CDR-009: deploying on mismatch turns a stale entry into a silent +second deployment, which is the behaviour this specification exists to remove. An +operator decides whether the entry is wrong or the chain is. + +### Recording a deployment + +**CDR-010.** WHEN a run deploys a contract, the run MUST emit a registry entry. +The entry MUST hold the chain, the contract role, the address, and the runtime +code hash. + +**CDR-011.** A run MUST NOT modify the compiled-in registry. It MUST write what +it deployed to a chain file of its own. An operator commits that file to make the +binary carry the chain. + +Rationale for CDR-011: the registry is the record other runs trust. A load test +that edits its own source of truth can record an address from a chain that no +longer exists. No human sees it happen. + +### Cost and behaviour + +**CDR-012.** WHILE a run starts, the requests it makes to verify recorded +addresses MUST grow with the contract count. They MUST NOT grow with the account +or transaction count. + +**CDR-013.** WHERE more than one run starts against one ephemeral chain, each run +MUST deploy its own contracts. A run MUST NOT wait for another run's deployment, and +MUST NOT drive a contract another run deployed. + +Rationale for CDR-013: isolation is the intent, not a tolerated outcome. Two runs +driving one StorageRW contract contend on the same slots. That contention is in +neither profile, so both runs measure a workload nobody configured. Separate +contracts keep each run's contention exactly what its profile asked for. + +Sharing one contract set across runs needs a preflight step that deploys once and +injects the addresses. That is orchestration this specification does not add. See +Deferred. + +## Success Criteria + +The specification closes when all of the following hold. A ticket cites these by +ID, so they carry one. + +- **SC-001** A profile naming a chain the registry covers starts and sends no + deployment transaction. *Verifier*: `go test ./registry/... ./generator/...` +- **SC-002** A profile naming an uncovered chain deploys, and writes a + reviewable chain file. *Verifier*: `go test ./generator/...` +- **SC-003** A recorded address holding no code, or unexpected code, fails the + run at startup with both hashes named. *Verifier*: `go test ./registry/...` +- **SC-004** A scenario driving more than one contract records one entry per + contract. *Verifier*: `go test ./registry/...` +- **SC-005** A profile that forces a deployment redeploys, even where an entry + exists. *Verifier*: `go test ./generator/...` +- **SC-006** No scenario holds a contract address, and no scenario opens a + connection to bind. *Verifier*: `go test ./generator/...` and `grep` for the + removed method. +- **SC-007** The registry package imports no sei-load package other than the + contract bindings. *Verifier*: `go test ./registry/ -run TestImportBoundary` +- **SC-008** Every requirement ID above has at least one test that names it. + *Verifier*: review against the traceability table in `tasks.md`. +- **SC-009** Every committed chain file parses, and holds each field the format + names. *Verifier*: the CI gate from T028. +- **SC-010** `make verify` passes. + +## Design + +How we propose to meet the requirements above, and what we considered instead. +The requirements are the contract. This is one way to satisfy them, and a later +reader can replace it without reopening them. +sees the mental model before the rules that constrain it. + +### Three parts, and which one is extractable + +**`registry`** answers one question: does this contract already exist on this chain, +and is the code there the code we recorded? It imports no sei-load package other +than the contract bindings, so it lifts out whole. + +**A preparation step** joins the registry to the scenarios. It reads the profile, +resolves each contract, deploys what is missing, binds the results, and hands +each scenario its contracts. It depends on both sides, so it is the one part that +cannot be extractable, and it stays small for that reason. + +**The scenarios** shape transactions against a bound contract. After this feature +they hold no address and open no connection. + +### The registry's surface + +```go +// Contract is one named contract on one chain. A scenario is a list of these, +// and a list of one is ordinary. +type Contract struct { + Name string + Address common.Address + CodeHash common.Hash +} + +// Chain is one chain and the contracts deployed on it. +type Chain struct { + ChainID int64 + ChainName string + GenesisHash string + GenesisS3URI string + Contracts []Contract +} + +// CodeReader is the one chain call this package makes. An *ethclient.Client +// satisfies it, and a test supplies a fake. +type CodeReader interface { + CodeAt(ctx context.Context, account common.Address, block *big.Int) ([]byte, error) +} + +// Load returns the compiled-in registry, with supplied files layered over it. +func Load(paths ...string) (*Registry, error) +func (r *Registry) Chain(chainID int64, genesisHash string) (Chain, bool) +func (r *Registry) Sources() map[string]string +func (c Chain) Contract(name string) (Contract, bool) + +// Resolve reports the address for a named contract, or that the caller needs to +// deploy. It verifies before it returns an address. +func Resolve(ctx context.Context, code CodeReader, r *Registry, + chainID int64, genesisHash, name string) (addr common.Address, deploy bool, err error) + +// Record reads the code at a freshly deployed address and returns the entry. +func Record(ctx context.Context, code CodeReader, name string, + addr common.Address) (Contract, error) + +// WriteEntry writes a chain file for review. It refuses a path inside the +// compiled-in chains directory. +func WriteEntry(path string, chain Chain) error + +// MismatchError is the failure a stale entry produces. Got is the zero hash when +// the address holds no code. +type MismatchError struct { + ChainName string + ChainID int64 + ContractName string + Address common.Address + Want, Got common.Hash +} +``` + +### What changes on the scenario side + +`ContractDeployer[T]` keeps the two methods the hand-off already uses: + +```go +GetBindFunc() ContractBindFunc[T] // the preparation step binds with this +SetContract(contract *T) // and hands the instance over with this +``` + +`AttachScenario(config, address) common.Address` **goes away.** It takes an +address, dials its own client, binds, and panics if either fails. The +preparation step takes over all three, and returns an error where it panicked. + +### The sequence, at startup + +Load generation does not begin until every step here has run. + +1. `main` reads the profile into a `LoadConfig`. +2. `main` calls `registry.Load(chainFiles...)`. With no files it returns the + compiled-in registry alone, touching no disk and no network. +3. `main` dials **one** client. That client serves every step below, and no + scenario dials its own. + + It is not on the load path. `CreateTransactionOpts` sets `auth.NoSend`, so a + bound contract hands the transaction back rather than sending it, and the + sender's own per-endpoint clients do the sending. This client binds and reads + code. Nothing else. +4. The preparation step collects the contracts the profile's scenarios need. A + contract no scenario drives is never resolved, verified, or deployed. +5. For each contract it calls `registry.Resolve`. + - An entry exists and its code matches: `Resolve` returns the address. + - No entry exists: `Resolve` reports that the caller needs to deploy. + - An entry exists and its code does not match: `Resolve` returns a + `*MismatchError`, and the run stops. +6. For each contract that needs one, the step deploys, then calls + `registry.Record` to read back the code hash. +7. The step binds every address, using `GetBindFunc` and that one client, and + calls `SetContract` on the scenario. +8. If step 6 recorded anything, the step writes a chain file beside the run + report. An operator commits it to make the binary carry that chain. +9. The generator starts. Every scenario holds a bound contract. + +### Where a failure stops the run + +| Failure | Step | What the run does | +| -- | -- | -- | +| A supplied chain file does not parse | 2 | Fails, naming the file. | +| The chain is in no registry, and no file supplies it | 4 | Fails, naming the chain id. | +| A recorded address holds no code, or different code | 5 | Fails with a `*MismatchError`. It does not deploy. | +| A deployment does not mine | 6 | Fails, naming the contract. | +| Binding fails | 7 | Fails, naming the contract. It does not panic. | +| The output path is inside the compiled-in registry | 8 | Fails, refusing to write. | + +Every one of these stops the run before it sends a transaction. That is +deliberate: a run that starts and then discovers its contracts are wrong has +already published numbers nobody can trust. + +### What we considered instead + +Each row is a choice the design makes, the alternative, and what decided it. + +**Binding in the preparation step, not in the scenario.** The scenario could keep +binding, as it does today, and simply receive an address from the registry. That +is a smaller diff. It leaves every scenario dialling its own client and panicking +on failure, and it leaves an address in a type whose job is shaping transactions. +The larger change is what makes CDR-017's boundary real: a scenario that never +holds an address cannot depend on the registry. + +**One shared client, not one per scenario.** Today each `AttachScenario` dials. +Per-scenario clients are simpler to reason about in isolation and cost a +connection each, and they hide a dial failure inside a panic. One client makes +the failure an error at a known step. + +**`Resolve` returns two values, not an error for a miss.** A `NotFound` error +would let one branch handle every outcome. A missing entry is ordinary: it is the +whole of the deploy path. An error would make the common case read as the +exceptional one, and every caller would type-switch to recover it. + +**`CodeReader` is one method, not `*ethclient.Client`.** Taking the concrete type +is less code. It would put `ethclient` in the registry's import list, and it +would force a test to stand up a client or an HTTP fake. One method keeps the +import list short enough to read, and a test supplies a struct. + +**Verify at startup, not lazily on first use.** Lazy verification costs nothing +for a role no scenario reaches. It also means the run has already started, and +already published numbers, when it discovers its contracts are wrong. Startup is +the last point where failing is free. + +**The registry records the code hash it observed, rather than deriving one.** A +binding's `Bin` is creation bytecode and `eth_getCode` returns runtime bytecode, +so there is nothing to derive from. Extracting the runtime portion from creation +bytecode is possible and fragile. The metadata Solidity appends also varies with +compiler settings, so two honest builds can disagree. + +**`ContractDeployer[T]` stays generic over one contract.** CDR-005 keys the +registry by role, which implies more than one. Changing the interface now would +touch four implementors for a case no scenario has yet. The registry's file +format is the one-way door and the interface is not. The format therefore carries +the role map, and the interface changes when TokenOps needs it. + +**A JSON file, not Go source.** Generated Go would fail to compile on a bad +entry, which is a better failure than a startup error. It also means an operator +cannot add a chain without writing Go, and CDR-011's reviewable sidecar becomes +code generation. + +## Decisions this specification does not make + +The clarification session resolved D-1 to D-4. See `## Clarifications`, and +CDR-004, CDR-011, CDR-014, CDR-015 and CDR-016, which carry those answers. + +One decision stays open, because the answer sits in a repository this +specification does not cover. + +**D-5, how a run learns its chain's genesis hash.** Resolved: the deployment +tells the run. A chain the binary knows carries its hash compiled in. A chain the harness created +carries it in the file the deployment supplies, which the harness fills from the +resource's status. sei-load reads no Kubernetes API, and queries no endpoint for +identity. + +CDR-015 therefore compares a value someone supplied, not one the run observed. A +deployment shipping the wrong chain file passes the identity check and fails the +code check at CDR-006 to CDR-008. That is what catches it, and it is why every run +runs the code check. + +## Deferred + +Named so a reader sees the line, and so nobody builds them by accident. + +- **A subcommand that deploys.** The run deploys when it finds no entry, and + writes a file. Separating deployment from load generation is the better shape + and it is not needed to serve the nightly harness. Un-defer when an operator + needs to deploy without starting a run. +- **Reading `SeiNetwork` directly.** The harness holds the resource and fills the + chain file from it. Un-defer if a run ever starts without a harness that knows + its chain. +- **Verifying the genesis against its S3 URI.** The registry records the URI. It + does not fetch it, which would put S3 credentials in a load generator. The + genesis hash already identifies the chain. Un-defer if a chain file's hash and + its URI are ever suspected of disagreeing. +- **A registry covering contracts this repo does not deploy.** CDR-004 lets a + profile name an address. Nothing else about foreign contracts is in scope. +- **A preflight step that deploys once for many runs.** CDR-013 gives each run + its own contracts, which is what keeps each run's contention its own. Sharing + one set needs a step that deploys before any run starts, and injects the + addresses. Un-defer when a workload needs many runs driving one contract on + purpose, rather than by accident. + +## Verification + +A requirement with no test that names its ID is not done. + +- CDR-001 to CDR-005: drive a mock chain and assert which addresses the run binds + and whether it sent a deployment. +- CDR-006 to CDR-009: return absent code, then wrong code, from the mock. Assert + the startup error names the chain, the role, the address, and both hashes. +- CDR-010 and CDR-011: assert the emitted entry's fields, and assert the run left + the committed registry unchanged. +- CDR-014 to CDR-016: assert an entry matches on chain id and genesis hash, and + that a forcing profile deploys where an entry exists. +- CDR-021 to CDR-023: assert no scenario field holds an address, and that a run + with one scenario resolves one role. +- CDR-012: count startup requests against a mock, across two contract counts. +- CDR-013: start two builders against one mock chain and assert both complete. diff --git a/specs/contract-deployment-registry/tasks.md b/specs/contract-deployment-registry/tasks.md new file mode 100644 index 0000000..302aed7 --- /dev/null +++ b/specs/contract-deployment-registry/tasks.md @@ -0,0 +1,224 @@ +# Tasks: Contract deployment and the contract registry + +**Input**: `spec.md` (23 requirements), `plan.md`, `data-model.md`, +`contracts/registry-api.md`, `quickstart.md` + +**Format**: `[ID] [P?] [Story] Description` + +- **[P]**: runs in parallel with its siblings. Different files, no shared edit. +- **[Story]**: US1, US2, US3, or blank for setup and foundation. +- Each task names the requirement IDs it closes. A requirement with no test that + names its ID is not done, so the mapping is the completion check. + +Revised 2026-08-23 after review. `Role` and `Deployment` became one `Contract`. +A task now deletes `AttachScenario` rather than calling it. The preparation step +gained the tasks it never had. + +## A note on delivery order before you read the phases + +US1 is the primary story: run against a chain the binary knows and deploy +nothing. Its code is testable against a mock from the start. + +It is not *useful* until US2 has run. No contracts exist on arctic-1, atlantic-2 +or pacific-1 today, so the compiled-in chain files for those chains cannot hold +addresses yet. Somebody runs US2 against each long-lived chain, reviews the +written file, and commits it. Only then does US1 do anything in production. + +Build US1 first, because US2 and US3 both need its types and its verification. +Expect US2 to be the first phase that changes what an operator sees. + +## Phase 1: Setup + +- [ ] T001 Create `registry/` with `doc.go` stating the package's one job and its + import boundary, and an empty `registry/chains/` holding a `.gitkeep`. +- [ ] T002 [P] Add `registry/boundary_test.go` asserting `go list -deps + ./registry` names no sei-load package except `generator/bindings`. + **Closes CDR-017.** Write it now so it guards every later task. + +T002 passes trivially against an empty package. That is the point: it fails the +moment a later task reaches for `config` or `types`. + +## Phase 2: Foundational + +Blocking prerequisites. No user story starts before these land. + +- [ ] T003 Add `Contract`, `Chain` and `Registry` to `registry/registry.go`, per + `data-model.md`. `Chain.Contracts` is a `[]Contract`, each carrying its + name, address and code hash. `common.Address` and `common.Hash` marshal + themselves, so add no custom marshaller. **Closes CDR-005.** +- [ ] T004 Add `chainKey{chainID, genesisHash}` and `Registry.Chain`, matching on + both fields and on nothing else. **Closes CDR-014, CDR-015.** +- [ ] T005 [P] Add `//go:embed chains/*.json` and `Load(paths ...string)`, + layering supplied files over compiled-in ones. **Closes CDR-018, CDR-019, + CDR-020.** +- [ ] T006 [P] Add `Registry.Sources()` so a run logs which file supplied each + chain. **Closes CDR-020.** +- [ ] T007 Add `registry/testdata/` with a two-chain fixture, one holding a + contract and one empty. Every later test reads it rather than building a + `Chain` inline. + +## Phase 3: User Story 1 - Run against a chain the binary knows (Priority: P1) + +**Goal**: the run finds the entry, verifies the code, hands each scenario its +bound contract, and sends no deployment transaction. + +**Independent test**: run a profile naming a chain in the fixture, against a mock +chain holding the recorded code. Assert the run sent no deployment, and that no +scenario holds an address. + +- [ ] T008 [P] [US1] Add `CodeReader`, and a fake in `registry/registry_test.go` + returning code, no code, or an error. +- [ ] T009 [US1] Add `Chain.Contract(name)`, returning two values because a miss + is ordinary. +- [ ] T010 [US1] Add `Verify(ctx, code, chain, name)`, hashing `eth_getCode` + output with Keccak-256 and comparing it to the recorded hash. + **Closes CDR-006, CDR-007.** +- [ ] T011 [US1] Add `Resolve(...)` returning an address to bind, or a signal to + deploy, and verifying before it returns an address. + **Closes CDR-001, CDR-002, CDR-003.** +- [ ] T012 [US1] Add the preparation step in `generator/`. It reads the profile, + collects the contracts its scenarios need, and dials one client for the + whole step. **Closes CDR-022, CDR-023.** A contract no scenario drives is + never resolved. +- [ ] T013 [US1] Bind each resolved address with `GetBindFunc` and that one + client, then call `SetContract`. **Delete `AttachScenario`** from + `ScenarioDeployer` and its implementors, and return an error where it + panicked. **Closes CDR-021.** +- [ ] T014 [US1] Assert a registry hit issues no deployment, and that no scenario + field holds an address. **Closes CDR-002, CDR-021.** Break the fixture + address to confirm the assertion fails before trusting it. + +T013 is the largest task in the feature. It removes a method from an interface +with four implementors, so it lands on its own rather than beside T012. + +## Phase 4: User Story 2 - Bring up a new chain and record what it deployed (Priority: P2) + +**Goal**: no entry exists, so the run deploys and writes a chain file an operator +reviews. + +**Independent test**: run against a chain absent from the fixture. Assert the run +deployed, and wrote a file naming the chain, the contract, the address, and the +code hash it observed. + +- [ ] T015 [P] [US2] Add `Record(ctx, code, name, addr)`, reading the deployed + code and returning the `Contract`. **Closes CDR-010.** +- [ ] T016 [US2] Add `WriteChain(path, chain)`. It refuses any path inside the + compiled-in `chains/` directory. **Closes CDR-011.** Assert the refusal, + not only the success. +- [ ] T017 [US2] Add a `--chain-file` flag to `main.go`, repeatable, passed into + `Load`. **Closes CDR-019.** +- [ ] T018 [US2] Have the preparation step call `Record` after a scenario + deploys, and write the chain file beside the run report. +- [ ] T019 [US2] Assert a supplied file beats a compiled-in chain of the same + identity. Assert the run logs which one it used. **Closes CDR-020.** + +## Phase 5: User Story 3 - A stale entry fails the run (Priority: P3) + +**Goal**: a recorded address holding nothing, or holding different code, stops +the run at startup instead of corrupting it. + +**Independent test**: point a run at a fixture address the mock serves no code +for. Assert startup fails and the message names the chain, the contract, the +address, and both hashes. + +- [ ] T020 [P] [US3] Add `MismatchError` with `ChainName`, `ChainID`, + `ContractName`, `Address`, `Want` and `Got`, where a zero `Got` means + absent code. **Closes CDR-008.** +- [ ] T021 [US3] Assert `Verify` returns `*MismatchError` for absent code and for + wrong code, and that the message distinguishes them. **Closes CDR-008.** +- [ ] T022 [US3] Assert no path deploys after a mismatch. **Closes CDR-009.** + This is the requirement most likely to erode, because redeploying looks + like a fix. +- [ ] T023 [US3] Assert the run fails before it sends any load, not after. + **Closes CDR-006.** + +## Phase 6: Polish and cross-cutting + +- [ ] T024 [P] Add the force-deploy flag to `config.Scenario`, and honour it in + the preparation step by skipping `Resolve`. **Closes CDR-016.** +- [ ] T025 [P] Add an explicit-address field to `config.Scenario` for a contract + deployed outside this repo. **Closes CDR-004.** +- [ ] T026 [P] Assert startup issues one `CodeAt` per contract, and that the + count does not move with account count. **Closes CDR-012.** +- [ ] T027 [P] Assert two runs against one ephemeral chain each deploy their own + contracts, and that neither drives the other's. **Closes CDR-013.** +- [ ] T028 Add a CI check that every committed chain file parses, and holds each + field the format names. **Closes success criterion 8.** +- [ ] T029 Walk `quickstart.md` end to end against a local chain and correct what + does not match. + +## Dependencies + +```text +Setup (T001, T002) + │ +Foundational (T003 → T004 → T005/T006 → T007) + │ + ├── US1 (T008 → T009 → T010 → T011 → T012 → T013 → T014) + │ │ + │ ├── US2 (T015 → T016 → T017 → T018 → T019) + │ └── US3 (T020 → T021 → T022 → T023) + │ + └── Polish (T024 … T029) +``` + +US2 and US3 both need US1's types and its `Verify`. They do not need each other, +so they run in parallel once US1 lands. + +## Parallel execution + +Within Foundational: T005 and T006 touch different functions in the same file, so +sequence them if one author holds both. + +Within US1: T008 is independent of T009 and T010. T012 and T013 are not parallel +with each other, because T013 consumes what T012 builds. + +Within US3: T020 lands first, then T021, T022 and T023 all consume it and touch +different assertions. + +Within Polish: every task carries `[P]` except T028 and T029. + +## Implementation strategy + +**MVP is US1 plus US2.** US1 alone reads a registry nothing has written. US2 +alone deploys every run, which is today's behaviour. Together they close the +loop: deploy once, record, and reuse. + +**US3 ships with the others despite being third.** Without it a stale entry +produces calls that land, do nothing, and report as included, because sei-load +reads inclusion rather than execution status. US1 and US2 shipping without US3 +would make that failure reachable in production. + +**Polish is genuinely deferrable, with one exception.** T024 to T027 close +requirements nothing depends on yet. T028 is worth pulling forward if a chain +file reaches `main` before the phase does. + +## Traceability + +| Requirement | Task | +| -- | -- | +| CDR-001 | T011 | +| CDR-002 | T011, T014 | +| CDR-003 | T011 | +| CDR-004 | T025 | +| CDR-005 | T003 | +| CDR-006 | T010, T023 | +| CDR-007 | T010 | +| CDR-008 | T020, T021 | +| CDR-009 | T022 | +| CDR-010 | T015 | +| CDR-011 | T016 | +| CDR-012 | T026 | +| CDR-013 | T027 | +| CDR-014 | T004 | +| CDR-015 | T004 | +| CDR-016 | T024 | +| CDR-017 | T002 | +| CDR-018 | T005 | +| CDR-019 | T005, T017 | +| CDR-020 | T005, T006, T019 | +| CDR-021 | T013, T014 | +| CDR-022 | T012 | +| CDR-023 | T012 | + +Every requirement has a task. No task closes nothing. diff --git a/specs/transaction-outcome-tracking/spec.md b/specs/transaction-outcome-tracking/spec.md new file mode 100644 index 0000000..cc96ad7 --- /dev/null +++ b/specs/transaction-outcome-tracking/spec.md @@ -0,0 +1,267 @@ +# Transaction outcome tracking + +Status: DRAFT +Owner: platform +Linear: PLT-466 + +## Anchors + +EARS · RFC 2119 · Spec Kit · goodput · coordinated omission · back-pressure · +conservation identity · fail-closed + +## Why this needs a specification + +sei-load reports how many transactions it offered, and how many reached a block. +It does not report how many did what the workload asked. + +A run can report a million accepted, near-perfect inclusion, and a healthy p99 +while every transaction reverted. Nothing in the output says so. + +One missing metric does not cause this. Two properties of the design do. + +**Two ledgers, and nothing joins them.** `stats.Collector` counts what the sender +submitted. `stats.InclusionTracker` counts what became of those submissions. The +tracker holds no reference to the collector. The collector never learns an +outcome. Neither one can produce a ratio of committed to offered. + +**The tracker observes delivery, not execution.** Its terminal states say whether +a transaction arrived. It derives `Included` from the transaction hashes in a +block, and a hash carries no execution status. `Included` therefore covers a +transaction that committed and one that reverted and burned its gas. + +This lands hardest where a revert is the expected failure. A contract call with a +bad argument reverts. A precompile call from an unassociated account reverts. Each +produces an included transaction and no effect. sei-load is growing exactly those +workloads. + +## Clarifications + +### Session 2026-08-23 + +- Q: Where does the tracker get execution status? → A: Replace the block fetch + with `eth_getBlockReceipts`. Receipts carry the transaction hash and the + status, so one call returns what two would. +- Q: How does the sender hand a transaction to the tracker? → A: Through a + channel. The overflow policy becomes a terminal state. +- Q: Does the tracker own the accounting or feed it? → A: It feeds the collector, + which already holds the per-scenario and per-operation counts. + +## Scope + +This specification covers how sei-load records the outcome of a transaction it +submitted, and how that outcome reaches the run's accounting. + +Out of scope: which workloads sei-load drives, how the sender schedules arrival, +and how the report presents these counts. The report consumes what this produces. + +## Current behaviour + +Stated so a reader sees what changes and what does not. + +- `sender.ShardedSender` calls `InclusionTracker.Register` after a successful + send, which makes `registered` a subset of `accepted`. +- `Register` is a synchronous call. It takes the tracker's mutex, and the tracker + holds that same mutex while it matches a block. +- The tracker subscribes to new heads. It fetches each block's transaction hashes + once per block, not once per transaction. +- An earlier design fetched a receipt per transaction. The rewrite removed that + path, because its cost grew with the throughput the run offered. +- A reap loop evicts a transaction still in flight after `reapAfter`, as + `Expired`. Registration past the registry cap becomes `DroppedAtCap`. +- `sender/doc.go` states a conservation identity: + `registered = included + expired + inflight_at_shutdown`. +- `Collector.RecordTransaction` records the submission and its send latency. + +## User stories + +**Story 1. An operator reads a run and learns whether the workload worked.** + +Today the operator reads an inclusion rate. A run whose transactions all reverted +looks like a run that succeeded. + +**Independent Test**: drive a mock chain that reverts every transaction. The run +report states zero committed. It does not state a healthy inclusion rate alone. + +**Story 2. An engineer compares two runs and trusts the difference.** + +A change that raises the revert rate looks like a change that raised throughput, +because a reverted transaction still counts as delivered. + +**Independent Test**: drive two mock runs with the same offered rate and different +revert rates. The reported goodput differs. The reported inclusion rate does not +carry the difference alone. + +**Story 3. The load generator does not slow itself down while measuring.** + +The sender hands each successful send to the tracker under a shared mutex. It +waits behind a block match in progress. + +**Independent Test**: measure the hand-off latency while the tracker matches a +block. The tail stays within the stated budget. The current design fails this. + +## Requirements + +Every requirement carries an ID. A test names the ID it covers. + +### Outcome states + +**TOT-001.** The tracker MUST distinguish a transaction that committed from one +that reverted. It MUST NOT report one state that covers both. + +**TOT-002.** WHEN the tracker matches a registered transaction in a block, the +tracker MUST resolve it to exactly one terminal state. + +**TOT-003.** The terminal states MUST partition every registered transaction. The +conservation identity MUST hold at shutdown over the new states. + +**TOT-014.** The hand-off from sender to tracker is a channel. IF the channel is +full, THEN the sender MUST NOT block. The run MUST count that transaction under a +terminal state naming the hand-off. + +Rationale for TOT-014: a channel that blocks when full brings back the stall +TOT-010 removes. A channel that drops in silence breaks TOT-003, because a +registered transaction would reach no terminal state. Naming the drop keeps both +true. It also makes the queue size a number an operator sees, rather than a +constant nobody revisits. + +**TOT-004.** IF the tracker matches a transaction but cannot determine its +execution status, THEN the tracker MUST record a state that names the missing +status. It MUST NOT record that transaction as committed. + +Rationale for TOT-004: a status source can fail on its own, while the block fetch +succeeds. Counting an unknown as a success is the defect this specification +removes. Counting it as a revert would invent a chain outcome. + +### Accounting + +**TOT-005.** The run's accounting MUST report offered, accepted, rejected, +committed, reverted, and dropped. + +**TOT-006.** The accounting MUST report those counts per scenario and per +operation. It MUST use the key the send path already labels its metrics with. + +**TOT-007.** The accounting MUST expose the ratio of committed to offered. + +**TOT-016.** The tracker MUST report each terminal outcome to the collector. The +collector MUST own the counts that TOT-005 to TOT-007 report. + +Rationale for TOT-016: the collector already holds per-scenario and +per-operation counts, so one ledger produces the ratio and one report reads it. +The alternative makes the collector key by transaction, which is state it does +not carry. + +**TOT-008.** The accounting MUST separate a reverted transaction from a dropped +one. The system under test produces a revert. The mempool or the run's own +timeout produces a drop. + +### Cost + +**TOT-009.** The tracker MUST NOT issue one request per transaction to determine +execution status. It MUST read status at block granularity. + +**TOT-015.** The tracker MUST get hashes and status from one call per block. It +MUST NOT keep a separate call for hashes. + +Rationale for TOT-015: `eth_getBlockReceipts` returns the transaction hash and +the status together. The call the tracker already makes per block becomes the +call that carries status. Sei serves it, and a `sei_` variant as well. Keeping +the hash-only call beside it would add a round trip, and a second thing to keep +consistent. + +Rationale for TOT-009: the rewrite removed the per-transaction receipt path. One +extra request per transaction scaled the tool's own load with the load it +offered. A block-granular source keeps the fixed per-block cost. + +**TOT-010.** Handing a submitted transaction to the tracker MUST NOT block the +send path behind work the tracker performs for another transaction. + +Rationale for TOT-010: `Register` shares one mutex with block matching. A sender +completing a send waits behind a match. The reported send latency hides that +wait, because the sender measures latency before the call. A load generator that +stalls its own send path understates the number it exists to report. + +**TOT-011.** WHILE a run is in progress, the registry cap MUST bound the +tracker's memory. Memory MUST NOT grow with the number of transactions sent. + +### Reporting + +**TOT-012.** WHERE a run writes a report, the report MUST state which counts +cover execution and which cover delivery only. + +**TOT-013.** IF execution status is unavailable for a whole run, THEN the report +MUST say so. It MUST NOT omit the execution counts silently. + +Rationale for TOT-013: an absent number reads as zero. A run against an endpoint +that serves no status MUST NOT look like a run with no reverts. + +## Success Criteria + +The specification closes when all of the following hold. + +1. A run against a chain that reverts every transaction reports zero committed + and a non-zero reverted count. +2. A run reports a goodput ratio, per scenario and per operation. +3. Every requirement ID above has at least one test that names it. +4. The tracker issues no more requests per block than it issues today. +5. The hand-off tail latency from sender to tracker stays inside a budget the + implementation states and measures. +6. `sender/doc.go` states the conservation identity over the new terminal states, + and a test asserts it. +7. `make verify` passes. + +## Decisions this specification makes + +The clarification session settled three. This section names the alternative each +one rejected, because each alternative reads as reasonable. + +**The status source is `eth_getBlockReceipts`.** Sei serves it, and it carries +the transaction hash beside the status. It replaces the tracker's per-block hash +fetch rather than adding to it, so the request count per block does not move. +TOT-015 states this. + +Rejected: a receipts-root derivation, and a trace call. Both are block-granular. +Both need more from the endpoint than a call Sei already serves. + +**The hand-off is a channel, not a shortened lock hold.** A channel decouples the +sender from block matching. Shortening the lock hold leaves the tail in place, +and that tail is the defect TOT-010 removes. + +The cost is a new terminal state. A full channel MUST NOT block the sender, so a +drop needs a name to keep TOT-003 true. TOT-014 states this. + +**The tracker feeds the collector.** The collector already holds the per-scenario +and per-operation counts. One ledger produces the ratio, and one report reads it. +TOT-016 states this. + +Rejected: the collector reads the tracker at report time. That needs the +collector to key by transaction, which is state it does not carry. + +## Decisions this specification does not make + +Both concern the channel from TOT-014. Neither changes an interface, so neither +blocks the design. + +**D-1. The queue depth.** The depth sets how long a tracker stall the sender +absorbs before it drops. Too small drops under ordinary jitter. Too large hides a +tracker that cannot keep up. + +An implementation MUST state the depth and report the drop count. An operator +then reads a number rather than inheriting a constant. + +**D-2. Whether a hand-off drop voids the run.** A drop means the run cannot +account for a transaction it submitted. A few drops leave the goodput ratio +usable. Many make it a lower bound rather than a measurement. + +The verdict rule needs a threshold. Nothing yet says what that threshold is. + +## Verification + +A requirement with no test that names its ID is not done. + +- TOT-001 to TOT-004 and TOT-008: drive a mock block source that returns a block + holding a reverted transaction. The tracker's tests already use a mock source. +- TOT-005 to TOT-007: assert the counts and the ratio from the collector. +- TOT-009: count the requests the tracker issues per block against a mock. +- TOT-010: measure the hand-off tail while the tracker matches a block. +- TOT-011: register past the cap, then assert the registry size. +- TOT-012 and TOT-013: assert against the report text.