raintree-technology · zacharyr0th · Jul 16, 2026 · Jul 16, 2026
diff --git a/Makefile b/Makefile
@@ -1,4 +1,4 @@
-.PHONY: clean clean-pyc clean-build clean-test help test test-inventory test-all-local benchmark benchmark-quick benchmark-parallel benchmark-compare benchmark-matrix benchmark-raindrop license-year metrics metrics-check metadata-check metadata-sync lint format release-pr release-publish release-publish-replace-tag release-dispatch
+.PHONY: clean clean-pyc clean-build clean-test help test test-inventory test-all-local benchmark benchmark-quick benchmark-parallel benchmark-compare benchmark-matrix benchmark-raindrop contracts-check contracts-sync license-year metrics metrics-check metadata-check metadata-sync lint format release-pr release-publish release-publish-replace-tag release-dispatch
 
 PYTHON ?= .venv/bin/python
 VERSION_ARG := $(if $(VERSION),--version $(VERSION),)
@@ -31,6 +31,8 @@ help:
 	@echo "metrics-check - fail if METRICS.md is older than METRICS_MAX_AGE_HOURS"
 	@echo "metadata-check - fail if generated release/plugin metadata is stale"
 	@echo "metadata-sync - refresh generated release/plugin metadata from source"
+	@echo "contracts-check - fail if bundled contract schemas are stale"
+	@echo "contracts-sync - regenerate bundled contract schemas"
 	@echo "lint - check style with ruff"
 	@echo "format - format code with ruff"
 	@echo "release-pr - push current release branch and open a protected-main PR"
@@ -127,7 +129,13 @@ metadata-check:
 metadata-sync:
 	$(PYTHON) scripts/sync_release_metadata.py --write
 
-lint: metadata-check
+contracts-check:
+	$(PYTHON) scripts/generate_contract_schemas.py --check
+
+contracts-sync:
+	$(PYTHON) scripts/generate_contract_schemas.py --write
+
+lint: metadata-check contracts-check
 	$(PYTHON) -m ruff check .
 
 format: license-year metadata-sync

diff --git a/README.md b/README.md
@@ -15,6 +15,11 @@ web sources an agent depends on, sync them into cited context packs, diff what
 changed, and export reproducible context for Cursor, Claude, OpenAI,
 LlamaIndex, LangChain, MCP clients, and RAG pipelines.
 
+Architecturally, DocPull is the evidence and acquisition engine: it acquires,
+versions, cites, hashes, and replays evidence. Downstream products own
+scheduling, review, approved claims, legal conclusions, and notifications. See
+the [architecture decision](docs/adr/0001-evidence-acquisition-engine.md).
+
 The core workflow is a `docpull.yaml` plus a `.docpull/context.lock.json`,
 similar in spirit to code dependency manifests and lockfiles:
 
@@ -170,6 +175,11 @@ docpull standards-pack rfc:9110 -o packs/standard
 docpull dataset-pack ./metrics.csv -o packs/dataset
 docpull transcript-pack ./meeting.vtt -o packs/transcript
 docpull wiki-pack wiki:Web_scraping -o packs/wiki
+docpull brand-pack example.com -o packs/brand
+docpull product-pack https://example.com/pricing -o packs/product
+docpull styleguide-pack example.com -o packs/styleguide
+docpull image-pack example.com -o packs/visuals
+docpull policy-pack example.com -o packs/policies
 docpull pack prepare packs/docs --eval-grade
 docpull pack validate packs/docs --level eval
 docpull export packs/docs --format openai-vector-jsonl -o exports/openai.jsonl
@@ -444,6 +454,11 @@ special handling for common web, documentation, and API surfaces.
 | Local datasets | `docpull dataset-pack` emits bounded schema, exact row counts where streamable, column, null-count, and sample summaries |
 | Transcripts | `docpull transcript-pack` emits timestamped segment records from VTT, SRT, text, JSON, or direct transcript URLs |
 | Wikimedia / Wikipedia | `docpull wiki-pack` emits MediaWiki REST page metadata, license/revision metadata, and section-level records |
+| Brand evidence | `docpull brand-pack` emits cited identity, firmographic, social, logo, and color observations |
+| Product and pricing | `docpull product-pack` emits plans, currencies, intervals, trials, feature gates, and page-text provenance |
+| Styleguide | `docpull styleguide-pack` emits cited CSS variables, colors, fonts, spacing, and component samples |
+| Visual assets | `docpull image-pack` emits bounded image candidates and optional validated local assets |
+| Policy documents | `docpull policy-pack` discovers policy/security pages and emits effective dates, stable clauses, and textual change candidates without legal conclusions |
 | Docusaurus / Sphinx / MkDocs | Extracts static article or document regions |
 | VitePress / VuePress / Astro Starlight | Extracts static content regions |
 | GitBook / ReadMe.io | Extracts available article or content regions |
@@ -496,6 +511,16 @@ Every file-backed run writes `corpus.manifest.json` with stable document IDs,
 chunk IDs, hashes, output paths, and chunk counts. See
 [Corpus Manifest](docs/corpus-manifest.md).
 
+Evidence-pack workflows also write `workflow.request.json`,
+`workflow.result.json`, and `artifact.manifest.json`. Export their Draft 2020-12
+schemas with `docpull contracts export -o schemas/`. The complete inventory and
+compatibility rules are in [Public Contracts](docs/contracts.md).
+
+For tracker imports, run `docpull pack intelligence-bundle PACK`. The canonical
+output is `intelligence.bundle.v1.json`; `company_brain.bundle.json` remains a
+compatibility alias. See the
+[competitor-tracker integration contract](docs/competitor-tracker-integration.md).
+
 ## Profiles
 
 ```bash
@@ -612,7 +637,8 @@ part of the package release contract.
   `docpull parse`, `docpull openapi-pack`, `docpull feed-pack`,
   `docpull paper-pack`, `docpull repo-pack`, `docpull package-pack`,
   `docpull standards-pack`, `docpull dataset-pack`, `docpull transcript-pack`,
-  `docpull wiki-pack`,
+  `docpull wiki-pack`, `docpull brand-pack`, `docpull product-pack`,
+  `docpull styleguide-pack`, `docpull image-pack`, `docpull policy-pack`,
   `docpull pack validate`,
   `docpull pack audit`, `docpull export`,
   `docpull serve`, `docpull share`, `docpull render`, `docpull auth check`,

diff --git a/bench/uv.lock b/bench/uv.lock
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
@@ -5,6 +5,38 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [6.2.0] - 2026-07-16
+
+### Added
+- Define DocPull as the evidence and acquisition engine and publish versioned
+  `WorkflowRequest`, `WorkflowResult`, `ArtifactManifest`,
+  `intelligence.bundle.v1`, and `ChangeEvent` contracts with bundled JSON
+  Schemas.
+- Add generic progress, warnings, failures, budget usage, SHA-256 artifact
+  hashes, pack/run identity, and scheduler-neutral replay settings.
+- Promote brand, product, styleguide, image, screenshot, and new policy packs to
+  public CLI, SDK, and MCP workflows, plus declarative project source types for
+  brand, product, styleguide, visual, and policy evidence.
+- Add policy discovery/classification, effective dates, stable clause hashes,
+  and clause-level before/after candidates without legal conclusions.
+- Add deterministic tracker bundles with source snapshots, document versions,
+  precise evidence spans, observations, confidence, evidence strength, source
+  authority, warnings, and change candidates.
+- Add idempotent change events separating structural, textual, and semantic
+  candidates for pricing, positioning, product, security, and policy changes.
+
+### Changed
+- Enrich product pricing extraction with plans, currencies, billing intervals,
+  trials, feature gating, page-text provenance, and incidental/visual price
+  exclusion.
+- Make JSON-only eval preparation retain the required `PACK_CARD.md` artifact.
+- Keep `company_brain.bundle.json` and its SDK builder as compatibility aliases
+  for `intelligence.bundle.v1`.
+
+### Compatibility
+- Validate old packs remain readable and retain legacy result filenames while
+  new workflow sidecars and schema fields are additive.
+
 ## [6.1.0] - 2026-07-14
 
 ### Added

diff --git a/docs/adr/0001-evidence-acquisition-engine.md b/docs/adr/0001-evidence-acquisition-engine.md
@@ -0,0 +1,64 @@
+# ADR 0001: DocPull is the evidence and acquisition engine
+
+- Status: Accepted
+- Date: 2026-07-16
+- Release: 6.2.0
+
+## Decision
+
+DocPull owns reproducible acquisition, normalization, evidence identity, rights
+and provenance metadata, budget accounting, artifact hashing, replay settings,
+and change-candidate production. It is the evidence and acquisition engine.
+
+DocPull does not own tracker scheduling, organizational review state, approved
+claims, legal conclusions, competitive scoring, notifications, or product UI.
+Those are downstream responsibilities. A scheduler may invoke DocPull, but no
+scheduler is required: every workflow and replay configuration runs locally.
+
+```mermaid
+flowchart LR
+    S["Public or explicitly authorized sources"] --> D["DocPull acquisition"]
+    D --> E["Documents, evidence spans, rights, provenance"]
+    E --> W["WorkflowResult and ArtifactManifest"]
+    E --> C["ChangeEvent candidates"]
+    W --> T["Downstream tracker or agent"]
+    C --> T
+    T --> R["Human review and approved claims"]
+```
+
+## Boundary rules
+
+- Local-first HTTP acquisition remains the default.
+- Browser rendering and paid/cloud routes remain off unless explicitly enabled
+  with their existing trust and budget controls.
+- Robots, source policy, rights, cache, lock, and provenance semantics remain
+  part of acquisition rather than downstream interpretation.
+- Machine-derived statements cross the boundary as `observation` or
+  `candidate`, never as an approved claim.
+- Authority tiers describe source relationship only. They are not review or
+  product-specific approval decisions.
+- Policy changes report structural and textual evidence. They do not express a
+  legal opinion.
+- Result contracts contain progress, warnings, failures, budget use, hashes,
+  artifacts, and replay settings so another repository does not need to infer
+  run state from logs.
+
+## Consequences
+
+DocPull publishes versioned JSON Schemas and maintains compatibility for the
+CLI, Python SDK, MCP, existing packs, and `company_brain.bundle.json`. A
+downstream tracker can pin a DocPull version and validate at the repository
+boundary without importing DocPull's internal modules.
+
+Schedulers, approval workflows, and notification systems remain replaceable.
+This keeps monitoring runnable in a local shell or CI job and prevents hosted
+control-plane assumptions from entering the acquisition contract.
+
+## Rejected alternatives
+
+- Embedding competitor-specific approval states in pack records would couple
+  evidence acquisition to one consumer.
+- Making browser rendering automatic would violate the local-first and explicit
+  trust boundary.
+- Treating extracted summaries as approved claims would erase the distinction
+  between evidence and review.
diff --git a/docs/competitor-tracker-integration.md b/docs/competitor-tracker-integration.md
@@ -0,0 +1,56 @@
+# Competitor-tracker integration contract
+
+Pin DocPull `6.2.0` and validate `intelligence.bundle.v1.json` against
+`intelligence-bundle.v1.schema.json` at import time.
+
+## Producer flow
+
+```bash
+docpull sync
+docpull pack intelligence-bundle .docpull/runs/<run-id> \
+  --objective "Track pricing, positioning, product, security, and policy"
+```
+
+Python:
+
+```python
+from pathlib import Path
+from docpull import build_intelligence_bundle
+
+bundle = build_intelligence_bundle(
+    Path(".docpull/runs/run_20260716"),
+    objective="Track competitor changes",
+    market="Developer tools",
+)
+```
+
+MCP clients call `intelligence_bundle`. Existing integrations may continue to
+read `company_brain.bundle.json`; it is an alias containing the v1 fields and a
+deprecated compatibility envelope.
+
+## Import requirements
+
+1. Reject an unknown `contract_version`.
+2. Recompute the canonical bundle core hash and compare `bundle_hash`.
+3. Deduplicate by `bundle_id` and change `idempotency_key`.
+4. Store pack ID, run ID, source snapshot ID, document ID, and document version.
+5. Treat `observations[*].status == "observation"` as unapproved machine output.
+6. Verify each evidence span against the referenced document version before
+   presenting it for review.
+7. Keep `change_candidates` in a review queue; do not infer approval from source
+   authority or confidence.
+8. Surface warnings and retain replay configuration for audit/reproduction.
+
+## Ownership split
+
+DocPull owns acquisition, evidence spans, source snapshots, rights/provenance,
+budget receipts, hashes, and change candidates. Competitor-tracker owns
+scheduling, entity resolution across imported packs, reviewer assignments,
+approval/rejection, product-specific severity, alert delivery, and UI.
+
+## Replay and monitoring
+
+`WorkflowRequest.replay` and `ChangeEvent.replay_configuration` deliberately set
+`scheduler` to null. A local cron job, CI workflow, desktop agent, or hosted
+scheduler can replay the same configuration. Browser and paid-route flags must
+remain false unless the operator explicitly enables and budgets those paths.
diff --git a/docs/context-packs.md b/docs/context-packs.md
@@ -18,6 +18,11 @@ docpull standards-pack rfc:9110 --output-dir packs/standard
 docpull dataset-pack ./metrics.csv --output-dir packs/dataset
 docpull transcript-pack ./meeting.vtt --output-dir packs/transcript
 docpull wiki-pack wiki:Web_scraping --output-dir packs/wiki
+docpull brand-pack example.com --output-dir packs/brand
+docpull product-pack https://example.com/pricing --output-dir packs/product
+docpull styleguide-pack example.com --output-dir packs/styleguide
+docpull image-pack example.com --output-dir packs/visuals
+docpull policy-pack example.com --output-dir packs/policies
 
 docpull pack prepare packs/example --eval-grade
 docpull pack validate packs/example --level eval
@@ -71,6 +76,21 @@ Remote typed lanes support `--cache --cache-dir .docpull-cache/typed-packs`
 for repeatable official API/metadata calls. Python SDK users can call the
 matching `async_build_*_pack` helpers when already inside an event loop.
 
+## Evidence Workflow Protocol
+
+Brand, product, styleguide, visual/image, screenshot, and policy lanes implement
+the common workflow protocol. Their existing result JSON and Markdown remain,
+and every run additionally writes:
+
+- `workflow.request.json` with stable input, policy, budget, and replay fields;
+- `workflow.result.json` with progress, warnings, failures, budget usage,
+  identities, and hashes;
+- `artifact.manifest.json` with byte sizes and SHA-256 for emitted artifacts.
+
+Use `docpull pack intelligence-bundle PACK` to create the deterministic tracker
+import. Use `docpull pack diff OLD NEW` to write `change.events.jsonl` alongside
+the existing diff artifacts.
+
 ## Release Smoke
 
 The full free/local public surface can be checked against real public data with

diff --git a/docs/contracts.md b/docs/contracts.md
@@ -0,0 +1,86 @@
+# Public contract inventory
+
+DocPull 6.2 freezes the existing artifact envelopes and adds transport-neutral
+cross-repository contracts. Bundled schemas live in `src/docpull/schemas/` and
+are installed with the Python package.
+
+Use `docpull contracts list`, `docpull contracts show NAME`, or
+`docpull contracts export -o schemas/` to inspect them.
+
+## Frozen compatibility envelopes
+
+| Contract | Version | Canonical implementation | Compatibility rule |
+|---|---:|---|---|
+| Pack metadata | existing v1/v3 | `*.pack.json`, `pack.v3.schema.json` | Readers continue accepting old pack metadata and unknown additive fields. |
+| Document record | v3 | `DocumentRecord`, `document.v3.schema.json` | Existing field names, IDs, citations, rights, route, and chunk fields remain readable. |
+| Run identity | v1 | `RunIdentity`, `run-identity.v1.schema.json` | Fingerprints remain deterministic and secret-free. |
+| Citation map | v1 | `citations.json`, `citation-map.v1.schema.json` | `citation_id`, URL, title, and record citations remain stable; authority and versions are additive. |
+| Rights | v1 envelope | `rights.manifest.json`, `rights.v1.schema.json` | Unknown remains the conservative default. |
+| Provenance | v1 envelope | `provenance.graph.json`, `provenance.v1.schema.json` | Additive nodes and edges are allowed. |
+| Basis | v2 | `basis.ndjson`, `basis.v2.schema.json` | Legacy basis rows normalize to v2. |
+| Company Brain | compatibility alias | `company_brain.bundle.json` | Alias remains written and readable; canonical contract is `intelligence.bundle.v1`. |
+
+The freeze means existing required fields are not renamed or removed during the
+6.x line. Additive fields and sidecars are allowed. Existing CLI commands and
+concrete SDK builders retain their result payloads while also writing the new
+generic contracts.
+
+## Cross-repository contracts
+
+### `workflow.request.v1`
+
+A scheduler-neutral invocation containing a stable request ID, workflow name,
+input, output location, options, source policy, budget, and replay settings.
+The request ID excludes ephemeral timestamps.
+
+### `workflow.result.v1`
+
+The common result for brand, product, styleguide, visual/image, screenshot, and
+policy workflows. It contains:
+
+- pack and run identities;
+- lifecycle progress events;
+- structured warnings and failures;
+- budget limit, estimated/actual spend, HTTP/cache counts, browser time, and
+  blocked actions;
+- SHA-256 request, manifest, legacy-result, and pack hashes;
+- replay configuration and compatibility artifact paths.
+
+### `artifact.manifest.v1`
+
+A sorted list of named artifacts with relative path, role, media type, bytes,
+and SHA-256. `aggregate_sha256` hashes the canonical sorted entry list. The
+manifest does not hash itself or `workflow.result.json`, avoiding circular
+identity.
+
+### `intelligence.bundle.v1`
+
+The supported tracker-import contract. It contains pack/run identity, source
+snapshots, document versions, precise observations, evidence strength,
+confidence, source authority, warnings, and before/after change candidates.
+The `bundle_hash` covers the canonical v1 core without the self-referential
+bundle ID/hash or legacy Company Brain envelope.
+
+### `change.event.v1`
+
+One idempotent event per changed URL. It carries old/new document IDs and
+hashes, precise old/new evidence, separate structural and textual changes, and
+semantic candidates classified as pricing, positioning, product, security,
+policy, or other. Classifications are candidates requiring review.
+
+## Evidence spans
+
+An evidence span contains the citation and record citation IDs, document ID,
+content-hash document version, URL, zero-based `char_start`/`char_end`, exact
+text, and exact-text SHA-256. Consumers must verify both the document version
+and exact text before promoting an observation.
+
+## Compatibility policy
+
+- JSON consumers must ignore unknown fields.
+- DocPull readers continue to accept old packs.
+- Existing legacy result filenames remain present.
+- New generic files are additive: `workflow.request.json`,
+  `workflow.result.json`, and `artifact.manifest.json`.
+- `company_brain.bundle.json` remains a deterministic compatibility alias.
+- Breaking changes require a new contract version and migration notes.