docs(plan): eval authoring restructure — promptfoo superset (DRAFT)#1594

Merged

christso merged 45 commits into

mainfrom

plan/promptfoo-aligned-eval-restructure

Jul 2, 2026

Commits on Jul 1, 2026

docs(plan): draft plan to restructure eval authoring as a promptfoo superset

christso
and
claude
committed
docs(plan): workspace as dataset+extension; drop on_run_complete; align lifecycle with promptfoo

christso
and
claude
committed
docs(plan): grader execution model — javascript in-process, python subprocess, code-grader power tool

christso
and
claude
committed
docs(plan): lock owner decisions (#1-#8) + reconcile with PR #1592

christso
and
claude
committed
docs(plan): grading.json contract for llm-rubric (main risk) + note PR #1592 amended

christso
and
claude
committed
docs(plan): reconcile grading.json with agentskills; rename skills extension -> agent_rules

christso
and
claude
committed
docs(plan): output format best-of-each (margin aggregate + vercel transcript/metrics); cleanup inventory; agent-rules kebab

christso
and
claude
committed
docs(plan): resolve index/manifest naming drift; adopt margin-lab .internal/ bundle layout (§6.0.1)

christso
and
claude
committed
docs(plan): drop maintained consolidated export (split bundle is source of truth); keep index.jsonl name

christso
and
claude
committed
docs(plan): merge timing.json into metrics.json (fixes ADR-0011/0012 split duplication)

christso
and
claude
committed

Commits on Jul 2, 2026