Skip to content

feat(evaluation): SWE-eval hardening - #1268

Closed
Teingi wants to merge 2 commits into
masterfrom
codex/swebench-pro-eval-0814-squash
Closed

feat(evaluation): SWE-eval hardening#1268
Teingi wants to merge 2 commits into
masterfrom
codex/swebench-pro-eval-0814-squash

Conversation

@Teingi

@Teingi Teingi commented Aug 19, 2026

Copy link
Copy Markdown
Member

Which issue or RFC does this PR close?

N/A — evaluation branch integration.

Rationale for this change

Port the accumulated SWE-bench Pro evaluation hardening from the eval-0814 line onto the canonical codex/swebench-pro-eval branch while keeping the destination history compact. The source history is represented as one squash commit after synchronizing the target branch with origin/master.

What changes are included in this PR?

  • Harden evaluation runner lifecycle, workspace preparation, checkpoint seeding, provider credentials, network relay setup, finalization, and resource cleanup.
  • Update the evaluation web API, worker, store, controls, reporting, usage accounting, configuration, and console behavior.
  • Add deployment service files and the source559 recovery helper.
  • Align Codex recall behavior and runtime integration details used by evaluation runs.
  • Update contract, unit, and web tests for the migrated behavior.
  • Add the multi-agent context runtime research note.

Are there any user-facing changes?

Yes. Evaluation operators receive updated runner and console behavior, deployment configuration, retry/finalization handling, and operational diagnostics. This PR does not intentionally change the public PowerContext HTTP contract or persisted application formats.

How was this change tested?

Full automated tests were not run in this environment because uv is not installed. Git validation confirmed a clean worktree, a single squash commit relative to codex/swebench-pro-eval, no imported eval-0814 commit history, and no docs/superpowers content in the PR diff.

AI usage statement

OpenAI Codex was used to inspect branch history, perform the squash migration, resolve Git operations, and prepare this pull request.

@Teingi Teingi changed the title feat(evaluation): port eval-0814 hardening feat(evaluation): SWE-eval hardening Aug 19, 2026
@Teingi
Teingi changed the base branch from codex/swebench-pro-eval to master August 19, 2026 08:58
@Teingi Teingi closed this Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant