Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions release/conformance/integrations/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,12 @@ non-closing evidence. The validator accepts `status: passed` only when every
applicable case and teardown is recorded as passed; maintainer review still
establishes whether the recorded hashes correspond to the restricted evidence.

After an independent operator completes the frozen journey, use
[`pilot-report.template.md`](pilot-report.template.md) to publish the bounded
outcome, findings, and public triage links without copying restricted evidence
into the repository. A plan or dry run is not evidence, and one pilot is not
proof of broad production readiness.

## Review the source packet

- [`profiles/opencrvs-dci-v1.9.profile.json`](profiles/opencrvs-dci-v1.9.profile.json)
Expand Down
84 changes: 84 additions & 0 deletions release/conformance/integrations/pilot-report.template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# External integration pilot report

Use this template only after an independent operator has completed the frozen
pilot journey. Publish a concise account of the outcome and link the validated,
sanitized result. Do not include credentials, network origins, operator or
source identifiers, record identifiers, raw audits, private evidence, or links
to restricted evidence.

Plans, dry runs, fixture runs, and source-built branch runs are not pilot
evidence. One completed pilot is not proof of broad production readiness.

## Closing evidence

- Sanitized run result: [link to the public result accepted by
`python3 release/scripts/integration-e2-runner.py validate`]
- Frozen Registry Stack candidate: [link to the published release]
- Independent operator: [confirmed or not confirmed; do not identify the
operator]
- Owner-approved non-production source: [confirmed or not confirmed; do not
identify the owner or source]
- Integration profile and reviewed operation: [safe public summary; the exact
values remain in the sanitized result]
- Supported topology and journey: [safe public summary]
- Generated Registry YAML edited by hand: [no, or explain why the pilot does
not close]
- Maintainer comparison of public hashes and flags with restricted evidence:
[confirmed or not confirmed; do not identify the maintainer or expose
restricted evidence]
- Overall outcome: [passed, failed, or incomplete]

Issue closure still requires a frozen published candidate, an independent
operator, an owner-approved source, and a confirmed maintainer comparison of
the public hashes and flags with the generated project, source audit records,
redaction report, and teardown evidence. A plan or maintainer-run substitute
cannot satisfy any of these requirements.

## What the pilot did and did not prove

The pilot showed: [summarize only the bounded journey completed from published
artifacts, including offline checks and the selected Relay and, when
applicable, Notary flow].

The pilot did not show: [summarize excluded versions, operations, topologies,
scale, availability, recovery, or other support claims]. It is not upstream
product certification, general country-system conformance, a security audit,
or evidence that Registry Stack is broadly production-ready.

## Findings and triage

Use safe summaries and public issue or pull-request links. Record `not
exercised` rather than inferring success.

| Area | Exercised | Sanitized outcome | Public triage links |
|---|---|---|---|
| Operator handoff and independence | [yes/no] | [summary] | [links or none] |
| Install or deployment | [yes/no] | [summary] | [links or none] |
| Configuration and environment binding | [yes/no] | [summary] | [links or none] |
| Diagnostics and ordinary source failures | [yes/no] | [summary] | [links or none] |
| Upgrade or rollback | [yes/no] | [summary] | [links or none] |
| Restart, teardown, and other operations | [yes/no] | [summary] | [links or none] |
| Security boundaries and redaction | [yes/no] | [summary] | [links or none] |
| Documentation and operator journey | [yes/no] | [summary] | [links or none] |

### Blocking findings

List each blocker with its public triage issue, fix, and independent
re-verification link. Unresolved blockers keep the pilot open.

- [none, or safe finding summary and links]

### Accepted limitations and narrowed support

List each owner-approved limitation or narrowed support claim with its public
decision, operator-guidance update, support-wording update, and review trigger.
Do not use this section to waive a blocker silently.

- [none, or safe limitation summary and links]

## Conclusion

[State whether the bounded pilot closes the external-pilot gate, remains
blocked, or requires a narrower support claim. It cannot close unless the
maintainer comparison above is confirmed. Reiterate any unexercised area that
affects the conclusion.]
53 changes: 53 additions & 0 deletions release/scripts/test_integration_e2_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,59 @@ def write_result(self, result: dict[str, object]) -> Path:
def test_checked_in_packet_is_closed_and_valid(self) -> None:
self.module.validate_packet()

def test_pilot_report_template_preserves_the_public_contract(self) -> None:
readme = (self.module.CONFIG_DIR / "README.md").read_text(encoding="utf-8")
template = (
self.module.CONFIG_DIR / "pilot-report.template.md"
).read_text(encoding="utf-8")
normalized_template = " ".join(template.split())
self.assertIn("(pilot-report.template.md)", readme)
self.assertIn(
"Do not include credentials, network origins, operator or source "
"identifiers, record identifiers, raw audits, private evidence, or "
"links to restricted evidence.",
normalized_template,
)
self.assertNotIn("Solmara", template)
self.assertIn(
"python3 release/scripts/integration-e2-runner.py validate",
normalized_template,
)
self.assertNotIn("schema-valid public result", normalized_template)
self.assertIn(
"Issue closure still requires a frozen published candidate, an "
"independent operator, an owner-approved source, and a confirmed "
"maintainer comparison of the public hashes and flags with the "
"generated project, source audit records, redaction report, and "
"teardown evidence.",
normalized_template,
)
self.assertIn(
"It cannot close unless the maintainer comparison above is confirmed.",
normalized_template,
)
for text in (
"Sanitized run result:",
"Plans, dry runs",
"One completed pilot is not proof of broad production readiness.",
"Frozen Registry Stack candidate:",
"Independent operator:",
"Owner-approved non-production source:",
"Maintainer comparison of public hashes and flags with restricted "
"evidence:",
"### Blocking findings",
"### Accepted limitations and narrowed support",
"Operator handoff and independence",
"Install or deployment",
"Configuration and environment binding",
"Diagnostics and ordinary source failures",
"Upgrade or rollback",
"Restart, teardown, and other operations",
"Security boundaries and redaction",
"Documentation and operator journey",
):
self.assertIn(text, template)

def test_nested_result_objects_must_remain_closed(self) -> None:
schema = self.module.load_json(self.module.SCHEMA_PATH)
schema["$defs"]["case"]["additionalProperties"] = True
Expand Down
Loading