docs: structured-output guide does not render its example (dangling "# content is auto-synced" placeholder)

[luciobaiocchi]

Problem

The Structured Output guide at https://docs.openhands.dev/sdk/guides/structured-output does not show its ready-to-run example. The "Ready-to-run Example" section renders only the literal placeholder line:

# content is auto-synced

…and the code block shows "See all 1 lines" instead of the actual example code.

The example itself exists in the SDK repo: https://github.com/OpenHands/software-agent-sdk/blob/main/examples/01_standalone_sdk/56_structured_output.py — it's just never appearing on the rendered docs page.

Root cause

In sdk/guides/structured-output.mdx, the example block was authored as:

```python icon="python" expandable examples/01_standalone_sdk/56_structured_output.py
# content is auto-synced
```

In this repo the icon="python" expandable examples/... fence is just a label — the example code must be inlined manually between the fences. There is no build-time sync step that injects the file contents; .github/scripts/check_documented_examples.py (in software-agent-sdk) only verifies that a reference string exists in the docs, not that the code is present. The author left # content is auto-synced as a placeholder that was never filled in.

Every other guide inlines its example code this way and renders correctly — e.g. sdk/guides/custom-tools.mdx ("See all 224 lines"), sdk/guides/hello-world.mdx, etc.

Fix

Inline the exact contents of examples/01_standalone_sdk/56_structured_output.py between the fences so the guide renders the ready-to-run example like every other guide. (Done in the linked PR.)

Broader note

This suggests a gap: nothing currently catches a "dangling" example block where the reference exists but the body is a placeholder. A lightweight lint that fails when an expandable examples/... fence contains only a placeholder comment (or fewer than N non-comment lines) would prevent regressions. Worth considering as a follow-up.

---

This issue was created by an AI agent (OpenHands) on behalf of @luciobaiocchi.

[luciobaiocchi]

Update: root cause found — the daily docs-sync workflow is broken

Investigation shows the automation that is supposed to keep these code blocks in sync already exists but has been failing on every run since February 2026, which is why this example (and others, e.g. the stale imports fixed in #615) drifted.

The automation that exists

• Workflow: OpenHands/docs/.github/workflows/sync-docs-code-blocks.yml — daily cron (0 2 * * *) + workflow_dispatch.
• Script: OpenHands/docs/.github/scripts/sync_code_blocks.py — scans MDX for ```python ... expandable examples/...py ``` blocks, reads the real file from a checkout of OpenHands/software-agent-sdk, and rewrites the block if it differs. On change it opens an auto-approved, auto-merged PR.

The script itself works

I cloned both repos and ran sync_code_blocks.py locally with AGENT_SDK_PATH set. It does fix this exact file — it inlined examples/01_standalone_sdk/56_structured_output.py into sdk/guides/structured-output.mdx (and updated ~47 other files). So the sync logic is correct.

Why it never lands — the workflow fails before any job runs

Every recent sync-docs-code-blocks run is completed / failure with 0 jobs, and gh run view reports:

✗ This run likely failed because of a workflow file issue.

The last successful sync PR was #323 (2026-02-11). Diffing the workflow at that point vs. current, the structural change that broke it is in the auto-approve / auto-merge steps (introduced by #318 → #330 → #336 → #462 → #473):

# current (broken) — references secrets.* directly in if:
- name: Auto-approve PR
  if: steps.cpr.outputs.pull-request-url && secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC != ''
  env:
    GH_TOKEN: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}

GitHub's official docs are explicit: "Secrets cannot be directly referenced in if: conditionals." GitHub statically validates the workflow file before queueing jobs and rejects secrets.* in step if: — producing the "workflow file issue" / 0-jobs failure. (Reference: https://docs.github.com/actions/using-workflows/workflow-syntax-for-github-actions — the jobs..steps[].if section, and the "Example: Using secrets" which shows the correct pattern.)

The version that last worked (#323) did it the documented way — pass the secret through env: and guard inside the shell:

- name: Auto-approve PR
  if: steps.cpr.outputs.pull-request-url
  env:
    GH_TOKEN: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}
  run: |
    if [ -n "$GH_TOKEN" ]; then
      gh pr review "${{ steps.cpr.outputs.pull-request-url }}" --approve --body "..."
    fi

Fix (two equivalent options)

1. Revert to the shell-guard pattern (proven — it's the last-working version): keep if: steps.cpr.outputs.pull-request-url, move the empty-secret check into if [ -n "$GH_TOKEN" ] inside the run block.
2. GitHub's documented pattern: hoist the secret to a job-level env var and reference env.* (not secrets.*) in if::

jobs:
  sync-code-blocks:
    runs-on: ubuntu-latest
    env:
      BOT_PAT: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}
    steps:
      - name: Auto-approve PR
        if: steps.cpr.outputs.pull-request-url && env.BOT_PAT != ''
        env:
          GH_TOKEN: ${{ env.BOT_PAT }}

Either fix unblocks the daily sync, after which dangling/empty example blocks and stale inlined snippets get corrected automatically (and PR #724 becomes unnecessary for this specific file, though it's still a fine manual fix in the meantime).

Side note — what the sync does not cover

Even once fixed, sync_code_blocks.py only resyncs whole-file expandable examples/...py blocks. It does not catch partial inlined snippets whose import paths drift (the #615 case: openhands.sdk.context.skills → openhands.sdk.skills inside a focus= block, not a whole-file sync). A companion lint that imports/executes inlined snippets, or a check that greps for known-renamed symbols, would close that gap. Worth a separate issue.

---

This comment was posted by an AI agent (OpenHands) on behalf of @luciobaiocchi.

[luciobaiocchi]

Follow-up: "auto-update everything + run snippets in CI"

This expands on the previous comment. The goal: all docs stay in sync with real SDK code, and snippets are proven to actually run. Here's what already exists, what the gaps are, and the minimal way to close them.

What already exists

Capability	Where	Status
Sync whole-file expandable examples/...py blocks from SDK → docs	docs/.github/scripts/sync_code_blocks.py + sync-docs-code-blocks.yml (daily cron)	✅ logic works, ❌ workflow broken (see comment above — secrets in if:)
Run the SDK example scripts end-to-end in CI	software-agent-sdk/.github/workflows/run-examples.yml → tests/examples/test_examples.py (--run-examples)	✅ works (label test-examples or manual); runs each example, requires EXAMPLE_COST marker, 10-min timeout
Verify every example is referenced somewhere in docs	software-agent-sdk/.github/scripts/check_documented_examples.py + check-documented-examples.yml	✅ works (reference-only check)
Generate API reference docs	docs/scripts/generate-api-docs.py (in the same sync workflow)	✅ logic works, ❌ blocked by the same broken workflow

Gap 1 — the sync workflow is broken (blocks everything)

Already diagnosed above. Fixing the two if: ... && secrets.X != '' lines (→ job-level env + env.X in if:, or shell if [ -n "$GH_TOKEN" ]) unblocks the daily whole-file sync. That alone would have auto-fixed the dangling structured-output example.

Gap 2 — partial snippets are not synced (this is what #615 was)

sync_code_blocks.py only matches whole-file blocks of the form:

```python icon="python" expandable examples/.../file.py

It does not touch partial focus= snippets like the one in #615:

```python icon="python" focus={3, 4}
from openhands.sdk.context.skills import load_project_skills   ← stale import path

I verified the regex: a focus= block with no file reference is not matched. So even with the workflow fixed, renamed imports inside hand-written partial snippets drift silently. #615 is exactly this class.

Gap 3 — nothing executes the snippets that appear in docs

tests/examples/test_examples.py runs the SDK example files (examples/**/*.py), not the snippets inlined in MDX. A snippet can compile-fail (e.g. ModuleNotFoundError: No module named 'openhands.sdk.context.skills' — the #615 bug) and nothing catches it, because:

• the SDK example file may have already been fixed, but
• the docs MDX still carries the old partial snippet, and
• no CI ever imports the MDX snippet.

Minimal, layered fix to get "docs auto-updated + proven runnable"

1. Fix the workflow (Gap 1) — unblocks existing whole-file sync + API-docs generation. One-line-class change.

2. Extend the sync script to cover partial snippets (Gap 2). Two options:

   • (a) Discourage partial snippets: lint that rejects python code blocks without a file reference (force every snippet to be expandable examples/... so it's auto-synced). Simplest, but loses the focus= highlighting UX.
   • (b) Anchor partial snippets to a file + line range: e.g. ```python icon="python" focus={3,4} examples/.../file.py ``` and have sync_code_blocks.py extract those exact lines from the SDK file. Keeps focus=, makes it auto-synced. This is the "coolest idea" made real — every visible snippet is a slice of a real, running file.

3. Compile-check / run-check inlined snippets in CI (Gap 3). The cheapest version that would have caught docs: fix load_skills_from_dir import path #615:

   • Add a docs-CI job that extracts every ```python block from MDX, writes each to a temp .py, and runs python -m py_compile (or ruff / python -c "import ast; ast.parse(...)"). A ModuleNotFoundError on an import fails the check. This is fast, needs no LLM, no API keys.
   • For snippets that are meant to be fully executable (the expandable whole-file ones), optionally run them via the existing tests/examples/test_examples.py harness — but note those already run from the SDK repo; the docs copy just needs to stay byte-identical (which Gap 1/2 already enforce).
   • Strict version: actually import/execute snippets that are self-contained, using the SDK installed from main. Heavier; gate behind a label like the existing test-examples.

TL;DR

• "Auto-update all docs from real files": already built (sync_code_blocks.py), just broken (workflow secrets in if:). Fix that and whole-file blocks stay current automatically.
• "Run snippets to prove the code works": not built for docs snippets — only for SDK example files (run-examples.yml). A lightweight py_compile/AST lint over extracted MDX snippets in docs CI would have caught docs: fix load_skills_from_dir import path #615, and is cheap and LLM-free.
• The docs: fix load_skills_from_dir import path #615 class specifically needs Gap 2 (sync partial/focus= snippets) or Gap 3 (compile-check snippets) to be caught automatically.

Happy to open a PR for any/all of these — the workflow fix (Gap 1) is the highest-leverage and lowest-risk starting point.

---

This comment was posted by an AI agent (OpenHands) on behalf of @luciobaiocchi.