Skip to content

refactor: simplify LiteLLM DirectiveDrafter example boundary - #65

Merged
rlippmann merged 7 commits into
mainfrom
litellm_example_drafter_refactor
Aug 10, 2026
Merged

refactor: simplify LiteLLM DirectiveDrafter example boundary#65
rlippmann merged 7 commits into
mainfrom
litellm_example_drafter_refactor

Conversation

@rlippmann

Copy link
Copy Markdown
Owner

Summary

Refactor the LiteLLM prompt-construction DirectiveDrafter example to consume the current DirectiveDrafter public boundary instead of duplicating drafter behavior in the example.

This PR updates the example to:

  • use DirectiveDrafter.draft_directive(...)
  • provide only a LiteLLM fallback callback that returns raw candidate text
  • consume DraftResult variants explicitly
  • read compiled state directly from engine.premise and engine.policies
  • remove example-owned directive grammar matching, resume/checkpoint behavior, and other runtime-style workflow logic

Why

The example had drifted away from the intended consumer boundary and was still re-implementing pieces of drafter behavior locally. That made the example misleading as documentation and also masked live behavior issues during end-to-end testing.

This change keeps the example focused on the provider/drafter/compiler handoff and updates the tests to verify the public API seam instead of internal helper behavior.

Example metadata

  • Enforcement point: host-side prompt construction from authoritative compiler state
  • Domain: generic assistant request shaping
  • Technology/runtime: Python, LiteLLM, DirectiveDrafter, Ollama-compatible provider flow
  • Example type: cookbook
  • Does this use directive drafter? should normally be no: yes
  • Does this derive state from model output? must be no for primary examples: no

Checklist

  • Repository scaffolding / governance change
  • One primary enforcement point
  • Explicit authoritative state
  • No model-derived state mutation
  • Adversarial stub or equivalent test
  • Observable runtime behavior change
  • Domain vocabulary is natural
  • Framework is secondary to the enforcement point
  • Docs updated
  • Tests/smoke checks added where practical

Notes

Focused validation run for the changed files:

  • uv run --no-sync pytest python/tests/test_litellm_with_directive_drafter.py
  • uv run --no-sync ruff check python/examples/prompt_construction/litellm/with_directive_drafter.py python/tests/test_litellm_with_directive_drafter.py
  • uv run --no-sync mypy python/examples/prompt_construction/litellm/with_directive_drafter.py python/tests/test_litellm_with_directive_drafter.py

Live testing against local Ollama on Monday, August 10, 2026 exposed that the example still over-applies drafted directives on some ordinary conversational turns. This PR keeps the boundary cleanup and test refactor in place, but the live test results indicate there is still follow-up behavior work to do in the example flow.

@rlippmann
rlippmann merged commit ed610cb into main Aug 10, 2026
6 checks passed
@rlippmann
rlippmann deleted the litellm_example_drafter_refactor branch August 10, 2026 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant