Skip to content

feat(closes OPEN-10364): make tracers init-based#647

Merged
gustavocidornelas merged 2 commits into
mainfrom
gustavo/open-10364-make-tracers-init-based
May 27, 2026
Merged

feat(closes OPEN-10364): make tracers init-based#647
gustavocidornelas merged 2 commits into
mainfrom
gustavo/open-10364-make-tracers-init-based

Conversation

@gustavocidornelas

Copy link
Copy Markdown
Contributor

Pull Request

Summary

Turns openlayer.lib.init() into an auto-instrumenting entry point.

Calling init() with no extra arguments now detects every installed Openlayer-supported LLM SDK (openai, anthropic, mistral, groq, gemini, oci, azure_content_understanding, litellm, portkey, google_adk) and patches it so that every newly-constructed client is auto-traced.

Existing trace_<x>(client) calls keep working. They're now also idempotent, so mixing auto-instrumentation with explicit wraps no longer double-wraps methods.

Changes

  • Add auto_instrument: bool | list[str] = True parameter to init() in src/openlayer/lib/tracing/tracer.py.
    Procedural (not stored in _tracer_config), evaluated fresh on every call. False short-circuits, a list patches
    only that subset.
  • New module src/openlayer/lib/integrations/_auto.py with: IntegrationSpec dataclass, 10-entry REGISTRY,
    auto_instrument(targets) and unpatch_all() orchestrators, plus shared _patch_class_init / _unpatch_class_init
    helpers that wrap a client class __init__ via functools.wraps (so __wrapped__ makes unpatch trivially reversible).
  • Per-integration _patch_<x>() / _unpatch_<x>() helpers added to 7 tracers (openai_tracer.py, anthropic_tracer.py, mistral_tracer.py, groq_tracer.py, gemini_tracer.py, oci_tracer.py, azure_content_understanding_tracer.py). Each wraps the SDK's client class __init__ so new instances auto-call the
    existing trace_<x>(client) function (the portkey_tracer.py pattern). Covers sync + async + Azure variants where
    applicable.
  • Bundled idempotency fix: every per-instance trace_<x>(client) in 9 tracer files gained a getattr(client, "_openlayer_patched", False) is True entry guard plus a marker assignment before return. Pre-existing correctness
    improvement — without it, calling trace_openai(client) twice nested the wrappers and double-counted every call. The
    strict is True comparison avoids false positives from MagicMock's auto-attribute behavior (caught by the bedrock
    test suite).
  • configure() deprecated alias updated to default auto_instrument=False via kwargs.setdefault. Existing
    configure() callers see no surprise patching on upgrade.
  • Re-export auto_instrument and unpatch_all from openlayer.lib.
  • _is_installed probes the full dotted path via importlib.util.find_spec (not just the top-level segment).
    Fixes a namespace-package collision where google.adk falsely reported installed because google.generativeai was
    installed under the same namespace.

Context

OPEN-10364: Make tracers init-based

Testing

  • Manual testing

Monitoring

  • No expected impact

Notes

  • Behavior change: anyone calling init() today (no args) will now auto-instrument every installed supported SDK
    on their next upgrade. This is the intended behavior — the "init-based" tracer model — but worth a callout in the
    changelog. Users can opt out with init(auto_instrument=False) or by switching to the deprecated configure()
    (which preserves the no-patching default).
  • Pre-existing client instances created before init() are NOT retroactively auto-traced. The patch wraps the
    class __init__, so only future construction goes through it. Class-method-level patching for catching pre-existing
    instances is deferred.
  • Out of scope (deferred to a follow-up): Bedrock auto-instrumentation (boto3 service-client shape needs a
    different mechanism), LangChain auto-instrumentation (callback-based, no clean global hook), and the
    auto_instrument env-var convenience.
  • Patch failure isolation: if one integration's patcher raises, the orchestrator logs a warning and continues
    with the rest. The returned dict[str, bool] reflects per-integration success.

Copy link
Copy Markdown
Contributor

@gustavocidornelas did you want me to review?

@gustavocidornelas gustavocidornelas force-pushed the gustavo/open-10364-make-tracers-init-based branch from 96fc8ee to 792e96c Compare May 19, 2026 17:45
IntegrationSpec(
"portkey",
"portkey_ai",
_patch_via("portkey_tracer", "trace_portkey"),

@viniciusdsmello viniciusdsmello May 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trace_portkey() doesn't have a class-level idempotency guard — it re-assigns Portkey.__init__ = traced_init unconditionally on every call. If init() is called twice with auto_instrument=True (a supported pattern), Portkey.__init__ gets stacked: each call wraps the previous traced_init. The per-instance _openlayer_portkey_patched marker prevents double-tracing of new instances, but the __init__ wrapping itself accumulates a layer per call.

Either add a _CLASS_PATCHED_ATTR-style guard at the top of trace_portkey(), or refactor it to use the shared _patch_class_init(Portkey, _wrap_portkey_instance) helper so it benefits from the same idempotency contract as the other integrations.

Same defect confirmed in _patch_google_adk(): it calls wrapt.wrap_function_wrapper(...) unconditionally for each target (BaseAgent.run_async, BaseLlmFlow._call_llm_async, etc.), and the _wrapped_methods.append(...) tracker is only used for unpatch — it doesn't dedupe on re-patch. A second init() call stacks wrappers on every ADK method too. Worth fixing both in the same pass.

),
IntegrationSpec(
"gemini",
"google.generativeai",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

google.generativeai (package google-generativeai) is the legacy SDK. Google's new "Google Gen AI SDK" is google.genai (package google-genai), and they're actively migrating users to it — the legacy SDK is in maintenance mode.

Worth a follow-up to either:

  • Register a second entry for google.genai pointing at a new tracer, or
  • At minimum, leave an inline TODO so this doesn't get lost.

Not a blocker for this PR, but users on the new SDK will silently get no auto-instrumentation today.

@gustavocidornelas

Copy link
Copy Markdown
Contributor Author

@viniciusdsmello, I addressed your comments. Can you review again? Thanks!

@viniciusdsmello viniciusdsmello left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Clean design — the IntegrationSpec registry + lazy _patch_via lambdas + two-level idempotency markers (_CLASS_PATCHED_ATTR on the class, _openlayer_patched on the instance) is the right shape, and configure() preserving the no-patch default is a thoughtful backward-compat call.

Approving — the inline comments (portkey/google_adk class-level idempotency, Gemini SDK migration) are non-blocking; happy to see them addressed here or as follow-ups.

@gustavocidornelas gustavocidornelas merged commit 818608f into main May 27, 2026
5 checks passed
@gustavocidornelas gustavocidornelas deleted the gustavo/open-10364-make-tracers-init-based branch May 27, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants