feat(closes OPEN-10364): make tracers init-based#647
Conversation
|
@gustavocidornelas did you want me to review? |
96fc8ee to
792e96c
Compare
| IntegrationSpec( | ||
| "portkey", | ||
| "portkey_ai", | ||
| _patch_via("portkey_tracer", "trace_portkey"), |
There was a problem hiding this comment.
trace_portkey() doesn't have a class-level idempotency guard — it re-assigns Portkey.__init__ = traced_init unconditionally on every call. If init() is called twice with auto_instrument=True (a supported pattern), Portkey.__init__ gets stacked: each call wraps the previous traced_init. The per-instance _openlayer_portkey_patched marker prevents double-tracing of new instances, but the __init__ wrapping itself accumulates a layer per call.
Either add a _CLASS_PATCHED_ATTR-style guard at the top of trace_portkey(), or refactor it to use the shared _patch_class_init(Portkey, _wrap_portkey_instance) helper so it benefits from the same idempotency contract as the other integrations.
Same defect confirmed in _patch_google_adk(): it calls wrapt.wrap_function_wrapper(...) unconditionally for each target (BaseAgent.run_async, BaseLlmFlow._call_llm_async, etc.), and the _wrapped_methods.append(...) tracker is only used for unpatch — it doesn't dedupe on re-patch. A second init() call stacks wrappers on every ADK method too. Worth fixing both in the same pass.
| ), | ||
| IntegrationSpec( | ||
| "gemini", | ||
| "google.generativeai", |
There was a problem hiding this comment.
google.generativeai (package google-generativeai) is the legacy SDK. Google's new "Google Gen AI SDK" is google.genai (package google-genai), and they're actively migrating users to it — the legacy SDK is in maintenance mode.
Worth a follow-up to either:
- Register a second entry for
google.genaipointing at a new tracer, or - At minimum, leave an inline TODO so this doesn't get lost.
Not a blocker for this PR, but users on the new SDK will silently get no auto-instrumentation today.
|
@viniciusdsmello, I addressed your comments. Can you review again? Thanks! |
viniciusdsmello
left a comment
There was a problem hiding this comment.
LGTM. Clean design — the IntegrationSpec registry + lazy _patch_via lambdas + two-level idempotency markers (_CLASS_PATCHED_ATTR on the class, _openlayer_patched on the instance) is the right shape, and configure() preserving the no-patch default is a thoughtful backward-compat call.
Approving — the inline comments (portkey/google_adk class-level idempotency, Gemini SDK migration) are non-blocking; happy to see them addressed here or as follow-ups.
Pull Request
Summary
Turns
openlayer.lib.init()into an auto-instrumenting entry point.Calling
init()with no extra arguments now detects every installed Openlayer-supported LLM SDK (openai,anthropic,mistral,groq,gemini,oci,azure_content_understanding,litellm,portkey,google_adk) and patches it so that every newly-constructed client is auto-traced.Existing
trace_<x>(client)calls keep working. They're now also idempotent, so mixing auto-instrumentation with explicit wraps no longer double-wraps methods.Changes
auto_instrument: bool | list[str] = Trueparameter toinit()insrc/openlayer/lib/tracing/tracer.py.Procedural (not stored in
_tracer_config), evaluated fresh on every call.Falseshort-circuits, a list patchesonly that subset.
src/openlayer/lib/integrations/_auto.pywith:IntegrationSpecdataclass, 10-entryREGISTRY,auto_instrument(targets)andunpatch_all()orchestrators, plus shared_patch_class_init/_unpatch_class_inithelpers that wrap a client class
__init__viafunctools.wraps(so__wrapped__makes unpatch trivially reversible)._patch_<x>()/_unpatch_<x>()helpers added to 7 tracers (openai_tracer.py,anthropic_tracer.py,mistral_tracer.py,groq_tracer.py,gemini_tracer.py,oci_tracer.py,azure_content_understanding_tracer.py). Each wraps the SDK's client class__init__so new instances auto-call theexisting
trace_<x>(client)function (theportkey_tracer.pypattern). Covers sync + async + Azure variants whereapplicable.
trace_<x>(client)in 9 tracer files gained agetattr(client, "_openlayer_patched", False) is Trueentry guard plus a marker assignment before return. Pre-existing correctnessimprovement — without it, calling
trace_openai(client)twice nested the wrappers and double-counted every call. Thestrict
is Truecomparison avoids false positives fromMagicMock's auto-attribute behavior (caught by the bedrocktest suite).
configure()deprecated alias updated to defaultauto_instrument=Falseviakwargs.setdefault. Existingconfigure()callers see no surprise patching on upgrade.auto_instrumentandunpatch_allfromopenlayer.lib._is_installedprobes the full dotted path viaimportlib.util.find_spec(not just the top-level segment).Fixes a namespace-package collision where
google.adkfalsely reported installed becausegoogle.generativeaiwasinstalled under the same namespace.
Context
OPEN-10364: Make tracers init-based
Testing
Monitoring
Notes
init()today (no args) will now auto-instrument every installed supported SDKon their next upgrade. This is the intended behavior — the "init-based" tracer model — but worth a callout in the
changelog. Users can opt out with
init(auto_instrument=False)or by switching to the deprecatedconfigure()(which preserves the no-patching default).
init()are NOT retroactively auto-traced. The patch wraps theclass
__init__, so only future construction goes through it. Class-method-level patching for catching pre-existinginstances is deferred.
different mechanism), LangChain auto-instrumentation (callback-based, no clean global hook), and the
auto_instrumentenv-var convenience.with the rest. The returned
dict[str, bool]reflects per-integration success.