feat(observability): expose projected mark payloads to OpenInference

[bbednarski9]

Overview

Expose Relay Mark payloads through standard OpenInference attributes when a Mark is projected as a TOOL span. This lets OpenInference consumers such as Phoenix display a canonical tool name, structured output, and metadata while retaining Relay's typed Mark attributes for filtering and inspection.

This builds on the optional TOOL projection introduced by #382 and the typed Mark attribute projection now present on main. It does not change native Mark event projection or introduce OpenTelemetry-specific concepts into Relay's core event model.

• I confirm this contribution is my own work, or I have the right to submit it under this project's license.
• I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

For TOOL-projected Marks, the OpenInference exporter now adds:

• tool.name and tool_call.function.name from the canonical Mark name.
• output.value containing the complete serialized Mark data when data is present.
• output.mime_type = application/json when output.value is emitted.
• metadata containing the complete serialized Mark metadata when metadata is present.

The existing Relay attributes remain authoritative and are preserved, including typed nemo_relay.mark.data.* and nemo_relay.mark.metadata.* attributes plus the Mark category, UUID, parent, and orphan information. Missing data or metadata does not create empty OpenInference payload attributes.

Native span-event Marks remain unchanged: the OpenInference TOOL aliases are added only by the explicit TOOL-projection path. Generic OTEL, ATOF, ATIF, plugin emission, parentage, and timing behavior are unchanged. This is additive and has no public configuration, wire-schema, or plugin API changes.

Validation on the rebased branch:

• cargo fmt --all -- --check
• git diff --check origin/main...HEAD
• cargo test -p nemo-relay 'observability::openinference::tests::tool_projection_' -- --nocapture — 5 passed
• cargo test -p nemo-relay registered_subscriber_emits_spans_for_scope_push_pop_and_marks -- --nocapture — 2 passed
• cargo test -p nemo-relay
• cargo clippy -p nemo-relay --all-targets -- -D warnings
• End-to-end Phoenix 13.22.2 smoke over OTLP/HTTP, including Phoenix API readback and visual UI inspection — confirmed an AGENT parent with a zero-duration TOOL Mark child, correct parentage, standard OpenInference tool/output/metadata fields, and preserved typed nemo_relay.mark.* attributes.

Breaking changes: none.

Where should the reviewer start?

Start with process_mark_as_tool and push_projected_mark_attributes in crates/core/src/observability/openinference.rs. The focused behavior and regression coverage are in crates/core/tests/unit/observability/openinference_tests.rs, including nested payloads, missing optional payloads, typed Relay attributes, parented TOOL spans, and unchanged native event projection.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• Closes Expose projected mark payloads through OpenInference semantic attributes #402

Summary by CodeRabbit

• Enhancements
  • Improved OpenInference tool event details with consistent tool and function identification for tool-projected marks.
  • Tool-projected marks now project serialized output values, application/json MIME type, and metadata when mark data/metadata are available.
  • When mark data/metadata are missing, the corresponding output/metadata fields are no longer emitted.
  • Removed the obsolete orphan-specific attribute from projected tool events.
• Tests
  • Strengthened unit test coverage and assertions for mark-to-tool span projection behavior, including partial data/metadata cases.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Walkthrough

Tool-projected marks now emit standard OpenInference tool identity, serialized output, MIME type, and metadata attributes when payloads exist. Generic mark events remain unchanged, and absent payloads omit standard output and metadata fields.

Changes

Projected mark OpenInference attributes

Layer / File(s)	Summary
Add projected mark attributes crates/core/src/observability/openinference.rs	Tool-projected marks use a helper to add tool identity and optional serialized data, MIME type, and metadata fields while removing orphan tagging.
Validate payload-dependent attributes crates/core/tests/unit/observability/openinference_tests.rs	Tests cover enriched, absent, data-only, and metadata-only payloads, plus exclusion of tool attributes from regular mark events.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• NVIDIA/NeMo-Relay#437: Updates the same tool-mark processing path with session identity and correlation attributes.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	Issue #402 requires retaining existing Relay attributes, but TOOL projection now drops the orphan-specific attribute.	Restore nemo_relay.mark.orphan in TOOL projection, or update the issue if orphan handling is intentionally changing.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	The changes stay focused on OpenInference TOOL-projected marks and their tests, with no obvious unrelated code paths added.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check	✅ Passed	The title uses Conventional Commits format, is lowercase, concise, and accurately summarizes the OpenInference mark projection change.
Description check	✅ Passed	The description follows the template with Overview, Details, reviewer start guidance, checklist items, and a related issue reference.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Fern docs preview: https://nvidia-preview-pull-request-403.docs.buildwithfern.com/nemo/relay (​https://nvidia-preview-pull-request-403.docs.buildwithfern.com/nemo/relay​)

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/core/src/observability/openinference.rs`:
- Around line 729-732: Remove the orphan computation and conditional
`nemo_relay.mark.orphan` attribute from the Tool-projected mark path around
`push_projected_mark_attributes`; retain orphan tagging on the native
generic-mark path. Update the existing orphan test fixture to assert that
Tool-projected marks omit this attribute.

In `@crates/core/tests/unit/observability/openinference_tests.rs`:
- Around line 2637-2671: Extend the Tool projection tests around
tool_projection_omits_standard_payload_attributes_when_mark_payload_is_absent
with separate data-only and metadata-only mark cases. Assert that data-only
emits the output.* attributes without metadata, while metadata-only emits
metadata without output.*, preserving the existing assertions for both-present
and both-absent marks.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 71407ff9-fdfa-45d1-83ba-3fa371719c3c

📥 Commits

Reviewing files that changed from the base of the PR and between 426f2fe and c7a6b2e.

📒 Files selected for processing (2)

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

📜 Review details

⏰ Context from checks skipped due to timeout. (2)

• GitHub Check: Check / Run
• GitHub Check: Preview docs

🧰 Additional context used

📓 Path-based instructions (14)

**/*.rs

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

**/*.rs: Any Rust change must run just test-rust
Any Rust change must run cargo fmt --all
Any Rust change must run cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all for all FFI work since it is Rust work
Run just test-rust to validate FFI changes
Run cargo clippy --workspace --all-targets -- -D warnings to enforce strict linting on FFI work

When Rust files changed as part of Go work, also run cargo fmt --all, just test-rust, and cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all when Rust files are changed as part of Node work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files are changed as part of Node work
Run just test-rust when Rust files are changed as part of Node work

When changing the core Rust runtime or Rust-facing API surface, format Rust code with cargo fmt (rustfmt defaults), keep cargo clippy -- -D warnings clean, and satisfy cargo deny check per deny.toml.

**/*.rs: If any Rust code changed, always run just test-rust.
If any Rust code changed, also run cargo fmt --all.
If any Rust code changed, also run cargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, run cargo fmt --all and cargo clippy --workspace --all-targets -- -D warnings even if relying on pre-commit.

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

{crates/core,crates/adaptive}/**/*

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

Changes to crates/core or crates/adaptive must run the full language matrix

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

crates/core/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)

If the change touched crates/core or shared runtime semantics, also use validate-change for broader validation

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

**/*.{rs,py}

📄 CodeRabbit inference engine (AGENTS.md)

Follow binding naming conventions in Rust and Python: use snake_case.

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

**/*.{rs,py,js,mjs,cjs,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{rs,py,js,mjs,cjs,ts,tsx}: Use Json = serde_json::Value in Rust-facing runtime APIs where the existing code expects JSON payloads.
Use Result<T> with FlowError in core runtime paths, and keep errors explicit and binding-appropriate at the wrapper layer.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

**/*.{rs,py,go,js,ts,c,h}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Use language-appropriate naming conventions: Rust snake_case, C FFI exports prefixed nemo_relay_, Go PascalCase, Node.js camelCase, and Python snake_case.

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

**/*.{rs,go,js,ts}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding // comment form.

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

{crates/**/src/**/*.rs,python/**/*.py}

📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)

Do not add tests under src; Rust tests belong in crate tests/ trees, and Python SDK tests belong under python/tests.

Files:

• crates/core/src/observability/openinference.rs

crates/core/src/observability/{atif,otel,openinference}.rs

📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)

When changing the core event model, emitted fields, exporter behavior, subscriber config, or lifecycle, keep the ATIF, OpenTelemetry, and OpenInference implementations in sync.

Files:

• crates/core/src/observability/openinference.rs

**/*

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, use maintain-dynamic-plugins and include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, prefer uv run pre-commit run --files <changed files...>.
Before review or handoff, run uv run pre-commit run --all-files.

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

crates/{core,adaptive}/**/*

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If crates/core or crates/adaptive changed, run the full validation matrix across Rust, Python, Go, and Node.js.

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

**/*.{rs,py,go,js,ts}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If a language surface changed, always run that language's test target even when Rust core did not change.

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

crates/{core,adaptive}/**/*.rs

⚙️ CodeRabbit configuration file

crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.

Files:

• crates/core/src/observability/openinference.rs
• crates/core/tests/unit/observability/openinference_tests.rs

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}

⚙️ CodeRabbit configuration file

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.

Files:

• crates/core/tests/unit/observability/openinference_tests.rs

🔇 Additional comments (3)

crates/core/src/observability/openinference.rs (2)

729-729: 📐 Maintainability & Code Quality

Run the required Rust/core validation matrix before merge.

Run cargo fmt --all, just test-rust, cargo clippy --workspace --all-targets -- -D warnings, cargo deny check, validate-change, the full Rust/Python/Go/Node matrix, and uv run pre-commit run --all-files. Also verify SPDX headers on both changed Rust files.

As per coding guidelines, “Any Rust change must run just test-rust,” cargo fmt --all, and cargo clippy --workspace --all-targets -- -D warnings; crates/core changes require broader and full-language validation.

Source: Coding guidelines

---

1535-1547: LGTM!

crates/core/tests/unit/observability/openinference_tests.rs (1)

1064-1068: LGTM!

Also applies to: 2549-2550, 2594-2624

[bbednarski9]

/ok to test c1675d5

[coderabbitai]

♻️ Duplicate comments (1)

crates/core/src/observability/openinference.rs (1)

717-732: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Orphan tagging still present on the Tool-projection path.

Lines 719 and 730-732 still compute orphan and conditionally push nemo_relay.mark.orphan, even though push_projected_mark_attributes (line 729) was added specifically to carry the Tool-projection contract, which per the PR objectives keeps orphan tagging only on the native generic-mark path. The line_range_change_details for this segment describes the conditional as already replaced, but the code shown still contains it — this looks unresolved from the prior review round. Note the existing test (Context snippet 1) always establishes a parent span, so orphan is never true there and the removal is not actually exercised/verified.

🐛 Proposed fix

 fn process_mark_as_tool(&mut self, event: &Event) {
     let timestamp = to_system_time(*event.timestamp());
-    let orphan = self.find_parent_span(event).is_none();
     let mut attributes = mark_attributes(event);
     if event.name() == "session.start" {
         push_session_identity_attributes(&mut attributes, event);
     }
     attributes.push(KeyValue::new("nemo_relay.mark.projection", "tool"));
     attributes.push(KeyValue::new(
         oi::OPENINFERENCE_SPAN_KIND,
         OpenInferenceSpanKind::Tool,
     ));
     push_projected_mark_attributes(&mut attributes, event);
-    if orphan {
-        attributes.push(KeyValue::new("nemo_relay.mark.orphan", true));
-    }
     apply_attribute_mappings(&mut attributes, &self.attribute_mappings);

Also worth adding an orphan-mark test case (no parent) to confirm nemo_relay.mark.orphan is truly absent on the Tool path.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/core/src/observability/openinference.rs` around lines 717 - 732,
Remove the orphan computation and conditional nemo_relay.mark.orphan attribute
from process_mark_as_tool, leaving orphan tagging only in the native
generic-mark path while preserving push_projected_mark_attributes. Add or update
a Tool-projection test with no parent span and assert that
nemo_relay.mark.orphan is absent.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@crates/core/src/observability/openinference.rs`:
- Around line 717-732: Remove the orphan computation and conditional
nemo_relay.mark.orphan attribute from process_mark_as_tool, leaving orphan
tagging only in the native generic-mark path while preserving
push_projected_mark_attributes. Add or update a Tool-projection test with no
parent span and assert that nemo_relay.mark.orphan is absent.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 9b5f1333-1d35-4486-8c03-a9dc69eb14ae

📥 Commits

Reviewing files that changed from the base of the PR and between 16fee71 and c1675d5.

📒 Files selected for processing (1)

• crates/core/src/observability/openinference.rs

📜 Review details

⏰ Context from checks skipped due to timeout. (2)

• GitHub Check: Check / Run
• GitHub Check: Preview docs

🧰 Additional context used

📓 Path-based instructions (13)

**/*.rs

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

**/*.rs: Any Rust change must run just test-rust
Any Rust change must run cargo fmt --all
Any Rust change must run cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all for all FFI work since it is Rust work
Run just test-rust to validate FFI changes
Run cargo clippy --workspace --all-targets -- -D warnings to enforce strict linting on FFI work

When Rust files changed as part of Go work, also run cargo fmt --all, just test-rust, and cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all when Rust files are changed as part of Node work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files are changed as part of Node work
Run just test-rust when Rust files are changed as part of Node work

When changing the core Rust runtime or Rust-facing API surface, format Rust code with cargo fmt (rustfmt defaults), keep cargo clippy -- -D warnings clean, and satisfy cargo deny check per deny.toml.

**/*.rs: If any Rust code changed, always run just test-rust.
If any Rust code changed, also run cargo fmt --all.
If any Rust code changed, also run cargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, run cargo fmt --all and cargo clippy --workspace --all-targets -- -D warnings even if relying on pre-commit.

Files:

• crates/core/src/observability/openinference.rs

{crates/core,crates/adaptive}/**/*

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

Changes to crates/core or crates/adaptive must run the full language matrix

Files:

• crates/core/src/observability/openinference.rs

crates/core/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)

If the change touched crates/core or shared runtime semantics, also use validate-change for broader validation

Files:

• crates/core/src/observability/openinference.rs

**/*.{rs,py}

📄 CodeRabbit inference engine (AGENTS.md)

Follow binding naming conventions in Rust and Python: use snake_case.

Files:

• crates/core/src/observability/openinference.rs

**/*.{rs,py,js,mjs,cjs,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{rs,py,js,mjs,cjs,ts,tsx}: Use Json = serde_json::Value in Rust-facing runtime APIs where the existing code expects JSON payloads.
Use Result<T> with FlowError in core runtime paths, and keep errors explicit and binding-appropriate at the wrapper layer.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.

Files:

• crates/core/src/observability/openinference.rs

**/*.{rs,py,go,js,ts,c,h}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Use language-appropriate naming conventions: Rust snake_case, C FFI exports prefixed nemo_relay_, Go PascalCase, Node.js camelCase, and Python snake_case.

Files:

• crates/core/src/observability/openinference.rs

**/*.{rs,go,js,ts}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding // comment form.

Files:

• crates/core/src/observability/openinference.rs

{crates/**/src/**/*.rs,python/**/*.py}

📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)

Do not add tests under src; Rust tests belong in crate tests/ trees, and Python SDK tests belong under python/tests.

Files:

• crates/core/src/observability/openinference.rs

crates/core/src/observability/{atif,otel,openinference}.rs

📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)

When changing the core event model, emitted fields, exporter behavior, subscriber config, or lifecycle, keep the ATIF, OpenTelemetry, and OpenInference implementations in sync.

Files:

• crates/core/src/observability/openinference.rs

**/*

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, use maintain-dynamic-plugins and include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, prefer uv run pre-commit run --files <changed files...>.
Before review or handoff, run uv run pre-commit run --all-files.

Files:

• crates/core/src/observability/openinference.rs

crates/{core,adaptive}/**/*

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If crates/core or crates/adaptive changed, run the full validation matrix across Rust, Python, Go, and Node.js.

Files:

• crates/core/src/observability/openinference.rs

**/*.{rs,py,go,js,ts}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If a language surface changed, always run that language's test target even when Rust core did not change.

Files:

• crates/core/src/observability/openinference.rs

crates/{core,adaptive}/**/*.rs

⚙️ CodeRabbit configuration file

crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.

Files:

• crates/core/src/observability/openinference.rs

🔇 Additional comments (1)

crates/core/src/observability/openinference.rs (1)

1056-1061: LGTM!

[willkill07]

/merge