Skip to content

OTel: classify tool-span error.type into a low-cardinality vocabulary #426

Description

@initializ-mk

Summary

Follow-up from the review of #422. Tool-execution spans set a single flat error.type = "tool_execution_error" on failure (forge-core/runtime/loop.go, tool-call branch). semconv's error.type is meant to be a low-cardinality classification, so operators can group failures by kind ("show me auth failures across all tools"). Refine the single constant into a small, fixed vocabulary.

The full error text is not lost today — it's preserved via toolSpan.RecordError(execErr) + SetStatus(codes.Error, execErr.Error()). This issue is purely about the classification attribute, so it stays additive and low-risk.

Proposed vocabulary

A closed set of low-cardinality values, defaulting to execution_error when nothing more specific is known:

error.type When
timeout ctx deadline / tool wall-clock limit exceeded
cancelled ctx cancelled mid-call (distinct from failure)
auth credential / permission rejection (e.g. MCP 401/403, OAuth failure)
not_found unknown tool, missing binary (cli_execute LookPath), 404 from an API/MCP tool
invalid_args argument validation / schema rejection before execution
egress_blocked request denied by the egress enforcer/allowlist
execution_error default — tool ran and failed for another reason

Keep the set closed and documented; unknown → execution_error (never echo the raw message into error.type, which would blow up cardinality).

Where to classify

e.tools.Execute returns execErr. Map it with errors.Is / errors.As against existing sentinels rather than string matching where possible:

  • context.DeadlineExceededtimeout; context.Canceledcancelled (note: the loop already special-cases ctx cancellation upstream — keep consistent).
  • Egress: the enforcer's denial error (forge-core/securityegress_blocked events already exist) → egress_blocked.
  • MCP tool errors: transport/auth error types in forge-core/mcpauth / not_found.
  • cli_execute: exec.ErrNotFound / LookPath failures → not_found; arg-validation rejections → invalid_args.
  • Fallback → execution_error.

A small classifier helper (e.g. classifyToolError(err) string in forge-core/runtime) keeps loop.go readable and unit-testable in isolation.

Acceptance criteria

  • Tool spans set error.type from the closed vocabulary above; default execution_error.
  • Raw error message never leaks into error.type (cardinality guard); full detail still on RecordError + span status.
  • classifyToolError unit tests cover each mapped category + the default.
  • docs/core-concepts/observability-tracing.md lists the error.type values.

References

  • otel: adopt gen_ai.* semantic conventions on tool + agent spans #422 review note (non-blocking) — origin of this refinement
  • forge-core/runtime/loop.go — tool-call branch, current error.type = "tool_execution_error"
  • forge-core/observability/attrs.goAttrErrorType
  • Existing sentinels to map against: forge-core/security (egress deny), forge-core/mcp (transport/auth), cli_execute (LookPath / arg validation)

No code changes in this issue — design + backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestepic:evalCross-repo observability + evaluation epic

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions