otel: adopt gen_ai.* semantic conventions on tool + agent spans

docs/core-concepts/observability-tracing.md

@@ -212,33 +212,46 @@ Forge mixes OTel GenAI semconv with Forge-specific `forge.*` namespaced attribut
 | `forge.correlation_id` | `agent.execute` | inbound `X-Forge-Correlation-Id` |
 | `forge.loop.iteration` | `agent.execute` (set at End) | turn count |
 | `forge.task.final_state` | `agent.execute` (set at End) | `completed` / `failed` / `canceled` |
-| `gen_ai.system` | `agent.execute`, `llm.completion` | `"anthropic"`, `"openai"`, `"ollama"` |
-| `gen_ai.request.model` / `.response.model` | `llm.completion` | provider request/response model |
+| `gen_ai.provider.name` | `agent.execute`, `llm.completion` | `"anthropic"`, `"openai"`, `"ollama"` — current key (see deprecation note below) |
+| `gen_ai.system` | `agent.execute`, `llm.completion` | same value as `gen_ai.provider.name`; **deprecated**, emitted one release for compatibility |
+| `gen_ai.operation.name` | `llm.completion` (`chat`), `tool.<name>` (`execute_tool`) | operation kind |
+| `gen_ai.agent.id` / `.name` / `.version` | `agent.execute` | `forge.yaml` `agent_id` (id + name) / `version` |
+| `gen_ai.conversation.id` | `agent.execute` | Forge session id (A2A task id) |
+| `gen_ai.request.model` | `agent.execute`, `llm.completion` | requested model |
+| `gen_ai.response.model` | `llm.completion` | vendor-reported model (falls back to request model) |
+| `gen_ai.response.id` | `llm.completion` | provider completion id |
 | `gen_ai.usage.input_tokens` / `.output_tokens` | `llm.completion` | provider usage block |
 | `gen_ai.response.finish_reasons` | `llm.completion` | provider stop reason |
-| `forge.tool.name` | `tool.<tool_name>` | tool function name |
-| `forge.tool.error` | `tool.<tool_name>` | error message on failure |
+| `gen_ai.tool.name` | `tool.<tool_name>` | tool function name |
+| `gen_ai.tool.call.id` | `tool.<tool_name>` | LLM-assigned tool-call id |
+| `gen_ai.tool.type` | `tool.<tool_name>` | `function` (builtin/skill) or `extension` (MCP-backed) |
+| `mcp.method.name` | `tool.<tool_name>` (MCP only) | `tools/call` |
+| `error.type` | `tool.<tool_name>` (on failure) | `tool_execution_error` |
 
-Tool errors do **not** fail the outer `agent.execute` span — they surface to the LLM as text and the loop continues. The tool span carries the failure detail so operators can pivot from a trace to the specific failed invocation.
+Tool spans follow the OTel **GenAI semantic conventions** (`gen_ai.tool.*`) — these replaced the former proprietary `forge.tool.*` keys, which never shipped to production. MCP-backed tools (namespaced `<server>__<tool>`) are typed `extension` and additionally carry `mcp.method.name=tools/call`. (`mcp.session.id` / `mcp.protocol.version` require plumbing the MCP manager to the executor and are tracked as a follow-up.)
+
+Tool errors do **not** fail the outer `agent.execute` span — they surface to the LLM as text and the loop continues. The tool span carries the failure detail (`error.type` + span status `Error`) so operators can pivot from a trace to the specific failed invocation.
 
 ### Span content capture
 
 Prompts, completions, tool args, and tool results are **off by default** — Phase 3 spans ship metadata only (provider, model, usage, finish reasons, tool name). Operators who need content attributes for in-trace debugging or supervised-learning corpora opt in via `observability.tracing.capture_content: true` (Phase 3.5 / issue #130).
 
 | `forge.yaml` knob | Span | Attribute keys added when `capture_content: true` |
 |---|---|---|
-| (always) | `llm.completion` | `gen_ai.system`, `gen_ai.request.model`, `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.response.finish_reasons` |
+| (always) | `agent.execute` | `gen_ai.provider.name`, `gen_ai.agent.id`, `gen_ai.agent.name`, `gen_ai.agent.version`, `gen_ai.conversation.id`, `gen_ai.request.model` |
+| `capture_content: true` | `agent.execute` | `gen_ai.tool.definitions` (JSON array of the tool catalog available to the agent — potentially large, hence opt-in) |
+| (always) | `llm.completion` | `gen_ai.operation.name` (`chat`), `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.response.model`, `gen_ai.response.id`, `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.response.finish_reasons` |
 | `capture_content: true` | `llm.completion` | `gen_ai.input.messages` (JSON array of role+content pairs sent to the model), `gen_ai.output.messages` (JSON single-element array of role+content for the model's response) — current OTel GenAI semconv, supersedes the deprecated flat-string `gen_ai.prompt` / `gen_ai.completion` |
-| (always) | `tool.<name>` | `forge.tool.name`, `forge.tool.error` (on failure) |
-| `capture_content: true` | `tool.<name>` | `forge.tool.args` (raw arguments JSON), `forge.tool.result` (raw output) |
+| (always) | `tool.<name>` | `gen_ai.operation.name` (`execute_tool`), `gen_ai.tool.name`, `gen_ai.tool.call.id`, `gen_ai.tool.type`, `mcp.method.name` (MCP only), `error.type` (on failure) |
+| `capture_content: true` | `tool.<name>` | `gen_ai.tool.call.arguments` (raw arguments JSON), `gen_ai.tool.call.result` (raw output), `gen_ai.tool.description` (from the tool definition) |
 
 When `capture_content: true` and `redact: true` (the default when capture is on), attribute values pass through a redactor that scrubs the same vendor secret-token shapes the runtime guardrails default rules cover (Anthropic `sk-ant-…`, OpenAI `sk-…`, GitHub `ghp_/gho_/ghs_/github_pat_…`, AWS `AKIA…`, Slack `xoxb-/xoxp-…`, RSA/EC/OPENSSH/PRIVATE key blocks, Telegram bot tokens). Matched values become `[REDACTED]`. Setting `redact: false` is the enterprise raw-capture path — content is stamped verbatim with the byte cap still applied.
 
 Every captured value is byte-capped at **4 KiB** (below the 5 KiB attribute soft-cap most backends apply). When the input exceeds the cap, the value ends with a `…[truncated:N]` marker where `N` is the original byte length. The marker is **byte-identical** to what the audit payload-capture path emits for the same input, so an operator grepping `[truncated:` across span attributes and audit rows sees aligned output.
 
-**Default posture** (no opt-in): the `gen_ai.input.messages`, `gen_ai.output.messages`, `forge.tool.args`, `forge.tool.result` keys are **absent** from spans — not set to empty string. Backends that gate dashboards on "is this key present?" can distinguish "metadata-only by default" from "operator opted in but the field happened to be empty."
+**Default posture** (no opt-in): the `gen_ai.input.messages`, `gen_ai.output.messages`, `gen_ai.tool.call.arguments`, `gen_ai.tool.call.result`, `gen_ai.tool.description`, and `gen_ai.tool.definitions` keys are **absent** from spans — not set to empty string. Backends that gate dashboards on "is this key present?" can distinguish "metadata-only by default" from "operator opted in but the field happened to be empty."
 
-**OTel semconv versioning note**: the GenAI semantic conventions moved from flat-string (`gen_ai.prompt`, `gen_ai.completion`) to structured (`gen_ai.input.messages`, `gen_ai.output.messages`) attributes. Forge emits only the **current** structured keys. Backends that only recognize the deprecated flat-string attributes will not show prompt / completion text on Forge spans — upgrade the backend's semconv mapping or use a span processor to translate.
+**OTel semconv versioning note**: the GenAI semantic conventions moved from flat-string (`gen_ai.prompt`, `gen_ai.completion`) to structured (`gen_ai.input.messages`, `gen_ai.output.messages`) attributes, and renamed `gen_ai.system` → `gen_ai.provider.name`. Forge emits the **current** structured keys and `gen_ai.provider.name`, and continues to emit the deprecated `gen_ai.system` for one release for compatibility. Backends that only recognize the older attributes should upgrade their semconv mapping or use a span processor to translate.
 
 ### Guardrail spans (issue #161)
 
@@ -268,7 +281,7 @@ Attribute reference:
 
 **Default posture**: `forge.guardrail.evidence` is absent unless `capture_content: true`. The other five attributes are always present when a gate fires (cheap, no PII risk). When tracing is disabled, the noop tracer short-circuits and the spans are not produced at all.
 
-**Content-capture parity**: the evidence attribute uses the exact same `PrepareSpanContent(redact, maxBytes)` pipeline as `gen_ai.input.messages` and `forge.tool.args` — same vendor secret-token scrub, same 4 KiB byte cap, same `…[truncated:N]` marker. Operators get one mental model across all four content streams (LLM input / LLM output / tool args / tool result / guardrail evidence).
+**Content-capture parity**: the evidence attribute uses the exact same `PrepareSpanContent(redact, maxBytes)` pipeline as `gen_ai.input.messages` and `gen_ai.tool.call.arguments` — same vendor secret-token scrub, same 4 KiB byte cap, same `…[truncated:N]` marker. Operators get one mental model across all four content streams (LLM input / LLM output / tool args / tool result / guardrail evidence).
 
 ## End-to-end propagation (Phase 5)
 

forge-cli/runtime/runner.go

@@ -1337,6 +1337,8 @@ func (r *Runner) Run(ctx context.Context) error {
 						Logger:        r.logger,
 						ModelName:     mc.Client.Model,
 						Provider:      mc.Provider,
+						AgentID:       r.cfg.Config.AgentID,
+						AgentVersion:  r.cfg.Config.Version,
 						MaxIterations: 100,
 						CharBudget:    charBudget,
 						FilesDir:      filepath.Join(r.cfg.WorkDir, ".forge", "files"),

forge-core/forgecore_test.go

@@ -54,6 +54,8 @@ func (m *mockToolExecutor) Execute(ctx context.Context, name string, arguments j
 	return "", nil
 }
 
+func (m *mockToolExecutor) IsMCPTool(string) bool { return false }
+
 func (m *mockToolExecutor) ToolDefinitions() []llm.ToolDefinition {
 	var defs []llm.ToolDefinition
 	for name := range m.tools {

forge-core/llm/providers/anthropic.go

@@ -333,6 +333,7 @@ func (c *AnthropicClient) convertMessage(m llm.ChatMessage) anthropicMessage {
 // Anthropic-specific response types.
 type anthropicResponse struct {
 	ID         string                  `json:"id"`
+	Model      string                  `json:"model"`
 	Content    []anthropicContentBlock `json:"content"`
 	StopReason string                  `json:"stop_reason"`
 	Usage      struct {
@@ -375,6 +376,7 @@ func (c *AnthropicClient) parseAnthropicResponse(body io.Reader) (*llm.ChatRespo
 
 	return &llm.ChatResponse{
 		ID:      resp.ID,
+		Model:   resp.Model,
 		Message: msg,
 		Usage: llm.UsageInfo{
 			InputTokens:  resp.Usage.InputTokens,

forge-core/llm/providers/openai.go

@@ -257,6 +257,7 @@ func derivePromptCacheKey(model string, req *llm.ChatRequest) string {
 // openaiResponse is the OpenAI-specific response format.
 type openaiResponse struct {
 	ID      string `json:"id"`
+	Model   string `json:"model"`
 	Choices []struct {
 		Message struct {
 			Role      string         `json:"role"`
@@ -284,7 +285,8 @@ func (c *OpenAIClient) parseOpenAIResponse(body io.Reader) (*llm.ChatResponse, e
 
 	choice := resp.Choices[0]
 	return &llm.ChatResponse{
-		ID: resp.ID,
+		ID:    resp.ID,
+		Model: resp.Model,
 		Message: llm.ChatMessage{
 			Role:      choice.Message.Role,
 			Content:   choice.Message.Content,

forge-core/llm/types.go

@@ -64,6 +64,13 @@ type ChatResponse struct {
 	Message      ChatMessage `json:"message"`
 	Usage        UsageInfo   `json:"usage"`
 	FinishReason string      `json:"finish_reason"`
+	// Model is the model the provider REPORTED generating the response,
+	// parsed from the provider response body when present (Anthropic /
+	// OpenAI both echo "model"). Often identical to the request model but
+	// enterprise routers can substitute a versioned suffix. Surfaced as
+	// the gen_ai.response.model span attribute; empty when the provider
+	// does not report it (the executor falls back to the request model).
+	Model string `json:"model,omitempty"`
 	// Endpoint is the URL the client POSTed to (base URL + provider path).
 	// Set by the provider client so the llm_call audit event can record the
 	// invoked path even when payload capture is off. Internal only (json:"-").

forge-core/observability/attrs.go

@@ -24,9 +24,16 @@ package observability
 // "llm.completion") and reads naturally inline.
 
 const (
-	// ─── GenAI semconv (draft, pinned to OTel semconv 1.26.0 GenAI). ──
+	// ─── GenAI semconv (draft). ──────────────────────────────────────
 	// Backends like Honeycomb / Datadog / Grafana Tempo group LLM
 	// activity by these. Naming follows the OTel GenAI spec exactly.
+	// The core token/model keys below track the 1.26.0 snapshot; the
+	// agent / operation / tool keys added later (provider.name,
+	// operation.name, agent.*, conversation.id, tool.*) track the newer
+	// GenAI registry. Because these are hand-declared string constants
+	// (not imported from the semconv package), a spec bump is this
+	// one-file sweep — the resource-level semconv import in otel.go is
+	// versioned separately and only carries service.* + schema URL.
 
 	// AttrGenAISystem identifies the LLM vendor: "anthropic",
 	// "openai", "ollama", "openai-compatible".
@@ -51,6 +58,70 @@ const (
 	// "max_tokens", "end_turn", etc.
 	AttrGenAIResponseFinishReasons = "gen_ai.response.finish_reasons"
 
+	// AttrGenAIProviderName is the current OTel key for the GenAI vendor
+	// ("anthropic", "openai", "ollama", ...). It supersedes the
+	// deprecated AttrGenAISystem (`gen_ai.system`); Forge emits BOTH for
+	// one release so dashboards keyed on either light up, then drops the
+	// alias. Same value as AttrGenAISystem.
+	AttrGenAIProviderName = "gen_ai.provider.name"
+
+	// AttrGenAIOperationName identifies the operation the span measures —
+	// "chat" on the llm.completion span, "execute_tool" on a tool.<name>
+	// span. Backends group GenAI activity by (operation.name, provider.name).
+	AttrGenAIOperationName = "gen_ai.operation.name"
+
+	// OpChat / OpExecuteTool are the two AttrGenAIOperationName values
+	// Forge emits today.
+	OpChat        = "chat"
+	OpExecuteTool = "execute_tool"
+
+	// AttrGenAIResponseID is the provider's completion id (Anthropic
+	// "msg_…", OpenAI "chatcmpl-…"). Read straight from ChatResponse.ID.
+	AttrGenAIResponseID = "gen_ai.response.id"
+
+	// AttrGenAIConversationID is the stable conversation/thread id. Forge
+	// maps this to the A2A task id — the session-store key that persists a
+	// transcript across turns (.forge/sessions/<task>.json). Per semconv
+	// this MUST be a real thread identifier, never a synthesized UUID or
+	// trace id; the Forge session id qualifies.
+	AttrGenAIConversationID = "gen_ai.conversation.id"
+
+	// AttrGenAIAgent* stamp the agent's identity on the agent.execute
+	// span from forge.yaml. `agent_id` doubles as the human-readable name
+	// (forge.yaml has no separate name field, matching the A2A card's
+	// AgentID fallback), so id and name carry the same value today.
+	AttrGenAIAgentID      = "gen_ai.agent.id"
+	AttrGenAIAgentName    = "gen_ai.agent.name"
+	AttrGenAIAgentVersion = "gen_ai.agent.version"
+
+	// AttrGenAITool* name the tool-call instrumentation on tool.<name>
+	// spans, following the OTel GenAI tool conventions. These REPLACE the
+	// former proprietary `forge.tool.*` keys (which never shipped to
+	// production). Content-bearing tool attributes (arguments / result /
+	// description / definitions) live in the content-capture block below.
+	AttrGenAIToolName   = "gen_ai.tool.name"
+	AttrGenAIToolCallID = "gen_ai.tool.call.id"
+	AttrGenAIToolType   = "gen_ai.tool.type"
+
+	// ToolTypeFunction / ToolTypeExtension are the AttrGenAIToolType
+	// values Forge emits: builtin + skill tools are "function"; MCP-backed
+	// tools (namespaced "<server>__<tool>") are "extension" — an
+	// agent-side bridge to an external system.
+	ToolTypeFunction  = "function"
+	ToolTypeExtension = "extension"
+
+	// AttrMCPMethodName is the MCP JSON-RPC method a span represents. For a
+	// Forge tool call backed by an MCP server it is always "tools/call".
+	// (mcp.session.id / mcp.protocol.version require plumbing the MCP
+	// manager down to the executor and are tracked as a follow-up.)
+	AttrMCPMethodName  = "mcp.method.name"
+	MCPMethodToolsCall = "tools/call"
+
+	// AttrErrorType is the OTel-standard error classification set on a
+	// tool.<name> span when execution fails (alongside RecordError +
+	// Status=Error). Replaces the former `forge.tool.error`.
+	AttrErrorType = "error.type"
+
 	// ─── Forge-specific attributes. ──────────────────────────────────
 
 	// AttrForgeAgentID is the agent_id from forge.yaml — the operator's
@@ -93,10 +164,12 @@ const (
 	// dashboards can chart "iterations per task."
 	AttrForgeLoopIteration = "forge.loop.iteration"
 
-	// AttrForgeToolName / AttrForgeToolError name the tool call
-	// instrumentation.
-	AttrForgeToolName  = "forge.tool.name"
-	AttrForgeToolError = "forge.tool.error"
+	// AttrForgeToolName cross-references which tool a guardrail.<gate>
+	// span evaluated (see forge-cli/runtime/guardrails_tracing.go). The
+	// tool.<name> execution span itself uses the semconv AttrGenAIToolName
+	// key instead. (The former forge.tool.error was removed in favor of the
+	// standard AttrErrorType.)
+	AttrForgeToolName = "forge.tool.name"
 
 	// AttrForgeTaskFinalState is the terminal A2A TaskState the loop
 	// resolved to — "completed", "failed", "canceled". Set on the
@@ -125,13 +198,26 @@ const (
 	// `gen_ai.completion` flat-string attribute.
 	AttrGenAIOutputMessages = "gen_ai.output.messages"
 
-	// AttrForgeToolArgs is the raw arguments JSON the agent passed to
-	// a tool. Set on tool.<name> spans.
-	AttrForgeToolArgs = "forge.tool.args"
-
-	// AttrForgeToolResult is the raw output the tool returned. Set on
-	// tool.<name> spans.
-	AttrForgeToolResult = "forge.tool.result"
+	// AttrGenAIToolCallArguments is the raw arguments JSON the agent passed
+	// to a tool (semconv `gen_ai.tool.call.arguments`). Set on tool.<name>
+	// spans. Replaces the former `forge.tool.args`.
+	AttrGenAIToolCallArguments = "gen_ai.tool.call.arguments"
+
+	// AttrGenAIToolCallResult is the raw output the tool returned (semconv
+	// `gen_ai.tool.call.result`). Set on tool.<name> spans. Replaces the
+	// former `forge.tool.result`.
+	AttrGenAIToolCallResult = "gen_ai.tool.call.result"
+
+	// AttrGenAIToolDescription is the tool's description from its
+	// definition (semconv `gen_ai.tool.description`). Flagged sensitive by
+	// semconv, so it rides the same CaptureContent opt-in.
+	AttrGenAIToolDescription = "gen_ai.tool.description"
+
+	// AttrGenAIToolDefinitions is the JSON array of tool definitions
+	// available to the agent (semconv `gen_ai.tool.definitions`), stamped
+	// once on the agent.execute span. Potentially large, so it is opt-in
+	// behind CaptureContent.
+	AttrGenAIToolDefinitions = "gen_ai.tool.definitions"
 
 	// ─── Guardrail span attributes (issue #161) ──────────────────────
 	//