Skip to content

Add AppSec agentic onboarding runtime telemetry (RFC-1110)#11983

Open
christophe-papazian wants to merge 2 commits into
masterfrom
christophe-papazian/appsec-agentic-onboarding-telemetry
Open

Add AppSec agentic onboarding runtime telemetry (RFC-1110)#11983
christophe-papazian wants to merge 2 commits into
masterfrom
christophe-papazian/appsec-agentic-onboarding-telemetry

Conversation

@christophe-papazian

@christophe-papazian christophe-papazian commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Implements RFC-1110 — a runtime telemetry signal that a service was onboarded by the AAP agentic onboarding solution.

When the presence-only flag DD_APPSEC_AGENTIC_ONBOARDING is set (via env var, JVM property, or local/fleet stable config), the tracer emits a single boolean configuration entry appsec.agentic_onboarding in its app-started instrumentation-telemetry payload, tagged with the config source origin.

  • The reported boolean is true only when AppSec fully came up at startup with its WAF module started, and false otherwise (AppSec disabled, remote-config-only, native WAF failed to load, or startup aborted). This mirrors the Python reference's "native WAF loaded" semantics and is stricter than bare isActive().
  • The flag is presence-only: any value (including empty) counts as set; its value is never read or transmitted. When unset in all sources, nothing is reported.

Motivation

Prerequisite for Agentic Onboarding GA (APPSEC-68812): lets the usage→activation funnel be measured in production. Java counterpart of the merged Python implementation (dd-trace-py#19068).

Changes

  • AppSecConfig — new APPSEC_AGENTIC_ONBOARDING config key; registered DD_APPSEC_AGENTIC_ONBOARDING in metadata/supported-configurations.json.
  • ActiveSubsystems — new bootstrap-visible APPSEC_WAF_STARTED flag.
  • AppSecSystem — sets it on full startup success (active + WAF module started), cleared per start attempt.
  • Agent.maybeStartAppSec — reports the signal in a finally so the disabled/failed cases are covered.
  • AgenticOnboarding (new) — presence/origin resolution (never via ConfigProvider, to keep the raw value out of telemetry) + reporting.

Testing

New JUnit 5 unit tests (AgenticOnboardingTest) cover presence-only semantics (incl. empty string), source precedence, the absent case, boolean value/origin, and the system-property end-to-end path. Behavioral coverage across languages is handled by system tests (APPSEC-69171).

🤖 Generated with Claude Code

@christophe-papazian christophe-papazian added type: feature Enhancements and improvements comp: asm waf Application Security Management (WAF) comp: telemetry Telemetry tag: ai generated Largely based on code generated by an AI or LLM labels Jul 17, 2026
@christophe-papazian

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 79d7375567

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jul 17, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 58.00%
Overall Coverage: 57.27% (-0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b91b3e2 | Docs | Datadog PR Page | Give us feedback!

christophe-papazian and others added 2 commits July 17, 2026 15:39
Report a runtime signal that a service was onboarded by the agentic
onboarding solution. When the presence-only flag
DD_APPSEC_AGENTIC_ONBOARDING is set (via env var, JVM property, or
local/fleet stable config), the tracer emits a single boolean
configuration entry `appsec.agentic_onboarding` in its app-started
instrumentation-telemetry payload, tagged with the flag's origin.

The reported boolean reflects whether AppSec fully came up at startup
with its WAF module started (new ActiveSubsystems.APPSEC_WAF_STARTED),
which matches the Python reference's "native WAF loaded" semantics and
is stricter than bare isActive(). The flag's value is never read or
transmitted; only its presence triggers reporting. When the flag is
unset in all sources, nothing is reported.

Reporting runs in a finally in maybeStartAppSec so it also covers the
AppSec-disabled and startup-failure cases (reported as false).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When DD_APPSEC_AGENTIC_ONBOARDING is set via fleet/local stable config,
attach the source's config_id to the reported entry so the signal stays
attributable to the deployed stable config, matching other stable-config
telemetry. No config_id for env/jvm origins.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@christophe-papazian
christophe-papazian force-pushed the christophe-papazian/appsec-agentic-onboarding-telemetry branch from 79d7375 to b91b3e2 Compare July 17, 2026 13:40
@dd-octo-sts

dd-octo-sts Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.00 s 13.96 s [-0.4%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent 12.94 s 13.07 s [-1.7%; -0.3%] (maybe better)
startup:petclinic:appsec:Agent 16.93 s 16.55 s [+1.2%; +3.3%] (significantly worse)
startup:petclinic:iast:Agent 16.90 s 16.95 s [-1.1%; +0.5%] (no difference)
startup:petclinic:profiling:Agent 16.60 s 16.95 s [-3.4%; -0.7%] (maybe better)
startup:petclinic:sca:Agent 16.93 s 16.80 s [-0.1%; +1.6%] (no difference)
startup:petclinic:tracing:Agent 15.94 s 15.66 s [-2.5%; +6.0%] (no difference)

Commit: b91b3e24 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@christophe-papazian
christophe-papazian marked this pull request as ready for review July 17, 2026 15:16
@christophe-papazian
christophe-papazian requested review from a team as code owners July 17, 2026 15:16
@christophe-papazian
christophe-papazian requested review from PerfectSlayer, dromanol, manuel-alvarez-alvarez and mhlidd and removed request for a team July 17, 2026 15:16

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

Validated across three standalone test programs covering 26 adversarial scenarios (origin precedence/presence-only semantics, key transformations, APPSEC_WAF_STARTED state management in all startup paths, and finally-block behavior). All scenarios pass: the DD_APPSEC_AGENTIC_ONBOARDING env var key is derived consistently across all lookup paths, the flag reports false for every non-WAF-started case (disabled, WAF init failure, exception path), and AgenticOnboarding.report() fires in the finally block under all conditions including AppSec errors.

Was this helpful? React 👍 or 👎

📊 Validated against 26 scenarios · Open Bits AI session

🤖 Datadog Autotest · Commit b91b3e2 · What is Autotest? · Any feedback? Reach out in #autotest

@e-n-0 e-n-0 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
However we might want to update the Feature Parity Dashboard to also set the config key that is special for java.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: asm waf Application Security Management (WAF) comp: telemetry Telemetry tag: ai generated Largely based on code generated by an AI or LLM type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants