fix: name the azure.yaml key in the RAI policy validation error by huimiu · Pull Request #9328 · Azure/azure-dev

Hui Miao (huimiu) · 2026-07-28T06:25:27Z

RAI policies already flow from unified azure.yaml agent definitions to the Foundry API, but this path lacked coverage and documentation. The validation error also suggested the legacy rai_policy_name key, even though unified azure.yaml uses raiPolicyName.

Sample PR: https://github.com/microsoft-foundry/foundry-samples-pr/pull/802
Doc PR: https://github.com/MicrosoftDocs/azure-ai-docs-pr/pull/13567

github-actions · 2026-07-28T06:25:40Z

📋 Prioritization Note

Thanks for the contribution! The linked issue isn't in the current milestone yet.
Thank you for logging this issue; our team is reviewing it. If you need urgent prioritization, tag Rick Winter (@RickWinter) and Kristen Womack (@kristenwomack) to let us know.

azure-pipelines · 2026-07-28T06:25:51Z

Azure Pipelines: Successfully started running 1 pipeline(s). 21 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

Copilot

Pull request overview

Clarifies RAI policy configuration for unified azure.yaml agent definitions.

Changes:

Improves policy validation messaging.
Adds policy round-trip and deployment tests.
Documents policy configuration and key naming.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
`README.md`	Documents content safety policies.
`agent_policies_test.go`	Adds unified-shape regression coverage.
`parse.go`	Names both supported policy keys.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

cli/azd/extensions/azure.ai.agents/README.md:62

azd-code-reviewer: Add the inline agent name to this example. Runtime validation requires AgentDefinition.Name and does not derive it from the services.my-agent map key, so copying this snippet fails with template.name not in valid format: name cannot be empty before the policy can be deployed.

    kind: hosted

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

cli/azd/extensions/azure.ai.agents/README.md:54

RAI expands to “Responsible AI,” not “Azure AI Content Safety.” Use the product terminology so readers can identify and search for the correct policy type.

A hosted agent can be bound to an Azure AI Content Safety (RAI) policy so every

cli/azd/extensions/azure.ai.agents/README.md:74

Document the effective cardinality of RAI policies. mapRaiConfig returns the first matching entry and validation accepts multiple entries, so users can list several policies while only the first is sent to Foundry. This is especially important for safety configuration; either state the first-entry behavior here or reject duplicate rai_policy entries.

- `type` is required. `rai_policy` is currently the only supported value.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

cli/azd/extensions/azure.ai.agents/README.md:77

azd-code-reviewer: The documented list accepts multiple rai_policy entries, but mapRaiConfig returns the first match and the data-plane model has only one RaiConfig; every later policy is silently ignored. For a safety setting, this can leave a policy the user intended to apply unused. Reject duplicate RAI policies during validation (with a regression test), or explicitly document that only the first entry is effective.

- `type` is required. `rai_policy` is currently the only supported value.

@imatiach-msft

…leases (#9371) * chore: prepare azure.ai.agents 1.0.0-beta.8 release Bump version and add changelog for 10 PRs since beta.7: - Features: max_stalls early-stopping (#9314), invocations protocol 2.0.0 default (#9327) - Bug fixes: #9365, #9328, #9291, #9290, #9212, #9211, #9280, #9237 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * chore: prepare azure.ai.projects 1.0.0-beta.4 release Bump version and add changelog for 1 PR since beta.3: - Bug fix: cap Foundry ARM deployment names to 64 chars (#9292) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * chore: add imatiach to cspell contributor aliases Fix cspell-ext CI gate failure for the azure.ai.agents CHANGELOG entry referencing @imatiach-msft (PR #9314). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* test(azure.ai.agents): validate azure.yaml examples in docs `azure.yaml` examples in this extension's docs weren't validated by anything, so they could drift out of sync with the code and ship broken. Two instances were found by hand recently: the README migration example omitted the required agent `name` (#9328), and a Learn article documented `rai_config.rai_policy_name`, which azd ignores entirely — deploying with no guardrail and no error. Adds TestDocExamplesAreValid, which extracts every fenced YAML block declaring an `azure.ai.agent` service from the extension's markdown and applies two checks: 1. Resolver — the snippet must survive AgentDefinitionFromService, the same entry point azd uses at deploy time. Catches the missing-`name` class. 2. Vocabulary — every property must be declared in schemas/azure.ai.agent.json or parsed by azd core. azd deliberately ignores unrecognized service properties for forward compatibility, which is exactly how a doc can advertise a setting that silently does nothing. Catches the `rai_config` class, which the resolver alone cannot. Not every snippet is meant to be complete: the three `azure.ai.agent` entries in docs/private-networking.md intentionally omit `kind` so azd falls back to the on-disk agent.yaml. Those opt out of check 1 with an `` marker, keeping the default strict rather than inferring intent. Fixes #9330 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e7ba2e0e-80cf-4226-b56e-ac7cbbc7338f * test(azure.ai.agents): validate doc snippets through core's field shapes and the full schema Co-authored-by: glharper <64209257+glharper@users.noreply.github.com> * test(azure.ai.agents): name the doc and the fix when a value can't reach structpb The two structpb.NewStruct assertions were the only doc-content-reachable ones without a message, so a value structpb can't represent failed with a bare "proto: invalid type: time.Time" and no hint about what to change. It is reachable from a snippet: an unquoted date under an extension-owned key (metadata: released: 2024-07-18) decodes to time.Time and fails exactly that way. Both assertions now report file:line/service and say to quote ambiguous scalars. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1d5b46e6-fc04-48bb-9a2c-156105966b48 * docs(agents): mark the config env snippet as a partial example The service-scoped env section added by #9079 shows only where `env:` belongs, so its snippet has no `kind:` and cannot resolve to a full agent definition. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: fed9e97b-e79b-4889-ac76-0d9a428599cd * Validate doc examples against the full schema and core YAML shapes Addresses review on #9368. Compile azure.ai.agent.json and validate the runtime-active inline or legacy config property map, so required, type, enum, pattern, and nested additionalProperties constraints are enforced in addition to the existing friendly vocabulary check. Reject extension properties in the inactive location because azd selects one shape rather than merging them. Replace untyped core service placeholders with strict test-only mirrors of Docker, AKS/Helm/Kustomize, infra/layers/deploymentStacks, and hook YAML. Mirror HooksConfig's mapping-or-list parsing and core's nil-hook validation, and reject nested core-field typos that azd would otherwise ignore. Regression coverage exercises each constraint and field family. The JSON Schema validator was already a transitive dependency; mark it direct now that the doc test imports it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b4617dfd-adfb-4b30-9222-477a041f8af9 --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Copilot-Session: e7ba2e0e-80cf-4226-b56e-ac7cbbc7338f Copilot-Session: 1d5b46e6-fc04-48bb-9a2c-156105966b48 Copilot-Session: fed9e97b-e79b-4889-ac76-0d9a428599cd Copilot-Session: b4617dfd-adfb-4b30-9222-477a041f8af9

fix: name the azure.yaml key in the RAI policy validation error

fac0419

Copilot AI review requested due to automatic review settings July 28, 2026 06:25

Hui Miao (huimiu) requested review from JeffreyCA, Glenn Harper (glharper), Zhijie Huang (hund030), John Miller (therealjohn), Travis Angevine (trangevi) and Travis Wilson (trrwilson) as code owners July 28, 2026 06:25

Copilot started reviewing on behalf of Hui Miao (huimiu) July 28, 2026 06:25 View session

microsoft-github-policy-service Bot assigned Hui Miao (huimiu) Jul 28, 2026

fix: remove redundant RAI policy validation comments

58e0d69

Copilot AI reviewed Jul 28, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.agents/README.md

Copilot AI review requested due to automatic review settings July 28, 2026 06:28

Copilot started reviewing on behalf of Hui Miao (huimiu) July 28, 2026 06:29 View session

Copilot AI reviewed Jul 28, 2026

View reviewed changes

github-actions Bot added the ext-agents azure.ai.agents extension label Jul 28, 2026

test: use maps.Copy for policy test case overrides

d0d16ca

Copilot AI review requested due to automatic review settings July 28, 2026 07:49

Copilot started reviewing on behalf of Hui Miao (huimiu) July 28, 2026 07:49 View session

Copilot AI reviewed Jul 28, 2026

View reviewed changes

fix: add required agent name to README azure.yaml examples

3f15ab7

Copilot AI review requested due to automatic review settings July 28, 2026 07:57

Copilot AI reviewed Jul 28, 2026

View reviewed changes

Hui Miao (huimiu) mentioned this pull request Jul 28, 2026

[Issue] azure.yaml examples in agent docs aren't validated and can ship broken #9330

Closed

1 task

Travis Angevine (trangevi) approved these changes Jul 28, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/parse.go

Amit Bhave (amitbhave10) approved these changes Jul 29, 2026

View reviewed changes

Travis Angevine (trangevi) merged commit f91b4c4 into main Jul 29, 2026
29 checks passed

Copilot AI mentioned this pull request Jul 29, 2026

Prepare azd core 1.29.0 release #9359

Merged

Hui Miao (huimiu) deleted the hui/rai-policy-unified-azure-yaml-coverage branch July 30, 2026 05:21

Glenn Harper (glharper) mentioned this pull request Jul 30, 2026

test(azure.ai.agents): validate azure.yaml examples in docs #9368

Merged

Travis Angevine (trangevi) mentioned this pull request Jul 30, 2026

chore: prepare azure.ai.agents beta.8 and azure.ai.projects beta.4 releases #9371

Merged

Navigation Menu

Uh oh!

fix: name the azure.yaml key in the RAI policy validation error - #9328

fix: name the azure.yaml key in the RAI policy validation error#9328
Travis Angevine (trangevi) merged 4 commits into
mainfrom
hui/rai-policy-unified-azure-yaml-coverage

Hui Miao (huimiu) commented Jul 28, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

azure-pipelines Bot commented Jul 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Hui Miao (huimiu) commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026

📋 Prioritization Note

Uh oh!

azure-pipelines Bot commented Jul 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Hui Miao (huimiu) commented Jul 28, 2026 •

edited

Loading