Skip to content

Support advanced prompt voice agent configuration - #9609

Draft
JianW (v1212) wants to merge 4 commits into
Azure:mainfrom
v1212:users/wujia/prompt-voice-agent-advanced-editing
Draft

Support advanced prompt voice agent configuration#9609
JianW (v1212) wants to merge 4 commits into
Azure:mainfrom
v1212:users/wujia/prompt-voice-agent-advanced-editing

Conversation

@v1212

@v1212 JianW (v1212) commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Builds on the previously merged prompt voice agent support by adding advanced, editable prompt voice configuration in azure.yaml.
  • Aligns the azd authoring shape with the current sample expectations and Vienna prompt voice implementation surface: structured inputs, audio input/output tuning, output modalities, tools, and avatar configuration.
  • Keeps existing simple prompt voice projects working by layering advanced overrides on top of the current defaults for model, instructions, voice, store, and audio pipeline.

Scope and compatibility

  • This PR is a public-preview follow-up for kind: prompt-voice; it does not change hosted/container agent deployment behavior.
  • Existing prompt voice YAML remains valid. Minimal managed/BYOM agents can still use only modelType, model.id, instructions, voice, and store.
  • Advanced fields are optional and map to the prompt voice service contract where supported. Missing audio fields keep azd defaults.
  • Other agent paths were reviewed for isolation: hosted/container mapping, code/container deploy, provisioning config, and non-voice service round-tripping remain unchanged.

Field coverage and source of truth

  • audio, outputModalities, structuredInputs, tools, avatar, and store follow the Vienna prompt voice contract/tests and current prompt voice samples.
  • azd validates stable local constraints that Vienna also validates, including supported audio formats (audio/pcm, audio/pcmu, audio/pcma), speed range, server VAD threshold range, and required avatar/voice fields.
  • Extensible service areas remain intentionally light: tool objects and avatar extension fields are passed through to prompt voice service validation so azd does not block future Vienna/Voice Live additions.
  • structuredInputs.defaultValue uses the azure.yaml camelCase authoring style and is converted to the service wire field default_value during deployment.

Implementation

  • Extends prompt voice authoring models and REST payload models for advanced voice config.
  • Updates mapping so audio, structuredInputs, outputModalities, tools, and avatar flow from YAML to the create request.
  • Adds validation for local constraints such as audio format, turn detection ranges, output speed, and blank modalities.
  • Updates the extension JSON schema and README with the advanced azure.yaml shape, known values, and pass-through boundaries.

Validation

  • go test ./... -count=1
  • go build ./...
  • cspell lint 'extensions/**/*.go' 'extensions/**/*.md' --config ./.vscode/cspell.yaml --no-progress
  • Live managed prompt voice deploy in France Central with advanced YAML: structured inputs, server VAD, transcription, Azure voice style, system tool, avatar, store; control-plane GET assertions passed.
  • Live BYOM realtime prompt voice deploy in France Central with advanced YAML: self-deployed realtime model, semantic VAD, PCMU input, audio+text output modalities, function tool, structured inputs; control-plane GET assertions passed.
  • Negative live smoke: audio/opus is rejected locally before deploy with a supported-format validation error.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
20 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions github-actions Bot added the ext-agents azure.ai.agents extension label Aug 18, 2026
Copilot AI added 2 commits August 19, 2026 09:05
…ice-agent-advanced-editing

# Conflicts:
#	cli/azd/extensions/azure.ai.agents/README.md
#	cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext-agents azure.ai.agents extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants