Improve audio and video file support

[paullizer]

Summary

Review and expand supported audio/video formats and playback/transcription/indexing behavior.

User Value

Tracking this as a GitHub issue moves the backlog item out of a private markdown file and gives the team a clear place to prioritize, design, implement, and validate the work.

Proposed Behavior

Distinguish upload allow-list, browser preview support, Video Indexer support, speech transcription support, and enhanced citation rendering.

Acceptance Criteria

• The requested behavior is designed or implemented for the relevant SimpleChat surfaces.
• Permission, configuration, and existing-feature interactions are accounted for.
• Tests, documentation, or validation notes are added as appropriate.

Notes

• Source backlog entry: Improve Audio And Video File Support
• Source file: priv-simplechat-2/grouped_feature_fix_backlog.md
• Backlog status: Extension
• Current anchors: application/single_app/config.py media extension lists, application/single_app/functions_documents.py video/audio processing, speech/video settings
• Proposed roadmap priority: P2
• Proposed roadmap size: M
• Related: video indexer limits #795 tracks Video Indexer limits and Create output as Audio file #628 tracks audio output. This issue focuses on broader media input support and citation/preview behavior.

[paullizer]

Repro: iPhone-recorded m4a fails during segmentation

A user uploaded an audio file recorded on an iPhone: 3400 Sunny View Dr 25.m4a.

Expected

The upload should be supported and process successfully. The same file was uploaded directly to Azure Speech Service and worked there.

Actual

The document appears in the Personal Workspace upload list with an error:

Error: Processing failed: Segmentation failed: [Errno 2] No such file or directory: 'ffmpeg'

Notes for implementation

• The UI allow-list includes m4a, so users reasonably expect iPhone audio uploads to work.
• The failure appears to be local/media preprocessing before Azure Speech receives the file, not a Speech Service format limitation.
• Validate the fix with an iPhone-style .m4a/AAC path and ensure missing local ffmpeg does not break supported audio transcription when Azure Speech can consume the source file.

[paullizer]

Implementation update

Implemented on branch issue-974-iphone-m4a-ffmpeg.

What changed

• Added default-on FFmpeg/FFprobe packaging for the SimpleChat container image with INSTALL_AUDIO_FFMPEG=true.
• Wired deployer control through SIMPLECHAT_INSTALL_FFMPEG and bumped deployer version to 1.0.21.
• Expanded recognized audio upload extensions for broad FFmpeg-decodable audio support.
• Added runtime audio capability reporting so Admin Settings shows whether FFmpeg broad transcoding is available and which upload extensions are recognized.
• Preserved the public Azure Speech source-file fallback for supported audio when local FFmpeg is unavailable.
• Updated app version to 0.250.010.
• Added feature/fix documentation and release notes under v0.250.010.

Validation

• python functional_tests/test_audio_ffmpeg_fallback.py -> 8/8 tests passed
• python -m py_compile application/single_app/functions_documents.py application/single_app/route_frontend_admin_settings.py application/single_app/config.py functional_tests/test_audio_ffmpeg_fallback.py ui_tests/test_admin_multimedia_guidance.py
• python -c "import yaml; yaml.safe_load(open('deployers/azure.yaml', encoding='utf-8'))"
• git diff --check
• Editor diagnostics reported no errors for the changed Python files.

Remaining validation note

The local venv does not have pytest, so the updated Playwright UI test was syntax-compiled but not run through pytest here. A full container/ACR build should also confirm the Azure Linux builder can install ffmpeg and that the final distroless image resolves ffmpeg on PATH.

[paullizer]

Closed as completed by PR #997: #997

PR #997 added ffmpeg fallback support for iPhone M4A uploads, documented audio runtime requirements, and surfaced supported file type guidance across the workspace/admin surfaces.

[paullizer]

Closed as completed by PR #997: #997

PR #997 added ffmpeg fallback support for iPhone M4A uploads, documented audio runtime requirements, and surfaced supported file type guidance across the workspace/admin surfaces.