fix: route timeline PLAY through embedded libmpv (WKWebView blocks plain-ws)#209
Merged
Conversation
added 2 commits
July 7, 2026 15:15
…ain-ws) WKWebView's secure tauri:// context silently blocks plain-ws:// loopback WebSockets as mixed content, so the in-house streaming engine's display leg (WS -> canvas) never painted more than the first frame. PLAY now rides an embedded mpv instance; pause/scrub/export keep the pixel-exact wgpu composite path. - previewEngine: delegate PLAY to libmpv (EDL over the primary video track, time-pos observer drives the playhead, startup watchdog + one-shot toast falls back to the legacy <video> path) - mpvEdl: Timeline -> edl:// translation (byte-escaped paths, lavfi black gap fill, missing media widens the gap) + vitest coverage - Preview/PanelShell/App/global.css: transparent hole over the canvas box while mpv drives (video paints on the native layer under the webview); margins re-synced via setVideoMarginRatio on layout changes - mpv_bootstrap: mirror the bundled libmpv-wrapper into exe_dir/lib at startup (the plugin only dlopens exe-adjacent paths; Tauri lands resources under Contents/Resources) - transport: keep a latest-frame /frame poll route (WKWebView-safe passive <img> load) as the engine's debuggable display leg + acceptance probe - export_probe: self-generate the fixture via ffmpeg lavfi (was a stale session-scoped path) - gitignore src-tauri/lib/ (dylibs come via tauri-plugin-libmpv-api setup-lib; documented in README prerequisites) Known MVP limits (follow-ups filed): independent audio tracks are silent during mpv playback (BGM follower sync is the next PR); overlay tracks render only in the paused composite.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
WKWebView 的安全
tauri://上下文把回环ws://WebSocket 当混合内容静默拦截(连 TCP 都不发起),自研流式引擎的显示腿(WS→canvas)永远只画得出第一帧 —— 播放整体不可用。修法
PLAY 委托内嵌 libmpv(社区引擎),暂停/擦洗/导出保持 wgpu 像素级合成不变。
previewEngine.tstime-pos观察者驱动播放头(整数帧截断),启动看门狗 + 单次 toast 降级 legacy<video>mpvEdl.ts(新)edl://翻译:UTF-8 字节转义、lavfi 黑场补间隙(时间线时间=播放时间)、缺失素材自动并入间隙;8 个 vitestPreview/PanelShell/App/global.csssetVideoMarginRatio按画布盒实时对位(ResizeObserver)mpv_bootstrap.rs(新)exe_dir/lib——插件只 dlopen exe 相邻路径,而 Tauri 把 resources 放Contents/Resources,不镜像则打包版必然降级transport.rs/frame单帧轮询路由(被动<img>加载不受混合内容限制)作为引擎可调试显示腿 + 验收探针export_probe.rssrc-tauri/lib/入 gitignore;wrapper 用tauri-plugin-libmpv-api setup-lib下载(README 已记);运行期需系统 libmpv(brew install mpv)兼容与降级
<video>路径 + 一次性 toast(沿用 feat(playback): make the Rust streaming engine the default preview path (#53) #189 的shouldFallBackToLegacy纯函数,测试未动仍绿)。--no-default-features最小构建不受影响(libmpv 插件运行期 dlopen,编译零系统依赖)。已知 MVP 限制(后续 PR)
自测
docs/architecture/PLAYBACK-ENGINE.md清单执行(构建→装 /Applications→MCP 驱动 + 截图),结果回填本 PR。