Skip to content

fix: route timeline PLAY through embedded libmpv (WKWebView blocks plain-ws)#209

Merged
appergb merged 2 commits into
betafrom
fix/ws-blocked-frame-transport
Jul 7, 2026
Merged

fix: route timeline PLAY through embedded libmpv (WKWebView blocks plain-ws)#209
appergb merged 2 commits into
betafrom
fix/ws-blocked-frame-transport

Conversation

@appergb

@appergb appergb commented Jul 7, 2026

Copy link
Copy Markdown
Owner

问题

WKWebView 的安全 tauri:// 上下文把回环 ws:// WebSocket 当混合内容静默拦截(连 TCP 都不发起),自研流式引擎的显示腿(WS→canvas)永远只画得出第一帧 —— 播放整体不可用。

修法

PLAY 委托内嵌 libmpv(社区引擎),暂停/擦洗/导出保持 wgpu 像素级合成不变。

部件 变更
previewEngine.ts 引擎路径整体切 mpv:EDL 载入主视频轨,time-pos 观察者驱动播放头(整数帧截断),启动看门狗 + 单次 toast 降级 legacy <video>
mpvEdl.ts(新) Timeline→edl:// 翻译:UTF-8 字节转义、lavfi 黑场补间隙(时间线时间=播放时间)、缺失素材自动并入间隙;8 个 vitest
Preview/PanelShell/App/global.css mpv 画在透明 WebView 下层原生窗口;播放时预览面板开"透明洞",setVideoMarginRatio 按画布盒实时对位(ResizeObserver)
mpv_bootstrap.rs(新) 启动时把 wrapper 从 Resources 镜像到 exe_dir/lib——插件只 dlopen exe 相邻路径,而 Tauri 把 resources 放 Contents/Resources,不镜像则打包版必然降级
transport.rs 保留 /frame 单帧轮询路由(被动 <img> 加载不受混合内容限制)作为引擎可调试显示腿 + 验收探针
export_probe.rs fixture 改 ffmpeg lavfi 自生成(原来指向会话级临时目录,跨机必挂)
构建 src-tauri/lib/ 入 gitignore;wrapper 用 tauri-plugin-libmpv-api setup-lib 下载(README 已记);运行期需系统 libmpv(brew install mpv

兼容与降级

已知 MVP 限制(后续 PR)

  1. mpv 播放期独立音轨(BGM/旁白)静音——EDL 只含主视频轨;follower 跟播修复已立项(下一个 PR)。
  2. Overlay 轨(PIP/文字)只在暂停合成帧可见,播放期不出现。
  3. 分发级 libmpv 随包与签名封条问题并入 [codec/preview] 播放预览编解码与无缝切换缺口(ffmpeg 未随包 / <video> 无回退 VP9·AV1 / clip 边界 remount) #131(ffmpeg sidecar)一起做。

自测

  • 本地六项全绿:fmt / clippy 双态 -D warnings / cargo test --workspace(40 组全 ok)/ web build / 506 vitest(含新增 8 个 mpvEdl)。
  • 真机视觉验收随后按 docs/architecture/PLAYBACK-ENGINE.md 清单执行(构建→装 /Applications→MCP 驱动 + 截图),结果回填本 PR。

baiqing added 2 commits July 7, 2026 15:15
…ain-ws)

WKWebView's secure tauri:// context silently blocks plain-ws:// loopback
WebSockets as mixed content, so the in-house streaming engine's display leg
(WS -> canvas) never painted more than the first frame. PLAY now rides an
embedded mpv instance; pause/scrub/export keep the pixel-exact wgpu
composite path.

- previewEngine: delegate PLAY to libmpv (EDL over the primary video track,
  time-pos observer drives the playhead, startup watchdog + one-shot toast
  falls back to the legacy <video> path)
- mpvEdl: Timeline -> edl:// translation (byte-escaped paths, lavfi black
  gap fill, missing media widens the gap) + vitest coverage
- Preview/PanelShell/App/global.css: transparent hole over the canvas box
  while mpv drives (video paints on the native layer under the webview);
  margins re-synced via setVideoMarginRatio on layout changes
- mpv_bootstrap: mirror the bundled libmpv-wrapper into exe_dir/lib at
  startup (the plugin only dlopens exe-adjacent paths; Tauri lands
  resources under Contents/Resources)
- transport: keep a latest-frame /frame poll route (WKWebView-safe passive
  <img> load) as the engine's debuggable display leg + acceptance probe
- export_probe: self-generate the fixture via ffmpeg lavfi (was a stale
  session-scoped path)
- gitignore src-tauri/lib/ (dylibs come via tauri-plugin-libmpv-api
  setup-lib; documented in README prerequisites)

Known MVP limits (follow-ups filed): independent audio tracks are silent
during mpv playback (BGM follower sync is the next PR); overlay tracks
render only in the paused composite.
@appergb appergb merged commit ac50dc8 into beta Jul 7, 2026
2 checks passed
@appergb appergb deleted the fix/ws-blocked-frame-transport branch July 7, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant