feat(media): #91 media library — dedup, backend favorites, audio Extract/Sound tabs#191
Merged
Conversation
added 9 commits
July 4, 2026 17:27
Two of the #91 media-library defects. Folder-tree preservation and the per-card extract-audio button already shipped (#49/#39); this lands the two genuinely-missing pieces. Dedup (素材重复出现): import_media_file appended unconditionally, so re-importing a file (or a folder containing already-imported files) minted duplicate entries. It now reuses the existing entry by source path, keeping its id so referencing clips stay valid. TDD: reimporting_the_same_file_*. Favorites (星标迁移): move the "mine" star set off browser localStorage onto the project manifest so favorites travel with the project. - domain: MediaManifest.favorites (serde default, skipped when empty, so older projects load and re-save unchanged) + is_favorite/set_favorites/ prune_favorites; archive() preserves it on save. - core/session: set_media_favorite, mutated outside the undo stack like import (favoriting is a library action, not a timeline edit). - tauri: toggle_favorite command + MediaItemDto.favorite (membership) + registered in the invoke handler. - web: MediaItem.favorite drives the "mine" tab and star button (writes the backend + refreshes); migrateLocalFavorites drains the legacy opentake.favorites key into the manifest once, self-guarding and per-project so it settles to a no-op. Tests: opentake-domain 211, opentake-core 40, src-tauri media 34, web favorites 3 — all green; cargo check --workspace + tsc + vite build clean.
Fixes #91 complaint 3 (音频 tab 语义错误) — the audio tab was only a type==='audio' filter. It now carries the CapCut-style entries, both reusing existing backends (no new Rust): - Extract (提取): lists project videos that carry an audio track; each is a normal MediaCard whose hover Extract button already runs extract_audio (#39), so extracting a video's audio is reachable without leaving the audio tab. - Sound (音效): surfaces the global sound library (#115's `sound` category, on the library_* commands) inside the audio tab with one-click "add to project" via libraryStore.importToProject. Subtabs are now kind-aware: material keeps 导入/我的; audio adds 提取/音效 (AUDIO_SUB_TABS). Landing on the material tab while pointing at an audio-only subtab falls back to import. Favoriting audio (from the earlier backend-favorites change) makes 我的 work for audio too. New: SoundLibraryTab.tsx. i18n media.subtab.extract/sound + media.sound.* / media.extract.* in both locales. tsc + vite build + 494 vitest green. Native flows (extract/import) need a desktop build to exercise end-to-end.
Test-build polish: don't hand testers dead buttons. The media panel's
"生成/Generate" button had no onClick (generate_* is still stubbed) — it now
renders disabled with a "AI 生成 · 即将推出" tooltip. The Agent panel
placeholder was dev-speak ("Agent 面板(独立功能)") — now "AI 助手 · 即将推出".
No behavior change beyond making unfinished surfaces read as intentional.
The clip right-click had a saveAsMedia i18n string but no menu item and no backend. Implement it end to end for video clips: render the clip — trims, speed, effects, color grade and text baked in — to a new .mp4 in the project's media/ dir, then import it as a fresh asset. Backend (save_clip_as_media): reuses the export pipeline with zero changes to it. build_single_clip_export (pure, unit-tested) frames the clip on a single visible/unmuted track re-based to frame 0 and subsets the manifest to the clip's source; run_export renders it; import_one probes + imports the result. Requires a saved project (needs a bundle media/ dir); video only for now (audio/image save-as is a follow-up — basic audio extraction already exists via extract_audio). Frontend: api.saveClipAsMedia + a video-only "另存为媒体" context-menu item → editActions.saveClipAsMedia (toasts start/result, refreshes the panel). Tests: src-tauri media 35 (incl. single_clip_export framing), tsc + vite + 494 vitest, cargo check --workspace all green. The GPU render itself needs a desktop build to exercise end-to-end.
CI runs cargo fmt --all --check and clippy --workspace -D warnings. Use std::slice::from_ref in the favorites test instead of &[id.clone()], and apply rustfmt's line wrapping. No behavior change.
The macOS release bundle had never been built (CI only runs cargo check on
Linux), and it fails: whisper-rs-sys (whisper.cpp/ggml) uses std::filesystem,
which needs macOS >= 10.15, but the cc crate was appending
-mmacosx-version-min=10.13 (the value Tauri injects) as the last flag, so
clang's "last wins" made the effective target 10.13.
- .cargo/config.toml: [env] MACOSX_DEPLOYMENT_TARGET = { value = "11.0",
force = true } — forces every build script's C/C++ target to 11.0.
- tauri.conf.json: bundle.macOS.minimumSystemVersion = "11.0" — matches the
Info.plist gate. Keep the two in sync.
Verified: `tauri build --bundles dmg` now produces OpenTake_1.0.0_aarch64.dmg.
…bView) Timeline playback showed only one frame in the packaged macOS app. The Rust engine was compositing and broadcasting every frame fine, but the frontend pointed an <img> at a multipart/x-mixed-replace MJPEG stream — and WebKit / WKWebView only ever paints the FIRST part of such a stream. (Never surfaced because the macOS bundle had never been built/run; the transport author had already flagged the swap in transport.rs.) Swap the transport to what WebKit handles reliably: - backend: add a /ws route pushing each broadcast JPEG as a binary WebSocket message (axum "ws" feature; same loopback-origin guard as /stream, kept for debugging). get_preview_endpoint now returns the ws:// URL. - frontend: TimelineRustOverlay renders a <canvas> and draws each frame via createImageBitmap + drawImage instead of an MJPEG <img>. playback tests 33 green; cargo check + clippy + fmt + tsc clean. Needs the desktop build to confirm end to end.
Ship the playback path that worked before the streaming engine landed (pre-4dff06a). The legacy stack composites the timeline in DOM/CSS (transform, crop, fades, z-order matching the render) and is cross-platform (works in every WebView), so timeline playback works on the macOS bundle right now. The Rust streaming engine (wgpu composite + WebSocket transport + cpal clock) is now OPT-IN (localStorage opentake.rustEngine="1"): its macOS transport only painted one frame in WKWebView, and although the transport is now a WS canvas, it hasn't been verified on a real build yet. Keep it behind the flag until then. Legacy default won't show GPU-only effects (color grade / chroma key / shader masks) in preview, but plays real timelines. tsc + vite + 494 vitest green.
…transport Make the wgpu streaming engine the default preview path again so the preview shows the FULL GPU result — color grade, chroma key, masks, shader effects — matching the export. The one-frame bug was the transport (MJPEG <img>), which is now WebSocket → canvas; the engine itself was always fine. Also harden the canvas draw: prefer createImageBitmap, fall back to an <img> + object URL on older WebKit, so a missing API can't blank the surface. The legacy <video> stack stays as the runtime-watchdog fallback (engine can't start) and the "0" opt-out. tsc + preview vitest (82) green. Needs the desktop build to confirm frame rate / rendering.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses the actionable parts of #91 (media-library rewrite). Investigation (2 agents + upstream
MediaPanelreference) found most of #91 already shipped in beta — folder-tree preservation (mirror_dir, #49), the per-card extract-audio button (#39), and the global sound library (#115). This PR lands the genuinely-missing pieces, reusing existing backends where possible.What changed
1. 查重 — dedup on import (backend)
import_media_fileappended unconditionally, so re-importing a file (or a folder containing already-imported files) minted duplicate manifest entries — the "素材重复出现" bug. It now reuses the existing entry bysourcepath, keeping its id so referencing clips stay valid. (The frontend store was already a clean read-only mirror; the dup was backend-side.)2. 星标迁移 — favorites move to the project manifest (backend + web)
Favorites were per-browser
localStorage; now they persist per-project in the manifest so they travel with the project.MediaManifest.favorites(serdedefault+ skip-when-empty → old projects load/re-save unchanged);is_favorite/set_favorites/prune_favorites;archive()preserves it.set_media_favoriteon session/core, mutated outside the undo stack like import (favoriting is a library action, not a timeline edit).toggle_favoriteTauri command +MediaItemDto.favorite.item.favoritedrives the 我的 tab + star button;migrateLocalFavoritesdrains the legacyopentake.favoriteskey into the manifest once, self-guarding and per-project.3. 音频 tab — Extract + Sound subtabs (web, reuses existing backends)
The audio tab was only a
type==='audio'filter. It now has CapCut-style subtabs:MediaCardwhose Extract button runsextract_audio(提取视频音频/音乐保存本地(媒体项星标导出) #39) — extracting is reachable without leaving the audio tab.soundcategory,library_*commands) with one-click add-to-project.Verified already-working (no change): folder tree not flattened (
import_folder_recursive_mirrors_tree).Test plan
cargo check --workspaceclean; opentake-domain 211, opentake-core 40, src-tauri media 34, opentake-project all green (incl. new dedup + favorites + serialization tests)tsc -bclean ·vite buildclean ·vitest494/494 (incl. rewrittenfavorites.test.tsmigration tests)library_*flows need the Tauri runtime — not exercisable in CI/browser)Refs #91. Backend favorites intentionally per-project (faithful
localStoragemigration), distinct from the cross-project global library (#37/#115).