Skip to content

feat(integration.ltcgi): drive LTCGI realtime area light from the media player#897

Open
towneh wants to merge 2 commits into
BasisVR:developerfrom
towneh:feat/ltcgi-media-player-integration
Open

feat(integration.ltcgi): drive LTCGI realtime area light from the media player#897
towneh wants to merge 2 commits into
BasisVR:developerfrom
towneh:feat/ltcgi-media-player-integration

Conversation

@towneh

@towneh towneh commented Jun 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds com.basis.integration.ltcgi — an optional bolt-on that drives LTCGI realtime area lighting from a Basis Media Player, so a video screen casts coloured light that matches whatever is playing.

BasisLTCGIVideoAdapter subscribes to BasisMediaPlayer.OnOutputTextureChanged, blits each frame into a RenderTexture it owns (the player exposes a plain Texture and never owns a RenderTexture; LTCGI samples one shared video texture through its blur chain), and hands that RT to LTCGI_UdonAdapter._SetVideoTexture. LTCGI's blur chain re-prefilters it every frame, so the emitted light tracks the video live. It also re-asserts the feed if LTCGI's global shader state is reset at runtime (e.g. a Desktop/VR mode switch), so the lighting self-heals.

Modelled on the existing com.basis.integration.audiolink / com.basis.integration.ytdlp packages: an embedded package that is inert until its dependencies are present — the assembly compiles to nothing unless both com.basis.mediaplayer and at.pimaker.ltcgi are in the project (asmdef define constraints BASIS_MEDIAPLAYER_EXISTS + LTCGI_EXISTS). Safe to land without forcing LTCGI into the project.

Scope: package source only — no manifest change. at.pimaker.ltcgi is intentionally not wired into the project manifest yet; the URP-compatible LTCGI surface shaders this relies on are being upstreamed to PiMaker's repo, after which the dependency can be added separately.

Required checks

All boxes below must be ticked before this PR can merge. If a check is genuinely N/A, tick it anyway and explain under Notes.

  • Tested — I built and ran this locally. The change works in the editor and (where relevant) in a built player.
  • Transform access is combined and limited — In hot paths, transform reads/writes go through TransformAccessArray or are otherwise batched. I have not added per-frame transform.position / transform.rotation / transform.localPosition calls inside loops. Whenever I need both position and rotation, I use the combined APIs — SetPositionAndRotation / SetLocalPositionAndRotation for writes, GetPositionAndRotation / GetLocalPositionAndRotation for reads — instead of two separate property accesses; the combined call does one local-to-world matrix traversal instead of two.
  • Addressables used for asset/memory loading — Any new asset loads go through Addressables. No new Resources.Load, no direct asset references that pull large content into memory on scene load.
  • No new GetComponent / AddComponent where avoidable — Where unavoidable, the result is cached on a field, and any GetComponent<T> is replaced with TryGetComponent<T>(out var x) — bare GetComponent will be denied. TryGetComponent is the modern API (Unity 2019.2+) and skips the Editor-only GC allocation GetComponent causes when a component is missing: Unity wraps the null return in a managed "fake null" object so its overloaded == operator can still detect destroyed C++ objects, and constructing that wrapper allocates; TryGetComponent returns a bool plus out parameter and never builds the wrapper. None of these calls run inside Update, LateUpdate, FixedUpdate, jobs, or other per-frame code paths.
  • Per-frame work is scheduled through BasisEventDriver — Any new per-frame work hooks into BasisEventDriver rather than adding standalone Update / LateUpdate / FixedUpdate callbacks on a MonoBehaviour.
  • Anything added to BasisEventDriver is bulletproof, or guarded by try/catchBasisEventDriver runs the single per-frame tick that drives the whole framework (network apply, local player sim, blendshapes, JigglePhysics, nameplates, and more) as one sequential chain. An unhandled exception anywhere in that chain aborts the rest of the tick, so every step after the throwing one is silently skipped for that frame. New work added to the driver must either be guaranteed not to throw, or be wrapped in a try/catch that contains the failure and surfaces it through BasisDebug — logged once / rate-limited, never every frame (see the existing HVRBasisBuiltInAddresses.Simulate() guard for the pattern). Expect this to be scrutinized closely in review.
  • Considered jobification — I asked whether this work can be moved to a Unity Job (Burst-compiled where possible). If it can, it is. If it cannot, the reason is in Notes.
  • No needless { get; set; } properties or access lockdowns — Public fields are fine; Basis is meant to be read and modified freely, so don't wall things off private/internal without a real reason. Don't wrap a field in { get; set; } when the accessors do nothing — property accessors have a real performance cost vs direct field access, and the lead maintainer prefers plain fields (or a method / setter-only property when only the setter needs logic) over a noop-getter pair. For .Instance singletons, callers reassigning Type.Instance is allowed; if that would break your code, log a warning or throw — don't block the assignment. Locking down access is not your call.
  • Camera access goes through BasisLocalCameraDriver — Code that needs the local camera (transform, projection, rig data, etc.) pulls it from BasisLocalCameraDriver rather than looking one up itself. Don't roll a separate camera discovery path.
  • Logging uses BasisDebug — All new logging calls go through BasisDebug.Log / BasisDebug.LogWarning / BasisDebug.LogError (with an appropriate LogTag) instead of UnityEngine.Debug.Log / Debug.LogWarning / Debug.LogError. BasisDebug routes through Basis's tagged, color-coded logger and respects the project-wide LoggingDisabled toggle so logging can be killed at runtime; bare Debug.Log calls bypass that and will be denied.
  • No scene-wide discovery for dependencies — New code is architected so it does not need FindObjectOfType / FindObjectsOfType / GameObject.Find / FindGameObjectsWithTag to locate what it depends on. References are wired in — registered through an existing manager/driver, injected at init, or passed in by the caller — rather than discovered by scanning the scene at runtime. If a scene scan is genuinely unavoidable, justify it under Notes.
  • No allocations in hot paths — Per-frame code (Update / LateUpdate / FixedUpdate, simulation loops, jobs, anything called once per frame or more) does not allocate. No new on reference types, no LINQ, no string concatenation/interpolation, no boxing, no foreach over interface-typed collections. Allocate once at init and reuse the buffer.
  • No debugging in hot paths — No log calls of any kind on per-frame paths, including BasisDebug. Hot-path logging floods the console and incurs cost on every frame regardless of whether the message is filtered out downstream. If a hot-path log is needed while iterating, gate it behind #if UNITY_EDITOR and remove (or leave gated) before merge.
  • Hot-path collection access is optimized — Cache .Count (lists) / .Length (arrays) into a local int before the loop instead of re-reading the property each iteration. Prefer T[] (with a separate length int when the array is over-sized) over List<T> where the data is hot — Unity's mono BCL doesn't expose CollectionsMarshal.AsSpan(List<T>), so a list can't be fed into Span<T> / unsafe paths cleanly. Where the perf justifies it, drop into Span<T> / ref locals / Unsafe.As / unsafe pointer code to skip bounds checks and copies, and call out the invariants you're relying on under Notes so reviewers can sanity-check them.

Testing details

Tick the platforms you actually tested on. Leave the rest unticked — these are informational and do not block merge.

  • Windows
  • Linux
  • Android
  • iOS
  • macOS

Input / control mode coverage:

  • Tested in VR (note headset under Notes)
  • Tested in desktop / non-VR mode
  • Tested with phone controls (mobile touch input)
  • N/A — change does not touch player/XR/input code

Where applicable, confirm these flows still work after your changes:

  • Hot-switching (desktop ↔ VR mode swap at runtime)
  • Avatar swapping
  • Server swapping (joining / leaving / changing servers)
  • N/A — change does not touch any of the above

Notes

Per-frame check (BasisEventDriver). The adapter's per-frame Blit/rebind runs in a standalone LateUpdate, matching the established pattern for leaf per-frame components in directly comparable packages — com.basis.integration.audiolink (BasisAudioLinkReactiveLight, BasisAudioLinkReactiveMaterial) and several com.basis.mediaplayer runtime scripts all use Update/LateUpdate. BasisEventDriver is a core, hard-wired call chain with no registration API; hooking a leaf, optional, inert-until-deps integration into it would mean editing core to reference an assembly that may not compile. Happy to move it onto the driver if you'd prefer that for this class of component — flag it and I'll wire it in.

BasisEventDriver bulletproofing — N/A: nothing is added to the driver (see above).

Scene discovery. The dependencies are serialized references (Player, Target) and are the primary path. The single FindFirstObjectByType<LTCGI_UdonAdapter>() is a one-shot fallback in OnEnable, used only when Target is left unassigned in the inspector; the result is cached. No per-frame scanning. Can make the reference mandatory and drop the fallback if you'd rather forbid the scan entirely.

GetComponent. The only lookup is GetComponentInParent<BasisMediaPlayer>(), once in OnEnable/Reset, cached on the Player field. No bare GetComponent, nothing on a per-frame path.

Allocations / hot path. The LateUpdate path (Blit + EnsureBound) is GC-free: Blit constructs only Vector2 structs (stack, no heap), and the LOD-rebind loop runs solely on a detected reset, not the steady per-frame path.

Transform / Addressables / Camera — N/A: the package touches no transforms, loads no assets, and accesses no camera.

Jobification — N/A: the per-frame work is a single Graphics.Blit (GPU); there is no CPU-side work to move to a Burst job.

Testing. Verified on Windows in both desktop and VR. Specifically exercised the desktop↔VR runtime mode switch: LTCGI's globals get cleared on the switch and the adapter's EnsureBound self-heal re-asserts the feed within a frame, so the screen keeps lighting the room after the swap. Avatar/server swapping are unrelated to this change. Headset: Quest Pro over Virtual Desktop (PCVR streaming).

towneh added 2 commits June 20, 2026 01:32
…ia player

Adds com.basis.integration.ltcgi: BasisLTCGIVideoAdapter feeds a
BasisMediaPlayer's live output texture into LTCGI's dynamic blur chain, so a
screen lit by the player emits matching realtime area light. The assembly
compiles to nothing unless both com.basis.mediaplayer and at.pimaker.ltcgi are
present (asmdef define constraints), so it is inert until both are installed.
Swap the three init-guard Debug.LogWarning calls for BasisDebug.LogWarning
with LogTag.Video, matching the sibling integration packages, so the warnings
respect the project-wide logging toggle. Adds the BasisDebug assembly
reference.
@towneh towneh requested a review from dooly123 June 20, 2026 00:49
@towneh towneh added the enhancement New feature or request label Jun 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant