Parallel asset loading, file-signature cache, and invalid-file tracking#1
Closed
HectorMu wants to merge 6 commits into
Closed
Parallel asset loading, file-signature cache, and invalid-file tracking#1HectorMu wants to merge 6 commits into
HectorMu wants to merge 6 commits into
Conversation
…ations
- CarCache: New file-signature based cache using LastWriteTime+FileSize instead of SHA256.
Cached bundles skip the GetALLAssetNames() scan on subsequent launches.
Stored at {modDir}\Settings\car_cache.json via JsonFx serialization.
- Parallel loading: AssetBundle.LoadFromFile runs in batches of 4 on the ThreadPool.
Main thread processes loaded bundles (scan, LoadAsset, CreateCar).
- Bundle unload: bundle.Unload(false) after loading assets frees memory immediately.
- Material array fix: renderer.materials accessed once per renderer instead of 2x.
- REflection cache: AssetBundleBridge resolves kernel methods once in static ctor.
- Section/DictionaryExtensions: Added GetOrCreate(string, Func<T>) factory overload
to avoid allocating default values when the key already exists.
- ProfileCarColors: Switched to factory overload; removed noisy Assigning logs.
- Plugin: Removed redundant second cache load (already done inside CreateCars).
- Stale entry cleanup: files deleted from disk are removed from the cache.
- Build infra: all external DLLs reference libs\*;
DistanceGameDir reads from DISTANCE_GAME_DIR env var; publicizer script fixed.
- Added README with full build and deploy instructions.
- New NUnit test project with tests for CarCache, benchmarks, and optimization
regression tests for Section/DictionaryExtensions factory overloads.
- Publicize.ps1: added assembly resolver to find UnityEngine.dll; made all
members public instead of protected; added property getter/setter publicization.
- Fixed targets file: Conditional copy for Centrifuge.Distance.dll.
…rupted files Changing batchDone.WaitOne() to WaitOne(30000) so a single hanging AssetBundle.LoadFromFile can't stall the entire startup. When a timeout occurs, incomplete results are marked with a TimeoutException and skipped in Phase 2.
…instantly - CarCache.InvalidFiles: persists file path + signature for bundles that failed to load. IsFileInvalid() checks both path and signature so a replaced file gets re-tried.\n\n- HasValidBundleSignature(): reads first 8 bytes of each file before calling AssetBundle.LoadFromFile. Rejects non- UnityFS/UnityRaw/UnityWeb headers in microseconds instead of waiting for LoadFromFile to timeout.\n\n- LoadAssetsBundles pre-filter: computes signatures for all files first, then skips any that are in InvalidFiles with a matching signature. Logs the skip count.\n\n- LoadBundle now accepts a pre-computed signature and uses the magic-byte gate.\n\n- Phase 2 error handler calls MarkFileInvalid() so the file is never retried until it changes on disk.\n\n- Stale InvalidFiles entries (files that no longer exist on disk) are cleaned up after processing.\n\n- Combined: first run detects corrupt files at near-zero cost (8 bytes + FileInfo). Subsequent runs skip them before any I/O at all.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces sequential asset-bundle scanning with parallel batched loading + a three-tier
invalid-file detection system (cache → magic bytes → LoadFromFile) to eliminate startup stalls.
Changes
Performance
AssetBundle.LoadFromFileruns in batches of 4 viaThreadPoolLastWriteTime + FileSizeinstead of SHA256 — cached files skipGetAllAssetNames()scanbundle.Unload(false)after processing each batch to free memory immediatelyrenderer.materialscaptured once per renderer instead of double-copiedAssetBundleBridgeresolves types once in static ctor instead of 247×Stall prevention
(path, signature)— never retried unless the file changes on diskUnityFS/UnityRaw/UnityWeb) before callingLoadFromFile— rejects non-bundles in microsecondsWaitOne(30000)on batch — one hanging file can't stall the entire startupFilesandInvalidFilesAPI improvements
Section+DictionaryExtensions:GetOrCreate(string, Func<T>)factory overloadProfileCarColors: switched to factory overload, removed noisy logsPlugin.cs: removed redundant second cache loadBuild infra
Distance.CustomCar.csproj: all refs point to$(SolutionDir)libs\*,DistanceGameDirreads from env varPublicize.ps1: assembly resolver forUnityEngine.dll, all members public, property publicization.gitignore:libs/excluded;README.md: build/deploy instructionsTool.BuildTargets: conditionalCentrifuge.Distance.dllcopyTests
CarCacheTests(17),CarLoadBenchmarks(5),OptimizationTests(8)