Validate cached model files before reusing them - #2242
Merged
Conversation
Currently, any file already present in server/models/ is trusted without validation, so a file downloaded only in part is loaded as if it were complete. The reranker then fails on every startup with "Protobuf parsing failed", and only deleting the directory by hand recovers. Cached files are now compared against the size the Hub reports, and downloads land on a sibling .part-<pid> path that is renamed once the whole body is on disk. Files are still checked one by one, so a missing tokenizer is fetched on its own while the 130MB model stays cached.
felladrin
marked this pull request as ready for review
July 28, 2026 18:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Currently,
downloadFileFromHuggingFaceRepositoryreturns early onfs.existsSync(localFilePath), so any file already inserver/models/is trusted without validation. A file downloaded only in part (process killed duringfs.writeFileSync, disk full) is never fetched again: the reranker fails withLoad model from ... failed:Protobuf parsing failed.on every startup, and only deleting the directory by hand recovers.This PR does two things:
fileDownloadInfo) and downloads it again when they differ, so an already corrupt cache recovers on its own. The downloaded body is checked against that size too, before anything touches the models directory..part-<pid>path and renames it once the whole body is on disk, so an interrupted write is never visible where the next startup would trust it.Files are still checked one by one, so a missing
tokenizer.jsonis fetched on its own while the 130MB model stays cached.This is pre-existing behavior, not something the ONNX migration introduced: the same short-circuit served the old GGUF download, and a truncated GGUF failed identically under llama-server.
Known limitations
The size check costs one metadata request per file at startup, about 600ms for the three of them. When the Hub is unreachable the failure is logged and the cached files are used as they are, so an offline server still starts with a warm cache (a corrupt cache can only be detected against the Hub).
Type of Change
How to test
npx vitest run --config vitest.integration.config.tsOn
mainthis step fails withProtobuf parsing failed, and keeps failing on every later run.4. Delete only
server/models/jinaai/jina-reranker-v1-tiny-en/tokenizer.jsonand run it once more. Just that file is fetched (the run finishes in about a second), so the model is not downloaded again.5.
npx vitest run server/downloadFileFromHuggingFaceRepository.test.tsfor the unit tests. Three of them fail onmain: the truncated cache is trusted, a short response resolves instead of throwing, and a write that fails mid-way leavesmodel.onnxbehind.Checklist
npm run lintpassesnpm run test), with tests added where it made sense