Skip to content

Validate cached model files before reusing them - #2242

Merged
felladrin merged 1 commit into
mainfrom
fix/validate-cached-model-files
Jul 28, 2026
Merged

Validate cached model files before reusing them#2242
felladrin merged 1 commit into
mainfrom
fix/validate-cached-model-files

Conversation

@felladrin

Copy link
Copy Markdown
Owner

Description

Currently, downloadFileFromHuggingFaceRepository returns early on fs.existsSync(localFilePath), so any file already in server/models/ is trusted without validation. A file downloaded only in part (process killed during fs.writeFileSync, disk full) is never fetched again: the reranker fails with Load model from ... failed:Protobuf parsing failed. on every startup, and only deleting the directory by hand recovers.

This PR does two things:

  • Compares each cached file against the size the Hub reports (fileDownloadInfo) and downloads it again when they differ, so an already corrupt cache recovers on its own. The downloaded body is checked against that size too, before anything touches the models directory.
  • Writes to a sibling .part-<pid> path and renames it once the whole body is on disk, so an interrupted write is never visible where the next startup would trust it.

Files are still checked one by one, so a missing tokenizer.json is fetched on its own while the 130MB model stays cached.

This is pre-existing behavior, not something the ONNX migration introduced: the same short-circuit served the old GGUF download, and a truncated GGUF failed identically under llama-server.

Known limitations

The size check costs one metadata request per file at startup, about 600ms for the three of them. When the Hub is unreachable the failure is logged and the cached files are used as they are, so an offline server still starts with a warm cache (a corrupt cache can only be detected against the Hub).

Type of Change

  • Bug fix
  • New feature
  • Documentation
  • Other (refactor, build, chore)

How to test

  1. Populate the cache: npx vitest run --config vitest.integration.config.ts
  2. Truncate the model:
head -c 5000000 server/models/jinaai/jina-reranker-v1-tiny-en/onnx/model.onnx > /tmp/t && mv /tmp/t server/models/jinaai/jina-reranker-v1-tiny-en/onnx/model.onnx
  1. Run the integration test again. It downloads the model again and passes, logging:
Cached onnx/model.onnx has 5000000 bytes instead of 132350375, so it will be downloaded again.

On main this step fails with Protobuf parsing failed, and keeps failing on every later run.
4. Delete only server/models/jinaai/jina-reranker-v1-tiny-en/tokenizer.json and run it once more. Just that file is fetched (the run finishes in about a second), so the model is not downloaded again.
5. npx vitest run server/downloadFileFromHuggingFaceRepository.test.ts for the unit tests. Three of them fail on main: the truncated cache is trusted, a short response resolves instead of throwing, and a write that fails mid-way leaves model.onnx behind.

Checklist

  • npm run lint passes
  • Tests pass (npm run test), with tests added where it made sense

Currently, any file already present in server/models/ is trusted without
validation, so a file downloaded only in part is loaded as if it were
complete. The reranker then fails on every startup with "Protobuf parsing
failed", and only deleting the directory by hand recovers.

Cached files are now compared against the size the Hub reports, and
downloads land on a sibling .part-<pid> path that is renamed once the whole
body is on disk. Files are still checked one by one, so a missing tokenizer
is fetched on its own while the 130MB model stays cached.
@felladrin
felladrin marked this pull request as ready for review July 28, 2026 18:13
@felladrin
felladrin merged commit 7148d98 into main Jul 28, 2026
6 of 7 checks passed
@felladrin
felladrin deleted the fix/validate-cached-model-files branch July 28, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant