feat(voyageai): add contextualized embeddings and refresh model support

[fzowl]

What

Refreshes the VoyageAI integration against the current VoyageAI catalog (text, contextualized, multimodal, reranker).

• Contextualized embeddings: voyage-context-* models (e.g. voyage-context-4) route to VoyageAI's contextualized embeddings API. VoyageAIVectorizer accepts a flat list[str] of chunks and calls contextualized_embed with enable_auto_chunking=True and chunk_size=32000, so each input document resolves to a single chunk and embed_many keeps its one-embedding-per-input contract (and cache alignment).
• Multimodal embed_many fix: previously the multimodal branch wrapped the whole batch as [batch], which VoyageAI treats as a single multimodal input with multiple parts — collapsing N requested contents into one embedding. Each content is now sent as its own input ([[item] for item in batch]), so the returned count matches the request count.
• Current model catalog: _get_batch_size now routes the voyage-4 family into the same token-limit tiers as their voyage-3.5 counterparts (voyage-4-lite → 30, voyage-4 → 10); voyage-4-large, voyage-code-4 and voyage-context-* keep the conservative default, matching their 120K-token tier. Every VoyageAI model id already works because the vectorizer passes model strings straight through — this refreshes the throughput heuristic and documents the current families (voyage-4-*, voyage-code-4, voyage-context-*, voyage-multimodal-*) on the class docstring.
• Dependency: bump the voyageai floor to >=0.5.0 (latest), where contextualized auto-chunking (enable_auto_chunking, chunk_size) and the voyage-4 series are available.
• Docs/examples: refresh notebook examples to current models — rerank-2.5 for the reranker and a voyage-context-4 contextualized example for the vectorizer guide.

Why

Contextualized (voyage-context-*) chunk embeddings were not exposed, current models weren't reflected in the docs or batch heuristics, and the multimodal batching bug meant embed_many(["a", "b"]) on a multimodal model could return a single embedding.

Notes

• Adds mocked (no-API-key) tests covering context-model routing, auto-chunking args, async parity, the multimodal no-collapse fix, and the per-model batch-size tiers; these run under make test.
• contextualized_embed calls carry # type: ignore[attr-defined] so type-checking works without vendored stubs; the mypy config uses ignore_missing_imports and does not set warn_unused_ignores, so the ignores stay valid whether or not the client is typed.
• API-dependent tests remain gated behind requires_api_keys.
• Housekeeping: uv.lock shows marker churn beyond the voyageai bump (nvidia/grpcio/etc. environment markers) because it was regenerated with a newer uv resolver than the one that produced the base lock. It pins voyageai==0.5.0 and is self-consistent, and CI installs via uv sync --frozen. Happy to regenerate it against the repo's pinned uv (0.12.3) for a minimal lock diff if preferred.
• Validation: mypy redisvl/utils/vectorize/voyageai.py is clean, isort --profile black + black --target-version py311 report no changes across redisvl/ and tests/, and the mocked VoyageAI routing/batch-size tests pass against voyageai==0.5.0. The full pytest invocation pulls in a session-scoped, Docker-backed Redis fixture that cannot start in this environment, so the mocked tests were also executed standalone (bypassing that fixture) and pass end-to-end. GitHub Actions is not enabled on this fork, so upstream CI / make check should still be the gate on merge.

---

Note

Medium Risk
Changes core embedding batching and API routing for VoyageAI (including a behavioral fix for multimodal embed_many); dependency floor bump may affect installs, but behavior is covered by new mocked tests.

Overview
VoyageAIVectorizer gains first-class support for voyage-context-* models via contextualized_embed, with auto-chunking for documents (and input_type defaulting to document when omitted), one embedding per input, and sync/async parity. Plain-text embed_many now batches by per-model token limits as well as item caps; voyage-4* models are included in the updated batch-size tiers.

Multimodal embed_many no longer wraps the whole batch as a single multi-part input—each item is sent separately so result count matches the request.

The optional voyageai dependency is raised to >=0.5.0 (lockfile pins 0.5.0). User guides add a contextualized embedding example and switch the reranker demo to rerank-2.5. Mocked tests cover context routing, token batching, multimodal batching, and batch-size tiers.

^{Reviewed by Cursor Bugbot for commit 7024c58. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit 7024c58. Configure here.}

[cursor]

Tokenize required for text embeds

Medium Severity

Plain-text embedding now calls Client.tokenize for every input via _batchify_by_tokens, including the single-string dimension probe in _set_model_dims. Voyage loads that tokenizer from Hugging Face for voyageai/{model}, so init and embed_many fail if the Hub is unreachable or no tokenizer exists for the model id, even when the Voyage embed API itself would succeed.

Additional Locations (2)

• redisvl/utils/vectorize/voyageai.py#L452-L454
• redisvl/utils/vectorize/voyageai.py#L303-L304

 

^{Reviewed by Cursor Bugbot for commit 7024c58. Configure here.}