Add OpenAIBatchChatGenerator for processing batch API requests (#8482)

[DhanushPillay]

Related Issues

• fixes OpenAI's Batch API Support #8482

Proposed Changes:

Adds the OpenAIBatchChatGenerator to support OpenAI's Batch API. This provides a 50% cost reduction for high-throughput, latency-insensitive workloads (like evals).

• Input/Output: Behaves similarly to the standard generator but takes a list of conversations (list[list[ChatMessage]]) instead of just one, and outputs the exact same structure back.
• Payload Building (_build_jsonl): Converts the conversations into a JSONL file in memory. Generates a custom_id for each request to keep track of their original order.
• Execution (run / run_async): Uploads the JSONL file to OpenAI, creates the batch job, and enters a polling loop (checking status every 30 seconds).
• Result Parsing (_parse_results): Once the batch completes, we download the output file. Because the Batch API returns raw JSON dicts instead of standard Pydantic models, we use a lightweight dictionary parser (_parse_choices) to convert them back into Haystack ChatMessage objects, mapping them to their original order using the custom_id.

How did you test it?

• Wrote a full suite of unit tests in test_openai_batch.py mocking out the OpenAI client. Tests cover serialization, building the jsonl, parsing results (including out-of-order responses), and error handling/timeouts.
• Ran hatch run test:unit and hatch run test:types (mypy is clean).

Notes for the reviewer

• Result parsing: Since the Batch API returns results as raw JSON lines instead of Pydantic models, I couldn't completely reuse the existing parsing logic from openai.py. I had to write a lightweight dictionary-based parser (_parse_choices), but the logic itself is pretty much identical.
• Tools: I kept explicit tool-calling out of the MVP scope for simplicity, but the parser does still extract tool calls if someone sneaks them in via generation_kwargs, just to make sure we don't silently drop data.

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

@DhanushPillay is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
haystack/components/generators/chat
__init__.py
openai_batch.py
Project Total

This report was generated by python-coverage-comment-action