fix(responses_converter): omit tool_choice when request has no tools#1925
Open
bg51717 wants to merge 1 commit into
Open
fix(responses_converter): omit tool_choice when request has no tools#1925bg51717 wants to merge 1 commit into
bg51717 wants to merge 1 commit into
Conversation
The responses->chat converter drops an absent/empty tools list but kept tool_choice, so no-tool requests (LLM judges / plain generation) carried a dangling tool_choice="auto" -- which vLLM rejects: "When using `tool_choice`, `tools` must be set." Pop tool_choice and parallel_tool_calls when there are no real tools so the emitted chat params are conformant. Adds unit tests for the absent-tools / empty-tools / with-tools paths. Signed-off-by: bg51717 <biguo@nvidia.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bg51717
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The responses->chat converter drops an absent/empty
toolslist but kepttool_choice, producing requests withtool_choiceset and notools— which vLLM rejects: "When usingtool_choice,toolsmust be set." Droptool_choiceandparallel_tool_callswhen there are no real tools, so no-tool requests (LLM judges / plain generation) are emitted conformantly.Reworked from the original
app.py-based patch: the converter now lives innemo_gym/responses_converter.py, so the fix moved there (rebased on currentmain). Adds unit tests for the absent-tools / empty-tools / with-tools paths.