fix: serialize and deserialize typing.Literal - #12286
Merged
Merged
Conversation
serialize_type rendered a Literal's values as bare tokens (e.g. typing.Literal[yes, no]), which deserialize_type then tried to resolve as types and failed on; values that looked like type names such as Literal["int", "str"] were silently converted to types on the round trip. Serialize each value with repr() and read it back with ast.literal_eval so the values (including strings, and commas inside a string) round-trip through pipeline serialization. Fixes deepset-ai#12285
|
@LK-maker-007 is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
|
@anakin87 I can take review of this one |
sjrl
reviewed
Aug 10, 2026
sjrl
reviewed
Aug 10, 2026
sjrl
reviewed
Aug 10, 2026
sjrl
reviewed
Aug 10, 2026
Contributor
Coverage reportClick to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
Contributor
Author
|
Applied all four: trimmed both comments, removed the test comment, and switched the release note to RST double backticks throughout (not just the flagged line). Thanks for the review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues
Proposed Changes:
serialize_typehad no handling fortyping.Literal, so it fell through to the generic-path logic and rendered the literal's values as if they were type names:deserialize_typethen tried to resolveyesandnoas types and raisedDeserializationError. A value that happened to look like a real type name silently corrupted instead of raising:Literal["int", "str"]round-tripped toLiteral[int, str](two types).The fix special-cases
typing.Literalon both sides:repr(), so strings keep their quotes and non-string values (int,bool,bytes,None) are emitted as valid Python literals.ast.literal_evalinstead of the type-resolving generic path.ast.literal_evalis quote-aware, so a comma inside a string value (Literal["a, b", "c"]) no longer splits the arguments, and it is limited to safe literals (str/int/bool/None/bytes/tuples).Scope note: this covers the literal value kinds Python's own
Literalaccepts and that survive a text round-trip (str, bytes, int, bool, None). Enum members asLiteralvalues are out of scope — they can't be reconstructed fromrepr()alone and would need import-path handling; I left that out rather than half-support it. Happy to follow up if you want it.How did you test it?
Unit tests in
test/utils/test_type_serialization.py(serialization,deserialization, andround_tripfor Literal, mirroring the existingCallabletrio), covering the failing case, the silent-corruption case (Literal["int", "str"]), non-string values, a comma inside a value, andLiteralnested inOptional/Union. Proven fail-without/pass-with: with the source change stashed the three new tests fail withDeserializationError; with it applied they pass.Full file green (
195 passed), plus the consumer suites:test_output_adapter.py,test_conditional_router.py,test_pipeline.py(80 passed).ruff check/ruff format --checkclean,mypyclean on the changed module.End-to-end check — this previously crashed on load, now round-trips:
Notes for the reviewer
The
Literalbranch is placed before the generic-args parsing on both sides on purpose:typing.get_origin(Literal[...])istyping.Literal, and its args are values, so they must not go through the type-resolving path (_parse_generic_args/deserialize_typeper arg).Checklist
fix:.