Repository navigation
fix(expressions): reject fractional strings converting to int/long - #4383
Open
Shahzeb-DataEngineer wants to merge 7 commits into
Open
Shahzeb-DataEngineer wants to merge 7 commits into
Shahzeb-DataEngineer wants to merge 7 commits into
Conversation
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
1 of 3 tasks
ebyhr
reviewed
Nov 16, 2026
| elif IntegerType.min > number: | ||
| return IntBelowMin() | ||
| return LongLiteral(int(number)) | ||
| return LongLiteral(_to_integral(number, type_var)) |
Member
There was a problem hiding this comment.
The range check runs before the fractional check. So "2147483647.5" to IntegerType returns IntAboveMax, but DecimalLiteral(Decimal("2147483647.5")).to(IntegerType()) raises. Intentional?
Contributor
Author
There was a problem hiding this comment.
Good catch, it was not intentional. I moved the fractional check before the range check in both StringLiteral.to(IntegerType) and .to(LongType), mirroring DecimalLiteral, so "2147483647.5" now raises ValueError instead of returning IntAboveMax. Added test_fractional_string_near_integral_type_bounds_raises covering both bounds for both types. Fixed in 472b861.
…iteral.to Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4382
Rationale for this change
StringLiteral.to(IntegerType)andStringLiteral.to(LongType)parsed withDecimalthen truncated with plainint(), so "2.6" became 2 and a bound predicate such asx < "2.6"on an int column silently becamex < 2, wrongly dropping rows with x = 2 and feeding the same wrong literal into partition pruning. The siblingDecimalLiteralpath already rejects fractional values via_to_integral, and the Java reference implementation refuses the conversion entirely. This change reuses_to_integralin both StringLiteral paths.Are these changes tested?
Yes. New test
test_fractional_string_to_integral_type_raisesmirrorstest_fractional_decimal_to_integral_type_raises; it fails on unpatched code and passes with the fix.test_string_literalupdated to no longer assert the truncating behavior.tests/expressions/test_literals.py: 180 passed.tests/expressions/: 594 passed, 2 failed (pre-existing pyparsing-version failures on clean main, verified viagit stash).ruff checkandruff format --checkclean on both changed files.Are there any user-facing changes?
Yes, behavior change for the better: converting a fractional string such as "2.6" to an int/long column now raises ValueError at bind time instead of silently truncating. Integral strings ("2.0", "1e3") still convert exactly.