Skip to content

[FLINK-40338][table-runtime] ELT() throws ClassCastException when index is not INT - #28931

Open
hulincup wants to merge 1 commit into
apache:masterfrom
hulincup:fix/elt-non-int-index-classcast
Open

[FLINK-40338][table-runtime] ELT() throws ClassCastException when index is not INT#28931
hulincup wants to merge 1 commit into
apache:masterfrom
hulincup:fix/elt-non-int-index-classcast

Conversation

@hulincup

@hulincup hulincup commented Aug 6, 2026

Copy link
Copy Markdown

Problem fixed & how

ELT(index, expr, exprs...) accepts any INTEGER_NUMERIC index (TINYINT/SMALLINT/INT/BIGINT). However, EltFunction#eval declares index as java.lang.Number and indexes the varargs array with exprs[(int) index - 1]. Per JLS 5.5, casting a Number reference to int compiles to a checkcast to Integer followed by unboxing, so a Byte, Short, or Long value throws ClassCastException on the success path (1 <= index <= exprs.length).

SELECT ELT(CAST(2 AS BIGINT), 'scala', 'java');
-- java.lang.ClassCastException: class java.lang.Long cannot be cast to class java.lang.Integer

Same for CAST(2 AS TINYINT) and CAST(2 AS SMALLINT).

The out-of-range guard above the cast uses index.longValue(), so out-of-range indices of any type still return NULL correctly; the exception only fires in the valid range. That is why the existing test ELT(9223372036854775807, 'ab', 'b') passes (returns NULL before reaching the cast) and every other existing test uses an INT literal.

Present since FLINK-35987 introduced ELT; confirmed absent from release-1.20 and present from release-2.0.

Fix: narrow the already-unboxed long idx (computed above for the range check) via primitive narrowing (int) idx (JLS 5.1.3, no checkcast) instead of casting the Number reference.

Behavior modified

  • previous: ELT with a non-INT INTEGER_NUMERIC index (TINYINT/SMALLINT/BIGINT) in the valid range 1 <= index <= exprs.length threw ClassCastException.
  • now: non-INT integer indices correctly return the corresponding expression.
  • impact: only the success path for non-INT integer indices; INT indices, NULL, and out-of-range behavior are unchanged.

Code refactored

EltFunction.eval: exprs[(int) index - 1]exprs[(int) idx - 1], with a 3-line comment explaining the JLS rationale.

Features added

N/A

Functions optimized

N/A

Test plan

  • Added 3 regression cases to StringFunctionsITCase.eltTestCases() covering TINYINT, SMALLINT, and BIGINT indices (the three types that previously threw ClassCastException). Each expects the correct expression ("java" for index 2).
  • The local environment runs Java 8 and cannot build Flink master (requires Java 11+), so verification relies on CI:
    mvn -pl flink-table/flink-table-planner -am test -Dtest=StringFunctionsITCase
    

…ex is not INT

EltFunction.eval declares index as java.lang.Number to support
TINYINT/SMALLINT/INT/BIGINT, but indexes the varargs array with
exprs[(int) index - 1]. Per JLS 5.5, casting a Number reference to int
compiles to a checkcast to Integer followed by unboxing, so a Byte,
Short, or Long value throws ClassCastException on the success path
(1 <= index <= exprs.length). The out-of-range guard above uses
index.longValue(), so out-of-range indices of any type still return
NULL correctly; the exception only fires in the valid range.

Narrow the already-unboxed long idx (computed above for the range
check) via primitive narrowing (JLS 5.1.3) instead, which emits no
checkcast and works for every INTEGER_NUMERIC type.

Adds TINYINT/SMALLINT/BIGINT index regression cases to
StringFunctionsITCase.
@flinkbot

flinkbot commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@SEPURI-SAI-KRISHNA

Copy link
Copy Markdown

Hi @hulincup — I'm the reporter of FLINK-40338 and had asked to be assigned before this PR was opened, with a patch and regression tests ready. Per the contribute-code guide, PRs on unassigned tickets aren't reviewed or merged, so could we wait for a committer to decide who takes it?

@raminqaf raminqaf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good. Left a nit

DataTypes.VARCHAR(5))
.testResult(
lit(2).elt("a", "b"), "ELT(2, 'a', 'b')", "b", DataTypes.CHAR(1))
// FLINK-40338: non-INT INTEGER_NUMERIC index must not throw ClassCastException

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this

Suggested change
// FLINK-40338: non-INT INTEGER_NUMERIC index must not throw ClassCastException

Comment on lines +186 to +200
.testResult(
lit(2).cast(DataTypes.TINYINT()).elt("scala", "java"),
"ELT(CAST(2 AS TINYINT), 'scala', 'java')",
"java",
DataTypes.VARCHAR(5))
.testResult(
lit(2).cast(DataTypes.SMALLINT()).elt("scala", "java"),
"ELT(CAST(2 AS SMALLINT), 'scala', 'java')",
"java",
DataTypes.VARCHAR(5))
.testResult(
lit(2).cast(DataTypes.BIGINT()).elt("scala", "java"),
"ELT(CAST(2 AS BIGINT), 'scala', 'java')",
"java",
DataTypes.VARCHAR(5))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up, the fix itself looks right. One coverage gap worth closing before merge.

All three new cases pass constant arguments, so ExpressionReducer folds the entire call during optimization and the generated runtime code is never reached. The plans:

-- ELT(CAST(2 AS TINYINT), 'scala', 'java')
== Optimized Execution Plan ==
Calc(select=[CAST('java' AS VARCHAR(5)) AS EXPR$0])     <- ELT folded away

-- ELT(b, 'scala', 'java')   where b TINYINT
== Optimized Execution Plan ==
Calc(select=[ELT(b, 'scala', 'java') AS EXPR$0])        <- ELT reaches the operator

To be clear, these cases do fail without the fix, because the reducer executes the function at plan time. But they only cover the constant-folding path, not the codegen'd operator path that a real job hits, so a future regression in the runtime path wouldn't be caught here.

The case just below at line 201 already uses the field-reference pattern, so extending the existing fields covers it:

.onFieldsWithData(null, null, null, new byte[] {1, 2, 3}, (byte) 2, (short) 2, 2L)
.andDataTypes(
        DataTypes.INT(), DataTypes.STRING(), DataTypes.BYTES(), DataTypes.BYTES(),
        DataTypes.TINYINT(), DataTypes.SMALLINT(), DataTypes.BIGINT())

and then $("f4").elt("scala", "java"), $("f5"), $("f6"), keeping one of the constant cases so the reducer path stays covered too.

I'm the reporter of FLINK-40338 and already have these written and verified locally (they fail with ClassCastException on java.lang.Byte/Short/Long without the fix). Happy to hand them over for you to include here, or to open them as a follow-up if you'd rather keep this PR as is, whichever you prefer.

Comment on lines +186 to +200
.testResult(
lit(2).cast(DataTypes.TINYINT()).elt("scala", "java"),
"ELT(CAST(2 AS TINYINT), 'scala', 'java')",
"java",
DataTypes.VARCHAR(5))
.testResult(
lit(2).cast(DataTypes.SMALLINT()).elt("scala", "java"),
"ELT(CAST(2 AS SMALLINT), 'scala', 'java')",
"java",
DataTypes.VARCHAR(5))
.testResult(
lit(2).cast(DataTypes.BIGINT()).elt("scala", "java"),
"ELT(CAST(2 AS BIGINT), 'scala', 'java')",
"java",
DataTypes.VARCHAR(5))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just in case write a test with non-literals too please

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@raminqaf agreed. Here's the patch I have locally, already verified, it fails with ClassCastException on java.lang.Byte, Short and Long before the fix and passes after:

.onFieldsWithData(null, null, null, new byte[] {1, 2, 3}, (byte) 2, (short) 2, 2L)
.andDataTypes(
        DataTypes.INT(), DataTypes.STRING(), DataTypes.BYTES(), DataTypes.BYTES(),
        DataTypes.TINYINT(), DataTypes.SMALLINT(), DataTypes.BIGINT())
.testResult(
        $("f4").elt("scala", "java"),
        "ELT(f4, 'scala', 'java')",
        "java",
        DataTypes.VARCHAR(5))
.testResult(
        $("f5").elt("scala", "java"),
        "ELT(f5, 'scala', 'java')",
        "java",
        DataTypes.VARCHAR(5))
.testResult(
        $("f6").elt("scala", "java"),
        "ELT(f6, 'scala', 'java')",
        "java",
        DataTypes.VARCHAR(5))

@hulincup feel free to take these directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants