Skip to content

[fix](be) Fix auto_partition_name crash for invalid runtime arguments #67218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
felixwluo wants to merge 3 commits into
base: master
Choose a base branch
from
+67 −41
Open

[fix](be) Fix auto_partition_name crash for invalid runtime arguments #67218

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0f28df0
[opt](scan) Use lazy-init for segment iterators and avoid caching all…
mrhhsg May 29, 2024
2652bbc
[fix](be) Fix auto_partition_name crash for invalid runtime arguments
felixwluo Aug 27, 2026
a938fea
Merge branch 'master' into fix-auto-partition-name
felixwluo Aug 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 59 additions & 41 deletions be/src/exprs/function/function_string_misc.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,20 +100,23 @@ class FunctionAutoPartitionName : public IFunction {
size_t get_number_of_arguments() const override { return 0; }
bool is_variadic() const override { return true; }
bool use_default_implementation_for_nulls() const override { return false; }
bool use_default_implementation_for_constants() const override { return false; }

@github-actions github-actions Bot Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve the constant-expression contract here

This override is also what VectorizedFnCall::is_constant() consults, so it does more than keep the control columns wrapped during execute_impl(). With folding skipped, auto_partition_name('list', 'x') now produces an ordinary ColumnString instead of a cached ColumnConst. For example, SELECT /*+SET_VAR(debug_skip_fold_constant=true)*/ trim('abc', auto_partition_name('list', 'x')) reaches FunctionTrim::get_arguments_that_are_always_constant() == {1} and is rejected by the generic constant-argument check, although the same deterministic nested expression was constant before this change. Literal-only projections also recompute and allocate the name once per input row (load planning is one path that explicitly skips folding). Please preserve VectorizedFnCall constness while validating the original control arguments through FunctionContext constant-column metadata or another decoupled mechanism, and add a no-fold nested regression.

DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
return std::make_shared<DataTypeString>();
}

Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
uint32_t result, size_t input_rows_count) const override {
size_t argument_size = arguments.size();
if (argument_size < 2) {
return Status::InvalidArgument(
"function auto_partition_name must contains at least two arguments");
}

auto const_null_map = ColumnUInt8::create(input_rows_count, 0);
auto null_map = ColumnUInt8::create(input_rows_count, 0);
std::vector<const ColumnString::Chars*> chars_list(argument_size);
std::vector<const ColumnString::Offsets*> offsets_list(argument_size);
std::vector<const ColumnString*> string_columns(argument_size);
std::vector<bool> is_const_args(argument_size);
std::vector<const ColumnUInt8::Container*> null_list(argument_size);
std::vector<ColumnPtr> argument_null_columns(argument_size);

std::vector<ColumnPtr> argument_columns(argument_size);
for (int i = 0; i < argument_size; ++i) {
Expand All @@ -122,40 +125,49 @@ class FunctionAutoPartitionName : public IFunction {
if (const auto* nullable =
check_and_get_column<const ColumnNullable>(*argument_columns[i])) {
null_list[i] = &nullable->get_null_map_data();

@github-actions github-actions Bot Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Keep the materialized null map alive

For a ColumnConst(ColumnNullable(...)), convert_to_full_column_if_const() creates a new full ColumnNullable with a freshly allocated null map. null_list[i] keeps only a raw pointer into that map, and the next assignment retains the nested string but releases the sole owner of the nullable parent and its null map. The later checks at lines 151/223/275 therefore read freed storage. The deleted argument_null_columns[i] = nullable->get_null_map_column_ptr() was the lifetime guard; the changed Consted tests build this exact const-nullable shape, as do supported LIST calls with NULL values. Please retain either the full materialized nullable column or its null-map ColumnPtr for the duration of execution, and cover both NULL and non-NULL const-nullable inputs under ASAN.

argument_null_columns[i] = nullable->get_null_map_column_ptr();
argument_columns[i] = nullable->get_nested_column_ptr();
} else {
null_list[i] = &const_null_map->get_data();
}

const auto& [col, is_const] =
unpack_if_const(block.get_by_position(arguments[i]).column);

const auto* col_str = assert_cast<const ColumnString*>(argument_columns[i].get());
chars_list[i] = &col_str->get_chars();
offsets_list[i] = &col_str->get_offsets();
is_const_args[i] = is_const;
string_columns[i] = col_str;
is_const_args[i] = is_column_const(*block.get_by_position(arguments[i]).column);
}

auto res = ColumnString::create();
auto& res_data = res->get_chars();
auto& res_offset = res->get_offsets();
res_offset.resize(input_rows_count);

std::string partition_type(chars_list[0]->raw_data(), (*offsets_list[0])[0]);
if (input_rows_count == 0) {
block.get_by_position(result).column = std::move(res);
return Status::OK();
}
if (!is_const_args[0]) {
return Status::InvalidArgument(
"auto_partition_name must accept literal for 1st argument");
}
if ((*null_list[0])[0]) {
return Status::InvalidArgument(
"function auto_partition_name must accept range|list for 1st argument");
}

std::string partition_type = string_columns[0]->get_data_at(0).to_string();
std::transform(partition_type.begin(), partition_type.end(), partition_type.begin(),
[](unsigned char c) { return static_cast<char>(std::tolower(c)); });
// partition type is list|range
if (partition_type == "list") {
return _auto_partition_type_of_list(chars_list, offsets_list, is_const_args, null_list,
res_data, res_offset, input_rows_count,
argument_size, block, result, res);
} else {
return _auto_partition_type_of_range(chars_list, offsets_list, is_const_args, res_data,
return _auto_partition_type_of_list(string_columns, is_const_args, null_list, res_data,
res_offset, input_rows_count, argument_size, block,
result, res);
} else if (partition_type == "range") {
return _auto_partition_type_of_range(string_columns, is_const_args, null_list, res_data,
res_offset, input_rows_count, argument_size, block,
result, res);
}
return Status::OK();
return Status::InvalidArgument(
"function auto_partition_name must accept range|list for 1st argument");
}

private:
Expand Down Expand Up @@ -194,8 +206,7 @@ class FunctionAutoPartitionName : public IFunction {

return first;
}
Status _auto_partition_type_of_list(std::vector<const ColumnString::Chars*>& chars_list,
std::vector<const ColumnString::Offsets*>& offsets_list,
Status _auto_partition_type_of_list(std::vector<const ColumnString*>& string_columns,
std::vector<bool>& is_const_args,
const std::vector<const ColumnUInt8::Container*>& null_list,
auto& res_data, auto& res_offset, size_t input_rows_count,
Expand All @@ -207,20 +218,15 @@ class FunctionAutoPartitionName : public IFunction {
res_p.reserve(argument_size * 5);
res_p += 'p';
for (int col = 1; col < argument_size; col++) {
const auto& current_offsets = *offsets_list[col];
const auto& current_chars = *chars_list[col];
const auto& current_nullmap = *null_list[col];

if (current_nullmap[row]) {
res_p += 'X';
} else {
auto idx = index_check_const(row, is_const_args[col]);

int size = current_offsets[idx] - current_offsets[idx - 1];
const char* raw_chars =
reinterpret_cast<const char*>(&current_chars[current_offsets[idx - 1]]);
// convert string to u16string in order to convert to unicode strings
const std::string raw_str(raw_chars, size);
const std::string raw_str = string_columns[col]->get_data_at(idx).to_string();
auto u16string = _string_to_u16string(raw_str);
res_p += _string_to_unicode(u16string) + std::to_string(u16string.size());
}
Expand Down Expand Up @@ -253,26 +259,38 @@ class FunctionAutoPartitionName : public IFunction {
return curr_len;
}

Status _auto_partition_type_of_range(std::vector<const ColumnString::Chars*>& chars_list,
std::vector<const ColumnString::Offsets*>& offsets_list,
std::vector<bool>& is_const_args, auto& res_data,
auto& res_offset, size_t input_rows_count,
size_t argument_size, Block& block, uint32_t result,
auto& res) const {
std::string range_type(chars_list[1]->raw_data(), (*offsets_list[1])[0]);
Status _auto_partition_type_of_range(
std::vector<const ColumnString*>& string_columns, std::vector<bool>& is_const_args,
const std::vector<const ColumnUInt8::Container*>& null_list, auto& res_data,
auto& res_offset, size_t input_rows_count, size_t argument_size, Block& block,
uint32_t result, auto& res) const {
if (argument_size != 3) {
return Status::InvalidArgument(
"range auto_partition_name must contains three arguments");
}
if (!is_const_args[1]) {
return Status::InvalidArgument(
"auto_partition_name must accept literal for 2nd argument");
}
if ((*null_list[1])[0]) {
return Status::InvalidArgument(
"range auto_partition_name must accept year|month|day|hour|minute|second for "
"2nd argument");
}
std::string range_type = string_columns[1]->get_data_at(0).to_string();
std::transform(range_type.begin(), range_type.end(), range_type.begin(),
[](unsigned char c) { return static_cast<char>(std::tolower(c)); });
if (range_type != "year" && range_type != "month" && range_type != "day" &&
range_type != "hour" && range_type != "minute" && range_type != "second") {
return Status::InvalidArgument(
"range auto_partition_name must accept year|month|day|hour|minute|second for "
"2nd argument");
}

res_data.resize(15 * input_rows_count);
for (int i = 0; i < input_rows_count; i++) {
const auto& current_offsets = *offsets_list[2];
const auto& current_chars = *chars_list[2];

auto idx = index_check_const(i, is_const_args[2]);
int size = current_offsets[idx] - current_offsets[idx - 1];
const char* tmp =
reinterpret_cast<const char*>(&current_chars[current_offsets[idx - 1]]);
std::string to_split_s(tmp, size);
std::string to_split_s = string_columns[2]->get_data_at(idx).to_string();

// check the str if it is date|datetime
RE2 date_regex(R"(^\d{4}-\d{2}-\d{2}( \d{2}:\d{2}:\d{2})?$)");
Expand All @@ -283,7 +301,7 @@ class FunctionAutoPartitionName : public IFunction {
// split date_str from (yyyy-mm-dd hh:mm:ss) to ([yyyy, mm, dd, hh, mm, ss])
std::vector<std::string> date_str(6);
date_str[0] = to_split_s.substr(0, 4);
for (int ni = 5, j = 1; ni <= size; ni += 3, j++) {
for (size_t ni = 5, j = 1; ni <= to_split_s.size(); ni += 3, j++) {
date_str[j] = to_split_s.substr(ni, 2);
}
int curr_len = 0;
Expand Down
8 changes: 8 additions & 0 deletions be/test/exprs/function/function_string_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ TEST(function_string_test, function_auto_partition_name_case_insensitive_test) {
const DataSet list_data_set = {
{{std::string("LIST"), std::string("edc_server2")}, std::string("pedc5fserver211")},
{{std::string("LiSt"), std::string("edc_server2")}, std::string("pedc5fserver211")},
{{std::string("LIST"), std::string("10")}, std::string("p102")},
};
for (const auto& data : list_data_set) {
ASSERT_TRUE(check_function<DataTypeString>("auto_partition_name", list_input_types, {data})
Expand All @@ -106,6 +107,13 @@ TEST(function_string_test, function_auto_partition_name_case_insensitive_test) {
ASSERT_TRUE(check_function<DataTypeString>("auto_partition_name", range_input_types, {data})
.ok());
}

const DataSet invalid_range_data_set = {
{{std::string("RANGE"), std::string("DAY")}, std::string("")},
};
ASSERT_FALSE(check_function<DataTypeString>("auto_partition_name", list_input_types,
invalid_range_data_set, -1, -1, true)
.ok());
}

TEST(function_string_test, function_string_substr_test) {
Expand Down
Loading