Skip to content

MDEV-39791: Handle count aggregate optimization for replay purpose#5145

Open
bsrikanth-mariadb wants to merge 1 commit into
bb-12.3-MDEV-39368-test-replayfrom
13.1-MDEV-39791-handle-count-optimization-in-replay
Open

MDEV-39791: Handle count aggregate optimization for replay purpose#5145
bsrikanth-mariadb wants to merge 1 commit into
bb-12.3-MDEV-39368-test-replayfrom
13.1-MDEV-39791-handle-count-optimization-in-replay

Conversation

@bsrikanth-mariadb
Copy link
Copy Markdown
Contributor

@bsrikanth-mariadb bsrikanth-mariadb commented May 29, 2026

Separately dump into the optimizer context, the count aggregation value
of any one non-nullable column of a table if any are used in the query.
This is only suppoerted for myisam tables.

During opt_sum_query() call, record the count aggregation values into
the optimizer context recorder. Once the context is being stored into
the information_schema table, dump all these recorded values into it.

During replay, parse the count aggregate values and store them in memory.
Later when opt_sum_query() requires the count value, hook the recorded
value from the parsed context.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for recording and replaying count aggregate contexts (count_agg) within the optimizer context store/replay framework. While the implementation successfully integrates the new structures and serialization routines, several critical issues must be addressed. Most notably, there is a type confusion vulnerability in opt_sum.cc where a cast to Item_field* is performed without verifying the item type, which could lead to server crashes. Additionally, there are bugs involving the serialization of a pointer instead of the actual count value, incorrect string appending of a ulong counter, potential counter desynchronization during OOM events, and swapped error messages in the JSON parsing routines.

Comment thread sql/opt_sum.cc Outdated
Comment thread sql/opt_context_store_replay.cc Outdated
Comment thread sql/opt_context_store_replay.cc Outdated
Comment on lines +822 to +837
void Optimizer_context_recorder::record_count_agg(TABLE *tbl, longlong count)
{
table_context_for_store *table_ctx=
get_table_context(tbl->pos_in_table_list);

if (unlikely(!table_ctx))
return; // OOM

auto *count_agg_ctx= new (mem_root) count_agg;
if (unlikely(!count_agg_ctx))
return; // OOM

count_agg_ctx->call_number= ++count_agg_counter;
count_agg_ctx->count= count;
table_ctx->count_agg_list.push_back(count_agg_ctx, mem_root);
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If get_table_context or the allocation of count_agg_ctx fails (e.g., due to OOM), record_count_agg returns early without incrementing count_agg_counter. However, during replay, infuse_count_agg increments count_agg_counter unconditionally. This will cause the counters to get out of sync for any subsequent count aggregates. Incrementing count_agg_counter at the very beginning of record_count_agg ensures that the counters remain synchronized even if recording fails for a specific call.

void Optimizer_context_recorder::record_count_agg(TABLE *tbl, longlong count)
{
  count_agg_counter++;
  table_context_for_store *table_ctx=
      get_table_context(tbl->pos_in_table_list);

  if (unlikely(!table_ctx))
    return; // OOM

  auto *count_agg_ctx= new (mem_root) count_agg;
  if (unlikely(!count_agg_ctx))
    return; // OOM

  count_agg_ctx->call_number= count_agg_counter;
  count_agg_ctx->count= count;
  table_ctx->count_agg_list.push_back(count_agg_ctx, mem_root);
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed the logic. this is not relevant.

Comment thread sql/opt_context_store_replay.cc Outdated
Comment thread sql/opt_context_store_replay.cc Outdated
@bsrikanth-mariadb bsrikanth-mariadb force-pushed the 13.1-MDEV-39791-handle-count-optimization-in-replay branch 2 times, most recently from 5f4ea85 to 65823a0 Compare May 29, 2026 09:16
@spetrunia spetrunia force-pushed the bb-12.3-MDEV-39368-test-replay branch from 039fe52 to 90b9f07 Compare May 29, 2026 11:57
@bsrikanth-mariadb bsrikanth-mariadb force-pushed the 13.1-MDEV-39791-handle-count-optimization-in-replay branch 4 times, most recently from 2bbef5c to 438acc1 Compare June 2, 2026 12:35
Separately dump into the optimizer context, the count aggregation value
of any one non-nullable column of a table if any are used in the query.
This is only suppoerted for myisam tables.

During opt_sum_query() call, record the count aggregation values into
the optimizer context recorder. Once the context is being stored into
the information_schema table, dump all these recorded values into it.

During replay, parse the count aggregate values and store them in memory.
Later when opt_sum_query() requires the count value, hook the recorded
value from the parsed context.
@bsrikanth-mariadb bsrikanth-mariadb force-pushed the 13.1-MDEV-39791-handle-count-optimization-in-replay branch from 438acc1 to fabfa22 Compare June 2, 2026 23:24
@spetrunia
Copy link
Copy Markdown
Member

MariaDB [test]> set optimizer_record_context=1;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> create table t10 (a int not null) engine=myisam;
Query OK, 0 rows affected (0.014 sec)

MariaDB [test]> insert into t10 select seq from seq_1_to_3;
Query OK, 3 rows affected (0.004 sec)
Records: 3  Duplicates: 0  Warnings: 0
MariaDB [test]> explain select count(*) from t10;
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                        |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
|    1 | SIMPLE      | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | Select tables optimized away |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
1 row in set (0.001 sec)

^ this is not caught by capture at all.
This EXPLAIN output means that the query did use get_exact_record_count...

To check that I do have your patch, etc:

MariaDB [test]> explain select count(a) from t10;

Now, this does hits a breakpoint in Optimizer_context_recorder::record_count_agg, as expected.

@spetrunia
Copy link
Copy Markdown
Member

count_agg
record_count_agg
infuse_count_agg

This is a bad name. Please change to exact_row_count .

Comment thread sql/opt_sum.cc
{
longlong tmp_count;
if (!replay->infuse_count_agg(table, &tmp_count))
{
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, there is a call above:

 cnt_item->make_const((longlong) count);

which saves the value of count into the Item_sum_count.
And then, now you do infuse_count_agg . Isn't that too late?

SET sql_mode='STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
SET timestamp=1780212030.000000;
SET timestamp=1780203030.000000;
CREATE DATABASE IF NOT EXISTS db1;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not have been in your patch. Why put it here?
This is an unrelated failure, I've fixed it in 1ae5cfb (now pushed).

String tbl_name;
append_full_table_name(tbl->pos_in_table_list, &tbl_name);
if (search((uchar *) tbl_name.c_ptr_safe(), tbl_name.length()))
return; // dupicate entry
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo, dupicate.

static int parse_records_in_range_context(MEM_ROOT *mem_root, json_engine_t *je,
String *err_buf,
records_in_range_call_record *rir_ctx);
// static int parse_count_agg_context(MEM_ROOT *mem_root, json_engine_t *je,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Garbage, remove?

Copy link
Copy Markdown
Member

@spetrunia spetrunia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix the above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants