Skip to content

Fix flaky query history reveal test via stable TreeItem ids#4468

Merged
josefs merged 1 commit into
mainfrom
cklin/fix-query-history-reveal-flaky
Jul 18, 2026
Merged

Fix flaky query history reveal test via stable TreeItem ids#4468
josefs merged 1 commit into
mainfrom
cklin/fix-query-history-reveal-flaky

Conversation

@cklin

@cklin cklin commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes a flaky test in query-history-manager.test.ts (the handleRemoveHistoryItem › "should not change the selection" cases), which was failing on CI (e.g. this windows-latest run).

Root cause

HistoryTreeDataProvider.getTreeItem never set a TreeItem.id. When no id is provided, VS Code identifies tree nodes by their label plus their position in the list (<parent>/<index>:<label>). When multiple history items share the same label, that positional identity is ambiguous, so treeView.reveal(item) can resolve to the wrong item or fail outright (Data tree node not found) whenever the list is re-sorted or refreshed.

The test overrides the label format to just ${queryName}, so all six mock variant-analysis items render with the identical label a-query-name (javascript). Combined with the randomised sort order (executionStartTime is faker.number.int()), reveal(selected) intermittently selected the wrong item, so getCurrent() no longer equalled selected.

Impact

Mainly test-only. The production default label format is ${queryName} on ${databaseName} - ${status} ${resultCount} [${startTime}], which includes the start time — so real history labels are effectively unique and users are very unlikely to hit this. The one exception is a user who customises codeQL.queryHistory.format to something non-unique. Because the impact is effectively test-only, no changelog entry is added.

Fix

Set a stable, unique id on each tree item so reveal resolves deterministically regardless of duplicate labels or sort order.

The id must be unique per item. getQueryId alone is not sufficient: a multi-query run (a query suite / multiple .ql files) produces several local-query history entries that are cloned from the same initialInfo and therefore share the same initialInfo.id (see completeQueries in query-history-manager.ts). Since VS Code de-duplicates nodes that share an id — which would break reveal/selection for all but one of the colliding rows — the id for local queries also includes the per-result outputBaseName, which is distinct for each entry (and is persisted, so this also disambiguates already-saved histories). Variant-analysis ids are already unique.

Validation

  • tsc --noEmit and eslint pass.
  • Ran the affected test in a loop (retries disabled) on macOS:
    • Before: ~20/40 failures (~50%).
    • After (initial fix and after addressing review feedback): 0 failures across repeated runs.

The query history tree data provider never set an id on its TreeItems, so
VS Code identified tree nodes by their label plus their position in the
list. When multiple history items share the same label, that positional
identity is ambiguous, so treeView.reveal could resolve to the wrong item
or fail outright ("Data tree node not found") when the list was re-sorted
or refreshed.

This mainly affects tests. The default label format includes the query
start time, so real query history labels are effectively unique and users
are unlikely to hit this (the only exception being a user who customises
the label format to something non-unique). In query-history-manager.test.ts
the label format is overridden to just the query name, producing identical
labels; this made the "should not change the selection" cases flaky,
reproducing locally around 50% of the time and disappearing entirely after
this change. Because the impact is effectively test-only, no changelog
entry is added.

Set a stable, unique id on each tree item so reveal resolves
deterministically regardless of duplicate labels or sort order. The id must
be unique per item: getQueryId is not sufficient on its own because a
multi-query run produces several local-query items that share the same
initialInfo.id, so for local queries we also include the per-result output
base name (VS Code de-duplicates nodes that share an id, which would
otherwise break reveal/selection for the colliding items).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f0809f2f-2c92-4acf-8f51-468c5e79c14a
@cklin
cklin force-pushed the cklin/fix-query-history-reveal-flaky branch from 308e76c to e65dfe8 Compare July 17, 2026 22:05
@cklin
cklin marked this pull request as ready for review July 17, 2026 22:22
@cklin
cklin requested a review from a team as a code owner July 17, 2026 22:22
Copilot AI review requested due to automatic review settings July 17, 2026 22:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds stable query-history tree identifiers to prevent flaky reveal and selection behavior.

Changes:

  • Assigns unique TreeItem.id values.
  • Disambiguates multi-query local results using outputBaseName.
Show a summary per file
File Description
extensions/ql-vscode/src/query-history/history-tree-data-provider.ts Adds stable IDs for local and variant-analysis history items.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 1/1 changed files
  • Comments generated: 2
  • Review effort level: Medium

Comment on lines +29 to +31
return `local:${element.initialInfo.id}:${
element.completedQuery?.query.outputBaseName ?? ""
}`;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that outputBaseName was introduced the same time as multi-query support, so there cannot be any legacy multi-query entry without outputBaseName.

Comment on lines +29 to +31
return `local:${element.initialInfo.id}:${
element.completedQuery?.query.outputBaseName ?? ""
}`;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mainly a test flakiness fix, and I don't think the behavior nuance is important enough to justify having its own tests.

@cklin
cklin requested a review from josefs July 17, 2026 22:52

@josefs josefs left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

A more natural home for the getTreeItemId might be in the query-history-info.ts module, but I don't have a strong opinion about that.

@josefs
josefs merged commit f8df6df into main Jul 18, 2026
33 checks passed
@josefs
josefs deleted the cklin/fix-query-history-reveal-flaky branch July 18, 2026 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants