MDEV-40827: prefetch MHNSW neighbours during search - #5571
Open
chanztuying wants to merge 2 commits into
Open
Conversation
Issue prefetches for unseen neighbour nodes before evaluating their distances. This overlaps later memory loads with distance calculations for earlier lanes without changing the search result. On a fixed 200k by 1024-dimensional cosine graph with M=6, a crossed AB/BA warm-cache run (30 paired observations) improved paired median QPS by 10.1% at ef_search=40 (95% CI 8.0%-14.4%) and 13.8% at ef_search=160 (95% CI 11.8%-20.0%), with identical exact-recall means.
|
|
gkodinov
added
the
External Contribution
gkodinov
approved these changes
Aug 19, 2026
gkodinov
left a comment
gkodinov
left a comment
Member
There was a problem hiding this comment.
Thank you for your contribution! This is a preliminary review.
Please consider increasing the compiler coverage as suggested below. And stand by for the final review.
| if (res == 0xff) | ||
| continue; | ||
|
|
||
| #if defined(__GNUC__) |
Member
There was a problem hiding this comment.
please enable this for clang and msvc. This is what grok says should be done:
#if defined(__GNUC__) || defined(__clang__)
__builtin_prefetch(link, 0, 3); // rw: 0=read, 1=write; locality: 0–3
#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
#include <xmmintrin.h> // or <intrin.h>
_mm_prefetch((const char*)link, _MM_HINT_T0); // _MM_HINT_T0, _MM_HINT_T1, _MM_HINT_T2, _MM_HINT_NTA
#else
// no-op or do nothing
(void)addr;
#endif
chanztuying
force-pushed
the
mdev-38721-prefetch
branch
from
August 19, 2026 11:05
a8075c1 to
edf9a46
Compare
Author
|
Addressed in the latest commit. Now supports GCC/Clang, MSVC x86/x64, and MSVC ARM64. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prefetch unseen MHNSW neighbour allocations before their distance evaluations.
Correctness
#elsepath also compiles.--do-test=vectorandmain.mysqld--helppass in both.Performance
Counterbalanced AB/BA, warm cache, both binaries querying the same persisted graph (200k x 1024, cosine, M=6); 200 queries, 30 paired observations per cell.
AMD EPYC 7713, RelWithDebInfo, 8-CPU cgroup quota on a shared host.
ef_searchThe gain grows with
ef_search: largerefexpands more nodes, so more neighbour groups are scanned and more memory latency is available to hide.