Intermittent SocketTimeoutExceptions seen on OpenSearch tests when using JVector with AVX512

[akash-shankaran]

This issue captures more details and the failure runs:
opensearch-project/opensearch-jvector#623

A mitigation PR on opensearch end to disable AVX512 use in our opensearch tests. This also has some details of the issue: opensearch-project/opensearch-jvector#646

This is the log where the failure happens (also in issue 623 above):

io.github.jbellis.jvector.vector.PanamaVectorUtilSupport.dotProduct64(PanamaVectorUtilSupport.java:188)
           io.github.jbellis.jvector.vector.PanamaVectorUtilSupport.dotProduct(PanamaVectorUtilSupport.java:162)
           io.github.jbellis.jvector.vector.VectorUtil.dotProduct(VectorUtil.java:60)
           io.github.jbellis.jvector.quantization.PQDecoder$CosineDecoder.<init>(PQDecoder.java:121)
           io.github.jbellis.jvector.quantization.PQVectors.precomputedScoreFunctionFor(PQVectors.java:217)
           io.github.jbellis.jvector.graph.similarity.BuildScoreProvider$2.searchProviderFor(BuildScoreProvider.java:204)
           io.github.jbellis.jvector.graph.GraphIndexBuilder.addGraphNode(GraphIndexBuilder.java:589)

To summarize: Opensearch runs several combinations of tests in our CI pipelines: Linux, Mac, Windows combined with JDK21 and JDK25, for a total of 6 combinations. These are run on github hosted runner machines, which can be any of the following: x86 - avx2, avx512, avx512* (intel sapphire rapids or newer) or arm machines. Machine selection is beyond our control.

The above failure is observed only on avx512, avx512* based machines running JDK25 on linux. The failure is a SocketTimeoutException on our end due to the stack above. More details captured in the issue.

None of the other combinations has failed on our tests. Please share if you have any ideas what could be causing this from the library.

[reta]

To clarify a few things:

• the timeout comes from the refresh (== merge) time budget, defaults to 60 seconds
• the issue seems to be not specific to JDK-25, we have seen it on JDK-21 as well
• we did not profile or captured any specific measurements / traces / flamegraphs since the machine comes from Github, we don't have one in our possession