Use Testcontainers for integration tests

[lsh1215]

Description

Adds Testcontainers-backed OpenSearch startup for Java client integration tests so developers and the released integration workflow no longer need to start OpenSearch with Docker Compose before running integrationTest.

The Testcontainers path is enabled by default for the integrationTest task. Existing externally managed clusters remain supported by setting tests.rest.cluster or disabling the test container with -Dtests.opensearch.testcontainers.enabled=false.

The Docker/Testcontainers startup path is limited to the integrationTest task. The released integration CI path remains Ubuntu-only, and the macOS/Windows build and unit-test jobs do not get a Docker runtime requirement.

Closes #1916.

Changes

• Add a JVM-singleton OpenSearchTestContainer helper for integration tests.
• Start the container before OpenSearchRestTestCase initializes its REST client.
• Use HTTPS/basic auth and handle the OpenSearch 2.12+ initial admin password requirement.
• Handle ambiguous OpenSearch 2.x/custom image version tags conservatively and cover that compatibility logic with focused tests.
• Disable disk threshold checks only for the disposable Testcontainers-managed node to avoid Docker host disk watermark state applying cluster.blocks.create_index=true during tests.
• Replace released integration workflow Docker Compose setup with Gradle/Testcontainers startup.
• Keep unreleased/source-build integration workflow on its existing external cluster path by disabling Testcontainers there.
• Document the new default, version/image selection, and external-cluster opt-out.

Verification

• git diff --check
• ./gradlew integrationTest --tests "org.opensearch.client.opensearch.integTest.OpenSearchTestContainerTests"
• ./gradlew spotlessJavaCheck
• ./gradlew integrationTest --tests "org.opensearch.client.opensearch.integTest.restclient.PingAndInfoIT" -Dtests.opensearch.version=3.2.0
• GitHub Actions for PR Use Testcontainers for integration tests #2033: all checks passed, including the released integration matrix.

[reta]

Please just use:

Suggested change

	// opensearch-testcontainers exposes OpenSearchContainer as a GenericContainer subclass, but publishes
	testImplementation("org.opensearch:opensearch-testcontainers:4.1.0")
	testImplementation"("org.testcontainers:testcontainers:2.0.4")
	testImplementation"("org.testcontainers:testcontainers-junit-jupiter:2.0.4")

[reta]

I don't think we need these tests - opensearch-testcontainers has bunch of them

[reta]

Please move under java-client/src/test/java21. Also, we should be using JUnit Jupiter extentions mechanism with proper pre / post lifycycle (see please examples here https://www.baeldung.com/junit-5-registerextension-annotation)

[lsh1215]

Moved the helper under src/test/java21. On the Jupiter extension part: these tests extend OpenSearchRestTestCase, so they run under RandomizedRunner (JUnit 4) and the Test task doesn't call useJUnitPlatform(). Jupiter's @RegisterExtension / BeforeAllCallback only run under the Jupiter engine, so they won't fire here.

The JUnit 4 equivalent is a @ClassRule ExternalResource, which the opensearch-testcontainers README documents: https://github.com/opensearch-project/opensearch-testcontainers#junit-4-integration

Would that work, or do you want to migrate these suites to the JUnit Platform (a larger change to the RandomizedRunner base classes)?

[reta]

Sorry, totally my bad, indeed we still use JUnit 4, @ClassRule would be the right equivalent (not extension), thank you

[reta]

Please remove, we should not see any leaking threads after https://github.com/opensearch-project/opensearch-java/pull/2033/changes#r3514648791 (the extension part)

[lsh1215]

Removing the filter surfaces thread leaks that RandomizedTesting flags at suite scope. These threads are JVM-global statics in the Testcontainers stack (duct-tape's Timeouts executor, the ryuk reaper, and the pull-watchdog/wait pools), so container.stop() doesn't end them.
open issue: testcontainers/testcontainers-java#9227

OpenSearch core keeps a @ThreadLeakFilters for the same threads: https://github.com/opensearch-project/OpenSearch/blob/main/plugins/ingestion-kafka/src/internalClusterTest/java/org/opensearch/plugin/kafka/TestContainerThreadLeakFilter.java

Can we keep a small filter like that instead? The alternatives (disabling ryuk, or reflecting into duct-tape's executor) are heavier and still miss pull-watchdog/wait.

[reta]

Can we keep a small filter like that instead? The alternatives (disabling ryuk, or reflecting into duct-tape's executor) are heavier and still miss pull-watchdog/wait.

I see, yeah, let's keep it for now, thank you for clarification