ci: extend Maven setup and retry dependency downloads

[sunchao]

Which issue does this PR close?

No issue is closed. This follows up on #5422 and addresses the dependency-download failures seen on #5481. The separate libhdfs crash is covered by #5036.

Rationale for this change

#5422 added Maven distribution caching and bootstrap retries to the Java test action. Other Maven callers still bootstrap independently, and downloading Maven itself does not protect subsequent artifact downloads.

On #5481, the Delta build gate failed downloading a Maven plugin dependency with Connection reset, while a Spark 4.1 SQL shard failed resolving a Guava dependency with HTTP 502 before its tests started.

What changes are included in this PR?

• Extract the existing Maven bootstrap/cache setup into a shared action and use it across CI. Preserve the existing cache keys and macOS cache exclusion, and configure bounded Maven Resolver retries for individual HTTP transfers without rerunning Maven goals.
• Select the actual Maven wrapper distribution directory for both cache operations. Hosted runners no longer traverse /root, which caused cache saving to fail with EACCES; root containers keep their existing distribution location. The changed cache path list causes one initial cache miss.
• Add bounded retries for dependency-only commands. Retry recognized transient download errors, preserve failure exit codes, and leave compilation, tests, permanent errors, and cancellation fatal.
• Resolve Spark test dependencies before compilation, restore SBT/Coursier caches, and save successful downloads before later build or test failures. Only the full build job can publish the shared cache, so a shard cannot populate it with partial dependencies.
• Preserve Delta gate Maven/Cargo logs, propagate effective-POM failures, and upload the logs when the gate fails.
• Add regression checks to preflight and update change detection for the shared action and scripts.

How are these changes tested?

• 24 regression tests passed, including real Maven 3.9.6 requests against a local HTTP server for transient 429/502, exhausted 503 retries, and permanent 404 failures. Additional cases cover retry limits, exit codes, interrupted backoff, dependency-only Spark tasks, and Delta compilation failures and log retention.
• Six real-Java cache-path tests cover host and container homes, wrapper-home overrides, JVM selection, option precedence, and invalid options. The computed path also matches the actual local Maven wrapper installation.
• The updated hosted preflight passed on JDK 11, ran all 24 regression tests, and successfully saved the Maven distribution cache under /home/runner/.m2/wrapper/dists without EACCES.
• Workflow and composite-action validation with actionlint, YAML parsing, and Bash syntax checks passed.
• Existing suite/benchmark consistency checks and Apache RAT passed.

The real Maven integration checks ran locally on JDK 21 and in preflight on JDK 11. Full Spark/JVM/native coverage is provided by the remaining PR CI matrix. This PR does not change the native crash fix in #5036.