[SPARK-58136][INFRA] Automate downloads.md and versions.json updates in the release script

[HyukjinKwon]

What changes were proposed in this pull request?

This PR adds a step to the finalize phase of dev/create-release/release-build.sh that updates two spark-website source files that were previously edited by hand:

• downloads.md: bump the Maven coordinate example (version:) to the latest stable release.
• site/static/versions.json: add an entry for the new release, keeping the list sorted in descending version order.

site/downloads.html is intentionally not edited here because it is regenerated from downloads.md by the existing bundle exec jekyll build step.

Both updates are guarded by the same -preview check used by the neighboring steps, so preview releases are not affected.

Why are the changes needed?

The release automation already updates most of the spark-website (documentation.md, js/downloads.js, the news post, the JIRA release notes, the docs, and the latest symlink), but it did not touch downloads.md or site/static/versions.json. As a result the release manager had to create these edits manually, e.g. apache/spark-website#701.

Does this PR introduce any user-facing change?

No. This only affects the release tooling.

How was this patch tested?

Extracted the new block from the script and ran it locally against realistic spark-website fixtures across many scenarios:

• Feature release (newest) updates both files and reproduces the manual spark-website#701 diff.
• Preview releases (-preview1, -preview2) skip both files, matching the sibling steps.
• Old-branch maintenance releases insert into the correct sorted slot in versions.json and never downgrade the downloads.md coordinate.
• downloads.md is only bumped when the release is strictly newer than what is shown.
• versions.json keeps every patch entry and is idempotent / dedup-safe on re-runs.
• Output remains valid JSON with the existing indent=4 formatting and trailing newline.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)

[cloud-fan]

0 blocking, 0 non-blocking, 0 nits.
Clean, self-consistent INFRA change that folds the manual spark-website#701 edits into the release script. Recommend approval.

No findings.

Verification

I verified the two transformations and the build ordering empirically against a fresh asf-site clone:

• downloads.md — the version: coordinate bumps only when the release is strictly newer than what's shown (feature/major-bump releases update; older-branch maintenance and same-version runs leave it unchanged / are idempotent), and the trailing newline is preserved by the (\s*)$ capture group.
• site/static/versions.json — the upsert inserts at the correct descending-sorted slot, dedups any prior same-version entry (idempotent on re-runs), and re-serializes byte-identically to the current file (no spurious whitespace/key-order diff on untouched entries).
• Ordering / clobber-safety — _config.yml sets destination: site with keep_files: ['docs','static',...], so site/static/versions.json (under static, no source counterpart) survives the step-6 jekyll build; downloads.md is a jekyll source whose downloads.html output is regenerated at step 6, which runs after this edit and before the shared git add . && git commit && git push. So omitting the site/downloads.html edit (which #701 did by hand) is correct.
• Consistency — the new step mirrors its sibling metadata steps (shared -preview guard, per-heredoc vercmp, don't-downgrade, idempotency), and the step renumbering (1..7) is contiguous with no stale references.

[HyukjinKwon]

Merge Summary:

• merged into master 74ea5d0
• merged into branch-4.x f8c5479
• merged into branch-4.2 f87f207
• merged into branch-4.1 986123b
• merged into branch-4.0 f22f287
• merged into branch-3.5 d781daf

Posted by merge_spark_pr.py