Update Spark 4.2.0 website metadata#701
Closed
huaxingao wants to merge 2 commits into
Closed
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
huaxingao
requested review from
HyukjinKwon,
cloud-fan,
dongjoon-hyun and
viirya
viirya
approved these changes
Jul 15, 2026
Member
|
You probably need to update other files too. |
Co-authored-by: Cursor <cursoragent@cursor.com>
Member
|
Hmm, looks like previous release docs update (e.g., Apache Spark 4.1.2, e5fb0b2) doesn't update these files. Maybe this update is not necessary? |
HyukjinKwon
approved these changes
Jul 15, 2026
Contributor
Author
|
Thanks @HyukjinKwon @viirya for the review! |
Contributor
|
@HyukjinKwon does it mean the Github Action based release process missed to update some files? |
Member
|
yeah I think so. we would need to udpate it |
Member
|
let me take a look |
HyukjinKwon
added a commit
to apache/spark
that referenced
this pull request
Jul 15, 2026
…in the release script ### What changes were proposed in this pull request? This PR adds a step to the `finalize` phase of `dev/create-release/release-build.sh` that updates two spark-website source files that were previously edited by hand: - `downloads.md`: bump the Maven coordinate example (`version:`) to the latest stable release. - `site/static/versions.json`: add an entry for the new release, keeping the list sorted in descending version order. `site/downloads.html` is intentionally not edited here because it is regenerated from `downloads.md` by the existing `bundle exec jekyll build` step. Both updates are guarded by the same `-preview` check used by the neighboring steps, so preview releases are not affected. ### Why are the changes needed? The release automation already updates most of the spark-website (`documentation.md`, `js/downloads.js`, the news post, the JIRA release notes, the docs, and the `latest` symlink), but it did not touch `downloads.md` or `site/static/versions.json`. As a result the release manager had to create these edits manually, e.g. apache/spark-website#701. ### Does this PR introduce _any_ user-facing change? No. This only affects the release tooling. ### How was this patch tested? Extracted the new block from the script and ran it locally against realistic spark-website fixtures across many scenarios: - Feature release (newest) updates both files and reproduces the manual [spark-website#701](apache/spark-website#701) diff. - Preview releases (`-preview1`, `-preview2`) skip both files, matching the sibling steps. - Old-branch maintenance releases insert into the correct sorted slot in `versions.json` and never downgrade the `downloads.md` coordinate. - `downloads.md` is only bumped when the release is strictly newer than what is shown. - `versions.json` keeps every patch entry and is idempotent / dedup-safe on re-runs. - Output remains valid JSON with the existing `indent=4` formatting and trailing newline. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) Closes #57269 from HyukjinKwon/SPARK-58136-automate-website-updates. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com>
HyukjinKwon
added a commit
to apache/spark
that referenced
this pull request
Jul 15, 2026
…in the release script ### What changes were proposed in this pull request? This PR adds a step to the `finalize` phase of `dev/create-release/release-build.sh` that updates two spark-website source files that were previously edited by hand: - `downloads.md`: bump the Maven coordinate example (`version:`) to the latest stable release. - `site/static/versions.json`: add an entry for the new release, keeping the list sorted in descending version order. `site/downloads.html` is intentionally not edited here because it is regenerated from `downloads.md` by the existing `bundle exec jekyll build` step. Both updates are guarded by the same `-preview` check used by the neighboring steps, so preview releases are not affected. ### Why are the changes needed? The release automation already updates most of the spark-website (`documentation.md`, `js/downloads.js`, the news post, the JIRA release notes, the docs, and the `latest` symlink), but it did not touch `downloads.md` or `site/static/versions.json`. As a result the release manager had to create these edits manually, e.g. apache/spark-website#701. ### Does this PR introduce _any_ user-facing change? No. This only affects the release tooling. ### How was this patch tested? Extracted the new block from the script and ran it locally against realistic spark-website fixtures across many scenarios: - Feature release (newest) updates both files and reproduces the manual [spark-website#701](apache/spark-website#701) diff. - Preview releases (`-preview1`, `-preview2`) skip both files, matching the sibling steps. - Old-branch maintenance releases insert into the correct sorted slot in `versions.json` and never downgrade the `downloads.md` coordinate. - `downloads.md` is only bumped when the release is strictly newer than what is shown. - `versions.json` keeps every patch entry and is idempotent / dedup-safe on re-runs. - Output remains valid JSON with the existing `indent=4` formatting and trailing newline. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) Closes #57269 from HyukjinKwon/SPARK-58136-automate-website-updates. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com> (cherry picked from commit 74ea5d0) Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com>
HyukjinKwon
added a commit
to apache/spark
that referenced
this pull request
Jul 15, 2026
…in the release script ### What changes were proposed in this pull request? This PR adds a step to the `finalize` phase of `dev/create-release/release-build.sh` that updates two spark-website source files that were previously edited by hand: - `downloads.md`: bump the Maven coordinate example (`version:`) to the latest stable release. - `site/static/versions.json`: add an entry for the new release, keeping the list sorted in descending version order. `site/downloads.html` is intentionally not edited here because it is regenerated from `downloads.md` by the existing `bundle exec jekyll build` step. Both updates are guarded by the same `-preview` check used by the neighboring steps, so preview releases are not affected. ### Why are the changes needed? The release automation already updates most of the spark-website (`documentation.md`, `js/downloads.js`, the news post, the JIRA release notes, the docs, and the `latest` symlink), but it did not touch `downloads.md` or `site/static/versions.json`. As a result the release manager had to create these edits manually, e.g. apache/spark-website#701. ### Does this PR introduce _any_ user-facing change? No. This only affects the release tooling. ### How was this patch tested? Extracted the new block from the script and ran it locally against realistic spark-website fixtures across many scenarios: - Feature release (newest) updates both files and reproduces the manual [spark-website#701](apache/spark-website#701) diff. - Preview releases (`-preview1`, `-preview2`) skip both files, matching the sibling steps. - Old-branch maintenance releases insert into the correct sorted slot in `versions.json` and never downgrade the `downloads.md` coordinate. - `downloads.md` is only bumped when the release is strictly newer than what is shown. - `versions.json` keeps every patch entry and is idempotent / dedup-safe on re-runs. - Output remains valid JSON with the existing `indent=4` formatting and trailing newline. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) Closes #57269 from HyukjinKwon/SPARK-58136-automate-website-updates. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com> (cherry picked from commit 74ea5d0) Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com>
HyukjinKwon
added a commit
to apache/spark
that referenced
this pull request
Jul 15, 2026
…in the release script ### What changes were proposed in this pull request? This PR adds a step to the `finalize` phase of `dev/create-release/release-build.sh` that updates two spark-website source files that were previously edited by hand: - `downloads.md`: bump the Maven coordinate example (`version:`) to the latest stable release. - `site/static/versions.json`: add an entry for the new release, keeping the list sorted in descending version order. `site/downloads.html` is intentionally not edited here because it is regenerated from `downloads.md` by the existing `bundle exec jekyll build` step. Both updates are guarded by the same `-preview` check used by the neighboring steps, so preview releases are not affected. ### Why are the changes needed? The release automation already updates most of the spark-website (`documentation.md`, `js/downloads.js`, the news post, the JIRA release notes, the docs, and the `latest` symlink), but it did not touch `downloads.md` or `site/static/versions.json`. As a result the release manager had to create these edits manually, e.g. apache/spark-website#701. ### Does this PR introduce _any_ user-facing change? No. This only affects the release tooling. ### How was this patch tested? Extracted the new block from the script and ran it locally against realistic spark-website fixtures across many scenarios: - Feature release (newest) updates both files and reproduces the manual [spark-website#701](apache/spark-website#701) diff. - Preview releases (`-preview1`, `-preview2`) skip both files, matching the sibling steps. - Old-branch maintenance releases insert into the correct sorted slot in `versions.json` and never downgrade the `downloads.md` coordinate. - `downloads.md` is only bumped when the release is strictly newer than what is shown. - `versions.json` keeps every patch entry and is idempotent / dedup-safe on re-runs. - Output remains valid JSON with the existing `indent=4` formatting and trailing newline. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) Closes #57269 from HyukjinKwon/SPARK-58136-automate-website-updates. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com> (cherry picked from commit 74ea5d0) Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com>
HyukjinKwon
added a commit
to apache/spark
that referenced
this pull request
Jul 15, 2026
…in the release script ### What changes were proposed in this pull request? This PR adds a step to the `finalize` phase of `dev/create-release/release-build.sh` that updates two spark-website source files that were previously edited by hand: - `downloads.md`: bump the Maven coordinate example (`version:`) to the latest stable release. - `site/static/versions.json`: add an entry for the new release, keeping the list sorted in descending version order. `site/downloads.html` is intentionally not edited here because it is regenerated from `downloads.md` by the existing `bundle exec jekyll build` step. Both updates are guarded by the same `-preview` check used by the neighboring steps, so preview releases are not affected. ### Why are the changes needed? The release automation already updates most of the spark-website (`documentation.md`, `js/downloads.js`, the news post, the JIRA release notes, the docs, and the `latest` symlink), but it did not touch `downloads.md` or `site/static/versions.json`. As a result the release manager had to create these edits manually, e.g. apache/spark-website#701. ### Does this PR introduce _any_ user-facing change? No. This only affects the release tooling. ### How was this patch tested? Extracted the new block from the script and ran it locally against realistic spark-website fixtures across many scenarios: - Feature release (newest) updates both files and reproduces the manual [spark-website#701](apache/spark-website#701) diff. - Preview releases (`-preview1`, `-preview2`) skip both files, matching the sibling steps. - Old-branch maintenance releases insert into the correct sorted slot in `versions.json` and never downgrade the `downloads.md` coordinate. - `downloads.md` is only bumped when the release is strictly newer than what is shown. - `versions.json` keeps every patch entry and is idempotent / dedup-safe on re-runs. - Output remains valid JSON with the existing `indent=4` formatting and trailing newline. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) Closes #57269 from HyukjinKwon/SPARK-58136-automate-website-updates. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com> (cherry picked from commit 74ea5d0) Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com>
HyukjinKwon
added a commit
to apache/spark
that referenced
this pull request
Jul 15, 2026
…in the release script ### What changes were proposed in this pull request? This PR adds a step to the `finalize` phase of `dev/create-release/release-build.sh` that updates two spark-website source files that were previously edited by hand: - `downloads.md`: bump the Maven coordinate example (`version:`) to the latest stable release. - `site/static/versions.json`: add an entry for the new release, keeping the list sorted in descending version order. `site/downloads.html` is intentionally not edited here because it is regenerated from `downloads.md` by the existing `bundle exec jekyll build` step. Both updates are guarded by the same `-preview` check used by the neighboring steps, so preview releases are not affected. ### Why are the changes needed? The release automation already updates most of the spark-website (`documentation.md`, `js/downloads.js`, the news post, the JIRA release notes, the docs, and the `latest` symlink), but it did not touch `downloads.md` or `site/static/versions.json`. As a result the release manager had to create these edits manually, e.g. apache/spark-website#701. ### Does this PR introduce _any_ user-facing change? No. This only affects the release tooling. ### How was this patch tested? Extracted the new block from the script and ran it locally against realistic spark-website fixtures across many scenarios: - Feature release (newest) updates both files and reproduces the manual [spark-website#701](apache/spark-website#701) diff. - Preview releases (`-preview1`, `-preview2`) skip both files, matching the sibling steps. - Old-branch maintenance releases insert into the correct sorted slot in `versions.json` and never downgrade the `downloads.md` coordinate. - `downloads.md` is only bumped when the release is strictly newer than what is shown. - `versions.json` keeps every patch entry and is idempotent / dedup-safe on re-runs. - Output remains valid JSON with the existing `indent=4` formatting and trailing newline. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) Closes #57269 from HyukjinKwon/SPARK-58136-automate-website-updates. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com> (cherry picked from commit 74ea5d0) Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Following the Apache Spark 4.2.0 release, this PR updates two pieces of
website metadata that the release tooling does not update automatically:
downloads.md: bump the Maven dependency coordinate example from4.1.0to4.2.0.site/static/versions.json: add a4.2.0entry (at the top) to thedocumentation version switcher.