Skip to content

[AURON #2444] Add build branch and build reversion info #2445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zhaohehuhu wants to merge 3 commits into
base: master
Choose a base branch
from
Open

[AURON #2444] Add build branch and build reversion info #2445

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ce56287
add build branch and build reversion info
zhaohehuhu Jul 30, 2026
c14edfd
refactor
zhaohehuhu Aug 3, 2026
d36db3a
populate the overrides in workflows
zhaohehuhu Aug 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/build-amd64-releases.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,8 @@ jobs:
- name: Build auron-${{ matrix.sparkver }}_${{ matrix.scalaver }}
env:
AURON_JAVA_VERSION: ${{ matrix.javaver }}
AURON_BUILD_BRANCH: ${{ github.head_ref || github.ref_name }}
AURON_BUILD_REVISION: ${{ github.event.pull_request.head.sha || github.sha }}
run: |
sed -i 's/docker-compose -f/docker compose -f/g' ./auron-build.sh
./auron-build.sh \
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/build-arm-releases.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,9 @@ jobs:
run: echo "short=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT

- name: Build auron-${{ matrix.sparkver }}_${{ matrix.scalaver }}
env:
AURON_BUILD_BRANCH: ${{ github.head_ref || github.ref_name }}
AURON_BUILD_REVISION: ${{ github.event.pull_request.head.sha || github.sha }}
run: |
rm -f .build-checksum_*.cache
build/mvn package -Prelease -P${{ matrix.sparkver }} -Pscala-${{ matrix.scalaver }} -DskipTests
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/build-macos-releases.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,9 @@ jobs:
run: echo "short=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT

- name: Build auron-${{ matrix.sparkver }}_${{ matrix.scalaver }}
env:
AURON_BUILD_BRANCH: ${{ github.head_ref || github.ref_name }}
AURON_BUILD_REVISION: ${{ github.event.pull_request.head.sha || github.sha }}
run: |
./auron-build.sh \
--release \
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/tpcds-reusable.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,9 @@ jobs:
cargo

- name: Build auron (Spark ${{ inputs.sparkver }}, Scala ${{ inputs.scalaver }}, JDK ${{ inputs.javaver }})
env:
AURON_BUILD_BRANCH: ${{ github.head_ref || github.ref_name }}
AURON_BUILD_REVISION: ${{ github.event.pull_request.head.sha || github.sha }}
run: |
rm -f .build-checksum_*.cache

Expand Down
12 changes: 12 additions & 0 deletions auron-build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ print_help() {

echo " -h, --help Show this help message"
echo
echo "Environment variables:"
echo " AURON_BUILD_BRANCH Override detected Git branch in generated build info"
echo " AURON_BUILD_REVISION Override detected Git revision in generated build info"
echo
echo "Examples:"
echo " $0 --pre --sparkver ${SUPPORTED_SPARK_VERSIONS[*]: -1}" \
"--scalaver ${SUPPORTED_SCALA_VERSIONS[*]: -1} -DskipBuildNative"
Expand Down Expand Up @@ -529,6 +533,8 @@ mkdir -p "$(dirname "$BUILD_INFO_FILE")"
JAVA_VERSION=$(java -version 2>&1 | head -n 1 | awk '{print $3}' | tr -d '"')
PROJECT_VERSION=$(./build/mvn help:evaluate -N -Dexpression=project.version -Pspark-${SPARK_VER} -q -DforceStdout 2>/dev/null)
RUST_VERSION=$(rustc --version | awk '{print $2}')
BUILD_BRANCH="${AURON_BUILD_BRANCH:-$(git rev-parse --abbrev-ref HEAD 2>/dev/null)}"

@slfan1989 slfan1989 Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the override and forwarding it into Docker. However, none of the workflows currently sets AURON_BUILD_BRANCH, so release jobs that check out ${{ github.event.pull_request.head.sha }} still fall back to git rev-parse and record build.branch=HEAD. Please populate the override in the relevant workflows, for example with ${{ github.head_ref || github.ref_name }}. The revision override could similarly use ${{ github.event.pull_request.head.sha || github.sha }} to make the recorded source explicit.

@zhaohehuhu zhaohehuhu Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing! @slfan1989

BUILD_REVISION="${AURON_BUILD_REVISION:-$(git rev-parse HEAD 2>/dev/null)}"

get_build_info() {
case "$1" in
Expand All @@ -543,6 +549,8 @@ get_build_info() {
"flink.version") echo "${FLINK_VER}" ;;
"iceberg.version") echo "${ICEBERG_VER}" ;;
"hudi.version") echo "${HUDI_VER}" ;;
"build.branch") echo "${BUILD_BRANCH}" ;;
"build.revision") echo "${BUILD_REVISION}" ;;
"build.timestamp") echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ")" ;;
*) echo "" ;;
esac
Expand All @@ -560,6 +568,8 @@ for key in \
"paimon.version" \
"flink.version" \
"iceberg.version" \
"build.branch" \
"build.revision" \
"build.timestamp"; do
value="$(get_build_info "$key")"
if [[ -n "$value" ]]; then
Expand Down Expand Up @@ -600,6 +610,8 @@ if [[ "$USE_DOCKER" == true ]]; then

echo "[INFO] Compiling inside Docker container..."
export AURON_BUILD_ARGS="${BUILD_ARGS[*]}"
export AURON_BUILD_BRANCH="${BUILD_BRANCH}"
export AURON_BUILD_REVISION="${BUILD_REVISION}"
export BUILD_CONTEXT="./${IMAGE_NAME}"
# Spark 4.x requires JDK 17+, auto-set if not specified
if [[ -z "$AURON_JAVA_VERSION" && "$SPARK_VER" == 4.* ]]; then
Expand Down
4 changes: 4 additions & 0 deletions common/src/main/scala/org/apache/auron/common/AuronBuildInfo.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ object AuronBuildInfo {
val UNIFFLE_VERSION_STRING: String = "Uniffle Version"
val PAIMON_VERSION_STRING: String = "Paimon Version"
val ICEBERG_VERSION_STRING: String = "Iceberg Version"
val BUILD_BRANCH_STRING: String = "Build Branch"
val BUILD_REVISION_STRING: String = "Build Revision"
val BUILD_DATE_STRING: String = "Build Timestamp"

val VERSION: String = props.getProperty("project.version", unknown)
Expand All @@ -62,5 +64,7 @@ object AuronBuildInfo {
val PAIMON_VERSION: String = props.getProperty("paimon.version", unknown)
val ICEBERG_VERSION: String = props.getProperty("iceberg.version", unknown)
val FLINK_VERSION: String = props.getProperty("flink.version", unknown)
val BUILD_BRANCH: String = props.getProperty("build.branch", unknown)
val BUILD_REVISION: String = props.getProperty("build.revision", unknown)
val BUILD_DATE: String = props.getProperty("build.timestamp", unknown)
}
2 changes: 2 additions & 0 deletions dev/docker-build/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ services:
environment:
RUSTFLAGS: "-C target-cpu=skylake"
AURON_BUILD_ARGS: "${AURON_BUILD_ARGS}"
AURON_BUILD_BRANCH: "${AURON_BUILD_BRANCH}"
AURON_BUILD_REVISION: "${AURON_BUILD_REVISION}"
AURON_JAVA_VERSION: "${AURON_JAVA_VERSION:-8}"
command: >
bash -c '
Expand Down
2 changes: 2 additions & 0 deletions spark-extension-shims-spark/src/main/scala/org/apache/spark/sql/auron/ShimsImpl.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,8 @@ class ShimsImpl extends Shims with Logging {
auronBuildInfo.put(AuronBuildInfo.PAIMON_VERSION_STRING, AuronBuildInfo.PAIMON_VERSION)
auronBuildInfo.put(AuronBuildInfo.ICEBERG_VERSION_STRING, AuronBuildInfo.ICEBERG_VERSION)
auronBuildInfo.put(AuronBuildInfo.FLINK_VERSION_STRING, AuronBuildInfo.FLINK_VERSION)
auronBuildInfo.put(AuronBuildInfo.BUILD_BRANCH_STRING, AuronBuildInfo.BUILD_BRANCH)
auronBuildInfo.put(AuronBuildInfo.BUILD_REVISION_STRING, AuronBuildInfo.BUILD_REVISION)
auronBuildInfo.put(AuronBuildInfo.BUILD_DATE_STRING, AuronBuildInfo.BUILD_DATE)
auronBuildInfo.retain { case (_, v) => v != null && v.nonEmpty }
val event = AuronBuildInfoEvent(auronBuildInfo)
Expand Down
Loading