Add riscv64 cross-compile support and update workflows#285
Add riscv64 cross-compile support and update workflows#285rodneyosodo wants to merge 25 commits into
Conversation
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
- Fix riscv64 tarball filename (riscv64gc-linux not riscv64gc-unknown-linux-gnu) - Skip riscv64 entry when --min flag is set (no minimal variant published) - Add tarfile path-traversal guard (filter=data on Python 3.12+) - Use mktemp for riscv64 backup in test.sh; restore on EXIT via trap - Fix stale version example in test.sh comment - Extract duplicate copy_c_api_headers into a function in local.sh - Add exit 1 after missing libwasmtime.a warning in local.sh - Pin wasmtime clone to v44.0.0 tag in CI; remove redundant rustup target add and dead-work cargo build (local.sh handles both) - Downgrade example/go.mod from go 1.26.2 to go 1.18 to match CI toolchain - Fix example module path to bytecodealliance namespace - Add Linux riscv64 to supported platforms in README
- Add linux-riscv64 to native local.sh build loop so go mod vendor works without cross-compiling (includebuild.go imports it unconditionally) - Add cd example/ to README riscv64 cross-compile snippet
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
There was a problem hiding this comment.
I'd prefer to keep CI configuration centralized if possible. Would it be possible to integrate this into the preexisting CI infrastructure and workflow?
There was a problem hiding this comment.
Yes. It is possible. Let me update it?
| WASMTIME_GO="$(cd "$SCRIPT_DIR/../.." && pwd)" | ||
| DOWNLOAD_SCRIPT="$WASMTIME_GO/ci/download-wasmtime.py" | ||
|
|
||
| # Step 1: Ensure the local wasmtime-go build directory has the v45 full libraries. | ||
| # The local build may contain libraries built from a different wasmtime version | ||
| # (e.g., a newer version built from local Rust source), which causes serialization format mismatches. | ||
| # We temporarily replace them with the official v45 release binaries. | ||
| # Backup linux-riscv64 FIRST since download-wasmtime.py clears all files in build/. | ||
| _riscv64_backup="" | ||
| if [ -f "$WASMTIME_GO/build/linux-riscv64/libwasmtime.a" ]; then | ||
| _riscv64_backup=$(mktemp "${TMPDIR:-/tmp}/linux-riscv64-libwasmtime-XXXXXX.a") | ||
| cp "$WASMTIME_GO/build/linux-riscv64/libwasmtime.a" "$_riscv64_backup" | ||
| fi | ||
| trap 'rm -rf vendor module.cwasm; [ -n "$_riscv64_backup" ] && [ -f "$_riscv64_backup" ] && mv "$_riscv64_backup" "$WASMTIME_GO/build/linux-riscv64/libwasmtime.a" || true' EXIT | ||
| ( | ||
| cd "$WASMTIME_GO" | ||
| python3 "$DOWNLOAD_SCRIPT" | ||
| ) | ||
| if [ -n "$_riscv64_backup" ] && [ -f "$_riscv64_backup" ]; then | ||
| mv "$_riscv64_backup" "$WASMTIME_GO/build/linux-riscv64/libwasmtime.a" | ||
| fi |
There was a problem hiding this comment.
What're all these changes for, especially the risc-v related ones?
There was a problem hiding this comment.
The riscv64 backup/restore logic is needed because download-wasmtime.py wipes the entire build/ directory before downloading release artefacts. If a developer has a locally-built riscv64 library (which isn’t available as a prebuilt binary for all use cases), it would be lost. The backup preserves it across the download step, and the trap restores it on exit.
There was a problem hiding this comment.
I don't understand why any of this is specific to riscv64, though? This all looks like generic logic. I'd prefer if this stayed pretty simple because it's already a bash script I don't really understand and this is extending that even further.
There was a problem hiding this comment.
Removed the riscv64-specific backup logic to keep the script simple. Developers can rebuild riscv64 artifacts locally after running this test if needed.
| # The minimal runtime is not published for riscv64; skip it so we | ||
| # do not overwrite a locally-built full library with a missing min | ||
| # artifact. | ||
| if args.min and dirname == 'linux-riscv64': | ||
| shutil.rmtree(src) | ||
| continue |
There was a problem hiding this comment.
I don't believe this is correct as https://github.com/bytecodealliance/wasmtime/releases/download/v45.0.0/wasmtime-v45.0.0-riscv64gc-linux-c-api.tar.xz has the min folder in it. Is this logic necessary?
There was a problem hiding this comment.
This file unfortuantely hasn't been maintained in awhile. Nowadays can this be updated to use CMake to build the C API? That'll handle headers and such. Also, can all the risc-v specific logic be removed? I don't think any of it should be necessary here (e.g. Zig shouldn't be needed)
There was a problem hiding this comment.
Updated to use CMake for building the C API as requested
There was a problem hiding this comment.
How come this directory was added when there are already tests/examples such as example_test.go?
There was a problem hiding this comment.
*_test.go files are for automated test suites run via go test.
The example/ directory is a standalone Go module intended to provide a binary we use to verify cross-compilation for architectures like riscv64, which go test doesn’t cover.
There was a problem hiding this comment.
I'd prefer to not take on new layouts/etc within this project. I barely understand things as-is so I think this is best left to something external or something like that.
| // #cgo darwin,arm64 LDFLAGS:-L${SRCDIR}/build/macos-aarch64 | ||
| // #cgo linux,amd64 LDFLAGS:-L${SRCDIR}/build/linux-x86_64 -lwasmtime -lm | ||
| // #cgo linux,arm64 LDFLAGS:-L${SRCDIR}/build/linux-aarch64 -lwasmtime -lm | ||
| // #cgo linux,riscv64 LDFLAGS:-L${SRCDIR}/build/linux-riscv64 ${SRCDIR}/build/linux-riscv64/libwasmtime.a -lm -lgcc -lgcc_s |
There was a problem hiding this comment.
Why is riscv different from other architectures?
There was a problem hiding this comment.
RISC-V often requires explicit linking of libgcc and libgcc_s for low-level runtime support (like atomics and exception handling) that more mature architectures handle implicitly. We also link the static library directly to ensure consistent behaviour across different RISC-V toolchains and avoid potential dynamic linker issues on embedded targets.
There was a problem hiding this comment.
For the static library I"m not sure what you're talking about and I don't understand why special logic is needed here. There's only a static library in the target folder so there's no possibility of using something else by accident. Does -lwasmtime not work? What toolchain does it not work on?
| ### Cross-compiling for Linux riscv64 | ||
|
|
||
| Pass `riscv64` as the second argument to `ci/local.sh`. This requires | ||
| [zig](https://ziglang.org/) as the C cross-compiler: | ||
|
|
||
| ```sh | ||
| $ ./ci/local.sh /path/to/wasmtime riscv64 | ||
| ``` | ||
|
|
||
| This will automatically install the Rust target, cross-compile the Wasmtime C | ||
| API artifact, and set up the `build/linux-riscv64` directory. Then cross-compile | ||
| your Go program: | ||
|
|
||
| ```sh | ||
| $ cd example | ||
| $ GOOS=linux GOARCH=riscv64 CGO_ENABLED=1 \ | ||
| CC="zig cc -target riscv64-linux-gnu" \ | ||
| go build -ldflags "-s -w" ./main.go | ||
| ``` |
There was a problem hiding this comment.
Why is Zig necessary here? Why can't the native toolchain be used like for all other platforms?
There was a problem hiding this comment.
Removed but Zig is only used here as a lightweight cross-compiler for the CI environment (x86_64 Linux), which doesn’t have a native riscv64 toolchain installed.
Because wasmtime-go uses CGO, Go cannot cross-compile alone; it needs a C compiler that targets riscv64. Zig is convenient because it supports cross-compilation out of the box without installing heavy system packages.
On a native riscv64 machine, you can use the standard system gcc - Zig is not required.
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
| if hasattr(tarfile, 'data_filter'): | ||
| t.extractall(filter='data') | ||
| else: | ||
| t.extractall() |
There was a problem hiding this comment.
The Python tarfile.extractall() method changed in Python 3.12+ to require a filter parameter to prevent path traversal https://docs.python.org/3.12/library/tarfile.html#tarfile.TarFile.extractall. Without the filter argument, Python 3.12+ raises a DeprecationWarning and will fail in future versions. The conditional check ensures backward compatibility with older Python versions.
There was a problem hiding this comment.
Can the test suite be built for another architecture without running it? Or can Go run tests with QEMU emulation? I'd prefer to avoid adding such weighty new CI configuration here if possible.
There was a problem hiding this comment.
Yes, Go can cross-compile the test suite without running it by using GOOS and GOARCH without executing the tests. However, go test with CGO dependencies cannot run under QEMU emulation - you’d need a native riscv64 environment or use user-mode QEMU with proper cross-compilation setup, which would be more complex than the current approach.
The current CI builds a binary to verify cross-compilation works, but we could simplify this by just running go build on the test files instead of the full test suite. Would that address your concern about avoiding weighty CI configuration?
| WASMTIME_GO="$(cd "$SCRIPT_DIR/../.." && pwd)" | ||
| DOWNLOAD_SCRIPT="$WASMTIME_GO/ci/download-wasmtime.py" | ||
|
|
||
| # Step 1: Ensure the local wasmtime-go build directory has the v45 full libraries. | ||
| # The local build may contain libraries built from a different wasmtime version | ||
| # (e.g., a newer version built from local Rust source), which causes serialization format mismatches. | ||
| # We temporarily replace them with the official v45 release binaries. | ||
| # Backup linux-riscv64 FIRST since download-wasmtime.py clears all files in build/. | ||
| _riscv64_backup="" | ||
| if [ -f "$WASMTIME_GO/build/linux-riscv64/libwasmtime.a" ]; then | ||
| _riscv64_backup=$(mktemp "${TMPDIR:-/tmp}/linux-riscv64-libwasmtime-XXXXXX.a") | ||
| cp "$WASMTIME_GO/build/linux-riscv64/libwasmtime.a" "$_riscv64_backup" | ||
| fi | ||
| trap 'rm -rf vendor module.cwasm; [ -n "$_riscv64_backup" ] && [ -f "$_riscv64_backup" ] && mv "$_riscv64_backup" "$WASMTIME_GO/build/linux-riscv64/libwasmtime.a" || true' EXIT | ||
| ( | ||
| cd "$WASMTIME_GO" | ||
| python3 "$DOWNLOAD_SCRIPT" | ||
| ) | ||
| if [ -n "$_riscv64_backup" ] && [ -f "$_riscv64_backup" ]; then | ||
| mv "$_riscv64_backup" "$WASMTIME_GO/build/linux-riscv64/libwasmtime.a" | ||
| fi |
There was a problem hiding this comment.
I don't understand why any of this is specific to riscv64, though? This all looks like generic logic. I'd prefer if this stayed pretty simple because it's already a bash script I don't really understand and this is extending that even further.
| # Determine host platform | ||
| host_os=$(uname -s) | ||
| host_arch=$(uname -m) | ||
| case "$host_os-$host_arch" in | ||
| Linux-x86_64) platform=linux-x86_64; rust_target=x86_64-unknown-linux-gnu ;; | ||
| Linux-aarch64) platform=linux-aarch64; rust_target=aarch64-unknown-linux-gnu ;; | ||
| Linux-riscv64) platform=linux-riscv64; rust_target=riscv64gc-unknown-linux-gnu ;; | ||
| Darwin-x86_64) platform=macos-x86_64; rust_target=x86_64-apple-darwin ;; | ||
| Darwin-arm64) platform=macos-aarch64; rust_target=aarch64-apple-darwin ;; | ||
| *) echo "Unsupported platform: $host_os-$host_arch"; exit 1 ;; | ||
| esac | ||
|
|
||
| if [ ! -f "$build/libwasmtime.a" ]; then | ||
| echo 'Missing libwasmtime.a. Did you `cargo build -p wasmtime-c-api`?' | ||
| # The CMake/cargo build puts artifacts in the wasmtime target directory. | ||
| # Native builds go to target/release/, cross builds to target/$rust_target/release/. | ||
| wasmtime_target_dir="$wasmtime/target/$rust_target/release" | ||
| if [ ! -d "$wasmtime_target_dir" ]; then | ||
| wasmtime_target_dir="$wasmtime/target/release" | ||
| fi |
There was a problem hiding this comment.
By using the install cmake target there shouldn't be any need to rummage around in the build directory to find files. Can that be used instead? (same for headers)
There was a problem hiding this comment.
CMake’s install target could be used, but it would require adjusting wasmtime-go’s directory structure expectations. CMake typically installs to $prefix/lib/ and $prefix/include/, whereas the current structure expects the library directly in the platform directory (build/$platform/libwasmtime.a) and headers in a shared build/include/. Using cmake --install would give us build/$platform/lib/libwasmtime.a, requiring either moving the file or updating the ffi.go cgo flags.
There was a problem hiding this comment.
I'd prefer to not take on new layouts/etc within this project. I barely understand things as-is so I think this is best left to something external or something like that.
| // #cgo darwin,arm64 LDFLAGS:-L${SRCDIR}/build/macos-aarch64 | ||
| // #cgo linux,amd64 LDFLAGS:-L${SRCDIR}/build/linux-x86_64 -lwasmtime -lm | ||
| // #cgo linux,arm64 LDFLAGS:-L${SRCDIR}/build/linux-aarch64 -lwasmtime -lm | ||
| // #cgo linux,riscv64 LDFLAGS:-L${SRCDIR}/build/linux-riscv64 ${SRCDIR}/build/linux-riscv64/libwasmtime.a -lm -lgcc -lgcc_s |
There was a problem hiding this comment.
For the static library I"m not sure what you're talking about and I don't understand why special logic is needed here. There's only a static library in the target folder so there's no possibility of using something else by accident. Does -lwasmtime not work? What toolchain does it not work on?
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
- Removed example/ directory as requested by reviewer - Cross-compilation verification handled by build-cross job in main.yml
- Rename build-example to build-cross - Remove example/ directory references - Simplify to just build test files for verification - Only run on ubuntu-latest for amd64, arm64, riscv64
- Remove direct static library path and -lgcc/-lgcc_s flags - Use -lwasmtime like other architectures for consistency - No special handling needed since only static library exists in target directory
linux-riscv64cross-compilation support: CGO flags,ci/local.shwith Zig cross-toolchain, prebuilt artifact downloads, andexample-buildCI workflowexample/module demonstrating basic usageincludebuild.goandBUILD.bazelto includelinux-riscv64artifacts