Skip to content

Docs & Packaging - streamline rocAL dependency documentation and fix CPack metadata - #509

Draft
levxn wants to merge 3 commits into
ROCm:developfrom
levxn:fix/verify-updated-dependency
Draft

Docs & Packaging - streamline rocAL dependency documentation and fix CPack metadata#509
levxn wants to merge 3 commits into
ROCm:developfrom
levxn:fix/verify-updated-dependency

Conversation

@levxn

@levxn levxn commented Aug 11, 2026

Copy link
Copy Markdown
Member

Motivation

Progress to issue #348

The issue's readelf -d/ldd dumps didn't match either the README's documented prerequisites or the current build's real NEEDED list. Auditing the CMake find_package/BUILD_ROCAL gating logic showed:

  • FFMPEG, rocDecode, and rocJPEG were documented as required, but CMake only disables the corresponding feature if they're missing - the build succeeds without them.
  • libsndfile, libtar, DLPack, and hipFile are all real optional dependencies that were never documented at all.
  • Building rocAL_pybind links the system Python3 runtime into the core librocal.so target itself (not just the pybind module), which is why libpython3.12.so shows up in the library's NEEDED list even for users who only care about the C++ API.
  • The packaged .deb/.rpm Depends:/Requires: lists were hand-maintained strings in CMakeLists.txt that had drifted from actual linkage: Protobuf's runtime .so (a hard-required dependency) was only declared in the -dev package lists, and LMDB/FFmpeg/libsndfile/hipFile were never declared even when found and linked at build time.
  • FFMPEG_FOUND and hipFile_FOUND were never cached as CACHE INTERNAL, so the root CMakeLists.txt couldn't see them after add_subdirectory(rocAL) to drive any conditional packaging logic.
  • OpenCV appears in the issue's dump (~50 libopencv_* entries) but is not find_package'd or used anywhere in the current rocAL/rocAL/{include,source} tree (only a stale comment in turbo_jpeg_decoder.cpp) — confirmed via readelf -d that the current build has zero OpenCV entries, so the issue's dump was captured against an older build/config. Left as a follow-up to clean up the Dockerfiles, which still build OpenCV from source unnecessarily.

Technical Details

  • README.md: split the "Libraries" prerequisites into Required and Optional (feature-enabling) sections; moved FFMPEG/rocDecode/rocJPEG into Optional; added libsndfile, Libtar (with from-source build steps, since no distro package exists), DLPack, and hipFile, none of which were previously documented; added a note about the pybind-into-core Python3 linkage; mirrored the same required/optional split in the "Install prerequisites" and "Package install" sections.
  • cmake/FindFFmpeg.cmake: added set(FFMPEG_FOUND ${FFMPEG_FOUND} CACHE INTERNAL "") at the end of the module, matching the pattern already used by FindLMDB.cmake/FindSndFile.cmake/FindLibTar.cmake, so the root CMakeLists.txt can see whether FFmpeg was actually linked.
  • rocAL/CMakeLists.txt: added the equivalent set(hipFile_FOUND ${hipFile_FOUND} CACHE INTERNAL "") right after the find_package(hipFile CONFIG QUIET) / legacy hipfile fallback block.
  • CMakeLists.txt (root):
    • Enabled automatic shared-library dependency detection: CPACK_DEBIAN_PACKAGE_SHLIBDEPS (+ per-component CPACK_DEBIAN_RUNTIME_PACKAGE_SHLIBDEPS/_DEV_/_ASAN_) and flipped CPACK_RPM_PACKAGE_AUTOREQPROV from "no" to "yes", so dpkg-shlibdeps/ rpmbuild's auto-Requires: derive the real, versioned dependency list from librocal.so's actual NEEDED entries going forward.
    • Added Protobuf's runtime package to ROCAL_DEBIAN_PACKAGE_LIST/ROCAL_RPM_PACKAGE_LIST, resolved dynamically via dpkg -S against the actual linked .so (falling back to a versioned alternation across Ubuntu 22.04/24.04 SONAMEs if that lookup fails, e.g. a from-source Protobuf build).
    • Conditionally appended LMDB/FFmpeg/libsndfile/hipFile package names guarded by their respective _FOUND variables, as a fallback safety net in case shlibdeps/autoreq isn't reliable in a given CI/packaging environment.
    • Added a configure-time diagnostic message for libtar, since no Debian/RPM distro package exists to declare it as a dependency on.

Affected files: README.md, CMakeLists.txt, cmake/FindFFmpeg.cmake,
rocAL/CMakeLists.txt.

Test Plan

Reconfigured the existing HIP-backend build and built the actual .deb
packages to confirm the packaging metadata reflects real linkage:

cd build
cmake ..
cpack -G DEB
dpkg-deb -f rocal_2.5.0-local_amd64.deb Depends
readelf -d lib/librocal.so.2.5.0 | grep NEEDED

Test Result

Before fix: FFMPEG_FOUND/hipFile_FOUND were invisible to the root CMakeLists.txt, so ffmpeg/hipfile never appeared in the DEB Depends: list despite being linked into librocal.so; Protobuf's runtime .so had no runtime package declared at all.

After fix: cmake .. configure log now shows ffmpeg, libsndfile1, hipfile present in the printed AMD ROCm rocAL DEB RunTime Package line, and the generated rocal_2.5.0-local_amd64.deb's Depends: field includes both the manual fallback entries and automatically-detected, correctly-versioned entries derived by dpkg-shlibdeps, including libavcodec60 (>= 7:6.0), liblmdb0 (>= 0.9.7), libprotobuf32t64 (>= 3.21.12), libsndfile1 (>= 1.0.20), and libpython3.12t64 (>= 3.12.1) (the pybind-into-core-library finding) — all of which cross-check against the real NEEDED entries from readelf -d. RPM Requires: auto-detection was not locally verifiable (rpmbuild is not installed in this environment) and should be confirmed in a RHEL/SLES-based CI job before merge.

@levxn
levxn requested review from a team, kiritigowda and rrawther as code owners August 11, 2026 13:36
@sruthi0107

sruthi0107 commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Update changelog.md with changes made

@sruthi0107 sruthi0107 self-assigned this Aug 11, 2026
@kiritigowda kiritigowda added enhancement New feature or request bug fix labels Aug 12, 2026
@levxn
levxn requested a review from sruthi0107 August 12, 2026 14:29
Comment thread CMakeLists.txt
# (e.g. libprotobuf23 on Ubuntu 22.04, libprotobuf32t64 on Ubuntu 24.04), so resolve it from the
# actual library dpkg found on the build machine instead of hardcoding one version.
set(PROTOBUF_DEBIAN_PACKAGE_NAME "")
if(EXISTS ${DPKG_EXE} AND PROTOBUF_LIBRARY)

@sruthi0107 sruthi0107 Aug 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PROTOBUF_LIBRARY is not defined at this scope so the dpkg auto-lookup path most likely does not execute and falls back to the hard-coded alternate. Also, FindProtobuf sets Protobuf_LIBRARY_RELEASE. Recommend either removing this block as dead code or adding find_package(Protobuf QUIET) before this block and modifying the variable name to Protobuf_LIBRARY_RELEASE.

Comment thread CMakeLists.txt
set(ROCAL_DEBIAN_DEV_PACKAGE_LIST "half, hip-dev, openmp-extras-dev, mivisionx-dev, libprotobuf-dev, libturbojpeg0-dev")
set(ROCAL_RPM_DEV_PACKAGE_LIST "half, hip-devel, openmp-extras-devel, mivisionx-devel, protobuf-devel")

# Protobuf is a hard-required core dependency, but its runtime .so was previously only declared

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@levxn not sure what this section is trying to achieve. @kiritigowda can you take a look. Seems like it's touching packaging

@levxn
levxn marked this pull request as draft August 18, 2026 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants