Skip to content

[build] Inline xxHash submodule into src-ThirdParty/#11657

Open
jonathanpeppers wants to merge 1 commit into
dotnet:mainfrom
jonathanpeppers:jonathanpeppers/inline-xxhash-submodule
Open

[build] Inline xxHash submodule into src-ThirdParty/#11657
jonathanpeppers wants to merge 1 commit into
dotnet:mainfrom
jonathanpeppers:jonathanpeppers/inline-xxhash-submodule

Conversation

@jonathanpeppers

@jonathanpeppers jonathanpeppers commented Jun 15, 2026

Copy link
Copy Markdown
Member

Context: aligned with the ongoing external/ cleanup campaign (#11529, #11568, #11580, #11608, #11613, #11631).

We only consume the xxhash.h header from xxHash, and we use it in inline mode only (XXH_INLINE_ALL) via src/native/common/include/shared/xxhash.hh:

#define XXH_NO_STREAM
#define XXH_INLINE_ALL
#define XXH_NAMESPACE xaInternal_
#include <xxHash/xxhash.h>

No .c from upstream is ever compiled. Vendor that one header into src-ThirdParty/xxHash/ and drop the git submodule.

Changes

  • Copy external/xxHash/xxhash.h verbatim to src-ThirdParty/xxHash/xxhash.h (BSD-2-Clause). Directory case is xxHash (capital H) to match the existing <xxHash/xxhash.h> include path on case-sensitive filesystems.
  • src/native/CMakeLists.txt:
    • Remove the dead XXHASH_DIR variable (it was set but never referenced).
    • Add a new THIRDPARTY_DIR = ${REPO_ROOT_DIR}/src-ThirdParty variable.
  • Add ${THIRDPARTY_DIR} next to ${EXTERNAL_DIR} in the six target_include_directories blocks that previously resolved <xxHash/xxhash.h> from external/:
    • src/native/CMakeLists.txt (CLR macro)
    • src/native/clr/host/CMakeLists.txt
    • src/native/clr/shared/CMakeLists.txt
    • src/native/nativeaot/host/CMakeLists.txt
    • src/native/mono/monodroid/CMakeLists.txt
    • src/native/mono/shared/CMakeLists.txt
  • Remove the [submodule "external/xxHash"] block from .gitmodules.
  • Remove external/xxHash from the gitsubmodule ignore list in .github/dependabot.yml.
  • Update .github/skills/update-tpn/SKILL.md - move xxHash from the "Git Submodules" table to the "Vendored Source" table; reword the "Native Libraries" line to reflect the new header-only inline usage.

THIRD-PARTY-NOTICES.TXT already has a full BSD-2-Clause Cyan4973/xxHash block, so no TPN edits are required.

Why THIRDPARTY_DIR points at src-ThirdParty/ (not src-ThirdParty/xxHash/)

The include in our wrapper header is written as <xxHash/xxhash.h>, so the search root we add to target_include_directories must be the parent of xxHash/ — i.e. src-ThirdParty/ — for the xxHash/ prefix to resolve. This mirrors how ${EXTERNAL_DIR} already works today: it's external/ (parent of all submodule dirs), not external/xxHash/. The wrapper header src/native/common/include/shared/xxhash.hh is intentionally unchanged.

The variable is named generically (THIRDPARTY_DIR rather than XXHASH_DIR) so future inlinings of other external/ submodules that use the same <dir-name/header.h> include style (e.g. robin-map, constexpr-xxh3, libunwind) can reuse the same variable instead of either duplicating the ${REPO_ROOT_DIR}/src-ThirdParty literal across six target_include_directories blocks or adding yet another per-project *_DIR variable.

Verification

  • git grep "external/xxHash" -> no matches.
  • git grep XXHASH_DIR -> no matches.
  • git submodule status -> external/xxHash is gone.
  • dotnet-local.cmd build Xamarin.Android.sln builds the Mono native projects (which transitively include <shared/xxhash.hh> -> <xxHash/xxhash.h>) cleanly: 0 Errors, all surviving warnings pre-date this PR.

We only consume the `xxhash.h` header from xxHash, and we use it in
inline mode (`XXH_INLINE_ALL`). Vendor the header into
`src-ThirdParty/xxHash/` and remove the `external/xxHash` git submodule
to reduce one more clone-time dependency and align with the ongoing
external/ cleanup (PRs dotnet#11529, dotnet#11568, dotnet#11580, dotnet#11608, dotnet#11613, dotnet#11631).

Changes:

* Copy `external/xxHash/xxhash.h` verbatim to
  `src-ThirdParty/xxHash/xxhash.h` (BSD-2-Clause, already attributed in
  `THIRD-PARTY-NOTICES.TXT`).
* Replace the dead `XXHASH_DIR` variable in `src/native/CMakeLists.txt`
  with a new `THIRDPARTY_DIR` pointing at `${REPO_ROOT_DIR}/src-ThirdParty`
  and add it next to `${EXTERNAL_DIR}` in the six
  `target_include_directories` blocks so `<xxHash/xxhash.h>` resolves from
  the new location (`shared/xxhash.hh` already wraps it transparently).
* Remove the submodule entry from `.gitmodules` and the corresponding
  dependabot ignore entry from `.github/dependabot.yml`.
* Update `.github/skills/update-tpn/SKILL.md` to list xxHash under
  "Vendored Source" instead of "Git Submodules".

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 15, 2026 18:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes the external/xxHash git submodule and vendors the single consumed upstream artifact (xxhash.h) into src-ThirdParty/xxHash/, updating native CMake include paths so #include <xxHash/xxhash.h> continues to resolve on case-sensitive filesystems. This fits the repo’s ongoing external/ cleanup by converting a header-only dependency from submodule → in-tree vendored source.

Changes:

  • Vendor xxhash.h under src-ThirdParty/xxHash/ and rely on existing header-only inline usage (XXH_INLINE_ALL).
  • Introduce THIRDPARTY_DIR in src/native/CMakeLists.txt and add it to the relevant target_include_directories blocks across CLR/Mono/NativeAOT native targets.
  • Remove the external/xxHash submodule wiring and related maintenance/docs entries (.gitmodules, Dependabot ignore, TPN update skill doc).

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src-ThirdParty/xxHash/xxhash.h Adds the vendored xxHash header (BSD-2-Clause) at the include path expected by #include <xxHash/xxhash.h>.
src/native/CMakeLists.txt Defines THIRDPARTY_DIR and wires it into CLR include directories; removes the unused XXHASH_DIR.
src/native/clr/host/CMakeLists.txt Adds ${THIRDPARTY_DIR} to host target include paths so xxHash/xxhash.h resolves.
src/native/clr/shared/CMakeLists.txt Adds ${THIRDPARTY_DIR} to shared CLR target include paths.
src/native/mono/monodroid/CMakeLists.txt Adds ${THIRDPARTY_DIR} to Mono monodroid target include paths.
src/native/mono/shared/CMakeLists.txt Adds ${THIRDPARTY_DIR} to Mono shared target include paths (BUILD_INTERFACE).
src/native/nativeaot/host/CMakeLists.txt Adds ${THIRDPARTY_DIR} to NativeAOT host target include paths.
.gitmodules Removes the external/xxHash submodule entry.
.github/dependabot.yml Drops external/xxHash from the gitsubmodule ignore list.
.github/skills/update-tpn/SKILL.md Moves xxHash from submodules to vendored sources and clarifies header-only usage.

@jonathanpeppers

Copy link
Copy Markdown
Member Author

I'm on the fence about doing this, we can discuss.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants