Skip to content

Optimize Radix sort. - #97

Merged
maleadt merged 7 commits into
JuliaGPU:mainfrom
shreyas-omkar:sh/radix-optim-no-ballot
Aug 19, 2026
Merged

Optimize Radix sort.#97
maleadt merged 7 commits into
JuliaGPU:mainfrom
shreyas-omkar:sh/radix-optim-no-ballot

Conversation

@shreyas-omkar

Copy link
Copy Markdown
Member

CUDA (RTX 5080), 4M elements

Type AK radix (ms) torch.sort (ms) AK / torch
UInt32 1.11 0.49 2.25×
Float32 1.12 0.49 2.31×
UInt64 1.85 1.35 1.37×
Float64 1.68 1.33 1.27×

AMD (RX 9060 XT), 4M elements

Type AK radix (ms) torch.sort (ms) AK / torch
UInt32 2.74 1.83 1.50×
Float32 2.76 1.75 1.58×
UInt64 5.70 5.84 0.98×
Float64 5.84 4.43 1.32×

@christiangnrd

christiangnrd commented Jul 20, 2026

Copy link
Copy Markdown
Member

Current OpenCL sort failures seem to be related (or surfaced by) radix sort. On my device, it hangs when julia isn't launched with --check-bounds=yes

MWE:

using pocl_jll, OpenCL; import AcceleratedKernels as AK; begin
v_arr = rand(Int32, 68931)
v = CLArray(v_arr);
@info "Sorting"
AK.sort!(v; alg=AK.RadixSort())
@info "Sorted"
issorted(Array(v))
end

This is probably POCL since this MWE passes on the Asahi driver

@shreyas-omkar

shreyas-omkar commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

histogram completes fine. AK.accumulate hangs for n > 512. Fix is pushed. Can you test the repro once on Mac?

@shreyas-omkar
shreyas-omkar force-pushed the sh/radix-optim-no-ballot branch from 64e0aec to e090e0c Compare July 22, 2026 07:26
@shreyas-omkar
shreyas-omkar force-pushed the sh/radix-optim-no-ballot branch 2 times, most recently from a227115 to e10caf2 Compare August 11, 2026 14:22
Comment thread src/sort/sort.jl Outdated
@shreyas-omkar
shreyas-omkar force-pushed the sh/radix-optim-no-ballot branch 4 times, most recently from 82aebf9 to 57660d5 Compare August 13, 2026 14:33
@shreyas-omkar
shreyas-omkar marked this pull request as ready for review August 17, 2026 11:12
@shreyas-omkar

Copy link
Copy Markdown
Member Author

@christiangnrd and @maleadt Please review it.

@shreyas-omkar
shreyas-omkar force-pushed the sh/radix-optim-no-ballot branch from 6798e18 to bc8ddbd Compare August 17, 2026 11:39
@maleadt
maleadt force-pushed the sh/radix-optim-no-ballot branch from bc8ddbd to 18639af Compare August 17, 2026 17:16
@maleadt

maleadt commented Aug 17, 2026

Copy link
Copy Markdown
Member

Reviewed and tightened this a little:

  • Floating-point ordering now matches Base, including negative NaNs, signed zero, infinities, and reverse sorting.
  • Radix sort now composes rev and order like Base does. In particular, rev=true, order=Reverse correctly gives ascending order.
  • Unsupported custom orderings now fail clearly instead of silently sorting by the wrong ordering.
  • Tuning is available through RadixSort(block_size=..., items_per_thread=...).
  • Configurations that would exceed Metal’s shared-memory limit use the portable path instead of failing kernel compilation.

I also removed the tile abstraction for now; it was pretty unused here. I'd prefer if we land this separately (porting more from GemmKernels.jl's abstraction like the 2D support, and adapting multiple algorithms here to validate the design).

Validated on OpenCL.jl and Metal.jl. Performance improvements are real, so this is looking good.

@maleadt

maleadt commented Aug 18, 2026

Copy link
Copy Markdown
Member

OpenCL.jl fixed in JuliaPackaging/Yggdrasil#14480, CUDA.jl failure is JuliaConcurrent/UnsafeAtomics.jl#23.

shreyas-omkar and others added 7 commits August 18, 2026 23:19
Rework the GPU LSD radix sort (8-bit, 256 buckets) for discrete-GPU
throughput while staying portable across backends (no sub-group
intrinsics):

- atomic histogram + chunked O(32)-rank scatter where the backend
  reports shared-memory atomics; portable scan/broadcast fallback
  otherwise
- process multiple items per thread in the histogram and scatter,
  shrinking the per-(digit, block) histogram and its scan
- single-kernel fast path for arrays that fit one tile (n <= 2*block_size),
  running the whole multi-pass sort in shared memory
- a small Tile primitive (src/tile.jl) for block-strided, coalesced,
  tail-safe indexing, shared by the histogram and scatter kernels
- block_size keyword: default 256 (safe on every backend, fits Metal's
  32 KiB threadgroup budget); pass 512 for ~1.3-1.4x on discrete NVIDIA

Measured vs CUDA.jl's built-in sort: 5-25x faster. Vs torch/CUB: level
on the RTX 5080 at large sizes, within ~1.2-2.1x on the RTX 3060.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shreyas-omkar
shreyas-omkar force-pushed the sh/radix-optim-no-ballot branch from 18639af to f8682c7 Compare August 18, 2026 17:49
@maleadt

maleadt commented Aug 19, 2026

Copy link
Copy Markdown
Member

Hm, I can't reproduce the oneAPI failure. In any case, looks unrelated to this branch.

@maleadt maleadt changed the title Optimize Radix Sort. Optimize Radix sort. Aug 19, 2026
@maleadt
maleadt merged commit f65f8cf into JuliaGPU:main Aug 19, 2026
54 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants