feat: add an example on using autotuning. by sayakpaul · Pull Request #754 · huggingface/kernels

sayakpaul · 2026-08-10T09:03:34Z

Fixes #733

HuggingFaceDocBuilderDev · 2026-08-10T09:04:59Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

github-actions · 2026-08-10T09:08:25Z

Coverage report — `kernels/`

Measured on: Python 3.10 / Torch 2.13.0.
Other CI configurations are not included in this number.
Hardware-gated code paths (ROCm/XPU/NPU/Darwin/Windows) are excluded or unreachable on the Linux+CUDA runner.

Total coverage: 84.3% — threshold: 80% — ✅

Per-file breakdown

Name	Stmts	Miss	Cover	Missing
src/kernels/__init__.py	13	0	100%
src/kernels/_system.py	6	1	83%	10
src/kernels/_versions.py	63	7	89%	46, 49, 52-53, 56-57, 100
src/kernels/backends.py	212	62	71%	40, 44, 48-51, 68, 90, 108, 117, 121, 125-127, 148, 157, 161, 165-167, 188, 199, 201, 208-211, 224, 228, 232-252, 260, 283-303
src/kernels/compat.py	8	1	88%	5
src/kernels/deps.py	58	4	93%	59-60, 101, 104
src/kernels/hf_hub.py	62	5	92%	18, 20, 114, 136-137
src/kernels/importer.py	48	3	94%	106, 110, 113
src/kernels/install.py	50	5	90%	78, 116-117, 155, 173
src/kernels/layer/__init__.py	6	0	100%
src/kernels/layer/_interval_tree.py	103	4	96%	23, 52, 147, 150
src/kernels/layer/device.py	48	14	71%	42, 47-49, 91, 96-98, 101, 149, 152, 155-157
src/kernels/layer/func.py	82	7	91%	83, 113, 185, 303, 309, 322, 340
src/kernels/layer/globals.py	5	0	100%
src/kernels/layer/kernelize.py	74	8	89%	255, 281, 289-290, 296, 300, 316-318
src/kernels/layer/layer.py	211	16	92%	169, 212, 218, 231, 339, 419-420, 432, 441, 449, 460, 489, 493, 506, 559, 589
src/kernels/layer/mode.py	14	0	100%
src/kernels/layer/repos.py	144	42	71%	27, 33, 36-43, 63-64, 70, 73-76, 90, 94, 103-104, 110, 113-116, 123-124, 130, 133-136, 143-144, 150, 153-156, 163-164, 170, 173-176, 257
src/kernels/load.py	74	9	88%	218, 224, 230-231, 252-264
src/kernels/locking.py	108	70	35%	42-109, 113-136, 144, 148-155, 159-169, 173-180
src/kernels/status.py	49	2	96%	23, 81
src/kernels/variants.py	278	22	92%	64, 95, 116, 146, 255-256, 298-301, 303, 387-394, 400-406, 437-443, 455-461, 611-613
src/kernels/verify.py	88	1	99%	32
TOTAL	1804	283	84%

Updated by the Test kernels workflow on commit f03a4cd450b6cd0d3c1c42f542f4e8a72200b3d4.

The tests directory was copied into the Docker test image, but the built kernel artifact was not, so LOCAL_KERNELS pointed at an unexpanded glob. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

danieldk

Really cool! Left a bunch of comments.

danieldk · 2026-08-17T09:11:59Z

+and logs a warning. The lookup is cached, so the file is read at most once
+per process:


I don't think this is true, since by default maxsize=128, so given enough N, K combinations, it will reload. Since the size will be bound by the number of files anyway, I think it's better to use functools.cache, which is equivalent to lru_cache(maxsize=None).

danieldk · 2026-08-17T09:15:51Z

+    N = b.shape[1]
+    if M == 0 or N == 0:
+        return
+    launch_gemm_kernel(a, b, out, get_config(M, N, K))


Question: how does this interact with torch.compile? Since there is a conditional in the main code path (dict lookup).

Compiles cleanly. Just verified:

import torch import kernels gemm_kernel = kernels.get_kernel("kernels-test/gemm-triton-autotune", version=1) def fn(a, b): return gemm_kernel.gemm(torch.nn.functional.silu(a), b) * 2 compiled = torch.compile(fn, fullgraph=True, dynamic=True) b = torch.randn(4096, 4096, device="cuda", dtype=torch.float16) with torch._dynamo.config.patch(error_on_recompile=True): torch.compiler.reset() for M in (3, 33, 333, 3333): compiled(torch.randn(M, 4096, device="cuda", dtype=torch.float16), b) torch.compiler.reset()

danieldk · 2026-08-17T09:18:38Z

          drv = sys: out: out.packages.${sys}.redistributable.torch-cuda;
        }
+        {
+          name = "gemm-triton-autotune-kernel";


We have an existing test that tests usage of pyext. Maybe this new example can replace it? (cuts down test build time)

Tracking here: #770

Co-authored-by: Daniël de Kok <me@danieldk.eu> Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

sayakpaul · 2026-08-17T10:10:20Z

@danieldk thanks for the comments. I believe I have addressed all of them.

danieldk

I think you forgot the docs link in one place.

danieldk · 2026-08-18T09:52:10Z

 - sections:
    - local: builder/writing-kernels
      title: Write kernels
+    - local: builder/triton-autotune


I think you forgot to remove it here.

Good catch. Removed.

feat: add an example on using autotuning.

f754782

sayakpaul temporarily deployed to testpypi August 10, 2026 09:03 — with GitHub Actions Inactive

sayakpaul added 2 commits August 10, 2026 09:11

update

8b46fb7

remove plan

132dc1a

sayakpaul temporarily deployed to testpypi August 10, 2026 09:11 — with GitHub Actions Inactive

sayakpaul marked this pull request as ready for review August 10, 2026 09:12

fix: copy gemm-triton-autotune built kernel into the test image

9805ed2

The tests directory was copied into the Docker test image, but the built kernel artifact was not, so LOCAL_KERNELS pointed at an unexpanded glob. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

sayakpaul temporarily deployed to testpypi August 10, 2026 10:06 — with GitHub Actions Inactive

sayakpaul requested a review from danieldk August 10, 2026 11:24

danieldk reviewed Aug 17, 2026

View reviewed changes

sayakpaul commented Aug 17, 2026

View reviewed changes

Comment thread docs/source/builder/triton-autotune.md Outdated

Apply suggestions from code review

2cd8593

Co-authored-by: Daniël de Kok <me@danieldk.eu> Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

sayakpaul temporarily deployed to testpypi August 17, 2026 09:27 — with GitHub Actions Inactive

remove tip from writing kernels.

f91727c

sayakpaul temporarily deployed to testpypi August 17, 2026 09:30 — with GitHub Actions Inactive

move autotuning guide to advanced practices' section.

9829990

sayakpaul temporarily deployed to testpypi August 17, 2026 09:33 — with GitHub Actions Inactive

Merge branch 'main' into autotuning-example

fa060f6

sayakpaul requested a review from danieldk August 17, 2026 10:10

sayakpaul temporarily deployed to testpypi August 17, 2026 10:10 — with GitHub Actions Inactive

danieldk reviewed Aug 18, 2026

View reviewed changes

sayakpaul mentioned this pull request Aug 18, 2026

We have an existing test that tests usage of pyext. Maybe this new example can replace it? (cuts down test build time) #770

Open

remove duplicate section.

4b024ff

sayakpaul requested a review from danieldk August 18, 2026 11:42

sayakpaul temporarily deployed to testpypi August 18, 2026 11:43 — with GitHub Actions Inactive

Merge branch 'main' into autotuning-example

83e6f10

sayakpaul temporarily deployed to testpypi August 18, 2026 11:44 — with GitHub Actions Inactive

fix sections

28b6d51

sayakpaul temporarily deployed to testpypi August 18, 2026 11:50 — with GitHub Actions Inactive

danieldk approved these changes Aug 19, 2026

View reviewed changes

Merge branch 'main' into autotuning-example

f03a4cd

sayakpaul temporarily deployed to testpypi August 19, 2026 11:48 — with GitHub Actions Inactive

sayakpaul merged commit 18c4d68 into main Aug 19, 2026
55 checks passed

sayakpaul deleted the autotuning-example branch August 19, 2026 13:36

		and logs a warning. The lookup is cached, so the file is read at most once
		per process:

Uh oh!

Conversation

sayakpaul commented Aug 10, 2026

Uh oh!

HuggingFaceDocBuilderDev commented Aug 10, 2026

Uh oh!

github-actions Bot commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage report — kernels/

Uh oh!

danieldk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sayakpaul commented Aug 17, 2026

Uh oh!

danieldk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Aug 10, 2026 •

edited

Loading

Coverage report — `kernels/`