Summary
Kubernetes v1.36 introduced Server-Side Sharded List and Watch as an alpha feature (KEP-5866). This adds a ShardSelector field to metav1.ListOptions that allows informers to request only objects within a specific hash range, so the API server filters watch events at the source.
The knative.dev/pkg injection framework currently has no support for injecting shardSelector into informer ListOptions. This blocks downstream projects (e.g., Tekton Pipelines) from adopting sharded watches while staying within the injection framework.
Background
The injection framework already supports filtering via WithTweakListOptions — the filtered informer factory uses this to inject label selectors. The ShardSelector would follow a similar pattern but targets the ListOptions.ShardSelector field instead of LabelSelector.
The ShardSelector field accepts CEL expressions like:
shardRange(object.metadata.uid, '0x0000000000000000', '0x8000000000000000')
Proposed Changes
-
New context decorator — similar to filteredinformerfactory.WithSelectors, add a mechanism to inject a shardSelector string into all informer ListOptions. For example:
ctx = sharding.WithShardSelector(ctx, "shardRange(object.metadata.uid, '0x0000000000000000', '0x8000000000000000')")
-
Plumb through informer factories — both the standard and filtered informer factories (and their generated CRD-specific counterparts from injection-gen) should respect the shard selector from context, appending it via WithTweakListOptions.
-
Coordinate with bucket-based leader election — when shard selectors are active, the informer cache only contains objects in the shard range. The bucket-based leader election partitioning must align with the shard boundaries, or a replica could be elected leader for keys it never receives via its watch. This likely means:
- When sharding is enabled, bucket count should equal the number of shards
- Or shard assignment should derive from the bucket/lease ownership
-
Graceful fallback — when the API server doesn't support the feature (pre-v1.36 or feature gate disabled), the controller should fall back to full watches transparently.
Use Case
Tekton Pipelines controllers processing thousands of concurrent PipelineRuns/TaskRuns. With N replicas today, each replica watches all objects and only reconciles its bucket — CPU, memory, and network scale with replica count rather than shard size. With sharded watches, each replica only receives 1/N of events.
Upstream issue: tektoncd/pipeline#10610
Kubernetes References
Summary
Kubernetes v1.36 introduced Server-Side Sharded List and Watch as an alpha feature (KEP-5866). This adds a
ShardSelectorfield tometav1.ListOptionsthat allows informers to request only objects within a specific hash range, so the API server filters watch events at the source.The knative.dev/pkg injection framework currently has no support for injecting
shardSelectorinto informerListOptions. This blocks downstream projects (e.g., Tekton Pipelines) from adopting sharded watches while staying within the injection framework.Background
The injection framework already supports filtering via
WithTweakListOptions— the filtered informer factory uses this to inject label selectors. TheShardSelectorwould follow a similar pattern but targets theListOptions.ShardSelectorfield instead ofLabelSelector.The
ShardSelectorfield accepts CEL expressions like:Proposed Changes
New context decorator — similar to
filteredinformerfactory.WithSelectors, add a mechanism to inject ashardSelectorstring into all informerListOptions. For example:Plumb through informer factories — both the standard and filtered informer factories (and their generated CRD-specific counterparts from injection-gen) should respect the shard selector from context, appending it via
WithTweakListOptions.Coordinate with bucket-based leader election — when shard selectors are active, the informer cache only contains objects in the shard range. The bucket-based leader election partitioning must align with the shard boundaries, or a replica could be elected leader for keys it never receives via its watch. This likely means:
Graceful fallback — when the API server doesn't support the feature (pre-v1.36 or feature gate disabled), the controller should fall back to full watches transparently.
Use Case
Tekton Pipelines controllers processing thousands of concurrent PipelineRuns/TaskRuns. With N replicas today, each replica watches all objects and only reconciles its bucket — CPU, memory, and network scale with replica count rather than shard size. With sharded watches, each replica only receives 1/N of events.
Upstream issue: tektoncd/pipeline#10610
Kubernetes References
ShardSelector stringinmetav1.ListOptionsShardedListAndWatch(alpha in v1.36)