XNNPACK: quantized slice fails to build when its output scale differs from its input

[john-rocky]

A PT2E-quantized slice_copy can end up with an output scale that differs from its input's. XNNPACK's static-slice builder rejects that, and because the rejection happens while the delegate initializes, the whole method fails to load — there is no partial fallback. The same graph converts and runs fine in fp32.

Environment: executorch 1.4.0 (pip), torch 2.13.0, torchao PT2E, macOS arm64.

Repro

import torch, torch.nn as nn
from executorch.backends.xnnpack.quantizer.xnnpack_quantizer import (
    XNNPACKQuantizer, get_symmetric_quantization_config)
from torchao.quantization.pt2e.quantize_pt2e import convert_pt2e, prepare_pt2e
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
from executorch.exir import to_edge_transform_and_lower
from executorch.runtime import Runtime

class M(nn.Module):
    def __init__(self):
        super().__init__()
        self.c = nn.Conv2d(3, 8, 3, padding=1)
    def forward(self, x):
        y = self.c(x)
        return y[:, :4] + y[:, 4:]

x = torch.randn(1, 3, 32, 32)
gm = torch.export.export(M().eval(), (x,)).module()
q = XNNPACKQuantizer()
q.set_global(get_symmetric_quantization_config(is_per_channel=True))
prepared = prepare_pt2e(gm, q)
prepared(x)
qm = convert_pt2e(prepared)

et = to_edge_transform_and_lower(torch.export.export(qm, (x,)),
                                 partitioner=[XnnpackPartitioner()]).to_executorch()
open("slice_q.pte", "wb").write(et.buffer)
Runtime.get().load_program("slice_q.pte").load_method("forward")

[XNNCompiler.cpp:1614] Failed to create static slice node 4 with code: xnn_status_invalid_parameter
[XNNPACKBackend.cpp:144] XNNCompiler::compileModel failed: 0x1
[method.cpp:132] Init failed for backend XnnpackBackend: 0x1
RuntimeError: Failed to load method forward, error: 0x:1

What distinguishes the failing node

Both slices read the same conv output. Reading the scales off the converted graph, one keeps the producer's scale and one does not:

node	input scale	output scale
y[:, :4]	0.019753	0.019753	same
y[:, 4:]	0.019753	0.016902	differs

Only the second is rejected. Two variants that keep the slice output on its producer's scale both build and run:

• conv -> slice -> output (single slice, no requantized consumer): OK
• conv -> slice -> relu -> output: OK
• conv -> two slices -> add (above): fails

So the trigger is the scale mismatch, not slicing itself, and not the INT64_MAX end index that y[:, 4:] carries.

Why the mismatch appears

propagate_annotation in xnnpack_quantizer_utils.py already treats slice_copy.Tensor as a share-observer op and gives it a SharedQuantizationSpec pointing at its producer. That pass skips any node already annotated:

if Q_ANNOTATION_KEY in n.meta and n.meta[Q_ANNOTATION_KEY]._annotated:
    continue

The add annotator runs first and annotates its two inputs with its own input qspecs. One slice is left alone and inherits the producer's scale; the other has been claimed and keeps add's input observer, which converges to a different scale. Nothing later reconciles the two.

Where it shows up in practice

Two public models, both conv-only segmentation networks whose blocks split a tensor and rejoin it: MODNet (portrait matting) and TwinLiteNet. Both convert cleanly in fp32 and both fail to load once quantized.

Workarounds

Dropping the slice partitioner config lets the node fall back to the portable kernel, and the rest of the graph still delegates:

from executorch.backends.xnnpack.partition.config import ALL_PARTITIONER_CONFIGS
cfgs = [c for c in ALL_PARTITIONER_CONFIGS if c.__name__ != "SliceCopyConfig"]
XnnpackPartitioner(configs=cfgs)

MODNet then runs at corr 0.99995 against fp32 eager, 26.1 MB down to 6.8 MB, with 93% of ops still on the delegate.

Annotating only conv and linear also avoids it, since the data-movement ops then carry no quantization at all:

cfg = get_symmetric_quantization_config(is_per_channel=True)
for t in (torch.ops.aten.conv2d.default, torch.ops.aten.linear.default):
    q.set_operator_type(t, cfg)

Two directions that would help

A partitioner constraint that refuses a quantized slice_copy whose output qspec is not shared with its input would turn a load failure into a clean portable fallback. Making the shared-observer propagation win over a consumer's annotation for these data-movement ops would fix the cause rather than route around it.

I'm happy to put together whichever of those you'd prefer, or to test a fix against the two models above.