XNNPACK: quantized slice fails to build when its output scale differs from its input

[john-rocky]

Summary

A PT2E-quantized slice_copy can be given an output scale that differs from its
input's. XNNPACK's static-slice builder rejects that, and because the rejection
happens while the delegate initializes, the whole method fails to load — there is
no partial fallback. The same graph is fine in fp32.

Environment: executorch 1.4.0 (pip), torch 2.13.0, torchao PT2E, macOS arm64.

Repro

import torch, torch.nn as nn
from executorch.backends.xnnpack.quantizer.xnnpack_quantizer import (
    XNNPACKQuantizer, get_symmetric_quantization_config)
from torchao.quantization.pt2e.quantize_pt2e import convert_pt2e, prepare_pt2e
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
from executorch.exir import to_edge_transform_and_lower
from executorch.runtime import Runtime

class M(nn.Module):
    def __init__(self):
        super().__init__()
        self.c = nn.Conv2d(3, 8, 3, padding=1)
    def forward(self, x):
        y = self.c(x)
        return y[:, :4] + y[:, 4:]

x = torch.randn(1, 3, 32, 32)
gm = torch.export.export(M().eval(), (x,)).module()
q = XNNPACKQuantizer()
q.set_global(get_symmetric_quantization_config(is_per_channel=True))
prepared = prepare_pt2e(gm, q)
prepared(x)
qm = convert_pt2e(prepared)

et = to_edge_transform_and_lower(torch.export.export(qm, (x,)),
                                 partitioner=[XnnpackPartitioner()]).to_executorch()
open("slice_q.pte", "wb").write(et.buffer)
Runtime.get().load_program("slice_q.pte").load_method("forward")

[XNNCompiler.cpp:1614] Failed to create static slice node 4 with code: xnn_status_invalid_parameter
[XNNPACKBackend.cpp:144] XNNCompiler::compileModel failed: 0x1
[method.cpp:132] Init failed for backend XnnpackBackend: 0x1
RuntimeError: Failed to load method forward, error: 0x:1

What distinguishes the failing node

Both slices read the same conv output. Reading the scales off the converted
graph, one keeps the producer's scale and one does not:

node	input scale	output scale
y[:, :4]	0.019753	0.019753	same
y[:, 4:]	0.019753	0.016902	differs

Only the second one is rejected. Two variants that keep the slice output on its
producer's scale both build and run:

• conv -> slice -> output (single slice, no requantized consumer): OK
• conv -> slice -> relu -> output: OK
• conv -> two slices -> add (above): fails

So the trigger is the scale mismatch, not slicing itself, and not the INT64_MAX
end index that y[:, 4:] carries.

Why the mismatch appears

propagate_annotation in xnnpack_quantizer_utils.py already treats
slice_copy.Tensor as a share-observer op and gives it a SharedQuantizationSpec
pointing at its producer. That pass skips any node already annotated:

if Q_ANNOTATION_KEY in n.meta and n.meta[Q_ANNOTATION_KEY]._annotated:
    continue

The add annotator runs first and annotates its two inputs with its own input
qspecs. One slice is left alone and inherits the producer's scale; the other has
been claimed and keeps add's input observer, which converges to a different
scale. Nothing later reconciles the two.

Where it shows up in practice

Two public models, both conv-only segmentation networks whose blocks split a
tensor and rejoin it: MODNet (portrait matting) and TwinLiteNet. Both convert
cleanly in fp32 and both fail to load once quantized.

Workarounds

Dropping the slice partitioner config lets the node fall back to the portable
kernel, and the rest of the graph still delegates:

from executorch.backends.xnnpack.partition.config import ALL_PARTITIONER_CONFIGS
cfgs = [c for c in ALL_PARTITIONER_CONFIGS if c.__name__ != "SliceCopyConfig"]
XnnpackPartitioner(configs=cfgs)

MODNet then runs at corr 0.99995 against fp32 eager, 26.1 MB down to 6.8 MB, with
93% of ops still on the delegate.

Annotating only conv and linear also avoids it, since the data-movement ops then
carry no quantization at all:

cfg = get_symmetric_quantization_config(is_per_channel=True)
for t in (torch.ops.aten.conv2d.default, torch.ops.aten.linear.default):
    q.set_operator_type(t, cfg)

Two things that would help

A partitioner constraint that refuses a quantized slice_copy whose output qspec
is not shared with its input would turn a load failure into a clean portable
fallback. Making the shared-observer propagation win over a consumer's annotation
for these data-movement ops would fix the cause rather than route around it.