Skip to content

[multicast] default the multicast feature and wire SoftNPU end-to-end#224

Draft
zeeshanlakhani wants to merge 17 commits into
mainfrom
multicast-e2e
Draft

[multicast] default the multicast feature and wire SoftNPU end-to-end#224
zeeshanlakhani wants to merge 17 commits into
mainfrom
multicast-e2e

Conversation

@zeeshanlakhani

@zeeshanlakhani zeeshanlakhani commented Feb 20, 2026

Copy link
Copy Markdown
Contributor

This enables multicast by default so the packaged artifacts include multicast support without requiring explicit feature flags at build time, and the separate multicast-test CI job is now folded into packet-test. This also wires up the softnpu ASIC backend to support multicast end-to-end by translating DPD's sidecar.p4 table operations into sidecar-lite.p4's simplified P4 pipeline.

asic/SoftNPU:

  • We replace the stubbed AsicMulticastOps implementation (which returned "OperationUnsupported") for group creation and port addition with in-memory group tracking via McGroupData, following the tofino_stub pattern. Group membership is used by the table translation layer to build port bitmaps for sidecar-lite's Replicate extern. - We map sidecar.p4 table names to sidecar-lite equivalents and translate action parameters where the designs differ for emulation.
  • We now track live replication entries per group and rewrite them when members are added or removed. SoftNPU bakes multicast port bitmaps into replication table entries at write time, unlike Tofino, where membership lives in the hardware PRE.
  • We implement a TODO -> Table::clear. The management protocol has no clear message, so we emulate one by dumping table state and removing entries individually, covering the tagged and untagged VLAN variants.
  • A fix for management reads: accumulate UART responses until the newline terminator rather than assuming a single 1024-byte read, and size the UDS receive buffer past any expected dump, since datagram recv truncates silently.

dpd/p4:

  • Fix egress multicast counter attribution. The external arm previously preceded the underlay tag check and matched on every encapsulated replica, leaving underlay_mcast_ctr unreachable. Underlay attribution now keys on the UNDERLAY and UNDERLAY_EXTERNAL tags, and the external arm is the residual case.
    dpd/mcast:
  • We fix an issue with stale in-memory sources after update with absent source list(s).
  • During external-group rollback, we now delete bitmap and replication entries only for internal groups; those entries are only extant for internal groups. External groups share the internal group's IDs, so deleting unconditionally would destroy the live internal group's entries.

swadm:

  • This adds a read-only swadm multicast subcommand (aliased as mcast as well) for inspecting the multicast groups programmed on the switch, with an #[ignore]d integration test run by the packet-test CI job.

References and Deps:

@zeeshanlakhani zeeshanlakhani force-pushed the multicast-e2e branch 5 times, most recently from c111615 to ab643d8 Compare February 27, 2026 01:33
@zeeshanlakhani zeeshanlakhani self-assigned this Apr 1, 2026
This wires up the softnpu ASIC backend to support multicast end-to-end by translating DPD's sidecar.p4 table operations into sidecar-lite.p4's simplified P4 pipeline.

## AsicMulticastOps

We replace the stubbed AsicMulticastOps implementation (which returned "OperationUnsupported" for group creation and port addition with in-memory group tracking via McGroupData, following the tofino_stub pattern.

Group membership is used by the table translation layer to build port bitmaps for sidecar-lite's Replicate extern. Ports >= 128 are rejected at add time to match sidecar-lite's 128-bit bitmap width.

## Table translation (asic/src/softnpu/table.rs)

We map sidecar.p4 table names to sidecar-lite equivalents and translate action parameters where the designs differ for emulation.

## Fix update semantics

We fix an issue with stale in-memory sources after update with absent source list(s). `modify_group_external` wrote back `new.sources.or(existing)` to the in-memory record while `update_external_tables` in the same call canonicalized `new.sources` directly to rewrite the P4 source-filter entries. A `None` update therefore cleared the P4 table but left a stale `Some([...])` in memory.

All multicast action arms are gated with #[cfg(feature = "multicast")].



## References and Deps

- [softnpu #183](oxidecomputer/softnpu#183)
- [propolis #1093](oxidecomputer/propolis#1093)
- [p4rs #240](oxidecomputer/p4#240)
- [sidecar-lite #152](oxidecomputer/sidecar-lite#152)
- tokio: 1.50 (due to softnpu)
- oxide-tokio-rt: 0.1.3 (following-up from tokio's move to 1.50)
- bump oxide-tokio-rt to 0.1.4, tokio to 1.52.1 (matching Omicron)
This adds a read-only `swadm multicast` subcommand (aliased as `mcast` as well)
for inspecting the multicast groups programmed on the switch:

- `multicast list [-t <tag>]` renders all groups as a table.
- `multicast get <group-ip>` shows one group's full config.

Formatting follows existing swadm conventions. To test varying display
points of interest, this includes an `#[ignore]`d integration test, run by
the `packet-test` CI job, that seeds groups via `dpd-client` and asserts each
display branch.
SoftNPU bakes multicast port bitmaps into replication table entries at
write time, unlike Tofino, where membership lives in the hardware PRE and
can change independently of match-action state. A membership change, therefore,
requires rewriting the affected replication entries.

Includes:

*asic/SoftNPU*:
  - We now track live replication entries per group and rewrite them when
    members are added or removed. The replication tracking lock is held across
    the snapshot and table writes so concurrent refreshes cannot interleave.
  - We implement a TODO -> `Table::clear`. The management protocol has no clear
    message, so we emulate one by dumping table state and removing entries
    individually, covering the tagged and untagged VLAN variants.
  - A fix for management reads: accumulate UART responses until the newline
    terminator rather than assuming a single 1024-byte read, and size
    the UDS receive buffer past any expected dump, since datagram recv
    truncates silently.

*dpd/p4*:
  - Fix egress multicast counter attribution. The external arm previously
    preceded the underlay tag check and matched on every encapsulated replica,
    leaving `underlay_mcast_ctr` unreachable. Underlay attribution now keys
    on the UNDERLAY and UNDERLAY_EXTERNAL tags, and the external arm is the
    residual case because external deliveries are decapsulated before the
    counters run and never carry geneve in that path.

*dpd/mcast*:
  - During external-group rollback, we now delete bitmap and replication entries
    only for internal groups; those entries are only extant for internal
    groups. External groups share the internal group's IDs, so deleting
    unconditionally would destroy the live internal group's entries.
@zeeshanlakhani zeeshanlakhani changed the title [multicast] default multicast feature across all crates [multicast] default the multicast feature and wire SoftNPU end-to-end Jul 3, 2026
A replica still carrying geneve at the egress counter block is bound for an
underlay port while external replicas are decapped earlier (at ingress for
external-only groups or in mcast_egress for bifurcated ones). We count underlay
vs external as independent conditions on geneve validity so that each counter
compiles to a gateway with payload counting, matching the form of the working
multicast counter.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant