diff --git a/CHANGELOG.md b/CHANGELOG.md index ca4ec57..0d58ca8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,94 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), Pre-releases are not listed separately. A beta is a step towards the next public version, so its changes are folded into that version's entry as they land and are described against the **last public release**, never against the beta before it. What one beta corrected in an earlier beta does not appear at all: from the point of view of somebody upgrading between released versions, it never happened. +## [3.1.0] + +A security release. Three things a caller could not previously find out — whether a control command was delivered, whether the panel's bootstrap traffic was encrypted, and whether the CA behind the MQTT broker is still the one that was there yesterday — +now have answers. **Install the matching adapter**: this release replaces four `SchemaAdapter` members, so `span-panel-api-schema-0` / `-1` must move to 1.1.0 at the same time. The extras (`span-panel-api[schema-0]`) carry the floor; a direct install of +the adapter distribution does not, and a 1.0.0 adapter against this bootstrap is rejected at discovery with a named error rather than misbehaving. + +### Fixed + +- **A control command that was never sent no longer looks like one that succeeded.** All five setters returned `None` on three separate paths that published nothing: after `close()` (the adapter survives, the bridge does not, so the setter returned having + done nothing at all), with no paho client, and — the one that matters — while the broker was unreachable. A caller had no way to tell any of them from a breaker that actually opened. +- **A publish while the broker is known to be down is refused instead of queued.** paho keeps a QoS-1 publish in its outbound queue across a disconnect and sends it when the connection returns, reusing the same client, so a relay command issued during an + outage fired whenever the broker came back — minutes later, against a panel nobody was watching, with nothing in the UI having said so. The bridge now checks its connection **before** handing the message to paho, which is the only point at which refusing + is still possible. A message the transport declines is reported `FAILED`, and `FAILED` is a promise that nothing will be delivered later. + + The refusal is bounded by what the transport can know: the check is only as fresh as paho's disconnect detection, which is a socket close or the keepalive. A broker that stops answering _without_ closing its socket leaves the connection looking healthy + for up to a keepalive interval and a half, and a publish in that window is still handed over and queued. That caller is told `UNCONFIRMED`, which promises nothing about delivery in either direction, so the outcome remains truthful — but "refused rather + than queued" describes a detected disconnect, not every disconnect. + +- **A discarded command settles instead of waiting out its deadline.** Rebuilding the paho client — or tearing it down — empties the outbound queue; anything still awaiting acknowledgement used to wait out its full deadline (five seconds, for a relay) for + a PUBACK that could no longer arrive. Those now resolve immediately with an explicit "the transport discarded this message; delivery is unknown", so an audit trail carries a terminal state rather than a gap. +- **A failed authentication no longer puts the rejected passphrase in the exception message.** `register_v2` interpolated the response body into `SpanPanelAuthError`, and the panel's validation layer answers a bad passphrase with a 422 that echoes the + submitted credential back. Home Assistant shows that message in the UI, writes it to the config-flow log, and captures it in a diagnostics download. The exception now carries the status code only; the body goes to `DEBUG` with every credential-valued key + replaced, found by a recursive walk rather than a top-level scan — the 422 nests the echo two levels down, so a top-level scan would redact nothing in exactly the response most likely to hold a secret. A body that is not JSON is described by length and + content-type rather than shown. +- **A TLS failure during reconnect no longer re-anchors trust to whatever is answering.** The reconnect path refetched the panel's CA over unauthenticated HTTP and built its trust store from the result, so a panel presenting a certificate from a + _different_ CA was silently accepted. With `ca_pem` configured (below) neither connect nor rebuild fetches anything. + +### Added + +- **`PublishOutcome`, returned by every setter, saying how far a command got.** Four states, because the differences are ones a person acts on differently: `CONFIRMED` (the property reported the requested value on its own topic), `ACCEPTED` (the broker + acknowledged it, no transition seen), `UNCONFIRMED` (handed over, nothing came back before the deadline) and `FAILED` (never handed to the broker, and will not be delivered). + + `UNCONFIRMED` **is not an error and does not raise.** It is the expected result of writing a value that is already current, and it is indistinguishable from a silent policy rejection until SPAN ships a reason code. A write whose value already matches + short-circuits to `UNCONFIRMED` with `no_op=True` immediately, compared in the panel's vocabulary rather than the caller's, so an automation that rewrites the same value on every run does not burn a deadline discovering that. + + `CONFIRMED` is strong evidence and not proof: the panel coalesces every API client into a single `USER` requester, so an observed transition cannot be attributed to one specific write. Nothing is retried — a relay write is not idempotent in its physical + effect, and a racing external change may have legitimately reverted it. + +- **`ca_pem` on `MqttClientConfig` — pin the panel CA instead of refetching it.** Supply it and the trust anchor is a configured value: no network call on connect, none on rebuild. Left unset, the previous behaviour stands (so this remains a minor release) + with one `WARNING` per bridge saying the anchor was obtained unauthenticated. + + A pinned handshake that fails is **not** assumed to mean the CA rotated, because it usually does not: an expired leaf after a panel's clock reset, or a hostname mismatch after the panel moved, produce the identical `SSLCertVerificationError`, and `ssl` + exposes no peer chain on a verification failure. The library performs a separate, display-only fetch of the panel's advertised CA and compares fingerprints. Same fingerprint, or the fetch failed — keep retrying, because a panel reachable on 8883 and not + on its HTTP port is a panel mid-reboot and declaring a permanent failure on missing evidence would convert a transient into an outage. Different fingerprint — `SpanPanelCAChangedError`, on the initial connect as well as in the reconnect loop. + +- **`build_panel_ssl_context(ca_pem)` and `ca_fingerprint(ca_pem)` are public.** A consumer that pins the CA needs the identical context and the identical fingerprint string on its own side of the pin; two implementations would drift, and the one that + drifted would be the security check. + +- **`register_fatal_error_callback` — a typed channel for a transport that has stopped for good.** The reconnect loop is fire-and-forget, so an exception raised inside it killed the task invisibly and a consumer learned nothing. Distinct from the + connection callback on purpose: "disconnected" is what an ordinary outage looks like and waiting through it is correct, while this fires only for a failure no amount of waiting fixes. A consumer that registers nothing is still not left guessing — + `ping()` and `get_snapshot()` re-raise the terminal error. + +- **`ControlInterceptor` — one veto-and-observe point covering all five setters.** `before_publish` may raise to refuse a command, and **the exception propagates unchanged**: the interceptor owns its type and its message, which is what lets a consumer + raise a framework-specific error with a translated message and have it reach the user intact. `after_publish` receives every command including the refusals (an audit that silently omits refusals is worse than no audit) and is fired as a task rather than + awaited, so a sink that merely hangs cannot stall every control call — the price being that ordering across commands is not guaranteed. + + **This is a boundary against callers of this library and nothing more.** It does not constrain anything holding the broker credential: such a process publishes to the panel's broker directly and never reaches this code. + +- **HTTPS for the bootstrap REST calls.** Every `auth.py` and `detection.py` function taking `host`/`port` now accepts `ssl_context`, as do `create_span_client` and `MqttClientConfig` (the MQTT client refetches the schema over HTTP at connect and on every + redispatch). Supplying one moves the call to `https://`; omitting it is byte-identical to 3.0.1. `download_ca_cert` is the one exception and stays on `http://` — it is the bootstrap, fetching the anchor everything else is checked against, so it has + nothing to check itself against. Its docstring now says so plainly, and it takes an `ssl_context` only for the caller that _already_ holds the anchor and wants a verified second copy. + +### Changed + +- **BREAKING FOR IMPLEMENTERS: the five control-protocol setters return `PublishOutcome` instead of `None`.** `CircuitControlProtocol`, `PanelControlProtocol`, `EvseControlProtocol` and `AdoptedControlProtocol` all move. **This is additive for callers** — + a call site that ignores the return value compiles and behaves exactly as before — **and breaking for implementers**: any class type-checked against one of these protocols with `-> None` stops conforming. Test fakes, simulators, and any + `Callable[..., Awaitable[None]]` typed against a setter are precisely that, and they must be updated in the same upgrade. + +- **BREAKING FOR IMPLEMENTERS: `SpanPanelClientProtocol` gains `register_fatal_error_callback`.** Same class of breakage as the setters and it needs the same treatment: additive for callers, but any fake, simulator or alternate transport implementing this + protocol stops conforming until it grows the method — under mypy, and at runtime too, since the protocol is `runtime_checkable` and a consumer that asks `isinstance` before offering a feature will silently stop offering it. It is declared on the protocol + rather than only on `SpanMqttClient` because the consumer codes against protocols, never against transport-specific classes, and it now depends on this channel. + +- **BREAKING FOR ADAPTERS: `set_*_topic` becomes `set_*_target`, returning a `ControlTarget`.** Verifying a write means matching the topic it went to against the property that reports it, and only the adapter knows both — the two schemas spell the same + control differently (flat's relay is `(serial, circuit_id, "relay")`, parent/child's is `(circuit_id, "switch", "relay")`), and parsing them back out of a topic string would put wire-format knowledge in the transport, which is the one component whose job + is not to have any. `ControlTarget` carries the topic and that triple together, produced by one call so they cannot disagree. + + The rename is deliberate rather than a return-type change under the old name. An adapter built for the old contract would keep the old name, pass discovery on presence, and then fail deep inside a setter with an `AttributeError` on a `str`; under a new + name it is rejected at discovery, where the remedy — upgrade the bootstrap and the adapter together — can still be named. `ADAPTER_CONTRACT_VERSION` stays **1**: the contract gained members and lost members, which discovery already detects by name, + rather than redefining one. + +- **`port` is `int | None` on every bootstrap call, defaulting to `None`.** With a plain `int = 80` there is no way to distinguish an omitted port from one a caller deliberately set to 80, and the two need opposite answers once a scheme is in play: `None` + resolves to 80 without an `ssl_context` and 443 with one. An explicit `port=80` **together with** an `ssl_context` raises `SpanPanelValidationError` naming both values rather than guessing — it is exactly what a consumer that stored a port before it + pinned a CA produces, and both readings are defensible. + +- **A supplied `ssl_context` now takes precedence over an injected `httpx.AsyncClient`.** httpx fixes `verify=` at construction, so a context cannot be applied to a client somebody else built; the previous behaviour of yielding an injected client untouched + would have meant a caller passing both got system trust while believing it had pinned the panel CA — a security control that appears to be on and is off. When both are supplied, a dedicated client is built for the call and closed after it. The cost is + named rather than hidden: those calls lose the injected client's connection pool, timeout and header policy. Acceptable because every caller here is bootstrap — registration, detection, schema, FQDN, status — a handful of calls per config entry. + ## [3.0.1] ### Fixed diff --git a/README.md b/README.md index 3717279..589af17 100644 --- a/README.md +++ b/README.md @@ -123,21 +123,25 @@ This ensures that the first `get_snapshot()` after connect returns human-readabl The library defines structural subtyping protocols (PEP 544). All are `runtime_checkable`, so a consumer asks `isinstance` before offering a control rather than assuming the panel in front of it supports one: -| Protocol | Purpose | -| -------------------------- | ------------------------------------------------------------------------------------------ | -| `SpanPanelClientProtocol` | Core lifecycle: `connect`, `close`, `ping`, `get_snapshot`, `register_connection_callback` | -| `CircuitControlProtocol` | Relay and shed-priority control: `set_circuit_relay`, `set_circuit_priority` | -| `PanelControlProtocol` | Panel-level control: `set_dominant_power_source` | -| `EvseControlProtocol` | Per-charger control: `set_evse_charge_limit(node_id, amps)` | -| `AdoptedControlProtocol` | Write to a settable property of a device this library models nothing for | -| `StreamingCapableProtocol` | Push-based updates: `register_snapshot_callback`, `start_streaming`, `stop_streaming` | +| Protocol | Purpose | +| ----------------------------- | ------------------------------------------------------------------------------------------ | +| `SpanPanelClientProtocol` | Core lifecycle: `connect`, `close`, `ping`, `get_snapshot`, `register_connection_callback` | +| `CircuitControlProtocol` | Relay and shed-priority control: `set_circuit_relay`, `set_circuit_priority` | +| `PanelControlProtocol` | Panel-level control: `set_dominant_power_source` | +| `EvseControlProtocol` | Per-charger control: `set_evse_charge_limit(node_id, amps)` | +| `AdoptedControlProtocol` | Write to a settable property of a device this library models nothing for | +| `ControlInterceptionProtocol` | Install one veto-and-observe point for every control command: `set_control_interceptor` | +| `StreamingCapableProtocol` | Push-based updates: `register_snapshot_callback`, `start_streaming`, `stop_streaming` | The first five differ in subject, not just in name. `EvseControlProtocol` is separate from `PanelControlProtocol` because several chargers may be commissioned at once and every call names which one. `AdoptedControlProtocol` differs in kind: the curated setters name a control this library understands and translate or bound the value on the way out, while this one names a property by its wire address and passes the value through, because the declaration is all anybody here knows about it. That write is authorised by the snapshot rather than by its arguments — the transport resolves the property against the current `adopted_devices` and refuses anything it does not find carrying a set topic, so a device this library _does_ model cannot be addressed through it. -A seventh protocol, `SchemaAdapter`, is the bootstrap-to-parser contract rather than a consumer-facing one; it is what an adapter distribution implements and what discovery checks. Integration code programs against the protocols above, not against +`ControlInterceptionProtocol` is separate from the four control protocols rather than a member of them because adding it there would break every implementer of those protocols a second time in one release, and separate from `StreamingCapableProtocol` +because a transport could reasonably offer one and not the other. + +One further protocol, `SchemaAdapter`, is the bootstrap-to-parser contract rather than a consumer-facing one; it is what an adapter distribution implements and what discovery checks. Integration code programs against the protocols above, not against transport-specific classes. ### Snapshots @@ -325,13 +329,70 @@ client.set_snapshot_interval(0) ```python # Set circuit relay (OPEN/CLOSED) -await client.set_circuit_relay("circuit-uuid", "OPEN") +outcome = await client.set_circuit_relay("circuit-uuid", "OPEN") await client.set_circuit_relay("circuit-uuid", "CLOSED") # Set circuit shed priority (NEVER / SOC_THRESHOLD / OFF_GRID) await client.set_circuit_priority("circuit-uuid", "NEVER") ``` +Every setter returns a `PublishOutcome` saying how far the command got. Ignoring it is fine and behaves as it always did; reading it is how a caller tells a breaker that opened from a command that was never sent. + +| `outcome.state` | What it means | +| -------------------------- | ------------------------------------------------------------------------------- | +| `PublishState.CONFIRMED` | The property reported the requested value on its own topic | +| `PublishState.ACCEPTED` | The broker acknowledged the message; no transition observed before the deadline | +| `PublishState.UNCONFIRMED` | Handed over, and nothing came back before the deadline | +| `PublishState.FAILED` | Never handed to the broker, and will not be delivered | + +`UNCONFIRMED` **is not an error and does not raise.** It is the expected result of writing a value that is already current — that case short-circuits immediately with `outcome.no_op` set rather than burning the deadline — and it is indistinguishable from a +silent policy rejection by the panel until SPAN ships a reason code. `FAILED` is the one state that is a promise about the future, which is why the transport refuses to publish while the broker is unreachable instead of letting paho queue the message and +deliver it minutes later. `CONFIRMED` is strong evidence rather than proof: the panel coalesces every API client into a single `USER` requester, so an observed transition cannot be attributed to one specific write. Nothing is retried — a relay write is not +idempotent in its physical effect. + +### Control Interception + +A consumer with a notion of who is asking can refuse a command before it is published, and record every command in one place rather than in five setters that will drift: + +```python +class Gate: + async def before_publish(self, command: ControlCommand) -> None: + if not authorised(command): + raise PermissionError(f"not allowed to write {command.property_id}") + + async def after_publish(self, command: ControlCommand, outcome: PublishOutcome) -> None: + audit.record(command, outcome.state) + +client.set_control_interceptor(Gate()) +``` + +One interceptor at a time, replaceable; pass `None` to remove it. A veto's exception propagates to the caller **unchanged**, so a consumer raising a framework-specific error with a translated message gets it through intact. `after_publish` fires for +refusals too — with `FAILED` and a `vetoed` detail — because an audit that silently omits refusals is worse than no audit; it runs as a task rather than being awaited, so a sink that hangs cannot stall every control call, and ordering across commands is +therefore not guaranteed. + +**This is a boundary against callers of this library and nothing more.** Anything holding the broker credential publishes to the panel directly and never reaches this code. + +### Pinning the Panel CA + +By default the MQTT bridge fetches the panel's CA over unauthenticated HTTP on every connect and trusts whatever answers, which also means a reconnect can silently re-anchor trust to a different CA. Supply the PEM instead and the anchor becomes a +configured value — no network call on connect or on rebuild: + +```python +config = MqttClientConfig(..., ca_pem=stored_pem) + +# The same context and the same fingerprint string the library uses, so a +# consumer's own HTTPS calls and its own pin cannot drift from the library's. +context = build_panel_ssl_context(stored_pem) +fingerprint = ca_fingerprint(stored_pem) +``` + +Leaving `ca_pem` unset keeps the previous behaviour, with one `WARNING` per bridge recording that the anchor was obtained unauthenticated. With it set, a certificate-verification failure is diagnosed rather than assumed: an expired leaf after a panel's +clock reset and a hostname mismatch after the panel moved both produce the identical error, so the library refetches the advertised CA for comparison only and keeps retrying unless the fingerprint has actually changed — at which point it raises +`SpanPanelCAChangedError` carrying both fingerprints and stops. Register `register_fatal_error_callback` to be told; a consumer that registers nothing still cannot mistake a dead bridge for a healthy one, because `ping()` and `get_snapshot()` re-raise. + +The bootstrap REST calls take an `ssl_context` for the same purpose. `download_ca_cert` is the one exception and stays on plain HTTP — it fetches the anchor everything else is checked against, so it has nothing to check itself against, and its result must +be fingerprint-confirmed out of band before it is trusted. + ### Pending-State Detection When the panel publishes Homie `$target` properties, `SpanCircuitSnapshot` exposes the desired state alongside the actual state: diff --git a/packages/schema-0/CHANGELOG.md b/packages/schema-0/CHANGELOG.md index e5f4035..185f758 100644 --- a/packages/schema-0/CHANGELOG.md +++ b/packages/schema-0/CHANGELOG.md @@ -9,6 +9,19 @@ rather than by this version number. A release here means this parser changed, ne Pre-releases are not listed separately. A beta is a step towards the next public version, so its changes are folded into that version's entry as they land and are described against the last public release, never against the beta before it. +## [1.1.0] + +Requires `span-panel-api` **3.1.0 or newer**, and the two must be upgraded together in both directions: this wheel is rejected at discovery by a 3.0.x bootstrap, and a 1.0.0 wheel is rejected by 3.1.0. + +### Changed + +- **BREAKING: `set_circuit_relay_topic`, `set_circuit_priority_topic`, `set_dominant_power_source_topic` and `set_evse_charge_limit_topic` become `set_*_target`, returning a `ControlTarget` instead of a topic string.** 3.1.0 verifies a control write by + matching the topic it published to against the property that reports the change, and only an adapter knows both — flat spells this control `(serial, circuit_id, "relay")` where parent/child spells it `(circuit_id, "switch", "relay")`, so the transport + cannot derive one from the other without learning two topic grammars. `ControlTarget` returns the topic and that triple from a single call, in the same spelling `_on_property_changed` reports under, so the two cannot disagree. + + Renamed rather than re-typed under the old name so that the mismatch is caught at discovery, where the remedy can be named, instead of surfacing as an `AttributeError` on a `str` deep inside a setter. `ADAPTER_CONTRACT` stays **1** — the contract's + member list changed, which discovery already checks by name. + ## [1.0.0] First release as a standalone distribution. Requires `span-panel-api` 3.0.0 or newer. diff --git a/packages/schema-0/pyproject.toml b/packages/schema-0/pyproject.toml index 2562a46..31e3aec 100644 --- a/packages/schema-0/pyproject.toml +++ b/packages/schema-0/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "span-panel-api-schema-0" -version = "1.0.0" +version = "1.1.0" description = "Flat-schema (data-model-version absent) parser for span-panel-api" authors = [ {name = "SpanPanel"} @@ -12,7 +12,11 @@ dependencies = [ # Stated as a stable version rather than as the prerelease the floor tracked # during development: naming a prerelease in a specifier is pip's own signal # that prereleases are acceptable for that requirement. - "span-panel-api>=3.0.0,<4.0", + # 3.1.0 is the first bootstrap that defines `ControlTarget` and asks adapters + # for `set_*_target` instead of `set_*_topic`. Against 3.0.x this wheel is + # rejected at discovery for carrying names that bootstrap does not require — + # so the floor moves with the rename rather than trailing it. + "span-panel-api>=3.1.0,<4.0", ] [project.urls] diff --git a/packages/schema-0/src/span_panel_api_schema_0/adapter.py b/packages/schema-0/src/span_panel_api_schema_0/adapter.py index 9e14bed..1c36d06 100644 --- a/packages/schema-0/src/span_panel_api_schema_0/adapter.py +++ b/packages/schema-0/src/span_panel_api_schema_0/adapter.py @@ -10,6 +10,7 @@ from collections.abc import Callable from typing import TYPE_CHECKING +from span_panel_api.models import ControlTarget from span_panel_api_schema_0.accumulator import HomiePropertyAccumulator from span_panel_api_schema_0.const import PROPERTY_SET_TOPIC_FMT, TYPE_CORE, WILDCARD_TOPIC_FMT from span_panel_api_schema_0.consumer import HomieDeviceConsumer @@ -62,17 +63,32 @@ def circuit_nodes_missing_names(self) -> list[str]: def find_node_by_type(self, type_str: str) -> str | None: return self._consumer.find_node_by_type(type_str) - def set_circuit_relay_topic(self, circuit_id: str) -> str: - return PROPERTY_SET_TOPIC_FMT.format(serial=self._serial_number, node=circuit_id, prop="relay") + def set_circuit_relay_target(self, circuit_id: str) -> ControlTarget: + return self._target(circuit_id, "relay") - def set_circuit_priority_topic(self, circuit_id: str) -> str: - return PROPERTY_SET_TOPIC_FMT.format(serial=self._serial_number, node=circuit_id, prop="shed-priority") + def set_circuit_priority_target(self, circuit_id: str) -> ControlTarget: + return self._target(circuit_id, "shed-priority") - def set_dominant_power_source_topic(self) -> str | None: + def set_dominant_power_source_target(self) -> ControlTarget | None: core_node = self._consumer.find_node_by_type(TYPE_CORE) if core_node is None: return None - return PROPERTY_SET_TOPIC_FMT.format(serial=self._serial_number, node=core_node, prop="dominant-power-source") + return self._target(core_node, "dominant-power-source") + + def _target(self, node: str, prop: str) -> ControlTarget: + """One node/property pair as both a topic and an observation address. + + `device_id` is the panel serial for every flat control, because the flat + schema is a single Homie device and its nodes hang directly off it. That + is the same string `register_property_callback` reports under, which is + what lets the transport match a write against the value that comes back. + """ + return ControlTarget( + topic=PROPERTY_SET_TOPIC_FMT.format(serial=self._serial_number, node=node, prop=prop), + device_id=self._serial_number, + node_id=node, + property_id=prop, + ) def dominant_power_source_payload(self, value: str) -> str | None: """Flat speaks this vocabulary already, so the caller's value passes through. @@ -91,7 +107,7 @@ def dominant_power_source_payload(self, value: str) -> str | None: candidate = value.strip().upper() return candidate if candidate in allowed else None - def set_evse_charge_limit_topic(self, node_id: str) -> str | None: # pylint: disable=unused-argument + def set_evse_charge_limit_target(self, node_id: str) -> ControlTarget | None: # pylint: disable=unused-argument """None: flat firmware publishes no charge-current ceiling to write. The flat `energy.ebus.device.evse` type carries `advertised-current` — @@ -101,7 +117,7 @@ def set_evse_charge_limit_topic(self, node_id: str) -> str | None: # pylint: di generation subscribes to. `node_id` is accepted and unused for the same reason - `set_dominant_power_source_topic` takes no arguments and still returns + `set_dominant_power_source_target` takes no arguments and still returns None on a panel with no core node: the answer does not depend on which charger is asked. """ @@ -112,4 +128,23 @@ def evse_charge_limit_payload(self, node_id: str, amps: int) -> str | None: # p return None def register_property_callback(self, callback: Callable[[str, str, str, str | None], None]) -> Callable[[], None]: - return self._consumer.register_property_callback(callback) + """Subscribe to per-property updates; returns an unregister callable. + + Adapts the accumulator's `(node_id, property_id, new_value, old_value)` + to the protocol's `(device_id, node_id, property_id, value)`, the same + way `SchemaOneAdapter` adapts the SDK's five arguments to the same four. + + This used to be a bare delegation, which handed the accumulator's tuple + straight to a consumer expecting the protocol's. The two agree on arity + and on nothing else: the fourth argument was the *previous* value where + the protocol wants the current one, and the device was missing entirely. + A consumer written against the protocol therefore read a flat panel's + node id as a device id and its old value as its new one, and could not + have noticed -- both are strings. The protocol has no place for the + previous value; a consumer that needs one keeps it. + """ + + def _adapt(node_id: str, property_id: str, value: str, _old_value: str | None) -> None: + callback(self._serial_number, node_id, property_id, value) + + return self._consumer.register_property_callback(_adapt) diff --git a/packages/schema-1/CHANGELOG.md b/packages/schema-1/CHANGELOG.md index 5f9bcb7..a3b0e7a 100644 --- a/packages/schema-1/CHANGELOG.md +++ b/packages/schema-1/CHANGELOG.md @@ -9,6 +9,19 @@ number. A release here means this parser changed, never that the panel did. Pre-releases are not listed separately. A beta is a step towards the next public version, so its changes are folded into that version's entry as they land and are described against the last public release, never against the beta before it. +## [1.1.0] + +Requires `span-panel-api` **3.1.0 or newer**, and the two must be upgraded together in both directions: this wheel is rejected at discovery by a 3.0.x bootstrap, and a 1.0.0 wheel is rejected by 3.1.0. + +### Changed + +- **BREAKING: `set_circuit_relay_topic`, `set_circuit_priority_topic`, `set_dominant_power_source_topic` and `set_evse_charge_limit_topic` become `set_*_target`, returning a `ControlTarget` instead of a topic string.** 3.1.0 verifies a control write by + matching the topic it published to against the property that reports the change, and only an adapter knows both. It matters more here than on the flat side: under parent/child a device is a peer of the panel rather than a node beneath it, so the topic + grammar the transport would have to parse is not even the same one. `ControlTarget` returns the topic and the `(device_id, node_id, property_id)` triple from a single call, in the spelling `_on_property_changed` reports under. + + Renamed rather than re-typed under the old name so that the mismatch is caught at discovery, where the remedy can be named, instead of surfacing as an `AttributeError` on a `str` deep inside a setter. `ADAPTER_CONTRACT` stays **1** — the contract's + member list changed, which discovery already checks by name. + ## [1.0.0] First release as a standalone distribution, and the first parser for the parent/child data model. Requires `span-panel-api` 3.0.0 or newer, and `ebus-sdk` `>=0.19,<0.24`. diff --git a/packages/schema-1/pyproject.toml b/packages/schema-1/pyproject.toml index 45bcd14..990027b 100644 --- a/packages/schema-1/pyproject.toml +++ b/packages/schema-1/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "span-panel-api-schema-1" -version = "1.0.0" +version = "1.1.0" description = "Parent/child schema (data-model-version 1.x) parser for span-panel-api" authors = [ {name = "SpanPanel"} @@ -17,7 +17,11 @@ dependencies = [ # Stated as a stable version rather than as the prerelease the floor tracked # during development: naming a prerelease in a specifier is pip's own signal # that prereleases are acceptable for that requirement. - "span-panel-api>=3.0.0,<4.0", + # Raised to 3.1.0 for the same reason it was 3.0.0 before: that is the first + # bootstrap defining everything this parser imports. 3.1.0 adds `ControlTarget` + # and replaces the `set_*_topic` members with `set_*_target`, so this wheel is + # rejected at discovery by any 3.0.x. + "span-panel-api>=3.1.0,<4.0", # Only this distribution depends on the eBus SDK. The bootstrap and # schema-0 stay clean, so a flat-panel install never pulls it in — which is # what bounds the release coupling this dependency introduces to panels on diff --git a/packages/schema-1/src/span_panel_api_schema_1/adapter.py b/packages/schema-1/src/span_panel_api_schema_1/adapter.py index 985e2d3..2422d03 100644 --- a/packages/schema-1/src/span_panel_api_schema_1/adapter.py +++ b/packages/schema-1/src/span_panel_api_schema_1/adapter.py @@ -22,6 +22,7 @@ from ebus_sdk import Controller +from span_panel_api.models import ControlTarget from span_panel_api_schema_1.charge_limit import ChargeLimitProperty, ChargeLimitSurface, resolve_charge_limit from span_panel_api_schema_1.const import ( HOMIE_DOMAIN, @@ -195,13 +196,13 @@ def find_node_by_type(self, type_str: str) -> str | None: # The adapter names the topic and the transport publishes it, so commanding # a panel needs no connection here either. - def set_circuit_relay_topic(self, circuit_id: str) -> str: - return self._set_topic(circuit_id, NODE_SWITCH, PROP_RELAY) + def set_circuit_relay_target(self, circuit_id: str) -> ControlTarget: + return self._target(circuit_id, NODE_SWITCH, PROP_RELAY) - def set_circuit_priority_topic(self, circuit_id: str) -> str: - return self._set_topic(circuit_id, NODE_LOAD_SHED, PROP_PRIORITY) + def set_circuit_priority_target(self, circuit_id: str) -> ControlTarget: + return self._target(circuit_id, NODE_LOAD_SHED, PROP_PRIORITY) - def set_dominant_power_source_topic(self) -> str | None: + def set_dominant_power_source_target(self) -> ControlTarget | None: """The settable successor: `shed/asserted-islanding-state` on the panel. `dominant-power-source` split in two. The read half became @@ -219,7 +220,7 @@ def set_dominant_power_source_topic(self) -> str | None: Payload translation is not optional: the flat enum this protocol speaks is not the one the panel accepts. See `dominant_power_source_payload`. """ - return self._set_topic(self._serial_number, NODE_SHED, PROP_ASSERTED_ISLANDING_STATE) + return self._target(self._serial_number, NODE_SHED, PROP_ASSERTED_ISLANDING_STATE) def dominant_power_source_payload(self, value: str) -> str | None: """Translate a flat `dominant-power-source` value into an assertion. @@ -245,8 +246,8 @@ def dominant_power_source_payload(self, value: str) -> str | None: "UNKNOWN": "NONE", }.get(value.strip().upper()) - def set_evse_charge_limit_topic(self, node_id: str) -> str | None: - """The set topic for one charger's charge-current limit, or None. + def set_evse_charge_limit_target(self, node_id: str) -> ControlTarget | None: + """Where one charger's charge-current limit is written, or None. Every part of the topic is resolved at runtime. The device id comes from matching `node_id` — the snapshot's harmonised key, which is the @@ -266,7 +267,7 @@ def set_evse_charge_limit_topic(self, node_id: str) -> str | None: if writable is None: return None device, surface, limit = writable - return self._set_topic(device.device_id, surface.node, limit.property_id) + return self._target(device.device_id, surface.node, limit.property_id) def evse_charge_limit_payload(self, node_id: str, amps: int) -> str | None: """The payload to publish for `amps`, or None if it may not be published. @@ -297,7 +298,7 @@ def evse_charge_limit_payload(self, node_id: str, amps: int) -> str | None: return str(amps) def register_property_callback(self, callback: Callable[[str, str, str, str | None], None]) -> Callable[[], None]: - """Subscribe to per-property updates; returns an unregister callable.""" + """Subscribe to per-property updates: `(device_id, node_id, property_id, value)`.""" self._property_callbacks.append(callback) def _unregister() -> None: @@ -333,8 +334,24 @@ def _writable_charge_limit( return device, surface, surface.limit return None - def _set_topic(self, device_id: str, node: str, prop: str) -> str: - return f"{HOMIE_DOMAIN}/{HOMIE_VERSION}/{device_id}/{node}/{prop}/set" + def _target(self, device_id: str, node: str, prop: str) -> ControlTarget: + """One device/node/property address as both a set topic and an observation key. + + The triple is returned alongside the topic rather than left for the + transport to parse back out of it: the transport is the one component + that is supposed to know nothing about this schema's topic grammar, and + under parent/child the device is a peer of the panel rather than a node + beneath it, so the grammar is not even the flat one. + + The spelling here is the spelling `_on_property_changed` reports under, + because a write is verified by matching one against the other. + """ + return ControlTarget( + topic=f"{HOMIE_DOMAIN}/{HOMIE_VERSION}/{device_id}/{node}/{prop}/set", + device_id=device_id, + node_id=node, + property_id=prop, + ) def _require_root(self) -> DiscoveredDevice: """The root, or a clear error if discovery has not finished. diff --git a/packages/schema-1/src/span_panel_api_schema_1/field_metadata.py b/packages/schema-1/src/span_panel_api_schema_1/field_metadata.py index 573c1d7..1d9f047 100644 --- a/packages/schema-1/src/span_panel_api_schema_1/field_metadata.py +++ b/packages/schema-1/src/span_panel_api_schema_1/field_metadata.py @@ -473,7 +473,7 @@ def _lookup( ), (TYPE_PANEL, NODE_SHED, "asserted-islanding-state"): ( "tier 2 of resolve_islanding_state (panel.py), shadowed wherever a MID answers " - "at tier 1, and the write target of set_dominant_power_source_topic" + "at tier 1, and the write target of set_dominant_power_source_target" ), (TYPE_LUGS, NODE_CONNECTION, "feeds-device-id"): ( "the downstream-lugs feedthrough branch of resolve_relative_position " diff --git a/packages/schema-1/src/span_panel_api_schema_1/transport.py b/packages/schema-1/src/span_panel_api_schema_1/transport.py index ea86025..812e3d1 100644 --- a/packages/schema-1/src/span_panel_api_schema_1/transport.py +++ b/packages/schema-1/src/span_panel_api_schema_1/transport.py @@ -78,7 +78,7 @@ def publish(self, topic: str, data: str, qos: int = 1, retain: bool = False) -> """Not supported, and deliberately loud about it. Commands do not travel this way. `SchemaAdapter` exposes - ``set_circuit_relay_topic`` and friends, and the transport publishes to + ``set_circuit_relay_target`` and friends, and the transport publishes to the topic it is handed — so an adapter never needs a socket to command a panel, and neither does this class. diff --git a/pyproject.toml b/pyproject.toml index e9256cb..378b0b1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "span-panel-api" -version = "3.0.1" +version = "3.1.0" description = "A client library for SPAN Panel API" authors = [ {name = "SpanPanel"} @@ -53,8 +53,13 @@ dependencies = [ # is pip's own signal that prereleases are acceptable for that requirement, so a # `>=1.0.0b5` floor here would leave a released install willing to resolve a # future beta of the adapter without anyone asking for one. -schema-0 = ["span-panel-api-schema-0>=1.0.0"] -schema-1 = ["span-panel-api-schema-1>=1.0.0"] +# Raised to 1.1.0 for 3.1.0, and this floor is not cosmetic. The `SchemaAdapter` +# protocol replaced `set_*_topic` with `set_*_target`, and `_derive_required_members` +# makes every public protocol member mandatory of every adapter wheel — so a 1.0.0 +# adapter installed against this bootstrap is rejected at discovery rather than +# working in a degraded way. The two must move together. +schema-0 = ["span-panel-api-schema-0>=1.1.0"] +schema-1 = ["span-panel-api-schema-1>=1.1.0"] [project.urls] Homepage = "https://github.com/SpanPanel/span-panel-api" diff --git a/src/span_panel_api/__init__.py b/src/span_panel_api/__init__.py index 0c311a2..af66103 100644 --- a/src/span_panel_api/__init__.py +++ b/src/span_panel_api/__init__.py @@ -6,6 +6,7 @@ from importlib.metadata import version as _pkg_version +from ._ssl import build_panel_ssl_context, ca_fingerprint from .auth import ( delete_fqdn, download_ca_cert, @@ -22,6 +23,7 @@ SpanPanelAdapterMissingError, SpanPanelAPIError, SpanPanelAuthError, + SpanPanelCAChangedError, SpanPanelConnectionError, SpanPanelError, SpanPanelSchemaVersionError, @@ -37,6 +39,7 @@ DISCOVERY_NAMESPACE, AdoptedDevice, AdoptedProperty, + ControlTarget, DiscoveredMetadata, ExtensionProperty, ExtensionSubject, @@ -54,7 +57,15 @@ V2StatusInfo, is_discovery_path, ) -from .mqtt import MqttClientConfig, SpanMqttClient +from .mqtt import ( + ControlCommand, + ControlDeadlines, + ControlInterceptor, + MqttClientConfig, + PublishOutcome, + PublishState, + SpanMqttClient, +) from .phase_validation import ( PhaseDistribution, are_tabs_opposite_phase, @@ -66,6 +77,7 @@ from .protocol import ( AdoptedControlProtocol, CircuitControlProtocol, + ControlInterceptionProtocol, EvseControlProtocol, PanelCapability, PanelControlProtocol, @@ -89,6 +101,11 @@ # snapshot rather than by its arguments -- a device the adapter models # produces no AdoptedDevice and so cannot be addressed through it. "AdoptedControlProtocol", + # Added 2026-08-25 (3.1.0): one veto/observe point for every control + # command. A protocol of its own rather than a member on the four control + # protocols, which would break their implementers a second time in one + # release. + "ControlInterceptionProtocol", "PanelCapability", "PanelControlProtocol", "SpanPanelClientProtocol", @@ -110,6 +127,9 @@ "ADOPTION_TOPOLOGY_NODE", "AdoptedDevice", "AdoptedProperty", + # Added 2026-08-25 (3.1.0): where a control command goes and which + # property reports it landing, produced by the adapter as one value. + "ControlTarget", "ExtensionProperty", "ExtensionSubject", # Snapshots @@ -129,6 +149,11 @@ "V2AuthResponse", "V2HomieSchema", "V2StatusInfo", + # Added 2026-08-25 with CA pinning: the consumer builds the same context for + # its own HTTPS calls and prints and compares the same fingerprint string, so + # both live here rather than being reimplemented on the other side. + "build_panel_ssl_context", + "ca_fingerprint", "delete_fqdn", "download_ca_cert", "get_fqdn", @@ -140,6 +165,14 @@ # Transport "MqttClientConfig", "SpanMqttClient", + # Added 2026-08-25 (3.1.0): what a control command did. The five setters + # returned None, which could not distinguish a breaker that opened from a + # command the transport never handed to the broker. + "ControlCommand", + "ControlDeadlines", + "ControlInterceptor", + "PublishOutcome", + "PublishState", # Phase validation "PhaseDistribution", "are_tabs_opposite_phase", @@ -152,6 +185,7 @@ "SpanPanelAdapterIncompatibleError", "SpanPanelAdapterMissingError", "SpanPanelAuthError", + "SpanPanelCAChangedError", "SpanPanelSchemaVersionError", "SpanPanelConnectionError", "SpanPanelError", diff --git a/src/span_panel_api/_http.py b/src/span_panel_api/_http.py index 7feeebb..f020165 100644 --- a/src/span_panel_api/_http.py +++ b/src/span_panel_api/_http.py @@ -10,6 +10,14 @@ import httpx +from .exceptions import SpanPanelValidationError + +#: What a bootstrap URL resolves to when the caller names no port. HTTP without a +#: context, HTTPS with one -- so a caller that pins the panel CA and leaves the +#: port alone reaches the right place rather than the plaintext one. +DEFAULT_HTTP_PORT = 80 +DEFAULT_HTTPS_PORT = 443 + @dataclass class _SSLCache: @@ -28,11 +36,37 @@ def get_lock(self) -> asyncio.Lock: _ssl_cache = _SSLCache() -def _build_url(host: str, port: int, path: str) -> str: - """Build an HTTP URL, omitting the port when it is the default (80).""" - if port == 80: - return f"http://{host}{path}" - return f"http://{host}:{port}{path}" +def _build_url(host: str, port: int | None, path: str, ssl_context: ssl.SSLContext | None = None) -> str: + """Build a bootstrap URL, choosing the scheme from whether a CA was supplied. + + ``port`` is ``None`` for "the caller did not say", which is the whole reason + it is nullable: with a plain ``int = 80`` there is no way to tell an omitted + port from one the caller deliberately set to 80, and the two need opposite + answers here. + + An explicit ``port=80`` alongside an ``ssl_context`` is refused rather than + reinterpreted. It is not a hypothetical combination -- a consumer that + persisted a port before it pinned a CA produces exactly this on its first + HTTPS call -- and both readings are defensible: the caller may mean "HTTPS on + the unusual port 80" or may simply not have migrated the stored value. + Guessing either way is a security control that silently does something other + than what it was asked to. + + Raises: + SpanPanelValidationError: ``ssl_context`` supplied with an explicit port 80. + """ + if ssl_context is None: + resolved = DEFAULT_HTTP_PORT if port is None else port + return f"http://{host}{path}" if resolved == DEFAULT_HTTP_PORT else f"http://{host}:{resolved}{path}" + + if port == DEFAULT_HTTP_PORT: + raise SpanPanelValidationError( + f"port={DEFAULT_HTTP_PORT} was passed together with an ssl_context for {host}. " + f"Port {DEFAULT_HTTP_PORT} is the plaintext default and {DEFAULT_HTTPS_PORT} is the TLS one; " + "pass the panel's HTTPS port explicitly, or omit port to take the default." + ) + resolved = DEFAULT_HTTPS_PORT if port is None else port + return f"https://{host}{path}" if resolved == DEFAULT_HTTPS_PORT else f"https://{host}:{resolved}{path}" async def _create_ssl_context() -> ssl.SSLContext: @@ -57,7 +91,31 @@ async def _create_ssl_context() -> ssl.SSLContext: async def _get_client( httpx_client: httpx.AsyncClient | None, timeout: float, + ssl_context: ssl.SSLContext | None = None, ) -> AsyncIterator[httpx.AsyncClient]: + """Yield the client this call should use, honouring both arguments truthfully. + + **A supplied ``ssl_context`` wins over an injected client, deliberately.** + httpx fixes ``verify=`` at construction, so a context cannot be applied to a + client somebody else built. This function used to yield an injected client + untouched, which meant a caller passing both would have got a plaintext-or- + system-trust connection while believing it had pinned the panel CA -- a + security control that appears to be on and is off. The only two honest + options are to refuse the combination or to build a dedicated client, and + building one keeps the pin working for the consumer that motivated it: Home + Assistant injects its shared client at every bootstrap call site. + + The cost is named rather than hidden: these calls lose the shared connection + pool and the injected client's timeout and header policy, and take this + function's ``timeout`` instead. Acceptable because every caller here is + bootstrap -- registration, detection, schema, FQDN, status -- made a handful + of times per config entry, not a hot path. The injected client is never + closed by this function on any path; the dedicated one always is. + """ + if ssl_context is not None: + async with httpx.AsyncClient(timeout=timeout, verify=ssl_context) as client: + yield client + return if httpx_client is not None: yield httpx_client return diff --git a/src/span_panel_api/_ssl.py b/src/span_panel_api/_ssl.py new file mode 100644 index 0000000..dabdee6 --- /dev/null +++ b/src/span_panel_api/_ssl.py @@ -0,0 +1,104 @@ +"""The panel's trust anchor: building a context from it, and naming it. + +Both functions here take a CA in PEM form and nothing else. They make no network +call and hold no state, which is the point -- a trust anchor that is fetched at +the moment it is used is not an anchor, it is whatever answered. The fetching +lives in ``auth.download_ca_cert``, and deciding whether a fetched PEM may be +trusted lives with the caller. + +Public rather than private (``_ssl`` is a module-name convention here, and both +names are re-exported from the package root) because the consumer needs exactly +these two: it builds the same context for its own HTTPS calls, and it prints and +compares the same fingerprint string. Two implementations of a fingerprint that +must agree byte-for-byte is a defect waiting for a firmware upgrade to find it. +""" + +from __future__ import annotations + +import base64 +import binascii +import hashlib +import ssl + +from .exceptions import SpanPanelValidationError + +_PEM_HEADER = "-----BEGIN CERTIFICATE-----" +_PEM_FOOTER = "-----END CERTIFICATE-----" + + +def build_panel_ssl_context(ca_pem: str) -> ssl.SSLContext: + """Build an SSLContext that trusts only the provided panel CA. + + The panel issues a private CA and a server cert signed by it. We do + not want to trust system CAs for this connection, so the context is + built fresh rather than via ``ssl.create_default_context()``. + + The panel's CA is a minimal self-signed certificate that omits the + Authority Key Identifier (AKI) X.509v3 extension. Python 3.13 enabled + ``VERIFY_X509_STRICT`` by default, and that flag rejects such a + certificate with "Missing Authority Key Identifier", which makes the + MQTTS handshake fail on otherwise healthy panels. The flag is cleared + here so the library keeps working across Python versions. + + This does not weaken the parts of verification that matter for this + connection: the trust anchor is still only the panel's own CA, hostname + checking stays enabled, and signature/expiry validation is unchanged. + + Raises: + ssl.SSLError: ``ca_pem`` is not a certificate the ssl module accepts. + ValueError: ``ca_pem`` is malformed in a way ``ssl`` reports as such. + """ + ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + ctx.verify_mode = ssl.CERT_REQUIRED + ctx.check_hostname = True + ctx.verify_flags &= ~ssl.VERIFY_X509_STRICT + ctx.load_verify_locations(cadata=ca_pem) + return ctx + + +def ca_fingerprint(ca_pem: str) -> str: + """SHA-256 over the certificate's DER bytes, lowercase hex, no separators. + + The identity of a trust anchor, in a form a user can compare by eye against + what the panel's label or another install reports, and a consumer can store + in a config entry. + + Taken over the DER rather than over the PEM text on purpose. PEM is a + presentation of the same bytes -- line width, line endings, surrounding + blank lines and any explanatory text a firmware chooses to put above the + header all vary without the certificate changing -- so a hash of the text + would report a rotation that did not happen. That is the worse error of the + two available: an integration that raises "your panel's CA changed" every + time a firmware reflows its PEM teaches its users to dismiss the one time it + matters. + + Only the first certificate in the PEM is read. The panel serves a single + self-signed CA; if a future firmware appends a chain, the anchor is still the + first element, and silently hashing a concatenation would change the + fingerprint of an unchanged anchor. + + Raises: + SpanPanelValidationError: no certificate block, or one whose body is not + valid base64. Distinct from an ``ssl`` error because nothing has been + asked of ``ssl`` yet -- this is a malformed input, and the caller + handling it has a different remedy from one whose certificate is + well-formed and unacceptable. + """ + start = ca_pem.find(_PEM_HEADER) + if start == -1: + raise SpanPanelValidationError("No PEM certificate block found; cannot fingerprint the CA") + body_start = start + len(_PEM_HEADER) + end = ca_pem.find(_PEM_FOOTER, body_start) + if end == -1: + raise SpanPanelValidationError("PEM certificate block is not terminated; cannot fingerprint the CA") + + # Every run of whitespace is dropped rather than only line breaks, so a PEM + # reflowed, re-indented or converted to CRLF fingerprints identically. + body = "".join(ca_pem[body_start:end].split()) + try: + der = base64.b64decode(body, validate=True) + except (binascii.Error, ValueError) as exc: + raise SpanPanelValidationError("PEM certificate body is not valid base64; cannot fingerprint the CA") from exc + if not der: + raise SpanPanelValidationError("PEM certificate block is empty; cannot fingerprint the CA") + return hashlib.sha256(der).hexdigest() diff --git a/src/span_panel_api/adapters.py b/src/span_panel_api/adapters.py index 44f2048..115751d 100644 --- a/src/span_panel_api/adapters.py +++ b/src/span_panel_api/adapters.py @@ -107,7 +107,16 @@ def _is_adapter_class(loaded: object) -> TypeGuard[type[SchemaAdapter]]: def _describe_defect(loaded: object) -> str: - """Explain why `loaded` failed _is_adapter_class. Only called on the error path.""" + """Explain why `loaded` failed _is_adapter_class, and say what to do about it. + + Naming the absent members is the diagnosis, not the remedy, and on its own it + reads as a fault in the adapter. It usually is not one: the ordinary cause is + two packages from different releases installed together, which is the one + thing member presence catches cleanly in both directions -- a new bootstrap + misses the members the adapter has not grown yet, an old one misses the + members the adapter has already renamed. Either way the answer is the same + and the reader should not have to infer it, so this says it. + """ if not isinstance(loaded, type): return f"expected a class, got {type(loaded).__name__}" missing = [member for member in _REQUIRED_MEMBERS if not hasattr(loaded, member)] @@ -120,7 +129,13 @@ def _describe_defect(loaded: object) -> str: f"versioning and was built against an older span-panel-api. Install an adapter " f"release built for contract {ADAPTER_CONTRACT_VERSION}." ) - return f"{loaded.__name__} does not implement SchemaAdapter (missing: {', '.join(missing)})." + return ( + f"{loaded.__name__} does not implement SchemaAdapter (missing: {', '.join(missing)}). " + f"This is what a mismatched pair of packages looks like: the bootstrap and its adapter " + f"are versioned separately and a member added or renamed in one release is absent until " + f"both move. Upgrade span-panel-api and the adapter distribution together — installing " + f"via the extra (span-panel-api[schema-N]) is what keeps their floors honest." + ) def _contract_defect(adapter_cls: type[SchemaAdapter]) -> str | None: diff --git a/src/span_panel_api/auth.py b/src/span_panel_api/auth.py index a4943a1..2d4d502 100644 --- a/src/span_panel_api/auth.py +++ b/src/span_panel_api/auth.py @@ -11,6 +11,8 @@ import asyncio import hashlib import json +import logging +import ssl import uuid import httpx @@ -25,6 +27,77 @@ ) from .models import HomieSchemaTypes, V2AuthResponse, V2HomieSchema, V2StatusInfo +_LOGGER = logging.getLogger(__name__) + +#: Keys whose value is a credential wherever it appears in a response body, +#: compared case-folded because the panel spells them lowerCamelCase and a +#: proxy or validation layer in between may not. +_CREDENTIAL_KEYS = frozenset( + { + "hoppassphrase", + "passphrase", + "ebusbrokerpassword", + "password", + "accesstoken", + "token", + } +) + +_REDACTED = "***" + + +def _redact(value: object) -> object: + """Replace every credential-valued key in a JSON-decoded body, at any depth. + + A top-level key scan is not enough, and the case that matters is the exact + one this exists for: a FastAPI-style 422 echoes the request that failed + validation back under ``detail[].input``, so the passphrase that was just + rejected reappears nested two levels down. Scanning only the outermost + object would redact nothing in precisely the response most likely to carry a + secret. + + Walks dicts and lists; anything else is returned as-is, because a scalar + reached here is a value whose key has already been judged. + """ + if isinstance(value, dict): + return {key: _REDACTED if str(key).lower() in _CREDENTIAL_KEYS else _redact(item) for key, item in value.items()} + if isinstance(value, list): + return [_redact(item) for item in value] + return value + + +def _log_auth_failure(endpoint: str, response: httpx.Response) -> None: + """Record why an auth call failed, at DEBUG and with credentials removed. + + The body is worth keeping — a 422's validation detail is the only thing that + says *which* field the panel objected to — but it is not worth putting in an + exception message, which Home Assistant surfaces in the UI, writes to the + config-flow log, and carries into a diagnostics download. DEBUG is opt-in and + is where a user chasing a registration failure is already looking. + + A body that is not JSON is described rather than shown. It could be an HTML + error page from a proxy, and it could equally be an echo of the request; with + no structure to walk there is no way to redact it, so only its shape is + logged. + """ + try: + parsed = response.json() + except ValueError: + _LOGGER.debug( + "%s failed with HTTP %d; body not JSON (%d bytes, content-type %r)", + endpoint, + response.status_code, + len(response.content), + response.headers.get("content-type", ""), + ) + return + _LOGGER.debug( + "%s failed with HTTP %d; body (credentials redacted): %s", + endpoint, + response.status_code, + json.dumps(_redact(parsed), sort_keys=True), + ) + def _str(val: object) -> str: """Extract a string from a JSON-decoded value.""" @@ -69,8 +142,9 @@ async def register_v2( name: str, passphrase: str | None = None, timeout: float = 10.0, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, ) -> V2AuthResponse: """Register with the SPAN Panel v2 API and obtain access + MQTT credentials. @@ -90,8 +164,13 @@ async def register_v2( passphrase: Panel passphrase (printed on label or set by owner). None for door bypass. timeout: Request timeout in seconds for the internally created client when ``httpx_client`` is None; ignored when a client is injected (caller configures timeouts). - port: HTTP port of the panel bootstrap API + port: Port of the panel bootstrap API. ``None`` means "unspecified" and takes + the scheme's default -- 80 without ``ssl_context``, 443 with one. httpx_client: Optional shared ``httpx.AsyncClient``; not closed by this function. + Not used when ``ssl_context`` is supplied -- httpx fixes its trust store at + construction, so a pinned CA needs a client built for it. See ``_get_client``. + ssl_context: Trust anchor for the panel's HTTPS certificate. Supplying one moves + this call to ``https://``; ``None`` is byte-identical to 3.0.1. Returns: V2AuthResponse with access token and MQTT broker credentials @@ -102,7 +181,7 @@ async def register_v2( SpanPanelTimeoutError: Request timed out SpanPanelAPIError: Unexpected response """ - url = _build_url(host, port, "/api/v2/auth/register") + url = _build_url(host, port, "/api/v2/auth/register", ssl_context) # The panel requires unique client names — append a random suffix. # The passphrase field must be "hopPassphrase" per the SPAN v2 API spec. suffix = uuid.uuid4().hex[:8] @@ -112,7 +191,7 @@ async def register_v2( payload["hopPassphrase"] = passphrase try: - async with _get_client(httpx_client, timeout) as client: + async with _get_client(httpx_client, timeout, ssl_context) as client: response = await client.post(url, json=payload) except httpx.ConnectError as exc: raise SpanPanelConnectionError(f"Cannot reach panel at {host}") from exc @@ -120,7 +199,13 @@ async def register_v2( raise SpanPanelTimeoutError(f"Timed out connecting to {host}") from exc if response.status_code in (401, 403, 422): - raise SpanPanelAuthError(f"Authentication failed (HTTP {response.status_code}): {response.text}") + # Status only, matching the shape the branch below already uses. The body + # is logged at DEBUG instead: a 422 from the panel's validation layer + # echoes the submitted `hopPassphrase` straight back, and interpolating + # `response.text` here put that secret into an exception message that + # Home Assistant shows in the UI and captures in diagnostics. + _log_auth_failure("/api/v2/auth/register", response) + raise SpanPanelAuthError(f"Authentication failed (HTTP {response.status_code})") if response.status_code != 200: raise SpanPanelAPIError(f"Unexpected response from /api/v2/auth/register: HTTP {response.status_code}") @@ -145,13 +230,30 @@ async def register_v2( async def download_ca_cert( host: str, timeout: float = 10.0, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, max_attempts: int = CA_CERT_MAX_ATTEMPTS, backoff_s: float = CA_CERT_BACKOFF_S, + ssl_context: ssl.SSLContext | None = None, ) -> str: """Download the PEM CA certificate from the SPAN Panel. + **This call is unauthenticated and unverified by construction, and its + result must be fingerprint-confirmed by the caller before it is trusted.** + It is the bootstrap: it fetches the very anchor everything else is checked + against, so there is nothing for it to check itself against. Anything on the + path between here and the panel can answer it with a CA of its own, and the + response carries no evidence that would distinguish that from the real one. + Whoever calls this owes the trust decision -- comparing the fingerprint + against one recorded out of band, or against one pinned on a previous + install -- and until that is done the PEM is a candidate, not an anchor. + + ``ssl_context`` exists here for the caller that *already holds* the anchor + and wants a second copy over a verified channel: refetching to compare + fingerprints, which is how a suspected CA rotation is told apart from an + ordinary TLS failure. It does not make the first fetch trustworthy, because + the first fetch has no context to pass. + The panel rate-limits this endpoint and replies with HTTP 429 once the limit is hit. A single reconnect storm — or another client polling the same panel — is enough to trigger it, and a one-shot request would turn @@ -161,8 +263,13 @@ async def download_ca_cert( Args: host: IP address or hostname of the SPAN Panel timeout: Request timeout in seconds when ``httpx_client`` is None; ignored when injected. - port: HTTP port of the panel bootstrap API + port: Port of the panel bootstrap API. ``None`` means "unspecified" and takes + the scheme's default -- 80 without ``ssl_context``, 443 with one. httpx_client: Optional shared ``httpx.AsyncClient``; not closed by this function. + Not used when ``ssl_context`` is supplied -- httpx fixes its trust store at + construction, so a pinned CA needs a client built for it. See ``_get_client``. + ssl_context: Trust anchor for a *re*-fetch by a caller that already holds one. + ``None`` -- the bootstrap case -- is plaintext HTTP, and has to be. max_attempts: Total attempts made when the panel replies HTTP 429. backoff_s: Base delay for exponential backoff between 429 retries. @@ -174,12 +281,12 @@ async def download_ca_cert( SpanPanelTimeoutError: Request timed out SpanPanelAPIError: Unexpected response or invalid PEM """ - url = _build_url(host, port, "/api/v2/certificate/ca") + url = _build_url(host, port, "/api/v2/certificate/ca", ssl_context) last_status: int | None = None for attempt in range(1, max_attempts + 1): try: - async with _get_client(httpx_client, timeout) as client: + async with _get_client(httpx_client, timeout, ssl_context) as client: response = await client.get(url) except httpx.ConnectError as exc: raise SpanPanelConnectionError(f"Cannot reach panel at {host}") from exc @@ -206,8 +313,9 @@ async def download_ca_cert( async def get_homie_schema( host: str, timeout: float = 10.0, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, ) -> V2HomieSchema: """Fetch the Homie property schema from the SPAN Panel. @@ -216,8 +324,13 @@ async def get_homie_schema( Args: host: IP address or hostname of the SPAN Panel timeout: Request timeout in seconds when ``httpx_client`` is None; ignored when injected. - port: HTTP port of the panel bootstrap API + port: Port of the panel bootstrap API. ``None`` means "unspecified" and takes + the scheme's default -- 80 without ``ssl_context``, 443 with one. httpx_client: Optional shared ``httpx.AsyncClient``; not closed by this function. + Not used when ``ssl_context`` is supplied -- httpx fixes its trust store at + construction, so a pinned CA needs a client built for it. See ``_get_client``. + ssl_context: Trust anchor for the panel's HTTPS certificate. Supplying one moves + this call to ``https://``; ``None`` is byte-identical to 3.0.1. Returns: V2HomieSchema with firmware version, schema hash, and type definitions @@ -227,10 +340,10 @@ async def get_homie_schema( SpanPanelTimeoutError: Request timed out SpanPanelAPIError: Unexpected response """ - url = _build_url(host, port, "/api/v2/homie/schema") + url = _build_url(host, port, "/api/v2/homie/schema", ssl_context) try: - async with _get_client(httpx_client, timeout) as client: + async with _get_client(httpx_client, timeout, ssl_context) as client: response = await client.get(url) except httpx.TimeoutException as exc: raise SpanPanelTimeoutError(f"Timed out connecting to {host}") from exc @@ -315,8 +428,9 @@ async def regenerate_passphrase( host: str, token: str, timeout: float = 10.0, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, ) -> str: """Rotate the MQTT broker password on the SPAN Panel. @@ -328,8 +442,13 @@ async def regenerate_passphrase( host: IP address or hostname of the SPAN Panel token: Valid JWT access token timeout: Request timeout in seconds when ``httpx_client`` is None; ignored when injected. - port: HTTP port of the panel bootstrap API + port: Port of the panel bootstrap API. ``None`` means "unspecified" and takes + the scheme's default -- 80 without ``ssl_context``, 443 with one. httpx_client: Optional shared ``httpx.AsyncClient``; not closed by this function. + Not used when ``ssl_context`` is supplied -- httpx fixes its trust store at + construction, so a pinned CA needs a client built for it. See ``_get_client``. + ssl_context: Trust anchor for the panel's HTTPS certificate. Supplying one moves + this call to ``https://``; ``None`` is byte-identical to 3.0.1. Returns: New MQTT broker password @@ -340,11 +459,11 @@ async def regenerate_passphrase( SpanPanelTimeoutError: Request timed out SpanPanelAPIError: Unexpected response """ - url = _build_url(host, port, "/api/v2/auth/passphrase") + url = _build_url(host, port, "/api/v2/auth/passphrase", ssl_context) headers = {"Authorization": f"Bearer {token}"} try: - async with _get_client(httpx_client, timeout) as client: + async with _get_client(httpx_client, timeout, ssl_context) as client: response = await client.put(url, headers=headers) except httpx.ConnectError as exc: raise SpanPanelConnectionError(f"Cannot reach panel at {host}") from exc @@ -366,8 +485,9 @@ async def register_fqdn( token: str, fqdn: str, timeout: float = 10.0, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, ) -> None: """Register an FQDN with the SPAN Panel for TLS certificate SAN inclusion. @@ -380,8 +500,13 @@ async def register_fqdn( token: Valid JWT access token from register_v2 fqdn: Fully qualified domain name to register timeout: Request timeout in seconds when ``httpx_client`` is None; ignored when injected. - port: HTTP port of the panel bootstrap API + port: Port of the panel bootstrap API. ``None`` means "unspecified" and takes + the scheme's default -- 80 without ``ssl_context``, 443 with one. httpx_client: Optional shared ``httpx.AsyncClient``; not closed by this function. + Not used when ``ssl_context`` is supplied -- httpx fixes its trust store at + construction, so a pinned CA needs a client built for it. See ``_get_client``. + ssl_context: Trust anchor for the panel's HTTPS certificate. Supplying one moves + this call to ``https://``; ``None`` is byte-identical to 3.0.1. Raises: SpanPanelAuthError: Token invalid or expired @@ -389,12 +514,12 @@ async def register_fqdn( SpanPanelTimeoutError: Request timed out SpanPanelAPIError: Unexpected response (including 404 if unsupported) """ - url = _build_url(host, port, "/api/v2/dns/fqdn") + url = _build_url(host, port, "/api/v2/dns/fqdn", ssl_context) headers = {"Authorization": f"Bearer {token}"} payload = {"ebusTlsFqdn": fqdn} try: - async with _get_client(httpx_client, timeout) as client: + async with _get_client(httpx_client, timeout, ssl_context) as client: response = await client.post(url, json=payload, headers=headers) except httpx.ConnectError as exc: raise SpanPanelConnectionError(f"Cannot reach panel at {host}") from exc @@ -412,8 +537,9 @@ async def get_fqdn( host: str, token: str, timeout: float = 10.0, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, ) -> str | None: """Retrieve the currently registered FQDN from the SPAN Panel. @@ -421,8 +547,13 @@ async def get_fqdn( host: IP address or hostname of the SPAN Panel token: Valid JWT access token from register_v2 timeout: Request timeout in seconds when ``httpx_client`` is None; ignored when injected. - port: HTTP port of the panel bootstrap API + port: Port of the panel bootstrap API. ``None`` means "unspecified" and takes + the scheme's default -- 80 without ``ssl_context``, 443 with one. httpx_client: Optional shared ``httpx.AsyncClient``; not closed by this function. + Not used when ``ssl_context`` is supplied -- httpx fixes its trust store at + construction, so a pinned CA needs a client built for it. See ``_get_client``. + ssl_context: Trust anchor for the panel's HTTPS certificate. Supplying one moves + this call to ``https://``; ``None`` is byte-identical to 3.0.1. Returns: The registered FQDN string, or ``None`` when no FQDN is configured @@ -435,11 +566,11 @@ async def get_fqdn( SpanPanelTimeoutError: Request timed out SpanPanelAPIError: Unexpected response """ - url = _build_url(host, port, "/api/v2/dns/fqdn") + url = _build_url(host, port, "/api/v2/dns/fqdn", ssl_context) headers = {"Authorization": f"Bearer {token}"} try: - async with _get_client(httpx_client, timeout) as client: + async with _get_client(httpx_client, timeout, ssl_context) as client: response = await client.get(url, headers=headers) except httpx.ConnectError as exc: raise SpanPanelConnectionError(f"Cannot reach panel at {host}") from exc @@ -466,8 +597,9 @@ async def delete_fqdn( host: str, token: str, timeout: float = 10.0, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, ) -> None: """Remove the registered FQDN from the SPAN Panel. @@ -478,8 +610,13 @@ async def delete_fqdn( host: IP address or hostname of the SPAN Panel token: Valid JWT access token from register_v2 timeout: Request timeout in seconds when ``httpx_client`` is None; ignored when injected. - port: HTTP port of the panel bootstrap API + port: Port of the panel bootstrap API. ``None`` means "unspecified" and takes + the scheme's default -- 80 without ``ssl_context``, 443 with one. httpx_client: Optional shared ``httpx.AsyncClient``; not closed by this function. + Not used when ``ssl_context`` is supplied -- httpx fixes its trust store at + construction, so a pinned CA needs a client built for it. See ``_get_client``. + ssl_context: Trust anchor for the panel's HTTPS certificate. Supplying one moves + this call to ``https://``; ``None`` is byte-identical to 3.0.1. Raises: SpanPanelAuthError: Token invalid or expired @@ -487,11 +624,11 @@ async def delete_fqdn( SpanPanelTimeoutError: Request timed out SpanPanelAPIError: Unexpected response """ - url = _build_url(host, port, "/api/v2/dns/fqdn") + url = _build_url(host, port, "/api/v2/dns/fqdn", ssl_context) headers = {"Authorization": f"Bearer {token}"} try: - async with _get_client(httpx_client, timeout) as client: + async with _get_client(httpx_client, timeout, ssl_context) as client: response = await client.delete(url, headers=headers) except httpx.ConnectError as exc: raise SpanPanelConnectionError(f"Cannot reach panel at {host}") from exc @@ -508,16 +645,22 @@ async def delete_fqdn( async def get_v2_status( host: str, timeout: float = 5.0, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, ) -> V2StatusInfo: """Lightweight v2 status probe (unauthenticated). Args: host: IP address or hostname of the SPAN Panel timeout: Request timeout in seconds when ``httpx_client`` is None; ignored when injected. - port: HTTP port of the panel bootstrap API + port: Port of the panel bootstrap API. ``None`` means "unspecified" and takes + the scheme's default -- 80 without ``ssl_context``, 443 with one. httpx_client: Optional shared ``httpx.AsyncClient``; not closed by this function. + Not used when ``ssl_context`` is supplied -- httpx fixes its trust store at + construction, so a pinned CA needs a client built for it. See ``_get_client``. + ssl_context: Trust anchor for the panel's HTTPS certificate. Supplying one moves + this call to ``https://``; ``None`` is byte-identical to 3.0.1. Returns: V2StatusInfo with serial number and firmware version @@ -527,10 +670,10 @@ async def get_v2_status( SpanPanelTimeoutError: Request timed out SpanPanelAPIError: Unexpected response or non-v2 panel """ - url = _build_url(host, port, "/api/v2/status") + url = _build_url(host, port, "/api/v2/status", ssl_context) try: - async with _get_client(httpx_client, timeout) as client: + async with _get_client(httpx_client, timeout, ssl_context) as client: response = await client.get(url) except httpx.ConnectError as exc: raise SpanPanelConnectionError(f"Cannot reach panel at {host}") from exc diff --git a/src/span_panel_api/detection.py b/src/span_panel_api/detection.py index 5ebf92e..3949c12 100644 --- a/src/span_panel_api/detection.py +++ b/src/span_panel_api/detection.py @@ -8,6 +8,7 @@ from __future__ import annotations from dataclasses import dataclass +import ssl import httpx @@ -32,8 +33,9 @@ class DetectionResult: async def detect_api_version( host: str, timeout: float = 5.0, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, ) -> DetectionResult: """Detect SPAN Panel API version. @@ -44,16 +46,21 @@ async def detect_api_version( Args: host: IP address or hostname of the SPAN Panel timeout: Request timeout in seconds when ``httpx_client`` is None; ignored when injected. - port: HTTP port of the panel bootstrap API + port: Port of the panel bootstrap API. ``None`` means "unspecified" and takes + the scheme's default -- 80 without ``ssl_context``, 443 with one. httpx_client: Optional shared ``httpx.AsyncClient``; not closed by this function. + Not used when ``ssl_context`` is supplied -- httpx fixes its trust store at + construction, so a pinned CA needs a client built for it. See ``_get_client``. + ssl_context: Trust anchor for the panel's HTTPS certificate. Supplying one moves + this call to ``https://``; ``None`` is byte-identical to 3.0.1. Returns: DetectionResult indicating which API version is available. On transport failures, ``api_version`` is ``"v1"`` and ``probe_failed`` is True. """ - url = _build_url(host, port, "/api/v2/status") + url = _build_url(host, port, "/api/v2/status", ssl_context) try: - async with _get_client(httpx_client, timeout) as client: + async with _get_client(httpx_client, timeout, ssl_context) as client: response = await client.get(url) except (httpx.ConnectError, httpx.TimeoutException, httpx.RemoteProtocolError): return DetectionResult(api_version="v1", probe_failed=True) diff --git a/src/span_panel_api/exceptions.py b/src/span_panel_api/exceptions.py index cb59d4d..b038bc6 100644 --- a/src/span_panel_api/exceptions.py +++ b/src/span_panel_api/exceptions.py @@ -39,6 +39,43 @@ class SpanPanelServerError(SpanPanelAPIError): """ +class SpanPanelCAChangedError(SpanPanelError): + """The panel is presenting a certificate chain from a different CA than the pin. + + Terminal, and deliberately so. Every other connection failure in this library + is retried, because every other one is something that fixes itself: a panel + rebooting, a broker restarting, a network dropping. This one does not fix + itself, and retrying it is the failure mode -- a client that keeps trying is + a client waiting to succeed against whatever is answering, which is the + outcome pinning exists to prevent. + + It is also not a conclusion drawn from a handshake failure, because that + conclusion cannot be drawn: an expired leaf (a panel whose clock reset after + a power outage) and a hostname mismatch (a panel whose address moved) both + raise the same verification error against a perfectly valid pinned CA, and + the ``ssl`` module exposes no peer chain when verification fails. This is + raised only after a separate fetch of the panel's advertised CA returned a + certificate whose fingerprint differs from the pinned one -- so + ``observed_fingerprint`` is what the panel says its anchor is now, not what + it presented on the connection that failed. + + The two remedies are opposite and only the user can choose between them, so + both fingerprints are carried: re-pin, if the panel's CA was legitimately + rotated by a firmware upgrade or a factory reset, or investigate, if it was + not. + """ + + def __init__(self, expected_fingerprint: str, observed_fingerprint: str) -> None: + self.expected_fingerprint = expected_fingerprint + self.observed_fingerprint = observed_fingerprint + super().__init__( + "The panel is advertising a different CA certificate than the pinned one. " + f"Pinned SHA-256 {expected_fingerprint}, panel now advertises {observed_fingerprint}. " + "Refusing to re-anchor: a rotated CA and an intercepted connection look identical " + "from here. Re-pin only after confirming the new fingerprint out of band." + ) + + class SpanPanelStaleDataError(SpanPanelError): """Raised when get_snapshot() is called while the client isn't live. diff --git a/src/span_panel_api/factory.py b/src/span_panel_api/factory.py index 246864e..5ab4464 100644 --- a/src/span_panel_api/factory.py +++ b/src/span_panel_api/factory.py @@ -8,6 +8,7 @@ import asyncio import logging +import ssl from typing import TYPE_CHECKING from .adapters import resolve_adapter @@ -31,8 +32,9 @@ async def create_span_client( passphrase: str | None = None, mqtt_config: MqttClientConfig | None = None, serial_number: str | None = None, - port: int = 80, + port: int | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, ) -> SpanMqttClient: """Create a SPAN Panel MQTT client. @@ -41,11 +43,26 @@ async def create_span_client( passphrase: Panel passphrase for v2 registration. mqtt_config: Pre-built MQTT broker configuration. serial_number: Panel serial number (extracted from detection/registration if omitted). - port: HTTP port of the panel bootstrap API used for registration and detection. + port: Port of the panel bootstrap API used for registration, detection and the + schema fetch. ``None`` takes the scheme default -- 80 plaintext, 443 with a + context. httpx_client: Optional shared ``httpx.AsyncClient``, used for every request this makes and handed to the client it builds. Not closed here; its timeouts and limits are the caller's, which is why the per-call ``timeout`` defaults are - ignored when one is given. + ignored when one is given. Superseded by ``ssl_context`` where one is given, + because httpx cannot have a trust store applied after construction. + ssl_context: Trust anchor for the panel's HTTPS certificate, applied to every + bootstrap call this makes and carried into the client it returns for the + schema refetches that client does on its own. + + ``register_v2`` is the reason this is not optional in practice: it carries + the panel passphrase up and brings the broker password back, which makes it + the most sensitive request this library issues. The schema fetches carry no + credential, but a plaintext one still hands an observer the panel's + topology, and one HTTP path left open is where the next one gets added. + + Separate from ``MqttClientConfig.ca_pem``, which anchors the *broker* + connection. Both are the same panel CA; supply both, from one PEM. Returns: A connected-ready SpanMqttClient instance. @@ -63,7 +80,9 @@ async def create_span_client( if mqtt_config is None: if passphrase is None: raise SpanPanelAuthError("Neither mqtt_config nor passphrase provided") - auth_response = await register_v2(host, _V2_CLIENT_NAME, passphrase, port=port, httpx_client=httpx_client) + auth_response = await register_v2( + host, _V2_CLIENT_NAME, passphrase, port=port, httpx_client=httpx_client, ssl_context=ssl_context + ) mqtt_config = MqttClientConfig( broker_host=auth_response.ebus_broker_host, username=auth_response.ebus_broker_username, @@ -77,7 +96,7 @@ async def create_span_client( if serial_number is None: # Try to detect from panel status - result = await detect_api_version(host, port=port, httpx_client=httpx_client) + result = await detect_api_version(host, port=port, httpx_client=httpx_client, ssl_context=ssl_context) if result.status_info is not None: serial_number = result.status_info.serial_number @@ -89,7 +108,7 @@ async def create_span_client( # flat-versus-parent/child signal, mirroring MQTT's `info/data-model-version` # — so the parser is chosen before a single message is consumed, rather than # a wrong parser being discovered by its output. - schema = await get_homie_schema(host, port=port, httpx_client=httpx_client) + schema = await get_homie_schema(host, port=port, httpx_client=httpx_client, ssl_context=ssl_context) adapter_key, dispatch_reason = select_adapter_key(schema.data_model_version) # In a thread: resolution reads distribution metadata and imports the adapter # package, and this is the first call in the process to do either. See @@ -106,6 +125,7 @@ async def create_span_client( schema_dispatch_reason=dispatch_reason, schema=schema, httpx_client=httpx_client, + ssl_context=ssl_context, ) await client.connect() return client diff --git a/src/span_panel_api/models.py b/src/span_panel_api/models.py index 3c1f338..24b65f3 100644 --- a/src/span_panel_api/models.py +++ b/src/span_panel_api/models.py @@ -600,6 +600,45 @@ def panel_size(self) -> int: """ +@dataclass(frozen=True, slots=True) +class ControlTarget: + """Where a control command goes, and which property will report it landing. + + Produced by the adapter and by nothing else. Verifying a write means + watching the property that reports it, and the transport holds only a topic + string -- it cannot derive the triple from that without learning two + schemas' topic grammars, which is exactly the wire knowledge the bootstrap + is supposed to be free of. The two are also schema-private and differ: + flat's relay is `(serial, circuit_id, "relay")`, v1.0's is + `(circuit_id, "switch", "relay")`. + + One value rather than two calls, so the topic a command is published to and + the property watched for its effect cannot come from different resolutions + of the same request -- which is how a control ends up confirming itself + against the wrong charger the day a harmonisation rule changes. + + `device_id`, `node_id` and `property_id` are the triple + `SchemaAdapter.register_property_callback` reports values under, and must be + spelled exactly as that stream spells them or nothing will ever match. + """ + + topic: str + """The topic the command is published to, ready to use.""" + + device_id: str + """The Homie device, as the observation stream names it. + + Under the flat schema every property belongs to the panel, so this is the + panel serial. Under parent/child it is whichever device owns the node. + """ + + node_id: str + """The Homie node the property lives on.""" + + property_id: str + """The Homie property that reports this control's value.""" + + @dataclass(frozen=True, slots=True) class AdoptedProperty: """One property of a device this library models no snapshot field for. diff --git a/src/span_panel_api/mqtt/__init__.py b/src/span_panel_api/mqtt/__init__.py index 9580610..3eac3a5 100644 --- a/src/span_panel_api/mqtt/__init__.py +++ b/src/span_panel_api/mqtt/__init__.py @@ -7,11 +7,17 @@ from .async_client import AsyncMQTTClient from .client import SpanMqttClient from .connection import AsyncMqttBridge +from .control import ControlCommand, ControlDeadlines, ControlInterceptor, PublishOutcome, PublishState from .models import MqttClientConfig __all__ = [ "AsyncMQTTClient", "AsyncMqttBridge", + "ControlCommand", + "ControlDeadlines", + "ControlInterceptor", "MqttClientConfig", + "PublishOutcome", + "PublishState", "SpanMqttClient", ] diff --git a/src/span_panel_api/mqtt/client.py b/src/span_panel_api/mqtt/client.py index 0d17eb7..7ff33de 100644 --- a/src/span_panel_api/mqtt/client.py +++ b/src/span_panel_api/mqtt/client.py @@ -10,8 +10,11 @@ import asyncio from collections.abc import Awaitable, Callable import contextlib +from dataclasses import dataclass +from functools import partial from importlib.metadata import version import logging +import ssl import time from typing import TYPE_CHECKING @@ -24,15 +27,17 @@ SpanPanelAdapterIncompatibleError, SpanPanelAdapterMissingError, SpanPanelConnectionError, + SpanPanelError, SpanPanelSchemaVersionError, SpanPanelServerError, SpanPanelStaleDataError, SpanPanelTimeoutError, ) -from ..models import AdoptedProperty, FieldMetadata, HomieSchemaTypes, SpanPanelSnapshot, V2HomieSchema +from ..models import AdoptedProperty, ControlTarget, FieldMetadata, HomieSchemaTypes, SpanPanelSnapshot, V2HomieSchema from ..protocol import PanelCapability, SchemaAdapter from .connection import AsyncMqttBridge from .const import MQTT_READY_TIMEOUT_S +from .control import ControlCommand, ControlDeadlines, ControlInterceptor, PublishOutcome, PublishState from .models import MqttClientConfig if TYPE_CHECKING: @@ -65,6 +70,45 @@ """ +@dataclass(slots=True) +class _Verification: + """One control command waiting for its property to report the value written. + + Not a `PublishOutcome`: this is the machinery underneath one, alive only + between a publish and its deadline. + """ + + key: tuple[str, str, str] + expected: str + observed: asyncio.Future[bool] + """`True` when the property reported the value, `False` when the transport + discarded the message and no report can arrive. Both are endings; only the + first is a transition, and carrying which in the result is what lets the + waiter stop at either without a second future to watch.""" + + +def _discard_verification(verification: _Verification, acknowledged: asyncio.Future[bool]) -> None: + """End a write's wait when the transport says the message is gone. + + Fired when the bridge settles a publish. `False` there means a rebuild + discarded paho's outbound queue, so the panel will never see this write and + nothing will ever report a transition for it -- the deadline would expire + on a certainty. `True` is an ordinary PUBACK and changes nothing: the broker + taking the message is not the panel acting on it, and the write is still + waiting on exactly what it was waiting on before. + + This resolves the wait rather than cancelling it. Cancelling would surface + in the setter as a `CancelledError` indistinguishable from the caller + cancelling the control call itself, and turning one into an outcome would + swallow the other. + """ + if acknowledged.cancelled() or acknowledged.exception() is not None: + # The setter's own cleanup, or a failure that has its own reporting. + return + if not acknowledged.result() and not verification.observed.done(): + verification.observed.set_result(False) + + def _metadata_for_the_log() -> tuple[list[str], str]: """Every distribution-metadata read connect() needs, in one place. @@ -85,12 +129,14 @@ def __init__( serial_number: str, broker_config: MqttClientConfig, snapshot_interval: float = 1.0, - panel_http_port: int = 80, + panel_http_port: int | None = None, adapter_factory: Callable[[str, V2HomieSchema], SchemaAdapter] | None = None, data_model_version: str | None = None, schema_dispatch_reason: str | None = None, schema: V2HomieSchema | None = None, httpx_client: httpx.AsyncClient | None = None, + ssl_context: ssl.SSLContext | None = None, + control_deadlines: ControlDeadlines | None = None, ) -> None: self._host = host self._serial_number = serial_number @@ -104,12 +150,23 @@ def __init__( # the reason this exists at all is that the runtime path was the one place # left without it. See `_get_client`. self._httpx_client = httpx_client + # Anchors this client's own REST calls -- the schema fetch at connect and + # every refetch the redispatch path makes. Separate from the broker's + # anchor in `MqttClientConfig.ca_pem` because they secure different + # transports, and identical in origin because a panel signs both its HTTPS + # certificate and its broker's with the one CA. `None` keeps these fetches + # on plaintext HTTP, which is 3.0.1's behaviour. + self._ssl_context = ssl_context + # How long each setter waits for the panel to report the change back. + # Injectable so a test asserting a refusal does not pay a real deadline. + self._control_deadlines = control_deadlines or ControlDeadlines() self._bridge: AsyncMqttBridge | None = None self._adapter: SchemaAdapter | None = None self._streaming = False self._snapshot_callbacks: list[Callable[[SpanPanelSnapshot], Awaitable[None]]] = [] self._connection_callbacks: list[Callable[[bool], None]] = [] + self._fatal_error_callbacks: list[Callable[[SpanPanelError], None]] = [] self._schema_change_callbacks: list[Callable[[str | None, str | None], None]] = [] self._live = False self._ready_event: asyncio.Event | None = None @@ -143,6 +200,33 @@ def __init__( # One reconsideration at a time. The MQTT trigger can fire repeatedly while a # fetch is retrying, and each would otherwise start its own retry loop. self._redispatch_in_flight = False + # The observation half of write-then-verify. `_observed_values` is the last + # value seen for each `(device_id, node_id, property_id)`, which is what + # answers "is this write a no-op" without a round trip; `_verifications` + # holds the writes currently waiting for their property to report back. + # Both are fed by one callback registered on the adapter in + # `_build_adapter`, so there is a single subscription rather than one per + # command. + self._observed_values: dict[tuple[str, str, str], str] = {} + self._verifications: list[_Verification] = [] + self._unregister_property_observer: Callable[[], None] | None = None + # One interceptor, replaceable. See `set_control_interceptor`. + self._control_interceptor: ControlInterceptor | None = None + + async def _fetch_schema(self) -> V2HomieSchema: + """One REST schema read, with this client's transport settings applied. + + Both callers -- ``connect()`` and the redispatch retry -- had the same + four arguments spelled out separately, and adding the trust anchor to one + and not the other is exactly how a session ends up bootstrapping over + HTTPS and refetching over HTTP for the rest of its life. One call site. + """ + return await get_homie_schema( + self._host, + port=self._panel_http_port, + httpx_client=self._httpx_client, + ssl_context=self._ssl_context, + ) async def _preload_adapter(self, schema: V2HomieSchema) -> None: """Resolve this schema's adapter in a thread, ahead of building it. @@ -206,6 +290,7 @@ def _build_adapter(self, schema: V2HomieSchema) -> SchemaAdapter: self._schema_dispatch_reason = dispatch_reason factory = resolve_adapter(adapter_key, dispatch_reason) self._adapter = factory(self._serial_number, schema) + self._observe(self._adapter) return self._adapter @property @@ -310,11 +395,7 @@ async def connect(self) -> None: # would be a second call to the same unauthenticated endpoint for a # value that cannot have changed. A directly-constructed client has no # schema yet, so it fetches here and dispatches in _build_adapter. - schema = ( - self._schema - if self._schema is not None - else await get_homie_schema(self._host, port=self._panel_http_port, httpx_client=self._httpx_client) - ) + schema = self._schema if self._schema is not None else await self._fetch_schema() self._schema = schema await self._preload_adapter(schema) adapter = self._build_adapter(schema) @@ -368,11 +449,13 @@ async def connect(self) -> None: use_tls=self._broker_config.use_tls, loop=self._loop, panel_http_port=self._panel_http_port, + ca_pem=self._broker_config.ca_pem, ) # Wire message handler self._bridge.set_message_callback(self._on_message) self._bridge.set_connection_callback(self._on_connection_change) + self._bridge.set_fatal_error_callback(self._on_fatal_error) # Pre-rebuild hook: reset Homie accumulator before the bridge swaps # paho clients, so retained messages on the new subscription start # from a clean slate (no stale `$state=disconnected` cached from @@ -469,9 +552,25 @@ async def close(self) -> None: self._live = False async def ping(self) -> bool: - """Check if MQTT connection is alive and device is ready.""" + """Check if MQTT connection is alive and device is ready. + + Raises rather than returning False when the transport has stopped for + good. The two answers are not the same fact: False means "not right now, + still trying", and a consumer's correct response to it is to wait. A + bridge that will never reconnect answering False would put that consumer + in a wait with no end, which is exactly the state the fatal-error channel + exists to make impossible — including for a consumer that registered no + callback. + + Raises: + SpanPanelCAChangedError: the panel is pinned and now advertises a + different CA. Terminal; see the bridge's `fatal_error`. + """ if self._bridge is None or self._adapter is None: return False + fatal = self._bridge.fatal_error + if fatal is not None: + raise fatal return self._bridge.is_connected() and self._adapter.is_ready() def register_connection_callback(self, callback: Callable[[bool], None]) -> Callable[[], None]: @@ -493,6 +592,45 @@ def unregister() -> None: return unregister + def register_fatal_error_callback(self, callback: Callable[[SpanPanelError], None]) -> Callable[[], None]: + """Subscribe to the transport stopping for good. + + Fires once, with the error, for a failure that retrying cannot fix. Today + that is exactly one condition -- the panel advertising a CA other than + the pinned one -- and the reason it needs a channel of its own is that + the reconnect loop runs fire-and-forget: raising inside it kills the task + silently, and the connection callback can only say "disconnected", which + is what a consumer sees during an ordinary outage and correctly waits + through. + + This is a notification, not the only notification. `ping()` and + `get_snapshot()` re-raise the same error, so a consumer that registers + nothing still cannot read a dead transport as a healthy one. + + Returns an unregister function. Calling it twice is safe. + """ + self._fatal_error_callbacks.append(callback) + + def unregister() -> None: + with contextlib.suppress(ValueError): + self._fatal_error_callbacks.remove(callback) + + return unregister + + def _on_fatal_error(self, error: SpanPanelError) -> None: + """Fan the bridge's terminal failure out to subscribers. + + Iterates a copy for the same reason the connection fan-out does: the + expected response is to tear this client down, and a subscriber + unregistering from inside its own callback must not mutate the list + being walked. + """ + for cb in list(self._fatal_error_callbacks): + try: + cb(error) + except Exception: # pylint: disable=broad-exception-caught + _LOGGER.warning("Fatal-error callback raised", exc_info=True) + def register_schema_change_callback(self, callback: Callable[[str | None, str | None], None]) -> Callable[[], None]: """Subscribe to the panel changing schema generation mid-session. @@ -529,6 +667,13 @@ async def get_snapshot(self) -> SpanPanelSnapshot: """ if self._bridge is None or self._adapter is None: raise SpanPanelStaleDataError("Client not connected — call connect() first") + # Ahead of the staleness checks, because it is the stronger statement: + # `SpanPanelStaleDataError` is documented as "panel currently + # unreachable" and consumers poll through it, which is the right response + # to a disconnect and the wrong one to a transport that has stopped. + fatal = self._bridge.fatal_error + if fatal is not None: + raise fatal if not self._bridge.is_connected(): raise SpanPanelStaleDataError("MQTT broker disconnected") if not self._adapter.is_ready(): @@ -537,31 +682,36 @@ async def get_snapshot(self) -> SpanPanelSnapshot: # -- CircuitControlProtocol -------------------------------------------- - async def set_circuit_relay(self, circuit_id: str, state: str) -> None: + async def set_circuit_relay(self, circuit_id: str, state: str) -> PublishOutcome: """Publish relay state change for a circuit. Args: circuit_id: Dashless UUID (matches wire format) state: "OPEN" or "CLOSED" + + Returns: + What happened to the command. `PublishState.UNCONFIRMED` is not an + error -- see `PublishState`. """ - topic = self._require_adapter().set_circuit_relay_topic(circuit_id) - if self._bridge is not None: - self._bridge.publish(topic, state, qos=1) + target = self._require_adapter().set_circuit_relay_target(circuit_id) + return await self._publish_control(target, state, self._control_deadlines.relay) - async def set_circuit_priority(self, circuit_id: str, priority: str) -> None: + async def set_circuit_priority(self, circuit_id: str, priority: str) -> PublishOutcome: """Publish a circuit priority change. Args: circuit_id: Dashless UUID (matches wire format) priority: v2 enum value (NEVER, SOC_THRESHOLD, OFF_GRID) + + Returns: + What happened to the command. See `PublishState`. """ - topic = self._require_adapter().set_circuit_priority_topic(circuit_id) - if self._bridge is not None: - self._bridge.publish(topic, priority, qos=1) + target = self._require_adapter().set_circuit_priority_target(circuit_id) + return await self._publish_control(target, priority, self._control_deadlines.priority) # -- PanelControlProtocol ---------------------------------------------- - async def set_dominant_power_source(self, value: str) -> None: + async def set_dominant_power_source(self, value: str) -> PublishOutcome: """Publish a dominant power source change for the panel. Args: @@ -574,18 +724,17 @@ async def set_dominant_power_source(self, value: str) -> None: would put a string outside that enum on the wire. """ adapter = self._require_adapter() - topic = adapter.set_dominant_power_source_topic() - if topic is None: + target = adapter.set_dominant_power_source_target() + if target is None: raise SpanPanelServerError("Core node not found in panel topology") payload = adapter.dominant_power_source_payload(value) if payload is None: raise SpanPanelServerError(f"{value!r} has no representation on this schema's control") - if self._bridge is not None: - self._bridge.publish(topic, payload, qos=1) + return await self._publish_control(target, payload, self._control_deadlines.dominant_power_source) # -- EvseControlProtocol ----------------------------------------------- - async def set_evse_charge_limit(self, node_id: str, amps: int) -> None: + async def set_evse_charge_limit(self, node_id: str, amps: int) -> PublishOutcome: """Publish a charge-current limit for one commissioned EV charger. Args: @@ -600,18 +749,17 @@ async def set_evse_charge_limit(self, node_id: str, amps: int) -> None: facts and a user can act on only one of them. """ adapter = self._require_adapter() - topic = adapter.set_evse_charge_limit_topic(node_id) - if topic is None: + target = adapter.set_evse_charge_limit_target(node_id) + if target is None: raise SpanPanelServerError(f"No settable charge-current limit on EVSE {node_id!r}") payload = adapter.evse_charge_limit_payload(node_id, amps) if payload is None: raise SpanPanelServerError(f"{amps} A is outside what EVSE {node_id!r} accepts") - if self._bridge is not None: - self._bridge.publish(topic, payload, qos=1) + return await self._publish_control(target, payload, self._control_deadlines.evse_charge_limit) # -- AdoptedControlProtocol -------------------------------------------- - async def set_adopted_property(self, device_id: str, node_id: str, property_id: str, value: str) -> None: + async def set_adopted_property(self, device_id: str, node_id: str, property_id: str, value: str) -> PublishOutcome: """Publish a write to one settable property of an adopted device. Args: @@ -641,8 +789,272 @@ async def set_adopted_property(self, device_id: str, node_id: str, property_id: surface = self._adopted_property(device_id, node_id, property_id) if surface is None or surface.set_topic is None: raise SpanPanelServerError(f"No settable adopted property {node_id}/{property_id} on device {device_id!r}") - if self._bridge is not None: - self._bridge.publish(surface.set_topic, value, qos=1) + target = ControlTarget( + topic=surface.set_topic, + device_id=device_id, + node_id=node_id, + property_id=property_id, + ) + return await self._publish_control(target, value, self._control_deadlines.adopted_property) + + # -- ControlInterceptionProtocol ---------------------------------------- + + def set_control_interceptor(self, interceptor: ControlInterceptor | None) -> None: + """Install the one interceptor every control command passes through. + + `None` removes it. Replacing rather than appending is deliberate: two + interceptors would raise ordering and precedence questions with no + principled answer, and a consumer that needs several composes them on + its own side where it knows which wins. + + See `ControlInterceptor` for the contract, and in particular for what + this is *not* -- it constrains callers of this client, not anything + holding the broker credential. + """ + self._control_interceptor = interceptor + + # -- The one place a control command reaches the wire ------------------- + + def _observe(self, adapter: SchemaAdapter) -> None: + """Watch every property this adapter reports, for write-then-verify. + + One subscription for the life of an adapter, rather than one per command: + registering per write would mean the no-op check had nothing to read, + because the *pre*-write value has to already be known when the write + arrives. + + Re-registered whenever the adapter is replaced -- a transport rebuild or + a schema-generation change -- because the old instance's callback list + does not survive it. The observed values are dropped at the same moment: + they describe a tree that is being replaced, and a stale one would answer + the no-op check for a panel that no longer exists. + + In-flight verifications are deliberately *not* dropped. A write whose + deadline outlives the rebuild is re-armed against the new tree for free, + and if the value never arrives it expires into `UNCONFIRMED`, which is + the honest answer. + """ + if self._unregister_property_observer is not None: + self._unregister_property_observer() + self._observed_values.clear() + self._unregister_property_observer = adapter.register_property_callback(self._on_property_value) + + def _on_property_value(self, device_id: str, node_id: str, property_id: str, value: str | None) -> None: + """Record one property's value and resolve any write waiting for it.""" + if value is None: + return + key = (device_id, node_id, property_id) + self._observed_values[key] = value + for verification in list(self._verifications): + if verification.key == key and verification.expected == value and not verification.observed.done(): + verification.observed.set_result(True) + + async def _publish_control(self, target: ControlTarget, value: str, deadline: float) -> PublishOutcome: + """Run one control command past the interceptor, then deliver it. + + Interception wraps *everything*, including the refusals and the no-op + short-circuit, because a consumer's authorisation decision has to be + made before this client decides anything -- and because an interceptor + that saw only the commands that reached the wire would be an audit with + a hole in it exactly where the interesting cases are. + + A veto's exception is re-raised untouched. `after_publish` still fires + for it, with `FAILED` and a `vetoed` detail. + """ + interceptor = self._control_interceptor + if interceptor is None: + return await self._deliver_control(target, value, deadline) + + command = ControlCommand( + device_id=target.device_id, + node_id=target.node_id, + property_id=target.property_id, + value=value, + topic=target.topic, + ) + try: + await interceptor.before_publish(command) + except Exception: # pylint: disable=broad-exception-caught + # Not caught to handle -- caught to record the refusal, then + # re-raised unchanged so the caller sees the interceptor's own + # exception type and message. `CancelledError` is a BaseException + # and correctly bypasses this: a cancelled call is not a refusal. + refusal = PublishOutcome( + state=PublishState.FAILED, + topic=target.topic, + value=value, + detail="vetoed", + ) + self._fire_after_publish(interceptor, command, refusal) + raise + + outcome = await self._deliver_control(target, value, deadline) + self._fire_after_publish(interceptor, command, outcome) + return outcome + + def _fire_after_publish( + self, + interceptor: ControlInterceptor, + command: ControlCommand, + outcome: PublishOutcome, + ) -> None: + """Hand the result to the interceptor without waiting for it. + + Awaiting would let a sink that merely hangs -- not raises -- stall every + control call in the process, which is a worse failure than a late audit + row. Tracked in `_background_tasks` so it is cancelled on `close()`, and + so it is not garbage-collected mid-flight. + """ + loop = self._loop or asyncio.get_running_loop() + task = loop.create_task(self._run_after_publish(interceptor, command, outcome), name="span_mqtt_after_publish") + self._background_tasks.add(task) + task.add_done_callback(self._background_tasks.discard) + + async def _run_after_publish( + self, + interceptor: ControlInterceptor, + command: ControlCommand, + outcome: PublishOutcome, + ) -> None: + """Await `after_publish`, absorbing whatever it does.""" + try: + await interceptor.after_publish(command, outcome) + except Exception: # pylint: disable=broad-exception-caught + _LOGGER.warning("Control interceptor's after_publish raised", exc_info=True) + + async def _deliver_control(self, target: ControlTarget, value: str, deadline: float) -> PublishOutcome: + """Publish one control command and report how far it got. + + Every setter funnels through here, which is what makes the refusals and + the verification below true of all of them rather than of whichever were + remembered. + + Two refusals, both of which used to be silent successes: + + - **No bridge.** `close()` clears `_bridge` and leaves `_adapter` in + place, so `_require_adapter()` passed and the setter returned `None` + having done nothing at all. + - **Not connected.** The bridge declines to hand the message to paho at + all, because paho would queue it and deliver it whenever the broker + returns. See `AsyncMqttBridge.publish`. + + Anything past those was handed over and may still arrive, so no outcome + beyond this point is `FAILED`. + + **The no-op short-circuit compares wire vocabulary, not the caller's.** + `value` here is already the adapter's translation -- a dominant-power- + source request of `BATTERY` reaches this as `OFF_GRID` under v1.0 -- and + the observed value is what the panel published. Comparing the caller's + string would compare two different vocabularies and never match, which + would burn a full deadline on every no-op write. + + **`CONFIRMED` is strong evidence, not proof.** The panel coalesces every + API client into a single `USER` requester, so an observed transition to + the value just written cannot be attributed to this write specifically. + A second client writing the same value at the same moment is + indistinguishable. + + **No retry, deliberately.** A relay write is not idempotent in its + physical effect, and a racing external change may have legitimately + reverted it. The state is reported and the caller decides. + """ + bridge = self._bridge + if bridge is None: + return PublishOutcome( + state=PublishState.FAILED, + topic=target.topic, + value=value, + detail="transport is closed", + ) + + key = (target.device_id, target.node_id, target.property_id) + if self._observed_values.get(key) == value: + # Nothing will transition, because nothing has to. Waiting out the + # deadline to discover that is the common case for an automation that + # writes the same value on every run. + return PublishOutcome( + state=PublishState.UNCONFIRMED, + topic=target.topic, + value=value, + no_op=True, + detail="the property already reports this value", + ) + + # Armed before the publish, so a panel that answers immediately cannot + # transition in the window between the two. + verification = _Verification(key=key, expected=value, observed=asyncio.get_running_loop().create_future()) + self._verifications.append(verification) + acknowledged: asyncio.Future[bool] | None = None + try: + acknowledged = bridge.publish(target.topic, value) + if acknowledged is None: + return PublishOutcome( + state=PublishState.FAILED, + topic=target.topic, + value=value, + detail="broker not connected; refused rather than queued", + ) + # A discarded message ends the wait as surely as a transition does. + # Without this the deadline is the only thing that ends it, so a + # relay write would sit out its full five seconds against a transport + # that had already thrown the message away and can never report back. + acknowledged.add_done_callback(partial(_discard_verification, verification)) + try: + transitioned = await asyncio.wait_for(verification.observed, timeout=deadline) + except TimeoutError: + return self._unverified_outcome(target, value, deadline, acknowledged) + if not transitioned: + return self._unverified_outcome(target, value, deadline, acknowledged) + return PublishOutcome(state=PublishState.CONFIRMED, topic=target.topic, value=value) + finally: + with contextlib.suppress(ValueError): + self._verifications.remove(verification) + # A transition can land before the PUBACK does, leaving this future + # pending with nobody left to read it. Cancelling settles it, which + # is what triggers the bridge to forget the message id -- otherwise + # the pending map grows by one for every confirmed write. + if acknowledged is not None and not acknowledged.done(): + acknowledged.cancel() + + def _unverified_outcome( + self, + target: ControlTarget, + value: str, + deadline: float, + acknowledged: asyncio.Future[bool], + ) -> PublishOutcome: + """What to say when the deadline passed without the property reporting back. + + Three different facts, and the broker's QoS-1 acknowledgement is what + separates them. Folding them together would discard information the + transport is already holding: "the broker took it and the panel did not + act" points at the panel, "nothing acknowledged it" points at the link, + and "the transport discarded it" points at neither. + """ + if not acknowledged.done() or acknowledged.cancelled(): + acknowledged.cancel() + return PublishOutcome( + state=PublishState.UNCONFIRMED, + topic=target.topic, + value=value, + detail=f"no broker acknowledgement and no transition within {deadline}s", + ) + if acknowledged.result(): + return PublishOutcome( + state=PublishState.ACCEPTED, + topic=target.topic, + value=value, + detail=f"acknowledged by the broker; no transition within {deadline}s", + ) + return PublishOutcome( + state=PublishState.UNCONFIRMED, + topic=target.topic, + value=value, + # Says what happened, not which of the two causes caused it. The + # bridge discards its outbound queue on a rebuild and on teardown + # alike, and naming one here reported a `close()` as a rebuild. + detail="the transport discarded this message before the broker acknowledged; delivery is unknown", + ) def _adopted_property(self, device_id: str, node_id: str, property_id: str) -> AdoptedProperty | None: """The named property of the named adopted device in the current snapshot. @@ -851,7 +1263,7 @@ async def _fetch_schema_with_retry(self) -> V2HomieSchema | None: attempts = 0 while True: try: - return await get_homie_schema(self._host, port=self._panel_http_port, httpx_client=self._httpx_client) + return await self._fetch_schema() except ( SpanPanelConnectionError, SpanPanelTimeoutError, diff --git a/src/span_panel_api/mqtt/connection.py b/src/span_panel_api/mqtt/connection.py index af17b99..20c431e 100644 --- a/src/span_panel_api/mqtt/connection.py +++ b/src/span_panel_api/mqtt/connection.py @@ -22,8 +22,16 @@ from paho.mqtt.properties import Properties from paho.mqtt.reasoncodes import ReasonCode +from .._ssl import build_panel_ssl_context, ca_fingerprint from ..auth import download_ca_cert -from ..exceptions import SpanPanelAPIError, SpanPanelConnectionError, SpanPanelTimeoutError +from ..exceptions import ( + SpanPanelAPIError, + SpanPanelCAChangedError, + SpanPanelConnectionError, + SpanPanelError, + SpanPanelTimeoutError, + SpanPanelValidationError, +) from .async_client import AsyncMQTTClient from .const import ( MQTT_CONNECT_TIMEOUT_S, @@ -41,33 +49,6 @@ _LOGGER = logging.getLogger(__name__) -def _build_ssl_context(ca_pem: str) -> ssl.SSLContext: - """Build an SSLContext that trusts only the provided panel CA. - - The panel issues a private CA and a server cert signed by it. We do - not want to trust system CAs for this connection, so the context is - built fresh rather than via ``ssl.create_default_context()``. - - The panel's CA is a minimal self-signed certificate that omits the - Authority Key Identifier (AKI) X.509v3 extension. Python 3.13 enabled - ``VERIFY_X509_STRICT`` by default, and that flag rejects such a - certificate with "Missing Authority Key Identifier", which makes the - MQTTS handshake fail on otherwise healthy panels. The flag is cleared - here so the library keeps working across Python versions. - - This does not weaken the parts of verification that matter for this - connection: the trust anchor is still only the panel's own CA (fetched - over the local network immediately before use), hostname checking stays - enabled, and signature/expiry validation is unchanged. - """ - ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) - ctx.verify_mode = ssl.CERT_REQUIRED - ctx.check_hostname = True - ctx.verify_flags &= ~ssl.VERIFY_X509_STRICT - ctx.load_verify_locations(cadata=ca_pem) - return ctx - - class AsyncMqttBridge: """Event-loop-driven paho-mqtt wrapper with async callback dispatch. @@ -87,7 +68,8 @@ def __init__( transport: MqttTransport = "tcp", use_tls: bool = True, loop: asyncio.AbstractEventLoop | None = None, - panel_http_port: int = 80, + panel_http_port: int | None = None, + ca_pem: str | None = None, ) -> None: self._host = host self._port = port @@ -99,6 +81,22 @@ def __init__( self._use_tls = use_tls self._loop = loop self._panel_http_port = panel_http_port + # The pin. See `_trust_anchor_pem` for what supplying it changes and + # `MqttClientConfig.ca_pem` for why `None` is still allowed. + self._ca_pem = ca_pem + self._warned_unpinned = False + + # Terminal state. Set only for a failure that retrying cannot fix, which + # today means exactly one thing: the panel's advertised CA no longer + # matches the pin. Everything else this bridge encounters is retried, so + # this staying `None` is the normal condition even during a long outage. + self._fatal_error: SpanPanelError | None = None + self._fatal_error_callback: Callable[[SpanPanelError], None] | None = None + + # QoS-1 publishes awaiting a PUBACK, keyed by paho message id. Emptied + # by `_on_publish` one at a time, and wholesale by + # `_resolve_pending_publishes` when the outbound queue ceases to exist. + self._pending_publishes: dict[int, asyncio.Future[bool]] = {} self._connected = False self._client: AsyncMQTTClient | None = None @@ -125,6 +123,46 @@ def set_connection_callback(self, callback: Callable[[bool], None]) -> None: """Set callback for connection state changes: callback(is_connected).""" self._connection_callback = callback + def set_fatal_error_callback(self, callback: Callable[[SpanPanelError], None]) -> None: + """Set the callback fired when this bridge stops for good. + + The reconnect loop is created fire-and-forget, so nothing awaits it and + nothing reads its exception. Raising inside it kills the task invisibly: + the consumer sees a bridge that is merely disconnected and waits for a + reconnect that will never be attempted. This is the channel that exists + so it cannot. + + Fires at most once per bridge, from the event loop, with the same error + `fatal_error` then holds. A subscriber that raises is logged and + otherwise ignored -- there is nothing left to protect at that point, but + an exception escaping here would still swallow the notification for a + second subscriber added later. + """ + self._fatal_error_callback = callback + + @property + def fatal_error(self) -> SpanPanelError | None: + """The failure this bridge stopped for, or None while it is still trying. + + Readable so a caller that registered no callback still cannot mistake a + dead bridge for a disconnected one. `SpanMqttClient.ping()` and + `get_snapshot()` consult it for exactly that reason. + """ + return self._fatal_error + + def _enter_terminal_state(self, error: SpanPanelError) -> None: + """Stop reconnecting, record why, and tell whoever asked to be told.""" + self._fatal_error = error + # Stops `_reconnect_loop`'s while-condition and prevents `_on_disconnect` + # from starting a replacement loop. + self._should_reconnect = False + _LOGGER.error("MQTT bridge to %s:%s has stopped permanently: %s", self._host, self._port, error) + if self._fatal_error_callback is not None: + try: + self._fatal_error_callback(error) + except Exception: # pylint: disable=broad-exception-caught + _LOGGER.warning("Fatal-error callback raised", exc_info=True) + def set_pre_rebuild_callback(self, callback: Callable[[], None]) -> None: """Set callback invoked just before the bridge rebuilds its paho client. @@ -138,6 +176,106 @@ def set_pre_rebuild_callback(self, callback: Callable[[], None]) -> None: """ self._pre_rebuild_callback = callback + async def _trust_anchor_pem(self) -> str: + """The CA this connection is verified against. + + **With `ca_pem` supplied this makes no network call, on any path.** That + is the whole of the pin, and it is the single most important line in this + class. Before it, `_rebuild_client` refetched the CA on every reconnect + and rebuilt the context from whatever came back — so a panel presenting a + certificate from a *different* CA was silently re-anchored to it on the + next reconnect. Automatic recovery from CA rotation and automatic + acceptance of an interception are the same code path; there is no way to + keep one without the other, and this chooses to keep neither. + + Without `ca_pem` the old behaviour stands, because requiring a pin would + break every install on upgrade. It warns once per bridge rather than per + connect: the fetch happens on every reconnect, and a warning per reconnect + during a day-long outage is a log the user stops reading. + + Raises: + SpanPanelConnectionError: unpinned, and the panel could not be reached. + SpanPanelTimeoutError: unpinned, and the CA request timed out. + SpanPanelAPIError: unpinned, and the panel did not answer with a PEM. + """ + pinned = self._ca_pem + if pinned is not None: + return pinned + if not self._warned_unpinned: + self._warned_unpinned = True + _LOGGER.warning( + "MQTT trust anchor for %s was obtained unauthenticated: no ca_pem was configured, " + "so the CA is fetched over plaintext HTTP from the panel on every connect and " + "whatever answers is trusted. Pin the CA by setting MqttClientConfig.ca_pem.", + self._panel_host, + ) + return await download_ca_cert(self._panel_host, port=self._panel_http_port) + + async def _diagnose_ca_change(self) -> SpanPanelCAChangedError | None: + """Decide whether a certificate-verification failure means the CA rotated. + + It usually does not, and the failure carries no evidence either way. A + valid pinned CA still produces `SSLCertVerificationError` when the leaf + has expired — a panel whose clock reset after a power outage, which for + an electrical panel is not a corner case — or when the hostname no longer + matches after the panel's address changed. And `ssl` exposes no peer + chain on a verification failure, so the certificate that was actually + offered cannot be read from the exception at all. + + So the observed fingerprint has to come from somewhere else: a separate, + unauthenticated fetch of the panel's advertised CA. That fetch is + **diagnostic only and is never used to re-anchor** — it is exactly the + request an attacker would answer, and treating its result as a new trust + anchor is the re-anchoring this class exists to stop. + + Three outcomes, and only one of them escalates: + + - Fingerprint matches the pin — the CA did not change. Some other TLS + problem; the caller keeps retrying. + - Fingerprint differs — the panel is advertising a different anchor. + Returns the error to raise. + - The fetch failed, or returned something that is not a certificate — + returns None. **Never escalate on missing evidence.** A panel that is + reachable on 8883 and not on its HTTP port is a panel mid-reboot, and + declaring its CA changed on that basis would convert a transient into a + permanent outage. + + Returns None immediately when nothing is pinned: with no anchor recorded + there is nothing to compare against, and the unpinned path already + re-anchors by design. + """ + pinned = self._ca_pem + if pinned is None: + return None + try: + # Plaintext deliberately: if the CA really has rotated, a fetch + # verified against the old pin would fail and tell us nothing. + advertised = await download_ca_cert(self._panel_host, port=self._panel_http_port) + except (OSError, SpanPanelError) as exc: + _LOGGER.warning( + "TLS verification failed for %s and the panel's CA could not be re-read to say why (%s). " + "Treating this as transient and continuing to retry.", + self._panel_host, + exc, + ) + return None + try: + expected = ca_fingerprint(pinned) + observed = ca_fingerprint(advertised) + except SpanPanelValidationError as exc: + _LOGGER.warning("Could not fingerprint a CA certificate while diagnosing a TLS failure: %s", exc) + return None + if expected == observed: + _LOGGER.warning( + "TLS verification failed for %s, but the panel still advertises the pinned CA " + "(SHA-256 %s). An expired certificate or a changed hostname would both look like " + "this. Continuing to retry.", + self._panel_host, + expected, + ) + return None + return SpanPanelCAChangedError(expected, observed) + def _make_paho_client(self, ssl_context: ssl.SSLContext | None) -> AsyncMQTTClient: """Build and wire a fresh paho client. @@ -159,6 +297,7 @@ def _make_paho_client(self, ssl_context: ssl.SSLContext | None) -> AsyncMQTTClie client.on_connect = self._on_connect client.on_disconnect = self._on_disconnect client.on_message = self._on_message + client.on_publish = self._on_publish if ssl_context is not None: client.tls_set_context(ssl_context) return client @@ -166,12 +305,18 @@ def _make_paho_client(self, ssl_context: ssl.SSLContext | None) -> AsyncMQTTClie async def connect(self) -> None: """Connect to the MQTT broker. - Fetches the CA certificate from the panel, configures TLS, - connects via executor (blocking I/O), and waits for CONNACK. + Resolves the TLS trust anchor — the configured pin, or a fetch from the + panel when there is none — configures TLS, connects via executor + (blocking I/O), and waits for CONNACK. Raises: SpanPanelConnectionError: Cannot connect to broker. SpanPanelTimeoutError: Connection timed out. + SpanPanelCAChangedError: The panel is pinned and now advertises a + different CA. If the CA rotated while the consumer was down, the + pinned handshake fails here rather than in the reconnect loop, + and wrapping it as a connection error would leave the consumer + retrying setup forever with nothing to act on. """ if self._loop is None: self._loop = asyncio.get_running_loop() @@ -179,19 +324,20 @@ async def connect(self) -> None: self._connect_event = asyncio.Event() self._should_reconnect = True - # Fetch CA cert from panel for TLS - _LOGGER.debug("BRIDGE: Fetching CA cert from %s (use_tls=%s)", self._panel_host, self._use_tls) + _LOGGER.debug( + "BRIDGE: Resolving CA for %s (use_tls=%s, pinned=%s)", self._panel_host, self._use_tls, self._ca_pem is not None + ) ssl_context: ssl.SSLContext | None = None if self._use_tls: try: - ca_pem = await download_ca_cert(self._panel_host, port=self._panel_http_port) + ca_pem = await self._trust_anchor_pem() except (OSError, SpanPanelConnectionError, SpanPanelTimeoutError) as exc: raise SpanPanelConnectionError(f"Failed to fetch CA certificate from {self._panel_host}") from exc # Build the SSLContext from PEM data in memory — no temp file. # A malformed PEM raises ssl.SSLError or ValueError; wrap both # so callers only see the documented SpanPanelConnectionError. try: - ssl_context = _build_ssl_context(ca_pem) + ssl_context = build_panel_ssl_context(ca_pem) except (ssl.SSLError, ValueError) as exc: raise SpanPanelConnectionError(f"Failed to build SSL context for {self._panel_host}") from exc @@ -215,6 +361,18 @@ def _blocking_connect() -> None: _LOGGER.debug("BRIDGE: Running connect in executor to %s:%s", self._host, self._port) try: await self._loop.run_in_executor(None, _blocking_connect) + except ssl.SSLCertVerificationError as exc: + # The pinned handshake failed on the very first attempt, which is + # what a CA rotated while the consumer was shut down looks like. + # Wrapped as a connection error this became a setup-retry loop + # with nothing for the user to act on, forever. `_diagnose_ca_change` + # is what distinguishes it from an expired leaf or a moved host, + # and returns None for both of those so they keep their old + # retryable shape. + fatal = await self._diagnose_ca_change() + if fatal is not None: + raise fatal from exc + raise SpanPanelConnectionError(f"Cannot connect to MQTT broker at {self._host}:{self._port}: {exc}") from exc except Exception as exc: # pylint: disable=broad-exception-caught # paho raises OSError for TCP failures and transport-specific # errors (e.g. WebsocketConnectionError) that do not inherit @@ -254,6 +412,8 @@ async def disconnect(self) -> None: self._misc_timer.cancel() self._misc_timer = None + self._resolve_pending_publishes(False, "bridge disconnected") + client = self._client if client is not None: client.disconnect() @@ -266,10 +426,120 @@ def subscribe(self, topic: str, qos: int = 0) -> None: if self._client is not None: self._client.subscribe(topic, qos=qos) - def publish(self, topic: str, payload: str, qos: int = 1) -> None: - """Publish a message. Must be called after connect().""" - if self._client is not None: - self._client.publish(topic, payload=payload, qos=qos) + def publish(self, topic: str, payload: str) -> asyncio.Future[bool] | None: + """Hand one QoS-1 control message to paho, or refuse to hand it over. + + Returns ``None`` when the message was **not** handed over -- no client, + or not connected. That is the only condition under which a caller may + say the command failed, and the reason the check is here rather than on + paho's return code afterwards. + + **The gate is only as fresh as paho's disconnect detection**, which is + socket close or the keepalive at ``MQTT_KEEPALIVE_S``. A broker that + stops answering without closing its socket -- a silent partition, a + dropped route -- leaves ``is_connected()`` true for up to a keepalive + interval and a half, and a publish in that window is handed over and + queued after all, then re-sent as DUP on the next reconnect of this same + client. Nothing here lies as a result: such a caller is told + ``UNCONFIRMED``, which promises nothing about delivery either way, and + ``FAILED``'s promise is unaffected because this path never produces it. + What is bounded is the refusal, not the queueing: it catches every + disconnect the transport knows about, and knows about a silent one only + once the keepalive expires. Closing that window means rebuilding rather + than reconnecting whenever un-PUBACKed publishes are in flight, which is + a larger change than this one. + + **paho queues a QoS-1 publish across a disconnect.** On + ``MQTT_ERR_NO_CONN`` it keeps the message in ``_out_messages`` with + ``state = mqtt_ms_publish`` -- its own comment reads "remove from + inflight messages so it will be send after a connection is made" -- and + the reconnect path reuses the same client object. So a relay command + published while the broker is down is not discarded: it fires whenever + the broker returns, which on a firmware upgrade is minutes later and + unannounced. Reading paho's return code and calling it a failure would + tell a user their breaker command failed while the command was still + pending delivery, and a user told that acts on it. The message must not + reach paho at all. + + Otherwise returns a future that resolves: + + - ``True`` when the broker PUBACKs, and + - ``False`` when a transport rebuild discards paho's outbound queue + before that happens. + + ``False`` is genuinely ambiguous and the caller must treat it as such: a + rebuilt client drops the message from this side, but the original may + already have reached the broker and been acted on. It resolves the + future rather than leaving it pending so nothing waits on a message no + longer in flight -- it does not license reporting a failure. + """ + client = self._client + if client is None or not self._connected: + return None + + info = client.publish(topic, payload=payload, qos=1) + + loop = self._loop + if loop is None: + loop = asyncio.get_running_loop() + self._loop = loop + acknowledged: asyncio.Future[bool] = loop.create_future() + # Registered after `publish()` returns, which is safe only because paho's + # callbacks here are driven by the event loop's reader callback: no + # `loop_read` can run between the line above and this one, so the PUBACK + # for this mid cannot arrive before there is a future to resolve. It + # would be a real race under paho's own threaded loop. + self._pending_publishes[info.mid] = acknowledged + # Cleans up when the caller's deadline cancels the future, which is the + # ordinary end for a message the broker never answers. Without it the + # entry outlives every waiter and the map grows for the life of the + # bridge. Guarded by identity in `_forget_publish` because paho's message + # ids wrap. + acknowledged.add_done_callback(partial(self._forget_publish, info.mid)) + return acknowledged + + def _forget_publish(self, mid: int, future: asyncio.Future[bool]) -> None: + """Drop a settled publish, but only if it is still the one we recorded.""" + if self._pending_publishes.get(mid) is future: + del self._pending_publishes[mid] + + def _resolve_pending_publishes(self, acknowledged: bool, reason: str) -> None: + """Settle every publish still awaiting a PUBACK. + + Called when the outbound queue stops existing -- a rebuilt paho client, + or teardown. Without this a caller waits out its whole deadline for an + acknowledgement that is now impossible, and an audit trail shows a + command in limbo with no terminal state. + + Settling the future is what *lets* a waiter stop early; it does not by + itself stop one. A caller waiting on something else -- the property + transition, in this client's case -- has to watch this future too, and + `_discard_verification` is where that is wired up. The distinction is + worth keeping straight: this method's job ends at making the evidence + available. + """ + if not self._pending_publishes: + return + _LOGGER.debug( + "Settling %d in-flight publish(es) as acknowledged=%s: %s", len(self._pending_publishes), acknowledged, reason + ) + for future in list(self._pending_publishes.values()): + if not future.done(): + future.set_result(acknowledged) + self._pending_publishes.clear() + + def _on_publish( + self, + _client: paho.Client, + _userdata: object, + mid: int, + _reason_code: ReasonCode, + _properties: Properties | None, + ) -> None: + """Handle PUBACK — resolve whoever is waiting on this message id.""" + future = self._pending_publishes.get(mid) + if future is not None and not future.done(): + future.set_result(True) # -- Socket callbacks (event-loop-driven I/O) --------------------------- @@ -453,13 +723,17 @@ async def _rebuild_client(self) -> bool: old_client = self._client - # Fetch fresh CA (TLS bridges only). Failure is non-fatal — old - # client stays in place and the loop retries on the next tick. + # Resolve the trust anchor (TLS bridges only). With `ca_pem` configured + # this is a pure read: a rebuild must not be a route back onto an anchor + # the panel is currently offering, which is precisely what re-fetching + # here used to make it. Unpinned, the fetch is the recovery it always + # was, and a failure is non-fatal — the old client stays in place and the + # loop retries on the next tick. ssl_context: ssl.SSLContext | None = None if self._use_tls: try: - ca_pem = await download_ca_cert(self._panel_host, port=self._panel_http_port) - ssl_context = _build_ssl_context(ca_pem) + ca_pem = await self._trust_anchor_pem() + ssl_context = build_panel_ssl_context(ca_pem) except ( OSError, SpanPanelConnectionError, @@ -471,6 +745,12 @@ async def _rebuild_client(self) -> bool: _LOGGER.warning("Client rebuild — CA fetch failed: %s", exc) return False + # The new paho client has an empty outbound queue, so anything the old + # one was still holding is gone. Settle those waiters now rather than + # letting each burn its full deadline on an acknowledgement that can no + # longer arrive. See `publish` for why `False` is not a failure. + self._resolve_pending_publishes(False, "transport rebuild discarded the outbound queue") + # Fire pre-rebuild hook before we touch any state. SpanMqttClient # uses this to discard its stale Homie accumulator so retained # messages on the new subscription start from a clean slate. @@ -547,11 +827,17 @@ async def _reconnect_loop(self) -> None: Every MQTT_FULL_REBUILD_AFTER_FAILURES consecutive non-SSL failures (or on any ssl.SSLError), rebuild the paho client from scratch — - re-fetching the panel CA and resetting any stale in-memory state. - Mirrors what a manual integration reload does without going through - HA's config_entry teardown. The counter resets after every rebuild - attempt (success or fail) and on `_connected == True`, so the - cadence holds throughout extended outages. + resetting any stale in-memory state, and re-fetching the panel CA when + no pin is configured. Mirrors what a manual integration reload does + without going through HA's config_entry teardown. The counter resets + after every rebuild attempt (success or fail) and on + `_connected == True`, so the cadence holds throughout extended outages. + + The loop ends on exactly one thing other than `disconnect()`: a + confirmed CA change, which `_enter_terminal_state` records and announces + before the `while` condition drops it out. Every other failure is + retried, because every other failure is one the panel can recover from + on its own. """ delay = MQTT_RECONNECT_MIN_DELAY_S failures_since_rebuild_attempt = 0 @@ -564,10 +850,39 @@ async def _reconnect_loop(self) -> None: self._client.on_socket_open = self._on_socket_open_sync self._client.on_socket_register_write = self._on_socket_register_write_sync await self._loop.run_in_executor(None, self._client.reconnect) + except ssl.SSLCertVerificationError as exc: + # Certificate verification, specifically -- caught ahead of + # `ssl.SSLError` because it is a subclass, and separated from + # it because they mean different things. This one is the + # *only* shape a CA change can take. `SSLEOFError`, which the + # broad clause below still handles, is what a broker restart + # looks like mid-handshake: the ordinary shape of a firmware + # upgrade, and reading "CA changed, stop forever" into it + # would turn a four-minute reboot into a permanent outage. + if self._ca_pem is None: + # Unpinned: unchanged from 3.0.1. The refetch inside the + # rebuild is the recovery, because with nothing pinned + # the anchor is by definition whatever the panel last + # served. + _LOGGER.warning("Reconnect TLS verification failure (%s), rebuilding client", exc) + await self._rebuild_client() + failures_since_rebuild_attempt = 0 + else: + fatal = await self._diagnose_ca_change() + if fatal is not None: + self._enter_terminal_state(fatal) + return + # Not the CA. An expired leaf or a moved host, both of + # which the panel or the network can still fix, so this + # is an ordinary failure. No immediate rebuild: with a + # pin, a rebuild changes nothing about trust and only + # discards whatever paho was still holding. + failures_since_rebuild_attempt += 1 + _LOGGER.warning("Reconnect TLS verification failed (%s), retrying in %ss", exc, delay) except ssl.SSLError as exc: - # TLS verification failure — most likely a CA rotation - # (firmware upgrade). ssl.SSLError must be caught before - # OSError because it is an OSError subclass. + # Every other TLS failure. Kept on the rebuild path it has + # always been on: a fresh paho client is a reasonable answer + # to a handshake that went wrong for a reason we cannot name. _LOGGER.warning("Reconnect TLS failure (%s), rebuilding client", exc) await self._rebuild_client() failures_since_rebuild_attempt = 0 diff --git a/src/span_panel_api/mqtt/control.py b/src/span_panel_api/mqtt/control.py new file mode 100644 index 0000000..e1ba004 --- /dev/null +++ b/src/span_panel_api/mqtt/control.py @@ -0,0 +1,166 @@ +"""What happened to a control command, said precisely enough to act on. + +Every setter used to return `None`, which meant the caller could not tell a +breaker that opened from one whose command was dropped on the floor. These types +are the vocabulary for saying which. + +The distinctions here are not decoration. A consumer renders each of them +differently to a person, and two of them are the difference between "your panel +did the thing" and "your panel may do the thing in four minutes". +""" + +from __future__ import annotations + +from dataclasses import dataclass +from enum import StrEnum +from typing import Protocol, runtime_checkable + + +class PublishState(StrEnum): + """How far a control command got. + + Ordered from most to least evidence. Only `FAILED` is a promise about the + future; the rest describe what was observed by a deadline and say nothing + about what happens after it. + """ + + CONFIRMED = "confirmed" + """The property reported the requested value on its own topic. + + The strongest statement available, and still not proof that *this* write + caused it -- see the correlation caveat on the setters. + """ + + ACCEPTED = "accepted" + """The broker acknowledged the message (QoS-1 PUBACK); no transition seen. + + A real and separate diagnosis, kept apart from `UNCONFIRMED` because paho + already gives it away for free and folding the two together discards + information the library is holding anyway. "The broker took it and the panel + did not act" and "nothing ever acknowledged it" send an investigation in + different directions. + """ + + UNCONFIRMED = "unconfirmed" + """Handed over, and nothing came back within the deadline. + + **Not an error, and must not raise.** It is the expected result of a write + whose value was already current, and it is indistinguishable from a silent + policy rejection by the panel until SPAN ships a reason code. A consumer + should say what it means -- the panel accepted the command and never reported + a change, most often because there was no change to report -- rather than + dressing it as a failure. + """ + + FAILED = "failed" + """Never handed to the broker. Will not be delivered. + + The one state that is a promise, and the reason the transport refuses to + publish while disconnected rather than checking paho's return code + afterwards. paho *queues* a QoS-1 publish across a disconnect and sends it + when the connection returns, so a command reported failed on a return code + could still fire minutes later against a panel nobody is watching. A user + told "failed" acts on that. `FAILED` is only ever produced by a refusal that + happens before paho sees the message at all. + """ + + +@dataclass(frozen=True, slots=True) +class PublishOutcome: + """The result of one control command. + + `detail` is free text for a human reading a log or an audit row -- which + refusal, which deadline. **It never carries a credential**, and nothing + should parse it; `state` is the machine-readable half. + """ + + state: PublishState + topic: str + value: str + no_op: bool = False + detail: str | None = None + + +@dataclass(frozen=True, slots=True) +class ControlDeadlines: + """How long each control waits for the panel to report the change. + + Per property rather than one number, because the thing being waited on + differs in kind. A relay is a physical actuation with a contactor at the end + of it; the rest are values the panel writes down. Sized so the common case + returns as soon as the transition arrives -- the deadline is only reached + when nothing does. + + Injectable rather than constant so tests do not each pay a real deadline to + assert a refusal. + """ + + relay: float = 5.0 + priority: float = 2.0 + dominant_power_source: float = 2.0 + evse_charge_limit: float = 2.0 + adopted_property: float = 2.0 + + +@dataclass(frozen=True, slots=True) +class ControlCommand: + """One control command, described in the terms a policy or an audit needs. + + Built from the `ControlTarget` the adapter produced plus the translated + payload, so the identifying fields are the ones actually on the wire rather + than the caller's arguments -- a dominant-power-source request of `BATTERY` + arrives here as the `OFF_GRID` that will be published under v1.0. + """ + + device_id: str + node_id: str + property_id: str + value: str + topic: str + + +@runtime_checkable +class ControlInterceptor(Protocol): + """One veto-and-observe point covering every control command. + + **This is a boundary against callers of this library, and nothing more.** + It does not constrain anything holding the broker credential: a process with + the credential publishes to the panel's broker directly and never reaches + this code, and neither does another copy of this library in another process. + Presenting it as a security boundary around the *panel* would be the most + damaging thing this feature could do, because a user would stop looking for + the real one. + + What it is good for is the boundary it can actually hold: everything + arriving through this client. A consumer with a notion of who is asking -- + Home Assistant has one, per service call -- can refuse here, and can record + every command in one place rather than in five setters that will drift. + + One interceptor at a time, replaceable. Several would raise ordering + questions with no principled answer. + """ + + async def before_publish(self, command: ControlCommand) -> None: + """Called before anything is published. Raise to veto. + + **The exception propagates to the caller unchanged.** The library does + not translate it: the interceptor raised it, and owns its type and its + message. That is load-bearing for a consumer that raises a + framework-specific error carrying a translated message and needs it to + reach the user intact. + + A veto still produces an `after_publish` with `PublishState.FAILED`, so + an audit built on this cannot silently omit the refusals -- which would + make it worse than no audit. + """ + + async def after_publish(self, command: ControlCommand, outcome: PublishOutcome) -> None: + """Called with the result of every command, refusals included. + + **Fired as a task and not awaited on the control path.** A sink that + merely hangs -- a slow event bus, a blocked writer -- would otherwise + stall every control call in the process. The consequences are the price: + ordering across commands is not guaranteed, so an audit consumer must + not assume it, and an exception raised here is logged and discarded + rather than reaching the caller who issued the command. + """ diff --git a/src/span_panel_api/mqtt/models.py b/src/span_panel_api/mqtt/models.py index 40bd40d..aa36187 100644 --- a/src/span_panel_api/mqtt/models.py +++ b/src/span_panel_api/mqtt/models.py @@ -14,8 +14,21 @@ class MqttClientConfig: """MQTT broker connection parameters from v2 auth response. - CA certificate is not stored here — AsyncMqttBridge fetches it - fresh from GET /api/v2/certificate/ca on every connect/reconnect. + ``ca_pem`` is the trust anchor for the broker connection, and supplying it is + what makes that connection pinned. With it set, ``AsyncMqttBridge`` builds + its SSL context from this value and makes no CA request on any path -- + neither at connect nor at rebuild. + + Leaving it ``None`` keeps 3.0.1's behaviour: the bridge fetches the CA from + ``GET /api/v2/certificate/ca`` on every connect and every rebuild, + unauthenticated and over plaintext HTTP, and trusts whatever comes back. It + is permitted because requiring a pin would break every existing install on + upgrade, and it logs a warning once per bridge because it is a real downgrade + from what this field offers, not a neutral default. + + Separate from ``create_span_client``'s ``ssl_context``, which anchors the + *REST* calls, because the two secure different transports. They are the same + panel CA; a caller holding the PEM supplies both from it. """ broker_host: str @@ -26,6 +39,9 @@ class MqttClientConfig: wss_port: int = MQTT_DEFAULT_WSS_PORT transport: MqttTransport = "tcp" use_tls: bool = True + # Last, so the positional signature every existing caller uses is unchanged, + # and beside `use_tls` because it is the other half of the same decision. + ca_pem: str | None = None @property def effective_port(self) -> int: diff --git a/src/span_panel_api/protocol.py b/src/span_panel_api/protocol.py index e92adaf..b9ea634 100644 --- a/src/span_panel_api/protocol.py +++ b/src/span_panel_api/protocol.py @@ -12,7 +12,9 @@ from typing import TYPE_CHECKING, Protocol, runtime_checkable if TYPE_CHECKING: - from .models import FieldMetadata, SpanPanelSnapshot, V2HomieSchema + from .exceptions import SpanPanelError + from .models import ControlTarget, FieldMetadata, SpanPanelSnapshot, V2HomieSchema + from .mqtt.control import ControlInterceptor, PublishOutcome class PanelCapability(Flag): @@ -48,21 +50,46 @@ async def get_snapshot(self) -> SpanPanelSnapshot: ... def register_connection_callback(self, callback: Callable[[bool], None]) -> Callable[[], None]: ... + def register_fatal_error_callback(self, callback: Callable[[SpanPanelError], None]) -> Callable[[], None]: + """Subscribe to the transport stopping for good. + + Declared here rather than only on the MQTT client because the consumer + depends on it and this module's rule is that the consumer codes against + protocols, never against transport-specific classes. + + Distinct from `register_connection_callback` because the two say + different things and the difference is the whole point. "Disconnected" is + what an ordinary outage looks like and a consumer is right to wait + through it; this fires only for a failure no amount of waiting fixes, + and the consumer is expected to surface it to a person. + """ + @runtime_checkable class CircuitControlProtocol(Protocol): - """Control protocol for relay and priority changes.""" + """Control protocol for relay and priority changes. - async def set_circuit_relay(self, circuit_id: str, state: str) -> None: ... + Every setter across the four control protocols returns a `PublishOutcome` + rather than `None`. **Additive for callers, breaking for implementers**: an + existing call site that ignores the return value is unaffected, but a class + type-checked against one of these protocols with `-> None` stops conforming. + Test fakes and simulators are exactly that. - async def set_circuit_priority(self, circuit_id: str, priority: str) -> None: ... + The change exists because `None` could not distinguish a breaker that opened + from a command that was never handed to the broker, and the transport had + three separate paths that returned `None` having published nothing. + """ + + async def set_circuit_relay(self, circuit_id: str, state: str) -> PublishOutcome: ... + + async def set_circuit_priority(self, circuit_id: str, priority: str) -> PublishOutcome: ... @runtime_checkable class PanelControlProtocol(Protocol): """Control protocol for panel-level settable properties.""" - async def set_dominant_power_source(self, value: str) -> None: ... + async def set_dominant_power_source(self, value: str) -> PublishOutcome: ... @runtime_checkable @@ -75,7 +102,7 @@ class EvseControlProtocol(Protocol): the control, exactly as it does for circuit and panel control. """ - async def set_evse_charge_limit(self, node_id: str, amps: int) -> None: ... + async def set_evse_charge_limit(self, node_id: str, amps: int) -> PublishOutcome: ... @runtime_checkable @@ -95,7 +122,26 @@ class AdoptedControlProtocol(Protocol): what stops this becoming a generic write around the curated setters. """ - async def set_adopted_property(self, device_id: str, node_id: str, property_id: str, value: str) -> None: ... + async def set_adopted_property(self, device_id: str, node_id: str, property_id: str, value: str) -> PublishOutcome: ... + + +@runtime_checkable +class ControlInterceptionProtocol(Protocol): + """Transport that can be given one veto-and-observe point for every command. + + A protocol of its own rather than a member added to the four control + protocols or to `StreamingCapableProtocol`. Adding it to the control + protocols would break every implementer of them a second time in one + release, and streaming has nothing to do with control -- a transport could + reasonably offer one and not the other. + + Declared here at all because the consumer's authorisation gate is built on + it, and this module's rule is that the consumer codes against protocols, + never against transport-specific classes. + """ + + def set_control_interceptor(self, interceptor: ControlInterceptor | None) -> None: + """Install the interceptor, or `None` to remove it. One at a time.""" @runtime_checkable @@ -179,14 +225,27 @@ def circuit_nodes_missing_names(self) -> list[str]: ... def find_node_by_type(self, type_str: str) -> str | None: ... - def set_circuit_relay_topic(self, circuit_id: str) -> str: ... + def set_circuit_relay_target(self, circuit_id: str) -> ControlTarget: + """Where a relay command goes, and the property that reports it. + + Renamed from `set_circuit_relay_topic`, which returned a bare string. + The rename is deliberate rather than a return-type change under the old + name: an adapter built against the old contract would still carry the + old name, pass discovery on presence, and fail deep inside a setter with + an `AttributeError` on a `str`. Under a new name it is rejected at + discovery, where the remedy -- upgrade both packages together -- can + still be named. That is also why `ADAPTER_CONTRACT_VERSION` does not + move: the change is additive plus a removal, not a redefinition. + """ - def set_circuit_priority_topic(self, circuit_id: str) -> str: ... + def set_circuit_priority_target(self, circuit_id: str) -> ControlTarget: + """Where a shed-priority command goes, and the property that reports it.""" - def set_dominant_power_source_topic(self) -> str | None: ... + def set_dominant_power_source_target(self) -> ControlTarget | None: + """Where a dominant-power-source command goes, or None if the panel has no such control.""" - def set_evse_charge_limit_topic(self, node_id: str) -> str | None: - """The topic that writes one charger's charge-current limit, or None. + def set_evse_charge_limit_target(self, node_id: str) -> ControlTarget | None: + """The target that writes one charger's charge-current limit, or None. `node_id` is the key the snapshot's `evse` map uses, so a caller needs nothing but the snapshot it already has. Returning None means this @@ -221,4 +280,16 @@ def dominant_power_source_payload(self, value: str) -> str | None: panel will reject. """ - def register_property_callback(self, callback: Callable[[str, str, str, str | None], None]) -> Callable[[], None]: ... + def register_property_callback(self, callback: Callable[[str, str, str, str | None], None]) -> Callable[[], None]: + """Subscribe to per-property updates; returns an unregister callable. + + **The callback receives `(device_id, node_id, property_id, value)`** -- + the same triple `ControlTarget` carries, spelled the same way, because + write-then-verify matches one against the other. `value` is `None` only + where the adapter can report a property with no value; a consumer that + needs the previous value keeps it itself. + + Under the flat schema every property belongs to the one device, so + `device_id` is the panel serial rather than being omitted. Filling it in + is what lets a consumer treat both schemas' streams as one shape. + """ diff --git a/tests/conftest.py b/tests/conftest.py index 8d9d026..ae05185 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -3,6 +3,7 @@ from __future__ import annotations import asyncio +import itertools import json import os from pathlib import Path @@ -16,6 +17,7 @@ import span_panel_api._http as _http_mod from span_panel_api.models import V2HomieSchema +from span_panel_api.mqtt.control import ControlDeadlines from span_panel_api_schema_0.const import TOPIC_PREFIX, TYPE_CORE _DOTENV = Path(__file__).parent.parent / ".env" @@ -146,6 +148,30 @@ def _connect( ) return 0 + mids = itertools.count(1) + + def _publish(topic: str, payload: object = None, qos: int = 0, **_kwargs: object) -> MagicMock: + """Simulate a QoS-1 publish, PUBACK included. + + The acknowledgement is scheduled rather than fired inline for two + reasons, and both are how a real broker behaves: a PUBACK is a round + trip, and `AsyncMqttBridge.publish` registers its waiter after + `client.publish()` returns, so an inline callback would arrive before + there was anything to resolve. Without this every setter test waits out + a real deadline for an acknowledgement that never comes. + """ + info = MagicMock(rc=0, mid=next(mids)) + if qos > 0: + loop.call_soon( + mock_client.on_publish, + mock_client, + None, + info.mid, + ReasonCode(packetType=4, aName="Success"), + None, + ) + return info + def _reconnect() -> int: """Simulate paho reconnect.""" mock_client.on_socket_open(mock_client, None, fake_sock) @@ -163,7 +189,7 @@ def _reconnect() -> int: with ( patch("span_panel_api.mqtt.connection.AsyncMQTTClient") as cls, patch("span_panel_api.mqtt.connection.download_ca_cert", return_value="FAKE-PEM"), - patch("span_panel_api.mqtt.connection._build_ssl_context", return_value=MagicMock()), + patch("span_panel_api.mqtt.connection.build_panel_ssl_context", return_value=MagicMock()), patch("span_panel_api.mqtt.client.get_homie_schema", return_value=MOCK_SCHEMA), patch("span_panel_api.factory.get_homie_schema", return_value=MOCK_SCHEMA), ): @@ -171,10 +197,43 @@ def _reconnect() -> int: mock_client.connect.side_effect = _connect mock_client.reconnect.side_effect = _reconnect mock_client.subscribe.return_value = (0, 1) - mock_client.publish.return_value = MagicMock(rc=0, mid=1) + mock_client.publish.side_effect = _publish mock_client.disconnect.return_value = 0 mock_client.loop_read.return_value = 0 mock_client.loop_write.return_value = 0 mock_client.loop_misc.return_value = paho.MQTT_ERR_SUCCESS yield mock_client + + +def acking_bridge() -> MagicMock: + """A bridge stub whose publishes are acknowledged immediately. + + `AsyncMqttBridge.publish` hands back a future that resolves when the broker + PUBACKs, and `SpanMqttClient._publish_control` awaits it to decide between + `ACCEPTED` and `UNCONFIRMED`. A bare `MagicMock` answers with a `Mock`, which + cannot be awaited, so every control test needs a real future -- and would + otherwise each grow its own. + """ + bridge = MagicMock() + + def _publish(_topic: str, _payload: str) -> asyncio.Future[bool]: + acknowledged: asyncio.Future[bool] = asyncio.get_running_loop().create_future() + acknowledged.set_result(True) + return acknowledged + + bridge.publish.side_effect = _publish + return bridge + + +#: Control deadlines short enough that a test asserting *where* a command went +#: does not also wait out the real 2--5 second window in which the panel would +#: have reported it back. The production defaults exist for a physical +#: contactor; a mock panel that never echoes anything would burn one per setter. +FAST_CONTROL_DEADLINES = ControlDeadlines( + relay=0.05, + priority=0.05, + dominant_power_source=0.05, + evse_charge_limit=0.05, + adopted_property=0.05, +) diff --git a/tests/test_adapters_discovery.py b/tests/test_adapters_discovery.py index 8138b6d..934068a 100644 --- a/tests/test_adapters_discovery.py +++ b/tests/test_adapters_discovery.py @@ -363,6 +363,29 @@ def test_an_adapter_predating_contract_versioning_is_rejected_by_age_not_by_shap assert "predates contract versioning" in str(exc.value) +def test_a_missing_member_names_the_remedy_not_just_the_diagnosis() -> None: + """A member-presence rejection is what a mismatched *pair* of packages looks + like, in both directions: a new bootstrap misses the member an old adapter + has not grown, an old bootstrap misses the one a new adapter has renamed. + Listing the absent names reads as a fault in the adapter and sends someone + looking for a bug. The remedy is the same either way, so the message says it + rather than leaving it to be inferred.""" + members = _conforming_members() + del members["set_circuit_relay_target"] + + _reset_adapter_cache() + with patch( + "span_panel_api.adapters.entry_points", + return_value=[_FakeEntryPoint("schema_9", type("HalfBuilt", (), members))], + ): + with pytest.raises(SpanPanelAdapterIncompatibleError) as exc: + resolve_adapter("schema_9", "test") + + message = str(exc.value) + assert "set_circuit_relay_target" in message, "the diagnosis must survive" + assert "together" in message, "and the remedy must be stated, not inferred" + + def test_a_rejected_adapter_is_reported_as_unusable_not_as_missing() -> None: """Absent and rejected are opposite remedies. Reporting a stale adapter as missing sends someone to install a package they already have.""" diff --git a/tests/test_adopted_control.py b/tests/test_adopted_control.py index caccd4e..0daba54 100644 --- a/tests/test_adopted_control.py +++ b/tests/test_adopted_control.py @@ -13,6 +13,8 @@ from unittest.mock import MagicMock import pytest + +from conftest import FAST_CONTROL_DEADLINES, acking_bridge from span_panel_api.exceptions import SpanPanelServerError from span_panel_api.models import AdoptedDevice, AdoptedProperty from span_panel_api.mqtt import MqttClientConfig @@ -37,14 +39,19 @@ def _client(*properties: AdoptedProperty) -> tuple[SpanMqttClient, MagicMock]: """A client whose adapter reports one adopted device carrying `properties`.""" config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) adapter = MagicMock() adapter.build_snapshot.return_value = MagicMock( adopted_devices=(AdoptedDevice(device_id=DEVICE, device_type="energy.ebus.device.generator", properties=properties),) ) client._adapter = adapter - bridge = MagicMock() + bridge = acking_bridge() client._bridge = bridge return client, bridge @@ -62,7 +69,7 @@ async def test_a_settable_adopted_property_publishes_to_its_own_topic() -> None: await client.set_adopted_property(DEVICE, "generator", "mode", "OFF") - bridge.publish.assert_called_once_with(f"ebus/5/{DEVICE}/generator/mode/set", "OFF", qos=1) + bridge.publish.assert_called_once_with(f"ebus/5/{DEVICE}/generator/mode/set", "OFF") @pytest.mark.asyncio @@ -128,7 +135,12 @@ def _two_generators() -> tuple[SpanMqttClient, MagicMock]: kind is exactly when a device id stops being decoration. """ config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) def control(device_id: str) -> AdoptedProperty: return AdoptedProperty( @@ -153,7 +165,7 @@ def control(device_id: str) -> AdoptedProperty: ) ) client._adapter = adapter - bridge = MagicMock() + bridge = acking_bridge() client._bridge = bridge return client, bridge diff --git a/tests/test_auth_redaction.py b/tests/test_auth_redaction.py new file mode 100644 index 0000000..b281b18 --- /dev/null +++ b/tests/test_auth_redaction.py @@ -0,0 +1,127 @@ +"""The auth-failure path must never carry a credential out of the library. + +`register_v2` is the one call that sends the panel passphrase, and the panel's +validation layer echoes what it rejected. Everything here is about that echo: +that it does not reach the exception message, and that where it *is* kept — the +DEBUG log, which a user chasing a registration failure needs — the credential is +gone and the surrounding detail is not. +""" + +from __future__ import annotations + +import json +import logging +from unittest.mock import AsyncMock, patch + +import httpx +import pytest + +from span_panel_api.auth import _redact, register_v2 +from span_panel_api.exceptions import SpanPanelAuthError + +# The shape a FastAPI-style 422 takes: the submitted body is echoed back under +# `detail[].input`, which is two levels below anything a top-level key scan sees. +SECRET = "correct-horse-battery-staple" + +VALIDATION_422 = { + "detail": [ + { + "type": "value_error", + "loc": ["body", "hopPassphrase"], + "msg": "Value error, passphrase does not match", + "input": {"name": "home-assistant-0badcafe", "hopPassphrase": SECRET}, + } + ] +} + + +def _response(status_code: int, *, json_data: object | None = None, text: str = "") -> httpx.Response: + if json_data is not None: + return httpx.Response( + status_code=status_code, + content=json.dumps(json_data).encode(), + headers={"content-type": "application/json"}, + request=httpx.Request("POST", "http://panel.invalid/api/v2/auth/register"), + ) + return httpx.Response( + status_code=status_code, + content=text.encode(), + headers={"content-type": "text/html"}, + request=httpx.Request("POST", "http://panel.invalid/api/v2/auth/register"), + ) + + +async def _register_against(response: httpx.Response) -> SpanPanelAuthError: + """Drive `register_v2` against one canned response and return what it raised.""" + injected = AsyncMock(spec=httpx.AsyncClient) + injected.post = AsyncMock(return_value=response) + with pytest.raises(SpanPanelAuthError) as excinfo: + await register_v2("panel.invalid", "home-assistant", SECRET, httpx_client=injected) + return excinfo.value + + +class TestRedactWalk: + def test_redacts_nested_credential(self) -> None: + redacted = _redact(VALIDATION_422) + assert SECRET not in json.dumps(redacted) + + def test_keeps_the_diagnostic_around_the_credential(self) -> None: + """Redaction must not flatten the body — the `loc` is what makes it useful.""" + redacted = _redact(VALIDATION_422) + rendered = json.dumps(redacted) + assert "hopPassphrase" in rendered # the key, as a location, is not a secret + assert "value_error" in rendered + assert "home-assistant-0badcafe" in rendered + + def test_case_insensitive_keys(self) -> None: + assert _redact({"HopPassphrase": SECRET, "ACCESSTOKEN": "jwt"}) == { + "HopPassphrase": "***", + "ACCESSTOKEN": "***", + } + + def test_walks_lists_of_lists(self) -> None: + assert _redact([[{"ebusBrokerPassword": SECRET}]]) == [[{"ebusBrokerPassword": "***"}]] + + def test_scalars_pass_through(self) -> None: + assert _redact(7) == 7 + assert _redact(None) is None + assert _redact("plain") == "plain" + + +class TestRegisterV2AuthFailure: + @pytest.mark.asyncio + async def test_secret_is_not_in_the_exception(self) -> None: + exc = await _register_against(_response(422, json_data=VALIDATION_422)) + assert SECRET not in str(exc) + assert "422" in str(exc) + + @pytest.mark.asyncio + async def test_secret_is_not_logged_at_info_or_above(self, caplog: pytest.LogCaptureFixture) -> None: + with caplog.at_level(logging.INFO, logger="span_panel_api.auth"): + await _register_against(_response(422, json_data=VALIDATION_422)) + assert SECRET not in caplog.text + + @pytest.mark.asyncio + async def test_debug_keeps_the_body_with_the_credential_removed(self, caplog: pytest.LogCaptureFixture) -> None: + with caplog.at_level(logging.DEBUG, logger="span_panel_api.auth"): + await _register_against(_response(422, json_data=VALIDATION_422)) + assert SECRET not in caplog.text + # The reason the body is kept at all: it names the field that failed. + assert "passphrase does not match" in caplog.text + + @pytest.mark.asyncio + async def test_non_json_body_is_described_not_shown(self, caplog: pytest.LogCaptureFixture) -> None: + """A proxy's HTML error page has no structure to redact, so only its shape is logged.""" + body = f"rejected {SECRET}" + with caplog.at_level(logging.DEBUG, logger="span_panel_api.auth"): + exc = await _register_against(_response(401, text=body)) + assert SECRET not in caplog.text + assert SECRET not in str(exc) + assert str(len(body.encode())) in caplog.text + assert "text/html" in caplog.text + + @pytest.mark.asyncio + async def test_403_takes_the_same_path(self) -> None: + exc = await _register_against(_response(403, json_data=VALIDATION_422)) + assert SECRET not in str(exc) + assert "403" in str(exc) diff --git a/tests/test_ca_pinning.py b/tests/test_ca_pinning.py new file mode 100644 index 0000000..bd9c86a --- /dev/null +++ b/tests/test_ca_pinning.py @@ -0,0 +1,458 @@ +"""Pinning the panel CA, and refusing to draw the wrong conclusion from a TLS failure. + +Two separate claims are under test here and they pull in opposite directions. + +The first is that a pinned bridge never re-anchors: it makes no CA request on +connect or on rebuild, so a panel presenting a chain from some other CA cannot +become trusted by being persistent. + +The second is that the bridge is *slow* to call something a CA change. A valid +pinned CA still produces `SSLCertVerificationError` when the panel's clock has +reset or its address has moved, and a broker restarting mid-handshake produces +`SSLEOFError` -- which is the ordinary shape of a firmware upgrade. Every one of +those has to stay retryable, because escalating one of them converts a +self-healing outage into a permanent one. +""" + +from __future__ import annotations + +import asyncio +import base64 +import logging +import ssl +from unittest.mock import MagicMock, patch + +from paho.mqtt.client import DisconnectFlags +from paho.mqtt.reasoncodes import ReasonCode +import pytest + +from span_panel_api._ssl import ca_fingerprint +from span_panel_api.exceptions import ( + SpanPanelCAChangedError, + SpanPanelConnectionError, + SpanPanelError, + SpanPanelStaleDataError, + SpanPanelTimeoutError, + SpanPanelValidationError, +) +from span_panel_api.mqtt import connection as conn_mod +from span_panel_api.mqtt.client import SpanMqttClient +from span_panel_api.mqtt.connection import AsyncMqttBridge +from span_panel_api.mqtt.models import MqttClientConfig + +from conftest import SERIAL + + +def _pem(marker: bytes) -> str: + """A PEM block whose body is `marker`. + + `ca_fingerprint` hashes the decoded DER bytes and asks nothing else of them, + and every test that also needs a *usable* context patches the context + builder. Minting real X.509 for these would add a `cryptography` dependency + to assertions that are about identity, not about validity -- + `test_ssl_context.py` already covers the certificate side. + """ + return "-----BEGIN CERTIFICATE-----\n" + base64.b64encode(marker).decode() + "\n-----END CERTIFICATE-----\n" + + +PINNED_PEM = _pem(b"the-panel-ca") +ROTATED_PEM = _pem(b"a-different-ca") +PINNED_FP = ca_fingerprint(PINNED_PEM) +ROTATED_FP = ca_fingerprint(ROTATED_PEM) + + +def _bridge(*, ca_pem: str | None) -> AsyncMqttBridge: + return AsyncMqttBridge( + host="broker.local", + port=8883, + username="user", + password="pass", + panel_host="panel.invalid", + serial_number=SERIAL, + use_tls=True, + ca_pem=ca_pem, + ) + + +def _drive_reconnect_loop(bridge: AsyncMqttBridge, client_mock: MagicMock) -> None: + """Push the bridge into its reconnect loop through a disconnect edge.""" + bridge._on_disconnect( + client_mock, + None, + DisconnectFlags(is_disconnect_packet_from_server=True), + ReasonCode(packetType=2, aName="Success"), + None, + ) + assert bridge._reconnect_task is not None + + +# --------------------------------------------------------------------------- +# ca_fingerprint +# --------------------------------------------------------------------------- + + +class TestCaFingerprint: + def test_stable_across_pem_whitespace(self) -> None: + """A firmware that reflows its PEM has not rotated its CA. + + Reporting one because the line width changed is the worse of the two + available errors: it teaches a user to dismiss the alert that matters. + """ + body = base64.b64encode(b"the-panel-ca").decode() + variants = [ + f"-----BEGIN CERTIFICATE-----\n{body}\n-----END CERTIFICATE-----\n", + f"-----BEGIN CERTIFICATE-----\r\n{body}\r\n-----END CERTIFICATE-----\r\n", + f" -----BEGIN CERTIFICATE-----\n {body[:4]}\n {body[4:]} \n-----END CERTIFICATE----- ", + f"issuer: test\n-----BEGIN CERTIFICATE-----\n{body}\n-----END CERTIFICATE-----\ntrailing\n", + ] + assert {ca_fingerprint(v) for v in variants} == {PINNED_FP} + + def test_lowercase_hex_without_separators(self) -> None: + assert len(PINNED_FP) == 64 + assert PINNED_FP == PINNED_FP.lower() + assert ":" not in PINNED_FP + + def test_different_certificates_differ(self) -> None: + assert PINNED_FP != ROTATED_FP + + def test_only_the_first_certificate_is_read(self) -> None: + """An appended chain must not change the anchor's fingerprint.""" + assert ca_fingerprint(PINNED_PEM + ROTATED_PEM) == PINNED_FP + + @pytest.mark.parametrize( + "bad", + [ + "no certificate here", + "-----BEGIN CERTIFICATE-----\nZm9v\n", + "-----BEGIN CERTIFICATE-----\n!!!not base64!!!\n-----END CERTIFICATE-----\n", + "-----BEGIN CERTIFICATE-----\n\n-----END CERTIFICATE-----\n", + ], + ids=["no-block", "unterminated", "not-base64", "empty"], + ) + def test_malformed_input_is_rejected(self, bad: str) -> None: + with pytest.raises(SpanPanelValidationError): + ca_fingerprint(bad) + + +# --------------------------------------------------------------------------- +# The pin itself: no CA request on any path +# --------------------------------------------------------------------------- + + +class TestTrustAnchor: + @pytest.mark.asyncio + async def test_pinned_connect_makes_no_ca_request(self, mqtt_client_mock: MagicMock) -> None: + bridge = _bridge(ca_pem=PINNED_PEM) + with patch("span_panel_api.mqtt.connection.build_panel_ssl_context") as build: + await bridge.connect() + + conn_mod.download_ca_cert.assert_not_called() + build.assert_called_once_with(PINNED_PEM) + await bridge.disconnect() + + @pytest.mark.asyncio + async def test_pinned_rebuild_makes_no_ca_request(self, mqtt_client_mock: MagicMock) -> None: + """The rebuild was the re-anchoring path, and is the one that mattered.""" + bridge = _bridge(ca_pem=PINNED_PEM) + await bridge.connect() + conn_mod.download_ca_cert.reset_mock() + + assert await bridge._rebuild_client() is True + conn_mod.download_ca_cert.assert_not_called() + + await bridge.disconnect() + + @pytest.mark.asyncio + async def test_unpinned_fetches_and_warns_once_per_bridge( + self, mqtt_client_mock: MagicMock, caplog: pytest.LogCaptureFixture + ) -> None: + """3.0.1's behaviour, kept, plus exactly one warning. + + Once per bridge rather than once per connect: the fetch happens on every + reconnect, and a line per reconnect through a day-long outage is a log + nobody reads. + """ + bridge = _bridge(ca_pem=None) + with caplog.at_level(logging.WARNING, logger="span_panel_api.mqtt.connection"): + await bridge.connect() + await bridge._rebuild_client() + await bridge._rebuild_client() + + assert conn_mod.download_ca_cert.call_count == 3 + unpinned = [r for r in caplog.records if "obtained unauthenticated" in r.message] + assert len(unpinned) == 1 + assert unpinned[0].levelno == logging.WARNING + + await bridge.disconnect() + + +# --------------------------------------------------------------------------- +# The disambiguation procedure +# --------------------------------------------------------------------------- + + +class TestDiagnoseCaChange: + @pytest.mark.asyncio + async def test_unpinned_never_escalates(self) -> None: + bridge = _bridge(ca_pem=None) + with patch("span_panel_api.mqtt.connection.download_ca_cert") as fetch: + assert await bridge._diagnose_ca_change() is None + fetch.assert_not_called() + + @pytest.mark.asyncio + async def test_same_fingerprint_is_not_a_ca_change(self, caplog: pytest.LogCaptureFixture) -> None: + """An expired leaf or a moved host: verification fails, the anchor did not move.""" + bridge = _bridge(ca_pem=PINNED_PEM) + with ( + patch("span_panel_api.mqtt.connection.download_ca_cert", return_value=PINNED_PEM), + caplog.at_level(logging.WARNING, logger="span_panel_api.mqtt.connection"), + ): + assert await bridge._diagnose_ca_change() is None + assert "still advertises the pinned CA" in caplog.text + + @pytest.mark.asyncio + async def test_different_fingerprint_carries_both(self) -> None: + bridge = _bridge(ca_pem=PINNED_PEM) + with patch("span_panel_api.mqtt.connection.download_ca_cert", return_value=ROTATED_PEM): + error = await bridge._diagnose_ca_change() + assert isinstance(error, SpanPanelCAChangedError) + assert error.expected_fingerprint == PINNED_FP + assert error.observed_fingerprint == ROTATED_FP + assert PINNED_FP in str(error) + assert ROTATED_FP in str(error) + + @pytest.mark.asyncio + @pytest.mark.parametrize( + "failure", + [ + SpanPanelConnectionError("panel unreachable"), + SpanPanelTimeoutError("timed out"), + OSError("network down"), + ], + ids=["unreachable", "timeout", "oserror"], + ) + async def test_fetch_failure_never_escalates(self, failure: Exception) -> None: + """Missing evidence is not evidence. A panel mid-reboot looks exactly like this.""" + bridge = _bridge(ca_pem=PINNED_PEM) + with patch("span_panel_api.mqtt.connection.download_ca_cert", side_effect=failure): + assert await bridge._diagnose_ca_change() is None + + @pytest.mark.asyncio + async def test_unfingerprintable_answer_never_escalates(self) -> None: + """A proxy's error page in place of a PEM says nothing about the CA.""" + bridge = _bridge(ca_pem=PINNED_PEM) + with patch("span_panel_api.mqtt.connection.download_ca_cert", return_value="404"): + assert await bridge._diagnose_ca_change() is None + + +# --------------------------------------------------------------------------- +# Initial connect +# --------------------------------------------------------------------------- + + +class TestInitialConnect: + @pytest.mark.asyncio + async def test_ca_changed_while_down_raises_rather_than_looping(self, mqtt_client_mock: MagicMock) -> None: + """The rotation-while-shut-down case, which used to be an endless setup retry.""" + bridge = _bridge(ca_pem=PINNED_PEM) + mqtt_client_mock.connect.side_effect = ssl.SSLCertVerificationError("unable to get local issuer certificate") + + with patch("span_panel_api.mqtt.connection.download_ca_cert", return_value=ROTATED_PEM): + with pytest.raises(SpanPanelCAChangedError) as excinfo: + await bridge.connect() + + assert excinfo.value.expected_fingerprint == PINNED_FP + assert excinfo.value.observed_fingerprint == ROTATED_FP + + @pytest.mark.asyncio + async def test_verification_failure_with_unchanged_ca_stays_retryable(self, mqtt_client_mock: MagicMock) -> None: + bridge = _bridge(ca_pem=PINNED_PEM) + mqtt_client_mock.connect.side_effect = ssl.SSLCertVerificationError("certificate has expired") + + with patch("span_panel_api.mqtt.connection.download_ca_cert", return_value=PINNED_PEM): + with pytest.raises(SpanPanelConnectionError): + await bridge.connect() + + +# --------------------------------------------------------------------------- +# The reconnect loop +# --------------------------------------------------------------------------- + + +class TestReconnectLoop: + @pytest.mark.asyncio + async def test_confirmed_ca_change_stops_the_loop_and_fires_the_callback( + self, mqtt_client_mock: MagicMock, monkeypatch: pytest.MonkeyPatch + ) -> None: + monkeypatch.setattr("span_panel_api.mqtt.connection.MQTT_RECONNECT_MIN_DELAY_S", 0.01) + bridge = _bridge(ca_pem=PINNED_PEM) + await bridge.connect() + + seen: list[SpanPanelError] = [] + bridge.set_fatal_error_callback(seen.append) + mqtt_client_mock.reconnect.side_effect = ssl.SSLCertVerificationError("unable to get local issuer certificate") + + with patch("span_panel_api.mqtt.connection.download_ca_cert", return_value=ROTATED_PEM): + _drive_reconnect_loop(bridge, mqtt_client_mock) + await asyncio.sleep(0.2) + + assert isinstance(bridge.fatal_error, SpanPanelCAChangedError) + assert [type(e) for e in seen] == [SpanPanelCAChangedError] + # The loop is out, and _on_disconnect cannot start a replacement. + assert bridge._should_reconnect is False + assert bridge._reconnect_task is not None + assert bridge._reconnect_task.done() + + @pytest.mark.asyncio + async def test_expired_leaf_keeps_retrying(self, mqtt_client_mock: MagicMock, monkeypatch: pytest.MonkeyPatch) -> None: + """A panel whose clock reset after a power cut must not be declared compromised.""" + monkeypatch.setattr("span_panel_api.mqtt.connection.MQTT_RECONNECT_MIN_DELAY_S", 0.01) + bridge = _bridge(ca_pem=PINNED_PEM) + await bridge.connect() + mqtt_client_mock.reconnect.side_effect = ssl.SSLCertVerificationError("certificate has expired") + + with patch("span_panel_api.mqtt.connection.download_ca_cert", return_value=PINNED_PEM): + _drive_reconnect_loop(bridge, mqtt_client_mock) + await asyncio.sleep(0.1) + + assert bridge.fatal_error is None + assert bridge._should_reconnect is True + assert mqtt_client_mock.reconnect.call_count > 1 + + await bridge.disconnect() + + @pytest.mark.asyncio + async def test_ssl_eof_keeps_retrying(self, mqtt_client_mock: MagicMock, monkeypatch: pytest.MonkeyPatch) -> None: + """A broker restarting mid-handshake -- the ordinary shape of a firmware upgrade.""" + monkeypatch.setattr("span_panel_api.mqtt.connection.MQTT_RECONNECT_MIN_DELAY_S", 0.01) + bridge = _bridge(ca_pem=PINNED_PEM) + await bridge.connect() + mqtt_client_mock.reconnect.side_effect = ssl.SSLEOFError("EOF occurred in violation of protocol") + + with patch("span_panel_api.mqtt.connection.download_ca_cert", return_value=ROTATED_PEM) as fetch: + _drive_reconnect_loop(bridge, mqtt_client_mock) + await asyncio.sleep(0.1) + + assert bridge.fatal_error is None + assert bridge._should_reconnect is True + # Never even asked: SSLEOFError is not a verification failure, so the + # diagnostic that could escalate it is never reached. + fetch.assert_not_called() + + await bridge.disconnect() + + @pytest.mark.asyncio + async def test_diagnostic_fetch_failure_keeps_retrying( + self, mqtt_client_mock: MagicMock, monkeypatch: pytest.MonkeyPatch + ) -> None: + monkeypatch.setattr("span_panel_api.mqtt.connection.MQTT_RECONNECT_MIN_DELAY_S", 0.01) + bridge = _bridge(ca_pem=PINNED_PEM) + await bridge.connect() + mqtt_client_mock.reconnect.side_effect = ssl.SSLCertVerificationError("unable to get local issuer certificate") + + with patch( + "span_panel_api.mqtt.connection.download_ca_cert", + side_effect=SpanPanelConnectionError("panel HTTP not up yet"), + ): + _drive_reconnect_loop(bridge, mqtt_client_mock) + await asyncio.sleep(0.1) + + assert bridge.fatal_error is None + assert bridge._should_reconnect is True + + await bridge.disconnect() + + @pytest.mark.asyncio + async def test_unpinned_verification_failure_still_rebuilds( + self, mqtt_client_mock: MagicMock, monkeypatch: pytest.MonkeyPatch + ) -> None: + """Unpinned behaviour is 3.0.1's, unchanged: the refetch *is* the recovery.""" + monkeypatch.setattr("span_panel_api.mqtt.connection.MQTT_RECONNECT_MIN_DELAY_S", 0.01) + bridge = _bridge(ca_pem=None) + await bridge.connect() + before = conn_mod.download_ca_cert.call_count + mqtt_client_mock.reconnect.side_effect = [ssl.SSLCertVerificationError("verify failed"), 0] + + _drive_reconnect_loop(bridge, mqtt_client_mock) + await asyncio.sleep(0.2) + + assert conn_mod.download_ca_cert.call_count == before + 1 + assert bridge.fatal_error is None + + await bridge.disconnect() + + +# --------------------------------------------------------------------------- +# Surfacing it through the client +# --------------------------------------------------------------------------- + + +class TestClientSurface: + """The client is wired to the bridge's terminal state, and to nothing else. + + Built by hand rather than through `connect()`: these assertions are about + what `ping()` and `get_snapshot()` consult and in what order, and driving a + full Homie handshake to reach them would test the handshake. + """ + + @staticmethod + def _client(*, connected: bool = True) -> tuple[SpanMqttClient, AsyncMqttBridge]: + client = SpanMqttClient( + host="panel.invalid", + serial_number=SERIAL, + broker_config=MqttClientConfig(broker_host="broker.local", username="u", password="p", ca_pem=PINNED_PEM), + ) + bridge = _bridge(ca_pem=PINNED_PEM) + bridge._connected = connected + bridge.set_fatal_error_callback(client._on_fatal_error) + client._bridge = bridge + adapter = MagicMock() + adapter.is_ready.return_value = True + client._adapter = adapter + return client, bridge + + @pytest.mark.asyncio + async def test_ping_and_get_snapshot_reraise(self) -> None: + """A consumer that registered no callback still cannot read dead as healthy.""" + client, bridge = self._client() + bridge._enter_terminal_state(SpanPanelCAChangedError(PINNED_FP, ROTATED_FP)) + + with pytest.raises(SpanPanelCAChangedError): + await client.ping() + with pytest.raises(SpanPanelCAChangedError): + await client.get_snapshot() + + @pytest.mark.asyncio + async def test_stale_data_is_still_stale_data(self) -> None: + """Without a terminal failure, a disconnect keeps its retryable shape.""" + client, _bridge_obj = self._client(connected=False) + with pytest.raises(SpanPanelStaleDataError): + await client.get_snapshot() + assert await client.ping() is False + + @pytest.mark.asyncio + async def test_registered_callback_fires_and_unregisters(self) -> None: + client, bridge = self._client() + seen: list[SpanPanelError] = [] + unregister = client.register_fatal_error_callback(seen.append) + + bridge._enter_terminal_state(SpanPanelCAChangedError(PINNED_FP, ROTATED_FP)) + assert len(seen) == 1 + + unregister() + unregister() # idempotent + client._on_fatal_error(SpanPanelCAChangedError(PINNED_FP, ROTATED_FP)) + assert len(seen) == 1 + + @pytest.mark.asyncio + async def test_a_raising_subscriber_does_not_swallow_the_rest(self) -> None: + client, _bridge_obj = self._client() + seen: list[SpanPanelError] = [] + + def _explode(_error: SpanPanelError) -> None: + raise RuntimeError("subscriber is broken") + + client.register_fatal_error_callback(_explode) + client.register_fatal_error_callback(seen.append) + client._on_fatal_error(SpanPanelCAChangedError(PINNED_FP, ROTATED_FP)) + assert len(seen) == 1 diff --git a/tests/test_control_interceptor.py b/tests/test_control_interceptor.py new file mode 100644 index 0000000..fe508bc --- /dev/null +++ b/tests/test_control_interceptor.py @@ -0,0 +1,235 @@ +"""One veto-and-observe point that every control command passes through. + +The contract has four edges that each exist for a reason a test can state: +a veto's exception reaches the caller untranslated (the consumer raises a +framework error carrying a translated message and needs it intact); a refusal +still produces an audit record (an audit that omits refusals is worse than +none); the observation half is fired as a task (a sink that merely hangs must +not stall control); and the interceptor sees the refusals and the no-op, not +only the commands that reached the wire. +""" + +from __future__ import annotations + +import asyncio +from unittest.mock import MagicMock + +import pytest + +from span_panel_api.models import ControlTarget +from span_panel_api.mqtt.client import SpanMqttClient +from span_panel_api.mqtt.connection import AsyncMqttBridge +from span_panel_api.mqtt.control import ControlCommand, ControlDeadlines, ControlInterceptor, PublishOutcome, PublishState +from span_panel_api.mqtt.models import MqttClientConfig +from span_panel_api.protocol import ControlInterceptionProtocol + +from conftest import SERIAL + +CIRCUIT = "aabbccdd112233445566778899001122" +RELAY_TOPIC = f"ebus/5/{SERIAL}/{CIRCUIT}/relay/set" + + +class _Recorder: + """An interceptor that records, and optionally refuses.""" + + def __init__(self, veto: Exception | None = None, hang: bool = False) -> None: + self.veto = veto + self.hang = hang + self.before: list[ControlCommand] = [] + self.after: list[tuple[ControlCommand, PublishOutcome]] = [] + + async def before_publish(self, command: ControlCommand) -> None: + self.before.append(command) + if self.veto is not None: + raise self.veto + + async def after_publish(self, command: ControlCommand, outcome: PublishOutcome) -> None: + self.after.append((command, outcome)) + if self.hang: + await asyncio.Event().wait() + + +class _Refusal(Exception): + """Stands in for the consumer's own framework-specific error.""" + + +def _client(*, connected: bool = True, deadline: float = 0.05) -> SpanMqttClient: + client = SpanMqttClient( + host="panel.invalid", + serial_number=SERIAL, + broker_config=MqttClientConfig(broker_host="broker.local", username="u", password="p"), + control_deadlines=ControlDeadlines(relay=deadline), + ) + adapter = MagicMock() + adapter.set_circuit_relay_target.return_value = ControlTarget( + topic=RELAY_TOPIC, device_id=SERIAL, node_id=CIRCUIT, property_id="relay" + ) + adapter.register_property_callback.side_effect = lambda cb: lambda: None + client._adapter = adapter + client._observe(adapter) + + paho_client = MagicMock() + paho_client.publish.return_value = MagicMock(rc=0, mid=11) + bridge = AsyncMqttBridge( + host="broker.local", + port=8883, + username="u", + password="p", + panel_host="panel.invalid", + serial_number=SERIAL, + ) + bridge._connected = connected + bridge._client = paho_client + bridge._loop = asyncio.get_event_loop() + client._bridge = bridge + return client + + +async def _settle() -> None: + """Let the fire-and-forget `after_publish` task run.""" + await asyncio.sleep(0) + await asyncio.sleep(0) + + +class TestRegistration: + def test_the_client_satisfies_the_protocol(self) -> None: + """The consumer codes against the protocol, never against the class.""" + client = SpanMqttClient( + host="panel.invalid", + serial_number=SERIAL, + broker_config=MqttClientConfig(broker_host="broker.local", username="u", password="p"), + ) + assert isinstance(client, ControlInterceptionProtocol) + + @pytest.mark.asyncio + async def test_one_at_a_time_and_removable(self) -> None: + client = _client() + first, second = _Recorder(), _Recorder() + + client.set_control_interceptor(first) + client.set_control_interceptor(second) + await client.set_circuit_relay(CIRCUIT, "OPEN") + + assert first.before == [] + assert len(second.before) == 1 + + client.set_control_interceptor(None) + await client.set_circuit_relay(CIRCUIT, "CLOSED") + assert len(second.before) == 1 + + +class TestVeto: + @pytest.mark.asyncio + async def test_the_exception_reaches_the_caller_untranslated(self) -> None: + client = _client() + client.set_control_interceptor(_Recorder(veto=_Refusal("only admins may do that"))) + + with pytest.raises(_Refusal, match="only admins may do that"): + await client.set_circuit_relay(CIRCUIT, "OPEN") + + @pytest.mark.asyncio + async def test_nothing_is_published(self) -> None: + client = _client() + client.set_control_interceptor(_Recorder(veto=_Refusal("no"))) + assert client._bridge is not None + + with pytest.raises(_Refusal): + await client.set_circuit_relay(CIRCUIT, "OPEN") + + client._bridge._client.publish.assert_not_called() + + @pytest.mark.asyncio + async def test_a_refusal_still_produces_an_audit_record(self) -> None: + """An audit that silently omits refusals is worse than no audit.""" + client = _client() + recorder = _Recorder(veto=_Refusal("no")) + client.set_control_interceptor(recorder) + + with pytest.raises(_Refusal): + await client.set_circuit_relay(CIRCUIT, "OPEN") + await _settle() + + assert len(recorder.after) == 1 + _command, outcome = recorder.after[0] + assert outcome.state is PublishState.FAILED + assert outcome.detail == "vetoed" + + +class TestObservation: + @pytest.mark.asyncio + async def test_the_command_carries_the_wire_address_and_the_translated_value(self) -> None: + client = _client() + recorder = _Recorder() + client.set_control_interceptor(recorder) + + await client.set_circuit_relay(CIRCUIT, "OPEN") + + command = recorder.before[0] + assert command == ControlCommand( + device_id=SERIAL, + node_id=CIRCUIT, + property_id="relay", + value="OPEN", + topic=RELAY_TOPIC, + ) + + @pytest.mark.asyncio + async def test_a_refused_publish_is_seen_too(self) -> None: + """Not only the commands that reached the wire -- the interesting ones do not.""" + client = _client(connected=False) + recorder = _Recorder() + client.set_control_interceptor(recorder) + + outcome = await client.set_circuit_relay(CIRCUIT, "OPEN") + await _settle() + + assert outcome.state is PublishState.FAILED + assert len(recorder.before) == 1 + assert recorder.after[0][1].state is PublishState.FAILED + + @pytest.mark.asyncio + async def test_a_no_op_is_seen_too(self) -> None: + client = _client() + recorder = _Recorder() + client.set_control_interceptor(recorder) + client._on_property_value(SERIAL, CIRCUIT, "relay", "OPEN") + + outcome = await client.set_circuit_relay(CIRCUIT, "OPEN") + await _settle() + + assert outcome.no_op is True + assert len(recorder.before) == 1 + assert recorder.after[0][1].no_op is True + + @pytest.mark.asyncio + async def test_a_hanging_sink_does_not_stall_control(self) -> None: + """Awaiting `after_publish` would make a slow event bus a control outage.""" + client = _client() + client.set_control_interceptor(_Recorder(hang=True)) + + await asyncio.wait_for(client.set_circuit_relay(CIRCUIT, "OPEN"), timeout=1.0) + + # The hung task is tracked, so it is cancelled with the client rather + # than garbage-collected mid-await. + assert client._background_tasks + await client.close() + + @pytest.mark.asyncio + async def test_a_raising_sink_does_not_reach_the_caller(self, caplog: pytest.LogCaptureFixture) -> None: + class _Exploding: + async def before_publish(self, command: ControlCommand) -> None: + return None + + async def after_publish(self, command: ControlCommand, outcome: PublishOutcome) -> None: + raise RuntimeError("the audit sink is broken") + + client = _client() + client.set_control_interceptor(_Exploding()) + + outcome = await client.set_circuit_relay(CIRCUIT, "OPEN") + await _settle() + + assert outcome.state is not PublishState.FAILED + + def test_a_conforming_object_satisfies_the_protocol(self) -> None: + assert isinstance(_Recorder(), ControlInterceptor) diff --git a/tests/test_https_transport.py b/tests/test_https_transport.py new file mode 100644 index 0000000..d5b7b4d --- /dev/null +++ b/tests/test_https_transport.py @@ -0,0 +1,252 @@ +"""HTTPS for the bootstrap REST calls, and the two ways it could silently not happen. + +Both failure modes this covers are ones where the control *appears* to be on: +an `ssl_context` handed to a call that also has an injected httpx client (httpx +fixes `verify=` at construction, so the context would have been dropped), and a +port left at the stored plaintext default while the caller believes it is +talking TLS. +""" + +from __future__ import annotations + +from collections.abc import Awaitable, Callable +import json +import ssl +from unittest.mock import AsyncMock, patch + +import httpx +import pytest + +from span_panel_api._http import _build_url, _get_client +from span_panel_api.auth import ( + delete_fqdn, + download_ca_cert, + get_fqdn, + get_homie_schema, + get_v2_status, + regenerate_passphrase, + register_fqdn, + register_v2, +) +from span_panel_api.detection import detect_api_version +from span_panel_api.exceptions import SpanPanelValidationError + +HOST = "panel.invalid" + +V2_AUTH_JSON = { + "accessToken": "jwt", + "tokenType": "Bearer", + "iatMs": 1700000000000, + "ebusBrokerUsername": "broker-user", + "ebusBrokerPassword": "broker-pass", + "ebusBrokerHost": HOST, + "ebusBrokerMqttsPort": 8883, + "ebusBrokerWsPort": 9001, + "ebusBrokerWssPort": 9002, + "hostname": "panel", + "serialNumber": "SYN-0000-0001", + "hopPassphrase": "hop", +} + +PEM = "-----BEGIN CERTIFICATE-----\nZm9v\n-----END CERTIFICATE-----\n" + + +@pytest.fixture +def context() -> ssl.SSLContext: + """Any context object will do — nothing here completes a handshake.""" + return ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + + +def _json_response(payload: object, status_code: int = 200) -> httpx.Response: + return httpx.Response( + status_code=status_code, + content=json.dumps(payload).encode(), + headers={"content-type": "application/json"}, + request=httpx.Request("GET", f"http://{HOST}/"), + ) + + +def _text_response(text: str, status_code: int = 200) -> httpx.Response: + return httpx.Response( + status_code=status_code, + content=text.encode(), + headers={"content-type": "text/plain"}, + request=httpx.Request("GET", f"http://{HOST}/"), + ) + + +class TestBuildUrl: + def test_plaintext_unchanged_from_3_0_1(self) -> None: + assert _build_url(HOST, 80, "/api/v2/status") == f"http://{HOST}/api/v2/status" + assert _build_url(HOST, 8080, "/api/v2/status") == f"http://{HOST}:8080/api/v2/status" + + def test_omitted_port_defaults_by_scheme(self, context: ssl.SSLContext) -> None: + assert _build_url(HOST, None, "/p") == f"http://{HOST}/p" + assert _build_url(HOST, None, "/p", context) == f"https://{HOST}/p" + + def test_https_names_a_non_default_port(self, context: ssl.SSLContext) -> None: + assert _build_url(HOST, 8443, "/p", context) == f"https://{HOST}:8443/p" + + def test_explicit_443_is_omitted(self, context: ssl.SSLContext) -> None: + assert _build_url(HOST, 443, "/p", context) == f"https://{HOST}/p" + + def test_explicit_port_80_with_a_context_is_refused(self, context: ssl.SSLContext) -> None: + """The unmigrated-consumer case: a port stored before the CA was pinned.""" + with pytest.raises(SpanPanelValidationError) as excinfo: + _build_url(HOST, 80, "/p", context) + message = str(excinfo.value) + assert "80" in message + assert "443" in message + assert HOST in message + + +class TestGetClient: + @pytest.mark.asyncio + async def test_context_beats_an_injected_client(self, context: ssl.SSLContext) -> None: + """The whole point of L2: the pin must not be silently dropped.""" + injected = AsyncMock(spec=httpx.AsyncClient) + injected.aclose = AsyncMock() + + with patch("span_panel_api._http.httpx.AsyncClient") as mock_cls: + dedicated = AsyncMock() + dedicated.__aenter__ = AsyncMock(return_value=dedicated) + dedicated.__aexit__ = AsyncMock(return_value=False) + mock_cls.return_value = dedicated + + async with _get_client(injected, timeout=9.0, ssl_context=context) as client: + assert client is dedicated + + mock_cls.assert_called_once_with(timeout=9.0, verify=context) + # The dedicated client is ours, so we close it; the injected one is not. + dedicated.__aexit__.assert_awaited_once() + injected.aclose.assert_not_called() + + @pytest.mark.asyncio + async def test_no_context_still_yields_the_injected_client(self) -> None: + injected = AsyncMock(spec=httpx.AsyncClient) + async with _get_client(injected, timeout=1.0) as client: + assert client is injected + + +class TestAuthCallsUseHttps: + """Every bootstrap endpoint moves to https:// when a context is supplied.""" + + @pytest.mark.asyncio + @pytest.mark.parametrize( + ("call", "method", "response", "path"), + [ + ( + lambda ctx, c: register_v2(HOST, "ha", "secret", httpx_client=c, ssl_context=ctx), + "post", + _json_response(V2_AUTH_JSON), + "/api/v2/auth/register", + ), + ( + lambda ctx, c: get_homie_schema(HOST, httpx_client=c, ssl_context=ctx), + "get", + _json_response({"firmwareVersion": "x", "types": {}}), + "/api/v2/homie/schema", + ), + ( + lambda ctx, c: regenerate_passphrase(HOST, "tok", httpx_client=c, ssl_context=ctx), + "put", + _json_response({"ebusBrokerPassword": "new"}), + "/api/v2/auth/passphrase", + ), + ( + lambda ctx, c: register_fqdn(HOST, "tok", "panel.example", httpx_client=c, ssl_context=ctx), + "post", + _json_response({}, 204), + "/api/v2/dns/fqdn", + ), + ( + lambda ctx, c: get_fqdn(HOST, "tok", httpx_client=c, ssl_context=ctx), + "get", + _json_response({"ebusTlsFqdn": "panel.example"}), + "/api/v2/dns/fqdn", + ), + ( + lambda ctx, c: delete_fqdn(HOST, "tok", httpx_client=c, ssl_context=ctx), + "delete", + _json_response({}, 204), + "/api/v2/dns/fqdn", + ), + ( + lambda ctx, c: get_v2_status(HOST, httpx_client=c, ssl_context=ctx), + "get", + _json_response({"serialNumber": "SYN-0000-0001", "firmwareVersion": "x"}), + "/api/v2/status", + ), + ( + lambda ctx, c: detect_api_version(HOST, httpx_client=c, ssl_context=ctx), + "get", + _json_response({"serialNumber": "SYN-0000-0001", "firmwareVersion": "x"}), + "/api/v2/status", + ), + ], + ids=[ + "register_v2", + "get_homie_schema", + "regenerate_passphrase", + "register_fqdn", + "get_fqdn", + "delete_fqdn", + "get_v2_status", + "detect_api_version", + ], + ) + async def test_https_url_and_dedicated_client( + self, + context: ssl.SSLContext, + call: Callable[[ssl.SSLContext, httpx.AsyncClient], Awaitable[object]], + method: str, + response: httpx.Response, + path: str, + ) -> None: + injected = AsyncMock(spec=httpx.AsyncClient) + with patch("span_panel_api._http.httpx.AsyncClient") as mock_cls: + dedicated = AsyncMock() + dedicated.__aenter__ = AsyncMock(return_value=dedicated) + dedicated.__aexit__ = AsyncMock(return_value=False) + getattr(dedicated, method).return_value = response + mock_cls.return_value = dedicated + + await call(context, injected) + + mock_cls.assert_called_once() + assert mock_cls.call_args.kwargs["verify"] is context + url = getattr(dedicated, method).call_args.args[0] + assert url == f"https://{HOST}{path}" + # The injected client must not have been used for a call that pins a CA. + getattr(injected, method).assert_not_called() + + @pytest.mark.asyncio + async def test_without_a_context_the_url_is_unchanged(self) -> None: + injected = AsyncMock(spec=httpx.AsyncClient) + injected.get = AsyncMock(return_value=_json_response({"firmwareVersion": "x", "types": {}})) + await get_homie_schema(HOST, httpx_client=injected) + assert injected.get.call_args.args[0] == f"http://{HOST}/api/v2/homie/schema" + + +class TestDownloadCaCert: + """The bootstrap fetch stays plaintext by default; the diagnostic refetch does not.""" + + @pytest.mark.asyncio + async def test_default_is_plaintext(self) -> None: + injected = AsyncMock(spec=httpx.AsyncClient) + injected.get = AsyncMock(return_value=_text_response(PEM)) + assert await download_ca_cert(HOST, httpx_client=injected) == PEM + assert injected.get.call_args.args[0] == f"http://{HOST}/api/v2/certificate/ca" + + @pytest.mark.asyncio + async def test_refetch_with_a_context_is_https(self, context: ssl.SSLContext) -> None: + with patch("span_panel_api._http.httpx.AsyncClient") as mock_cls: + dedicated = AsyncMock() + dedicated.__aenter__ = AsyncMock(return_value=dedicated) + dedicated.__aexit__ = AsyncMock(return_value=False) + dedicated.get.return_value = _text_response(PEM) + mock_cls.return_value = dedicated + + assert await download_ca_cert(HOST, ssl_context=context) == PEM + + assert dedicated.get.call_args.args[0] == f"https://{HOST}/api/v2/certificate/ca" diff --git a/tests/test_mqtt_client_connection.py b/tests/test_mqtt_client_connection.py index f1a0176..1611a74 100644 --- a/tests/test_mqtt_client_connection.py +++ b/tests/test_mqtt_client_connection.py @@ -36,6 +36,9 @@ class _FakeBridge(AsyncMqttBridge): def __init__(self, connected: bool = True) -> None: # Intentionally do not call super().__init__ — avoids I/O setup. self._connected = connected + # No terminal failure: get_snapshot() and ping() consult this before the + # liveness checks, so a stub that omits it is a bridge with no answer. + self._fatal_error = None self.subscribed_topics: list[tuple[str, int]] = [] def is_connected(self) -> bool: diff --git a/tests/test_mqtt_connect_flow.py b/tests/test_mqtt_connect_flow.py index d4e83cc..a2feb53 100644 --- a/tests/test_mqtt_connect_flow.py +++ b/tests/test_mqtt_connect_flow.py @@ -20,7 +20,7 @@ from span_panel_api.mqtt.const import MQTT_FULL_REBUILD_AFTER_FAILURES, MQTT_RECONNECT_MIN_DELAY_S from span_panel_api.mqtt.models import MqttClientConfig -from conftest import MINIMAL_DESCRIPTION, SERIAL, TOPIC_PREFIX_SERIAL +from conftest import FAST_CONTROL_DEADLINES, MINIMAL_DESCRIPTION, SERIAL, TOPIC_PREFIX_SERIAL def _make_bridge() -> AsyncMqttBridge: @@ -103,7 +103,7 @@ async def test_malformed_ca_pem_raises_connection_error(self, mqtt_client_mock: """Malformed CA PEM must surface as SpanPanelConnectionError, not ssl.SSLError.""" bridge = _make_bridge() with patch( - "span_panel_api.mqtt.connection._build_ssl_context", + "span_panel_api.mqtt.connection.build_panel_ssl_context", side_effect=ssl.SSLError("malformed PEM"), ): with pytest.raises(SpanPanelConnectionError, match="Failed to build SSL context"): @@ -153,8 +153,11 @@ async def test_publish_after_connect(self, mqtt_client_mock: MagicMock) -> None: bridge = _make_bridge() await bridge.connect() - bridge.publish("test/topic", "hello", qos=1) + acknowledged = bridge.publish("test/topic", "hello") mqtt_client_mock.publish.assert_called_once_with("test/topic", payload="hello", qos=1) + # Handed over, and still waiting: the PUBACK has not arrived. + assert acknowledged is not None + assert not acknowledged.done() # --------------------------------------------------------------------------- @@ -295,6 +298,7 @@ def _make_span_client(snapshot_interval: float = 1.0) -> SpanMqttClient: serial_number=SERIAL, broker_config=config, snapshot_interval=snapshot_interval, + control_deadlines=FAST_CONTROL_DEADLINES, ) diff --git a/tests/test_mqtt_homie.py b/tests/test_mqtt_homie.py index 9a109ee..3b1405b 100644 --- a/tests/test_mqtt_homie.py +++ b/tests/test_mqtt_homie.py @@ -41,7 +41,7 @@ from span_panel_api.mqtt.connection import AsyncMqttBridge from span_panel_api.mqtt.models import MqttClientConfig -from conftest import flat_schema +from conftest import FAST_CONTROL_DEADLINES, acking_bridge, flat_schema from span_panel_api.protocol import ( PanelCapability, ) @@ -52,7 +52,7 @@ class _ConnectedBridge(AsyncMqttBridge): def __init__(self) -> None: # noqa: D107 # Bypass AsyncMqttBridge.__init__ — avoids TLS/network setup. - pass + self._fatal_error = None def is_connected(self) -> bool: # noqa: D102 return True @@ -998,7 +998,12 @@ def test_capabilities(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) caps = client.capabilities assert PanelCapability.EBUS_MQTT in caps assert PanelCapability.PUSH_STREAMING in caps @@ -1018,10 +1023,15 @@ async def test_set_circuit_relay_publishes(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) client._adapter = SchemaZeroAdapter(serial_number=SERIAL, schema=flat_schema(32)) - mock_bridge = MagicMock() + mock_bridge = acking_bridge() client._bridge = mock_bridge await client.set_circuit_relay("aabbccdd112233445566778899001122", "OPEN") @@ -1029,7 +1039,6 @@ async def test_set_circuit_relay_publishes(self): mock_bridge.publish.assert_called_once_with( f"{TOPIC_PREFIX}/{SERIAL}/aabbccdd112233445566778899001122/relay/set", "OPEN", - qos=1, ) @pytest.mark.asyncio @@ -1037,10 +1046,15 @@ async def test_set_circuit_priority_publishes(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) client._adapter = SchemaZeroAdapter(serial_number=SERIAL, schema=flat_schema(32)) - mock_bridge = MagicMock() + mock_bridge = acking_bridge() client._bridge = mock_bridge await client.set_circuit_priority("aabbccdd112233445566778899001122", "NEVER") @@ -1048,7 +1062,6 @@ async def test_set_circuit_priority_publishes(self): mock_bridge.publish.assert_called_once_with( f"{TOPIC_PREFIX}/{SERIAL}/aabbccdd112233445566778899001122/shed-priority/set", "NEVER", - qos=1, ) @pytest.mark.asyncio @@ -1056,7 +1069,12 @@ async def test_set_dominant_power_source_publishes(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) client._adapter = SchemaZeroAdapter(serial_number=SERIAL, schema=flat_schema(32)) # Populate the homie description so core node is known @@ -1064,7 +1082,7 @@ async def test_set_dominant_power_source_publishes(self): client._adapter.handle_message(f"{PREFIX}/$state", HOMIE_STATE_READY) client._adapter.handle_message(f"{PREFIX}/$description", desc) - mock_bridge = MagicMock() + mock_bridge = acking_bridge() client._bridge = mock_bridge await client.set_dominant_power_source("BATTERY") @@ -1072,7 +1090,6 @@ async def test_set_dominant_power_source_publishes(self): mock_bridge.publish.assert_called_once_with( f"{TOPIC_PREFIX}/{SERIAL}/core/dominant-power-source/set", "BATTERY", - qos=1, ) @pytest.mark.asyncio @@ -1081,7 +1098,12 @@ async def test_set_dominant_power_source_no_core_node_raises(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) client._adapter = SchemaZeroAdapter(serial_number=SERIAL, schema=flat_schema(32)) # No description loaded — core node not found @@ -1100,7 +1122,12 @@ async def test_get_snapshot_returns_homie_state(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) client._adapter = SchemaZeroAdapter(serial_number=SERIAL, schema=flat_schema(32)) client._bridge = _ConnectedBridge() @@ -1118,7 +1145,12 @@ async def test_ping_false_no_bridge(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) assert await client.ping() is False @pytest.mark.asyncio @@ -1126,10 +1158,18 @@ async def test_ping_true_when_connected_and_ready(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) mock_bridge = MagicMock() mock_bridge.is_connected.return_value = True + # ping() consults this first: a MagicMock would answer with a Mock, which + # is neither None nor raisable. + mock_bridge.fatal_error = None client._bridge = mock_bridge client._adapter = SchemaZeroAdapter(serial_number=SERIAL, schema=flat_schema(32)) @@ -1150,7 +1190,12 @@ async def test_register_and_unregister_snapshot_callback(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) callback = AsyncMock() unregister = client.register_snapshot_callback(callback) @@ -1163,7 +1208,12 @@ async def test_start_stop_streaming(self): from span_panel_api.mqtt.client import SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=SERIAL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=SERIAL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) assert client._streaming is False await client.start_streaming() diff --git a/tests/test_protocol_conformance.py b/tests/test_protocol_conformance.py index 5b23423..a2a2e9e 100644 --- a/tests/test_protocol_conformance.py +++ b/tests/test_protocol_conformance.py @@ -65,11 +65,11 @@ def test_schema_adapter_declares_its_methods() -> None: "build_field_metadata", "circuit_nodes_missing_names", "find_node_by_type", - "set_circuit_relay_topic", - "set_circuit_priority_topic", - "set_dominant_power_source_topic", + "set_circuit_relay_target", + "set_circuit_priority_target", + "set_dominant_power_source_target", "dominant_power_source_payload", - "set_evse_charge_limit_topic", + "set_evse_charge_limit_target", "evse_charge_limit_payload", "register_property_callback", ): diff --git a/tests/test_public_api_unchanged.py b/tests/test_public_api_unchanged.py index 10bfa64..eb628bd 100644 --- a/tests/test_public_api_unchanged.py +++ b/tests/test_public_api_unchanged.py @@ -54,9 +54,16 @@ "ADOPTION_TOPOLOGY_NODE", "AdoptedDevice", "AdoptedProperty", + # Added 2026-08-25 (3.1.0): the adapter answers a control request with the + # topic *and* the property that reports it, in one value. + "ControlTarget", "ExtensionProperty", "ExtensionSubject", "AdoptedControlProtocol", + # Added 2026-08-25 (3.1.0): one veto/observe point for every control + # command, the consumer-side half of its authorisation gate. A protocol of + # its own so the four control protocols are not broken twice in one release. + "ControlInterceptionProtocol", "is_discovery_path", # Snapshots "SpanBatterySnapshot", @@ -82,6 +89,12 @@ "V2AuthResponse", "V2HomieSchema", "V2StatusInfo", + # Added 2026-08-25 with CA pinning (3.1.0). Deliberate additions: the + # integration builds the same SSL context for its own HTTPS calls and stores + # and compares the same fingerprint string, so both live here rather than + # being reimplemented on the far side of the pin where they could drift. + "build_panel_ssl_context", + "ca_fingerprint", "delete_fqdn", "download_ca_cert", "get_fqdn", @@ -93,6 +106,15 @@ # Transport "MqttClientConfig", "SpanMqttClient", + # Added 2026-08-25 (3.1.0): the control-outcome vocabulary. Additive for + # callers -- a call site that ignores the return value is unaffected -- and + # breaking for anything type-checked against the control protocols with + # `-> None`, which the release notes name. + "ControlCommand", + "ControlDeadlines", + "ControlInterceptor", + "PublishOutcome", + "PublishState", # Phase validation "PhaseDistribution", "are_tabs_opposite_phase", @@ -110,6 +132,11 @@ "SpanPanelAdapterIncompatibleError", "SpanPanelAdapterMissingError", "SpanPanelAuthError", + # Added 2026-08-25 (3.1.0): the one connection failure this library will not + # retry, because retrying it means waiting to succeed against whatever is + # answering. Additive -- nothing raised it before, so no caller's except + # clause changes meaning. + "SpanPanelCAChangedError", "SpanPanelSchemaVersionError", "SpanPanelConnectionError", "SpanPanelError", diff --git a/tests/test_publish_outcome.py b/tests/test_publish_outcome.py new file mode 100644 index 0000000..a36b9de --- /dev/null +++ b/tests/test_publish_outcome.py @@ -0,0 +1,438 @@ +"""Control commands report what happened to them, and `FAILED` is a promise. + +Three paths used to return `None` having published nothing, and a fourth -- +publishing while the broker was down -- looked like a discard and was not: paho +queues a QoS-1 message across a disconnect and sends it when the connection +returns. That last one is why the refusal has to happen before paho sees the +message. Reading paho's return code afterwards would report a failure for a +breaker command that fires four minutes later, and a user told "failed" acts on +it. + +So the assertions that matter here are as much about what is *not* claimed -- +nothing is `FAILED` once it has been handed over, and nothing is queued when it +has not -- as about what is. +""" + +from __future__ import annotations + +import asyncio +from collections.abc import Awaitable, Callable +from unittest.mock import MagicMock + +import pytest + +from span_panel_api.models import ControlTarget +from span_panel_api.mqtt.client import SpanMqttClient +from span_panel_api.mqtt.connection import AsyncMqttBridge +from span_panel_api.mqtt.control import ControlDeadlines, PublishOutcome, PublishState +from span_panel_api.mqtt.models import MqttClientConfig + +from conftest import SERIAL + +CIRCUIT = "aabbccdd112233445566778899001122" +FAST = ControlDeadlines(relay=0.05, priority=0.05, dominant_power_source=0.05, evse_charge_limit=0.05, adopted_property=0.05) + + +def _client(*, deadlines: ControlDeadlines | None = None) -> SpanMqttClient: + client = SpanMqttClient( + host="panel.invalid", + serial_number=SERIAL, + broker_config=MqttClientConfig(broker_host="broker.local", username="u", password="p"), + control_deadlines=deadlines or FAST, + ) + adapter = MagicMock() + adapter.set_circuit_relay_target.return_value = ControlTarget( + topic=f"ebus/5/{SERIAL}/{CIRCUIT}/relay/set", device_id=SERIAL, node_id=CIRCUIT, property_id="relay" + ) + adapter.set_circuit_priority_target.return_value = ControlTarget( + topic=f"ebus/5/{SERIAL}/{CIRCUIT}/shed-priority/set", device_id=SERIAL, node_id=CIRCUIT, property_id="shed-priority" + ) + adapter.set_dominant_power_source_target.return_value = ControlTarget( + topic=f"ebus/5/{SERIAL}/core/dominant-power-source/set", + device_id=SERIAL, + node_id="core", + property_id="dominant-power-source", + ) + adapter.dominant_power_source_payload.return_value = "BATTERY" + adapter.set_evse_charge_limit_target.return_value = ControlTarget( + topic="ebus/5/evse-1/config/user-max-charge-current/set", + device_id="evse-1", + node_id="config", + property_id="user-max-charge-current", + ) + adapter.evse_charge_limit_payload.return_value = "24" + # One observer is registered per adapter in `_build_adapter`; these clients + # never connect, so wire it here or nothing feeds the no-op check. + adapter.register_property_callback.side_effect = lambda cb: lambda: None + client._adapter = adapter + client._observe(adapter) + return client + + +def _bridge(*, connected: bool, paho_client: MagicMock | None) -> AsyncMqttBridge: + bridge = AsyncMqttBridge( + host="broker.local", + port=8883, + username="u", + password="p", + panel_host="panel.invalid", + serial_number=SERIAL, + ) + bridge._connected = connected + bridge._client = paho_client + bridge._loop = asyncio.get_event_loop() + return bridge + + +def _paho() -> MagicMock: + client = MagicMock() + client.publish.return_value = MagicMock(rc=0, mid=7) + return client + + +# --------------------------------------------------------------------------- +# The two refusals +# --------------------------------------------------------------------------- + + +class TestFailedMeansNeverDelivered: + @pytest.mark.asyncio + async def test_no_bridge_is_failed_not_a_silent_return(self) -> None: + """`close()` clears the bridge and leaves the adapter, so this path passed + `_require_adapter()` and returned `None` having done nothing.""" + client = _client() + client._bridge = None + + outcome = await client.set_circuit_relay(CIRCUIT, "OPEN") + + assert outcome.state is PublishState.FAILED + assert outcome.value == "OPEN" + assert outcome.detail is not None + + @pytest.mark.asyncio + async def test_no_paho_client_is_failed(self) -> None: + client = _client() + client._bridge = _bridge(connected=False, paho_client=None) + + outcome = await client.set_circuit_relay(CIRCUIT, "OPEN") + + assert outcome.state is PublishState.FAILED + + @pytest.mark.asyncio + async def test_disconnected_is_failed_and_nothing_is_queued_in_paho(self) -> None: + """The substantive fix. paho would have queued this and sent it later.""" + paho_client = _paho() + client = _client() + client._bridge = _bridge(connected=False, paho_client=paho_client) + + outcome = await client.set_circuit_relay(CIRCUIT, "CLOSED") + + assert outcome.state is PublishState.FAILED + paho_client.publish.assert_not_called() + + @pytest.mark.asyncio + @pytest.mark.parametrize( + "call", + [ + lambda c: c.set_circuit_relay(CIRCUIT, "OPEN"), + lambda c: c.set_circuit_priority(CIRCUIT, "NEVER"), + lambda c: c.set_dominant_power_source("BATTERY"), + lambda c: c.set_evse_charge_limit("evse-1", 24), + ], + ids=["relay", "priority", "dps", "evse"], + ) + async def test_every_setter_refuses_while_down( + self, call: Callable[[SpanMqttClient], Awaitable[PublishOutcome]] + ) -> None: + paho_client = _paho() + client = _client() + client._bridge = _bridge(connected=False, paho_client=paho_client) + + outcome = await call(client) + + assert outcome.state is PublishState.FAILED + paho_client.publish.assert_not_called() + + +# --------------------------------------------------------------------------- +# Everything past the gate +# --------------------------------------------------------------------------- + + +class TestHandedOver: + @pytest.mark.asyncio + async def test_puback_yields_accepted(self) -> None: + paho_client = _paho() + client = _client() + bridge = _bridge(connected=True, paho_client=paho_client) + client._bridge = bridge + + started = asyncio.get_running_loop().time() + task = asyncio.ensure_future(client.set_circuit_relay(CIRCUIT, "OPEN")) + await asyncio.sleep(0) # let the publish register its waiter + bridge._on_publish(paho_client, None, 7, MagicMock(), None) + outcome = await task + + assert outcome.state is PublishState.ACCEPTED + assert outcome.no_op is False + paho_client.publish.assert_called_once() + # A PUBACK must NOT end the wait. The broker taking the message says + # nothing about the panel acting on it, so the deadline still has to run + # its course -- otherwise every write would report ACCEPTED the instant + # the broker answered and a transition arriving later would go unseen. + assert asyncio.get_running_loop().time() - started >= FAST.relay + + @pytest.mark.asyncio + async def test_deadline_expiry_yields_unconfirmed_and_does_not_raise(self) -> None: + """An unacknowledged write is not an error; it is most often a no-op write.""" + client = _client() + client._bridge = _bridge(connected=True, paho_client=_paho()) + + outcome = await client.set_circuit_relay(CIRCUIT, "OPEN") + + assert outcome.state is PublishState.UNCONFIRMED + assert outcome.detail is not None and "0.05" in outcome.detail + + @pytest.mark.asyncio + async def test_rebuild_settles_in_flight_publishes_without_calling_them_failed(self) -> None: + """A rebuilt paho client drops the outbound queue, but the original may + already have reached the broker -- so this resolves, and does not claim + the command will never be delivered. + + The deadline here is a realistic relay deadline rather than a fast one, + because the point is that it is never reached: a discarded message ends + the wait immediately. Waiting it out would be five seconds spent on a + transport that had already thrown the message away. + """ + client = _client(deadlines=ControlDeadlines(relay=5.0)) + bridge = _bridge(connected=True, paho_client=_paho()) + client._bridge = bridge + + started = asyncio.get_running_loop().time() + task = asyncio.ensure_future(client.set_circuit_relay(CIRCUIT, "OPEN")) + await asyncio.sleep(0) + bridge._resolve_pending_publishes(False, "test rebuild") + outcome = await task + elapsed = asyncio.get_running_loop().time() - started + + assert outcome.state is PublishState.UNCONFIRMED + assert outcome.state is not PublishState.FAILED + # "discarded", not "rebuilt": the bridge empties its outbound queue on a + # rebuild and on teardown alike, and naming one cause reported a close() + # as a rebuild. + assert outcome.detail is not None and "discarded" in outcome.detail + # Generous enough not to be flaky on a loaded machine, and still two + # orders of magnitude below the deadline it would otherwise have burnt. + assert elapsed < 1.0 + + @pytest.mark.asyncio + async def test_a_settled_publish_is_forgotten(self) -> None: + """The pending map must not grow for the life of the bridge.""" + paho_client = _paho() + bridge = _bridge(connected=True, paho_client=paho_client) + + acknowledged = bridge.publish("some/topic/set", "OPEN") + assert acknowledged is not None + assert bridge._pending_publishes == {7: acknowledged} + + bridge._on_publish(paho_client, None, 7, MagicMock(), None) + await asyncio.sleep(0) + assert bridge._pending_publishes == {} + + @pytest.mark.asyncio + async def test_a_cancelled_waiter_is_forgotten(self) -> None: + """The ordinary end for a message the broker never answers.""" + bridge = _bridge(connected=True, paho_client=_paho()) + + acknowledged = bridge.publish("some/topic/set", "OPEN") + assert acknowledged is not None + acknowledged.cancel() + await asyncio.sleep(0) + + assert bridge._pending_publishes == {} + + @pytest.mark.asyncio + async def test_a_late_puback_after_the_deadline_does_not_explode(self) -> None: + """`asyncio.wait_for` cancels the future; the PUBACK still arrives.""" + paho_client = _paho() + client = _client() + bridge = _bridge(connected=True, paho_client=paho_client) + client._bridge = bridge + + outcome = await client.set_circuit_relay(CIRCUIT, "OPEN") + assert outcome.state is PublishState.UNCONFIRMED + + # Whatever paho does next must not raise out of the callback. + bridge._on_publish(paho_client, None, 7, MagicMock(), None) + + +class TestDisconnectSettlesWaiters: + @pytest.mark.asyncio + async def test_teardown_does_not_leave_a_caller_waiting(self) -> None: + """A close() empties the same outbound queue a rebuild does, so it takes + the same path out -- and must not be described as the other one.""" + client = _client(deadlines=ControlDeadlines(relay=5.0)) + bridge = _bridge(connected=True, paho_client=_paho()) + client._bridge = bridge + + started = asyncio.get_running_loop().time() + task = asyncio.ensure_future(client.set_circuit_relay(CIRCUIT, "OPEN")) + await asyncio.sleep(0) + await bridge.disconnect() + outcome = await task + elapsed = asyncio.get_running_loop().time() - started + + assert outcome.state is PublishState.UNCONFIRMED + assert elapsed < 1.0, "a torn-down transport must not hold the caller to its deadline" + assert outcome.detail is not None + assert "discarded" in outcome.detail + assert "rebuilt" not in outcome.detail, "this was a teardown, not a rebuild" + + +# --------------------------------------------------------------------------- +# Write-then-verify +# --------------------------------------------------------------------------- + + +class TestWriteThenVerify: + """The panel reporting the value back is the only thing that says it landed.""" + + @staticmethod + def _observe(client: SpanMqttClient, target: ControlTarget, value: str) -> None: + """Feed the observation stream as the adapter would.""" + client._on_property_value(target.device_id, target.node_id, target.property_id, value) + + @pytest.mark.asyncio + async def test_a_transition_yields_confirmed(self) -> None: + paho_client = _paho() + client = _client(deadlines=ControlDeadlines(relay=1.0)) + bridge = _bridge(connected=True, paho_client=paho_client) + client._bridge = bridge + target = client._adapter.set_circuit_relay_target(CIRCUIT) + + task = asyncio.ensure_future(client.set_circuit_relay(CIRCUIT, "OPEN")) + await asyncio.sleep(0) + self._observe(client, target, "OPEN") + outcome = await task + + assert outcome.state is PublishState.CONFIRMED + assert outcome.no_op is False + + @pytest.mark.asyncio + async def test_a_transition_to_some_other_value_is_not_a_confirmation(self) -> None: + """A racing external change is not evidence that this write landed.""" + client = _client() + client._bridge = _bridge(connected=True, paho_client=_paho()) + target = client._adapter.set_circuit_relay_target(CIRCUIT) + + task = asyncio.ensure_future(client.set_circuit_relay(CIRCUIT, "OPEN")) + await asyncio.sleep(0) + self._observe(client, target, "CLOSED") + outcome = await task + + assert outcome.state is not PublishState.CONFIRMED + + @pytest.mark.asyncio + async def test_puback_without_a_transition_is_accepted(self) -> None: + paho_client = _paho() + client = _client() + bridge = _bridge(connected=True, paho_client=paho_client) + client._bridge = bridge + + task = asyncio.ensure_future(client.set_circuit_relay(CIRCUIT, "OPEN")) + await asyncio.sleep(0) + bridge._on_publish(paho_client, None, 7, MagicMock(), None) + outcome = await task + + assert outcome.state is PublishState.ACCEPTED + assert outcome.detail is not None and "no transition" in outcome.detail + + @pytest.mark.asyncio + async def test_nothing_at_all_is_unconfirmed(self) -> None: + client = _client() + client._bridge = _bridge(connected=True, paho_client=_paho()) + + outcome = await client.set_circuit_relay(CIRCUIT, "OPEN") + + assert outcome.state is PublishState.UNCONFIRMED + assert outcome.no_op is False + + @pytest.mark.asyncio + async def test_a_no_op_short_circuits_without_publishing(self) -> None: + """A write whose value is already current would burn the whole deadline.""" + paho_client = _paho() + client = _client(deadlines=ControlDeadlines(relay=30.0)) + client._bridge = _bridge(connected=True, paho_client=paho_client) + target = client._adapter.set_circuit_relay_target(CIRCUIT) + self._observe(client, target, "OPEN") + + outcome = await client.set_circuit_relay(CIRCUIT, "OPEN") + + assert outcome.state is PublishState.UNCONFIRMED + assert outcome.no_op is True + paho_client.publish.assert_not_called() + + @pytest.mark.asyncio + async def test_the_no_op_check_compares_wire_vocabulary(self) -> None: + """The caller says BATTERY; the wire says OFF_GRID under v1.0. + + Comparing the caller's string against the observed one would compare two + different vocabularies, never match, and burn a deadline on every + repeated write. + """ + paho_client = _paho() + client = _client(deadlines=ControlDeadlines(dominant_power_source=30.0)) + client._bridge = _bridge(connected=True, paho_client=paho_client) + client._adapter.dominant_power_source_payload.return_value = "OFF_GRID" + target = client._adapter.set_dominant_power_source_target() + self._observe(client, target, "OFF_GRID") + + outcome = await client.set_dominant_power_source("BATTERY") + + assert outcome.no_op is True + assert outcome.value == "OFF_GRID" + paho_client.publish.assert_not_called() + + @pytest.mark.asyncio + async def test_a_different_property_does_not_confirm_this_one(self) -> None: + client = _client() + client._bridge = _bridge(connected=True, paho_client=_paho()) + + task = asyncio.ensure_future(client.set_circuit_relay(CIRCUIT, "OPEN")) + await asyncio.sleep(0) + client._on_property_value(SERIAL, "some-other-circuit", "relay", "OPEN") + outcome = await task + + assert outcome.state is not PublishState.CONFIRMED + + @pytest.mark.asyncio + async def test_a_confirmed_write_leaves_nothing_pending(self) -> None: + """Neither the verification list nor the bridge's publish map may grow.""" + paho_client = _paho() + client = _client(deadlines=ControlDeadlines(relay=1.0)) + bridge = _bridge(connected=True, paho_client=paho_client) + client._bridge = bridge + target = client._adapter.set_circuit_relay_target(CIRCUIT) + + task = asyncio.ensure_future(client.set_circuit_relay(CIRCUIT, "OPEN")) + await asyncio.sleep(0) + self._observe(client, target, "OPEN") + assert (await task).state is PublishState.CONFIRMED + await asyncio.sleep(0) + + assert client._verifications == [] + assert bridge._pending_publishes == {} + + @pytest.mark.asyncio + async def test_swapping_the_adapter_drops_stale_observations(self) -> None: + """The values describe a tree that is being replaced.""" + client = _client() + target = client._adapter.set_circuit_relay_target(CIRCUIT) + self._observe(client, target, "OPEN") + assert client._observed_values + + replacement = MagicMock() + replacement.register_property_callback.side_effect = lambda cb: lambda: None + client._observe(replacement) + + assert client._observed_values == {} diff --git a/tests/test_redispatch_on_reconnect.py b/tests/test_redispatch_on_reconnect.py index 59ae667..553b0f3 100644 --- a/tests/test_redispatch_on_reconnect.py +++ b/tests/test_redispatch_on_reconnect.py @@ -24,6 +24,8 @@ from __future__ import annotations +from collections.abc import Callable + import asyncio from typing import Any from unittest.mock import patch @@ -57,6 +59,7 @@ def __init__(self, serial: str, schema: _Schema) -> None: self.serial = serial self.schema = schema self.schema_major = f"schema_for_{schema.data_model_version}" + self.property_callback: Callable[[str, str, str, str | None], None] | None = None def topics_to_subscribe(self) -> list[str]: return [f"topics/for/{self.schema.data_model_version}"] @@ -70,6 +73,16 @@ def is_ready(self) -> bool: def handle_message(self, topic: str, payload: str) -> None: return None + def register_property_callback(self, callback: Callable[[str, str, str, str | None], None]) -> Callable[[], None]: + """The transport subscribes once per adapter, for write-then-verify. + + Recorded rather than ignored: the swap must leave the transport observing + the *new* parser, and a stub that silently accepted the registration + could not show that. + """ + self.property_callback = callback + return lambda: None + class _Bridge: def __init__(self) -> None: diff --git a/tests/test_schema_one_adapter.py b/tests/test_schema_one_adapter.py index 23e416b..837f032 100644 --- a/tests/test_schema_one_adapter.py +++ b/tests/test_schema_one_adapter.py @@ -382,8 +382,8 @@ def test_field_metadata_is_empty_before_discovery() -> None: def test_command_topics_address_the_child_device(adapter: SchemaOneAdapter) -> None: """Under parent/child a circuit is its own device, so its command topic is rooted at the circuit rather than nested under the panel.""" - assert adapter.set_circuit_relay_topic(SOLAR_CIRCUIT) == f"ebus/5/{SOLAR_CIRCUIT}/switch/relay/set" - assert adapter.set_circuit_priority_topic(SOLAR_CIRCUIT) == f"ebus/5/{SOLAR_CIRCUIT}/load-shed/priority/set" + assert adapter.set_circuit_relay_target(SOLAR_CIRCUIT).topic == f"ebus/5/{SOLAR_CIRCUIT}/switch/relay/set" + assert adapter.set_circuit_priority_target(SOLAR_CIRCUIT).topic == f"ebus/5/{SOLAR_CIRCUIT}/load-shed/priority/set" def test_dominant_power_source_writes_the_panel_assertion(adapter: SchemaOneAdapter) -> None: @@ -395,7 +395,7 @@ def test_dominant_power_source_writes_the_panel_assertion(adapter: SchemaOneAdap the whole time — `shed/asserted-islanding-state`, `settable=True` — and the adapter simply never named it. """ - assert adapter.set_dominant_power_source_topic() == f"ebus/5/{PANEL}/shed/asserted-islanding-state/set" + assert adapter.set_dominant_power_source_target().topic == f"ebus/5/{PANEL}/shed/asserted-islanding-state/set" def test_the_flat_vocabulary_is_translated_not_forwarded(adapter: SchemaOneAdapter) -> None: diff --git a/tests/test_schema_one_charge_limit.py b/tests/test_schema_one_charge_limit.py index f1045fa..7991c0c 100644 --- a/tests/test_schema_one_charge_limit.py +++ b/tests/test_schema_one_charge_limit.py @@ -20,6 +20,8 @@ import pytest +from conftest import FAST_CONTROL_DEADLINES, acking_bridge + from ebus_sdk.homie import DiscoveredDevice from span_panel_api.exceptions import SpanPanelServerError @@ -299,7 +301,7 @@ def test_the_catalogued_spelling_is_written_to_its_own_topic() -> None: adapter = _adapter(_renamed_to_catalog(EVSE)) key = _key(adapter, EVSE) - assert adapter.set_evse_charge_limit_topic(key) == f"ebus/5/{EVSE}/charge-limit/owner-limit/set" + assert adapter.set_evse_charge_limit_target(key).topic == f"ebus/5/{EVSE}/charge-limit/owner-limit/set" def test_the_catalogued_spelling_wins_where_both_are_declared() -> None: @@ -340,14 +342,16 @@ def test_the_set_topic_addresses_the_device_and_the_declared_property() -> None: key = _key(adapter, EVSE) assert key != EVSE - assert adapter.set_evse_charge_limit_topic(key) == f"ebus/5/{EVSE}/config/user-max-charge-current/set" + assert adapter.set_evse_charge_limit_target(key).topic == f"ebus/5/{EVSE}/config/user-max-charge-current/set" def test_each_charger_gets_its_own_set_topic() -> None: adapter = _adapter() - assert adapter.set_evse_charge_limit_topic(_key(adapter, EVSE)) == (f"ebus/5/{EVSE}/config/user-max-charge-current/set") - assert adapter.set_evse_charge_limit_topic(_key(adapter, EVSE_2)) == ( + assert adapter.set_evse_charge_limit_target(_key(adapter, EVSE)).topic == ( + f"ebus/5/{EVSE}/config/user-max-charge-current/set" + ) + assert adapter.set_evse_charge_limit_target(_key(adapter, EVSE_2)).topic == ( f"ebus/5/{EVSE_2}/config/user-max-charge-current/set" ) @@ -358,7 +362,7 @@ def test_no_topic_for_a_charger_that_does_not_declare_the_limit_settable() -> No adapter = _adapter(_without_settable(EVSE)) key = _key(adapter, EVSE) - assert adapter.set_evse_charge_limit_topic(key) is None + assert adapter.set_evse_charge_limit_target(key) is None assert adapter.evse_charge_limit_payload(key, 16) is None @@ -366,14 +370,14 @@ def test_no_topic_for_a_charger_with_no_charge_limit_node() -> None: adapter = _adapter(_without_node(EVSE)) key = _key(adapter, EVSE) - assert adapter.set_evse_charge_limit_topic(key) is None + assert adapter.set_evse_charge_limit_target(key) is None assert adapter.evse_charge_limit_payload(key, 16) is None def test_no_topic_for_a_charger_the_panel_does_not_have() -> None: adapter = _adapter() - assert adapter.set_evse_charge_limit_topic("not-a-charger") is None + assert adapter.set_evse_charge_limit_target("not-a-charger") is None assert adapter.evse_charge_limit_payload("not-a-charger", 16) is None @@ -429,7 +433,7 @@ def test_a_charger_with_no_ceiling_is_not_second_guessed() -> None: key = _key(adapter, EVSE) assert adapter.evse_charge_limit_payload(key, 1000) == "1000" - assert adapter.set_evse_charge_limit_topic(key) == f"ebus/5/{EVSE}/config/user-max-charge-current/set" + assert adapter.set_evse_charge_limit_target(key).topic == f"ebus/5/{EVSE}/config/user-max-charge-current/set" # --------------------------------------------------------------------------- @@ -441,9 +445,14 @@ def _client(adapter: SchemaOneAdapter) -> tuple[object, MagicMock]: from span_panel_api.mqtt.client import MqttClientConfig, SpanMqttClient config = MqttClientConfig(broker_host="h", username="u", password="p") - client = SpanMqttClient(host="192.168.1.1", serial_number=PANEL, broker_config=config) + client = SpanMqttClient( + host="192.168.1.1", + serial_number=PANEL, + broker_config=config, + control_deadlines=FAST_CONTROL_DEADLINES, + ) client._adapter = adapter - bridge = MagicMock() + bridge = acking_bridge() client._bridge = bridge return client, bridge @@ -457,7 +466,7 @@ async def test_the_transport_publishes_the_topic_and_payload_the_adapter_named() await client.set_evse_charge_limit(key, asked) - bridge.publish.assert_called_once_with(f"ebus/5/{EVSE}/config/user-max-charge-current/set", str(asked), qos=1) + bridge.publish.assert_called_once_with(f"ebus/5/{EVSE}/config/user-max-charge-current/set", str(asked)) @pytest.mark.asyncio diff --git a/tests/test_schema_zero_adapter.py b/tests/test_schema_zero_adapter.py index b567857..2ccd035 100644 --- a/tests/test_schema_zero_adapter.py +++ b/tests/test_schema_zero_adapter.py @@ -39,8 +39,8 @@ def test_subscribes_to_the_single_panel_wildcard(adapter: SchemaZeroAdapter) -> def test_circuit_setter_topics_address_the_panel_device(adapter: SchemaZeroAdapter) -> None: circuit = "ac3dccda46a94b98878a227df6fed588" - assert adapter.set_circuit_relay_topic(circuit) == f"ebus/5/{SERIAL}/{circuit}/relay/set" - assert adapter.set_circuit_priority_topic(circuit) == f"ebus/5/{SERIAL}/{circuit}/shed-priority/set" + assert adapter.set_circuit_relay_target(circuit).topic == f"ebus/5/{SERIAL}/{circuit}/relay/set" + assert adapter.set_circuit_priority_target(circuit).topic == f"ebus/5/{SERIAL}/{circuit}/shed-priority/set" def test_dominant_power_source_topic_is_none_before_the_core_node_is_known( @@ -48,7 +48,7 @@ def test_dominant_power_source_topic_is_none_before_the_core_node_is_known( ) -> None: """The core node id is discovered from $description, so it is unavailable until a description has been routed through handle_message.""" - assert adapter.set_dominant_power_source_topic() is None + assert adapter.set_dominant_power_source_target() is None def test_is_not_ready_before_any_message(adapter: SchemaZeroAdapter) -> None: @@ -85,3 +85,41 @@ def test_schema_zero_presence_follows_the_lugs_fallback() -> None: assert metadata["panel.upstream_l1_current_a"].resolved is False assert metadata["panel.downstream_l2_current_a"].resolved is False assert "circuit.instant_power_w" not in metadata + + +def test_property_callbacks_speak_the_protocol_not_the_accumulator() -> None: + """The shim between the accumulator's tuple and the protocol's is load-bearing + and arity-compatible, which is the dangerous combination. + + The accumulator fires `(node_id, property_id, new_value, old_value)`; the + protocol declares `(device_id, node_id, property_id, value)`. Four strings + either way, so a bare delegation type-checks, runs, and silently feeds a + consumer the node id as a device id and the *previous* value as the current + one. That is the bug this shim fixes, and nothing else pins it: every + publish-outcome test drives a MagicMock adapter, so write-then-verify on a + flat panel -- both `CONFIRMED` and the no-op pre-check, which match on + `(device_id, node_id, property_id)` and compare the reported value -- rests + entirely on these four arguments arriving in this order. + + Two writes rather than one, because a single write cannot distinguish the + new value from the old: the first arrives with no previous value at all. + """ + adapter = SchemaZeroAdapter(serial_number=SERIAL, schema=flat_schema(40)) + seen: list[tuple[str, str, str, str | None]] = [] + unregister = adapter.register_property_callback(lambda d, n, p, v: seen.append((d, n, p, v))) + + adapter.handle_message(f"ebus/5/{SERIAL}/core/power", "100") + adapter.handle_message(f"ebus/5/{SERIAL}/core/power", "200") + + assert seen == [ + (SERIAL, "core", "power", "100"), + (SERIAL, "core", "power", "200"), + ] + # Stated separately from the tuple comparison above, because these are the + # two ways a regression here stays silent rather than failing loudly. + assert seen[1][0] == SERIAL, "the device is the panel serial, never the node id" + assert seen[1][3] == "200", "the fourth argument is the new value, never the previous one" + + unregister() + adapter.handle_message(f"ebus/5/{SERIAL}/core/power", "300") + assert len(seen) == 2 diff --git a/tests/test_ssl_context.py b/tests/test_ssl_context.py index 7e6df0e..84ad1a0 100644 --- a/tests/test_ssl_context.py +++ b/tests/test_ssl_context.py @@ -20,7 +20,7 @@ import pytest -from span_panel_api.mqtt.connection import _build_ssl_context +from span_panel_api._ssl import build_panel_ssl_context cryptography = pytest.importorskip("cryptography", reason="cryptography needed to mint a test CA") @@ -176,7 +176,7 @@ def _serve() -> None: class TestBuildSslContext: def test_loads_ca_without_authority_key_identifier(self) -> None: """The panel's AKI-less CA must load — this is the actual regression.""" - ctx = _build_ssl_context(_self_signed_ca(with_aki=False)) + ctx = build_panel_ssl_context(_self_signed_ca(with_aki=False)) assert ctx.verify_mode is ssl.CERT_REQUIRED assert ctx.check_hostname is True @@ -185,7 +185,7 @@ def test_loads_ca_without_authority_key_identifier(self) -> None: def test_handshake_succeeds_against_panel_style_cert(self) -> None: """End-to-end proof: a TLS handshake completes against an AKI-less chain.""" ca_pem, leaf_pem, leaf_key_pem = _ca_and_leaf(with_aki=False) - ctx = _build_ssl_context(ca_pem) + ctx = build_panel_ssl_context(ca_pem) with _tls_server(leaf_pem, leaf_key_pem) as (host, port): with socket.create_connection((host, port), timeout=5) as raw: @@ -201,7 +201,7 @@ def test_strict_x509_would_reject_the_panel_chain(self) -> None: no longer needed. """ ca_pem, leaf_pem, leaf_key_pem = _ca_and_leaf(with_aki=False) - strict = _build_ssl_context(ca_pem) + strict = build_panel_ssl_context(ca_pem) strict.verify_flags |= ssl.VERIFY_X509_STRICT with _tls_server(leaf_pem, leaf_key_pem) as (host, port): @@ -210,25 +210,25 @@ def test_strict_x509_would_reject_the_panel_chain(self) -> None: strict.wrap_socket(raw, server_hostname="localhost") def test_strict_flag_is_cleared(self) -> None: - ctx = _build_ssl_context(_self_signed_ca(with_aki=False)) + ctx = build_panel_ssl_context(_self_signed_ca(with_aki=False)) assert not (ctx.verify_flags & ssl.VERIFY_X509_STRICT) def test_hostname_and_peer_verification_stay_enabled(self) -> None: """Clearing the strict flag must not weaken the checks that matter.""" - ctx = _build_ssl_context(_self_signed_ca(with_aki=True)) + ctx = build_panel_ssl_context(_self_signed_ca(with_aki=True)) assert ctx.check_hostname is True assert ctx.verify_mode is ssl.CERT_REQUIRED def test_system_ca_bundle_is_not_trusted(self) -> None: """Only the panel CA is a trust anchor — no system roots.""" - ctx = _build_ssl_context(_self_signed_ca(with_aki=False)) + ctx = build_panel_ssl_context(_self_signed_ca(with_aki=False)) assert len(ctx.get_ca_certs()) == 1 def test_conventional_ca_still_loads(self) -> None: - ctx = _build_ssl_context(_self_signed_ca(with_aki=True)) + ctx = build_panel_ssl_context(_self_signed_ca(with_aki=True)) assert ctx.get_ca_certs() def test_malformed_pem_raises(self) -> None: with pytest.raises((ssl.SSLError, ValueError)): - _build_ssl_context("-----BEGIN CERTIFICATE-----\nnot base64\n-----END CERTIFICATE-----\n") + build_panel_ssl_context("-----BEGIN CERTIFICATE-----\nnot base64\n-----END CERTIFICATE-----\n") diff --git a/uv.lock b/uv.lock index a00bbfc..dbb9c06 100644 --- a/uv.lock +++ b/uv.lock @@ -962,7 +962,7 @@ wheels = [ [[package]] name = "span-panel-api" -version = "3.0.1" +version = "3.1.0" source = { editable = "." } dependencies = [ { name = "httpx" }, @@ -1032,7 +1032,7 @@ dev = [ [[package]] name = "span-panel-api-schema-0" -version = "1.0.0" +version = "1.1.0" source = { editable = "packages/schema-0" } dependencies = [ { name = "span-panel-api" }, @@ -1043,7 +1043,7 @@ requires-dist = [{ name = "span-panel-api", editable = "." }] [[package]] name = "span-panel-api-schema-1" -version = "1.0.0" +version = "1.1.0" source = { editable = "packages/schema-1" } dependencies = [ { name = "ebus-sdk" },