Python FFI for bundling in zigpy-ziggurat

[puddly]

As a test, this PR abstracts away the serial side and the websocket side of Ziggurat. This lets us treat it as a black box library that sends/receives radio bytes and sends/receives API messages.

The benefit here is that we don't have to run in Ziggurat in a separate process or as an app: we build native Python bindings for Ziggurat and directly embed the stack into the Python radio library. The architecture has zigpy-ziggurat itself own the serial connection but proxy the serial data to/from Ziggurat, using a "websocketless" JSON transport as the API.

The zigpy-ziggurat side is here: https://github.com/zigpy/zigpy-ziggurat/tree/puddly/ziggurat-ffi

[burmistrzak]

I'm wondering how we could add Z2M driver support on our end... 🤔
ZoH for example is just straight TypeScript, so that's easy.

For ziggurat, we could either write a generic WebSocket driver or go the FFI route as well. In terms of performance, the latter is probably preferable.

While a Zigbee stack written in TypeScript might be more approachable, the benefits of a fast, compiled and memory-safe language are hard to beat, IMHO..

[puddly]

For ziggurat, we could either write a generic WebSocket driver or go the FFI route as well. In terms of performance, the latter is probably preferable.

For zigpy, I'm supporting both serial, WebSocket, and WebSocket over UNIX domain sockets: zigpy/zigpy-ziggurat#3. The round trip IPC latency even with WebSocket over localhost TCP is just so minimal that FFI doesn't really provide much of a benefit (and costs complexity with weird wiring).

Stdio mode is I think a middle ground, where Ziggurat takes control of an adapter but sends/receives its wire protocol over standard input and standard output. That'd let you spawn it as a subprocess and not have to deal with FFI complexity on both sides.

[burmistrzak]

For zigpy, I'm supporting both serial, WebSocket, and WebSocket over UNIX domain sockets: zigpy/zigpy-ziggurat#3. The round trip IPC latency even with WebSocket over localhost TCP is just so minimal that FFI doesn't really provide much of a benefit (and costs complexity with weird wiring).

@puddly I think a modular and containerized architecture, a bit similar to how Matter-over-Thread works in HA, would be a godsend for the greater Zigbee community. My pitch:

• One app handles the Zigbee stack (e.g. ziggurat) and exposes a lightweight WebSocket for IPC.
• Another app (Z2M or ZHA) consumes that API and handles ZCL and integration with HA (and/or other systems).

Such an architecture would likely prevent the ecosystem from fracturing further (especially at the stack/adapter level) and foster more collaboration between the two communities. 🥰

[puddly]

Totally! https://github.com/zigpy/addons/tree/dev/ziggurat does just that. I have the addon set up to also run as a normal Docker container.