Implement fbs schema & json to fbs converter tool - #738
Conversation
Thomas-Mikhael
commented
Jul 21, 2026
Thomas-Mikhael
commented
- Define the .fbs schema file that mirrors the existing JSON configuration structure.
- Build a JSON-to-FlatBuffer converter tool for generating FlatBuffer config from existing JSON configurations
- Flag to control compiling the Flatbuffers (--config=flatbuffers), default is off
- PyTest for the tool
+ Define the .fbs schema file that mirrors the existing JSON configuration structure. + Build a JSON-to-FlatBuffer converter tool for generating FlatBuffer config from existing JSON configurations + Flag to control compiling the Flatbuffers (--config=flatbuffers) + PyTest for the tool
a2ca224 to
2510ac0
Compare
|
As discussed via Slack - I love that we go for the Flatbuffers support and the contribution you perform here! But it would be very welcome to see the roadmap and map for its qualification before continuing here. Stating again: This does not require a full and ready qualification, we would just like to judge if the qualification at the end can succeed. |
Hi Jan, I agree. |
castler
left a comment
castler
left a comment
There was a problem hiding this comment.
Thanks this is going into the right direction.
I still believe that we need a flatbuffers certifcation for the 1.0 - and not only in 2027. And we should have a parallel workstream with weekly (or bi-weekly) reporting about the tasks there.
Anyhow - for this PR, my major concern is that we know have 4 places where we need to know the different schemas:
- In the JSON Parser (soonish also flatbuffers parser +1)
- In the JSON Schema
- In the Flatbuffers Schema
- In the JSON to Flatbuffers converter
I think this is to much duplication and will make it not maintainable. We should strive to have a generic flatbuffers schema generatioin and also a generci json to flatbuffers converter - to reduce the duplication as much as possible.
|
|
||
| # Enables FlatBuffers-based mw::com configuration tooling (disabled by default, | ||
| # see score/mw/com/impl/configuration/flatbuffers_flags.bzl). | ||
| common:flatbuffers --//score/mw/com/impl/configuration:enable_flatbuffers=true |
There was a problem hiding this comment.
Can we please name this experimental_enable_flatbuffer_configuration
| //score/mw/com/impl/configuration/converter:json_to_flatbuffer | ||
| //score/mw/com/impl/configuration/converter:mw_com_config.bin |
There was a problem hiding this comment.
We should guard this via the experimental flag - and since the experimental flag is per default off, these targets should net yet be publicly visible.
| the pinned ``@flatbuffers//:flatc`` compiler, so ``.bin`` artifacts become part | ||
| of the build graph and can be consumed by other targets (e.g. as ``data``). | ||
| """ | ||
|
|
There was a problem hiding this comment.
you are missing a visibility declaration in this *.bzl file
| // | ||
| // FlatBuffers schema mirroring the mw::com runtime configuration. | ||
| // | ||
| // This schema is the binary-serialization counterpart of the JSON configuration |
There was a problem hiding this comment.
I would have expected that we generate this file out of the JSON schema - otherwise, we get a problem with maintainability - as we have to take care in various places the the configuration is maintained.
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
| """Convert an mw::com configuration JSON file into a FlatBuffer binary. |
There was a problem hiding this comment.
My hope is, that if we auto generate the *.fbs, we also get rid of this tool (or make it way more easy - as we do not have yet another place where we translate between the two tools
2 participants
