Search before asking
Paimon version
2.0.0-rc4
Compute Engine
Flink 2.2
Minimal reproduce step
Create a Paimon table with Iceberg compatibility enabled, try to read the table from Snowflake.
What doesn't meet your expectations?
Snowflake cannot read the Iceberg metadata, returning this error: Unsupported: external field ID in metadata file cannot be zero.
It looks like Snowflake enforces all field IDs to be > 0 (which is how the Iceberg implementation works), but Paimon generates metadata with field IDs starting from 0, so Snowflake treats that as invalid metadata.
Anything else?
I'm not really sure what the best option is here. Looks like field mappings are encoded in Parquet footers, so maybe an option could be to specify an option at table creation to start all field IDs from 1 rather than 0 in order to produce more universally Iceberg-compatible layout?
Are you willing to submit a PR?
Search before asking
Paimon version
2.0.0-rc4
Compute Engine
Flink 2.2
Minimal reproduce step
Create a Paimon table with Iceberg compatibility enabled, try to read the table from Snowflake.
What doesn't meet your expectations?
Snowflake cannot read the Iceberg metadata, returning this error:
Unsupported: external field ID in metadata file cannot be zero.It looks like Snowflake enforces all field IDs to be > 0 (which is how the Iceberg implementation works), but Paimon generates metadata with field IDs starting from 0, so Snowflake treats that as invalid metadata.
Anything else?
I'm not really sure what the best option is here. Looks like field mappings are encoded in Parquet footers, so maybe an option could be to specify an option at table creation to start all field IDs from 1 rather than 0 in order to produce more universally Iceberg-compatible layout?
Are you willing to submit a PR?