Skip to content

test: add mixed-format rename column coverage#365

Open
liujiwen-up wants to merge 1 commit into
apache:mainfrom
liujiwen-up:fix/258-mixed-format-rename-column
Open

test: add mixed-format rename column coverage#365
liujiwen-up wants to merge 1 commit into
apache:mainfrom
liujiwen-up:fix/258-mixed-format-rename-column

Conversation

@liujiwen-up
Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #258

Add Spark-generated consistency coverage for schema evolution with RENAME COLUMN across mixed file formats.

Old files are written with the original physical column name, while files written after the rename use the new column name. Rust readers should expose the current column name and read values by field id.

Brief change log

  • Add schema_evolution_rename_column to the Spark provisioning script.
  • Write the table across Parquet, ORC, and Avro before/after RENAME COLUMN.
  • Add a Rust integration test covering full reads, projected reads, output column names, and scanned file formats.

Tests

  • cargo fmt: passed
  • cargo test -p paimon arrow::schema_evolution: passed
  • cargo test -p paimon-integration-tests --test read_tables --no-run: passed

API and Format

No public API or storage format changes.

Documentation

No documentation update required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add more consistency file tests from Java data files

1 participant