Skip to content

XTable performs incorrect schema sync during Delta to Iceberg conversion when a column is renamed in the Delta table.#711

Description

@brishi19791

Search before asking

  • I had searched in the issues and found no similar issues.

Please describe the bug 馃悶

I'm performing a Delta to Iceberg conversion using XTable. I renamed a column in the Delta table from "name" to "FullName". In the generated Iceberg schema, XTable removed the original "name" field (field ID: 2) and added a new field "FullName" with a new field ID: 5. Below is the Iceberg schema XTable generated:
"schemas" : [ {
"type" : "struct",
"schema-id" : 0,
"fields" : [ {
"id" : 1,
"name" : "id",
"required" : false,
"type" : "int"
}, {
"id" : 2,
"name" : "name",
"required" : false,
"type" : "string"
}, {
"id" : 3,
"name" : "role",
"required" : false,
"type" : "string"
}, {
"id" : 4,
"name" : "address",
"required" : false,
"type" : "string"
} ]
}, {
"type" : "struct",
"schema-id" : 1,
"fields" : [ {
"id" : 1,
"name" : "id",
"required" : false,
"type" : "int"
}, {
"id" : 3,
"name" : "role",
"required" : false,
"type" : "string"
}, {
"id" : 4,
"name" : "address",
"required" : false,
"type" : "string"
}, {
"id" : 5,
"name" : "FullName",
"required" : false,
"type" : "string"
} ]
} ]

XTable should have retained the original field ID (2) and simply updated the column name to "FullName" to reflect a rename, not a new field addition.

Are you willing to submit PR?

  • I am willing to submit a PR!
  • I am willing to submit a PR but need help getting started!

Code of Conduct

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions