Fix: Do not normalize table names when deserializing from protobuf (backport to branch-48) #18188
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This fixes a semantic error when reading a logical plan from protobuf. Existing behavior is to use the `relation` field of `ColumnRelation` message to construct a `TableReference`. However, the `relation` field is a string and `From<String> for TableReference` always calls parse_identifiers_normalized with `ignore_case: False`, which always normalizes the identifier to lower case. New behavior is to implement `From<protobuf::ColumnRelation> for TableReference` which calls a new method, `parse_str_normalized`, with `ignore_case: True`. Overall, if normalization occurs, it should happen prior to serialization to protobuf; thus, deserialization from protobuf should not normalize (if it is desirable, though, `parse_str_normalized` propagates its boolean parameter to `parse_identifiers_normalized` unlike `parse_str`). Issue: apache#18122
A new test case, `roundtrip_mixed_case_table_reference`, exercises a scenario where a logical plan containing a table reference with uppercase characters is roundtripped through protobuf and the deserialization side erroneously normalizes the table reference to lowercase. Issue: apache#18122
For parse_identifiers_normalized, the `ignore_case` parameter controls whether the parsing should be case-sensitive (ignore_case: true) or insensitive (ignore_case: false). The name of the parameter is counter-intuitive to the behavior, so this adds a clarifying comment for the method.
github-actions
bot
added
common
|
Closing as we apparently don't need the backport, feel free to reopen if someone wants it |
drin
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
This also addresses #18122 but for branch-48 of datafusion
Rationale for this change
The PR #18187 addresses the fix on main, but for compatibility with iceberg-rust, we also want this fix available for branch-48.
For any other information, please see #18187, which has quite a bit of information.