-
Notifications
You must be signed in to change notification settings - Fork 1.4k
fix: Ensure ConstantTypedExpr and variant types are compatible #15217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Ensure ConstantTypedExpr and variant types are compatible #15217
Conversation
✅ Deploy Preview for meta-velox canceled.
|
velox/core/Expressions.h
Outdated
| value_{std::move(value)} {} | ||
| value_{std::move(value)} { | ||
| VELOX_CHECK_EQ( | ||
| value_.kind(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This check is not sufficient for complex types. Use isTypeCompatible instead. Please, add tests to ensure that ConstantTypedExpr(ARRAY(VARCHAR()), variant::array({1,2,3}) fails.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @mbasmanova, updated to use isTypeCompatible. Could you PTAL?
| // Constant expr created using variant | ||
| auto nullVariantToExpr = | ||
| std::make_shared<core::ConstantTypedExpr>(VARCHAR(), Variant{}); | ||
| auto nullVariantToExpr = std::make_shared<core::ConstantTypedExpr>( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change is incorrect. Once you use isTypeCompatible it won't be needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Variant{} is initialized with TypeKind::INVALID here, so the null variant needs to be initialized with matching type, otherwise the test fails with below error using isTypeCompatible:
C++ exception with description "Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Expression type does not match variant type
Retriable: False
Expression: value_.isTypeCompatible(ITypedExpr::type())
Function: ConstantTypedExpr
Is this fine?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pramodsatya I see. This is interesting. We may need to look into the use of TypeKind::INVALID. It feels like this shouldn't exist in the first place. Especially since I assume Variant::isNull() returns true for Variant{}. Perhaps, we need another fix to replace TypeKind::INVALID with TypeKind::UNKNOWN in Variant{} constructor or just handle INVALID the same way as UNKNOWN everywhere else. Would you create an issue about this? The change in this PR is fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @mbasmanova, I agree the use of TypeKind::INVALID seems incorrect here and changing to TypeKind::UNKNOWN would be better: pramodsatya@b760e87. Should I open an issue for further discussion or shall I open a PR with this commit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pramodsatya Please, open a PR. Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @mbasmanova, opened #15250, please take a look when you get a chance
3df1b49 to
de923ea
Compare
velox/core/Expressions.h
Outdated
| value_{std::move(value)} { | ||
| VELOX_CHECK( | ||
| value_.isTypeCompatible(ITypedExpr::type()), | ||
| "Expression type does not match variant type"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's include both types in the error message to help troubleshooting?
|
|
||
| TEST_F(ConstantTypedExprTest, variantTypeCheck) { | ||
| VELOX_ASSERT_THROW( | ||
| std::make_shared<core::ConstantTypedExpr>(INTEGER(), "abc"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's introduce a helper lambda and a constant for error message to reduce copy-paste.
VELOX_ASSERT_THROW(makeConstant(INTEGER(), "abc"), ...), kTypeMismatch);
| std::make_shared<core::ConstantTypedExpr>(INTEGER(), "abc"), | ||
| "Expression type does not match variant type"); | ||
| VELOX_ASSERT_THROW( | ||
| std::make_shared<core::ConstantTypedExpr>(INTEGER(), variant(123LL)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add tests that use null variants.
de923ea to
d908948
Compare
|
|
||
| TEST_F(ConstantTypedExprTest, variantTypeCheck) { | ||
| auto testVariantExpr = | ||
| [&](Variant value, const TypePtr type, const TypePtr expectedType) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pass arguments by const &
| "Expression type {} does not match variant type {}", | ||
| type->toString(), | ||
| expectedType->toString())); | ||
| if (type->isPrimitiveType()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you add an else branch to verify that createVariantExpr succeeds?
d908948 to
a017702
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks.
|
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this in D85276445. |
|
@mbasmanova merged this pull request in cf403b8. |
…ookincubator#15217) Summary: Pull Request resolved: facebookincubator#15217 Reviewed By: peterenescu Differential Revision: D85276445 Pulled By: mbasmanova fbshipit-source-id: ddf6a5f25541343d479bf0c00e6d48f3de79da0b
No description provided.