fix: lazy evaluation for coalesce #17357
Conversation
github-actions
bot
added
sqllogictest
| # due to the reason describe in https://github.com/apache/datafusion/issues/8927, | ||
| # the following queries will fail |
There was a problem hiding this comment.
Since the test below now works, I think we could move this comment to the test below this one.
There was a problem hiding this comment.
Thanks @chenkovsky and @nuno-faria -- I think this PR is quite good and probably can be merged. My only potential concern is that we may mess up comet. Let's see if we get any more comments
| 02)--TableScan: t projection=[x, y] | ||
| physical_plan | ||
| 01)ProjectionExec: expr=[coalesce(1, CAST(y@1 / x@0 AS Int64)) as coalesce(Int64(1),t.y / t.x), coalesce(2, CAST(y@1 / x@0 AS Int64)) as coalesce(Int64(2),t.y / t.x)] | ||
| 01)ProjectionExec: expr=[CASE WHEN true THEN 1 ELSE CAST(y@1 / x@0 AS Int64) END as coalesce(Int64(1),t.y / t.x), CASE WHEN true THEN 2 ELSE CAST(y@1 / x@0 AS Int64) END as coalesce(Int64(2),t.y / t.x)] |
There was a problem hiding this comment.
It seems like this should be able to be simplifed even more -- CASE WHEN true THEN ... should go to 1
I filed a ticket to track this idea:
| } | ||
|
|
||
| let n = args.len(); |
There was a problem hiding this comment.
This is quite a clever implementation.
However, I worry it may cause problems for comet which uses physical evaluation directly
@comphead or @andygrove do you know if comet uses the COALESCE implementation directly?
Comet was actually just thinking about coalesce: Maybe @coderfender has some thoughts about this? |
It seems like this PR does the same thing described in the comet PR Thus it seems like a good thing to merge Let's wait a while to see if anyone else has comments, otherwise I'll plan to merge this PR |
|
@alamb , @mbutrovich I made changes to comet to fallback to CASE statement to replicate |
Which issue does this PR close?
Rationale for this change
arguments of coalesce are evaluated eagerly.
What changes are included in this PR?
simplify coalesce to case when expr.
Are these changes tested?
UT
Are there any user-facing changes?
No