Add audit log if target dag partition mapper is wrong#62509
Open
Lee-W wants to merge 4 commits intoapache:mainfrom
Open
Add audit log if target dag partition mapper is wrong#62509Lee-W wants to merge 4 commits intoapache:mainfrom
Lee-W wants to merge 4 commits intoapache:mainfrom
Conversation
boring-cyborg
bot
added
the
area:Scheduler
Lee-W
force-pushed
the
add-audit-log-if-target-dag-partition-mapper-is-wrong
branch
from
f3e41fb to
2fe2499
Compare
…by target Dag partition_mapper Thighs needs to be logged in the source Dag since the target Dag won't have task instance created Also revert changes in apache#59375, it does not work.
…mapped by target Dag partition_mapper
…not be mapped by target Dag partition_mapper
…ey can not be mapped by target Dag partition_mapper
Lee-W
force-pushed
the
add-audit-log-if-target-dag-partition-mapper-is-wrong
branch
from
2fe2499 to
90073be
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
If a target partitioned Dag has a partition mapper that cannot map a partition key correctly (e.g., map "abc" through HourlyMapper), it raises an exception in the scheduler which makes sense but not obvious to user.
What
Add AuditLog to the source partitioned Dag if the target partitioned Dag has a partition mapper that cannot map a partition key correctly (e.g., map "abc" through HourlyMapper)
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.