branch-4.1: [improvement](compaction) split time series compaction score metric #63900 - #67607
Open
Yukang-Lian wants to merge 1 commit into
Open
Conversation
…pache#63900) ### What problem does this PR solve? Before this change, BE exposes only the aggregate cumulative compaction score: ```bash curl -s http://<be_host>:<webserver_port>/metrics \ | grep 'doris_be_tablet_cumulative_max_compaction_score' ``` The aggregate value cannot show which cumulative compaction policy contributes the maximum score. After this change, observe the aggregate and per-policy scores with: ```bash curl -s http://<be_host>:<webserver_port>/metrics \ | grep -E 'doris_be_tablet_(cumulative|size_based|time_series)_max_compaction_score' ``` | Metric | Meaning | | --- | --- | | `doris_be_tablet_cumulative_max_compaction_score` | Maximum score across all cumulative compaction policies; semantics unchanged | | `doris_be_tablet_size_based_max_compaction_score` | Maximum score from the size-based policy | | `doris_be_tablet_time_series_max_compaction_score` | Maximum score from the time-series policy | New behavior: - Local and Cloud BEs publish the two per-policy metrics. - The existing aggregate metric and BE report value `max(base, cumulative)` remain unchanged. - Compaction score calculation, tablet ranking, and tablet selection are unchanged. - A complete score check can reset a missing policy score to zero; an incomplete local scan caused by capacity-limited disks does not lower the cumulative score metrics. ### Release note Add separate size-based and time-series cumulative compaction score metrics while preserving the existing aggregate metric.
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Collaborator
Author
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: #63900
Problem Summary: Backport the compaction score metric split from #63900 to
branch-4.1.The change keeps the existing cumulative score as the aggregate maximum and adds separate size-based and time-series policy gauges for both local and Cloud BEs. Score collection remains independent from whether a tablet is selected for execution. Complete checks may clear a missing policy to zero, while incomplete local scans caused by capacity-limited disks do not lower the gauges.
The backport preserves this branch's
std::vector<TabletSharedPtr>scheduling API and its existing compaction eligibility behavior. This release branch has no cumulative-binlog compaction score path, so the master-only binlog clauses and tests are intentionally omitted.The branch is rebased onto the latest
branch-4.1.Release note
Add separate size-based and time-series cumulative compaction score metrics while preserving the aggregate metric.
Check List (For Author)
clang-format16--dry-run --Werror: passed for all changed C++ filesgit diff --check: passedrun buildall