Filter by `maxOtherDoc` first to reduce the overhead of `filterCompetitiveHits` #15081

HUSTERGS · 2025-08-16T00:55:52Z

Description

This PR propose to filter the docAndScoreBuffer by maxOtherDoc, so the cost following filterCompetitiveHits might be reduced since the elements are less. Here is the result under wikimediumall after 20 iterations:

                            TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
              CombinedAndHighMed       19.94      (4.9%)       19.55      (4.7%)   -1.9% ( -11% -    8%) 0.243
               CombinedOrHighMed       19.39      (6.0%)       19.02      (4.4%)   -1.9% ( -11% -    9%) 0.295
      FilteredOr2Terms2StopWords       46.08      (5.7%)       45.20      (4.8%)   -1.9% ( -11% -    9%) 0.291
                       CountTerm     5513.24      (2.5%)     5419.88      (2.7%)   -1.7% (  -6% -    3%) 0.057
                      OrHighRare       90.97      (6.5%)       89.47      (7.2%)   -1.6% ( -14% -   12%) 0.485
               FilteredOrHighMed       36.35      (4.9%)       35.76      (4.0%)   -1.6% (  -9% -    7%) 0.289
                FilteredOr3Terms       41.11      (4.6%)       40.46      (3.8%)   -1.6% (  -9% -    7%) 0.276
             CountFilteredOrMany        4.32      (3.0%)        4.25      (2.8%)   -1.6% (  -7% -    4%) 0.114
                          IntSet      287.97      (3.3%)      283.60      (4.4%)   -1.5% (  -8% -    6%) 0.257
                     CountOrMany        4.86      (2.9%)        4.79      (2.8%)   -1.4% (  -6% -    4%) 0.157
             And2Terms2StopWords       52.86      (7.4%)       52.16      (7.1%)   -1.3% ( -14% -   14%) 0.594
              FilteredOrHighHigh       12.39      (3.9%)       12.23      (2.6%)   -1.3% (  -7% -    5%) 0.257
                    FilteredTerm       60.82      (2.6%)       60.05      (3.1%)   -1.3% (  -6% -    4%) 0.191
                          OrMany        4.11      (5.2%)        4.05      (4.5%)   -1.3% ( -10% -    8%) 0.443
             FilteredAndHighHigh       10.33      (3.0%)       10.20      (2.2%)   -1.2% (  -6% -    4%) 0.169
                 DismaxOrHighMed       44.93      (4.4%)       44.38      (3.3%)   -1.2% (  -8% -    6%) 0.359
                          IntNRQ       42.96      (3.0%)       42.44      (3.1%)   -1.2% (  -7% -    5%) 0.244
                        Wildcard       46.61      (3.5%)       46.05      (2.0%)   -1.2% (  -6% -    4%) 0.216
               FilteredAnd3Terms       99.45      (3.8%)       98.27      (2.7%)   -1.2% (  -7% -    5%) 0.291
                  FilteredIntNRQ       42.57      (3.1%)       42.09      (3.0%)   -1.1% (  -6% -    5%) 0.270
                  FilteredOrMany        3.85      (3.6%)        3.80      (2.8%)   -1.1% (  -7% -    5%) 0.310
                      TermDTSort      138.65      (4.0%)      137.10      (3.6%)   -1.1% (  -8% -    6%) 0.389
             CountFilteredPhrase        8.66      (3.9%)        8.56      (3.7%)   -1.1% (  -8% -    6%) 0.396
            FilteredAndStopWords        8.30      (3.2%)        8.21      (2.4%)   -1.1% (  -6% -    4%) 0.280
               TermDayOfYearSort      254.17      (1.8%)      251.60      (2.4%)   -1.0% (  -5% -    3%) 0.168
     FilteredAnd2Terms2StopWords       55.64      (5.5%)       55.09      (4.9%)   -1.0% ( -10% -    9%) 0.583
                 CountOrHighHigh       49.57      (1.8%)       49.12      (1.8%)   -0.9% (  -4% -    2%) 0.140
                    SloppyPhrase        1.12      (3.9%)        1.11      (3.4%)   -0.9% (  -7% -    6%) 0.486
                CountAndHighHigh       48.49      (2.0%)       48.06      (1.8%)   -0.9% (  -4% -    2%) 0.177
                 AndHighOrMedMed       13.72      (3.1%)       13.61      (3.3%)   -0.8% (  -7% -    5%) 0.446
              FilteredAndHighMed       30.43      (3.1%)       30.19      (2.5%)   -0.8% (  -6% -    4%) 0.403
                         Respell       34.92      (3.7%)       34.65      (3.4%)   -0.8% (  -7% -    6%) 0.526
             FilteredOrStopWords        7.82      (3.7%)        7.76      (2.3%)   -0.8% (  -6% -    5%) 0.473
                    CombinedTerm       10.69      (3.5%)       10.61      (5.0%)   -0.7% (  -8% -    8%) 0.627
                  CountOrHighMed       71.71      (2.5%)       71.20      (1.2%)   -0.7% (  -4% -    3%) 0.287
                          Fuzzy2       33.77      (4.4%)       33.54      (4.2%)   -0.7% (  -8% -    8%) 0.651
              CombinedOrHighHigh        5.44      (4.1%)        5.41      (3.6%)   -0.6% (  -7% -    7%) 0.641
             CombinedAndHighHigh        5.62      (1.1%)        5.59      (1.4%)   -0.6% (  -3% -    1%) 0.158
                  FilteredPhrase        9.36      (2.9%)        9.31      (3.0%)   -0.6% (  -6% -    5%) 0.552
             CountFilteredIntNRQ       16.42      (1.3%)       16.33      (1.2%)   -0.6% (  -3% -    1%) 0.163
                   TermMonthSort     2066.73      (1.9%)     2055.02      (2.6%)   -0.6% (  -4% -    3%) 0.465
                          Phrase        7.38      (2.8%)        7.35      (2.8%)   -0.5% (  -5% -    5%) 0.611
                IntervalsOrdered        2.42      (3.1%)        2.41      (4.0%)   -0.5% (  -7% -    6%) 0.712
                 CountAndHighMed       70.28      (2.9%)       69.97      (1.8%)   -0.4% (  -4% -    4%) 0.596
         CountFilteredOrHighHigh       15.79      (1.0%)       15.73      (0.8%)   -0.4% (  -2% -    1%) 0.175
          CountFilteredOrHighMed       17.88      (0.8%)       17.82      (0.7%)   -0.3% (  -1% -    1%) 0.203
                          Fuzzy1       37.19      (5.0%)       37.09      (4.3%)   -0.3% (  -9% -    9%) 0.857
                      DismaxTerm      446.12      (2.8%)      445.45      (3.7%)   -0.2% (  -6% -    6%) 0.895
                         Term100      398.15      (3.8%)      397.58      (5.0%)   -0.1% (  -8% -    9%) 0.926
                         Term10K      397.62      (3.7%)      397.06      (5.1%)   -0.1% (  -8% -    9%) 0.926
                        SpanNear        2.47      (4.4%)        2.47      (4.5%)   -0.1% (  -8% -    9%) 0.932
                         TermB1M      397.33      (3.8%)      397.16      (5.1%)   -0.0% (  -8% -    9%) 0.979
                            Term      396.99      (3.8%)      396.98      (5.0%)   -0.0% (  -8% -    9%) 0.998
                          Term1M      397.20      (3.7%)      397.21      (5.1%)    0.0% (  -8% -    9%) 0.998
                       TermB1M1P      397.20      (3.8%)      397.27      (5.2%)    0.0% (  -8% -    9%) 0.991
                       And3Terms       66.67      (5.1%)       66.69      (4.7%)    0.0% (  -9% -   10%) 0.981
                         Prefix3       73.64      (4.2%)       73.72      (3.8%)    0.1% (  -7% -    8%) 0.937
                 FilteredPrefix3       68.69      (3.8%)       68.87      (3.5%)    0.3% (  -6% -    7%) 0.840
              Or2Terms2StopWords       52.85      (6.4%)       53.02      (6.4%)    0.3% ( -11% -   14%) 0.884
                AndMedOrHighHigh       15.57      (2.7%)       15.64      (2.4%)    0.4% (  -4% -    5%) 0.614
                     CountPhrase        2.58      (4.8%)        2.60      (4.4%)    0.5% (  -8% -   10%) 0.731
                    AndStopWords        8.30      (3.6%)        8.38      (3.0%)    1.0% (  -5% -    7%) 0.376
                   TermTitleSort       50.31      (4.6%)       51.27      (4.4%)    1.9% (  -6% -   11%) 0.216
                        Or3Terms       58.12      (4.4%)       59.26      (4.7%)    2.0% (  -6% -   11%) 0.209
                      AndHighMed       48.60      (4.9%)       49.56      (4.4%)    2.0% (  -6% -   11%) 0.215
                       OrHighMed       57.57      (4.9%)       58.85      (5.3%)    2.2% (  -7% -   13%) 0.208
                DismaxOrHighHigh       31.71      (3.5%)       32.49      (3.5%)    2.5% (  -4% -    9%) 0.039
                     OrStopWords        8.07      (3.4%)        8.34      (3.7%)    3.4% (  -3% -   10%) 0.005
                     AndHighHigh       19.66      (6.2%)       20.65      (4.7%)    5.0% (  -5% -   17%) 0.008
                      OrHighHigh       17.70      (3.6%)       18.73      (4.6%)    5.8% (  -2% -   14%) 0.000

github-actions · 2025-08-16T00:56:44Z

This PR does not have an entry in lucene/CHANGES.txt. Consider adding one. If the PR doesn't need a changelog entry, then add the skip-changelog label to it and you will stop receiving this reminder on future updates to the PR.

HUSTERGS · 2025-08-16T16:16:44Z

I did some experiment, if I only filter docs when minCompetitiveScore > 0 (which means this change should only affect filterCompetitiveHit, set aside the extra branching, since if the minCompetitveScore == 0f, it essentially did nothing), the speedup disappeared, so my previous thought about this change is helpful for filterCompetitiveHit should be wrong

HUSTERGS · 2025-08-16T23:44:47Z

Since I'm little bit confused by the cause of performance gain, I re-run the util to double-check it, here is the second benchmark result:

                            TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                      OrHighRare       91.58      (5.4%)       90.35      (7.9%)   -1.3% ( -13% -   12%) 0.527
                         Prefix3       74.58      (3.6%)       73.61      (4.8%)   -1.3% (  -9% -    7%) 0.336
                     CountPhrase        2.66      (2.1%)        2.62      (3.9%)   -1.3% (  -7% -    4%) 0.188
                 FilteredPrefix3       69.51      (3.4%)       68.76      (4.6%)   -1.1% (  -8% -    7%) 0.400
                       TermB1M1P      397.72      (6.2%)      395.01      (6.7%)   -0.7% ( -12% -   13%) 0.738
                            Term      397.58      (6.1%)      395.33      (6.8%)   -0.6% ( -12% -   13%) 0.782
                         Term100      397.36      (6.1%)      395.17      (6.8%)   -0.6% ( -12% -   13%) 0.787
                         Term10K      397.53      (6.2%)      395.35      (6.7%)   -0.5% ( -12% -   13%) 0.787
                    CombinedTerm       10.82      (3.5%)       10.77      (4.1%)   -0.4% (  -7% -    7%) 0.721
                IntervalsOrdered        2.44      (2.6%)        2.43      (3.6%)   -0.4% (  -6% -    5%) 0.676
                          Term1M      397.47      (6.0%)      395.92      (7.0%)   -0.4% ( -12% -   13%) 0.850
                         TermB1M      397.18      (6.1%)      395.64      (6.9%)   -0.4% ( -12% -   13%) 0.850
              CombinedOrHighHigh        5.49      (3.5%)        5.47      (3.7%)   -0.4% (  -7% -    7%) 0.737
                     CountOrMany        4.80      (3.4%)        4.79      (3.2%)   -0.4% (  -6% -    6%) 0.740
                   TermTitleSort       51.38      (4.2%)       51.22      (4.6%)   -0.3% (  -8% -    8%) 0.827
                        SpanNear        2.46      (5.0%)        2.45      (6.1%)   -0.3% ( -10% -   11%) 0.867
                CountAndHighHigh       48.08      (2.1%)       48.00      (1.9%)   -0.2% (  -4% -    3%) 0.769
                 CountOrHighHigh       49.17      (2.6%)       49.09      (2.3%)   -0.2% (  -4% -    4%) 0.817
             CountFilteredOrMany        4.28      (2.8%)        4.27      (2.9%)   -0.2% (  -5% -    5%) 0.845
                        Wildcard       46.51      (2.1%)       46.46      (3.1%)   -0.1% (  -5% -    5%) 0.910
            FilteredAndStopWords        8.22      (2.2%)        8.22      (3.0%)   -0.0% (  -5% -    5%) 0.970
         CountFilteredOrHighHigh       15.76      (0.8%)       15.76      (0.9%)   -0.0% (  -1% -    1%) 0.996
          CountFilteredOrHighMed       17.84      (0.7%)       17.85      (0.8%)    0.0% (  -1% -    1%) 0.920
             FilteredOrStopWords        7.85      (2.3%)        7.85      (3.0%)    0.0% (  -5% -    5%) 0.967
                  CountOrHighMed       71.51      (2.1%)       71.55      (2.2%)    0.1% (  -4% -    4%) 0.935
             CombinedAndHighHigh        5.61      (1.3%)        5.61      (1.6%)    0.1% (  -2% -    2%) 0.899
                 CountAndHighMed       70.68      (2.8%)       70.75      (2.6%)    0.1% (  -5% -    5%) 0.906
                    SloppyPhrase        1.11      (3.8%)        1.11      (4.0%)    0.1% (  -7% -    8%) 0.920
             FilteredAndHighHigh       10.22      (2.4%)       10.23      (3.3%)    0.1% (  -5% -    6%) 0.886
                          Phrase        7.39      (2.2%)        7.40      (2.3%)    0.2% (  -4% -    4%) 0.772
                  FilteredOrMany        3.86      (3.4%)        3.87      (3.4%)    0.2% (  -6% -    7%) 0.830
             CountFilteredIntNRQ       16.33      (1.2%)       16.37      (1.1%)    0.2% (  -2% -    2%) 0.516
                          IntNRQ       42.56      (2.7%)       42.66      (2.7%)    0.2% (  -5% -    5%) 0.776
                      DismaxTerm      448.07      (4.6%)      449.36      (3.8%)    0.3% (  -7% -    9%) 0.828
               TermDayOfYearSort      252.26      (2.2%)      253.08      (1.4%)    0.3% (  -3% -    4%) 0.577
               CombinedOrHighMed       19.30      (5.5%)       19.37      (5.9%)    0.3% ( -10% -   12%) 0.846
                  FilteredIntNRQ       42.16      (2.7%)       42.31      (2.6%)    0.4% (  -4% -    5%) 0.656
             CountFilteredPhrase        8.58      (3.4%)        8.61      (3.8%)    0.4% (  -6% -    7%) 0.728
              FilteredOrHighHigh       12.35      (2.9%)       12.40      (3.3%)    0.4% (  -5% -    6%) 0.684
                          OrMany        4.06      (6.2%)        4.08      (6.6%)    0.4% ( -11% -   14%) 0.838
                 AndHighOrMedMed       13.49      (2.7%)       13.56      (3.0%)    0.5% (  -5% -    6%) 0.593
                FilteredOr3Terms       41.01      (4.2%)       41.22      (4.4%)    0.5% (  -7% -    9%) 0.713
                          Fuzzy2       33.72      (4.4%)       33.89      (5.0%)    0.5% (  -8% -   10%) 0.734
                       CountTerm     5519.28      (2.6%)     5547.79      (2.1%)    0.5% (  -4% -    5%) 0.491
                    FilteredTerm       60.30      (2.3%)       60.63      (3.4%)    0.6% (  -5% -    6%) 0.544
                          IntSet      286.83      (3.3%)      288.51      (4.0%)    0.6% (  -6% -    8%) 0.611
               FilteredOrHighMed       36.14      (4.2%)       36.36      (4.6%)    0.6% (  -7% -    9%) 0.657
                         Respell       34.65      (3.8%)       34.87      (4.0%)    0.6% (  -6% -    8%) 0.610
      FilteredOr2Terms2StopWords       45.74      (5.2%)       46.05      (5.4%)    0.7% (  -9% -   11%) 0.692
                          Fuzzy1       37.23      (5.0%)       37.48      (5.1%)    0.7% (  -8% -   11%) 0.673
              CombinedAndHighMed       19.64      (4.8%)       19.77      (5.3%)    0.7% (  -8% -   11%) 0.671
                      TermDTSort      136.72      (3.4%)      137.70      (3.5%)    0.7% (  -5% -    7%) 0.511
               FilteredAnd3Terms       99.36      (3.7%)      100.18      (3.6%)    0.8% (  -6% -    8%) 0.475
                   TermMonthSort     2067.37      (2.0%)     2085.00      (1.8%)    0.9% (  -2% -    4%) 0.161
                  FilteredPhrase        9.33      (2.8%)        9.43      (3.5%)    1.0% (  -5% -    7%) 0.308
                 DismaxOrHighMed       44.36      (6.9%)       44.81      (4.6%)    1.0% (  -9% -   13%) 0.577
                AndMedOrHighHigh       15.36      (4.2%)       15.54      (3.3%)    1.1% (  -6% -    9%) 0.353
              FilteredAndHighMed       30.00      (4.0%)       30.46      (3.2%)    1.5% (  -5% -    9%) 0.183
     FilteredAnd2Terms2StopWords       54.75      (6.0%)       55.79      (5.3%)    1.9% (  -8% -   13%) 0.286
             And2Terms2StopWords       51.39      (8.7%)       52.87      (6.9%)    2.9% ( -11% -   20%) 0.248
                    AndStopWords        8.16      (6.3%)        8.40      (1.8%)    2.9% (  -4% -   11%) 0.046
              Or2Terms2StopWords       51.97      (7.9%)       53.69      (6.8%)    3.3% ( -10% -   19%) 0.155
                       And3Terms       65.09      (8.6%)       67.50      (4.4%)    3.7% (  -8% -   18%) 0.087
                DismaxOrHighHigh       31.54      (4.9%)       32.74      (4.4%)    3.8% (  -5% -   13%) 0.010
                     OrStopWords        8.08      (6.5%)        8.41      (3.7%)    4.1% (  -5% -   15%) 0.015
                        Or3Terms       57.13      (8.6%)       59.89      (5.1%)    4.8% (  -8% -   20%) 0.031
                      AndHighMed       47.53      (9.3%)       49.88      (3.2%)    4.9% (  -6% -   19%) 0.024
                       OrHighMed       56.33      (9.6%)       59.54      (5.3%)    5.7% (  -8% -   22%) 0.020
                     AndHighHigh       19.41      (8.9%)       20.76      (2.4%)    7.0% (  -3% -   19%) 0.001
                      OrHighHigh       17.52      (8.1%)       18.91      (4.5%)    7.9% (  -4% -   22%) 0.000

I suspect that pre-filter doc can help because it reduces some useless advance of some clauses, since they now have some kind of global view (docs filtered by maxOtherDoc), so the applyRequiredClause always makes some true progress.

jpountz · 2025-08-17T20:33:22Z

This is interesting, this suggests that we're scoring too many docs with the leading clause. With your change, we're skipping these docs after scoring them with the first clause, but we should be able to skip these docs before scoring them as well? With something like that:

diff --git a/lucene/core/src/java/org/apache/lucene/search/BlockMaxConjunctionBulkScorer.java b/lucene/core/src/java/org/apache/lucene/search/BlockMaxConjunctionBulkScorer.java
index 21f8af990b8..c2030dbc8fb 100644
--- a/lucene/core/src/java/org/apache/lucene/search/BlockMaxConjunctionBulkScorer.java
+++ b/lucene/core/src/java/org/apache/lucene/search/BlockMaxConjunctionBulkScorer.java
@@ -178,6 +178,7 @@ final class BlockMaxConjunctionBulkScorer extends BulkScorer {
 
       docAndScoreAccBuffer.copyFrom(docAndScoreBuffer);
 
+      int maxOtherDoc = -1;
       for (int i = 1; i < scorers.length; ++i) {
         double sumOfOtherClause = sumOfOtherClauses[i];
         if (sumOfOtherClause != sumOfOtherClauses[i - 1]) {
@@ -188,20 +189,17 @@ final class BlockMaxConjunctionBulkScorer extends BulkScorer {
         }
 
         ScorerUtil.applyRequiredClause(docAndScoreAccBuffer, iterators[i], scorables[i]);
+        maxOtherDoc = Math.max(iterators[i].docID(), maxOtherDoc);
       }
 
       for (int i = 0; i < docAndScoreAccBuffer.size; ++i) {
         scorable.score = (float) docAndScoreAccBuffer.scores[i];
         collector.collect(docAndScoreAccBuffer.docs[i]);
       }
-    }
 
-    int maxOtherDoc = -1;
-    for (int i = 1; i < iterators.length; ++i) {
-      maxOtherDoc = Math.max(iterators[i].docID(), maxOtherDoc);
-    }
-    if (lead.docID() < maxOtherDoc) {
-      lead.advance(maxOtherDoc);
+      if (lead.docID() < maxOtherDoc) {
+        lead.advance(maxOtherDoc);
+      }
     }
   }

HUSTERGS · 2025-08-18T02:56:45Z

but we should be able to skip these docs before scoring them as well?

Yeah, I remember I tried something like that before, didn't workout well, I'll run the luceneutil to verify again.

HUSTERGS · 2025-08-18T07:33:31Z

I'll run the luceneutil to verify again. Here is the result, based on https://github.com/HUSTERGS/lucene/tree/bench_advance, hope I'm not get anything wrong

                            TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                 FilteredPrefix3       68.14      (4.6%)       67.56      (5.1%)   -0.8% ( -10% -    9%) 0.578
                         Prefix3       73.02      (5.6%)       72.44      (5.5%)   -0.8% ( -11% -   10%) 0.651
                 AndHighOrMedMed       13.32      (3.5%)       13.25      (4.3%)   -0.5% (  -8% -    7%) 0.687
                      OrHighHigh       17.49     (10.0%)       17.41      (9.4%)   -0.5% ( -18% -   21%) 0.882
                          IntSet      283.49      (4.4%)      282.71      (5.6%)   -0.3% (  -9% -   10%) 0.864
                     OrStopWords        7.97      (8.2%)        7.95      (8.3%)   -0.2% ( -15% -   17%) 0.952
                   TermTitleSort       50.51      (7.6%)       50.46      (8.5%)   -0.1% ( -15% -   17%) 0.969
                        Wildcard       45.77      (4.0%)       45.76      (3.7%)   -0.0% (  -7% -    7%) 0.984
                IntervalsOrdered        2.39      (4.7%)        2.39      (4.3%)   -0.0% (  -8% -    9%) 0.988
          CountFilteredOrHighMed       17.72      (1.5%)       17.71      (1.4%)   -0.0% (  -2% -    2%) 0.981
         CountFilteredOrHighHigh       15.64      (1.8%)       15.64      (1.6%)    0.0% (  -3% -    3%) 0.969
                CountAndHighHigh       47.62      (3.2%)       47.64      (2.9%)    0.0% (  -5% -    6%) 0.974
                     CountPhrase        2.60      (4.5%)        2.61      (4.4%)    0.0% (  -8% -    9%) 0.980
                 CountAndHighMed       70.08      (3.0%)       70.10      (3.9%)    0.0% (  -6% -    7%) 0.973
             CountFilteredIntNRQ       16.20      (1.7%)       16.23      (1.8%)    0.1% (  -3% -    3%) 0.792
                    FilteredTerm       59.72      (3.7%)       59.85      (3.9%)    0.2% (  -7% -    8%) 0.859
               TermDayOfYearSort      247.90      (3.9%)      248.46      (4.5%)    0.2% (  -7% -    8%) 0.866
                          Phrase        7.22      (3.4%)        7.24      (3.6%)    0.2% (  -6% -    7%) 0.827
                    CombinedTerm       10.65      (5.7%)       10.68      (4.9%)    0.2% (  -9% -   11%) 0.884
                  CountOrHighMed       70.80      (2.4%)       71.00      (2.8%)    0.3% (  -4% -    5%) 0.724
                  FilteredIntNRQ       41.43      (3.1%)       41.56      (3.3%)    0.3% (  -5% -    6%) 0.752
                   TermMonthSort     2008.62      (5.9%)     2015.27      (6.0%)    0.3% ( -10% -   12%) 0.860
                 CountOrHighHigh       48.55      (3.3%)       48.73      (2.4%)    0.4% (  -5% -    6%) 0.686
                     CountOrMany        4.70      (4.7%)        4.72      (4.0%)    0.4% (  -7% -    9%) 0.776
            FilteredAndStopWords        8.13      (3.1%)        8.17      (3.1%)    0.4% (  -5% -    6%) 0.689
                       OrHighMed       55.43     (10.6%)       55.65     (10.4%)    0.4% ( -18% -   23%) 0.904
                  FilteredPhrase        9.16      (4.0%)        9.20      (3.8%)    0.4% (  -7% -    8%) 0.733
                          IntNRQ       41.73      (3.2%)       41.91      (3.4%)    0.4% (  -5% -    7%) 0.682
                    SloppyPhrase        1.08      (6.6%)        1.09      (6.4%)    0.5% ( -11% -   14%) 0.815
             CountFilteredOrMany        4.19      (4.4%)        4.21      (3.6%)    0.5% (  -7% -    8%) 0.699
             FilteredAndHighHigh       10.13      (2.8%)       10.19      (3.0%)    0.6% (  -5% -    6%) 0.542
                      TermDTSort      134.03      (4.0%)      134.79      (4.6%)    0.6% (  -7% -    9%) 0.679
                  FilteredOrMany        3.79      (2.7%)        3.82      (3.8%)    0.6% (  -5% -    7%) 0.573
                        SpanNear        2.41      (6.7%)        2.42      (6.9%)    0.6% ( -12% -   15%) 0.775
                         Respell       34.21      (2.9%)       34.43      (3.4%)    0.6% (  -5% -    7%) 0.521
                      DismaxTerm      443.25      (7.0%)      446.24      (6.8%)    0.7% ( -12% -   15%) 0.757
             CountFilteredPhrase        8.47      (3.5%)        8.53      (3.3%)    0.7% (  -5% -    7%) 0.502
              FilteredOrHighHigh       12.14      (2.8%)       12.23      (3.7%)    0.7% (  -5% -    7%) 0.481
                FilteredOr3Terms       40.17      (3.0%)       40.47      (4.2%)    0.7% (  -6% -    8%) 0.514
             FilteredOrStopWords        7.72      (2.9%)        7.78      (3.4%)    0.8% (  -5% -    7%) 0.431
                          Fuzzy2       33.07      (4.4%)       33.34      (4.9%)    0.8% (  -8% -   10%) 0.588
               FilteredAnd3Terms       97.61      (3.3%)       98.40      (4.3%)    0.8% (  -6% -    8%) 0.510
             CombinedAndHighHigh        5.51      (4.6%)        5.55      (4.2%)    0.8% (  -7% -   10%) 0.548
                 DismaxOrHighMed       43.24      (7.7%)       43.61      (7.8%)    0.8% ( -13% -   17%) 0.729
              CombinedOrHighHigh        5.39      (5.6%)        5.44      (4.8%)    0.9% (  -9% -   12%) 0.595
              Or2Terms2StopWords       50.73      (8.5%)       51.20      (8.7%)    0.9% ( -14% -   19%) 0.734
                         Term10K      397.11      (8.3%)      400.79      (6.8%)    0.9% ( -13% -   17%) 0.700
               CombinedOrHighMed       18.74      (6.0%)       18.92      (4.6%)    0.9% (  -9% -   12%) 0.579
               FilteredOrHighMed       35.39      (3.6%)       35.72      (4.2%)    0.9% (  -6% -    8%) 0.439
      FilteredOr2Terms2StopWords       44.69      (4.0%)       45.12      (5.2%)    1.0% (  -7% -   10%) 0.507
                        Or3Terms       56.27      (9.1%)       56.84      (9.2%)    1.0% ( -15% -   21%) 0.728
              CombinedAndHighMed       19.12      (4.5%)       19.31      (5.0%)    1.0% (  -8% -   11%) 0.499
                          Term1M      397.05      (8.7%)      401.40      (6.3%)    1.1% ( -12% -   17%) 0.650
                          OrMany        3.96      (6.2%)        4.00      (6.8%)    1.1% ( -11% -   15%) 0.594
                AndMedOrHighHigh       15.22      (4.8%)       15.39      (4.1%)    1.1% (  -7% -   10%) 0.436
                         TermB1M      396.44      (8.8%)      400.87      (6.6%)    1.1% ( -13% -   18%) 0.648
                       TermB1M1P      397.07      (8.5%)      401.57      (6.2%)    1.1% ( -12% -   17%) 0.630
                          Fuzzy1       36.35      (5.0%)       36.78      (5.0%)    1.2% (  -8% -   11%) 0.460
                            Term      396.76      (8.6%)      401.50      (6.9%)    1.2% ( -13% -   18%) 0.626
                         Term100      396.43      (8.5%)      401.41      (6.5%)    1.3% ( -12% -   17%) 0.601
                DismaxOrHighHigh       31.12      (6.2%)       31.52      (6.2%)    1.3% ( -10% -   14%) 0.514
              FilteredAndHighMed       29.61      (4.3%)       30.08      (3.2%)    1.6% (  -5% -    9%) 0.186
     FilteredAnd2Terms2StopWords       53.17      (6.1%)       54.30      (5.6%)    2.1% (  -9% -   14%) 0.251
                       And3Terms       63.65      (8.5%)       65.18      (5.3%)    2.4% ( -10% -   17%) 0.281
             And2Terms2StopWords       49.84      (8.0%)       51.06      (6.9%)    2.4% ( -11% -   18%) 0.303
                       CountTerm     5207.97      (7.6%)     5340.28      (7.6%)    2.5% ( -11% -   19%) 0.291
                    AndStopWords        7.95      (8.7%)        8.18      (5.4%)    2.8% ( -10% -   18%) 0.215
                      OrHighRare       89.54      (6.2%)       92.12      (4.0%)    2.9% (  -6% -   13%) 0.080
                      AndHighMed       46.21     (10.0%)       47.59      (6.5%)    3.0% ( -12% -   21%) 0.262
                     AndHighHigh       18.84     (10.6%)       19.47      (7.5%)    3.4% ( -13% -   24%) 0.246
WARNING: cat=CombinedAndHighHigh: hit counts differ: 231389+ vs 231396+
WARNING: cat=CombinedOrHighHigh: hit counts differ: 256699+ vs 256705+
WARNING: cat=CombinedOrHighMed: hit counts differ: 71326+ vs 71325+
WARNING: cat=DismaxOrHighHigh: hit counts differ: 144527+ vs 144388+
WARNING: cat=FilteredAndHighHigh: hit counts differ: 53017+ vs 53016+
WARNING: cat=FilteredAndHighMed: hit counts differ: 12420+ vs 12419+
WARNING: cat=Fuzzy1: hit counts differ: 11814+ vs 11813+
WARNING: cat=Or3Terms: hit counts differ: 32433+ vs 32418+
WARNING: cat=OrHighMed: hit counts differ: 43152+ vs 43139+

HUSTERGS · 2025-08-18T07:35:39Z

I suspect that advance is too heavy for some cases that only skip one or two docs, so the advance approach doesn't help much?

jpountz · 2025-08-18T13:40:09Z

Hmm, it's annoying when the most logical fix doesn't work. :) I'm not sure if it's due to advance() being too heavy, your approach has a call to VectorUtil#findNextGEQ which should be as heavy as advance()? Another theory is that it makes calls to bitSetIntoArray more frequently start a few bits after the start of a long word rather than from the beginning. Related to this, given recent work on increasing batching while evaluating queries, I've been wondering if we'd see speedups by switching to bigger blocks in postings lists, e.g. 256 docs per block instead of 128 today.

jpountz · 2025-08-18T15:03:11Z

I ran this PR on my machine, it seems to trigger a small slowdown on combined tasks (p-value is low) and it's not clear if there's any speedup:

                            TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
              CombinedOrHighHigh       21.80      (3.2%)       21.30      (2.7%)   -2.3% (  -8% -    3%) 0.014
               CombinedOrHighMed       82.43      (3.3%)       80.56      (2.7%)   -2.3% (  -7% -    3%) 0.017
                    CombinedTerm       37.39      (1.9%)       37.01      (1.6%)   -1.0% (  -4% -    2%) 0.071
                     CountPhrase        4.18      (2.3%)        4.14      (3.1%)   -0.8% (  -6% -    4%) 0.349
             CountFilteredOrMany       26.87      (1.8%)       26.66      (1.8%)   -0.8% (  -4% -    2%) 0.178
              CombinedAndHighMed       85.34      (0.9%)       84.68      (0.7%)   -0.8% (  -2% -    0%) 0.004
             CombinedAndHighHigh       22.41      (1.0%)       22.26      (0.8%)   -0.7% (  -2% -    1%) 0.018
                CountAndHighHigh      355.09      (2.4%)      353.53      (1.8%)   -0.4% (  -4% -    3%) 0.505
         CountFilteredOrHighHigh      135.69      (1.0%)      135.19      (0.9%)   -0.4% (  -2% -    1%) 0.230
                 AndHighOrMedMed       48.73      (2.2%)       48.56      (1.4%)   -0.4% (  -3% -    3%) 0.524
          CountFilteredOrHighMed      147.59      (0.7%)      147.08      (0.6%)   -0.3% (  -1% -    1%) 0.106
               TermDayOfYearSort      285.69      (1.3%)      284.77      (1.9%)   -0.3% (  -3% -    2%) 0.536
                    FilteredTerm      153.91      (2.3%)      153.47      (2.2%)   -0.3% (  -4% -    4%) 0.688
                  FilteredIntNRQ      298.60      (0.9%)      297.83      (0.7%)   -0.3% (  -1% -    1%) 0.286
                 CountOrHighHigh      335.61      (2.8%)      334.77      (1.8%)   -0.3% (  -4% -    4%) 0.736
            FilteredAndStopWords       64.09      (1.7%)       64.00      (1.2%)   -0.1% (  -3% -    2%) 0.757
                     CountOrMany       28.56      (2.1%)       28.54      (1.2%)   -0.1% (  -3% -    3%) 0.856
                FilteredOr3Terms      161.75      (1.0%)      161.69      (1.1%)   -0.0% (  -2% -    2%) 0.911
                  FilteredPhrase       31.15      (1.4%)       31.14      (1.7%)   -0.0% (  -3% -    3%) 0.966
      FilteredOr2Terms2StopWords      142.53      (1.2%)      142.54      (1.2%)    0.0% (  -2% -    2%) 0.983
                  CountOrHighMed      342.17      (1.8%)      342.40      (1.6%)    0.1% (  -3% -    3%) 0.897
               FilteredOrHighMed      148.34      (1.1%)      148.45      (1.0%)    0.1% (  -1% -    2%) 0.814
                       CountTerm     8770.53      (3.1%)     8777.75      (3.2%)    0.1% (  -6% -    6%) 0.934
                  FilteredOrMany       15.86      (1.9%)       15.88      (1.6%)    0.1% (  -3% -    3%) 0.827
     FilteredAnd2Terms2StopWords      212.24      (1.5%)      212.54      (1.2%)    0.1% (  -2% -    2%) 0.746
                 CountAndHighMed      298.80      (1.2%)      299.23      (0.9%)    0.1% (  -1% -    2%) 0.677
             FilteredAndHighHigh       77.57      (1.7%)       77.68      (1.2%)    0.2% (  -2% -    3%) 0.748
             CountFilteredPhrase       24.92      (1.6%)       24.97      (1.8%)    0.2% (  -3% -    3%) 0.693
                   TermMonthSort     3305.82      (2.0%)     3313.86      (2.0%)    0.2% (  -3% -    4%) 0.702
             FilteredOrStopWords       43.63      (3.0%)       43.78      (2.1%)    0.4% (  -4% -    5%) 0.666
               FilteredAnd3Terms      184.72      (2.7%)      185.50      (2.2%)    0.4% (  -4% -    5%) 0.593
              FilteredOrHighHigh       64.39      (2.5%)       64.67      (1.9%)    0.4% (  -3% -    4%) 0.533
              FilteredAndHighMed      151.49      (3.8%)      152.26      (2.9%)    0.5% (  -6% -    7%) 0.639
                      TermDTSort      381.66      (2.4%)      383.70      (2.7%)    0.5% (  -4% -    5%) 0.508
                 FilteredPrefix3      149.35      (2.6%)      150.35      (2.6%)    0.7% (  -4% -    6%) 0.415
                        Or3Terms      219.76      (8.4%)      221.28      (6.4%)    0.7% ( -13% -   16%) 0.769
              Or2Terms2StopWords      194.85      (6.9%)      196.43      (5.4%)    0.8% ( -10% -   14%) 0.682
                AndMedOrHighHigh       80.68      (5.6%)       81.38      (4.1%)    0.9% (  -8% -   11%) 0.578
                   TermTitleSort       83.74      (3.6%)       84.58      (3.3%)    1.0% (  -5% -    8%) 0.355
                       And3Terms      229.24      (8.1%)      231.68      (6.6%)    1.1% ( -12% -   17%) 0.649
             And2Terms2StopWords      195.53      (6.3%)      197.67      (5.6%)    1.1% ( -10% -   13%) 0.559
                     OrStopWords       45.72     (11.2%)       46.22      (8.5%)    1.1% ( -16% -   23%) 0.727
                       OrHighMed      239.52      (9.7%)      242.30      (7.5%)    1.2% ( -14% -   20%) 0.672
                      OrHighHigh       72.35     (12.7%)       73.23      (9.7%)    1.2% ( -18% -   26%) 0.733
                          OrMany       22.17      (4.0%)       22.47      (3.5%)    1.3% (  -5% -    9%) 0.268
                      AndHighMed      189.15     (11.6%)      192.00      (9.6%)    1.5% ( -17% -   25%) 0.653
                    AndStopWords       44.46     (10.2%)       45.14      (8.4%)    1.5% ( -15% -   22%) 0.607
                            Term      614.05      (9.7%)      623.36      (7.9%)    1.5% ( -14% -   21%) 0.589
                     AndHighHigh       64.25     (12.8%)       65.24     (10.5%)    1.5% ( -19% -   28%) 0.678
                      OrHighRare      272.53     (11.1%)      277.09      (9.7%)    1.7% ( -17% -   25%) 0.612

HUSTERGS · 2025-08-18T15:19:07Z

Actually I did another experiment, I was worried about the findNextGEQ operation might cause a unexpected slowdown on those machine that do not have fast SIMD instructions, so I added a new copyWithMinDocRequired method to DocAndScoreAccBuffer, now instead of call findNextGEQ, I directly filter those docs when copy, here is the result:

                            TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
              CombinedOrHighHigh        5.54      (3.2%)        5.41      (3.6%)   -2.5% (  -9% -    4%) 0.023
                   TermTitleSort       51.50      (3.9%)       50.60      (4.1%)   -1.7% (  -9% -    6%) 0.169
             CombinedAndHighHigh        5.60      (2.2%)        5.53      (1.3%)   -1.2% (  -4% -    2%) 0.032
                 DismaxOrHighMed       45.35      (7.4%)       44.93      (9.2%)   -0.9% ( -16% -   16%) 0.726
                        SpanNear        2.51      (4.1%)        2.50      (4.1%)   -0.4% (  -8% -    8%) 0.733
                    CombinedTerm       10.96      (4.0%)       10.91      (4.8%)   -0.4% (  -8% -    8%) 0.761
                 AndHighOrMedMed       13.50      (2.6%)       13.47      (3.0%)   -0.2% (  -5% -    5%) 0.789
             FilteredAndHighHigh       10.23      (2.9%)       10.24      (2.7%)    0.0% (  -5% -    5%) 0.959
                         Term10K      438.89      (6.1%)      439.16      (6.6%)    0.1% ( -11% -   13%) 0.975
                            Term      439.07      (6.0%)      439.36      (6.6%)    0.1% ( -11% -   13%) 0.974
                         TermB1M      439.19      (5.9%)      439.51      (6.6%)    0.1% ( -11% -   13%) 0.971
                    SloppyPhrase        1.12      (3.3%)        1.12      (4.1%)    0.1% (  -7% -    7%) 0.929
               CombinedOrHighMed       19.36      (5.3%)       19.40      (5.7%)    0.2% ( -10% -   11%) 0.926
            FilteredAndStopWords        8.22      (3.0%)        8.23      (2.5%)    0.2% (  -5% -    5%) 0.849
                         Term100      438.48      (6.0%)      439.22      (6.6%)    0.2% ( -11% -   13%) 0.933
                       TermB1M1P      438.65      (6.0%)      439.45      (6.7%)    0.2% ( -11% -   13%) 0.927
                          Term1M      438.40      (6.1%)      439.36      (6.5%)    0.2% ( -11% -   13%) 0.913
                     CountPhrase        2.64      (3.2%)        2.64      (3.8%)    0.2% (  -6% -    7%) 0.836
          CountFilteredOrHighMed       17.85      (0.7%)       17.90      (0.7%)    0.3% (  -1% -    1%) 0.186
         CountFilteredOrHighHigh       15.76      (0.7%)       15.81      (0.9%)    0.3% (  -1% -    1%) 0.173
                      DismaxTerm      473.34      (4.1%)      475.20      (4.9%)    0.4% (  -8% -    9%) 0.782
                      OrHighRare       93.56      (3.0%)       93.93      (3.3%)    0.4% (  -5% -    6%) 0.683
             CountFilteredIntNRQ       16.34      (1.2%)       16.42      (1.2%)    0.5% (  -1% -    2%) 0.193
                          IntSet      286.37      (4.4%)      287.83      (4.7%)    0.5% (  -8% -    9%) 0.723
                 FilteredPrefix3       68.84      (4.4%)       69.19      (3.6%)    0.5% (  -7% -    8%) 0.682
                 CountAndHighMed       71.34      (2.8%)       71.73      (2.6%)    0.6% (  -4% -    6%) 0.516
                        Wildcard       46.52      (2.5%)       46.78      (2.6%)    0.6% (  -4% -    5%) 0.476
                IntervalsOrdered        2.43      (2.9%)        2.44      (3.1%)    0.6% (  -5% -    6%) 0.528
             FilteredOrStopWords        7.87      (2.0%)        7.93      (2.1%)    0.7% (  -3% -    4%) 0.251
                         Prefix3       73.58      (4.8%)       74.13      (3.7%)    0.8% (  -7% -    9%) 0.578
             CountFilteredOrMany        4.29      (2.5%)        4.33      (2.6%)    0.9% (  -4% -    6%) 0.278
                CountAndHighHigh       48.27      (1.6%)       48.70      (1.5%)    0.9% (  -2% -    4%) 0.072
                 CountOrHighHigh       49.25      (1.9%)       49.71      (1.8%)    0.9% (  -2% -    4%) 0.101
                     OrStopWords        8.91      (8.4%)        9.00     (12.2%)    1.0% ( -18% -   23%) 0.771
               TermDayOfYearSort      252.19      (1.9%)      254.74      (1.9%)    1.0% (  -2% -    4%) 0.093
                          Phrase        7.36      (2.4%)        7.43      (2.1%)    1.0% (  -3% -    5%) 0.145
                    FilteredTerm       60.82      (2.3%)       61.48      (2.6%)    1.1% (  -3% -    6%) 0.165
                  CountOrHighMed       71.77      (2.2%)       72.56      (1.9%)    1.1% (  -2% -    5%) 0.094
                  FilteredPhrase        9.30      (2.9%)        9.41      (2.7%)    1.2% (  -4% -    6%) 0.188
              CombinedAndHighMed       19.58      (5.1%)       19.81      (5.0%)    1.2% (  -8% -   11%) 0.460
                     CountOrMany        4.81      (2.3%)        4.87      (2.6%)    1.2% (  -3% -    6%) 0.132
                          OrMany        4.17      (6.2%)        4.22      (5.3%)    1.2% (  -9% -   13%) 0.509
                  FilteredOrMany        3.87      (3.0%)        3.92      (2.9%)    1.2% (  -4% -    7%) 0.196
                          IntNRQ       42.46      (2.9%)       42.99      (3.0%)    1.3% (  -4% -    7%) 0.179
              FilteredOrHighHigh       12.37      (2.8%)       12.54      (2.9%)    1.3% (  -4% -    7%) 0.134
                         Respell       34.57      (2.7%)       35.05      (3.5%)    1.4% (  -4% -    7%) 0.162
                      OrHighHigh       20.69     (10.5%)       20.98     (14.8%)    1.4% ( -21% -   29%) 0.730
             CountFilteredPhrase        8.57      (3.3%)        8.69      (3.3%)    1.4% (  -5% -    8%) 0.181
                   TermMonthSort     2048.69      (2.8%)     2078.78      (1.8%)    1.5% (  -3% -    6%) 0.049
                       CountTerm     5432.73      (2.5%)     5516.44      (2.7%)    1.5% (  -3% -    6%) 0.063
                          Fuzzy2       33.85      (3.7%)       34.38      (3.7%)    1.6% (  -5% -    9%) 0.180
                  FilteredIntNRQ       42.05      (2.9%)       42.75      (3.1%)    1.7% (  -4% -    7%) 0.080
                DismaxOrHighHigh       33.48      (5.4%)       34.03      (7.0%)    1.7% ( -10% -   14%) 0.400
               FilteredAnd3Terms       99.67      (3.3%)      101.33      (3.2%)    1.7% (  -4% -    8%) 0.104
                      TermDTSort      136.90      (3.5%)      139.43      (4.0%)    1.9% (  -5% -    9%) 0.118
               FilteredOrHighMed       36.11      (4.1%)       36.80      (4.3%)    1.9% (  -6% -   10%) 0.146
                          Fuzzy1       37.33      (4.3%)       38.06      (4.1%)    2.0% (  -6% -   10%) 0.136
                FilteredOr3Terms       41.00      (3.8%)       41.84      (4.2%)    2.0% (  -5% -   10%) 0.106
              FilteredAndHighMed       30.09      (4.2%)       30.78      (3.6%)    2.3% (  -5% -   10%) 0.061
      FilteredOr2Terms2StopWords       45.64      (4.9%)       46.83      (5.3%)    2.6% (  -7% -   13%) 0.105
                        Or3Terms       59.85      (9.7%)       61.50     (12.9%)    2.8% ( -18% -   28%) 0.444
                       OrHighMed       60.34     (11.4%)       62.06     (15.6%)    2.9% ( -21% -   33%) 0.507
              Or2Terms2StopWords       53.33      (8.4%)       55.96     (10.6%)    4.9% ( -12% -   26%) 0.103
     FilteredAnd2Terms2StopWords       54.99      (5.3%)       58.02      (5.9%)    5.5% (  -5% -   17%) 0.002
                     AndHighHigh       21.28     (10.6%)       22.68     (10.7%)    6.6% ( -13% -   31%) 0.051
                       And3Terms       65.77      (9.0%)       70.21      (8.8%)    6.8% ( -10% -   27%) 0.017
                      AndHighMed       49.53     (10.6%)       52.99     (10.4%)    7.0% ( -12% -   31%) 0.036
             And2Terms2StopWords       51.74      (8.1%)       55.41      (8.8%)    7.1% (  -9% -   26%) 0.008
                AndMedOrHighHigh       14.90      (4.2%)       16.03      (4.0%)    7.6% (   0% -   16%) 0.000
                    AndStopWords        8.18      (6.7%)        8.86      (7.4%)    8.3% (  -5% -   24%) 0.000

BTW, This benchmark contains #15039, which is different from previous benchmark setup. The affected tasks also changed, e.g. OrHighHigh no longer got a speedup.

I'd push the newest code latter, maybe you can help run the benchmark to verify it ?

HUSTERGS · 2025-08-18T15:44:36Z

Maybe we can call other iterator's advance according to maxOtherDoc instead of manually filter the docAndScoreBuffer? Just a thought, will run a benchmark to verify it

HUSTERGS · 2025-08-19T02:57:55Z

Maybe we can call other iterator's advance according to maxOtherDoc instead of manually filter the docAndScoreBuffer? Just a thought, will run a benchmark to verify it

Didn't work

HUSTERGS · 2025-08-21T05:06:38Z

Since both @jpountz and my last benchmark results showed a slowdown on Combined tasks, I did a micro benchmark locally (not commit with this pr) and change the way copyWithMinDocRequired works. It seems the related tasks no longer suffered from this change. Although I still don't know why this speedup cannot be reproduced on other machines : (

                            TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                         Prefix3       71.47     (13.1%)       64.90     (17.1%)   -9.2% ( -34% -   24%) 0.056
                 FilteredPrefix3       66.61     (12.3%)       60.87     (16.3%)   -8.6% ( -33% -   22%) 0.059
                        Wildcard       45.67      (5.0%)       44.48      (7.1%)   -2.6% ( -14% -   10%) 0.179
                   TermTitleSort       51.37      (4.3%)       50.39      (3.6%)   -1.9% (  -9% -    6%) 0.127
              CombinedOrHighHigh        5.50      (3.3%)        5.43      (4.4%)   -1.3% (  -8% -    6%) 0.275
                     CountPhrase        2.63      (4.4%)        2.60      (4.5%)   -1.1% (  -9% -    8%) 0.440
                        SpanNear        2.45      (5.7%)        2.44      (4.9%)   -0.7% ( -10% -   10%) 0.664
             CombinedAndHighHigh        5.64      (1.6%)        5.60      (1.5%)   -0.7% (  -3% -    2%) 0.147
               CombinedOrHighMed       19.30      (5.6%)       19.19      (6.3%)   -0.6% ( -11% -   11%) 0.756
            FilteredAndStopWords        8.27      (2.6%)        8.23      (2.7%)   -0.5% (  -5% -    4%) 0.532
                IntervalsOrdered        2.42      (3.9%)        2.41      (4.6%)   -0.5% (  -8% -    8%) 0.695
                  FilteredPhrase        9.41      (3.4%)        9.36      (3.0%)   -0.5% (  -6% -    6%) 0.615
                          Phrase        7.39      (3.0%)        7.36      (3.2%)   -0.4% (  -6% -    6%) 0.657
                    CombinedTerm       10.97      (4.8%)       10.92      (4.7%)   -0.4% (  -9% -    9%) 0.782
             FilteredAndHighHigh       10.28      (2.5%)       10.24      (3.1%)   -0.4% (  -5% -    5%) 0.673
                 CountAndHighMed       71.02      (2.3%)       70.84      (3.3%)   -0.3% (  -5% -    5%) 0.773
                 AndHighOrMedMed       13.59      (3.1%)       13.56      (2.9%)   -0.2% (  -5% -    5%) 0.813
                   TermMonthSort     2027.04      (2.4%)     2022.76      (2.8%)   -0.2% (  -5% -    5%) 0.798
                  FilteredOrMany        3.87      (3.1%)        3.86      (3.7%)   -0.2% (  -6% -    6%) 0.876
              CombinedAndHighMed       19.70      (5.4%)       19.67      (5.2%)   -0.1% ( -10% -   11%) 0.931
          CountFilteredOrHighMed       17.78      (0.6%)       17.77      (0.8%)   -0.0% (  -1% -    1%) 0.858
                         Respell       34.74      (3.5%)       34.73      (3.9%)   -0.0% (  -7% -    7%) 0.977
             FilteredOrStopWords        7.84      (2.8%)        7.84      (2.1%)   -0.0% (  -4% -    4%) 0.995
         CountFilteredOrHighHigh       15.70      (0.8%)       15.70      (1.0%)    0.0% (  -1% -    1%) 0.995
                CountAndHighHigh       48.12      (1.6%)       48.13      (1.9%)    0.0% (  -3% -    3%) 0.959
                  CountOrHighMed       71.81      (2.1%)       71.86      (2.6%)    0.1% (  -4% -    4%) 0.933
                    SloppyPhrase        1.10      (3.8%)        1.10      (4.3%)    0.1% (  -7% -    8%) 0.941
               FilteredAnd3Terms       99.65      (3.0%)       99.79      (4.2%)    0.1% (  -6% -    7%) 0.899
              FilteredOrHighHigh       12.32      (3.0%)       12.34      (3.0%)    0.2% (  -5% -    6%) 0.843
             CountFilteredPhrase        8.59      (4.8%)        8.60      (3.7%)    0.2% (  -7% -    9%) 0.889
             CountFilteredIntNRQ       16.31      (1.2%)       16.35      (1.2%)    0.2% (  -2% -    2%) 0.584
                      TermDTSort      137.28      (3.4%)      137.57      (3.6%)    0.2% (  -6% -    7%) 0.852
                FilteredOr3Terms       40.96      (3.9%)       41.05      (4.5%)    0.2% (  -7% -    9%) 0.876
                    FilteredTerm       60.63      (3.0%)       60.76      (3.1%)    0.2% (  -5% -    6%) 0.818
               FilteredOrHighMed       36.01      (4.1%)       36.09      (4.6%)    0.2% (  -8% -    9%) 0.867
               TermDayOfYearSort      251.59      (1.9%)      252.18      (2.3%)    0.2% (  -3% -    4%) 0.727
             CountFilteredOrMany        4.26      (2.9%)        4.27      (3.1%)    0.3% (  -5% -    6%) 0.790
      FilteredOr2Terms2StopWords       45.71      (5.0%)       45.84      (5.6%)    0.3% (  -9% -   11%) 0.870
                          Fuzzy2       34.01      (3.8%)       34.11      (3.8%)    0.3% (  -7% -    8%) 0.798
                          IntNRQ       42.39      (3.2%)       42.54      (3.0%)    0.4% (  -5% -    6%) 0.718
                     CountOrMany        4.76      (2.9%)        4.79      (3.5%)    0.4% (  -5% -    7%) 0.662
                 CountOrHighHigh       48.94      (2.0%)       49.19      (2.2%)    0.5% (  -3% -    4%) 0.454
                 DismaxOrHighMed       45.32      (7.5%)       45.57      (5.7%)    0.6% ( -11% -   14%) 0.789
                  FilteredIntNRQ       41.96      (3.1%)       42.22      (3.1%)    0.6% (  -5% -    7%) 0.518
                       CountTerm     5386.09      (3.1%)     5421.82      (2.2%)    0.7% (  -4% -    6%) 0.436
                          OrMany        4.18      (5.8%)        4.21      (4.5%)    0.7% (  -9% -   11%) 0.655
                      DismaxTerm      464.76      (4.1%)      468.41      (2.7%)    0.8% (  -5% -    7%) 0.471
                          Fuzzy1       37.46      (4.4%)       37.81      (4.4%)    0.9% (  -7% -   10%) 0.502
              FilteredAndHighMed       30.45      (2.6%)       30.73      (3.2%)    1.0% (  -4% -    6%) 0.305
                          IntSet      286.66      (3.7%)      289.69      (4.0%)    1.1% (  -6% -    9%) 0.385
                          Term1M      430.69      (5.6%)      437.59      (3.4%)    1.6% (  -7% -   11%) 0.274
                       TermB1M1P      430.98      (5.5%)      438.19      (3.6%)    1.7% (  -6% -   11%) 0.252
                         Term100      431.18      (5.4%)      438.44      (3.6%)    1.7% (  -6% -   11%) 0.244
                         TermB1M      430.94      (5.4%)      438.20      (3.5%)    1.7% (  -6% -   11%) 0.245
                            Term      431.05      (5.5%)      438.47      (3.5%)    1.7% (  -6% -   11%) 0.240
                         Term10K      430.57      (5.4%)      438.14      (3.6%)    1.8% (  -6% -   11%) 0.227
                      OrHighRare       90.01      (6.8%)       91.85      (6.0%)    2.1% ( -10% -   15%) 0.312
                DismaxOrHighHigh       33.24      (4.7%)       34.11      (4.9%)    2.6% (  -6% -   12%) 0.083
     FilteredAnd2Terms2StopWords       55.60      (5.1%)       57.06      (5.8%)    2.6% (  -7% -   14%) 0.127
             And2Terms2StopWords       52.76      (7.4%)       54.44      (7.6%)    3.2% ( -10% -   19%) 0.179
              Or2Terms2StopWords       53.71      (8.4%)       55.44      (7.6%)    3.2% ( -11% -   21%) 0.206
                       And3Terms       67.49      (4.3%)       70.15      (5.3%)    3.9% (  -5% -   14%) 0.010
                     OrStopWords        9.03      (6.1%)        9.41      (7.1%)    4.2% (  -8% -   18%) 0.045
                        Or3Terms       60.45      (8.4%)       62.99      (7.9%)    4.2% ( -11% -   22%) 0.104
                       OrHighMed       61.26      (9.7%)       64.00      (8.9%)    4.5% ( -12% -   25%) 0.128
                    AndStopWords        8.40      (2.1%)        8.83      (2.7%)    5.2% (   0% -   10%) 0.000
                      AndHighMed       51.02      (3.3%)       53.74      (4.2%)    5.3% (  -2% -   13%) 0.000
                      OrHighHigh       21.02      (7.4%)       22.18      (8.9%)    5.5% ( -10% -   23%) 0.033
                AndMedOrHighHigh       14.97      (2.5%)       15.93      (2.6%)    6.4% (   1% -   11%) 0.000
                     AndHighHigh       21.94      (2.2%)       23.42      (3.1%)    6.7% (   1% -   12%) 0.000

init

a0e8b3f

github-actions bot added the module:core/search label Aug 16, 2025

changes & tidy & revert MaxScoreBulkScorer

be1e03a

github-actions bot added this to the 10.3.0 milestone Aug 16, 2025

gesong.samuel added 2 commits August 16, 2025 09:04

refactor

5cd0f01

only filter when minCompetitveScore > 0

4fd22eb

revert

78a7b79

merge main

b54008c

remove findNextGEQ

5810408

gesong.samuel added 2 commits August 20, 2025 23:27

tmp

5fe0f0a

merge main branch

3103139

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Filter by `maxOtherDoc` first to reduce the overhead of `filterCompetitiveHits` #15081

Filter by `maxOtherDoc` first to reduce the overhead of `filterCompetitiveHits` #15081

Uh oh!

HUSTERGS commented Aug 16, 2025

Uh oh!

github-actions bot commented Aug 16, 2025

Uh oh!

HUSTERGS commented Aug 16, 2025 •

edited

Loading

Uh oh!

HUSTERGS commented Aug 16, 2025

Uh oh!

jpountz commented Aug 17, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

jpountz commented Aug 18, 2025

Uh oh!

jpountz commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 19, 2025

Uh oh!

HUSTERGS commented Aug 21, 2025

Uh oh!

Uh oh!

Filter by maxOtherDoc first to reduce the overhead of filterCompetitiveHits #15081

Are you sure you want to change the base?

Filter by maxOtherDoc first to reduce the overhead of filterCompetitiveHits #15081

Uh oh!

Conversation

HUSTERGS commented Aug 16, 2025

Description

Uh oh!

github-actions bot commented Aug 16, 2025

Uh oh!

HUSTERGS commented Aug 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HUSTERGS commented Aug 16, 2025

Uh oh!

jpountz commented Aug 17, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

jpountz commented Aug 18, 2025

Uh oh!

jpountz commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 18, 2025

Uh oh!

HUSTERGS commented Aug 19, 2025

Uh oh!

HUSTERGS commented Aug 21, 2025

Uh oh!

Uh oh!

Filter by `maxOtherDoc` first to reduce the overhead of `filterCompetitiveHits` #15081

Filter by `maxOtherDoc` first to reduce the overhead of `filterCompetitiveHits` #15081

HUSTERGS commented Aug 16, 2025 •

edited

Loading