Skip to content

Commit d8257b4

Browse files
Add documentation about script rescorer (#133359)
1 parent 1337476 commit d8257b4

12 files changed

+236
-135
lines changed

docs/redirects.yml

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,4 +85,16 @@ redirects:
8585
- to: 'reference/query-languages/esql/commands/stats-by.md'
8686
anchors: {'esql-stats-by'}
8787
- to: 'reference/query-languages/esql/commands/where.md'
88-
anchors: {'esql-where'}
88+
anchors: {'esql-where'}
89+
90+
# https://github.com/elastic/elasticsearch/pull/133359
91+
'reference/elasticsearch/rest-apis/filter-search-results.md':
92+
to: 'reference/elasticsearch/rest-apis/filter-search-results.md'
93+
anchors: {} # pass-through unlisted anchors in the `many` ruleset
94+
many:
95+
- to: 'reference/elasticsearch/rest-apis/rescore-search-results.md'
96+
anchors: {'rescore'}
97+
- to: 'reference/elasticsearch/rest-apis/rescore-search-results.md'
98+
anchors: {'query-rescorer'}
99+
- to: 'reference/elasticsearch/rest-apis/rescore-search-results.md'
100+
anchors: {'multiple-rescores'}

docs/reference/aggregations/search-aggregations-metrics-top-hits-aggregation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ If you **only** need `docvalue_fields`, `size`, and `sort` then [Top metrics](/r
4343
::::
4444

4545

46-
`top_hits` does not support the [`rescore`](/reference/elasticsearch/rest-apis/filter-search-results.md#rescore) parameter. Query rescoring applies only to search hits, not aggregation results. To change the scores used by aggregations, use a [`function_score`](/reference/query-languages/query-dsl/query-dsl-function-score-query.md) or [`script_score`](/reference/query-languages/query-dsl/query-dsl-script-score-query.md) query.
46+
`top_hits` does not support the [`rescore`](/reference/elasticsearch/rest-apis/rescore-search-results.md#rescore) parameter. Query rescoring applies only to search hits, not aggregation results. To change the scores used by aggregations, use a [`function_score`](/reference/query-languages/query-dsl/query-dsl-function-score-query.md) or [`script_score`](/reference/query-languages/query-dsl/query-dsl-script-score-query.md) query.
4747

4848

4949
## Example [_example_6]

docs/reference/elasticsearch/rest-apis/collapse-search-results.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@ GET /my-index-000001/_search
173173

174174
## Rescore collapse results [rescore-collapse-results]
175175

176-
You can use field collapsing alongside the [`rescore`](/reference/elasticsearch/rest-apis/filter-search-results.md#rescore) search parameter. Rescorers run on every shard for the top-ranked document per collapsed field. To maintain a reliable order, it is recommended to cluster documents sharing the same collapse field value on one shard. This is achieved by assigning the collapse field value as the [routing key](/reference/elasticsearch/rest-apis/search-shard-routing.md#search-routing) during indexing:
176+
You can use field collapsing alongside the [`rescore`](/reference/elasticsearch/rest-apis/rescore-search-results.md#rescore) search parameter. Rescorers run on every shard for the top-ranked document per collapsed field. To maintain a reliable order, it is recommended to cluster documents sharing the same collapse field value on one shard. This is achieved by assigning the collapse field value as the [routing key](/reference/elasticsearch/rest-apis/search-shard-routing.md#search-routing) during indexing:
177177

178178
```console
179179
POST /my-index-000001/_doc?routing=xyz <1>
@@ -189,7 +189,7 @@ POST /my-index-000001/_doc?routing=xyz <1>
189189

190190
By doing this, you guarantee that only one top document per collapse key gets rescored globally.
191191

192-
The following request utilizes field collapsing on the `user.id` field and then rescores the top groups with a [query rescorer](/reference/elasticsearch/rest-apis/filter-search-results.md#query-rescorer):
192+
The following request utilizes field collapsing on the `user.id` field and then rescores the top groups with a [query rescorer](/reference/elasticsearch/rest-apis/rescore-search-results.md#query-rescorer):
193193

194194
```console
195195
GET /my-index-000001/_search

docs/reference/elasticsearch/rest-apis/filter-search-results.md

Lines changed: 0 additions & 123 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,6 @@ You can use two methods to filter search results:
1212
* Use a boolean query with a `filter` clause. Search requests apply [boolean filters](/reference/query-languages/query-dsl/query-dsl-bool-query.md) to both search hits and [aggregations](/reference/aggregations/index.md).
1313
* Use the search API’s `post_filter` parameter. Search requests apply [post filters](#post-filter) only to search hits, not aggregations. You can use a post filter to calculate aggregations based on a broader result set, and then further narrow the results.
1414

15-
You can also [rescore](#rescore) hits after the post filter to improve relevance and reorder results.
16-
17-
18-
1915
## Post filter [post-filter]
2016

2117
When you use the `post_filter` parameter to filter search results, the search hits are filtered after the aggregations are calculated. A post filter has no impact on the aggregation results.
@@ -125,122 +121,3 @@ GET /shirts/_search
125121
2. The `colors` agg returns popular colors for shirts by Gucci.
126122
3. The `color_red` agg limits the `models` sub-aggregation to **red** Gucci shirts.
127123
4. Finally, the `post_filter` removes colors other than red from the search `hits`.
128-
129-
130-
131-
## Rescore filtered search results [rescore]
132-
133-
Rescoring can help to improve precision by reordering just the top (eg 100 - 500) documents returned by the [`query`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) and [`post_filter`](#post-filter) phases, using a secondary (usually more costly) algorithm, instead of applying the costly algorithm to all documents in the index.
134-
135-
A `rescore` request is executed on each shard before it returns its results to be sorted by the node handling the overall search request.
136-
137-
Currently the rescore API has only one implementation: the query rescorer, which uses a query to tweak the scoring. In the future, alternative rescorers may be made available, for example, a pair-wise rescorer.
138-
139-
::::{note}
140-
An error will be thrown if an explicit [`sort`](/reference/elasticsearch/rest-apis/sort-search-results.md) (other than `_score` in descending order) is provided with a `rescore` query.
141-
::::
142-
143-
144-
::::{note}
145-
when exposing pagination to your users, you should not change `window_size` as you step through each page (by passing different `from` values) since that can alter the top hits causing results to confusingly shift as the user steps through pages.
146-
::::
147-
148-
149-
150-
### Query rescorer [query-rescorer]
151-
152-
The query rescorer executes a second query only on the Top-K results returned by the [`query`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) and [`post_filter`](#post-filter) phases. The number of docs which will be examined on each shard can be controlled by the `window_size` parameter, which defaults to 10.
153-
154-
By default the scores from the original query and the rescore query are combined linearly to produce the final `_score` for each document. The relative importance of the original query and of the rescore query can be controlled with the `query_weight` and `rescore_query_weight` respectively. Both default to `1`.
155-
156-
For example:
157-
158-
```console
159-
POST /_search
160-
{
161-
"query" : {
162-
"match" : {
163-
"message" : {
164-
"operator" : "or",
165-
"query" : "the quick brown"
166-
}
167-
}
168-
},
169-
"rescore" : {
170-
"window_size" : 50,
171-
"query" : {
172-
"rescore_query" : {
173-
"match_phrase" : {
174-
"message" : {
175-
"query" : "the quick brown",
176-
"slop" : 2
177-
}
178-
}
179-
},
180-
"query_weight" : 0.7,
181-
"rescore_query_weight" : 1.2
182-
}
183-
}
184-
}
185-
```
186-
187-
The way the scores are combined can be controlled with the `score_mode`:
188-
189-
| Score Mode | Description |
190-
| --- | --- |
191-
| `total` | Add the original score and the rescore query score. The default. |
192-
| `multiply` | Multiply the original score by the rescore query score. Usefulfor [`function query`](/reference/query-languages/query-dsl/query-dsl-function-score-query.md) rescores. |
193-
| `avg` | Average the original score and the rescore query score. |
194-
| `max` | Take the max of original score and the rescore query score. |
195-
| `min` | Take the min of the original score and the rescore query score. |
196-
197-
198-
### Multiple rescores [multiple-rescores]
199-
200-
It is also possible to execute multiple rescores in sequence:
201-
202-
```console
203-
POST /_search
204-
{
205-
"query" : {
206-
"match" : {
207-
"message" : {
208-
"operator" : "or",
209-
"query" : "the quick brown"
210-
}
211-
}
212-
},
213-
"rescore" : [ {
214-
"window_size" : 100,
215-
"query" : {
216-
"rescore_query" : {
217-
"match_phrase" : {
218-
"message" : {
219-
"query" : "the quick brown",
220-
"slop" : 2
221-
}
222-
}
223-
},
224-
"query_weight" : 0.7,
225-
"rescore_query_weight" : 1.2
226-
}
227-
}, {
228-
"window_size" : 10,
229-
"query" : {
230-
"score_mode": "multiply",
231-
"rescore_query" : {
232-
"function_score" : {
233-
"script_score": {
234-
"script": {
235-
"source": "Math.log10(doc.count.value + 2)"
236-
}
237-
}
238-
}
239-
}
240-
}
241-
} ]
242-
}
243-
```
244-
245-
The first one gets the results of the query then the second one gets the results of the first, etc. The second rescore will "see" the sorting done by the first rescore so it is possible to use a large window on the first rescore to pull documents into a smaller window for the second rescore.
246-

docs/reference/elasticsearch/rest-apis/reciprocal-rank-fusion.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ The `rrf` retriever does not currently support:
114114

115115
* [scroll](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#search-api-scroll-query-param)
116116
* [sort](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search#search-sort-param)
117-
* [rescore](/reference/elasticsearch/rest-apis/filter-search-results.md#rescore)
117+
* [rescore](/reference/elasticsearch/rest-apis/rescore-search-results.md#rescore)
118118

119119
Using unsupported features as part of a search with an `rrf` retriever results in an exception.
120120

0 commit comments

Comments
 (0)