feat(elasticsearch-plugin): avoid redundant reindex on stock movements

[LeftoversTodayAppAdmin]

Closes #50.

Adds two opt-in options that stop stock movements from needlessly re-indexing a product, including the delete-then-recreate window during which the product drops out of search results. Both default to the current behavior, so existing installs are unaffected.

Changes

reindexOnStockMovement: 'always' | 'onStockStatusChange' (default 'always')

Pre-enqueue guard in the StockMovementEvent subscriber. In 'onStockStatusChange' it only enqueues an update when the movement flips a variant's inStock or its product's productInStock, so a 50 to 49 change never creates a job. It inspects only the built-in stock booleans, so it is meant for setups without a stock-derived custom mapping. Others should stay on 'always' and use the option below.

skipUnchangedIndexUpdates: boolean (default false)

Write-path guard. Before the delete-then-recreate, updateProductsOperations builds the product's target documents, compares them against what is currently indexed, and skips the write when they are identical. It compares the whole document, so it stays correct for any mapping configuration including stock-derived custom mappings, and it removes the flicker for any no-op update regardless of trigger. A full reindex is never skipped.

The two are complementary: the first avoids creating a job for stock movements that cannot change search results, the second is a general safety net against a redundant delete-then-recreate from any trigger.

Backwards compatibility

• Both options default to today's behavior.
• No change to the SearchClientAdapter interface (the comparison uses the existing search), and no index mapping or schema change.
• The stock check uses ProductVariantService.getSaleableStockLevel and the comparison reads through the adapter, so both features behave identically on Elasticsearch and OpenSearch.

Implementation notes

• updateProductsOperationsOnly was split into buildProductVariantOperations (build) and a thin execute wrapper, so the incremental path can build the target documents, diff them, and skip the write. Output is unchanged and writes are still chunked via executeBulkOperationsByChunks.
• The document comparison lives in src/indexing/index-diff.ts (stableStringify, indexedDocumentsMatch) so it can be unit tested in isolation.

Tests

Unit (src/indexing/index-diff.spec.ts): stableStringify, targetDocumentsById (pairing the update and doc bulk operations back into a document map), and indexedDocumentsMatch, including that a stock-derived custom field difference still counts as a change, which is the correctness invariant behind onStockStatusChange.

E2E (e2e/elasticsearch-plugin.e2e-spec.ts): the existing suite now runs with skipUnchangedIndexUpdates enabled, so every existing test exercises the guard on real updates. A new skipUnchangedIndexUpdates describe adds explicit stock-movement cases: a non-boundary movement leaves search correct, and movements that flip inStock in and out of stock are reflected. Runs under both backends via SEARCH_BACKEND.

I validated onStockStatusChange (the pre-enqueue guard) manually against a running OpenSearch instance: a non-boundary movement created no reindex job, a boundary movement flipped inStock/productInStock, and a full reindex rebuilt everything. I did not enable onStockStatusChange in the e2e config because the existing spec defines a stock-derived custom mapping, and that mode is not intended for such setups. Happy to add dedicated coverage for it if you would like, and to adjust the option names or defaults.

[biggamesmallworld]

Nice approach on skipUnchangedIndexUpdates. Diffing against the real builder output is the right call.

Three blockers:

1. The refactor dropped bulk-operation chunking. Hits everyone, including full reindex, with both options off. See inline.
2. Please drop reindexOnStockMovement. Your description says skipUnchangedIndexUpdates is correct for any config and covers any trigger, which makes the other one a subset that's only correct sometimes. Add a stock-derived custom mapping later and you get a silently stale index. It also does an ES search plus a Product query plus per-variant getSaleableStockLevel in the subscriber, so order placement pays for it. Dropping it takes productStockStatusDiffersFromIndex and half the PR with it.
3. stableStringify reimplements fast-deep-equal, which we already depend on, and it's buggy. See inline.

Also missing @since on both options, and elasticsearch-options.mdx needs regenerating.

[biggamesmallworld]

Both mid-build flushes are gone, so this buffers every operation for a product before returning. The description says writes are still chunked, which is true, but the build isn't anymore.

Count is channels × languages × variants × 2, each holding a full VariantIndexItem. 5000 variants, 3 channels, 4 languages = 120k buffered docs. reindex() at line 388 goes through here, so full reindex on a big catalog can now OOM. No opt-in needed.

The diff path needs them in memory, that's fine, it's opt-in. But updateProductsOperationsOnly has to keep streaming. A flush callback would do it.

[biggamesmallworld]

fast-deep-equal is already in package.json and imported in elasticsearch.service.ts:19.

Also: stableStringify(new Date()) returns {}. A custom mapping returning a Date never matches its own _source (an ISO string), so the skip never fires and the option quietly does nothing. Tests only cover primitives and plain objects.

equal(JSON.parse(JSON.stringify(doc)), hit._source) handles undefined, Dates, and key order, and compares exactly what gets stored.

[biggamesmallworld]

buildProductVariantOperations has the id and the doc at construction time. Return { operations, documentsById } and this function plus its four tests go away. Array<{ operation: any }> also throws away the index field.

[biggamesmallworld]

This recomputes stock status differently from createVariantIndexItem and they already disagree. ProductVariant.deletedAt is a plain @Column, not @DeleteDateColumn, so line 249 pulls in soft-deleted variants. The builder filters them and also force-disables everything when product.enabled is false, which changes productInStock.

Both differences fail safe today. The issue is that isProductIndexUnchanged gets this right by calling the real builder, and this one will go stale next time createVariantIndexItem changes with nothing to catch it.

[biggamesmallworld]

10000 is the default index.max_result_window, so this truncates at the ceiling with no error. Same literal at 795.

Safe there (fewer hits = mismatch = write), not here: dropped hits mean dropped channel buckets, if (!channelDocs) continue skips them, guard says unchanged. Named constant plus handle the boundary. Both call sites are the same term: { productId } search too.

[biggamesmallworld]

These pass with the flag off: they assert search is correct, which it already was. Nothing checks a write was skipped.

Sentinel makes it deterministic: write an extra field into the index doc, run the no-op update, assert it survived. Proves the delete-then-recreate didn't happen.

it('finds an in-stock variant to exercise') is a beforeAll in disguise, and the next two break with undefined if it fails. Fixture is fixed, so assert the exact variant. take: 200 will silently truncate once the fixture grows.

Flipping the whole suite also means these 96 tests stop covering the default path. The uuid spec still does, but that's 7 tests. Prefer a targeted describe.