From 8269db64a181eb0c2b37d53b893aac12d31db406 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Wed, 16 Jul 2025 13:02:32 +0300 Subject: [PATCH] Add recommendations on non-idempotent operations Fixes #5242 --- doc/platform/sharding/vshard_admin.rst | 87 +++++++++++++++++++ .../reference_rock/vshard/vshard_router.rst | 6 ++ 2 files changed, 93 insertions(+) diff --git a/doc/platform/sharding/vshard_admin.rst b/doc/platform/sharding/vshard_admin.rst index ae338eca3..0ce6255f7 100644 --- a/doc/platform/sharding/vshard_admin.rst +++ b/doc/platform/sharding/vshard_admin.rst @@ -551,6 +551,93 @@ In a router application, you can define the ``put`` function that specifies how Learn more at :ref:`vshard-process-requests`. +.. _vshard-deduplication: + +Deduplication of non-idempotent requests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Idempotent requests** produce the same result every time they are executed. +For example, a data read request or a multiplication by one are both idempotent. +Therefore, incrementing by one is an example of a non-idempotent operation. +When such an operation is applied again, the value for the field increases by 2 instead of just 1. + +.. note:: + + Any write requests that are intended to be executed repeatedly should be idempotent. + The operations' idempotency ensures that the change is applied **only once**. + +A request may need to be run again if an error occurs on the server or client side. +In this case: + +- Read requests can be executed repeatedly. + For this purpose, :ref:`vshard.router.call() ` (with ``mode=read``) uses the ``request_timeout`` parameter + (since ``vshard`` 0.1.28). + It is necessary to pass the ``request_timeout`` and ``timeout`` parameters together, with the following requirement: + + .. code-block:: text + + timeout > request_timeout + + + For example, if ``timeout = 10`` and ``request_timeout = 2``, + within 10 seconds the router is able to make 5 attempts (2 seconds each) to send a request to different replicas + until the request finally succeeds. + +- Write requests (:ref:`vshard.router.callrw() `) generally **cannot be re-executed** without verifying + that they have not been applied before. + Lack of such a check might lead to duplicate records or unplanned data changes. + + For example, a client has sent a request to the server. The client is waiting for a response within a specified timeout. + If the server sends a successful response after this time has elapsed, the client will receive an error. + When re-executing this request without additional check, the operation may be applied twice. + + A write request can be executed repeatedly without a check only if the error occurred on the server side -- + for example, `ER_READONLY`. + +**Deduplication examples** + +To ensure that the write requests (INSERT, UPDATE, UPSERT, and autoincrement) are idempotent, +you should implement a check that the request is applied for the first time. + +For example, when you add a new tuple to a space, you can use a unique insert ID to check the request. +In the example below within a single transaction: + +1. It is checked whether a tuple with the ``key`` ID exists in the ``bands`` space. +2. If there is no tuple with this ID in the space, the tuple is inserted. + +.. code-block:: lua + + box.begin() + if box.space.bands:get{key} == nil then + box.space.bands:insert{key, value} + end + box.commit() + +For update and upsert requests, you can create a *deduplication space* where the request IDs will be saved. +*Deduplication space* is a user space that contains a list of unique identifiers. +Each identifier corresponds to one applied request. +This space can have any name, in the example it is called ``deduplication``. + +In the example below, within a single transaction: + +1. It is checked whether the ``deduplication_key`` request ID exists in the ``deduplication`` space. +2. If there is no such ID, The ID is added to the deduplication space. +3. If the request hasn't been applied before, it increments the specified field in the ``bands`` space by one. + +This approach ensures that each data modification request will be executed **only once**. + +.. code-block:: lua + + function update_1(deduplication_key, key) + box.begin() + if box.space.deduplication:get{deduplication_key} == nil then + box.space.deduplication:insert{deduplication_key} + box.space.bands:update(key, {{'+', 'value', 1 }}) + end + box.commit() + end + + .. _vshard-maintenance: Sharded cluster maintenance diff --git a/doc/reference/reference_rock/vshard/vshard_router.rst b/doc/reference/reference_rock/vshard/vshard_router.rst index 66c23d2ad..c7e832f03 100644 --- a/doc/reference/reference_rock/vshard/vshard_router.rst +++ b/doc/reference/reference_rock/vshard/vshard_router.rst @@ -163,6 +163,12 @@ Router public API optional attribute containing a message with the human-readable error description, and other attributes specific for the error code. + .. note:: + + Any write requests that are intended to be executed repeatedly should be idempotent. + The operations' idempotency ensures that the change is applied **only once**. + Read more: :ref:``. + **Examples:** To call ``customer_add`` function from ``vshard/example``, say: