feat: add stdio tool retry and improve readme

Author: ptbsare

README.md

@@ -174,27 +174,110 @@ Example `config/tool_config.json`:
     export TOOLS_FOLDER=/srv/mcp_tools
     ```
 
-### 4. Proxy Behavior Configuration (Error Handling & Retries)
-While some general proxy behaviors might be configured here in the future, the primary retry and error handling settings are now managed via **Environment Variables** for easier deployment-specific overrides. Values in `config/mcp_server.json` for these specific settings will be overridden by environment variables if set.
+-   **`RETRY_SSE_TOOL_CALL_ON_DISCONNECT`**: (Optional) Controls whether to automatically reconnect and retry on SSE tool call failures. Set to `"true"` to enable, `"false"` to disable. Default: `true`. See the "Enhanced Reliability Features" section for details.
+    ```bash
+    export RETRY_SSE_TOOL_CALL_ON_DISCONNECT="true"
+    ```
+-   **`RETRY_HTTP_TOOL_CALL`**: (Optional) Controls whether to retry on HTTP tool call connection errors. Set to `"true"` to enable, `"false"` to disable. Default: `true`. See the "Enhanced Reliability Features" section for details.
+    ```bash
+    export RETRY_HTTP_TOOL_CALL="true"
+    ```
+-   **`HTTP_TOOL_CALL_MAX_RETRIES`**: (Optional) Maximum number of retry attempts for HTTP tool calls (after the initial failure). Default: `2`. See the "Enhanced Reliability Features" section for details.
+    ```bash
+    export HTTP_TOOL_CALL_MAX_RETRIES="3"
+    ```
+-   **`HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS`**: (Optional) Base delay in milliseconds for HTTP tool call retries, used in exponential backoff. Default: `300`. See the "Enhanced Reliability Features" section for details.
+    ```bash
+    export HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS="500"
+    ```
+-   **`RETRY_STDIO_TOOL_CALL`**: (Optional) Controls whether to retry on Stdio tool call connection errors (attempts to restart the process). Set to `"true"` to enable, `"false"` to disable. Default: `true`. See the "Enhanced Reliability Features" section for details.
+    ```bash
+    export RETRY_STDIO_TOOL_CALL="true"
+    ```
+-   **`STDIO_TOOL_CALL_MAX_RETRIES`**: (Optional) Maximum number of retry attempts for Stdio tool calls (after the initial failure). Default: `2`. See the "Enhanced Reliability Features" section for details.
+    ```bash
+    export STDIO_TOOL_CALL_MAX_RETRIES="5"
+    ```
+-   **`STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS`**: (Optional) Base delay in milliseconds for Stdio tool call retries, used in exponential backoff. Default: `300`. See the "Enhanced Reliability Features" section for details.
+    ```bash
+    export STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS="1000"
+    ```
 
-Example `config/mcp_server.json` showing other potential proxy settings:
-```json
-{
-  "mcpServers": {
-    "...": "..."
-  },
-  "proxy": {
-    "someOtherProxySettingNotOverriddenByEnv": "example_value"
-    // Specific retry settings like retrySseToolCallOnDisconnect, retryHttpToolCall, 
-    // httpToolCallMaxRetries, and httpToolCallRetryDelayBaseMs are now
-    // preferably set via environment variables (see below).
-  }
-}
+## Enhanced Reliability Features
+
+The MCP Proxy Server includes features to improve its resilience and the reliability of interactions with backend MCP services, ensuring smoother operations and more consistent tool execution.
+
+### 1. Error Propagation
+The proxy server ensures that errors originating from backend MCP services are consistently propagated to the requesting client. These errors are formatted as standard JSON-RPC error responses, making it easier for clients to handle them uniformly.
+
+### 2. SSE Connection Retry for Tool Calls
+When a `tools/call` operation is made to an SSE-based backend server, and the underlying connection is lost or experiences an error, the proxy server will automatically attempt to:
+1.  Re-establish the connection to the SSE backend.
+2.  If reconnection is successful, it will retry the original `tools/call` request **once**.
+
+This behavior helps mitigate transient network issues that might temporarily disrupt SSE connections.
+
+**Configuration:**
+This feature is primarily controlled by the **`RETRY_SSE_TOOL_CALL_ON_DISCONNECT`** environment variable.
+-   **`RETRY_SSE_TOOL_CALL_ON_DISCONNECT`** (environment variable):
+    -   Set to `"true"` to enable the automatic reconnect and retry.
+    -   Set to `"false"` to disable this feature.
+    -   **Default Behavior:** `true` (if the environment variable is not set, is empty, or is an invalid value).
+
+**Example (Environment Variable):**
+```bash
+export RETRY_SSE_TOOL_CALL_ON_DISCONNECT="true"
 ```
-See the "Enhanced Reliability Features" and "Environment Variables" sections for details on these options.
 
+### 3. HTTP Request Retry for Tool Calls
+For `tools/call` operations directed to HTTP-based backend servers, the proxy implements a retry mechanism for connection errors (e.g., "failed to fetch", network timeouts).
+
+**Retry Mechanism:**
+If an initial HTTP request fails due to a connection error, the proxy will retry the request using an exponential backoff strategy. This means the delay before each subsequent retry attempt increases exponentially, with a small amount of jitter (randomness) added to prevent thundering herd scenarios.
 
-## Enhanced Reliability Features
+**Configuration:**
+These settings are primarily controlled by environment variables.
+
+-   **`RETRY_HTTP_TOOL_CALL`** (environment variable):
+    -   Set to `"true"` to enable retries for HTTP tool calls.
+    -   Set to `"false"` to disable this feature.
+    -   **Default Behavior:** `true` (if the environment variable is not set, is empty, or is an invalid value).
+
+-   **`HTTP_TOOL_CALL_MAX_RETRIES`** (environment variable):
+    -   Specifies the maximum number of retry attempts *after* the initial failed attempt. For example, if set to `"2"`，there will be one initial attempt and up to two retry attempts, totaling a maximum of three attempts.
+    -   **Default Behavior:** `2` (if the environment variable is not set, is empty, or is not a valid integer).
+
+-   **`HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS`** (environment variable):
+    -   The base delay in milliseconds used in the exponential backoff calculation. The delay before the *n*-th retry (0-indexed) is roughly `HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS * (2^n) + jitter`.
+    -   **Default Behavior:** `300` (milliseconds) (if the environment variable is not set, is empty, or is not a valid integer).
+
+### 4. Stdio Connection Retry for Tool Calls
+For `tools/call` operations directed to Stdio-based backend servers, the proxy implements a retry mechanism for connection errors (e.g., process crash or unresponsiveness).
+
+**Retry Mechanism:**
+If an initial Stdio connection or tool call fails, the proxy will attempt to restart the Stdio process and retry the request. This mechanism follows an exponential backoff strategy similar to HTTP retries.
+
+**Configuration:**
+These settings are primarily controlled by environment variables.
+
+-   **`RETRY_STDIO_TOOL_CALL`** (environment variable):
+    -   Set to `"true"` to enable Stdio tool call retries.
+    -   Set to `"false"` to disable this feature.
+    -   **Default Behavior:** `true` (if the environment variable is not set, is empty, or is an invalid value).
+
+-   **`STDIO_TOOL_CALL_MAX_RETRIES`** (environment variable):
+    -   Specifies the maximum number of retry attempts *after* the initial failed attempt. For example, if set to `"2"`，there will be one initial attempt and up to two retry attempts, totaling a maximum of three attempts.
+    -   **Default Behavior:** `2` (if the environment variable is not set, is empty, or is not a valid integer).
+
+-   **`STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS`** (environment variable):
+    -   The base delay in milliseconds used in the exponential backoff calculation. The delay before the *n*-th retry (0-indexed) is roughly `STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS * (2^n) + jitter`.
+    -   **Default Behavior:** `300` (milliseconds) (if the environment variable is not set, is empty, or is not a valid integer).
+
+**General Notes on Environment Variable Parsing:**
+-   Boolean environment variables (`RETRY_SSE_TOOL_CALL_ON_DISCONNECT`, `RETRY_HTTP_TOOL_CALL`, `RETRY_STDIO_TOOL_CALL`) are considered `true` if their lowercase value is exactly `"true"`. Any other value (including empty or not set) results in the default being applied or `false` if the default is `false` (though for these specific variables, the default is `true`).
+-   Numeric environment variables (`HTTP_TOOL_CALL_MAX_RETRIES`, `HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS`, `STDIO_TOOL_CALL_MAX_RETRIES`, `STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS`) are parsed as base-10 integers. If parsing fails (e.g., the value is not a number, or the variable is empty/not set), the default value is used.
+
+## Development
 
 The MCP Proxy Server includes features to improve its resilience and the reliability of interactions with backend MCP services, ensuring smoother operations and more consistent tool execution.
 

README_ZH.md

@@ -175,6 +175,109 @@
     export TOOLS_FOLDER=/srv/mcp_tools
     ```
 
+-   **`RETRY_SSE_TOOL_CALL_ON_DISCONNECT`**: (可选) 控制 SSE 工具调用失败时是否自动重连并重试。设置为 `"true"` 启用，`"false"` 禁用。默认: `true`。有关详细信息，请参阅“增强的可靠性特性”部分。
+    ```bash
+    export RETRY_SSE_TOOL_CALL_ON_DISCONNECT="true"
+    ```
+-   **`RETRY_HTTP_TOOL_CALL`**: (可选) 控制 HTTP 工具调用连接错误时是否重试。设置为 `"true"` 启用，`"false"` 禁用。默认: `true`。有关详细信息，请参阅“增强的可靠性特性”部分。
+    ```bash
+    export RETRY_HTTP_TOOL_CALL="true"
+    ```
+-   **`HTTP_TOOL_CALL_MAX_RETRIES`**: (可选) HTTP 工具调用最大重试次数（在初始失败后）。默认: `2`。有关详细信息，请参阅“增强的可靠性特性”部分。
+    ```bash
+    export HTTP_TOOL_CALL_MAX_RETRIES="3"
+    ```
+-   **`HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS`**: (可选) HTTP 工具调用重试延迟基准（毫秒），用于指数退避。默认: `300`。有关详细信息，请参阅“增强的可靠性特性”部分。
+    ```bash
+    export HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS="500"
+    ```
+-   **`RETRY_STDIO_TOOL_CALL`**: (可选) 控制 Stdio 工具调用连接错误时是否重试（尝试重启进程）。设置为 `"true"` 启用，`"false"` 禁用。默认: `true`。有关详细信息，请参阅“增强的可靠性特性”部分。
+    ```bash
+    export RETRY_STDIO_TOOL_CALL="true"
+    ```
+-   **`STDIO_TOOL_CALL_MAX_RETRIES`**: (可选) Stdio 工具调用最大重试次数（在初始失败后）。默认: `2`。有关详细信息，请参阅“增强的可靠性特性”部分。
+    ```bash
+    export STDIO_TOOL_CALL_MAX_RETRIES="5"
+    ```
+-   **`STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS`**: (可选) Stdio 工具调用重试延迟基准（毫秒），用于指数退避。默认: `300`。有关详细信息，请参阅“增强的可靠性特性”部分。
+    ```bash
+    export STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS="1000"
+    ```
+
+## 增强的可靠性特性
+
+MCP 代理服务器包含多项特性，用以提升其自身弹性以及与后端 MCP 服务交互的可靠性，确保更平稳的操作和更一致的工具执行。
+
+### 1. 错误传播
+代理服务器确保从后端 MCP 服务产生的错误能够一致地传播给请求客户端。这些错误被格式化为标准的 JSON-RPC 错误响应，使客户端更容易统一处理它们。
+
+### 2. SSE 工具调用的连接重试
+当对基于 SSE 的后端服务器执行 `tools/call` 操作时，如果底层连接丢失或遇到错误，代理服务器将自动尝试：
+1.  重新建立与 SSE 后端的连接。
+2.  如果重新连接成功，它将重试原始的 `tools/call` 请求**一次**。
+
+此行为有助于缓解可能暂时中断 SSE 连接的瞬时网络问题。
+
+**配置:**
+此功能主要通过 **`RETRY_SSE_TOOL_CALL_ON_DISCONNECT`** 环境变量控制。
+-   **`RETRY_SSE_TOOL_CALL_ON_DISCONNECT`** (环境变量):
+    -   设置为 `"true"` 以启用自动重新连接和重试。
+    -   设置为 `"false"` 以禁用此功能。
+    -   **默认行为:** `true` (如果环境变量未设置、为空或为无效值)。
+
+**示例 (环境变量):**
+```bash
+export RETRY_SSE_TOOL_CALL_ON_DISCONNECT="true"
+```
+
+### 3. HTTP 工具调用的请求重试
+对于定向到基于 HTTP 的后端服务器的 `tools/call` 操作，代理服务器为连接错误（例如，“failed to fetch”、网络超时）实现了一套重试机制。
+
+**重试机制：**
+如果初始 HTTP 请求因连接错误而失败，代理将使用指数退避策略重试该请求。这意味着每次后续重试尝试之前的延迟会指数级增加，并加入少量抖动（随机性）以防止“惊群效应”。
+
+**配置：**
+这些设置主要通过环境变量控制。
+
+-   **`RETRY_HTTP_TOOL_CALL`** (环境变量):
+    -   设置为 `"true"` 以启用 HTTP 工具调用的重试。
+    -   设置为 `"false"` 以禁用此功能。
+    -   **默认行为:** `true` (如果环境变量未设置、为空或为无效值)。
+
+-   **`HTTP_TOOL_CALL_MAX_RETRIES`** (环境变量):
+    -   指定在初次失败尝试*之后*的最大重试次数。例如，如果设置为 `"2"`，则会有一次初始尝试和最多两次重试尝试，总共最多三次尝试。
+    -   **默认行为:** `2` (如果环境变量未设置、为空或不是一个有效的整数)。
+
+-   **`HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS`** (环境变量):
+    -   用于指数退避计算的基准延迟（以毫秒为单位）。第 *n* 次重试（0索引）之前的延迟大约是 `HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS * (2^n) + 抖动`。
+    -   **默认行为:** `300` (毫秒) (如果环境变量未设置、为空或不是一个有效的整数)。
+
+### 4. Stdio 工具调用的连接重试
+对于指向基于 Stdio 的后端服务器的 `tools/call` 操作，代理实现了针对连接错误（例如，进程崩溃或无响应）的重试机制。
+
+**重试机制:**
+如果初始 Stdio 连接或工具调用失败，代理将尝试重新启动 Stdio 进程并重试请求。此机制类似于 HTTP 重试，使用指数退避策略。
+
+**配置:**
+这些设置主要由环境变量控制。
+
+-   **`RETRY_STDIO_TOOL_CALL`** (环境变量):
+    -   设置为 `"true"` 以启用 Stdio 工具调用重试。
+    -   设置为 `"false"` 以禁用此功能。
+    -   **默认行为:** `true` (如果环境变量未设置、为空或为无效值)。
+
+-   **`STDIO_TOOL_CALL_MAX_RETRIES`** (环境变量):
+    -   指定在初次失败尝试*之后*的最大重试尝试次数。例如，如果设置为 `"2"`，则将有一次初始尝试和最多两次重试尝试，总共最多三次尝试。
+    -   **默认行为:** `2` (如果环境变量未设置、为空或不是一个有效的整数)。
+
+-   **`STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS`** (环境变量):
+    -   用于指数退避计算的基准延迟（以毫秒为单位）。第 *n* 次重试（从 0 开始索引）之前的延迟大约是 `STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS * (2^n) + 抖动`。
+    -   **默认行为:** `300` (毫秒) (如果环境变量未设置、为空或不是一个有效的整数)。
+
+**环境变量解析通用说明:**
+-   布尔环境变量（`RETRY_SSE_TOOL_CALL_ON_DISCONNECT`，`RETRY_HTTP_TOOL_CALL`，`RETRY_STDIO_TOOL_CALL`）如果其小写值恰好是 `"true"`，则被视为 `true`。任何其他值（包括空或未设置）将应用默认值，或者如果默认值为 `false` 则为 `false`（尽管对于这些特定变量，默认值为 `true`）。
+-   数字环境变量（`HTTP_TOOL_CALL_MAX_RETRIES`，`HTTP_TOOL_CALL_RETRY_DELAY_BASE_MS`，`STDIO_TOOL_CALL_MAX_RETRIES`，`STDIO_TOOL_CALL_RETRY_DELAY_BASE_MS`）被解析为十进制整数。如果解析失败（例如，值不是数字，或变量为空/未设置），则使用默认值。
+
 ## 开发
 
 安装依赖: