Add docs for updated Undici Agent functionality in Transport (#2957)

Author: JoshMock

docs/reference/advanced-config.md

@@ -19,7 +19,7 @@ For information about the `Transport` class, refer to [Transport](/reference/tra
 
 ## `ConnectionPool` [_connectionpool]
 
-This class is responsible for keeping in memory all the {{es}} Connection that you are using. There is a single Connection for every node. The connection pool handles the resurrection strategies and the updates of the pool.
+This class is responsible for keeping in memory all the {{es}} connections that you are using. There is a single `Connection` for every node. The connection pool handles the resurrection strategies and the updates of the pool.
 
 ```js
 const { Client, ConnectionPool } = require('@elastic/elasticsearch')
@@ -41,7 +41,7 @@ const client = new Client({
 
 ## `Connection` [_connection]
 
-This class represents a single node, it holds every information we have on the node, such as roles, id, URL, custom headers and so on. The actual HTTP request is performed here, this means that if you want to swap the default HTTP client (Node.js core), you should override the `request` method of this class.
+This class represents a single node, it holds every information we have on the node, such as roles, id, URL, custom headers and so on. The actual HTTP request is performed here, this means that if you want to swap the default HTTP client ([Undici `Pool`](https://undici.nodejs.org/#/docs/api/Pool.md)), you should override the `request` method of this class.
 
 ```js
 const { Client, BaseConnection } = require('@elastic/elasticsearch')
@@ -59,6 +59,10 @@ const client = new Client({
 })
 ```
 
+`@elastic/transport` provides two `Connection` implementations:
+
+- `UndiciConnection`: manages HTTP connections using [Undici](https://undici.nodejs.org/), Node.js's high-performance HTTP client implementation; this is the default value of `Connection` and is recommended unless you have a use case that is not yet supported by Undici or `UndiciConnection`
+- `HttpConnection`: manages HTTP connections using [the `http` package](https://nodejs.org/api/http.html) from Node.js's standard library
 
 ## `Serializer` [_serializer]
 
@@ -175,5 +179,5 @@ try {
 
 ## Migrate to v8 [_migrate_to_v8]
 
-The Node.js client can be configured to emit an HTTP header `Accept: application/vnd.elasticsearch+json; compatible-with=7` which signals to Elasticsearch that the client is requesting `7.x` version of request and response bodies. This allows for upgrading from 7.x to 8.x version of Elasticsearch without upgrading everything at once. Elasticsearch should be upgraded first after the compatibility header is configured and clients should be upgraded second. To enable to setting, configure the environment variable `ELASTIC_CLIENT_APIVERSIONING` to `true`.
+The Node.js client can be configured to emit an HTTP header `Accept: application/vnd.elasticsearch+json; compatible-with=7` which signals to {{es}} that the client is requesting `7.x` version of request and response bodies. This allows for upgrading from 7.x to 8.x version of {{es}} without upgrading everything at once. {{es}} should be upgraded first after the compatibility header is configured and clients should be upgraded second. To enable to setting, configure the environment variable `ELASTIC_CLIENT_APIVERSIONING` to `true`.
 

docs/reference/basic-config.md

@@ -184,12 +184,21 @@ const client = new Client({
 })
 ```
 
-### `agent`
+### `agent` [agent-config]
 
-Type: `http.AgentOptions, function`<br>
+Type: `http.AgentOptions, undici.PoolOptions, function, false`<br>
 Default: `null`
 
-http agent [options](https://nodejs.org/api/http.html#http_new_agent_options), or a function that returns an actual http agent instance. If you want to disable the http agent use entirely (and disable the `keep-alive` feature), set the agent to `false`.
+If using the default `UndiciConnection` from `@elastic/transport`, this value can be:
+
+- an [Undici `PoolOptions` object](https://undici.nodejs.org/#/docs/api/Pool?id=parameter-pooloptions)
+- a function that receives all connection-related options and returns an [Undici `Agent`](https://undici.nodejs.org/#/docs/api/Agent.md) instance (or any other object that follows [Undici's `Dispatch.request()`](https://undici.nodejs.org/#/docs/api/Dispatcher?id=dispatcherrequestoptions-callback) conventions)
+
+If using the legacy `HttpConnection` from `@elastic/transport`, this value can be:
+
+- [the options object passed to an `http.Agent`](https://nodejs.org/api/http.html#new-agentoptions)
+- a function that returns an `http.Agent` (and thus also an [`https.Agent`](https://nodejs.org/api/https.html#class-httpsagent), or any implementation that follows the same conventions, like [`hpagent`](https://www.npmjs.com/package/hpagent))
+- `false` to disable all agent usage, including the `keep-alive` feature
 
 ```js
 const client = new Client({
@@ -211,6 +220,10 @@ const client = new Client({
 })
 ```
 
+::::{warning}
+If you have set [the `agent` option](/reference/basic-config.md#agent-config) on your client instance to a function and are using `UndiciConnection`&mdash;the default [`Connection`](/reference/advanced-config.md#_connection) value starting in 8.0&mdash;all `caFingerprint` and `tls` options will be ignored. It is your responsibility to ensure that your custom agent will properly verify HTTPS connections.
+::::
+
 ### `nodeFilter`
 
 Type: `function`

docs/reference/client-helpers.md

@@ -93,7 +93,7 @@ To create a new instance of the Bulk helper, access it as shown in the example a
     ```
 
 `onSuccess`
-:   A function that is called for each successful operation in the bulk request, which includes the result from Elasticsearch along with the original document that was sent, or `null` for delete operations.
+:   A function that is called for each successful operation in the bulk request, which includes the result from {{es}} along with the original document that was sent, or `null` for delete operations.
 
     ```js
     const b = client.helpers.bulk({
@@ -307,7 +307,7 @@ console.log(result)
 
 Added in `v8.8.2`
 
-If you need to modify documents in your datasource before it is sent to Elasticsearch, you can return an array in the `onDocument` function rather than an operation object. The first item in the array must be the operation object, and the second item must be the document or partial document object as you’d like it to be sent to Elasticsearch.
+If you need to modify documents in your datasource before it is sent to {{es}}, you can return an array in the `onDocument` function rather than an operation object. The first item in the array must be the operation object, and the second item must be the document or partial document object as you’d like it to be sent to {{es}}.
 
 ```js
 const { Client } = require('@elastic/elasticsearch')

docs/reference/connecting.md

@@ -77,6 +77,10 @@ When you start {{es}} for the first time you’ll see a distinct block like the
 
 Depending on the circumstances there are two options for verifying the HTTPS connection, either verifying with the CA certificate itself or via the HTTP CA certificate fingerprint.
 
+::::{warning}
+If you have set [the `agent` option](/reference/basic-config.md#agent-config) on your client instance to a function and are using `UndiciConnection`—the default `Connection` value starting in 8.0—all `caFingerprint` and `tls` options will be ignored. It is your responsibility to ensure that your custom agent will properly verify HTTPS connections.
+::::
+
 ### TLS configuration [auth-tls]
 
 The generated root CA certificate can be found in the `certs` directory in your {{es}} config location (`$ES_CONF_PATH/certs/http_ca.crt`). If you’re running {{es}} in Docker there is [additional documentation for retrieving the CA certificate](docs-content://deploy-manage/deploy/self-managed/install-elasticsearch-with-docker.md).
@@ -332,7 +336,7 @@ The supported request specific options are:
 | Option | Description |
 | --- | ----------- |
 | `ignore` | `number[]` -  HTTP status codes which should not be considered errors for this request.<br>*Default:* `null` |
-| `requestTimeout` | `number` or `string` - Max request timeout for the request in milliseconds. This overrides the client default, which is to not time out at all. See [Elasticsearch best practices for HTML clients](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md#_http_client_configuration) for more info.<br>_Default:_ No timeout |
+| `requestTimeout` | `number` or `string` - Max request timeout for the request in milliseconds. This overrides the client default, which is to not time out at all. See [{{es}} best practices for HTML clients](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md#_http_client_configuration) for more info.<br>_Default:_ No timeout |connecting
 | `retryOnTimeout` | `boolean` - Retry requests that have timed out.*Default:* `false` |
 | `maxRetries` | `number` - Max number of retries for the request, it overrides the client default.<br>*Default:* `3` |
 | `compression` | `string` or  `boolean` - Enables body compression for the request.<br>*Options:* `false`, `'gzip'`<br>*Default:* `false` |
@@ -477,9 +481,9 @@ You can find the errors exported by the client in the table below.
 
 ## Keep-alive connections [keep-alive]
 
-By default, the client uses persistent, keep-alive connections to reduce the overhead of creating a new HTTP connection for each Elasticsearch request. If you are using the default `UndiciConnection` connection class, it maintains a pool of 256 connections with a keep-alive of 10 minutes. If you are using the legacy `HttpConnection` connection class, it maintains a pool of 256 connections with a keep-alive of 1 minute.
+By default, the client uses persistent, keep-alive connections to reduce the overhead of creating a new HTTP connection for each {{es}} request. If you are using the default `UndiciConnection` connection class, it maintains a pool of 256 connections with a keep-alive of 10 minutes. If you are using the legacy `HttpConnection` connection class, it maintains a pool of 256 connections with a keep-alive of 1 minute.
 
-If you need to disable keep-alive connections, you can override the HTTP agent with your preferred [HTTP agent options](https://nodejs.org/api/http.md#http_new_agent_options):
+If you need to disable keep-alive connections, you can override the HTTP agent with your preferred [HTTP agent options](/reference/basic-config.md#agent-config):
 
 ```js
 const client = new Client({
@@ -500,6 +504,48 @@ const client = new Client({
 })
 ```
 
+## Managing open connection limits [limit-open-connections]
+
+Starting in client 9.0, when using `@elastic/transport` 9.2.0 or later, you can provide a custom `agent` function to share a singleton [Undici `Agent`](https://undici.nodejs.org/#/docs/api/Agent.md) instance that can enforce client-wide connection limits.
+
+```typescript
+import { Agent } from 'undici'
+import { HttpConnection } from '@elastic/transport'
+
+// `maxOrigins * connections` (50 in this case) is the total connection limit
+const maxSocketAgent = new Agent({
+  keepAliveTimeout: 1000,
+  maxOrigins: 5,
+  connections: 10
+})
+
+const client = new Client({
+  node: '...',
+  auth: { ... },
+  agent: () => maxSocketAgent
+})
+```
+
+If using the legacy `HttpConnection`, you can use an [`Agent`](https://nodejs.org/api/https.html#class-httpsagent) singleton that enforces `maxTotalSockets`:
+
+```typescript
+import { Agent } from 'node:http'
+import { HttpConnection } from '@elastic/transport'
+
+const maxSocketAgent = new Agent({
+  keepAlive: true,
+  keepAliveMsecs: 1000,
+  maxTotalSockets: 50
+})
+
+const client = new Client({
+  node: '...',
+  auth: { ... },
+  Connection: HttpConnection,
+  agent: () => maxSocketAgent
+})
+```
+
 ## Closing a client’s connections [close-connections]
 
 If you would like to close all open connections being managed by an instance of the client, use the `close()` function:
@@ -513,4 +559,4 @@ client.close();
 
 ## Automatic product check [product-check]
 
-Since v7.14.0, the client performs a required product check before the first call. This pre-flight product check allows the client to establish the version of Elasticsearch that it is communicating with. The product check requires one additional HTTP request to be sent to the server as part of the request pipeline before the main API call is sent. In most cases, this will succeed during the very first API call that the client sends. Once the product check completes, no further product check HTTP requests are sent for subsequent API calls.
+Since v7.14.0, the client performs a required product check before the first call. This pre-flight product check allows the client to establish the version of {{es}} that it is communicating with. The product check requires one additional HTTP request to be sent to the server as part of the request pipeline before the main API call is sent. In most cases, this will succeed during the very first API call that the client sends. Once the product check completes, no further product check HTTP requests are sent for subsequent API calls.

docs/reference/getting-started.md

@@ -6,7 +6,7 @@ mapped_pages:
 
 # Getting started [getting-started-js]
 
-This page guides you through the installation process of the Node.js client, shows you how to instantiate the client, and how to perform basic Elasticsearch operations with it.
+This page guides you through the installation process of the Node.js client, shows you how to instantiate the client, and how to perform basic {{es}} operations with it.
 
 
 ### Requirements [_requirements]
@@ -28,7 +28,7 @@ Refer to the [*Installation*](/reference/installation.md) page to learn more.
 
 ### Connecting [_connecting]
 
-You can connect to the Elastic Cloud using an API key and the Elasticsearch endpoint.
+You can connect to the Elastic Cloud using an API key and the {{es}} endpoint.
 
 ```js
 const { Client } = require('@elastic/elasticsearch')
@@ -43,9 +43,9 @@ const client = new Client({
 })
 ```
 
-Your Elasticsearch endpoint can be found on the **My deployment** page of your deployment:
+Your {{es}} endpoint can be found on the **My deployment** page of your deployment:
 
-![Finding Elasticsearch endpoint](images/es-endpoint.jpg)
+![Finding {{es}} endpoint](images/es-endpoint.jpg)
 
 You can generate an API key on the **Management** page under Security.
 
@@ -56,7 +56,7 @@ For other connection options, refer to the [*Connecting*](/reference/connecting.
 
 ### Operations [_operations]
 
-Time to use Elasticsearch! This section walks you through the basic, and most important, operations of Elasticsearch.
+Time to use {{es}}! This section walks you through the basic, and most important, operations of {{es}}.
 
 
 #### Creating an index [_creating_an_index]

docs/reference/observability.md

@@ -5,7 +5,7 @@ mapped_pages:
 
 # Observability [observability]
 
-To observe and measure Elasticsearch client usage, several client features are provided.
+To observe and measure {{es}} client usage, several client features are provided.
 
 First, as of 8.15.0, the client provides native support for OpenTelemetry, which allows you to send client usage data to any endpoint that supports OpenTelemetry without having to make any changes to your JavaScript codebase.
 
@@ -17,9 +17,9 @@ All of these observability features are documented below.
 
 ## OpenTelemetry [_opentelemetry]
 
-The client supports OpenTelemetry’s [zero-code instrumentation](https://opentelemetry.io/docs/zero-code/js/) to enable tracking each client request as an [OpenTelemetry span](https://opentelemetry.io/docs/concepts/signals/traces/#spans). These spans follow all of the [semantic OpenTelemetry conventions for Elasticsearch](https://opentelemetry.io/docs/specs/semconv/database/elasticsearch/) except for `db.query.text`.
+The client supports OpenTelemetry’s [zero-code instrumentation](https://opentelemetry.io/docs/zero-code/js/) to enable tracking each client request as an [OpenTelemetry span](https://opentelemetry.io/docs/concepts/signals/traces/#spans). These spans follow all of the [semantic OpenTelemetry conventions for {{es}}](https://opentelemetry.io/docs/specs/semconv/database/elasticsearch/) except for `db.query.text`.
 
-To start sending Elasticsearch trace data to your OpenTelemetry endpoint, follow [OpenTelemetry’s zero-code instrumentation guide](https://opentelemetry.io/docs/zero-code/js/), or the following steps:
+To start sending {{es}} trace data to your OpenTelemetry endpoint, follow [OpenTelemetry’s zero-code instrumentation guide](https://opentelemetry.io/docs/zero-code/js/), or the following steps:
 
 1. Install `@opentelemetry/api` and `@opentelemetry/auto-instrumentations-node` as Node.js dependencies
 2. Export the following environment variables with the appropriate values:

docs/reference/timeout-best-practices.md

@@ -9,5 +9,5 @@ Starting in 9.0.0, this client is configured to not time out any HTTP request by
 
 Prior to 9.0, this client was configured by default to operate like many HTTP client libraries do, by using a relatively short (30 second) timeout on all requests sent to {{es}}, raising a `TimeoutError` when that time period elapsed without receiving a response.
 
-If you need to set timeouts on Elasticsearch requests, setting the `requestTimeout` value to a millisecond value will cause this client to operate as it did prior to 9.0.
+If you need to set timeouts on {{es}} requests, setting the `requestTimeout` value to a millisecond value will cause this client to operate as it did prior to 9.0.
 

docs/reference/typescript.md

@@ -5,7 +5,7 @@ mapped_pages:
 
 # TypeScript support [typescript]
 
-The client offers a first-class support for TypeScript, shipping a complete set of type definitions of Elasticsearch’s API surface.
+The client offers a first-class support for TypeScript, shipping a complete set of type definitions of {{es}}'s API surface.
 
 The types are not 100% complete yet. Some APIs are missing (the newest ones, e.g. EQL), and others may contain some errors, but we are continuously pushing fixes & improvements. Contribute type fixes and improvements to [elasticsearch-specification github repository](https://github.com/elastic/elasticsearch-specification).
 

src/client.ts

@@ -164,7 +164,7 @@ export interface ClientOptions {
     * @defaultValue true */
   enableMetaHeader?: boolean
   /** @property cloud Custom configuration for connecting to Elastic Cloud, in lieu of a `node` or `nodes` configuration
-    * @remarks Read https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/client-connecting.html#client-usage for more details
+    * @remarks Read https://www.elastic.co/docs/reference/elasticsearch/clients/javascript/connecting#client-usage for more details
     * @defaultValue null */
   cloud?: {
     id: string
@@ -182,7 +182,7 @@ export interface ClientOptions {
     * @defaultValue null */
   maxCompressedResponseSize?: number
   /** @property redaction Options for how to redact potentially sensitive data from metadata attached to `Error` objects
-    * @remarks Read https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/advanced-config.html#redaction for more details
+    * @remarks Read https://www.elastic.co/docs/reference/elasticsearch/clients/javascript/advanced-config#redaction for more details
     * @defaultValue Configuration that will replace known sources of sensitive data */
   redaction?: RedactionOptions
   /** @property serverMode Setting to "serverless" will change some default behavior, like enabling compression and disabling features that assume the possibility of multiple Elasticsearch nodes.
@@ -443,7 +443,7 @@ export default class Client extends API {
 
   /**
    * Creates a child client instance that shared its connection pool with the parent client
-   * @see {@link https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/child.html}
+   * @see {@link https://www.elastic.co/docs/reference/elasticsearch/clients/javascript/child}
    */
   child (opts: ClientOptions): Client {
     // Merge the new options with the initial ones