Skip to content

Commit 1bb7370

Browse files
JuanLeeeJuanLeee
committed
docs: limitless docs
# Conflicts: # docs/using-the-python-driver/UsingThePythonDriver.md
1 parent 12f68b8 commit 1bb7370

File tree

3 files changed

+83
-0
lines changed

3 files changed

+83
-0
lines changed

‎docs/examples/PGLimitless.py

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License").
4+
# You may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import psycopg
16+
17+
from aws_advanced_python_wrapper import AwsWrapperConnection
18+
19+
if __name__ == "__main__":
20+
with AwsWrapperConnection.connect(
21+
psycopg.Connection.connect,
22+
host="limitless-cluster.limitless-xyz.us-east-1.rds.amazonaws.com",
23+
dbname="postgres_limitless",
24+
user="user",
25+
password="password",
26+
plugins="limitless",
27+
autocommit=True
28+
) as awsconn, awsconn.cursor() as awscursor:
29+
awscursor.execute("SELECT * FROM aurora_db_instance_identifier()")
30+
31+
res = awscursor.fetchone()
32+
print(res)

‎docs/using-the-python-driver/UsingThePythonDriver.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ The AWS Advanced Python Driver has several built-in plugins that are available t
7373
| [Read Write Splitting Plugin](./using-plugins/UsingTheReadWriteSplittingPlugin.md) | `read_write_splitting` | Aurora | Enables read write splitting functionality where users can switch between database reader and writer instances. | None |
7474
| [Fastest Response Strategy Plugin](./using-plugins/UsingTheFastestResponseStrategyPlugin.md) | `fastest_response_strategy` | Aurora | A host selection strategy plugin that uses a host monitoring service to monitor each reader host's response time and choose the host with the fastest response. | None |
7575
| [Blue/Green Deployment Plugin](./using-plugins/UsingTheBlueGreenPlugin.md) | `bg` | Aurora, <br>RDS Instance | Enables client-side Blue/Green Deployment support. | None |
76+
| [Limitless Plugin](using-plugins/UsingTheLimitlessPlugin.md) | `limitless` | Aurora | Enables client-side load-balancing of Transaction Routers on Amazon Aurora Limitless Databases. | None |
7677

7778
In addition to the built-in plugins, you can also create custom plugins more suitable for your needs.
7879
For more information, see [Custom Plugins](../development-guide/LoadablePlugins.md#using-custom-plugins).

‎docs/using-the-python-driver/using-plugins/UsingTheLimitlessPlugin.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Using the Limitless Plugin
2+
3+
## What is Amazon Aurora Limitless Database?
4+
5+
Amazon Aurora Limitless Database is a new type of database that can horizontally scale to handle millions of write transactions per second and manage petabytes of data.
6+
Users will be able to use the AWS Python Driver with Aurora Limitless Databases and optimize their experience using the Limitless Plugin.
7+
To learn more about Aurora Limitless Database, see the [Amazon Aurora Limitless documentation](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/limitless.html).
8+
9+
## Why use the Limitless Plugin?
10+
11+
Aurora Limitless Database introduces a new endpoint for the databases - the DB shard group (limitless) endpoint that's managed by Route 53.
12+
When connecting to Aurora Limitless Database, clients will connect using this endpoint, and be routed to a transaction router via Route 53.
13+
Unfortunately, Route 53 is limited in its ability to load balance, and can allow uneven work loads on transaction routers.
14+
The Limitless Plugin addresses this by performing client-side load balancing with load awareness.
15+
16+
The Limitless Plugin achieves this with a monitoring thread that periodically polls for available transaction routers and their load metrics, and then caches them.
17+
When a new connection is made, the plugin directs the connection to a transaction router selected from the cache using a weighted random strategy.
18+
Routers with a higher load are assigned a lower weight, and routers with a lower load are assigned a higher weight.
19+
20+
## How to use the Limitless Plugin with the AWS Python Driver
21+
To enable the Limitless Plugin, add the plugin code `limitless` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) value.
22+
23+
The URL used to connect to a limitless database is the DB shard group URL.
24+
25+
### Limitless Plugin Parameters
26+
| Parameter | Value | Required | Description | Default Value | Example Value |
27+
|---------------------------------------------------------|:-------:|:--------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|-------------|
28+
| `limitless_transaction_router_monitor_interval_ms` | Integer | No | This property is the interval in milliseconds, that the plugin polls the database for available transaction routers and their load metrics. A lower value will increase the frequency of polling, and a higher value will decrease the frequency of polling. <br><br>Note that there will always be a delay between when the database updates its load metric info and when the Limitless Plugin polls for it. If your Limitless Database experiences fluctuating load between transaction routers, you may want to consider lowering `limitlessTransactionRouterMonitorIntervalMs` to reduce this delay and ensure the Limitless Plugin load balancing has fresher info to work with. <br><br>The default value of this property is 7.5 seconds. This is half the interval that the database updates its load metric metadata. This value was chosen as a compromise between having fresher load metric info, but also being conscious of the associated overhead. | `7500` | `30000` |
29+
| `limitless_transaction_router_monitor_disposal_time_ms` | Integer | No | This property is the time in milliseconds that a Limitless transaction router monitor can remain unused before it is disposed. This ensures that in periods of long inactivity, the database isn't being needlessly polled and the resources associated with the monitor can be cleaned up. Note that when a new connection is created, a new Limitlesstransaction router monitor will also be created to resume polling the database. | `600000` | `300000` |
30+
| `limitless_max_retries_ms` | Integer | No | This property is the max number of retries the Limitless Plugin will attempt when failing to connect to the database. During these retries, the plugin will attempt to connect to the least loaded transaction router that is available. If the max number of connection retries is exceeded, then the plugin will throw a `AwsWrapperError`. In this scenario, it is likely that the database is in an unhealthy state, and the `AwsWrapperError` should be caught and handled by your application. | `5` | `13` |
31+
| `limitless_wait_for_transaction_router_info` | Boolean | No | In scenarios such as application start-up, the cache of available transaction routers and their load metric info will be empty. If `limitlessWaitForTransactionRouterInfo` is set to `True`, the plugin will wait until the cache is populated before selecting a transaction router and connecting to it. This may be beneficial for applications that create a large number of connections on start-up, since these connections will be load-balanced. <br><br>Alternatively, if this property set to `False` and the cache is empty, the plugin will not wait for the cache to be populated and default to using the DB Shard Group endpoint to connect to until the cache is populated. This will result in connections being routed to a transaction router via Route 53 until the cache is populated. This may be beneficial for applications that prioritize quicker start-up times at the expense of some early connections not being load-balanced by the Limitless Plugin. | True | False |
32+
| `limitless_get_transaction_router_max_retries` | Integer | No | This property is the max number of times the Limitless Plugin will retry fetching available transaction routers and their load metrics. These retries will occur if the fetched transaction router information is `None` or empty. If this max is reached, a `AwsWrapperError` will be thrown. In this scenario, it is likely that the database is in an unhealthy state, and the thrown `AwsWrapperError` should be caught and handled by your application. <br><br>If your application prioritizes failing fast, then consider a lower value for this property. However, if your application prioritizes durability, then consider a higher value. | `5` | `10` |
33+
| `limitless_get_transaction_router_retry_interval_ms` | Integer | No | This property is the interval in milliseconds between retries of fetching available transaction routers and their load metrics. <br><br>If your application prioritizes failing fast, then consider a lower value for this property. However, if your application prioritizes durability, then consider a higher value. | `300` | `1000` |
34+
| Weighted random parameters | | | See [Reader Selection Strategies](../ReaderSelectionStrategies.md) for more information on weighted random parameter. | `300` | `1000` |
35+
36+
### Use with other plugins
37+
The Limitless Plugin is compatible with authentication type plugins such as the IAM and AWS Secrets Manager Plugins.
38+
39+
> [!IMPORTANT]\
40+
> The Failover, Host Monitoring, and Read Write Splitting Plugins are also compatible with the Limitless Plugin.
41+
However, we don't recommend using them with the Limitless Plugin because they're not designed to be used with Aurora Limitless Database.
42+
They don't provide any extra value, and add unnecessary computation and memory overhead.
43+
44+
### Use with Connection Pools
45+
Connection pools keep connections open for reuse, but this can work against the client-side load-balancing of the Limitless Plugin and cause an imbalanced load on transaction routers.
46+
To mitigate this, consider setting connection properties that can reduce the number of idle connections or increase the lifetime of connections.
47+
If you're using the internal connection pool, some of these properties are `pool_expiration_check_ns`, `pool_cleanup_interval_ns`.
48+
49+
## Sample Code
50+
[PGLimitless.py](../../examples/PGLimitless.py)

0 commit comments

Comments
 (0)