Fix RDS IAM Cross Account Auth and Clarify Dev Container Docs#27632
Fix RDS IAM Cross Account Auth and Clarify Dev Container Docs#27632aniruddhaadak80 wants to merge 8 commits intoopen-metadata:mainfrom
Conversation
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR adds cross-account support for AWS RDS IAM auth token generation by optionally assuming an STS role, and updates developer documentation/devcontainer configs to clarify Dev Container initialization.
Changes:
- Add optional
assumeRoleArnJDBC query param support inAwsRdsDatabaseAuthenticationProviderusing STS assume-role credentials. - Document Dev Container workflows in
DEVELOPER.md, clarifying thatpost-create.shis the one-time initialization script shared by both devcontainer modes. - Add clarifying inline notes to Dev Container
postCreateCommandconfiguration.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| openmetadata-service/src/main/java/org/openmetadata/service/util/jdbi/AwsRdsDatabaseAuthenticationProvider.java | Add optional STS assume-role credentials provider for cross-account RDS IAM token generation. |
| DEVELOPER.md | Add Dev Containers section clarifying the two configs and initialization flow. |
| .devcontainer/full-stack/devcontainer.json | Add clarification near postCreateCommand (but currently via JSON comment). |
| .devcontainer/dev/devcontainer.json | Add clarification near postCreateCommand (but currently via JSON comment). |
| AwsCredentialsProvider credentialsProvider = DefaultCredentialsProvider.create(); | ||
|
|
||
| if (assumeRoleArn != null) { | ||
| StsClient stsClient = | ||
| StsClient.builder() | ||
| .region(Region.of(awsRegion)) | ||
| .credentialsProvider(credentialsProvider) | ||
| .build(); | ||
|
|
||
| AssumeRoleRequest assumeRoleRequest = | ||
| AssumeRoleRequest.builder() | ||
| .roleArn(assumeRoleArn) | ||
| .roleSessionName("OpenMetadata-RDS-IAM-Auth") | ||
| .build(); | ||
|
|
||
| credentialsProvider = | ||
| StsAssumeRoleCredentialsProvider.builder() | ||
| .stsClient(stsClient) | ||
| .refreshRequest(assumeRoleRequest) | ||
| .build(); | ||
| } |
There was a problem hiding this comment.
assumeRoleArn triggers creating a new StsClient + StsAssumeRoleCredentialsProvider on every authenticate() call. In the IAM-auth path this runs per DB connection, so this will repeatedly call STS (latency + throttling risk) and also leaves the StsClient/provider unclosed, potentially leaking HTTP resources/threads. Consider constructing and reusing an assume-role credentials provider (e.g., cached by awsRegion+assumeRoleArn or initialized once per pool) and ensuring any SDK clients/providers are closed on shutdown.
| if (assumeRoleArn != null) { | ||
| StsClient stsClient = | ||
| StsClient.builder() | ||
| .region(Region.of(awsRegion)) | ||
| .credentialsProvider(credentialsProvider) | ||
| .build(); | ||
|
|
||
| AssumeRoleRequest assumeRoleRequest = | ||
| AssumeRoleRequest.builder() | ||
| .roleArn(assumeRoleArn) | ||
| .roleSessionName("OpenMetadata-RDS-IAM-Auth") | ||
| .build(); |
There was a problem hiding this comment.
The check if (assumeRoleArn != null) will attempt an STS assume-role even when the query param is present but empty/whitespace (e.g. assumeRoleArn=), which will fail with an AWS SDK validation error. Treat blank values as “not provided” (e.g., check isBlank() and skip) or raise a clear configuration error.
| StsClient stsClient = | ||
| StsClient.builder() | ||
| .region(Region.of(awsRegion)) | ||
| .credentialsProvider(credentialsProvider) | ||
| .build(); | ||
|
|
||
| AssumeRoleRequest assumeRoleRequest = | ||
| AssumeRoleRequest.builder() | ||
| .roleArn(assumeRoleArn) | ||
| .roleSessionName("OpenMetadata-RDS-IAM-Auth") | ||
| .build(); | ||
|
|
||
| credentialsProvider = | ||
| StsAssumeRoleCredentialsProvider.builder() | ||
| .stsClient(stsClient) | ||
| .refreshRequest(assumeRoleRequest) | ||
| .build(); | ||
| } |
There was a problem hiding this comment.
With the new STS assume-role path, failures (STS call errors, invalid role ARN, missing permissions, etc.) will throw AWS SDK runtime exceptions that currently bypass the catch (MalformedURLException e) and won’t be wrapped in DatabaseAuthenticationProviderException. Consider catching broader exceptions in authenticate() and wrapping them consistently to keep error handling aligned with other providers (e.g., AzureDatabaseAuthenticationProvider).
| "version": "22.17.0" | ||
| } | ||
| }, | ||
| // Use post-create script for one-time environment initialization (ANTLR, dependencies, venv) |
There was a problem hiding this comment.
devcontainer.json is parsed by the repo’s pre-commit check-json hook (strict JSON). Adding // comments makes this file invalid JSON and will cause the hook/CI to fail. Please remove the comment or replace it with a normal JSON field (e.g., a _comment property) if you want to keep the explanation in-file.
| // Use post-create script for one-time environment initialization (ANTLR, dependencies, venv) | |
| "_comment": "Use post-create script for one-time environment initialization (ANTLR, dependencies, venv)", |
| "version": "22.17.0" | ||
| } | ||
| }, | ||
| // Use post-create script for one-time environment initialization (ANTLR, dependencies, venv) |
There was a problem hiding this comment.
devcontainer.json is parsed by the repo’s pre-commit check-json hook (strict JSON). Adding // comments makes this file invalid JSON and will cause the hook/CI to fail. Please remove the comment or replace it with a normal JSON field (e.g., a _comment property) if you want to keep the explanation in-file.
| // Use post-create script for one-time environment initialization (ANTLR, dependencies, venv) | |
| "_comment": "Use post-create script for one-time environment initialization (ANTLR, dependencies, venv)", |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
…-docs-27552-27517
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
| if (CommonUtil.nullOrEmpty(awsRegion)) { | ||
| throw new DatabaseAuthenticationProviderException( | ||
| "Parameter `awsRegion` shall be provided in the jdbc url."); | ||
| } | ||
| if (CommonUtil.nullOrEmpty(allowPublicKeyRetrieval)) { | ||
| throw new DatabaseAuthenticationProviderException( | ||
| "Parameter `allowPublicKeyRetrieval` shall be provided in the jdbc url."); | ||
| } |
There was a problem hiding this comment.
CommonUtil.nullOrEmpty only checks isEmpty() and will treat whitespace-only values as present. That means values like awsRegion=%20 or assumeRoleArn=%20 will pass validation and then fail later (e.g., Region.of(" ") or STS AssumeRole with an invalid ARN), producing a harder-to-diagnose error. Consider validating these parameters with a blank-aware check (e.g., trim + empty, or StringUtils.isBlank) so whitespace-only inputs are rejected as missing.
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
… restore shard null-checks, clean up STS resources, and add service name AOSS detection
…-docs-27552-27517
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
I have addressed all the feedback from Gitar-bot and Copilot-bot. Could a maintainer please add the |
harshach
added
the
safe to test
|
The Java checkstyle failed. Please run You can install the pre-commit hooks with |
…n to resolve Checkstyle and Copilot feedback
Code Review ✅ Approved 5 resolved / 5 findingsFixes RDS IAM cross-account authentication and updates development container documentation. Resolves resource leaks in StsClient, fixes AOSS detection logic, removes duplicate methods, and adds necessary null safety checks. ✅ 5 resolved✅ Bug: StsClient is never closed, causing resource leak on every auth call
✅ Performance: StsClient and STS credentials provider recreated on every connection
✅ Edge Case: AOSS detection misses SEARCH_AWS_SERVICE_NAME=aoss config
✅ Bug: Duplicate postDelete method will cause compilation error
✅ Bug: Null check on shards().total() was dropped, risking NPE
OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
|
The Java checkstyle failed. Please run You can install the pre-commit hooks with |
This PR addresses two issues: #27552 and #27517. It implements support for the optional assumeRoleArn parameter in AwsRdsDatabaseAuthenticationProvider.java to enable cross-account IAM authentication for RDS. It also enhances the DEVELOPER.md documentation with a dedicated section for Dev Containers and adds clarifying comments to .devcontainer configs to clarify that post-create.sh is the primary initialization script.
Summary by Gitar
AutoCloseableinAwsRdsDatabaseAuthenticationProviderto ensure proper cleanup ofstsClientCacheandcredentialsProviderCache.testCaseResultandtestCaseResolutionStatusinTestCaseRepository.isAoss()detection toOpenSearchClientto support AWS OpenSearch Serverless configurations.SearchIndexClusterValidatorandSearchClusterMetricsfor cluster statistics to prevent runtime exceptions.This will update automatically on new commits.