-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Lazily build MongoClient in MongoClientModule #26912
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Reviewer's GuideThis PR introduces a MongoClientFactory to defer DNS lookups and replaces the singleton MongoClient in MongoSession with per-operation clients created and closed via the factory, updating dependent classes and module bindings accordingly. Class diagram for MongoClientFactory and MongoSession changesclassDiagram
class MongoClientFactory {
+OpenTelemetry openTelemetry
+Set<MongoClientSettingConfigurator> configurators
+MongoClient create()
}
class MongoSession {
-MongoClientFactory clientFactory
+MongoClient createClient()
+List<HostAddress> getAddresses()
+List<String> getAllSchemas()
+void createSchema(String schemaName)
+void dropSchema(String schemaName, boolean cascade)
+void createTable(RemoteTableName name, List<MongoColumnHandle> columns, Optional<String> comment)
+void dropTable(RemoteTableName remoteTableName)
+void setTableComment(MongoTableHandle table, Optional<String> comment)
+void setColumnComment(MongoTableHandle table, String columnName, Optional<String> comment)
+void renameTable(MongoTableHandle table, SchemaTableName newName)
+void addColumn(MongoTableHandle table, ColumnMetadata columnMetadata)
+void renameColumn(MongoTableHandle table, String source, String target)
+void dropColumn(MongoTableHandle table, String columnName)
+void setColumnType(MongoTableHandle table, String columnName, Type type)
+void insertMany(RemoteTableName table, List<Document> documents, InsertManyOptions options)
+List<MongoIndex> getIndexes(String schemaName, String tableName)
+long deleteDocuments(RemoteTableName remoteTableName, TupleDomain<ColumnHandle> constraint)
+MongoCursor<Document> execute(MongoClient client, MongoTableHandle tableHandle, List<MongoColumnHandle> columns)
}
MongoSession --> MongoClientFactory : uses
Class diagram for MongoPageSource and MongoPageSink changesclassDiagram
class MongoPageSource {
-MongoClient client
-MongoCursor<Document> cursor
+MongoPageSource(MongoSession mongoSession, MongoTableHandle tableHandle, List<MongoColumnHandle> columns, String implicitPrefix)
+void close()
}
class MongoPageSink {
+CompletableFuture<?> appendPage(Page page)
}
MongoPageSource --> MongoSession : uses
MongoPageSink --> MongoSession : uses
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey there - I've reviewed your changes - here's some feedback:
- MongoSession now opens and closes a MongoClient on every operation, which can severely impact performance—consider reusing a single client instance per session to leverage connection pooling and reduce overhead.
- Dropping Closeable from MongoSession removes any clear shutdown path for the clients created by MongoClientFactory—ensure there’s a defined lifecycle to close those clients and avoid resource leaks.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- MongoSession now opens and closes a MongoClient on every operation, which can severely impact performance—consider reusing a single client instance per session to leverage connection pooling and reduce overhead.
- Dropping Closeable from MongoSession removes any clear shutdown path for the clients created by MongoClientFactory—ensure there’s a defined lifecycle to close those clients and avoid resource leaks.
## Individual Comments
### Comment 1
<location> `plugin/trino-mongodb/src/test/java/io/trino/plugin/mongodb/TestMongoPlugin.java:48-49` </location>
<code_context>
connector.shutdown();
}
+
+ @Test
+ void testMongoDbSrv()
+ {
+ ConnectorFactory factory = getOnlyElement(new MongoPlugin().getConnectorFactories());
</code_context>
<issue_to_address>
**suggestion (testing):** Consider adding assertions to verify connector initialization and error handling for SRV URLs.
Please add assertions to confirm successful initialization and proper error handling, including cases with invalid SRV URLs and connection failures.
</issue_to_address>
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This pull request introduces a lazy MongoClientFactory
to defer DNS lookups during connector initialization when using MongoDB SRV connection strings, addressing the issue where connectors were accessing remote datasources during initialization.
Key changes:
- Replaces eager
MongoClient
creation with a factory pattern that creates clients on-demand - Refactors all MongoDB operations to use try-with-resources for client lifecycle management
- Adds integration test to verify SRV connection string handling
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
File | Description |
---|---|
MongoClientFactory.java | New factory class that creates MongoClient instances lazily to avoid DNS lookups during initialization |
MongoClientModule.java | Updated to bind MongoClientFactory instead of creating MongoClient eagerly |
MongoSession.java | Refactored to use factory and wrap all operations in try-with-resources blocks |
MongoPageSource.java | Updated to create and manage client lifecycle through the session |
MongoPageSink.java | Simplified to use session's insertMany method instead of direct collection access |
MongoMetadata.java | Updated to use session's createClient method for client access |
TestMongoPlugin.java | Added integration test for MongoDB SRV connection handling |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
plugin/trino-mongodb/src/main/java/io/trino/plugin/mongodb/MongoSession.java
Outdated
Show resolved
Hide resolved
plugin/trino-mongodb/src/main/java/io/trino/plugin/mongodb/MongoSession.java
Outdated
Show resolved
Hide resolved
plugin/trino-mongodb/src/main/java/io/trino/plugin/mongodb/MongoSession.java
Outdated
Show resolved
Hide resolved
ab49a6b
to
115f0af
Compare
plugin/trino-mongodb/src/main/java/io/trino/plugin/mongodb/MongoClientModule.java
Show resolved
Hide resolved
plugin/trino-mongodb/src/main/java/io/trino/plugin/mongodb/MongoClientModule.java
Outdated
Show resolved
Hide resolved
❤️ |
plugin/trino-mongodb/src/main/java/io/trino/plugin/mongodb/MongoPageSink.java
Show resolved
Hide resolved
115f0af
to
21a68b2
Compare
public void close() | ||
{ | ||
client.close(); | ||
getClient().close(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This requires a comment that this is lazy-loaded shared client instance.
Otherwise it look like we're creating a new client just to close it.
Alternatively, we could inject supplier without memoization and explicitly cache the client on a field in here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me leave a comment instead of injecting a supplier.
plugin/trino-mongodb/src/test/java/io/trino/plugin/mongodb/TestMongoPlugin.java
Outdated
Show resolved
Hide resolved
The existing MongoClientModule.defaultConfigurator looks up DNS record when SRV is used.
21a68b2
to
518a6ba
Compare
Description
The existing
MongoClientModule. defaultConfigurator
looks up DNS record when SRV is used.Connectors shouldn't try accessing remote datasources during the initialization.
Release notes
(x) This is not user-visible or is docs only, and no release notes are required.