Trino: Add Trino Docker Compose for integration testing #2220
+573
−108
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dingo4dev
force-pushed
the
test-trino-integration
branch
3 times, most recently
from
fdb090e to
04d1f61
Compare
Adds a Docker Compose configuration for setting up Trino with Iceberg, including support for Hive Metastore and REST catalog types. This allows for easier testing and development with Trino and Iceberg. closes apache#2219
- Introduced new tests for registering tables and schema existence in Trino. - Added Trino connection fixtures to support integration testing. - Updated `pyproject.toml` to include the Trino dependency. - Enhanced the Makefile with a new command to run Trino integration tests.
Add Trino as alternative tool for test uuid partitions as Java Iceberg 1.9.2 in spark is not yet supported. <!--related to apache#2002-->
dingo4dev
force-pushed
the
test-trino-integration
branch
from
04d1f61 to
a0e1a11
Compare
dingo4dev
changed the title
There was a problem hiding this comment.
Pull Request Overview
This PR introduces Trino integration testing capabilities to the PyIceberg project by adding Docker Compose configuration and test fixtures for Trino services. The changes enable testing of Iceberg operations through Trino's SQL interface to ensure compatibility and proper integration between PyIceberg and Trino.
Key changes include:
- Added Trino Docker services configuration with both REST and Hive catalog support
- Introduced new test fixtures for Trino database connections
- Created integration tests that verify PyIceberg operations work correctly with Trino
Reviewed Changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
tests/integration/test_writes/test_writes.py |
Added new Trino-specific UUID partitioning test and skipped existing Spark test |
tests/integration/test_rest_catalog.py |
Added test to verify Iceberg namespace appears as Trino schema |
tests/integration/test_register_table.py |
Added test to verify table registration works with Trino |
tests/conftest.py |
Added Trino connection fixtures and command-line options |
pyproject.toml |
Added Trino dependency and integration test marker |
dev/trino/catalog/*.properties |
Trino catalog configuration files for REST and Hive catalogs |
dev/run-trino.sh |
Shell script to manage Trino Docker services |
dev/docker-compose-trino.yml |
Standalone Trino Docker Compose configuration |
dev/docker-compose-integration.yml |
Added Trino service to main integration environment |
Makefile |
Added target for running Trino integration tests |
Comments suppressed due to low confidence (2)
tests/integration/test_register_table.py:109
- [nitpick] The test doesn't handle the case where Trino connection might fail or timeout. Consider adding error handling or retries for database connectivity issues that could make the test flaky.
assert table_name in inspect(trino_conn).get_table_names(schema=namespace)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
This pull request introduces a Trino service to the Docker Compose development environment, addressing the need for broader integration testing capabilities as discussed in issue #2219
Are these changes tested?
Are there any user-facing changes?