Updating release_branch#6
Merged
MT255026 merged 36 commits intoTeradata:release_branch_0.0.6from Oct 6, 2025
Merged
Conversation
* require polars and deltalake recent versions with fix * move overwrite schema to delta write options * disallow mode arg in delta_write_options * ruff and fix of wrong delta write option syntax * rollback dependency bump * add safeguard for incompatible polars and deltalake versions * add compatibility for older versions * fix tests with overwrite schema invalid passing method * remove legacy overwrite_schema mode and raise valueErr for incompatible versions * docstring and error clarification * additional safeguard for deltalake old version * bump uv.lock deltalake * add workaround for polars panic in tests on recent versions * fix wrong walrus use * update unreleased changelog
* use engine argument instead of streaming argument * ruff * ignore pyright for older syntax * ruff * parse polars version * require polars and deltalake recent versions with fix * move overwrite schema to delta write options * disallow mode arg in delta_write_options * ruff and fix of wrong delta write option syntax * rollback dependency bump * add safeguard for incompatible polars and deltalake versions * add compatibility for older versions * fix tests with overwrite schema invalid passing method * remove legacy overwrite_schema mode and raise valueErr for incompatible versions * docstring and error clarification * additional safeguard for deltalake old version * bump uv.lock deltalake * add workaround for polars panic in tests on recent versions * update changelog * cleanup version parse multiple syntax * readjust changelog after yanked release
* [pipes-java] fix JAR building and test dependencies fixes #221 - the jar building does not include the dependencies anymore - in order to run the MainTest class in the tests the application plugin is used to build a start script and libs directory - the test dependencies are declared separately now so that a user of the library doesn't depend on them - mockito was needed in the library to create a mock class. to remove mockito from the library the PipesContext was converted to an interface with an empty implementation instead of the mockito mock * [pipes-java] move javadoc comments to the PipesContext interface * [pipes-java] remove unneeded mockito dependency
…220) * [dagster-polars] fix: sanitize config passed to polars.scan_parquet * doc: update CHANGELOG.md --------- Co-authored-by: Orion Yeung <11580988+orionyeung001@users.noreply.github.com>
21d3447 removed setuptools-scm, which was responsible to include package files tracked by Git into the sdist. This lead version 0.27.5 missing 'dagster_polars.io_managers', b/c the default setuptools discovery does not include subpackages. Assuming that removing setuptools-scm was done for a reason, another way to make sure subpackages are included is by using 'setuptools.packages.find'.
* [dagster-dataform] Initial README * [dagster-dataform] Root files * [dagster-dataform] Integration modules * [dagster-dataform] Integration unit tests * [dagster-dataform] Add prereq for freshness checks --------- Co-authored-by: 2705815 <Justin.Cohen2@aetna.com>
* [dagster-dataform] Initial README * [dagster-dataform] Root files * [dagster-dataform] Integration modules * [dagster-dataform] Integration unit tests * [dagster-dataform] Add prereq for freshness checks * [dagster-dataform] import module objects into __init__.py * [dagster-dataform] fix import names * [dagster-dataform] Resolve linting warnings. * [dagster-dataform] Add # noqa for stubborn type warnings --------- Co-authored-by: 2705815 <Justin.Cohen2@aetna.com>
* [dagster-dataform] Initial README * [dagster-dataform] Root files * [dagster-dataform] Integration modules * [dagster-dataform] Integration unit tests * [dagster-dataform] Add prereq for freshness checks * [dagster-dataform] import module objects into __init__.py * [dagster-dataform] fix import names * [dagster-dataform] Resolve linting warnings. * [dagster-dataform] Add # noqa for stubborn type warnings * [dagster-dataform] Fix; remove non-existent function import * [dagster-dataform] Modify ignore comments to silence certain pyright errors. --------- Co-authored-by: 2705815 <Justin.Cohen2@aetna.com>
…Checks (#230) * [dagster-dataform] Initial README * [dagster-dataform] Root files * [dagster-dataform] Integration modules * [dagster-dataform] Integration unit tests * [dagster-dataform] Add prereq for freshness checks * [dagster-dataform] import module objects into __init__.py * [dagster-dataform] fix import names * [dagster-dataform] Resolve linting warnings. * [dagster-dataform] Add # noqa for stubborn type warnings * [dagster-dataform] Fix; remove non-existent function import * [dagster-dataform] Modify ignore comments to silence certain pyright errors. * [dagster-dataform] Add support for inclusion patterns to help filter which invocations actions are detected and processed. Add asset checks processing. * [dagster-dataform] Add capability to create asset checks from Dataform Assertions in the repository. * [dagster-dataform] New utils function to handle creating AssetCheckEvaluation event from assertion action in workflow invocation * [dagster-dataform] Fix log message * [dagster-dataform] Due to resource refactoring, remove import of load functions * [dagster-dataform] Mods to conditional logic for handling asset check evaluations; improve logging * [dagster-dataform] Improve design of resource class by moving load functions into class as methods and call load functions on initializationto so that code location reload performs full refresh of remote Dataform assets * [dagster-dataform] Refactor function to obtain asset_key for AssetCheckEvaluation from new asset_checks property of resource class * [dagster-dataform] Add freshness lag minutes param to init method * [dagster-dataform] Tune tests given core code changes * [dagster-dataform] Tune orchestration schedule tests given core code changes * [dagster-dataform] Improve sensor logs readability * [dagster-dataform] add pydantic config class to __init__.py for usage when using custom failure notification jobs in the sensor. * [dagster-dataform] Fix a bug in the way that the ListWorkflowInvocations response was being handled, which was leading to some workflow invocations being absent * [dagster-dataform] Fix test_resource.py * [dagster-dataform] Add proper error handling to not fail the sensor tick when asset evaluation found for non existent asset check; tick was failing on if not asset_key as asset_key was not defined * [dagster-dataform] Create pydantic config class; move ops and jobs out of function; improve logging; add run request if asset_check_evaluation outcome failed with run config to pass metadata to job/op * [dagster-dataform] Remove unnecessary error log message * [dagster-dataform] Remove completed TODO * [dagster-dataform] Modify test accordingly to reflect assertion action in mock compilation result * [dagster-dataform] Add parametrize decorator to tests where it was missing; general improvements to existing tests; add new tests for asset check functionality * [dagster-dataform] Remove unused code * [dagster-dataform] General improvements to existing configuration and mocks; add new mocks for asset check functionality * [dagster-dataform] Update lock file * [dagster-dataform] Run ruff format and check * [dagster-dataform] Rename op to proper name * [dagster-dataform] Correct return type. * Update readme * [dagster-dataform] Resolve or silence pyright warnings * [dagster-dataform] Run ruff format * [dagster-dataform] Convert to list to be able to check len of responses * [dagster-dataform] Run ruff format --------- Co-authored-by: 2705815 <Justin.Cohen2@aetna.com>
* feat: first rough draft - we stil have to figure out how to do the testing for CI * feat: no mocking, tests green * fix: make things work for good * chore: update docs for new API
* first commit * add quality check ci * update readme and add changelog * fix some linting * fix some linting
* Adding basic tests that append mode will be passed through handler correctly * Adding basic test that append mode works through table_writer * implementing append mode for table_writer * allow type handler to pass write mode via asset definition or output metadata. if provided, write mode on output metata will allow overriding write mode defined in asset definition metadata. * simplifying * factoring out test functionality * adding error handling and cleanup * Adding tests for append and overwrite write_modes * moor tests * formatting * refactoring * Adding changelog message * cleanup * Seems that you can't currently set output_metadata through build_output_context * updating test to show that output_metadata overrides defintion_metadtata * ruff * Fix tests * ruff * use if_not_exists instead of suppress * remove unused import * append() only allows Dict, not None * Add more specific changelog update * Add docstring example * Forgot that StrEnum is not available in python 3.10
* Release dagster-iceberg 0.3.5 * [github] consolidate workflows for quality check and integration releases * Potential fix for code scanning alert no. 72: Workflow does not contain permissions Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * Potential fix for code scanning alert no. 74: Workflow does not contain permissions Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * apply fix to release --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.