Skip to content

Conversation

blaginin
Copy link
Contributor

Part of #15791 .

Surpasses part of #16978

blaginin and others added 30 commits October 16, 2025 10:58
Replace old-style expected string arrays with insta snapshot assertions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Replace old-style expected string arrays with insta snapshot assertions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Replace old-style expected string arrays with insta snapshot assertions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Replace old-style expected string arrays with insta snapshot assertions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Replace old-style expected string arrays with insta snapshot assertions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
- smj_join_key_ordering
- reorder_join_keys_to_left_input (inline snapshots with filter)
- parallelization_ignores_limit
- parallelization_prior_to_sort_preserving_merge
- parallelization_sort_preserving_merge_with_union
- parallelization_does_not_benefit

Replace old-style expected string arrays with insta inline snapshot assertions.
For reorder_join_keys_to_left_input, use regex filter to replace all join_type
values with '...' and add separate assertion for top join type.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Migrated the following tests to use inline insta snapshots:
- repartition_sorted_limit
- repartition_sorted_limit_with_filter
- repartition_ignores_limit
- repartition_ignores_union
- repartition_through_sort_preserving_merge
- repartition_ignores_sort_preserving_merge
- repartition_ignores_sort_preserving_merge_with_union
- repartition_does_not_destroy_sort
- repartition_does_not_destroy_sort_more_complex
- repartition_ignores_transitively_with_projection
- repartition_transitively_past_sort_with_projection

All tests pass without --accept flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Migrated the following tests to use inline insta snapshots:
- remove_redundant_roundrobins
- remove_unnecessary_spm_after_filter
- preserve_ordering_through_repartition
- no_need_for_sort_after_filter
- do_not_preserve_ordering_through_repartition3
- do_not_add_unnecessary_hash
- do_not_add_unnecessary_hash2
- optimize_away_unnecessary_repartition

All tests pass without --accept flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Migrated the following tests to use inline insta snapshots:
- optimize_away_unnecessary_repartition2
- parallelization_ignores_transitively_with_projection_csv

All tests pass without --accept flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Migrated the following tests to use inline insta snapshots:
- parallelization_single_partition
- parallelization_multiple_files

All tests pass without --accept flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
…tputs

Migrated the following tests to use inline insta snapshots:
- repartition_transitively_with_projection
- repartition_transitively_past_sort_with_filter
- repartition_transitively_past_sort_with_projection_and_filter

These tests have different outputs for DISTRIB_DISTRIB_SORT vs SORT_DISTRIB_DISTRIB,
so each gets two separate inline snapshots.

All tests pass without --accept flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Migrated test_distribute_sort_parquet to use inline insta snapshots.

This test uses different optimizer configurations:
- Initial plan before optimization
- After Run::Distribution
- After Run::Distribution + Run::Sorting

All three states now use inline snapshots instead of string arrays.

All tests pass without --accept flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Migrated test_distribute_sort_memtable to use inline insta snapshot.

This test checks the final optimized plan after creating and executing
a SQL query on a MemTable.

All tests pass without --accept flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
This test uses a loop to test different compression types with conditional
logic for expected outputs. Used insta::allow_duplicates! to handle inline
snapshots within the loop.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
…to insta snapshots

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
blaginin and others added 9 commits October 20, 2025 20:23
Add hide_first() helper function that replaces the first occurrence
of a regex pattern in a plan, returning the captured value and modified
string. Use this to test join key reordering across multiple join types
while keeping snapshots readable.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Converted the complex multi_smj_joins test to use insta snapshot testing
with filters. This test loops over 6 join types and tests multiple
configurations with different optimizer runs.

Key changes:
- Wrapped loop with insta::allow_duplicates!
- Added Settings filter to replace join_type={actual_value} with join_type=...
- Converted all test_config.run() calls to run2() + assert_plan!() with inline snapshots
- Preserved all existing comments explaining test behavior
- Removed unused join_plan helper functions and dynamic string variables

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Converted the reorder_join_keys_to_right_input test to use insta snapshot
testing with the hide_first helper function. This test loops over 8 join
types and verifies that join key ordering is adjusted based on children inputs.

Key changes:
- Wrapped loop with insta::allow_duplicates!
- Used hide_first() to replace only the first join_type occurrence with "..."
- Converted assert_plan_txt! to insta::assert_snapshot! with inline snapshot
- Preserved comment explaining the test behavior
- Removed dynamic top_join_plan string variable

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Migrated the last two tests using assert_plan_txt! macro to insta snapshots
and removed the macro entirely.

Tests migrated:
- do_not_put_sort_when_input_is_invalid
- put_sort_when_input_is_valid

Key changes:
- Converted assert_plan_txt! calls to assert_plan! with inline snapshots
- Preserved all comments explaining expected behavior
- Removed assert_plan_txt! macro definition
- Removed unused get_plan_string import

The assert_plan_txt! macro is now completely eliminated from the codebase.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@blaginin blaginin changed the title Move enforce distrivution Insta for enforce_distrubution Oct 20, 2025
@github-actions github-actions bot added the core Core DataFusion crate label Oct 20, 2025
let mut settings = Settings::clone_current();
settings.add_filter(&format!("join_type={join_type}"), "join_type=...");

#[rustfmt::skip]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screen.Recording.2025-10-21.at.11.25.18.mov

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant