Skip to content

Conversation

@rluvaton
Copy link
Member

Which issue does this PR close?

N/A

Rationale for this change

I have a PR to improve zip perf for scalar but I don't see any benchmarks for it:

What changes are included in this PR?

created zip benchmarks for scalar and non scalar with different masks

Are these changes tested?

N/A

Are there any user-facing changes?

Nope

@github-actions github-actions bot added the arrow Changes to the arrow crate label Oct 19, 2025
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rluvaton

I did a quick profile on this benchmark and confirmed that zip was indeed taking most of the time, as expected

Screenshot 2025-10-19 at 3 41 18 PM

/// filling it with random bytes with lengths in the specified range,
/// all starting with the provided `prefix`, generated using the provided `seed`.
///
pub fn create_binary_array_with_len_range_and_prefix_and_seed<Offset: OffsetSizeTrait>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The maze of these functions is becoming bewildering. Maybe it is time (not in this PR) to make some sort of builder API or something

) -> GenericBinaryArray<Offset> {
let rng = &mut seedable_rng();
let range_rng = &mut seedable_rng();
create_binary_array_with_seed(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a behavior change, right (now it uses a fixed seed).

But that is an improvement as it now matches the doc comment

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a behavioral change as it used the same seed

@alamb alamb merged commit f131b54 into apache:main Oct 19, 2025
24 checks passed
@rluvaton rluvaton deleted the create_zip_benchmark branch October 19, 2025 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants