Skip to content

Remove per-benchmark wiggle room#546

Merged
glbrntt merged 1 commit intoapple:mainfrom
glbrntt:remove-wiggle-room
Jul 4, 2025
Merged

Remove per-benchmark wiggle room#546
glbrntt merged 1 commit intoapple:mainfrom
glbrntt:remove-wiggle-room

Conversation

@glbrntt
Copy link
Contributor

@glbrntt glbrntt commented Jul 4, 2025

Motivation:

The benchmarks have a 2000 alloc wiggle room defined at the per-test level. It's not obvious why this is there as the numbers seem stable. Their presence allows for improvements/regressions to go unnoticed which isn't good.

Modifications:

  • Remove the wiggle room
  • Rename the benchmark target from NIOSSHBenchmarks to NIOSSLBenchmarks

Result:

Easier to catch regressions/improvements

Motivation:

The benchmarks have a 2000 alloc wiggle room defined at the per-test
level. It's not obvious why this is there as the numbers seem stable.
Their presence allows for improvements/regressions to go unnoticed which
isn't good.

Modifications:

- Remove the wiggle room
- Rename the benchmark target from NIOSSHBenchmarks to NIOSSLBenchmarks

Result:

Easier to catch regressions/improvements
@glbrntt glbrntt added the semver/none No version bump required. label Jul 4, 2025
@glbrntt glbrntt marked this pull request as ready for review July 4, 2025 12:17
@glbrntt glbrntt enabled auto-merge (squash) July 4, 2025 12:19
@glbrntt glbrntt merged commit f76bfe9 into apple:main Jul 4, 2025
40 of 41 checks passed
@glbrntt glbrntt deleted the remove-wiggle-room branch July 4, 2025 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver/none No version bump required.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants