Skip to content

Add reproducible performance benchmark suite - #396

Open
ousamabenyounes wants to merge 1 commit into
OpenBMB:mainfrom
ousamabenyounes:feat/issue-158
Open

Add reproducible performance benchmark suite#396
ousamabenyounes wants to merge 1 commit into
OpenBMB:mainfrom
ousamabenyounes:feat/issue-158

Conversation

@ousamabenyounes

Copy link
Copy Markdown

Summary

  • add an easy-to-run benchmark for model-load time, per-run RTF, batch throughput, and peak CUDA memory
  • emit reproducible JSON with actual hardware, device, optimization-wrapper, generation, and immutable Hub snapshot metadata
  • document benchmark usage, metric definitions, and a verified-results hardware comparison matrix

Validation

  • untouched baseline: 56 tests passed
  • exact upstream RED: benchmark test collection fails because the runner is absent
  • focused GREEN: 27 tests passed with 222/222 benchmark statements covered (100%)
  • full GREEN: 83 tests passed
  • Black and Flake8 passed for all changed Python files
  • source distribution and wheel built successfully; source archive contains the benchmark guide and runner
  • two fresh independent Codex review gates passed

Fixes #158

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] Create performance benchmark suite

1 participant