v0.0.2

What's Changed

• Add MT-Bench and PR-Bench Support by @danmcp in #9
• Implement MMLU_Evaluator.run() by @alinaryan in #10
• Updating to > 1.0 openai by @danmcp in #17
• mmlu branch run() complete by @alimaredia in #19
• add cpu support for MMLU bench by @cdoern in #21

New Contributors

• @danmcp made their first contribution in #9
• @alinaryan made their first contribution in #10
• @alimaredia made their first contribution in #19
• @cdoern made their first contribution in #21

Full Changelog: v0.0.1...v0.0.2

v0.0.1

What's Changed

• Fixes pyproject.toml by @bjhargrave in #4
• Bump step-security/harden-runner from 2.8.0 to 2.8.1 by @dependabot in #5
• Bump rojopolis/spellcheck-github-actions from 0.37.0 to 0.38.0 by @dependabot in #7
• Bump pypa/gh-action-pypi-publish from 1.8.14 to 1.9.0 by @dependabot in #8
• Initial skeleton for Evaluator classes and exceptions by @nathan-weinberg in #6

New Contributors

• @bjhargrave made their first contribution in #4

Full Changelog: v0.0.1rc1...v0.0.1

v0.0.1rc1

What's Changed

• Bootstrap eval repo by @nathan-weinberg in #1
• Bump step-security/harden-runner from 2.8.0 to 2.8.1 by @dependabot in #2
• Copy pypi release automation from instructlab/sdg repo by @nathan-weinberg in #3

New Contributors

• @nathan-weinberg made their first contribution in #1
• @dependabot made their first contribution in #2

Full Changelog: https://github.com/instructlab/eval/commits/v0.0.1rc1