Is your feature request related to a problem? Please describe.
The dokimos-examples module could benefit from additional example evaluators, which showcase more common evaluation patterns. This would allow developers to explore more real-world evaluation scenarios beyond exact match and regex in the repo.
Describe the solution you'd like
The examples module you be extended with some custom implementations for:
- semantic similarity / deduplication of context chunks
- A length-based evaluator that makes sure outputs are in certain length ranges
Is your feature request related to a problem? Please describe.
The dokimos-examples module could benefit from additional example evaluators, which showcase more common evaluation patterns. This would allow developers to explore more real-world evaluation scenarios beyond exact match and regex in the repo.
Describe the solution you'd like
The examples module you be extended with some custom implementations for: