Skip to content

harshraj172/ConsistencyBench

Repository files navigation

ConsistencyBench

Setup

Install the requirements

conda create -n venv python=3.10
conda activate venv
pip install -r requirements.txt

To start generation and scoring

Set the arguments from run_eval.py

python run_eval.py --openai_api_key <OPENAI_API_KEY>

Example Output file: consistencybench/result_gpt-3.5-turbo_paraphrasing.csv

Example Jupyter Notebook: example.ipynb

About

Benchmark for evaluating consistency in LLMs

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 3

  •  
  •  
  •