Adding Arena Hard Auto

[asuvarna31]

Sample command :

python -m eval.eval\
    --model hf\
    --tasks arena_hard_auto\
    --model_args "pretrained=Qwen/Qwen1.5-1.8B-Chat" \
    --batch_size 2 \
    --output_path logs \
    --annotator_model gpt-4o-mini-2024-07-18 \ 

Sample Output :

"results": {
        "score": 5.56,
        "avg_tokens": 635.0
      }

[asuvarna31]

@neginraoof @RyanMarten feel free to review and test this.

[neginraoof]

Thanks a lot! Can you also update the reproduced_benchmarks.md with results?

[neginraoof]

Thanks a lot @asuvarna31 !
Can we just avoid read/write to file? seems like we can just use the input model_results here?

[asuvarna31]

Addressed your comments @neginraoof