fix: add offline dataset support via LOCAL_DATASET_PATH

[xjli360]

This PR implements offline mode support for lighteval, allowing users to run evaluations without internet access.

By setting the environment variable:

export LOCAL_DATASET_PATH=/path/to/local/dataset

More details in: 802

[xjli360]

hi, I'd appreciate it if you could review my changes. I'm happy to make any adjustments or answer any questions.
Thank you for your time and for maintaining this project!

[mjpost]

@xjli360 I think this implementation is unnecessary. If you want to do offline evaluation, you can follow the following procedure:

• Run evaluation somewhere you have internet access. This will download files to ~/.cache/huggingface
• Copy that cache directory to your offline serve

I have done this successfully with a custom evaluation script.

[xjli360]

@xjli360 I think this implementation is unnecessary. If you want to do offline evaluation, you can follow the following procedure:

• Run evaluation somewhere you have internet access. This will download files to ~/.cache/huggingface
• Copy that cache directory to your offline serve

I have done this successfully with a custom evaluation script.

Thank you for the suggestion, but I believe this implementation is still necessary for the following reasons:

1. In many production or enterprise environments, machines with GPUs are often restricted from accessing the internet for security or policy reasons. In such cases, datasets can only be downloaded on CPU-only machines with internet access and then transferred manually to GPU machines. The proposed approach of relying on evaluation "somewhere with internet access" is not always feasible or convenient in these scenarios.

2. Additionally, not all resources required during evaluation are cached under ~/.cache/huggingface. For example, certain benchmarks—such as tinybenchmark—are downloaded at runtime and stored elsewhere. Relying solely on the Hugging Face cache does not guarantee complete offline compatibility.

Therefore, having explicit support for an offline dataset path (e.g., LOCAL_DATASET_PATH) provides a more robust and generalizable solution for offline evaluation in constrained environments.