Skip to content

Commit 0bbb0e8

Browse files
committed
docs: update eval
1 parent ddffc8b commit 0bbb0e8

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -228,7 +228,10 @@ You are strongly recommended to use a sandbox such as [docker](https://docs.dock
228228
# If you want to change the RAM address space limit (in MB, 128 GB by default): `--max-as-limit XXX`
229229
# If you want to change the RAM data segment limit (in MB, 4 GB by default): `--max-data-limit`
230230
# If you want to change the RAM stack limit (in MB, 4 MB by default): `--max-stack-limit`
231-
docker run -v $(pwd):/app bigcodebench/bigcodebench-evaluate:latest --subset [complete|instruct] --samples samples-sanitized-calibrated
231+
docker run -v $(pwd):/app bigcodebench/bigcodebench-evaluate:latest --subset [complete|instruct] --samples samples-sanitized-calibrated.jsonl
232+
233+
# If you only want to check the ground truths
234+
docker run -v $(pwd):/app bigcodebench/bigcodebench-evaluate:latest --subset [complete|instruct] --samples samples-sanitized-calibrated.jsonl --check-gt-only
232235
```
233236
234237
...Or if you want to try it locally regardless of the risks ⚠️:

0 commit comments

Comments
 (0)