AlgoGrader

A CLI tool for testing algorithmic solutions against test cases with time and memory limits. Designed for competitive programming — test your solutions locally before submitting.

Supported Languages

Language	Extension	Compiler/Runtime
C	.c	gcc
C++	.cpp	g++
Java	.java	javac + java
Python	.py	python3
Rust	.rs	rustc
Go	.go	go build
OCaml	.ml	ocamlfind/ocamlopt
Standard ML	.sml	MLton
Kotlin	.kt	kotlinc + java
Haskell	.hs	ghc
Prolog	.pl	swipl (SWI-Prolog)

Installation

git clone https://github.com/geokoko/AlgoGrader.git
cd AlgoGrader
pip install -r requirements.txt

Usage

python3 main.py <source_file> [test_path] [-t TIME_LIMIT] [-m MEMORY_LIMIT] [-O OPTIMIZATION_LEVEL]

Arguments

• source_file (required): Path to your solution source file.
• test_path (optional): Path to a directory or zip file containing test cases. If omitted, looks in test_cases/<program_name>_tests/.

Options

• -t, --time-limit: Time limit per test in seconds (default: 12).
• -m, --memory-limit: Memory limit in MB (default: 128).
• -O, --optimization-level: Compiler/runtime optimization level: default, 0, 1, 2, 3, or s for size. The default keeps each language's existing behavior; explicit values map to the closest compiler or runtime optimization mode available for that language.

Examples

# Test a C++ solution against tests in a folder
python3 main.py solution.cpp ./my_tests/

# Test with custom limits
python3 main.py solution.py ./tests/ -t 5 -m 256

# Compile with a specific optimization level
python3 main.py solution.cpp ./tests/ -O 3

# Test from a zip file
python3 main.py solution.rs tests.zip

# Legacy mode (uses test_cases/solution_tests/ directory)
python3 main.py solution.cpp

Test Case Format

Two formats are supported:

Format 1: inputN.txt / outputN.txt

tests/
  input1.txt
  input2.txt
  output1.txt
  output2.txt

Format 2: *.in / *.out

tests/
  01.in
  01.out
  02.in
  02.out

Verdicts

Verdict	Meaning
Accepted	Output matches expected
Wrong Answer	Output differs from expected
Time Limit Exceeded	Program ran longer than the time limit
Runtime Error	Non-zero exit code (assertion failure, etc.)
Segmentation Fault	Memory access violation (SIGSEGV)
Floating Point Error	Division by zero, etc. (SIGFPE)
Killed (likely MLE)	Process killed by OS, usually due to memory limit (SIGKILL)

Notes

• Output comparison is whitespace-tolerant: trailing whitespace per line and trailing empty lines are ignored.
• Memory limits are enforced on Linux only (via RLIMIT_AS).
• stderr output from the tested program is ignored (many CP solutions write debug info to stderr).
• For Prolog, the solution must define a main predicate.
• For Java, the class name must match the filename.