Summary
The GLR-specific benchmarks in the performance suite are measuring simple Vec operations, not the sophisticated GLR parsing algorithms they claim to test.
Evidence
Fork Operations Benchmark:
group.bench_function("single_fork", |b| {
b.iter(|| {
let mut stacks = vec![vec![1, 2, 3]];
// Simulate fork
let forked = stacks[0].clone();
stacks.push(forked);
black_box(stacks)
});
});
Multiple Forks Benchmark:
group.bench_function("multiple_forks_10", |b| {
b.iter(|| {
let mut stacks = vec![vec![1, 2, 3, 4, 5]];
for _ in 0..10 {
let forked = stacks[0].clone();
stacks.push(forked);
}
black_box(stacks)
});
});
What This Actually Measures
- Vec::clone() performance (~85ns) - Not GLR fork complexity
- Vec::push() operations - Not parse stack management
- Simple integer arrays - Not parse states, symbols, or grammar rules
- No conflict resolution - Not shift/reduce or reduce/reduce handling
- No parse table access - Not LR(1) state transitions
What GLR Benchmarks Should Measure
- Parse state forking with actual grammar states
- Stack merging when paths converge
- Conflict resolution (shift/reduce, reduce/reduce)
- Parse table lookups and transitions
- Symbol handling and reduction operations
- Memory management for parse forests
- Ambiguity tracking and disambiguation
Impact
- Misleading performance characteristics: Vec operations are vastly simpler than GLR parsing
- False optimization targets: Developers might optimize Vec::clone instead of real bottlenecks
- Incomplete testing: Real GLR performance remains unmeasured
- Architecture validation missing: No verification that GLR design performs well
Required Actions
- Remove or clearly label mock benchmarks as "infrastructure only"
- Implement real GLR benchmarks once parser is functional
- Test actual conflict scenarios with ambiguous grammars
- Measure parse forest operations and memory usage
- Validate scalability with complex grammar states
Priority
MEDIUM - Important for accurate performance assessment once parser works
Context
Part of comprehensive performance analysis revealing that claimed GLR performance is based on simple data structure operations, not parsing complexity.
Summary
The GLR-specific benchmarks in the performance suite are measuring simple Vec operations, not the sophisticated GLR parsing algorithms they claim to test.
Evidence
Fork Operations Benchmark:
Multiple Forks Benchmark:
What This Actually Measures
What GLR Benchmarks Should Measure
Impact
Required Actions
Priority
MEDIUM - Important for accurate performance assessment once parser works
Context
Part of comprehensive performance analysis revealing that claimed GLR performance is based on simple data structure operations, not parsing complexity.