Skip to content

GLR benchmarks measure Vec::clone(), not actual GLR parsing operations #75

@EffortlessSteven

Description

@EffortlessSteven

Summary

The GLR-specific benchmarks in the performance suite are measuring simple Vec operations, not the sophisticated GLR parsing algorithms they claim to test.

Evidence

Fork Operations Benchmark:

group.bench_function("single_fork", |b| {
    b.iter(|| {
        let mut stacks = vec![vec![1, 2, 3]];
        // Simulate fork
        let forked = stacks[0].clone();
        stacks.push(forked);
        black_box(stacks)
    });
});

Multiple Forks Benchmark:

group.bench_function("multiple_forks_10", |b| {
    b.iter(|| {
        let mut stacks = vec![vec![1, 2, 3, 4, 5]];
        for _ in 0..10 {
            let forked = stacks[0].clone();
            stacks.push(forked);
        }
        black_box(stacks)
    });
});

What This Actually Measures

  • Vec::clone() performance (~85ns) - Not GLR fork complexity
  • Vec::push() operations - Not parse stack management
  • Simple integer arrays - Not parse states, symbols, or grammar rules
  • No conflict resolution - Not shift/reduce or reduce/reduce handling
  • No parse table access - Not LR(1) state transitions

What GLR Benchmarks Should Measure

  • Parse state forking with actual grammar states
  • Stack merging when paths converge
  • Conflict resolution (shift/reduce, reduce/reduce)
  • Parse table lookups and transitions
  • Symbol handling and reduction operations
  • Memory management for parse forests
  • Ambiguity tracking and disambiguation

Impact

  • Misleading performance characteristics: Vec operations are vastly simpler than GLR parsing
  • False optimization targets: Developers might optimize Vec::clone instead of real bottlenecks
  • Incomplete testing: Real GLR performance remains unmeasured
  • Architecture validation missing: No verification that GLR design performs well

Required Actions

  1. Remove or clearly label mock benchmarks as "infrastructure only"
  2. Implement real GLR benchmarks once parser is functional
  3. Test actual conflict scenarios with ambiguous grammars
  4. Measure parse forest operations and memory usage
  5. Validate scalability with complex grammar states

Priority

MEDIUM - Important for accurate performance assessment once parser works

Context

Part of comprehensive performance analysis revealing that claimed GLR performance is based on simple data structure operations, not parsing complexity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions