feat[venom]: loop invariant

[harkal]

What I did

How I did it

How to verify it

Commit message

Commit message for the final, squashed PR. (Optional, but reviewers will appreciate it! Please see our commit message style guide for what we would ideally like to see in a commit message.)

Description for the changelog

Cute Animal Picture

[harkal]

@HodanPlodky this is my implementation I told you I will push.
The comparison against #4175 by Codex
I suggest we take the good ideas from each to make one final implementation. What do you think?

LICM Implementation Comparison Report                                                                                                  
                                                                                                                                         
  Overview                                                                                                                               
  ┌────────────────┬─────────────────────────────────────────┬───────────────────────────────────────┐                                   
  │     Aspect     │   feat/venom/loop_invariant (current)   │         loop_invariant (old)          │                                   
  ├────────────────┼─────────────────────────────────────────┼───────────────────────────────────────┤                                   
  │ File           │ loop_invariant_code_motion.py (169 LOC) │ loop_invariant_hosting.py (138 LOC)   │                                   
  ├────────────────┼─────────────────────────────────────────┼───────────────────────────────────────┤                                   
  │ Class          │ LoopInvariantCodeMotionPass             │ LoopInvariantHoisting                 │                                   
  ├────────────────┼─────────────────────────────────────────┼───────────────────────────────────────┤                                   
  │ Loop Detection │ Self-contained in pass                  │ Separate NaturalLoopDetectionAnalysis │                                   
  ├────────────────┼─────────────────────────────────────────┼───────────────────────────────────────┤                                   
  │ Status         │ Integrated in O2/O3/Os                  │ Not integrated in pipeline            │                                   
  └────────────────┴─────────────────────────────────────────┴───────────────────────────────────────┘                                   
  ---                                                                                                                                    
  Algorithm Comparison                                                                                                                   
                                                                                                                                         
  Loop Detection                                                                                                                         
  ┌─────────────────────────────────────────────┬─────────────────────────────────────────────┐                                          
  │               Current Branch                │                 Old Branch                  │                                          
  ├─────────────────────────────────────────────┼─────────────────────────────────────────────┤                                          
  │ Uses dominator tree for back-edge detection │ Uses DFS stack for back-edge detection      │                                          
  ├─────────────────────────────────────────────┼─────────────────────────────────────────────┤                                          
  │ dom.dominates(succ, bb) → back edge         │ succ in stack → back edge                   │                                          
  ├─────────────────────────────────────────────┼─────────────────────────────────────────────┤                                          
  │ Computes preheader explicitly               │ Uses cfg_in(header).first() as hoist target │                                          
  ├─────────────────────────────────────────────┼─────────────────────────────────────────────┤                                          
  │ Tracks multiple latches per loop            │ Implicit single latch assumption            │                                          
  └─────────────────────────────────────────────┴─────────────────────────────────────────────┘                                          
  Verdict: Current branch is more correct. Dominator-based detection is the canonical approach. Old branch's first() heuristic is        
  fragile.                                                                                                                               
                                                                                                                                         
  Invariant Detection                                                                                                                    
  ┌───────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────────┐              
  │                      Current Branch                       │                        Old Branch                         │              
  ├───────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────┤              
  │ Requires read_effects == EMPTY AND write_effects == EMPTY │ Only checks (read_effects & loop_write_effects) != EMPTY  │              
  ├───────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────┤              
  │ Uses domination check for safety                          │ No domination check                                       │              
  ├───────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────┤              
  │ Fixed-point iteration to cascade dependencies             │ Per-instruction scan with separate _assign_dependencies() │              
  └───────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────┘              
  Verdict: Current branch is safer. Old branch's looser effect check could hoist memory reads unsafely.                                  
                                                                                                                                         
  Safety Checks                                                                                                                          
  ┌───────────────────────────────┬─────────────────────────────┬───────────────────────────────────────┐                                
  │             Check             │           Current           │                  Old                  │                                
  ├───────────────────────────────┼─────────────────────────────┼───────────────────────────────────────┤                                
  │ Dominates all latches         │ ✅ Yes                      │ ❌ No                                 │                                
  ├───────────────────────────────┼─────────────────────────────┼───────────────────────────────────────┤                                
  │ Pure computation (no effects) │ ✅ Strict (both read/write) │ ⚠️ Partial (only read vs loop writes) │                                
  ├───────────────────────────────┼─────────────────────────────┼───────────────────────────────────────┤                                
  │ Handles conditional paths     │ ✅ Via domination           │ ❌ May hoist incorrectly              │                                
  ├───────────────────────────────┼─────────────────────────────┼───────────────────────────────────────┤                                
  │ Handles nested loops          │ ✅ Inner-first ordering     │ ⚠️ Dict iteration order               │                                
  └───────────────────────────────┴─────────────────────────────┴───────────────────────────────────────┘                                
  ---                                                                                                                                    
  Unique Features                                                                                                                        
                                                                                                                                         
  Current Branch Only                                                                                                                    
                                                                                                                                         
  1. LoopInfo dataclass - Clean encapsulation of loop metadata                                                                           
  2. Preheader validation - Requires exactly one non-loop predecessor                                                                    
  3. Latch domination check - Prevents unsafe hoisting from conditional paths                                                            
  4. Inner-first loop ordering - Enables cascading hoists through nested loops                                                           
  5. DFG invalidation - Correct analysis cache management                                                                                
                                                                                                                                         
  Old Branch Only                                                                                                                        
                                                                                                                                         
  1. Separate NaturalLoopDetectionAnalysis - Reusable analysis module                                                                    
  2. _assign_dependencies() - Special handling for assign chains                                                                         
  3. returndatasize exclusion - Specific volatile instruction handling                                                                   
  4. Parametric tests - @pytest.mark.parametrize for depth/count combinations                                                            
                                                                                                                                         
  ---                                                                                                                                    
  Test Coverage                                                                                                                          
  ┌──────────────────────────────────────┬──────────────────────────────────────────────────┐                                            
  │            Current Branch            │                    Old Branch                    │                                            
  ├──────────────────────────────────────┼──────────────────────────────────────────────────┤                                            
  │ 3 focused unit tests                 │ 4 parametric tests (3×3 = 9 cases each)          │                                            
  ├──────────────────────────────────────┼──────────────────────────────────────────────────┤                                            
  │ Uses PrePostChecker helper           │ Uses assert_ctx_eq                               │                                            
  ├──────────────────────────────────────┼──────────────────────────────────────────────────┤                                            
  │ Tests: simple, variant, branch-local │ Tests: detection, simple, dependent, unhoistable │                                            
  ├──────────────────────────────────────┼──────────────────────────────────────────────────┤                                            
  │ + codesize regression test           │ No codesize test                                 │                                            
  └──────────────────────────────────────┴──────────────────────────────────────────────────┘                                            
  Verdict: Old branch has better test coverage via parametric testing across nested loop depths.                                         
                                                                                                                                         
  ---                                                                                                                                    
  Recommendations                                                                                                                        
                                                                                                                                         
  1. Base Implementation: Current Branch (feat/venom/loop_invariant)                                                                     
                                                                                                                                         
  The current branch is the better foundation because:                                                                                   
  - Correct dominator-based loop detection                                                                                               
  - Strict safety checks (both read/write effects must be empty)                                                                         
  - Proper latch domination check prevents unsafe hoisting                                                                               
  - Already integrated into O2/O3/Os pipeline                                                                                            
  - Cleaner code structure with LoopInfo dataclass                                                                                       
                                                                                                                                         
  2. Features to Merge from Old Branch                                                                                                   
                                                                                                                                         
  1. Separate NaturalLoopDetectionAnalysis                                                                                               
    - Extract loop detection into reusable analysis module                                                                               
    - Other passes (loop unrolling, strength reduction) could benefit                                                                    
    - Use current branch's dominator-based algorithm, not old DFS approach                                                               
  2. Parametric Test Structure                                                                                                           
    - Adopt @pytest.mark.parametrize(depth, count) pattern                                                                               
    - Much better coverage for nested loops                                                                                              
    - Add to current test file                                                                                                           
  3. returndatasize Exclusion (investigate)                                                                                              
    - Old branch explicitly excludes this opcode                                                                                         
    - Current branch relies on is_volatile - verify this covers it                                                                       
                                                                                                                                         
  3. Priority Order for Merging                                                                                                          
                                                                                                                                         
  1. High: Parametric tests from old branch → better coverage                                                                            
  2. Medium: Extract loop detection to separate analysis (architectural cleanup)                                                         
  3. Low: _assign_dependencies handling (current fixed-point handles this implicitly)                                                    
                                                                                                                                         
  ---                                                                                                                                    
  Summary                                                                                                                                
                                                                                                                                         
  Winner: Current branch (feat/venom/loop_invariant)                                                                                     
                                                                                                                                         
  The current implementation is more correct and safer. The old branch has some good ideas (reusable analysis, parametric tests) but has 
  algorithmic issues (no domination check, loose effect checking).                                                                       
                                                                                                                                         
  Merge the test patterns from the old branch into the current implementation, and consider extracting loop detection into a separate    
  analysis module for reuse.

[codecov]

Codecov Report

❌ Patch coverage is 93.42105% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.90%. Comparing base (cf880fd) to head (763deda).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
vyper/venom/passes/loop_invariant_code_motion.py	93.37%	5 Missing and 5 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4819      +/-   ##
==========================================
+ Coverage   92.89%   92.90%   +0.01%     
==========================================
  Files         155      156       +1     
  Lines       21744    21896     +152     
  Branches     3885     3929      +44     
==========================================
+ Hits        20199    20343     +144     
- Misses       1024     1027       +3     
- Partials      521      526       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.