Skip to content

Conversation

evkotov
Copy link
Contributor

@evkotov evkotov commented Jul 18, 2025

Details:

  • rewrite StatefulSDPAFusion transformation
  • Fix PassConfig State Management in SymbolicOptimizations

Problem

SymbolicOptimizations::run_on_model() disabled EliminateSqueeze and EliminateUnsqueeze passes in shared PassConfig without properly restoring their original state. This broke subsequent transformations like NgramFusion that depend on these
passes for symbolic pattern matching.

Root Cause

The original restore logic was incomplete - PassConfig has three states (Default, Disabled, Enabled) but the code only handled two, incorrectly converting Default state to Enabled state after restoration.

Solution

  • Added PassConfig::restore_default() method to properly restore passes to default state
  • Enhanced SymbolicOptimizations to track and restore exact original state of each pass
  • Added exception safety to guarantee state restoration even on errors

Benefits

  • Fixes NgramFusion and other transformations that rely on symbolic information
  • Enables safe sharing of PassConfig between multiple transformations
  • Maintains all existing functionality while preventing state pollution
  • Exception-safe state restoration

Tickets:

  • 170030

@evkotov evkotov self-assigned this Jul 18, 2025
@evkotov evkotov added the category: transformations OpenVINO Runtime library - Transformations label Jul 18, 2025
@evkotov evkotov requested review from a team as code owners July 18, 2025 15:08
@github-actions github-actions bot added category: CPU OpenVINO CPU plugin and removed category: transformations OpenVINO Runtime library - Transformations labels Jul 18, 2025
@evkotov evkotov requested a review from CuriousPanCake July 21, 2025 14:24
@evkotov evkotov force-pushed the CVS-170030 branch 2 times, most recently from 396e855 to 848c30b Compare July 28, 2025 10:29
@mryzhov mryzhov requested a review from Copilot July 28, 2025 11:29
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Refactors the StatefulSDPAFusion transformation to use OpenVINO's new symbolic API instead of the deprecated gen_pattern utilities. This modernizes the pattern matching implementation while maintaining the same functionality.

  • Replaces gen_pattern-based pattern definitions with symbolic API equivalents
  • Updates the transformation manager to use SymbolicOptimizations instead of the basic Manager
  • Adds validation for concat axis consistency between K and V tensors

@evkotov evkotov requested a review from praasz July 30, 2025 10:57
@evkotov evkotov requested a review from CuriousPanCake August 4, 2025 09:22
@evkotov evkotov requested review from a team as code owners August 4, 2025 13:26
@CuriousPanCake CuriousPanCake added this pull request to the merge queue Aug 11, 2025
@CuriousPanCake CuriousPanCake removed this pull request from the merge queue due to a manual request Aug 11, 2025
…g State Management in SymbolicOptimizations

  Problem: SymbolicOptimizations modified shared PassConfig without restoring
  original state, breaking subsequent transformations like NgramFusion.

  Solution: Added proper state preservation with new restore_default() method
  in PassConfig and complete state tracking in SymbolicOptimizations.

  - Add PassConfig::restore_default() method to remove from both disabled/enabled sets
  - Track original pass state (Default/Disabled/Enabled) before modification
  - Restore exact original state after SymbolicOptimizations execution
  - Add exception safety with try/catch for guaranteed state restoration
@github-actions github-actions bot added category: Core OpenVINO Core (aka ngraph) category: CPP API OpenVINO CPP API bindings labels Aug 12, 2025
@evkotov evkotov requested a review from mryzhov August 12, 2025 12:14
@github-actions github-actions bot removed the category: CPP API OpenVINO CPP API bindings label Aug 12, 2025
@github-actions github-actions bot removed the category: Core OpenVINO Core (aka ngraph) label Aug 12, 2025
// So we decided to disable these passes in SymbolicOptimizations.
const auto& pass_config = m_manager->get_pass_config();

const auto old_pass_config = *pass_config;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about the make a copy of config at L:212:
const auto pass_config = m_manager->get_pass_config();

then disable some passes on this copy, in case of error the restoration of original pass config should not be required.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, there is no method Manager::set_pass_config. Should we add this method?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The set pass should not be require the idea was to refactor like:

    // make a copy of config
    const auto pass_config = m_manager->get_pass_config();
    
    // disable passes on copy only
    pass_config->disable<EliminateSqueeze>();
    pass_config->disable<EliminateUnsqueeze>();
    
    auto result = m_manager->run_passes(m);
    
    ov::remove_skip_invalidation_rti(m);
    return result;

but maybe there is something missed and it will not work correctly

github-merge-queue bot pushed a commit that referenced this pull request Aug 13, 2025
)

### Details:
- #31386
 - rewrite StatefulSDPAFusion transformation
 - Fix SymbolicOptimizations
SymbolicOptimizations permanently disables EliminateSqueeze and
EliminateUnsqueeze passes in the shared PassConfig, affecting other
pipeline components that may need these optimizations.
Save the original disabled state of the passes before temporarily
disabling them, then restore only those passes that weren't originally
disabled.

### Tickets:
 - 170030
@evkotov
Copy link
Contributor Author

evkotov commented Aug 14, 2025

merged #31709 instead of this

@evkotov evkotov closed this Aug 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants