-
Notifications
You must be signed in to change notification settings - Fork 2.7k
rewrite StatefulSDPAFusion transformation using new symbolic API #31386
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Show resolved
Hide resolved
396e855
to
848c30b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Refactors the StatefulSDPAFusion transformation to use OpenVINO's new symbolic API instead of the deprecated gen_pattern utilities. This modernizes the pattern matching implementation while maintaining the same functionality.
- Replaces gen_pattern-based pattern definitions with symbolic API equivalents
- Updates the transformation manager to use SymbolicOptimizations instead of the basic Manager
- Adds validation for concat axis consistency between K and V tensors
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Show resolved
Hide resolved
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp
Outdated
Show resolved
Hide resolved
….CompareWithRefs/targetDevice=CPU*
...ansformations/src/transformations/common_optimizations/fuse_rotary_positional_embeddings.cpp
Outdated
Show resolved
Hide resolved
…g State Management in SymbolicOptimizations Problem: SymbolicOptimizations modified shared PassConfig without restoring original state, breaking subsequent transformations like NgramFusion. Solution: Added proper state preservation with new restore_default() method in PassConfig and complete state tracking in SymbolicOptimizations. - Add PassConfig::restore_default() method to remove from both disabled/enabled sets - Track original pass state (Default/Disabled/Enabled) before modification - Restore exact original state after SymbolicOptimizations execution - Add exception safety with try/catch for guaranteed state restoration
// So we decided to disable these passes in SymbolicOptimizations. | ||
const auto& pass_config = m_manager->get_pass_config(); | ||
|
||
const auto old_pass_config = *pass_config; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about the make a copy of config at L:212:
const auto pass_config = m_manager->get_pass_config();
then disable some passes on this copy, in case of error the restoration of original pass config should not be required.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately, there is no method Manager::set_pass_config. Should we add this method?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The set pass should not be require the idea was to refactor like:
// make a copy of config
const auto pass_config = m_manager->get_pass_config();
// disable passes on copy only
pass_config->disable<EliminateSqueeze>();
pass_config->disable<EliminateUnsqueeze>();
auto result = m_manager->run_passes(m);
ov::remove_skip_invalidation_rti(m);
return result;
but maybe there is something missed and it will not work correctly
) ### Details: - #31386 - rewrite StatefulSDPAFusion transformation - Fix SymbolicOptimizations SymbolicOptimizations permanently disables EliminateSqueeze and EliminateUnsqueeze passes in the shared PassConfig, affecting other pipeline components that may need these optimizations. Save the original disabled state of the passes before temporarily disabling them, then restore only those passes that weren't originally disabled. ### Tickets: - 170030
merged #31709 instead of this |
Details:
Problem
SymbolicOptimizations::run_on_model() disabled EliminateSqueeze and EliminateUnsqueeze passes in shared PassConfig without properly restoring their original state. This broke subsequent transformations like NgramFusion that depend on these
passes for symbolic pattern matching.
Root Cause
The original restore logic was incomplete - PassConfig has three states (Default, Disabled, Enabled) but the code only handled two, incorrectly converting Default state to Enabled state after restoration.
Solution
Benefits
Tickets: