[GPU][CPU] Use GatherMatmul as MOE building block

[EgorDuplensky]

Details:

• Align MOE processing and transformations between CPU and GPU plugins
• Use existing CPU op GatherMatmul as GPU MOE op building block
• Previous flow: TiledMoeBlock -> MOEOp -> MOECompressedOp -> MOECompressedOpWithRouting
• New flow: TiledMoeBlock -> MoeBlockViaGatherMatmuls(compressed) -> MOEOp(compressed) -> MOECompressedOpWithRouting
• The idea is that in case complex MOE / MOEFused patterns are not matched, the models should still perform well enough with GatherMatmuls as separate ops

TODO

• Add ascii diagram for every transformation
• Finalize GPU GatherMatmul implementation

[EgorDuplensky]

Converted to a draft for now to prevent CI executions

[p-durandin]

@EgorDuplensky please provide description of approach (either in PR or in ticket)

[EgorDuplensky]

@EgorDuplensky please provide description of approach (either in PR or in ticket)

The high-level description has been already provided. Could you please specify, what extra information is needed?

[riverlijunjie]

1. GatherMatmul is a key building block for expert computation in full MoE, but it does not represent the entire MoE workflow. Should we consider introducing a dedicated op/primitive to encapsulate end-to-end MoE computation in the GPU plugin (similar to PA/SDPA)? This may reduce scalability to some extent, but it could improve performance both on the host side and in kernel fusion opportunities.

2. In the GPU plugin, GatherMatmul currently uses two micro-GEMM kernel paths for prefill and decode phases. This design helps both compute-bound and memory-bound scenarios. We may further improve performance by fusing post-processing into micro-GEMM (similar to PR [GPU] enable post_proc for moe micro_gemm #33723) to avoid extra write-back/read-back traffic.

3. gather_sort currently uses a single thread for sorting, which is a clear bottleneck. Performance degrades significantly for long token sequences and large top-k values. This path should be optimized with parallel sorting.

4. oneDNN is introducing grouped_gemm as an alternative to micro-GEMM for MoE support. Should we consider adopting grouped_gemm in this path? If yes, what are the expected trade-offs and rationale (e.g., maintainability, portability, peak performance)? If no, why?

5. Do we have performance data for these paths? It would be helpful to quantify current efficiency and estimate the gap to the roofline.