Skip to content

Conversation

@10vliu13
Copy link

@10vliu13 10vliu13 commented Oct 8, 2025

Description

This PR intends to integrate ASR pipeline which include paraformer ASR.
WIP-1: Create ASRPipeline and paraformerASR constructor.

@github-actions github-actions bot added category: CPP API Changes in GenAI C++ public headers no-match-files labels Oct 8, 2025
Copy link
Collaborator

@rkazants rkazants left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please create/provide JIRA ticket to access if we need arch review for these changes.
In Jira ticket, we expect description of reason of these changes.

namespace genai {

// forward declaration
class ASRPipelineImplBase;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is about Whisper pipeline? Will it work with ASRPipeline? I expect -yes. Please add tests for this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, ASRPipeline should work with Whisper model. Once we add ASRPipeline I expect whisper pipeline to be deprecated.

Copy link
Author

@10vliu13 10vliu13 Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think Whisper pipeline will work with ASRPipeline. Let me also work on this to integrate with Whisper. @rkazants also mentioned tests for whisper. could you also show me the current tests for Whisper pipeline? I can refer to the current implementation. Thanks

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

namespace ov {
namespace genai {

ParaformerImpl::ParaformerImpl(const std::filesystem::path& model_dir,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do you get IRs for paraformer models? Do exporting and inference work in optimum-intel? If no, then we must add it for the full consistent user experience.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The peraformer IRs upstream to huggingface. Here is the link.
https://huggingface.co/apinge/paraformer-zh-int8-ov

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that is not how it should work. All supported models must pass optimum-cli tool conversion.
This is our deployment pipeline. If not optimum-intel support, model is not enabled and no claim in our documentation.
Create jira ticket for the corresponding task.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make it explicit. The ticket was created: CVS-174743

@rkazants rkazants self-assigned this Oct 8, 2025
@as-suvorov as-suvorov requested a review from Copilot October 8, 2025 08:09
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR creates the foundational structure for an ASR (Automatic Speech Recognition) pipeline with a focus on integrating Paraformer ASR functionality.

  • Introduces ASRPipeline class with a base implementation interface
  • Adds ParaformerImpl as a concrete implementation of the ASR pipeline
  • Provides a factory method for creating Paraformer ASR instances

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/cpp/include/openvino/genai/speech_recognition/asr_pipeline.hpp Public API header defining ASRPipeline class and paraformerASR factory method
src/cpp/src/speech_recognition/asr_pipeline.cpp Implementation of ASRPipeline constructors and paraformerASR factory method
src/cpp/src/speech_recognition/asr_pipelinee_impl_base.hpp Base class interface for ASR pipeline implementations
src/cpp/src/speech_recognition/paraformer_impl.hpp Header for ParaformerImpl class extending the base interface
src/cpp/src/speech_recognition/paraformer_impl.cpp Implementation of ParaformerImpl constructor and destructor

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.


#pragma once

#include "asr_pipelinee_impl_base.hpp"
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected spelling of 'pipelinee' to 'pipeline' in include filename.

Suggested change
#include "asr_pipelinee_impl_base.hpp"
#include "asr_pipeline_impl_base.hpp"

Copilot uses AI. Check for mistakes.
Comment on lines +16 to +18
ASRPipelineImplBase(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties = {}) {};
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Constructor should be explicit and the empty body should be on a separate line for better readability.

Suggested change
ASRPipelineImplBase(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties = {}) {};
explicit ASRPipelineImplBase(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties = {})
{}

Copilot uses AI. Check for mistakes.
Comment on lines +15 to +16
const ov::AnyMap& properties) {

Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constructor is empty but should initialize m_impl member variable or provide implementation details.

Suggested change
const ov::AnyMap& properties) {
const ov::AnyMap& properties)
: m_impl(std::make_shared<ParaformerImpl>(model_dir, device, properties)) {
assert(m_impl != nullptr);

Copilot uses AI. Check for mistakes.
@as-suvorov as-suvorov marked this pull request as draft October 8, 2025 08:35
@10vliu13
Copy link
Author

10vliu13 commented Oct 9, 2025

Please create/provide JIRA ticket to access if we need arch review for these changes. In Jira ticket, we expect description of reason of these changes.

Do you have any existing JIRA link i can refers to? I can create new one based on this. Thanks

@rkazants
Copy link
Collaborator

rkazants commented Oct 9, 2025

Please create/provide JIRA ticket to access if we need arch review for these changes. In Jira ticket, we expect description of reason of these changes.

Do you have any existing JIRA link i can refers to? I can create new one based on this. Thanks

Please consult with @EfimovIlia, @moslex about this JIRA tickets creation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants