Skip to content

WIP-1: Create asr pipeline and paraformerASR constructor. #2804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

WIP-1: Create asr pipeline and paraformerASR constructor. #2804

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions src/cpp/include/openvino/genai/speech_recognition/asr_pipeline.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
// Copyright (C) 2023-2025 Intel Corporation
// SPDX-License-Identifier: Apache-2.0

#pragma once

#include "utils.hpp"

namespace ov {
namespace genai {

// forward declaration
class ASRPipelineImplBase;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is about Whisper pipeline? Will it work with ASRPipeline? I expect -yes. Please add tests for this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, ASRPipeline should work with Whisper model. Once we add ASRPipeline I expect whisper pipeline to be deprecated.

Copy link
Author

@10vliu13 10vliu13 Oct 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think Whisper pipeline will work with ASRPipeline. Let me also work on this to integrate with Whisper. @rkazants also mentioned tests for whisper. could you also show me the current tests for Whisper pipeline? I can refer to the current implementation. Thanks

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whisper tests are here:
https://github.com/openvinotoolkit/openvino.genai/blob/master/tests/python_tests/test_whisper_pipeline.py
https://github.com/openvinotoolkit/openvino.genai/blob/master/tests/python_tests/test_whisper_pipeline_static.py


class OPENVINO_GENAI_EXPORTS ASRPipeline {
public:
explicit ASRPipeline(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties = {});

static ASRPipeline paraformerASR(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties = {});

private:
std::shared_ptr<ASRPipelineImplBase> m_impl;

explicit ASRPipeline(const std::shared_ptr<ASRPipelineImplBase>& impl);
};

} // namespace genai
} // namespace ov
32 changes: 32 additions & 0 deletions src/cpp/src/speech_recognition/asr_pipeline.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// Copyright (C) 2023-2025 Intel Corporation
// SPDX-License-Identifier: Apache-2.0

#include <cassert>

#include "openvino/genai/speech_recognition/asr_pipeline.hpp"

#include "speech_recognition/paraformer_impl.hpp"

namespace ov {
namespace genai {

ASRPipeline::ASRPipeline(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties) {

Comment on lines +15 to +16
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constructor is empty but should initialize m_impl member variable or provide implementation details.

Suggested change
const ov::AnyMap& properties) {
const ov::AnyMap& properties)
: m_impl(std::make_shared<ParaformerImpl>(model_dir, device, properties)) {
assert(m_impl != nullptr);

Copilot uses AI. Check for mistakes.
}

ASRPipeline::ASRPipeline(const std::shared_ptr<ASRPipelineImplBase>& impl)
: m_impl(impl) {
assert(m_impl != nullptr);
}

ASRPipeline ASRPipeline::paraformerASR(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties) {
auto impl = std::make_shared<ParaformerImpl>(model_dir, device, properties);
return ASRPipeline(impl);
}

} // namespace genai
} // namespace ov
23 changes: 23 additions & 0 deletions src/cpp/src/speech_recognition/asr_pipelinee_impl_base.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Copyright (C) 2023-2025 Intel Corporation
// SPDX-License-Identifier: Apache-2.0

#pragma once

#include <filesystem>
#include "openvino/openvino.hpp"
#include "openvino/runtime/intel_gpu/properties.hpp"
#include "openvino/runtime/intel_npu/properties.hpp"

namespace ov {
namespace genai {

class ASRPipelineImplBase {
public:
ASRPipelineImplBase(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties = {}) {};
Comment on lines +16 to +18
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Constructor should be explicit and the empty body should be on a separate line for better readability.

Suggested change
ASRPipelineImplBase(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties = {}) {};
explicit ASRPipelineImplBase(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties = {})
{}

Copilot uses AI. Check for mistakes.
virtual ~ASRPipelineImplBase() = default;
};

} // namespace genai
} // namespace ov
21 changes: 21 additions & 0 deletions src/cpp/src/speech_recognition/paraformer_impl.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
// Copyright (C) 2023-2025 Intel Corporation
// SPDX-License-Identifier: Apache-2.0

#include "paraformer_impl.hpp"

namespace ov {
namespace genai {

ParaformerImpl::ParaformerImpl(const std::filesystem::path& model_dir,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do you get IRs for paraformer models? Do exporting and inference work in optimum-intel? If no, then we must add it for the full consistent user experience.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The peraformer IRs upstream to huggingface. Here is the link.
https://huggingface.co/apinge/paraformer-zh-int8-ov

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that is not how it should work. All supported models must pass optimum-cli tool conversion.
This is our deployment pipeline. If not optimum-intel support, model is not enabled and no claim in our documentation.
Create jira ticket for the corresponding task.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make it explicit. The ticket was created: CVS-174743

const std::string& device,
const ov::AnyMap& properties)
: ASRPipelineImplBase(model_dir, device, properties)
{

}

ParaformerImpl::~ParaformerImpl()
{}

} // namespace genai
} // namespace ov
20 changes: 20 additions & 0 deletions src/cpp/src/speech_recognition/paraformer_impl.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
// Copyright (C) 2023-2025 Intel Corporation
// SPDX-License-Identifier: Apache-2.0

#pragma once

#include "asr_pipelinee_impl_base.hpp"
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected spelling of 'pipelinee' to 'pipeline' in include filename.

Suggested change
#include "asr_pipelinee_impl_base.hpp"
#include "asr_pipeline_impl_base.hpp"

Copilot uses AI. Check for mistakes.

namespace ov {
namespace genai {

class ParaformerImpl : public ASRPipelineImplBase {
public:
explicit ParaformerImpl(const std::filesystem::path& model_dir,
const std::string& device,
const ov::AnyMap& properties = {});
virtual ~ParaformerImpl() override;
};

} // namespace genai
} // namespace ov
Loading