Skip to content

feat: support configurable ONNX Runtime providers#181

Draft
tuannx wants to merge 3 commits into
pnnbao97:mainfrom
tuannx:agent/configurable-onnx-providers
Draft

feat: support configurable ONNX Runtime providers#181
tuannx wants to merge 3 commits into
pnnbao97:mainfrom
tuannx:agent/configurable-onnx-providers

Conversation

@tuannx

@tuannx tuannx commented Jul 20, 2026

Copy link
Copy Markdown

Summary

  • select ONNX Runtime execution providers from the providers actually installed on the host
  • support CUDA, TensorRT, ROCm/MIGraphX, OpenVINO, CoreML, DirectML, and CPU device preferences
  • expose an ordered onnx_providers override with early validation and clear errors
  • apply the resolved provider chain consistently to legacy Turbo encoder/decoder sessions and all v3 Turbo ONNX sessions, including codec, denoiser, and speaker encoder paths
  • retain CPUExecutionProvider as the automatic fallback when it is installed

Fixes #167.

Root cause

The Turbo paths constructed fixed CUDA/CPU or CPU-only provider lists. As a result, installing another ONNX Runtime distribution did not make its execution provider usable, and callers could not explicitly control provider order.

Usage

from vieneu import Vieneu

tts = Vieneu(
    mode="v3turbo",
    backend="onnx",
    device="coreml",
    onnx_providers=["CoreMLExecutionProvider", "CPUExecutionProvider"],
)

If an explicitly requested provider is not reported by onnxruntime.get_available_providers(), construction now fails early with the requested and available provider names.

Validation

  • pytest -q tests/test_base_utils.py tests/test_engine_turbo.py tests/test_engine_v3turbo.py — 15 passed
  • focused Ruff check passed (excluding pre-existing findings in touched legacy modules)
  • Arcade before/after: 5 components and 3 smells unchanged; the reusable provider policy adds one entity and four static edges, with A2A similarity 0.9757

@tuannx

tuannx commented Jul 20, 2026

Copy link
Copy Markdown
Author

Arcade Agent architecture review

I ran lemduc/arcade-agent v0.1.1 against src/ before and after this provider change.

Overall result

  • source files: 32 → 33
  • recovered components: 5 → 5
  • entities: 263 → 264
  • dependency edges: 81 → 85
  • architecture smells: 3 → 3
  • A2A similarity: 0.9757
  • RCI: 0.8519 → 0.7882
  • TurboMQ: 0.5010 → 0.4810
  • Balanced score: 0.6414 → 0.6238

The four additional static edges come from routing the legacy Turbo and v3 Turbo ONNX paths through one shared provider-policy module. No component, cycle, or concern-overload smell was added. I am reporting the score movement explicitly rather than treating it as a release gate; the dependency direction is intentional so provider detection and validation are implemented once.

The repository-level report currently flags the existing low-severity Vieneu/VieneuUtils cycle and concern-overload signals in Vieneu and V3TurboEngine. The focused tests for this PR pass across the policy, legacy Turbo integration, and v3 Turbo integration (15 tests).

Optional follow-up proposal

If this analysis is useful to maintainers, I can open a separate PR adding the pinned Arcade GitHub Action (lemduc/arcade-agent/actions/analyze@v0.1.1, package 0.1.1) with source-path: src and language: python. It would store the baseline as an artifact and update a single PR summary comment. I kept the workflow separate from this feature diff to avoid coupling adoption of the tool to the provider fix.

@sonarqubecloud

Copy link
Copy Markdown

@pnnbao97

Copy link
Copy Markdown
Owner

Nice, clean separation — select_onnx_providers() as a pure, testable helper is the right call. I confirmed the providers are applied to every InferenceSession in the turbo / v3turbo paths (including the lazily-loaded codec encoder, denoiser, and speaker encoder), and I ran the added tests locally: 15 passed ✅.

Since it's still a draft, these are just notes for when you un-draft — nothing blocking:

  1. API parity between enginesturbo exposes onnx_providers as an explicit typed parameter, but v3turbo pulls it via kwargs.pop("onnx_providers", None). The kwargs form is undiscoverable from the signature/IDE and is silently ignored on the PyTorch/GPU backend. Suggest making it an explicit parameter on V3TurboVieNeuTTS.__init__ to match turbo.
  2. Behavior change on mps — turbo now forwards the raw device string, so mps can map to CoreMLExecutionProvider when that provider is installed (previously it always fell back to CPU). Worth a line in the PR description, since a CoreML-unsupported op would now hard-fail instead of silently running on CPU.
  3. Test gap — the happy paths and the explicit-unavailable-provider rejection are well covered, but there's no test for the auto/default device-based selection path (requested=None) through either engine, nor for the available[0] warning fallback.
  4. Nitutils.py adds a stray blank line; unrelated cosmetic churn, can drop it.
  5. Optional — consider documenting that an explicit provider list intentionally gets no CPU fallback (fail-early), so users know a bad list errors out rather than degrading to CPU.

Also, heads-up: NeuCodecOnnx (utils.py) still hardcodes ["CPUExecutionProvider"]. Out of scope here since turbo doesn't hit that path, but "configurable providers" won't be fully uniform until it's covered too.

Thanks! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feature request] sử dụng các execution providers khác trong onnxruntime

2 participants