A comprehensive C# wrapper for OpenVINO and OpenVINO GenAI, providing idiomatic .NET APIs for AI inference and generative AI tasks.
**This is very much so in progress. Please do not use yet :). If this is something that might be helpful, leave an issue and let us know your usecase and we can try to incorporate as needed or else the main focus will be on Windows .NET apps **
- OpenVINO.NET.Core: Core OpenVINO functionality for model inference
- OpenVINO.NET.GenAI: Generative AI capabilities including LLM pipelines
- OpenVINO.NET.Native: Native library management and deployment
- Modern C# API: Async/await, IAsyncEnumerable, SafeHandle resource management
- Windows x64 Support: Optimized for Windows deployment scenarios
- .NET 8.0 or later
- Windows x64
- OpenVINO GenAI 2025.2.0.0 runtime
The easiest way to get started is with the QuickDemo application that automatically downloads a model:
By default the script downloads for ubuntu 24, if have another version, change it in the script
scripts/download-openvino-runtime.sh
OPENVINO_RUNTIME_PATH=/home/brandon/OpenVINO.GenAI.NET/build/native/runtimes/linux-x64/native dotnet run --project samples/QuickDemo/ --configuration Release -- --device CPUFor Windwos
.\scripts\download-openvino-runtime.ps1
$env:OPENVINO_RUNTIME_PATH = "C:\Users\brand\code\OpenVINO.GenAI.NET\build\native\runtimes\win-x64\native"
dotnet run --project samples/QuickDemo/ --configuration Release -- --device CPUSample Output:
OpenVINO.NET Quick Demo
=======================
Model: Qwen3-0.6B-fp16-ov
Temperature: 0.7, Max Tokens: 100
✓ Model found at: ./Models/Qwen3-0.6B-fp16-ov
Device: CPU
Prompt 1: "Explain quantum computing in simple terms:"
Response: "Quantum computing is a revolutionary technology that uses quantum mechanics principles..."
Performance: 12.4 tokens/sec, First token: 450ms
For integrating into your own applications:
using OpenVINO.NET.GenAI;
using var pipeline = new LLMPipeline("path/to/model", "CPU");
var config = GenerationConfig.Default.WithMaxTokens(100).WithTemperature(0.7f);
string result = await pipeline.GenerateAsync("Hello, world!", config);
Console.WriteLine(result);using OpenVINO.NET.GenAI;
using var pipeline = new LLMPipeline("path/to/model", "CPU");
var config = GenerationConfig.Default.WithMaxTokens(100);
await foreach (var token in pipeline.GenerateStreamAsync("Tell me a story", config))
{
Console.Write(token);
}OpenVINO.NET.Core- Core OpenVINO wrapperOpenVINO.NET.GenAI- GenAI functionalityOpenVINO.NET.Native- Native library managementQuickDemo- Quick start demo with automatic model downloadTextGeneration.Sample- Basic text generation exampleStreamingChat.Sample- Streaming chat application
┌─────────────────────────────────────────────────────────────┐
│ Your Application │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ OpenVINO.NET.GenAI │
│ • LLMPipeline (High-level API) │
│ • GenerationConfig (Fluent configuration) │
│ • ChatSession (Conversation management) │
│ • IAsyncEnumerable streaming │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ OpenVINO.NET.Core │
│ • Core OpenVINO functionality │
│ • Model loading and inference │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ OpenVINO.NET.Native │
│ • P/Invoke declarations │
│ • SafeHandle resource management │
│ • MSBuild targets for DLL deployment │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ OpenVINO GenAI C API │
│ • Native OpenVINO GenAI runtime │
│ • Version: 2025.2.0.0 │
└─────────────────────────────────────────────────────────────┘
- Memory Safe: SafeHandle pattern for automatic resource cleanup
- Async/Await: Full async support with cancellation tokens
- Streaming: Real-time token generation with
IAsyncEnumerable<string> - Fluent API: Chainable configuration methods
- Error Handling: Comprehensive exception handling and device fallbacks
- Performance: Optimized for both throughput and latency
-
Install .NET 9.0 SDK or later
- Download from: https://dotnet.microsoft.com/download
-
Install OpenVINO GenAI Runtime 2025.2.0.0
- Download from: https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/
- Extract to a directory in your PATH, or place DLLs in your application's output directory
# Clone the repository
git clone https://github.com/your-repo/OpenVINO-C-Sharp.git
cd OpenVINO-C-Sharp
# Build the solution
dotnet build OpenVINO.NET.sln
# Run the quick demo
dotnet run --project samples/QuickDemo| Device | Tokens/Second | First Token Latency | Notes |
|---|---|---|---|
| CPU | 12-15 | 400-600ms | Always available |
| GPU | 20-30 | 200-400ms | Requires compatible GPU |
| NPU | 15-25 | 300-500ms | Intel NPU required |
# Compare all available devices
dotnet run --project samples/QuickDemo -- --benchmarkError: The specified module could not be found. (Exception from HRESULT: 0x8007007E)
Solution: Ensure OpenVINO GenAI runtime DLLs are in your PATH or application directory.
Error: Failed to create LLM pipeline on GPU: Device GPU is not supported
Solutions:
- Check device availability:
dotnet run --project samples/QuickDemo -- --benchmark - Use CPU fallback:
dotnet run --project samples/QuickDemo -- --device CPU - Install appropriate drivers (Intel GPU driver for GPU support, Intel NPU driver for NPU)
Error: Failed to download model files from HuggingFace
Solutions:
- Check internet connectivity
- Verify HuggingFace is accessible
- Manually download model files to
./Models/Qwen3-0.6B-fp16-ov/
Error: Insufficient memory to load model
Solutions:
- Use a smaller model
- Reduce max_tokens parameter
- Close other memory-intensive applications
- Consider using INT4 quantized models
Enable detailed logging by setting environment variable:
# Windows
set OPENVINO_LOG_LEVEL=DEBUG
# Linux/macOS
export OPENVINO_LOG_LEVEL=DEBUG-
Install Prerequisites
- Visual Studio 2022 or VS Code with C# extension
- .NET 9.0 SDK
- OpenVINO GenAI runtime
-
Build and Test
dotnet build OpenVINO.NET.sln dotnet test tests/OpenVINO.NET.GenAI.Tests/ -
Code Style
- Follow Microsoft C# coding conventions
- Use async/await patterns
- Implement proper resource disposal (using statements)
- Add XML documentation for public APIs
- P/Invoke Layer: Add native method declarations in
GenAINativeMethods.cs - SafeHandle: Create appropriate handle classes for resource management
- High-level API: Implement user-friendly wrapper classes
- Tests: Add comprehensive unit tests
- Documentation: Update README and XML docs
This project is licensed under the MIT License - see the LICENSE file for details.
- OpenVINO GenAI Documentation
- OpenVINO GenAI C API Reference
- .NET P/Invoke Documentation
- HuggingFace Model Hub
dotnet build OpenVINO.NET.sln