Skip to content
This repository was archived by the owner on Nov 27, 2024. It is now read-only.

Commit 9c1aa87

Browse files
saddam213saddam213
committed
Update README
1 parent d7b17a2 commit 9c1aa87

File tree

3 files changed

+69
-188
lines changed

3 files changed

+69
-188
lines changed

‎OnnxStack.StableDiffusion/README.md

Lines changed: 69 additions & 87 deletions
Original file line numberDiff line numberDiff line change
@@ -1,110 +1,92 @@
1-
# OnnxStack.StableDiffusion - Onnx Stable Diffusion Services for .NET Applications
1+
# OnnxStack.StableDiffusion - Onnx Stable Diffusion Library for .NET
22

3-
`OnnxStack.StableDiffusion` is a library that provides higher-level Stable Diffusion services for use in .NET applications. It offers extensive support for features such as dependency injection, .NET configuration implementations, ASP.NET Core integration, and IHostedService support.
3+
`OnnxStack.StableDiffusion` is a library that provides access to Stable Diffusion processes in .NET.
4+
It offers extensive support for features such as TextToImage, ImageToImage, VideoToVideo, ControlNet and more
45

6+
## Getting Started
57

6-
### **Prompt**
7-
8-
Stable Diffusion models take a text prompt and create an image that represents the text.
9-
10-
*Example:*
11-
`
12-
High-fashion photography in an abandoned industrial warehouse, with dramatic lighting and edgy outfits, detailed clothing, intricate clothing, seductive pose, action pose, motion, beautiful digital artwork, atmospheric, warm sunlight, photography, neo noir, bokeh, beautiful dramatic lighting, shallow depth of field, photorealism, volumetric lighting, Ultra HD, raytracing, studio quality, octane render
13-
`
14-
15-
### **Negative Prompt**
16-
17-
A negative prompt can be provided to guide the inference to exclude in calculations
18-
19-
*Example:*
20-
21-
`
22-
painting, drawing, sketches, monochrome, grayscale, illustration, anime, cartoon, graphic, text, crayon, graphite, abstract, easynegative, low quality, normal quality, worst quality, lowres, close up, cropped, out of frame, jpeg artifacts, duplicate, morbid, mutilated, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, glitch, deformed, mutated, cross-eyed, ugly, dehydrated, bad anatomy, bad proportions, gross proportions, cloned face, disfigured, malformed limbs, missing arms, missing legs fused fingers, too many fingers,extra fingers, extra limbs,, extra arms, extra legs,disfigured,
23-
`
24-
25-
### **Schedulers**
26-
27-
Many different scheduler algorithms can be used for this computation, each having its pro- and cons.
28-
So far `OnnxStack.StableDiffusion` as included `LMS Discrete`, `Euler Ancestral`, `DDPM`, `DDIM`, and `KDPM2` options with more in the works.
29-
30-
*Example:*
31-
| LMS Scheduler | Euler Ancestral Scheduler | DDPM Scheduler |
32-
| :--- | :--- | :--- |
33-
<img src="../Assets/Samples/624461087_22_8_LMSScheduler.png" width="256" alt="Image of browser inferencing on sample images."/> | <img src="../Assets/Samples/624461087_22_8_EulerAncestralScheduler.png" width="256" alt="Image of browser inferencing on sample images."/> |<img src="../Assets/Samples/624461087_22_8_DDPMScheduler.png" width="256" alt="Image of browser inferencing on sample images."/> |
34-
35-
Seed: 624461087 GuidanceScale: 8 NumInferenceSteps: 22
36-
37-
### **Text To Image**
38-
Text To Image Stable Diffusion is a powerful machine learning technique that allows you to generate high-quality images from textual descriptions. It combines the capabilities of text understanding and image synthesis to convert natural language descriptions into visually coherent and meaningful images
39-
40-
| Input Text | Output Image | Diffusion Steps |
41-
| :--- | :--- | :--- |
42-
<img src="../Assets/Samples/Text2Img_Start.png" width="256" alt="Image of browser inferencing on sample images."/> | <img src="../Assets/Samples/Text2Img_End.png" width="256" alt="Image of browser inferencing on sample images."/> |<img src="../Assets/Samples/Text2Img_Animation.webp" width="256" alt="Image of browser inferencing on sample images."/> |
43-
44-
### **Image To Image**
45-
Image To Image Stable Diffusion is an advanced image processing and generation method that excels in transforming one image into another while preserving the visual quality and structure of the original content. Using stable diffusion, this technique can perform a wide range of image-to-image tasks, such as style transfer, super-resolution, colorization, and more
46-
47-
| Input Image | Output Image | Diffusion Steps |
48-
| :--- | :--- | :--- |
49-
<img src="../Assets/Samples/Img2Img_Start.bmp" width="256" alt="Image of browser inferencing on sample images."/> | <img src="../Assets/Samples/Img2Img_End.png" width="256" alt="Image of browser inferencing on sample images."/> |<img src="../../Assets/Samples/Img2Img_Animation.webp" width="256" alt="Image of browser inferencing on sample images."/> |
50-
8+
OnnxStack.StableDiffusion can be found via the nuget package manager, download and install it.
519
```
52-
Prompt: Dog wearing storm trooper helmet, head shot
10+
PM> Install-Package OnnxStack.StableDiffusion
5311
```
5412

55-
### **Image Inpainting**
56-
Image inpainting is an image modification/restoration technique that intelligently fills in missing or damaged portions of an image while maintaining visual consistency. It's used for tasks like photo restoration and object removal, creating seamless and convincing results.
57-
58-
In the below example we use a simple mask image + prompt to add a rider to the horse
59-
The black part of the mask will be used buy the process to generate new content, in this case the rider
60-
61-
| Input Image | Mask Image | Masked Image | Result
62-
| :--- | :--- | :--- | :--- |
63-
<img src="../Assets/Samples/Inpaint-Original.png" width="256" alt="Image of browser inferencing on sample images."/> | <img src="../Assets/Samples/Inpaint-Mask.png" width="256" alt="Image of browser inferencing on sample images."/> |<img src="../Assets/Samples/Inpaint-MaskedImage.PNG" width="256" alt="Image of browser inferencing on sample images."/> |<img src="../Assets/Samples/Inpaint-Result.png" width="256" alt="Image of browser inferencing on sample images."/> |
13+
## OnnxRuntime
14+
Depending on the devices you have and the platform you are running on, you will want to install the `Microsoft.ML.OnnxRuntime` package that best suits your needs.
6415

16+
**DirectML** - CPU-GPU support for Windows (Windows)
6517
```
66-
Prompt: Rider on horse
18+
PM> Install-Package Microsoft.ML.OnnxRuntime.DirectML
6719
```
6820

21+
**CUDA** - GPU support for NVIDIA (Windows, Linux)
22+
```
23+
PM> Install-Package Microsoft.ML.OnnxRuntime.Gpu
24+
```
6925

70-
## **Realtime Stable Diffusion**
71-
Realtime stable diffusion is a process where the results are constantly rendered as you are working with the image or changing the settings, This can be fantastic if you are creating new artworks or editing existing images.
72-
73-
Performance will depend on hardware and models selected, but for `Latent Consistency Models` you can get up to 4fps with a 3090 :)
74-
75-
### Text To Image
76-
https://user-images.githubusercontent.com/4353684/285347887-99db7f37-cff4-48b6-805b-3ca55e8f0c3a.mp4
26+
**CoreML** - CPU-GPU support for Mac (Apple)
27+
```
28+
PM> Install-Package Microsoft.ML.OnnxRuntime.CoreML
29+
```
7730

78-
### Image To Image
79-
https://user-images.githubusercontent.com/4353684/285348410-c19a2111-6745-4f01-8400-d137d40180fe.mp4
8031

81-
### Image Inpaint
82-
https://user-images.githubusercontent.com/4353684/285347894-9d044d7d-7c22-4379-8187-9cf7b9cac89c.mp4
32+
## Dependencies
33+
Video processing support requires `FFMpeg` and `FFProbe` binaries, files must be present in your output folder
34+
```
35+
https://ffbinaries.com/downloads
36+
https://github.com/ffbinaries/ffbinaries-prebuilt/releases/download/v6.1/ffmpeg-6.1-win-64.zip
37+
https://github.com/ffbinaries/ffbinaries-prebuilt/releases/download/v6.1/ffprobe-6.1-win-64.zip
38+
```
8339

84-
### Paint To Image
85-
https://user-images.githubusercontent.com/4353684/285347896-8da6709b-fea6-4cd4-ba65-ec692401f475.mp4
8640

87-
https://user-images.githubusercontent.com/4353684/285547207-3a7ea067-fcbf-47f0-9372-fafa94d301f7.mp4
41+
# C# Stable Diffusion
42+
Example Model: https://huggingface.co/runwayml/stable-diffusion-v1-5 (onnx branch)
8843

8944

45+
## Basic Stable Diffusion Example
46+
Run a simple Stable Diffusion process with a basic prompt
47+
```csharp
48+
// Create Pipeline
49+
var pipeline = StableDiffusionPipeline.CreatePipeline("D:\\Repositories\\stable-diffusion-v1-5");
9050

91-
## ONNX Model Download
92-
You will need an ONNX compatible model to use, Hugging Face is a great place to download the Stable Diffusion models
51+
// Set Prompt Options
52+
var promptOptions = new PromptOptions { Prompt = "Photo of a cute dog." };
9353

94-
- [OpenJourney V4](https://huggingface.co/TheyCallMeHex/OpenJourney-V4-ONNX)
95-
- [DreamLike PhotoReal 2.0](https://huggingface.co/TheyCallMeHex/DreamLike-PhotoReal-2.0-ONNX)
96-
- [CyberPunk Anime Diffusion](https://huggingface.co/TheyCallMeHex/Cyberpunk-Anime-Diffusion-ONNX)
97-
- [InkPunk Diffusion](https://huggingface.co/TheyCallMeHex/Inkpunk-Diffusion-ONNX)
98-
- [Mo-Di Diffusion](https://huggingface.co/TheyCallMeHex/Mo-Di-Diffusion-ONNX)
99-
- [epiCRealism](https://huggingface.co/TheyCallMeHex/epiCRealism-ONNX)
100-
- [Comic Diffusion](https://huggingface.co/TheyCallMeHex/Comic-Diffusion-ONNX)
101-
- [Redshift Diffusion](https://huggingface.co/TheyCallMeHex/Redshift-Diffusion-ONNX)
54+
// Run Pipleine
55+
var result = await pipeline.RunAsync(promptOptions);
10256

57+
// Save image result
58+
var image = result.ToImage();
59+
await image.SaveAsPngAsync("D:\\Results\\Image.png");
10360

104-
## Resources
105-
- [Hugging Face Stable Diffusion Blog](https://huggingface.co/blog/stable_diffusion)
106-
- [ONNX Runtime tutorial for Stable Diffusion in C#](https://onnxruntime.ai/docs/tutorials/csharp/stable-diffusion-csharp.html)
61+
// Unload Pipleine
62+
await pipeline.UnloadAsync();
63+
```
10764

65+
## Stable Diffusion Batch Example
66+
Run Stable Diffusion process and return a batch of results
67+
```csharp
68+
// Create Pipeline
69+
var pipeline = StableDiffusionPipeline.CreatePipeline("D:\\Repositories\\stable-diffusion-v1-5");
70+
71+
// Prompt
72+
var promptOptions = new PromptOptions{ Prompt = "Photo of a cat" };
73+
74+
// Batch Of 5 Images with unique seeds
75+
var batchOptions = new BatchOptions
76+
{
77+
ValueTo = 5,
78+
BatchType = BatchOptionType.Seed
79+
};
80+
81+
// Run Pipleine
82+
await foreach (var result in pipeline.RunBatchAsync(batchOptions, promptOptions))
83+
{
84+
// Save Image result
85+
var image = result.ImageResult.ToImage();
86+
await image.SaveAsPngAsync($"D:\\Results\\Image_{result.SchedulerOptions.Seed}.png");
87+
}
88+
89+
// Unload Pipleine
90+
await pipeline.UnloadAsync();
10891

109-
## Reference
110-
This work is based on the original C# implementation of Stable Diffusion by Cassie Breviu here: [Stable Diffusion with C# and ONNX Runtime](https://github.com/cassiebreviu/stablediffusion).
92+
```

‎OnnxStack.StableDiffusion/README_Example.md

Lines changed: 0 additions & 99 deletions
This file was deleted.

‎README.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,8 +65,6 @@ __________________________
6565

6666
More information can be found in the `OnnxStack.StableDiffusion` project **[README](OnnxStack.StableDiffusion/README.md)**
6767

68-
Code and development examples for `OnnxStack.StableDiffusion` can be founds here **[Dev Examples](OnnxStack.StableDiffusion/README_Example.md)**
69-
7068
__________________________
7169
__________________________
7270
## **[OnnxStack.ImageUpscaler]()**

0 commit comments

Comments
 (0)