How much RAM do i need to use Chroma with stable-diffusion.cpp?

[JohnClaw]

...

[stduhpf]

You need enough system RAM to fit the T5xxl model (so that's ~9GB for full fp16, only 1.2GB for q2_k) + you need some RAM for T5 compute buffer ( ~250MB).

You also need to be able to fit the diffusion model file in memory (~5.1GB for Q4_0 for example) + compute buffer (that depends on resolution, around 542 MB for 512x512 images). If you're not running on CPU only, this is part on VRAM and not system RAM.

For the VAE, you need about 100 MB for the model + 416 MB if you're using tiling, it depends on resolution if you're not using tiling.

[JohnClaw]

Thanks for reply. It's seems that it's too much for my laptop with 16gb ram and igpu Vega 7 (vulkan compatible) which can allocate 8gb vram from ram, right? Or i have a small chance to run it on cpu if i will use worst q2_k quants both for diffusion model and t5xxl model?

[stduhpf]

If you use very quantized models it might barely be able to run at low resolution I think? Best way to know for sure is to try.

[Green-Sky]

If --diffusion-fa works, q4_k with 1024x1024 should be possible.

[advanced-lvl-up]

@Green-Sky --- I had a question related to this. i guess it's not talked about much, but is the T5 encoder requirement really needed? I've always used the plain clip_l.safetensors alone for prompt testing Flux Schnell since the early days. i guess some UI implementations just zero out the input / output. It works totally fine without the T5. You just use the older style SD1.5 prompting and lose a bit of prompt accuracy / complexity.

as far as i know, Chroma is basically just Schnell with some minor differences.

[stduhpf]

#645

[stduhpf]

Though Chroma uses T5 as it's only text encoder, so skipping T5 for Chroma will completely disable text conditioning altogether.

[advanced-lvl-up]

i see. that's quite interesting. well, i guess you could save some default prompt testing embeddings,"Prompt: cat holds a sign reading Your T5 failed to load" , just warn the user that the prompt is a preconfigured test embedding, haha. ok, thats too much.

[Green-Sky]

The quality of the text embedder can significantly change the image. I saw someone recommend use the f32 t5xxl version instead with their model.