Skip to content

Conversation

DefTruth
Copy link

@DefTruth DefTruth commented Sep 10, 2025

This PR fix pipe device error while using device=cpu in model loading and enable user controllable compile setting. After this PR, users can run HunyuanImage-2.1 on many device with low GPU VRAM (< 48GiB). For example:

model_name = "hunyuanimage-v2.1"
pipe = HunyuanImagePipeline.from_pretrained(
    model_name=model_name,
    torch_dtype="bf16",
    # NOTE: load in CPU first, this will enable HunyuanImage run
    # on many device with low GPU VRAM (<96 GiB):
    # CPU -> GPU VRAM < 96GiB ? -> FP8 weight only on CPU -> GPU
    device="cpu" if GiB() < 96 else "cuda",
    use_compile=False,
)

Please also check another PR for more details: #13

@KimbingNg @yestinl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant