Skip to content

Add low_cpu_mem_usage option to from_single_file to align with from_pretrained #12114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

IrisRainbowNeko
Copy link
Contributor

@IrisRainbowNeko IrisRainbowNeko commented Aug 10, 2025

What does this PR do?

from_single_file can lead to errors in training model:

File "/home/dongziyi/miniconda3/envs/pt25/lib/python3.11/site-packages/diffusers/loaders/single_file_model.py", line 447, in from_single_file                                       
    dispatch_model(model, **device_map_kwargs)                                                                                                                                        
  File "/home/dongziyi/miniconda3/envs/pt25/lib/python3.11/site-packages/accelerate/big_modeling.py", line 502, in dispatch_model                                                     
    model.to(device)                                                                                                                                                                  
  File "/home/dongziyi/miniconda3/envs/pt25/lib/python3.11/site-packages/diffusers/models/modeling_utils.py", line 1383, in to                                                        
    return super().to(*args, **kwargs)                                                                                                                                                
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                
  File "/home/dongziyi/miniconda3/envs/pt25/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1343, in to                                                                
    return self._apply(convert)                                                                                                                                                       
           ^^^^^^^^^^^^^^^^^^^^                                                                                                                                                       
  File "/home/dongziyi/miniconda3/envs/pt25/lib/python3.11/site-packages/torch/nn/modules/module.py", line 903, in _apply                                                             
    module._apply(fn)                                                                                                                                                                 
  File "/home/dongziyi/miniconda3/envs/pt25/lib/python3.11/site-packages/torch/nn/modules/module.py", line 903, in _apply                                                             
    module._apply(fn)                                                                                                                                                                 
  File "/home/dongziyi/miniconda3/envs/pt25/lib/python3.11/site-packages/torch/nn/modules/module.py", line 903, in _apply                                                             
    module._apply(fn)                                                                                                                                                                 
  [Previous line repeated 5 more times]                                                                                                                                               
  File "/home/dongziyi/miniconda3/envs/pt25/lib/python3.11/site-packages/torch/nn/modules/module.py", line 930, in _apply                                                             
    param_applied = fn(param)                                                                                                                                                         
                    ^^^^^^^^^                                                                                                                                                         
  File "/home/dongziyi/miniconda3/envs/pt25/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1336, in convert                                                           
    raise NotImplementedError(                                                                                                                                                        
NotImplementedError: Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device.                                                                   

Add low_cpu_mem_usage option for from_single_file to align with from_pretrained. Allow users to set not to create the model on the meta device.

Example:

UNet2DConditionModel.from_single_file(ckpt_path, torch_dtype=dtype, low_cpu_mem_usage=False)

Fixes #8320

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@IrisRainbowNeko IrisRainbowNeko changed the title Add low_cpu_mem_usage option for from_single_file to align with from_pretrained Add low_cpu_mem_usage option to from_single_file to align with from_pretrained Aug 10, 2025
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@DN6
Copy link
Collaborator

DN6 commented Aug 11, 2025

@bot /style

Copy link
Contributor

github-actions bot commented Aug 11, 2025

Style bot fixed some files and pushed the changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[0.27.2]: from_single_file: 'AutoencoderKL' object has no attribute '__name__'. Did you mean: '__ne__'?
3 participants