Skip to content

Conversation

cyyever
Copy link
Contributor

@cyyever cyyever commented Aug 18, 2025

Fix the failure of QuantState.as_dict then followed by QuantState.from_dict.
A reproducing example is

import torch

from bitsandbytes.functional import (
    QuantState,
    dequantize_4bit,
    quantize_4bit,
)


a = torch.rand(2, 3)
quantized, quant_state = quantize_4bit(A=a, quant_type="nf4")
quant_dict = quant_state.as_dict()
quant_state = QuantState.from_dict(quant_dict, device=torch.device("cuda"))
quantized = dequantize_4bit(A=quantized, quant_state=quant_state)

Before this PR, it failed with (on bitsandbytes 0.47.0):

  File "/home/cyy/a.py", line 13, in <module>
    quant_state = QuantState.from_dict(quant_dict, device=torch.device("cuda"))
  File "/home/cyy/.local/lib/python3.13/site-packages/bitsandbytes/functional.py", line 469, in from_dict
    raise ValueError(
        f"There should be exactly one `quant_state` item with ending from {cls.valid_qs_type_keys}.\nDetected {qs_key}.",
    )
ValueError: There should be exactly one `quant_state` item with ending from ['bitsandbytes__fp4', 'bitsandbytes__nf4'].
Detected [].

@matthewdouglas
Copy link
Member

Hi,
Can you provide more detail on the issue that this fixes? A simple reproducer with example output would be ideal. Thanks!

@cyyever
Copy link
Contributor Author

cyyever commented Sep 4, 2025

@matthewdouglas Added

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants