[ERROR] Error during image-to-video generation: Input type (float) and bias type (c10::BFloat16) should be the same

[azterreza]

i have thi error when i load a picture to make a video

[ERROR] Error during image-to-video generation: Input type (float) and bias type (c10::BFloat16) should be the same

im in a mac m1 max

can someone help please ?

[ahmetselman]

same in here apple m3 pro

[mattanapol]

Same here on Apple M4 pro, this is traceback

Traceback (most recent call last):
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/app.py", line 261, in generate_image_to_video
    frames = model.generate_i2v(
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 912, in generate_i2v
    input_image_latent = (self.vae.encode(input_image_tensor.to(self.vae.device, dtype=self.vae.dtype)).latent_dist.sample() - self.vae_shift_factor) * self.vae_scale_factor  # [b c 1 h w]
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/video_vae/modeling_causal_vae.py", line 294, in encode
    return self.tiled_encode(x, return_dict=return_dict, is_init_image=is_init_image,
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/video_vae/modeling_causal_vae.py", line 444, in tiled_encode
    tile = self.encoder(tile, is_init_image=True, temporal_chunk=False)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/video_vae/modeling_enc_dec.py", line 188, in forward
    sample = down_block(sample, is_init_image=is_init_image, temporal_chunk=temporal_chunk)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/video_vae/modeling_block.py", line 530, in forward
    hidden_states = resnet(hidden_states, temb=None, is_init_image=is_init_image, temporal_chunk=temporal_chunk)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/video_vae/modeling_resnet.py", line 132, in forward
    hidden_states = self.conv1(hidden_states, is_init_image=is_init_image, temporal_chunk=temporal_chunk)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/video_vae/modeling_causal_conv.py", line 137, in forward
    x = self.conv(x)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 725, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/Users/kaewsai/pinokio/api/pyramidflow.git/app/env/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 720, in _conv_forward
    return F.conv3d(
RuntimeError: Input type (float) and bias type (c10::BFloat16) should be the same

Found similar issue here, hope it fix in the next update

[rockytriton]

I had the same issue on mac M4 MAX processor.

Open the app.py

Update the line:
model_dtype = "bf16"

to:
model_dtype = "fp32"

and it worked for me.

[djtrance]

I had the same issue on mac M4 MAX processor.

Open the app.py

Update the line: model_dtype = "bf16"

to: model_dtype = "fp32"

and it worked for me.

this makes my Mac to crash, but if I put this line:
model_dtype = "bf16" , torch.bfloat16
everything works