[Bug] There is a bug when running bevfusion demo

[bigyaofei]

Prerequisite

• I have searched Issues and Discussions but cannot get the expected help.
• I have read the FAQ documentation but cannot get the expected help.
• The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

why the shape of the model and the ckpt are mismatch? the ckpt is bevfusion_lidar_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d-2628f933.pth

Reproduces the problem - code sample

python demo/pcd_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__LIDAR_TOP__1532402927647951.pcd.bin configs/bevfusion/bevfusion_lidar_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d.py ./checkpoints/bevfusion_lidar_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d-2628f933.pth --show

Reproduces the problem - command or script

python demo/pcd_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__LIDAR_TOP__1532402927647951.pcd.bin configs/bevfusion/bevfusion_lidar_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d.py ./checkpoints/bevfusion_lidar_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d-2628f933.pth --show

Reproduces the problem - error message

ize mismatch for pts_middle_encoder.encoder_layers.encoder_layer1.0.conv1.weight: copying a param with shape torch.Size([16, 3, 3, 3, 16]) from checkpoint, the shape in current model is torch.Size([3, 3, 3, 16, 16]).
size mismatch for pts_middle_encoder.encoder_layers.encoder_layer1.0.conv2.weight: copying a param with shape torch.Size([16, 3, 3, 3, 16]) from checkpoint, the shape in current model is torch.Size([3, 3, 3, 16, 16]).
size mismatch for pts_middle_encoder.encoder_layers.encoder_layer1.1.conv1.weight: copying a param with shape torch.Size([16, 3, 3, 3, 16]) from checkpoint, the shape in current model is torch.Size([3, 3, 3, 16, 16]).
size mismatch for pts_middle_encoder.encoder_layers.encoder_layer1.1.conv2.weight: copying a param with shape torch.Size([16, 3, 3, 3, 16]) from checkpoint, the shape in current model is torch.Size([3, 3, 3, 16, 16]).

Additional information

No response

[thisimyusername]

any solution？

[thisimyusername]

you could try this code to convert model：(Sorry I forgot origin Link of this code )
import torch

path = 'bevfusion_lidar_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d-2628f933.pth'
model = torch.load(path)

for key in model['state_dict'].keys():
if (key == 'pts_middle_encoder.conv_input.0.weight') or (key == 'pts_middle_encoder.conv_out.0.weight') or (key == 'pts_middle_encoder.encoder_layers.encoder_layer3.2.0.weight') or (key == 'pts_middle_encoder.encoder_layers.encoder_layer2.2.0.weight') or (key == 'pts_middle_encoder.encoder_layers.encoder_layer1.2.0.weight') or (('pts_middle_encoder.encoder_layers.encoder_layer' in key) and ('conv' in key)):
model['state_dict'][key] = torch.transpose(model['state_dict'][key],0,1)
model['state_dict'][key] = torch.transpose(model['state_dict'][key],1,2)
model['state_dict'][key] = torch.transpose(model['state_dict'][key],2,3)
model['state_dict'][key] = torch.transpose(model['state_dict'][key],3,4)

torch.save(model, "official_convert_epoch_20.pth")