Inference steps on a custom dataset

[haseeb33]

Thank you for making this excellent work publicly available.

Without going into too much detail, could you please provide a simple, high-level set of steps to run inference on a custom dataset using one of the provided pre-trained models?
Specifically, I am interested in:

• Using a custom dataset with 6 camera images
• Running inference with an existing pre-trained model
• Visualizing the output as a 3D occupancy map similar to the results shown in the paper

Any guidance or pointers to the relevant scripts or documentation would be greatly appreciated.

[cdb342]

Hi, thanks for your interest in ALOcc! Here is a simple instruction to get you started to run inference on your custom dataset:

1. Data Alignment

The most critical step is formatting your custom data to match the expected input. ALOcc uses infos.pkl files to store camera intrinsics/extrinsics, image paths, etc.

• I recommend looking at the data preparation script: tools/create_data_bevdet.py.
• You essentially need to create a similar script to wrap your images and calibration matrices into the MMDetection3D format.

2. Configuration Adjustment

You can take one of the provided configs (e.g., configs/alocc/alocc_3d_256x704_bevdet_preatrain.py) and update the data_root and ann_file paths to point to your custom dataset.

3. Inference & Visualization

Use the provided test script with the --save flag to dump the predictions:

bash tools/dist_test.sh [CONFIG_FILE] [CHECKPOINT_PATH] --save

Then, use tools/visual.py to render the 3D grids via Mayavi.

It is worth noting that models trained solely on nuScenes may not generalize seamlessly to custom datasets out-of-the-box. The Lifting (2D-to-3D) process is inherently sensitive to camera intrinsics and extrinsics, which remain relatively fixed during nuScenes training. If your custom sensor setup differs significantly, the model may struggle to adapt to the new spatial mapping, leading to suboptimal occupancy predictions. For reliable performance on out-of-distribution data, fine-tuning the model on your specific setup is usually recommended.

[haseeb33]

Thank you very much for your response.

I think it's necessary to prepare dataset according to tools/create_data_bevdet.py but for testing when you don't have ground truth, how can your convert your data?

def get_gt(info):
    """Generate gt labels from info.

    Args:
        info(dict): Infos needed to generate gt labels.

    Returns:
        Tensor: GT bboxes.
        Tensor: GT labels.
    """

Let's say I have custom data consists of RGB images (6 camera angles) and camera intrinsic parameter files. then what would be the essential steps to see some output of ALOcc?

I think visualization might be difficult, maybe because of not having ground truth values but something to check the results would be nice. (I might be wrong).

During testing, what's the input of model? -> Can you please pointout the exact code line.
isn't it just RGB images and camera parameters?
are 6 cameras at the same time necessary or can we run with fewer images as well?