agibot converter with camera parameters

[haofuly]

Feature request

The original agilexworld-beta dataset has provided the camera parameters, like extrinsics and intrinsics. Could the agibot2lerobot scripts add these camera parameter keys during convertion?

Motivation

I want to add the camera paremeters into the agibot lerobot datasets during convertion.

Your contribution

None.

[Tavish9]

okay, I will add the camera params in the next update

[Tavish9]

I find that there many camera parameters for each task. So, it's not recommended to save them during conversion, since it'll slow down the loading speed when reading metadata.

[haofuly]

Thanks. If I must use a certain camera parameters, which strategy do you recommend:

1. saving it to the lerobotdataset during conversion;
2. not saving it during conversion, but additionally reading the corresponding json file of these external parameters when using the lerobotdataset.

[Tavish9]

After reconsidering, I think way 1 is more suitable.

For 1, you only need to save them in meta/episodes/chunk-xxx/file-xxx.parquet, including each camera's intrinsic and extrinsic parameters, roughly 9*2 columns.
Fo 2, what you need to address is properly handling the correspondence of episode index before and after the conversion, since the conversion process may involve operations like skipping, and the converted dataset does not record this mapping.