Release Highlights
-
New Research Projects: We have added comprehensive implementation scripts for notable research projects, including AuxThink and FineGrasp.
-
Streamlined Inference & Model Sharing: A new set of APIs, including TorchModelMixin and InferencePipelineMixin, dramatically simplifies the process of saving, loading, and sharing models. This system is fully compatible with the Hugging Face Hub, enabling seamless import and export of trained models and entire inference pipelines.
-
Unified Data Format: Introducing the RoboOrchard Dataset, a sophisticated data structure designed to bridge the entire robotics workflow. It is natively compatible with both ROS bags for data collection and Hugging Face Datasets for training. Its design supports multi-frequency signals, efficient metadata querying, and direct visualization, solving many common data management challenges in robotics.
New Features
-
Project Implementations:
-
Inference and Model APIs:
- TorchModelMixin: An API for easily saving and loading model weights and configurations, compatible with the Hugging Face Hub.
- InferencePipelineMixin: An integrated API that packages a model and its data processor into a single, portable inference pipeline, similar to the
transformers.pipelineand compatible with the Hugging Face Hub.
-
RoboOrchard Dataset
- Compatibility: Natively designed to be compatible with both ROS (mcap) message definitions and the arrow serialization format of Hugging Face Datasets, enabling a smooth transition from rosbag -> dataset.
- High-Fidelity Data: Each row in the dataset table represents a "message chunk," preserving multi-frequency sensor data without downsampling.
- Efficient Metadata Querying: Non-timeseries metadata (e.g., episode, task, robot info) is managed via an embedded database (like DuckDB/SQLite) for fast and convenient querying.
- Easy Visualization: Supports direct export of episodes to mcap format for visualization in tools like Foxglove.
-
Simulation Environments: Added apis for the Maniskill and RoboTwin simulation environments.
Improvements
- SEM: Updated to fully support the RoboTwin 2.0 environment.