awesome-world-models-for-robots

overview

• World Models
• arxiv 2024, 11, Understanding World or Predicting Future? A Comprehensive Survey of World Models Paper.

benchmark

• arXiv 2024, 03, HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation Paper Website. $15$ whole-body manipulation and $12$ locomotion tasks. This repo contains the code for environments and training.

dataset

• Pysical AI Website.
• AgiBot World Website. 1 million+ trajectories from 100 robots.
• LeRobotDataset Website. A bunch of models, datasets, and tools for real-world robotics in PyTorch.
• 1xgpt Website.
• OXE Paper.

models

• V-JEPA2 Website. Paper Code.
• Cosmos Website. Paper. Autoregressive Video2World/Text2World foundation models.

toolbox

• Menagerie Website MuJoCo physics engines. System identification toolbox has not been released.(up to 2025.1)
• MuJoCo Playground Website Paper Training environments in mjx. Humanoid Locomotion, Quadruped Locomotion and Manipulation (most robot arms and hand) tasks are included.

papers

• arxiv 2026, 02, World Action Models are Zero-shot Policies. Webiste. Paper.
• arxiv 2026, 01, PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation. Website. Paper.
• arxiv 2026, 01, Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning. Paper. Website.
• arxiv 2025, 12, What Drives Success in Physical Planning with Joint-Embedding Predictive World Models? Paper. Code.
• arxiv 2025, 12, World Models Can Leverage Human Videos for Dexterous Manipulation. Paper. Website.
• arxiv 2025, 12, Closing the Train-Test Gap in World Models for Gradient-Based Planning. Paper. Code.
• arxiv 2025, 10, Ego-Vision World Model for Humanoid Contact Planning. Website.
• unitree world model, UnifoLM-WMA-0: A World-Model-Action (WMA) Framework under UnifoLM Family. Website.
• arxiv 2025, 08, Genie Envisioner: A Unified World Foundation Model for Robotic Manipulation. Website. Paper.
• RSS 2025 Best Systems Paper finalist, Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation. Website. Paper.
• RSS 2025, Particle-Grid Neural Dynamics for Learning Deformable Object Models from RGB-D Videos. Website. Paper.
• 1x-world-model. Paper.
• arxiv 2025, 05, Evaluating Robot Policies in a World Model. Paper.
• arxiv 2025, 05, RLVR-World: Training World Models with Reinforcement Learning. Paper.
• arxiv 2025, 04, TesserAct: Learning 4D Embodied World Models. Website.
• arxiv 2025, 02, Strengthening Generative Robot Policies through Predictive World Modeling. Paper. Stengthen imitation learning with world model.
• RSS 2025, Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets. Website.
• RSS 2025, Unified Video Action Model. Website.
• ICRA 2025, World Model-based Perception for Visual Legged Locomotion. Website. Code.
• arxiv 2025, 03, Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning. Paper. Website.
• arxiv 2025, 01, RoboHorizon: An LLM-Assisted Multi-View World Model for Long-Horizon Robotic Manipulation Papaer.
• arxiv 2025, 01, Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics Paper. MBPO sim2real using world models. Quadruped locomotion tasks.
• ICML 2025, Trajectory World Models for Heterogeneous Environments. Paper.
• ICLR 2025 (Spotlight), DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes. Website.
• CVPR 2025 (Oral), Navigation World Models. Website. Paper.
• ICRA 2024, MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation. Paper.
• CoRL 2024, Multi-Task Interactive Robot Fleet Learning with Visual World Models. Paper. Visual world model for anomaly detection.
• Neulps 2024, iVideoGPT: Interactive VideoGPTs are Scalable World Models. Website. Code.
• RSS 2024, HRP: Human Affordances for Robotic Pre-Training. Paper.
• ICLR 2024 (Outstanding Paper), UniSim: Learning Interactive Real-World Simulators Website.
• ICLR 2024, Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation Website.
• arxiv 2024, 11, DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning. Website. World model for MPC. DINOv2 for representation.
• CVPR 2023, Affordances from Human Videos as a Versatile Representation for Robotics. Paper. Prediction contact points and trajectory waypoints, then use it for downstream tasks (suitable for different learning paradigms).
• RSS 2023, Structured World Models from Human Videos. Paper. Robot arm manipulation tasks. World Models with structured action space design.
• CoRL 2023 (Oral), Finetuning Offline World Models in the Real World Website Paper Offline pretraining and online finetuning of world models. Robot arm manipulation tasks.
• CoRL 2022, Daydreamer: World models for physical robot learning. Paper.

workshop

• Neurlps 2025, Embodied World Models for Decision Making Website.
• CoRL 2025, Robotics World Modeling Website.
• ICCV 2025, Reliable and Interactive World Model Website.
• RSS 2025, Structured World Models for Robotic Manipulation Website.
• ICML 2024, Multi-modal Foundation Model meets Embodied AI Website.
• ICLR 2025, Generative Models for Robot Learning. Website.
• ICLR 2025, World Models. Website.
• ICML 2025, Building Physically Plausible World Models. Website.

related: World Models

• Leo Fan's List. Website.
• ICML 2025 (Oral), Temporal Difference Flows. Paper.
• ICML 2025 (Spotlight), Novelty Detection in Reinforcement Learning with World Models. Paper.
• arxiv 2025, 03, Denoising Hamiltonian Network for Physical Reasoning. Paper.
• arxiv 2024, 05, Hierarchical World Models as Visual Whole-Body Humanoid Controllers. Website.
• ICML 2024, Offline Transition Modeling via Contrastive Energy Learning. Code.
• ICML 2024, 3D-VLA: A 3DVision-Language-Action Generative World Model. Paper.
• ICML 2024 (Oral), Genie: Generative Interactive Environments. Paper.
• ICML 2024 (Oral), Learning to Model the World with Language. Paper. Website.
• 2024, 12, Genie2 Blog.
• ICML 2025, PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Drop. Paper. Webstite.

related: LLM as WM

• ICLR 2025, Monte Carlo Planning with Large Language Model for Text-Based Games. Paper.
• arxiv 2024, AgentGym: Evolving Large Language Model-based Agents across Diverse Environments. Paper. Code.
• NIPS 2023, Language Models Meet World Models: Embodied Experiences Enhance Language Models. Paper. Openreview.
• NIPS 2023, Large Language Models as Commonsense Knowledge for Large-Scale Task Planning. Website. Paper.
• NIPS 2023, ChessGPT: Bridging Policy Learning and Language Modeling. Paper. Code.

related: Transfer Learning

• arxiv 2022, 01, Transferability in Deep Learning: A Survey. Paper.

related: Robotics & Foundation models

• 2025, 03, GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. Code.
• RSS 2024, OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics. Paper.
• CoRL 2023 (Oral), VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models. Website.
• ICLR 2024, Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models. Paper.
• ICRA 2025, WildLMA: Long Horizon Loco-MAnipulation in the Wild. Website.
• arxiv, 2024, 12, NaVILA: Legged Robot Vision-Language-Action Model for Navigation. Website.
• arxiv, 2024, 10, GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs. Paper.

related: Robotics & Vision-based RL

• CoRL 2022 (Oral), Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion. Paper.
• CoRL 2022 (Oral), Legged Locomotion in Challenging Terrains using Egocentric Vision. Paper.
• ICML 2023 (Oral), Efficient RL via Disentangled Environment and Agent Representations. Website.
• CoRL 2022, VideoDex: Learning Dexterity from Internet Videos. Website.
• CVPR 2022, Coupling Vision and Proprioception for Navigation of Legged Robots. Paper.
• CoRL 2024, Continuously Improving Mobile Manipulation with Autonomous Real-World RL. Paper. Mobile Manipulation.
• RSS 2023, Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trials. Paper.
• CoRL 2024, Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance. Website.

related: Robotics & Visual representations

• NeurIPS 2024, DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control. Paper.
• RSS 2024, HRP: Human Affordances for Robotic Pre-Training. Paper.
• ICML 2023 (Oral), Efficient RL via Disentangled Environment and Agent Representations. Website.
• CVPR 2023, Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture. Paper.
• ICML 2022, On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline. Paper.
• IROS 2023, Visual Reinforcement Learning with Self-Supervised 3D Representations. Paper.

related: Generative models for Decision-Making

• ICML 2025, History-Guided Video Diffusion. Website. Paper.
• arxiv 2025, 01, Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review. Paper.
• arxiv 2024, 05, Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models. Paper.
• NIPS 2024, Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion. Website.
• ICML 2022, Learning Iterative Reasoning through Energy Minimization. Paper.
• ICRA 2023, NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration. Paper.
• ICML 2024, Video as the New Language for Real-World Decision Making. Paper.

related: Generative simulation

• arxiv 2024, 06, RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots. Paper.

related: RL in the Real World

• arxiv 2021, 02, NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning. Paper. Website