LIFTER: Legged Incline Fall-recovery Technique for Enhanced Recovery

This repository contains the code from my Master's degree research on deep reinforcement learning for quadruped robot fall-recovery control.

📄 Thesis: Deep Reinforcement Learning for Fall-Recovery Control on Non-Flat Terrain of Quadruped Robots

📊 Presentation: Canva Slides

This environment builds on the legged gym environment by Nikita Rudin, Robotic Systems Lab, ETH Zurich (Paper: https://arxiv.org/abs/2109.11978) and the Isaac Gym simulator from NVIDIA (Paper: https://arxiv.org/abs/2108.10470). Training code builds on the rsl_rl repository, also by Nikita Rudin, Robotic Systems Lab, ETH Zurich. All redistributed code retains its original license.

Robot Platforms

This project uses the AiDIN series quadruped walking robots developed for multi-legged locomotion and fall recovery research.

Features

• Train reinforcement learning policies for the AiDIN robot using PPO, IsaacGym
• K-ACC clustering for efficient initial state sampling of fallen poses
• CENet (Context Encoder Network) for context-aided fall-recovery learning

Setup

Requirements

• Python 3.8
• Isaac Gym v4 (Preview Release)
• PyTorch >= 1.4.0
• NVIDIA GPU with CUDA support

Installation

# 1. Create conda environment
conda create -n lifter python=3.8 -y
conda activate lifter

# 2. Install Isaac Gym (download from NVIDIA)
cd /path/to/isaacgym/python
pip install -e .

# 3. Install LIFTER packages
cd /path/to/LIFTER
pip install -e recovery_gym
pip install -e recovery_rl

# 4. Install dependencies
pip install -r requirements.txt

Project Structure

LIFTER/
├── accessibility/          # Stage 1: Pose sampling & K-ACC clustering
│   ├── k-access/           # K-means accessibility clustering
│   ├── chord/              # Chord matrix visualization
│   └── tsne/               # t-SNE dimensionality reduction
│
├── recovery_gym/           # Stage 2: Isaac Gym environments
│   ├── envs/               # Environment definitions
│   │   ├── base/           # Base task implementations
│   │   ├── aidin_6/        # 6-leg robot configs
│   │   └── aidin_8/        # 8-leg robot configs
│   ├── scripts/            # Training & evaluation scripts
│   └── utils/              # Terrain, logging utilities
│
├── recovery_rl/            # RL algorithms
│   ├── algorithms/         # PPO, SAC implementations
│   ├── modules/            # Actor-Critic networks
│   ├── runners/            # Training loops
│   └── context/            # CENet encoder
│
├── resources/              # Robot URDF & mesh files
├── scripts/analysis/       # W&B result analysis scripts
└── results/figures/        # Generated plots

Usage

Stage 1: Sampling Fallen Poses (Accessibility)

Before training, generate diverse fallen poses and compute accessibility metrics using PyBullet simulation.

cd accessibility/k-access

# Step 1: Sample 3000 random fallen poses
python sample_poses_aidin_8.py      # Generates poses_aidin_8.pickle

# Step 2: Compute accessibility time table (3000x3000 matrix)
# Warning: This step takes a long time due to pairwise pose transition computation
python access_poses_aidin_8.py      # Generates aidin_8_AccessTimeTable.npy

# Step 3: (Optional) Compute Euclidean distance matrix for comparison
python euclidean_poses_aidin_8.py

For AiDIN-6 robot, use the corresponding *_aidin_6.py scripts.

After generating the accessibility matrix, use the Jupyter notebooks in accessibility/chord/ to:

• Visualize the accessibility matrix
• Perform K-ACC clustering
• Extract cluster centroids for training initial states

Stage 2: Training

python recovery_gym/scripts/train.py --task=aidin_8_stair_rec_context

Evaluation

python recovery_gym/scripts/play.py --task=aidin_8_stair_rec_context --init_mode=acc

Task Naming Convention

Tasks follow the pattern: {robot}_{terrain}_{task}_{method}

• Robots: aidin_6, aidin_8
• Terrains: flat, rough, stair
• Tasks: loco (locomotion), rec (recovery)
• Methods: base, context (CENet), contextlstm, oracle