This project uses the NuBot rescue robot as the platform and obstacles crossing as the task. It builds a reinforcement learning training system in Isaac Sim and implements commonly used reinforcement learning algorithms.
The major objective of FTR-Bench is to establish a learning framework that enables the articulated tracked robot to navigate obstacles in various terrains. Specifically, FTR-Bench consists of three primary components: the simulation, tasks, and learning algorithms.
Due to GitHub's limitations on video and file sizes, we have made the videos related to this work and experiments available Video for easier access and viewing.
The flipper tracked robots have the capability to learn in various terrains. Considering real-world rescue robot applications, we have created four types of terrains, including flat ground, obstacles, bollards, and stairs. The task defines the conditions and objectives that the tracked robots need to meet in each scenario.
Eventually, our experiments demonstrate that RL can facilitate the robots to achieve some remarkable performance on such challenging tasks, enabling the agent to learn the optimal obstacle-navigation strategies within the environment. And there is still some room for improvement and more difficult tasks for future work.
We design multiple different terrains.
| Task Name | Description | Demo |
|---|---|---|
| Up the Steps | Simulate the tracked robot ascending steps. The height of the steps ranges from 0.2$m$ to 0.4$m$. |  |
| Down the Steps | Simulate the tracked robot descending steps. The height of the steps ranges from 0.2$m$ to 0.6$m$. |  |
| Cross the Uplifts | Simulate the tracked robot crossing over uplifts. The height of the uplifts ranges from 0.2$m$ to 0.4$m$, and the width ranges from 0.2$m$ to 0.5$m$. |  |
| Through Single-sided Steps | Simulate a tracked robot going through a single-side step. The height of the single-side step ranges from 0.1$m$ to 0.3$m$. |  |
| Drive on Plum Piles | Simulate the tracked vehicle navigating on plum piles. The size of each individual plum pile ranges from 0.2$m$ to 0.5$m$, and the height ranges from 0.2$m$ to 0.5$m$. |  |
| Drive on Wave Terrains | Simulate the tracked vehicle navigating through waveform terrains. The size of each individual peak in the waveform terrain ranges from 0.25$m$ to 1$m$, and the height ranges from 0.1$m$ to 0.4$m$. |  |
| Cross the Rails | Simulate the tracked robot crossing over rails. The height of the railing ranges from 0.2$m$ to 0.3$m$, and the angle with the vehicle ranges from 10 to 45 degrees. |  |
| Up the Stairs | Simulate the tracked vehicle climbing stairs. The width of each step of the stairs ranges from 0.2$m$ to 0.3$m$, and the slope of the stairs ranges from 20 to 45 degrees. |  |
| Down the Stairs | Simulate the tracked vehicle descending stairs. The width of each step of the stairs ranges from 0.2$m$ to 0.3$m$, and the slope of the stairs ranges from 20 to 45 degrees. |  |
| Mixed Terrains | Simulate the tracked vehicle navigating through various complex terrains, with the successful completion of the task defined when the vehicle reaches the specified endpoint. |  |
Different flipper structures can be achieved by modifying the configuration file.
| Task Name | Description | Demo |
|---|---|---|
| Push Cube | Three robots collaborate to push a heavy object forward |  |
| Transfer Objects | Three robots work together to transfer a platform carrying a heavy load forward. |  |
| Predation | Three red robots collaborate to encircle and capture a green robot. |  |
The simulation platform used in this project is Isaac Sim, and version 4.2.0 is recommended. For detailed installation instructions, please refer to the official website installation address.
download and install Isaac Sim
Since this project is built on Isaac Lab, it is essential to install Isaac Lab first. You can refer to the following steps: Github or Documentation
Run the following commands to install the project dependencies:
~/.local/share/ov/pkg/isaac-sim-4.2.0/python.sh -m pip install -r requirements.txtTo start training, you can execute the following code
~/.local/share/ov/pkg/isaac-sim-4.2.0/python.sh scripts/ftr_algo/train.py \
--task Ftr-Crossing-Direct-v0 \
--num_envs 256 \
--algo sac \
--terrain cur_base \
--epoch 30000 \
--experiment "test" \
--headlessIf you want to test the training results, you can use the following command
~/.local/share/ov/pkg/isaac-sim-4.2.0/python.sh scripts/ftr_algo/train.py \
--task Ftr-Crossing-Direct-v0 \
--num_envs 8 \
--algo sac \
--terrain cur_base \
--epoch 30000 \
--experiment "<checkpoint_path>" \
--playTo start training, you can execute the following code
~/.local/share/ov/pkg/isaac-sim-4.2.0/python.sh scripts/skrl/train.py \
--task Ftr-Prey-v0 \
--num_envs 64 \
--algorithm IPPO \
--headless \
--seed 20 ~/.local/share/ov/pkg/isaac-sim-4.2.0/python.sh scripts/skrl/train.py \
--task Isaac-Velocity-Rough-Anymal-D-v1 \
--num_envs 8- Proximal Policy Optimization (PPO) (Paper, Config)
- Trust Region Policy Optimization (TRPO) (Paper, Config)
- Twin Delayed DDPG (TD3) (Paper, Config)
- Soft Actor-Critic (SAC) (Paper, Config)
- Deep Deterministic Policy Gradient (DDPG) (Paper, Config)
- Multi-Agent PPO (MAPPO)(Paper, Config)
- Herogeneous-Agent PPO (HAPPO)(Paper, Config)
- Herogeneous-Agent TRPO (HATRPO)(Paper, Config)
| Up the Steps | Down the Steps | ||
|---|---|---|---|
 |
 |
 |
 |
| Cross the Uplifts | Through Single-sided Steps | ||
 |
 |
 |
 |
| Drive on Plum Piles | Drive on Wave Terrains | ||
 |
 |
 |
 |
| Cross the Rails | Up the Stairs | ||
 |
 |
 |
 |
| Down the Stairs | Mixed Terrains | ||
 |
 |
 |
 |
Related experimental videos can be viewed：Video
| Push Cube |  |
| PTransfer Objects |  |
| Predation |  |
(Click to Collapse)
- Research paper