Lecture

Reinforcement Learning for Robotics

12 Sep 2025 • Richard Kuo

Introduction to Robot Gym

Gym-Gazebo

Code: erlerobot/gym-gazebo

DART (Dynamic Animation and Robotics Toolkit)

Dartsim/dart

Python bindings: dartpy, pydart2 (deprecated)
OpenAI Gym with DART support: gym-dart (dartpy based), DartEnv (pydart2 based, deprecated)

PyBullet-Gym

PyBullet Quickstart Guide

code: rkuo2000/pybullet-gym

installation

pip install gym
pip install stable-baselines3
git clone https://github.com/rkuo2000/pybullet-gym
export PYTHONPATH=$PATH:/home/yourname/pybullet-gym

Train
python train.py Ant 10000000

Enjoy with trained-model
python enjoy.py Ant

Enjoy with pretrained weights
python enjoy_Ant.py
python enjoy_HumanoidFlagrunHarder.py (a copy from pybulletgym/examples/roboschool-weights/enjoy_TF_*.py)

PyBullet-Robots

env_name = “AtlasPyBulletEnv-v0”
atlas_v4_with_multisense.urdf

OP3 Soccer

Paper: Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

RoboCar Gym

Pybullet-RoboCar

Blog:
Creating OpenAI Gym Environments with PyBullet (Part 1)
Creating OpenAI Gym Environments with PyBullet (Part 2)

Quadruped Gym

Code: PyTorch
For Training:
python motion_imitation/run_torch.py --mode train --motion_file 'dog_pace.txt|dog_spin.txt' \ --int_save_freq 10000000 --visualize --num_envs 50 --type_name 'dog_pace'
For Testing:
python motion_imitation/run_torch.py --mode test --motion_file 'dog_pace.txt' --model_file 'file_path' \ --encoder_file 'file_path' --visualize

Rex: an open-source quadruped robot

Code: nicrusso7/rex-gym

Drones Gym

PyBullet-Gym for Drones

Installation

sudo apt install ffmpeg
pip install numpy pillow matplotlib cycler
pip install gym pybullet stable_baselines3 ray[rllib]
git clone https://github.com/rkuo2000/gym-pybullet-drones.git
cd gym-pybullet-drones

Train & Enjoy
python train.py # modify train.py for different env, algorithm and timesteps
python enjoy.py # modify enjoy.py for different env
Fly using DSLPIDControl.py:（PID飛行）
python examples/fly.py --num_drones 1
To learn take-off:（起飛）
python examples/learn.py
compare.py which replays and compare to a trace saved in files/example_trace.pkl

Experiments
cd experiments/learning

env : hover, takeoff, flythrugate, tune（旋停, 起飛, 穿越, 調整）
algo: a2c, ppo, sac, td3, ddpg

To learn hover:（旋停）
python singleagent.py --env hover --algo a2c

To visualize the best trained agent:
python test_singleagent.py --exp ./results/save-hover-a2c

For multi-agent RL, using rllib:
python multiagent.py --num_drones 3 --env hover --algo a2c --num_workers 2

Flightmare

Flightmare is a flexible modular quadrotor simulator.

Introduction
Prerequisites
Install Python Packages
Install ROS

running ROS

roslaunch flightros rotors_gazebo.launch

flighRL

cd /path/to/flightmare/flightrl
pip install .
cd examples
python3 run_drone_control.py --train 1

AirSim

Assistive Gym

Paper: Assistive Gym: A Physics Simulation Framework for Assistive Robotics

Four collaborative robots (PR2, Jaco, Baxter, Sawyer)
Support for the Stretch and PANDA robots

Code: Healthcare-Robotics/assistive-gym

Assistive VR Gym

Paper: Assistive VR Gym: Interactions with Real People to Improve Virtual Assistive Robots
Code: Healthcare-Robotics/assistive-vr-gym

Learning Dexity

Dexterous Gym

ALOHA unleashed

Paper: ALOHA Unleashed: A Simple Recipe for Robot Dexterity
![](<iframe width="942" height="530" src="https://www.youtube.com/embed/1EAsn71O9yA" title="ALOHA Unleashed: Autonomous Policies" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>)