Reinforcement Learning for Robotics

Introduction to Reinforcement Learning for Robot / Drone.


AI-Robots

Figure AI


Sanctuary AI


ADAM

Paper: ADAM: a robotic companion for enhanced quality of life in aging populations

End-to-end Reinforcement Learning of Robotic Manipulation with Robust Keypoints Representation


Embodied AI

Blog: Overview of Embodied Artificial Intelligence
Embodied AI is the field for solving AI problems for virtual robots that can move, see, speak, and interact in the virtual world and with other virtual robots — these simulated robot solutions are then transferred to real world robots
The simulated environments for Embodied AI training: SUNCG, Matterport3D, iGibson, Replica, Habitat, and DART


Matterport3D

Paper: Matterport3D: Learning from RGB-D Data in Indoor Environments
Code: Matterport3D


Replica

Code: Replica Dataset


iGibson

Paper: iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks
Code: StanfordVL/iGibson


Habitat 2.0

Paper: Habitat 2.0: Training Home Assistants to Rearrange their Habitat
Code: facebookresearch/habitat-sim


Indoor Navigation

Autonomous Indoor Robot Navigation

Paper: Deep Reinforcement learning for real autonomous mobile robot navigation in indoor environments
Code:


DDPG 路徑規劃

Blog: 智慧送餐服務型機器人導航路徑之設計
路徑跟隨器有四個主軸:

  • 送餐路徑生成:從文件或上層發佈訊息獲取預先定義的路徑。
  • 編輯航線路徑點:清除路徑中不合適的航線路徑點。
  • MFAC無模型自適應控制之航段管制:自動調整送餐路徑之導航點之間的航段長度,依序共分成路徑跟隨之依據以及MFAC無模型自適應控制之應用。
  • DWA之區域路徑傳遞:依照MFAC調整之結果,產出相關生成路徑,並以DWA進行區域設定。

  • 自走車基於DDPG的室內路徑規劃

Long-Range Indoor Navigation

Paper: Long-Range Indoor Navigation with PRM-RL


Gym-Gazebo

Code: erlerobot/gym-gazebo


DART (Dynamic Animation and Robotics Toolkit)

Dartsim/dart

  • Python bindings: dartpy, pydart2 (deprecated)
  • OpenAI Gym with DART support: gym-dart (dartpy based), DartEnv (pydart2 based, deprecated)

PyBullet-Gym

PyBullet Quickstart Guide

code: rkuo2000/pybullet-gym

  • installation
    pip install gym
    pip install stable-baselines3
    git clone https://github.com/rkuo2000/pybullet-gym
    export PYTHONPATH=$PATH:/home/yourname/pybullet-gym
    

Train
python train.py Ant 10000000

Enjoy with trained-model
python enjoy.py Ant

Enjoy with pretrained weights
python enjoy_Ant.py
python enjoy_HumanoidFlagrunHarder.py (a copy from pybulletgym/examples/roboschool-weights/enjoy_TF_*.py)


PyBullet-Robots

env_name = “AtlasPyBulletEnv-v0”
atlas_v4_with_multisense.urdf


OP3 Soccer

Paper: Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning


RoboCar Gym

Pybullet-RoboCar

Blog:
Creating OpenAI Gym Environments with PyBullet (Part 1)
Creating OpenAI Gym Environments with PyBullet (Part 2)


Quadruped Gym

Motion Imitation

Code: TF 1.15

Code: PyTorch
For Training:
python motion_imitation/run_torch.py --mode train --motion_file 'dog_pace.txt|dog_spin.txt' \ --int_save_freq 10000000 --visualize --num_envs 50 --type_name 'dog_pace'
For Testing:
python motion_imitation/run_torch.py --mode test --motion_file 'dog_pace.txt' --model_file 'file_path' \ --encoder_file 'file_path' --visualize


Rex: an open-source quadruped robot

Code: nicrusso7/rex-gym


Drones Gym

PyBullet-Gym for Drones

  • Installation
    sudo apt install ffmpeg
    pip install numpy pillow matplotlib cycler
    pip install gym pybullet stable_baselines3 ray[rllib]
    git clone https://github.com/rkuo2000/gym-pybullet-drones.git
    cd gym-pybullet-drones
    
  • Train & Enjoy
    python train.py # modify train.py for different env, algorithm and timesteps
    python enjoy.py # modify enjoy.py for different env

  • Fly using DSLPIDControl.py:(PID飛行)
    python examples/fly.py --num_drones 1

  • To learn take-off:(起飛)
    python examples/learn.py

  • compare.py which replays and compare to a trace saved in files/example_trace.pkl

Experiments
cd experiments/learning

env : hover, takeoff, flythrugate, tune(旋停, 起飛, 穿越, 調整)
algo: a2c, ppo, sac, td3, ddpg

  • To learn hover:(旋停)
    python singleagent.py --env hover --algo a2c

To visualize the best trained agent:
python test_singleagent.py --exp ./results/save-hover-a2c

For multi-agent RL, using rllib:
python multiagent.py --num_drones 3 --env hover --algo a2c --num_workers 2


Flightmare

Flightmare is a flexible modular quadrotor simulator.


AirSim


Assistive Gym

Paper: Assistive Gym: A Physics Simulation Framework for Assistive Robotics

  • Four collaborative robots (PR2, Jaco, Baxter, Sawyer)

  • Support for the Stretch and PANDA robots

Code: Healthcare-Robotics/assistive-gym


Assistive VR Gym

Paper: Assistive VR Gym: Interactions with Real People to Improve Virtual Assistive Robots
Code: Healthcare-Robotics/assistive-vr-gym


Learning Dexity


Dexterous Gym


DexPilot

Paper: DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System


TriFinger

Paper: TriFinger: An Open-Source Robot for Learning Dexterity

Code: TriFinger Robot Simulation


Multi-Task Reset-Free (MTRF) Learning

Paper: Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention


Dexterous Anthropomorphic Robotic Hand

Blog: Robotic hand can crush beer cans and hold eggs without breaking them

Paper: Integrated linkage-driven dexterous anthropomorphic robotic hand


Robotic Telekinesis

Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube


Fixed-Finger Gripper

Paper: F1 Hand: A Versatile Fixed-Finger Gripper for Delicate Teleoperation and Autonomous Grasping


Learning Diverse Dexterous Grasps

Paper: Learning Diverse and Physically Feasible Dexterous Grasps with Generative Model and Bilevel Optimization


ViLa

Blog: https://bangqu.com/9Fa2ra.html


Mobile ALOHA

Paper: Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation
Github: https://github.com/MarkFzp/mobile-aloha
Code: https://github.com/MarkFzp/act-plus-plus


ALOHA 2

Paper: ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation


ALOHA unleashed

Paper: ALOHA Unleashed: A Simple Recipe for Robot Dexterity
![](<iframe width="942" height="530" src="https://www.youtube.com/embed/1EAsn71O9yA" title="ALOHA Unleashed: Autonomous Policies" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>)


DemoStart

Paper: DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots
Blog: Our latest advances in robot dexterity


ReKep

Paper: ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation
Code: https://github.com/huangwl18/ReKep



This site was last updated December 15, 2024.