Skip to content

Multi-agent collaboration using Reinforcement Learning

License

Notifications You must be signed in to change notification settings

abhijaysingh/pizzaro

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PizzaRo - Multi-agent Collaboration using Reinforcement Learning

This research project explores multi-agent collaboration in completing assembly tasks. Reinforcement Learning is employed to learn a policy to place toppings on a pizza base using 2 UR10s. This project adds a UR10Assembler environment based on OmniIsaacGymEnvs to control a UR10 with the policy learned by reinforcement learning in Omniverse Isaac Gym/Sim.

We target Isaac Sim 2022.1.1 and tested the RL code on Ubuntu 20.04.

Preview

Installation

Prerequisites:

Make sure to install Isaac Sim in the default directory and clone this repository to the home directory. Otherwise, you will encounter issues if you didn't modify the commands below accordingly.

We will use Anaconda to manage our virtual environment:

  1. Clone this repository:

    cd ~
    git clone https://github.com/abhijaysingh/pizzaro.git
  2. Generate instanceable UR10 assets for training:

    Launch the Script Editor in Isaac Sim. Copy the content in omniisaacgymenvs/utils/usd_utils/create_instanceable_ur10.py and execute it inside the Script Editor window. Wait until you see the text Done!.

    • Copy the two folders from the assets/models folder to omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/. Check if the props and thr robots are properly copied into the specified folders. These assets will be instantiated in the training environment.
  3. (Optional) Install ROS Melodic for Ubuntu and Set up a catkin workspace for UR10.

    Please change all catkin_ws in the commands to ur_ws, and make sure you can control the robot with rqt-joint-trajectory-controller.

    ROS support is not tested on Windows.

  4. Download and Install Anaconda.

    wget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh
    bash Anaconda3-2022.10-Linux-x86_64.sh
  5. Patch Isaac Sim 2022.1.1

    export ISAAC_SIM="$HOME/.local/share/ov/pkg/isaac_sim-2022.1.1"
    cp $ISAAC_SIM/setup_python_env.sh $ISAAC_SIM/setup_python_env.sh.bak
    cp ~/pizzaro/isaac_patch/setup_python_env.sh $ISAAC_SIM/setup_python_env.sh
  6. Set up conda environment for Isaac Sim

    # conda remove --name isaac-sim --all
    export ISAAC_SIM="$HOME/.local/share/ov/pkg/isaac_sim-2022.1.1"
    cd $ISAAC_SIM
    conda env create -f environment.yml
    conda activate isaac-sim
    cd ~/pizzaro
    pip install -e .
    # Below is optional
    pip install pyyaml rospkg
  7. Activate conda & ROS environment

    export ISAAC_SIM="$HOME/.local/share/ov/pkg/isaac_sim-2022.1.1"
    cd $ISAAC_SIM
    conda activate isaac-sim
    source setup_conda_env.sh
    # Below are optional
    cd ~/ur_ws
    source devel/setup.bash # or setup.zsh if you're using zsh

Please note that you should execute the commands in Step 7 for every new shell.

Training

You can launch the training in headless mode as follows:

cd ~/pizzaro
python omniisaacgymenvs/scripts/rlgames_train.py task=UR10Assembler headless=True

The number of environments is set to 512 by default. If your GPU has small memory, you can decrease the number of environments by changing the arguments num_envs as below:

cd ~/pizzaro
python omniisaacgymenvs/scripts/rlgames_train.py task=UR10Assembler headless=True num_envs=512

You can also skip training by downloading the pre-trained model checkpoint from here and unzip it to ~/pizzaro/: model_checkpoint

or use this (Recommended) :

cd ~/pizzaro
wget https://github.com/abhijaysingh/pizzaro/releases/download/v1.0.0/runs.zip
unzip runs.zip

Testing

Make sure you have model checkpoints at ~/pizzaro/runs, you can check it with the following command:

ls ~/pizzaro/runs/UR10Assembler/nn/

You can visualize the learned policy by the following command:

cd ~/pizzaro
python omniisaacgymenvs/scripts/rlgames_train.py task=UR10Assembler test=True num_envs=512 checkpoint=./runs/UR10Assembler/nn/UR10Assembler.pth

Likewise, you can decrease the number of environments by modifying the parameter num_envs=512.

Help

For any queries, raise an issue and I will get back to you.