ddodel/as-penguins repository overview

⁠as-lab-group-project

Reinforcement Learning Group Project for the 2020 Autonomous Systems Lab at LMU Munich

⁠Installation

You'll need something to manage your Python venv, I'm using miniconda⁠

# Create new virtual environment
conda create -n as-penguins python=3.7
# Activate environment
conda activate as-penguins
# Install requirements
pip install -r requirements.txt
# Train using default SAC
bash train.sh first_training
# Perform inference based on SAC trained model (From repo root directory)
python policy-penguins/examples/gym_unity_stable-baselines_sac_inference.py

The train.sh script is just a convenience for the following execution:

mlagents-learn --env environment_executable.x86_64 --run_id $1 [config]

If no config is passed as the second argument, it uses the SAC one; parameters were taken from the SAC default training parameters on the Unity repo.

⁠Usage

The available scripts in this package are:

• pp-info : Script that lists all available scripts and their functionality
• pp-optuna : Training using Hyperparameter optimization
• pp-inference : Inference using models saved in /models
• pp-manual-play : Manual Play on the Unity Penguins Environment

To see what command line arguments are available for optimization and inference, use the help tool description after installation:

$ pp-optuna --help
$ pp-inference --help

⁠Examples

policy-penguins/examples
├── gym_unity_openai-baselines_deepq.py
├── gym_unity_stable-baselines_sac_inference.py
└── gym_unity_stable-baselines_sac.py

⁠Stable (OpenAI) Baselines⁠

SAC examples for comparison with mlagents-learn results. Example trained model is as-penguins_baselines-test.zip, stable-baselines saves models as .zip files. File paths are currently hard-coded and relative from the repo root directory, so make sure to execute the scripts from there. Something like argparse⁠ or Click⁠ could be used to make it more flexible.

⁠OpenAI Baselines

DeepQ example is currently not working although it's straight from the gym_unity documentation⁠. Might have something to do with them not using a specific version, rather just installing it from the repo with pip install git+git://github.com/openai/baselines which is a moving target. Using the latest baselines version 0.1.5 on pip doesn't work as well.

⁠Unity Config

There is also a .gitignore file in the UnityPenguinLearner Directory. This .gitignore is used for Unity settings only.