README.md

# Anti-Poaching Game

This is the environment for the Anti-Poaching game. This is a muli-agent competitive game between poachers and rangers. This is currently on v2, and implements the model as described in the formal model description.


## Setting up the Virtual Environment

To have a ready-to-go environment created for you, execute the following line of code
```bash
$ source init.sh
```

This uses `python 3.8` (by default) and requires `virtualenv` to create it. To install the environment with a GPU-enabled version of pytorch enabled, you can supply the `full` option as follows:
```bash
$ source init.sh full
```

In case you want to install it yourself, either locally or if you want to manage the environment yourself, the following command should do it:
```bash
$ pip  install -e .[code,gpu] # For GPU-enabled torch
$ pip  install -e .[code,cpu] # For CPU-only torch
```

## Running tests

Tests can be run from the root folder using `pytest` as follows:

```bash
$ pytest [test/]
```

Furthermore, tools like `coverage` can be used to generate more extensive data like branch coverage as followS:
```bash
$ coverage --branch -m pytest
$ coverage html 
```

## Using the PettingZoo Environment

The main environment is implemented in [anti_poaching_game.py](./anti_poaching/env/anti_poaching_game.py), following the PettingZoo API. The `init.sh` script will automatically install the package `anti_poaching` for you (as a pip-editable package), so the following code should run:

```python
from anti_poaching.anti_poaching_v0 import anti_poaching
  
cg = anti_poaching.parallel_env(render_mode="rgb")
done, observations, terminations, truncations = False, None, None, None
action_mask = {
    agent: cg.grid.permitted_movements(agent) for agent in cg.agents
}

while not done:
    # sample the actions for each agent randomly
    actions = {
        agent: cg.action_space(agent).sample(mask=action_mask[agent])
        for agent in cg.agents
    }
    observations, _, terminations, truncations, _ = cg.step(actions)
    action_mask = {
        agent: observations[agent]["action_mask"] for agent in cg.agents
    }
    done = all(
        x or y for x, y in zip(terminations.values(), truncations.values())
    )
    cg.render()
```

Alternatively, try running the examples from [manual_policies](./examples/manual_policies/).

## Examples + RLlib

A few examples are found in the [examples](examples/) folder. 

### Manual policies

The [fixed_policy.py](examples/manual_policies/fixed_policy.py) and the [random_policy.py](examples/manual_policies/random_policy.py) show how to use the game using hand-coded policies, or just to show the basic RL loop.

### Rllib examples

The examples run MARL algorithms (Policy Gradients, PPO, QMIX) on the developed model using RLlib. All experiments can be launched the central script [main.py](examples/rllib_examples/main.py). This runs an RLlib algorithm (PPO) in Multi-Agent Independent Learning mode for an `AntiPoachingGame` instance by default. All examples have parameters that can be specified via command line (use --help to see all options); everything is wrapped to provide compatibility with RLlib.

```bash
# from the repository root
$ cd examples/rllib_examples
$ python main.py
```

To see all the configuration options possible, run
```bash
$ python main.py --help
```