Skip to content
Snippets Groups Projects
README.md 3.26 KiB
Newer Older
# Anti-Poaching Game
This is the environment for the Anti-Poaching game. This is a muli-agent competitive game between poachers and rangers. This is currently on v2, and implements the model as described in the formal model description.
## Setting up the Virtual Environment
To have a ready-to-go environment created for you, execute the following line of code
```bash
$ source init.sh
This uses `python 3.8` (by default) and requires `virtualenv` to create it. To install the environment with a GPU-enabled version of pytorch enabled, you can supply the `full` option as follows:
```bash
$ source init.sh full
```

In case you want to install it yourself, either locally or if you want to manage the environment yourself, the following command should do it:
```bash
$ pip  install -e .[code,gpu] # For GPU-enabled torch
$ pip  install -e .[code,cpu] # For CPU-only torch
```

## Running tests
Tests can be run from the root folder using `pytest` as follows:
```bash
$ pytest [test/]
```
Furthermore, tools like `coverage` can be used to generate more extensive data like branch coverage as followS:
```bash
$ coverage --branch -m pytest
$ coverage html 
```
## Using the PettingZoo Environment

The main environment is implemented in [anti_poaching_game.py](./anti_poaching/env/anti_poaching_game.py), following the PettingZoo API. The `init.sh` script will automatically install the package `anti_poaching` for you (as a pip-editable package), so the following code should run:

```python
from anti_poaching.anti_poaching_v0 import anti_poaching
  
cg = anti_poaching.parallel_env(render_mode="rgb")
done, observations, terminations, truncations = False, None, None, None
action_mask = {
    agent: cg.grid.permitted_movements(agent) for agent in cg.agents
}

while not done:
    # sample the actions for each agent randomly
    actions = {
        agent: cg.action_space(agent).sample(mask=action_mask[agent])
        for agent in cg.agents
    }
    observations, _, terminations, truncations, _ = cg.step(actions)
    action_mask = {
        agent: observations[agent]["action_mask"] for agent in cg.agents
    }
    done = all(
        x or y for x, y in zip(terminations.values(), truncations.values())
    )
    cg.render()
```

Alternatively, try running the examples from [manual_policies](./examples/manual_policies/).

## Examples + RLlib

MERITXELL VINYALS's avatar
MERITXELL VINYALS committed
A few examples are found in the [examples](examples/) folder. 

### Manual policies

The [fixed_policy.py](examples/manual_policies/fixed_policy.py) and the [random_policy.py](examples/manual_policies/random_policy.py) show how to use the game using hand-coded policies, or just to show the basic RL loop.
### Rllib examples
The examples run MARL algorithms (Policy Gradients, PPO, QMIX) on the developed model using RLlib. All experiments can be launched the central script [main.py](examples/rllib_examples/main.py). This runs an RLlib algorithm (PPO) in Multi-Agent Independent Learning mode for an `AntiPoachingGame` instance by default. All examples have parameters that can be specified via command line (use --help to see all options); everything is wrapped to provide compatibility with RLlib.
```bash
MERITXELL VINYALS's avatar
MERITXELL VINYALS committed
# from the repository root
$ cd examples/rllib_examples
$ python main.py
```

To see all the configuration options possible, run
```bash
$ python main.py --help
MERITXELL VINYALS's avatar
MERITXELL VINYALS committed
```