This code library contains functions to facilitate the development of relational reinforcement learning agents using Vejde and the MAL Simulator. It contains functions and wrappers to instatiate environments that produce observations in the format Vejde expects.

How to Use

1. Install the package and dependencies

This project uses uv for dependency management. Run uv sync --extra cu128 for PyTorch compiled with CUDA 12.8 and uv sync --extra cpu for CPU only.

2. Set up an environment

Use the function register_multi_env to register an environment with Gymnasium. You can then instatiate the environment with gymnasium.make.

The wrappers/ module contains various functional wrappers to alter the problem and observations in different ways. graph_wrapper.py shows to use of various wrappers set up in a pipeline we have used.

3. Train an agent

How and why you train your RL agent will likely vary a lot, but will probably involve 1) setting up the environment and 2) calling the training functions from Vejde. 

The example/ directory contains the code used to train the defender agents. The code is provided as we used it, and could benefit from some refinement, but it provides a useful starting point for anyone interested in training their own agents with Vejde and the MAL simulator.

Compatibility

This code library has been tested and run on Ubuntu LTS 24.04 and Debian Bookworm.