This repository contains one of the projects I did during my PhD in Artificial Intelligence Applied to Automation and Robotics. The idea was to propose a neural network architecture for cooperative multi-agent systems with heterogeneous agents, in which agents could be grouped into classes of homogeneous agents and policies as well as communication protocols could be learned faster by doing so.

• HMAG-QNet stands for Heterogeneous Multi-Agent Graph Q-Network. Paper and code information is available in the v8 tag.
• HCA-Net stands for Neural Network for Heterogeneous Communicative Agents. Paper and code information is available in the v9 tag.

To configure an environment, use conda env create in the root folder of the repository, which will configure a conda environment ready for use.

On a PC with the following specs:

• 16~32 GB RAM
• i7 7th generation
• NVIDIA GTX 1070 GPU

It takes anywhere from 4h30 to 14h to train a neural network for 1 million to 1.5 million steps. In some cases, training has to run for around 5 to 6 million steps for some models to learn (see paper referenced in the v9 tag).

I log my experiments on Weights & Biases. I don't know how this code behaves if ran on another account or if W&B is not installed or not logged in.

• 3m - a scenario with a single type of agent, to test all networks.
• 3s5z - an easy scenario, but there is still room for VDN to improve.
• MMM2 - an asymmetric heterogeneous environment in which VDN fails to win.
• 1c3s5z - an easy scenario with high heterogeneity.
• MMM - a symmetric heterogeneous environment, should be easier than MMM2.