MARLAnts

Project Overview

MARLAnts was my first foray into reinforcement learning. I created a custom environment where simple agents, modelled on ants, must gather resources and stack blocks to build a tower. Each agent has limited vision and cannot communicate, so cooperation emerges solely through the reward structure.

Key Features

• Q‑learning agents – each ant learns a policy using Q‑learning to decide when to move, pick up or drop a block
• Reward shaping – agents are rewarded for increasing tower height and penalised for aimless wandering
• Custom environment – built on top of Pygame/OpenAI Gym with discrete actions and partial observability
• Emergent behaviour – after ~1,000 training episodes the ants learn to coordinate implicitly and build efficient towers

Technical Implementation

The environment is a 2D grid rendered with Pygame. Agents have a local view of their surroundings and an action space of either moving forward, grabbing or dropping blocks. Q‑tables are updated after each action. Training scripts log metrics and allow hyper‑parameter tuning. Visualisation tools render episode replays as GIFs.