Multi-Agent Reinforcement Learning for Cooperative Drone Swarm Coordination: Decentralized Policy Learning, Communication Protocol Design, and Emergent Collective Behavior in Search and Rescue Operations
Autonomous drone swarms operating in search and rescue environments must coordinate complex collective behaviors -- area coverage, target localization, survivor extraction, and obstacle avoidance -- in GPS-denied, communication-constrained, and dynamically hazardous conditions that preclude centralized coordination architectures. Multi-agent reinforcement learning (MARL) offers a pathway to decentralized cooperative policy learning that is robust to agent failure and communication disruption, but the engineering of MARL systems that produce reliable cooperative behavior at swarm scales of 10 to 100 agents in realistic environments remains an open challenge. This paper presents SwarmRL, a MARL framework for cooperative drone swarm coordination evaluated in three progressively realistic environments: 2D grid simulation, 3D physics simulation using AirSim, and a real-world outdoor evaluation with a 12-drone swarm. SwarmRL employs a shared policy architecture with agent-specific observation encoders, a learned communication protocol using Graph Attention Networks for neighbor-to-neighbor message passing, and a centralized training with decentralized execution (CTDE) approach using QMIX as the cooperative value decomposition baseline. In the simulated search and rescue benchmark, SwarmRL achieves 94.2 percent target discovery rate within 600 seconds for a 20-agent swarm, outperforming both handcrafted coverage algorithms (78.3 percent) and independent RL agents without communication (67.1 percent). The learned communication protocol transmits 89 percent less data than broadcast communication while maintaining equivalent coordination quality. Real-world evaluation with 12 drones achieves 87.4 percent target discovery rate, demonstrating meaningful sim-to-real transfer.
