Optimizing mmWave Base Station Deployment in Smart Campuses via Multi-Agent Offline Deep Reinforcement Learning

Background and Context

Fifth-generation and future wireless networks rely on mmWave spectrum to deliver the large bandwidths needed for capacity growth, but millimeter-wave signals suffer heavy propagation loss and highly directional beamforming, meaning the physical placement of base stations directly determines coverage quality and user experience. Smart campuses represent a particularly difficult deployment environment: buildings cluster densely and user demand concentrates in specific venues, producing a non-convex radio topology that defies simple geometric reasoning. Traditional placement methods built on convex optimization or closed-form analysis assume smooth, well-behaved geometry and therefore struggle to capture the realities of such campuses.

The study frames the core challenge as jointly maximizing coverage completeness and fairness among users. Because the max-min fairness objective function is both non-convex and non-smooth, and because real campus topologies are non-convex, the siting problem is classified as NP-hard. Rather than seeking exact optimal solutions, which are computationally infeasible at scale, the authors recast base station siting as a Markov Decision Process (MDP). This reformulation converts an otherwise intractable combinatorial problem into a sequential decision-making task in which an agent learns through interaction with the environment, enabling deep reinforcement learning (DRL) to discover approximate solutions.

Deep Analysis

The paper's central methodological contribution is a systematic comparison of four DRL architectures spanning two action-space types and two collaboration modes. The first is a discrete single-agent Deep Q-Network (DQN), which discretizes candidate base station locations into a finite action set and lets one agent make global placement decisions. The second is a spatially partitioned multi-agent DQN, which divides the campus into geographic sub-regions so that independent agents make localized choices, mitigating the exponential explosion of the action space.

To address continuous action requirements, the authors introduce a continuous single-agent Deep Deterministic Policy Gradient (DDPG) alongside a geographically partitioned multi-agent DDPG framework. This design deliberately covers the full matrix of discrete and continuous actions combined with single-agent and multi-agent coordination. In the multi-agent partitioning scheme, each agent handles reduced local decision complexity, which accelerates convergence in dense scenarios, while DDPG's deterministic policy suits the fine-grained continuous outputs needed for precise placement.

Industry Impact

Numerical experiments conducted on a realistic non-convex campus topology evaluate network coverage, the Jain fairness index, and algorithmic convergence. In dense user scenarios, the multi-agent DDPG approach substantially outperforms single-agent methods, validating the combined value of geographic partitioning and continuous action design. Most notably, it achieves full coverage together with a Jain fairness index of 0.94, indicating that user experience disparities are tightly constrained and that no small group of users is marginalized.

The multi-agent framework also demonstrates efficient computational convergence in a dense scenario serving 400 users, confirming that solution quality remains stable as user scale grows. These ablation-style comparisons reveal a clear pattern: as scenario complexity increases, single-agent methods degrade because their action and state spaces balloon, whereas multi-agent partitioning combined with a continuous DDPG policy better balances performance and efficiency. This makes the approach a credible algorithmic choice for operators balancing construction cost against user experience during 5G and 6G network rollouts.

Outlook

By unifying an NP-hard deployment problem under a single MDP formulation and publishing a shared comparison framework, the work lowers the research barrier for follow-up studies and encourages standardized evaluation in wireless resource management.

The reproducible benchmark offers a concrete reference point rather than merely proposing a single high-performing solution. The authors suggest the combined ideas of spatial partitioning and continuous actions extend naturally toward more demanding problems, including mobile user scenarios, time-varying topologies, and joint beamforming optimization, pointing to substantial room for future expansion.