Advanced sleep modes in 5G multiple base stations using non-cooperative multi-agent reinforcement learning

Amal Abdel Razzac (1) , Tijani Chahed (2, 3, 4) , Zahi Shamseddine , Wafik Zahwa (5, 6, 7)

1 LU / ULB - الجامعة اللبنانية [بيروت] = Lebanese University [Beirut] = Université libanaise [Beyrouth]
2 IP Paris - Institut Polytechnique de Paris
3 TSP - RS2M - Département Réseaux et Services Multimédia Mobiles
4 NeSS-SAMOVAR - Network Systems and Services
5 PESTO - Proof techniques for security protocols
6 RESIST - Resilience and Elasticity for Security and ScalabiliTy of dynamic networked systems
7 NUMERYX

We consider in this paper multiple 5G base stations (BSs) implementing Advanced Sleep Modes (ASM) wherein each base station is able to deactivate some of its components when it does not transport any traffic and save thus energy. Thanks to so-called lean carrier, ASM define four levels of sleep, the deeper the level the larger the energy gain but the more delay to wakeup and serve the incoming user. We specifically study this energy saving versus delay performance trade-off taking into account the effect of inter-cell interference and its impact on whether to wakeup and serve the transmission request immediately upon arrival or to continue to sleep; this latter decision is a main novelty of our work. We treat the case where arrivals of those requests are unknown and a reinforcement learning agent is implemented in each BS in order to (selfishly) derive the optimal sleep policy that achieves a target energy saving versus delay performance tradeoff. Our results show the optimal policies in terms of the value of the timer after which the BS goes into sleep, the time spent in each sleep level, and whether the BS should continue to sleep or wake up immediately upon request arrival. We eventually show the corresponding achieved power saving and delay performance. Index Terms—5G, multiple base stations, Advanced Sleep Modes, Multi-Agent Reinforcement Learning, energy saving versus delay performance trade-off.

Mots clés: 5G; multiple base stations; Advanced Sleep Modes; Multi-Agent Reinforcement Learning; energy saving versus delay performance trade-off

Domaines: Réseaux et télécommunications [cs.NI]; Ingénierie de l'environnement

Amal Abdel Razzac, Tijani Chahed, Zahi Shamseddine, Wafik Zahwa. Advanced sleep modes in 5G multiple base stations using non-cooperative multi-agent reinforcement learning. IEEE Global Communications Conference (GLOBECOM), Dec 2023, Kuala Lumpur, Malaysia. pp.7025-7030, ⟨10.1109/GLOBECOM54140.2023.10437599⟩. ⟨hal-04492371⟩