Multi-Agent Reinforcement Learning for CAV Cooperative Merging Considering Communication Failure

Abstract

Highway on-ramp merging sections are prone to oscillations caused by merging vehicles cutting into and disrupting the mainline flow. With the advancements in vehicle-to-everything (V2X) communications, more intelligent traffic control strategies have been studied to migrate on-ramp congestions using available information. However, most studies assume perfect V2X communications and complete information, overlooking the impact of communication failures that may degrade the effectiveness of cooperative merging strategies. In learning-based approaches, such failures can disrupt or distort the information provided to neural networks, leading to inaccurate decisions or predictions. This study introduces a Multi-Agent Reinforcement Learning (MARL) strategy with an attention mechanism, namely Attention-based Multi-Agent Proximal Policy Optimization (AMAPPO), for cooperative merging at highway on-ramp sections. Additionally, the significance of various V2X information sources and their impact on traffic control performance are evaluated. Experimental results demonstrate that our AMAPPO is more robust for impaired information compared to standard MAPPO.