Multi-agent Reinforcement Learning based Multi-model Running Latency Optimization in Vehicular Edge Computing Paradigm

Abstract

With the advancement of edge computing, more and more intelligent applications are being deployed at the edge in proximity to end devices to provide in-vehicle services. However, the implementation of some complex services requires the collaboration of multiple AI models to handle and analyze various types of sensory data. In this context, the simultaneous scheduling and execution of multiple model inference tasks is an emerging scenario and faces many challenges. One of the major challenges is to reduce the completion time of time-sensitive services. In order to solve this problem, a multiagent reinforcement learning-based multi-model inference task scheduling method was proposed in this paper, with a newly designed reward function to jointly optimize the overall running time and load imbalance. Firstly, the Multi-agent Proximal Policy Optimization algorithm is utilized for designing the task scheduling method. Secondly, the designed method can generate near-optimal task scheduling decisions and then dynamically allocate inference tasks to different edge applications based on their status and task characteristics. Thirdly, one assessment index, Quality of Method, is defined and the proposed method is compared with the other five benchmark methods. Experimental results reveal that the proposed method can reduce the running time of multi-model inference by at least 25% or more, closing to the optimal solution.