Signal optimization at oversaturated intersections based on deep learning

In this paper, a discrete model of delays at oversaturated intersections was established by using the graphic analysis method and a reward function. The problem of optimizing signal control at an oversaturated intersection was transformed into a Markov decision process by utilizing queue length as the state variable and choice of green ratio as the action. Following this, a self-learning method for signal optimization (SLSO) at oversaturated intersections based on deep learning (dueling Q-learning, Dueling DQN) with priority experience replay (PER) was proposed. Moreover, dynamic learning rate and dynamic ε-greedy strategy were used to accelerate convergence and achieve better model optimization. Constraints and penalties on queue length were set to avoid traffic overflow. The results of comparative simulations showed that the traditional method yielded the longest delay while the two-stage bang-bang control method led to the shortest delay, with a reduction in the delay of about 13.3%. The proposed SLSO could reduce the delay by about 12.3% and the maximum queue length by 1.7% without constraints on queuing, and by 5.1% and 13%, respectively, with constraints on queuing in place. Finally, a real-world 4-phase intersection was selected to prove that the proposed SLSO method is widely applicable.