Fuzzy Q-Learning Traffic Light Control based on Traffic Flow and Pedestrian Number Estimated from Visual Information

Document Type : Machine learning-Sadoghi

Authors

• Marjan Jalali moghaddam ¹
• Matin Hosseini ¹
• Reza Safabakhsh ²

¹ Amirkabir

² Amirkabir University of Technology Tehran, Iran

10.22067/cke.v2i1.77384

Abstract

A vision-based intelligent traffic control system is a robust framework that controls the traffic flow in real-time by estimating the traffic density near traffic lights. In this paper, a traffic light control system based on fuzzy Q-learning is proposed according to the vehicle density and the pedestrian number estimated from the visual information. The aim of proposed approach is to minimize the pedestrian and the car waiting time and maximize throughput for an isolated 4-way traffic intersection. Also, the pedestrian traffic light is controlled based on the fuzzy logic. The states and actions of the Q-learning variables are set by a fuzzy algorithm which can be learned through environmental interactions. The system can detect the number of pedestrians and vehicles using visual information from cameras and machine vision algorithms. The fuzzy control system can adjust the sequence of green phases to decrease the total waiting time and the mean of the queue length. The proposed algorithm was simulated for one hour for each of 14 different traffic conditions and was assessed and compared with the preset cycle time and vehicle actuated approaches. The results showed the proposed algorithm could decrease the total waiting time and the mean of the queue length effectively.

Keywords

• Intelligent traffic control system
• Traffic density
• Fuzzy logic
• traffic light control

---

References

[1] M. Jalali Moghaddam, E. Shaabani, and R. Safabakhsh, “Crowd density estimation for outdoor environment”’. In Proceedings of the 8th International Conference on Bioinspired Information and Communications Technologies. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), December 2014, pp. 306-310.

[2] A. Ghasemi, and R. Safabakhsh, “A real-time multiple vehicle classification and tracking system with occlusion handling”. IEEE 8th International Conference on Intelligent Computer Communication and Processing, Romania, Sep 2012, pp. 109-115.

[3] A. Ghasemi, and R. Safabakhsh, “Unsupervised foreground-background segmentation using growing self-organizing map in noisy background”. 3rd International Conference on Computer Research and Development, Shanghai, China, March 2010, pp. 334-338.

[4] A. S. Kalaki, and R. Safabakhsh, “Current and adjacent lanes detection for an autonomous vehicle to facilitate obstacle avoidance using a monocular camera”. 2014 Iranian Conference on Intelligent Systems (ICIS), IEEE, February 2014, pp. 1-6.

[5] A. S. Kalaki, and R. Safabakhsh, “Vision based real-time lane and obstacle detection and tracking in intelligent vehicles”. 13th International Conference on Traffic and Transportation Engineering, Tehran, Iran, February 2014, pp. 25-26.

[6] M. Keyarsalan, and A. Gholam, “Designing an intelligent ontological system for traffic light control in isolated intersections”, Engineering Applications of Artificial Intelligence, vol. 24, no. 8, pp. 1328-1339, 2011.

[7] WEBSTE COBBE, 1996,WEBSTER F. V., COBBE B. M. (1996). Technical Paper 56: Traffic Signals

[8] M. Dotoli, M. P. Fanti, and C. Meloni, “A signal timing plan formulation for urban traffic control”, Control Engineering Practice, vol. 14, no. 11, pp. 1297-1311, 2006.

[9] R. H. Smith, and D C. Chin, “Evaluation of an adaptive traffic control technique with underlying system changes”. Proceedings of the 27th conference on Winter simulation. IEEE Computer Society, 1995, pp. 1124-1130.

[10] R. Hoyer, and U. Jumar, “Fuzzy control of traffic lights’. In Fuzzy Systems, IEEE World Congress on Computational Intelligence., Proceedings of the Third IEEE Conference on Computational Intelligence, June 1994, pp. 1526-1531.

[11] W. Hejun, and M. Changyun, “Design of intelligent traffic light control system based on traffic flow”. In 2010 International Conference on Computer and Communication Technologies in Agriculture Engineering, 2010, 3, pp. 368-371.

[12] M. Arora, and V. K. Banga, “Intelligent traffic light control system using morphological edge detection and fuzzy logic”. In International Conference on Intelligent Computational Systems (ICICS'2012), January 2012, pp. 7-8.

[13] T. Tari, L. T. Koczy, C. Gaspar, et al. “Control of traffic lights in high complexity intersections using hierarchical interpolative fuzzy methods”. In Fuzzy Systems, 2006 IEEE International Conference, 2006, pp. 1045-1048.

[14] M. Shakeri, H. Deldari, A. Rezvanian, et al. “A novel fuzzy method to traffic light control based on unidirectional selective cellular automata for urban traffic”. 11th International Conference on Computer and Information Technology, 2008, pp. 300-305.

[15] B. Abdulhai, R. Pringle, and G. J. Karakoulas, “Reinforcement learning for true adaptive traffic signal control”, Journal of Transportation Engineering, vol. 129, no. 3, pp. 278-285, 2003.

[16] M. Abdoos, N. Mozayani, A. L. Bazzan, “Traffic light control in non-stationary environments based on multi agent Q-learning”. Intelligent Transportation Systems (ITSC), 2011 14th International IEEE Conference, 2011, pp. 1580-1585.

[17] T. Riedel, and U. Brunner, “Traffic control using graph theory”, Control Engineering Practice, vol. 2, no. 3, pp. 397-404, 1994.

[18] Q. Liu, and J. Xu, “Traffic signal timing optimization for isolated intersections based on differential evolution bacteria foraging algorithm”, Procedia-Social and Behavioral Sciences, vol. 43, pp. 210-215, 2012.

[19] P. Choudekar, S. Banerjee, and M. K. Muju, “Implementation of image processing in real time traffic light control”. In Electronics Computer Technology (ICECT), 2011 3rd International Conference on, April 2011, pp. 94-98.

[20] Y. Dujardin, F. Boillot, D. Vanderpooten, et al. “Multiobjective and multimodal adaptive traffic light control on single junctions”. InIntelligent Transportation Systems (ITSC), 2011 14th International IEEE Conference, October 2011, pp. 1361-1368.

[21] M. J. Moghaddam, M. Hosseini, and R. Safabakhsh, “Traffic light control based on fuzzy Q-leaming”. In 2015 The International Symposium on Artificial Intelligence and Signal Processing (AISP), IEEE, pp. 124-128, March 2015.

[22] A. L. Bazzan, D. de Oliveira, and B. C. da Silva, “Learning in groups of traffic signals”, Engineering Applications of Artificial Intelligence, vol. 23, no. 4, pp. 560-568, 2010.

[23] S. Mikami, and Y. Kakazu, “Genetic reinforcement learning for cooperative traffic signal control”. In Evolutionary Computation, IEEE World Congress on Computational Intelligence., Proceedings of the First IEEE Conference, June 1994, pp. 223-228.

[24] F. Zhu, , H. A. Aziz, X. Qian, et al. “A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework”, Transportation Research Part C: Emerging Technologies,‏ vol. 58, pp. 487-501, 2015

[25] B. Cesme, and P. G. Furth, “Self-organizing traffic signals using secondary extension and dynamic coordination’. Transportation Research Part C: Emerging Technologies, 48, pp. 1-15, 2014.

[26] A. Stevanovic, J. Stevanovic, J. So, et al. “Multi-criteria optimization of traffic signals: Mobility, safety, and environment”, Transportation Research Part C: Emerging Technologies, vol. 55, pp. 46-68, 2015.

[27] D. Pescaru, and D. I. Curiac, “Ensemble based traffic light control for city zones using a reduced number of sensors”, Transportation Research Part C: Emerging Technologies, vol. 46, pp. 261-273, 2014.

[28] T. Le, P. Kovacs, N. Walton, et al. “Decentralized signal control for urban road networks”, Transportation Research Part C: Emerging Technologies,‏ vol. 58, pp. 431-450, 2015

[29] Z. Cong, B. De Schutter, and R. Babuška, “Co-design of traffic network topology and control measures”, Transportation Research Part C: Emerging Technologies, vol. 54, pp. 56-73, 2015.

[30] D. Sun, R. F. Benekohal, and S. T. Waller, “Bi‐level programming formulation and heuristic solution approach for dynamic traffic signal optimization”, Computer‐Aided Civil and Infrastructure Engineering, vol. 21, no. 5, pp. 321-333, 2006.

[31] J. Li, Y. Zhang, and Y. Chen, “A self-adaptive traffic light control system based on speed of vehicles. Software Quality, Reliability and Security Companion (QRS-C), 2016 IEEE International Conference, August 2016, pp. 382-388.

[32] L. Qi, M. Zhou, and W. Luan, “Emergency traffic-light control system design for intersections subject to accidents”. IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 1, pp. 170-183, 2016.

[33] Z. Cao, S. Jiang, J. Zhang, and H. Guo, “A unified framework for vehicle rerouting and traffic light control to reduce traffic congestion. “ IEEE Transactions on Intelligent Transportation Systems, vol, 18, no. 7, pp. 1958-1973, 2016

[34] J. L. Fleck, C. G. Cassandras, and Y. Geng, “Adaptive quasi-dynamic traffic light control”. IEEE Transactions on Control Systems Technology, vol. 24, no. 3, pp. 830-842, 2016.

[35] R. S. Sutton, and A. G. Barto, Reinforcement learning: An introduction, vol. 2, no. 4. Cambridge: MIT press, 1998.

[36] T. N. Tan, “Texture feature extraction via visual cortical channel modelling’. In Proceedings., 11th IAPR International Conference on Pattern Recognition, August 1992, pp. 607-610.

[37] ‘LS-SVMlab version 1.8’, http://www.esat. kuleuven. be /sista/lssvmlab/, accessed 1 March 2015.

[38] C. Stauffer, and W. E. L. Grimson, “Adaptive background mixture models for real-time tracking”. In Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), vol. 2, pp. 246-252, IEEE, June 1999.

[39] A. Bonarini, A. Lazaric, F. Montrone, et al. “Reinforcement distribution in fuzzy Q-learning”, Fuzzy Sets and Systems, vol. 160, no. 10, 1420-1443, 2009.

[40] Y. Chong, C. Quek, and P. Loh, “A novel neuro-cognitive approach to modeling traffic control and flow based on fuzzy neural techniques”, Expert Systems with Applications, vol. 36, no. 3, 4788-4803, 2009.

[41] M. Rezzai, et al. “Design and realization of a new architecture based on multi-agent systems and reinforcement learning for traffic signal control”, 2018 6th International Conference on Multimedia Computing and Systems (ICMCS). IEEE, 2018.

[42] M. Rezzai, et al. “Reinforcement learning for traffic control system: Study of Exploration methods using Q-learning”, International Research Journal of Engineering and Technology, vol. 04, no. 10, pp. 1838-1848, 2017.