Model for the identification of license plates in Mexico City using machine learning algorithms

Modelo para la identificación de matrículas en la Ciudad de México mediante algoritmos de aprendizaje automático

Modelo para a identificação de placas na Cidade do México usando algoritmos de aprendizado de máquina

 

Martínez-Prado Gustavo

Escuela Superior de Ingeniería Mecánica y Eléctrica, México

Miguel Pescador-Hernandez 

Escuela Superior de Ingeniería Mecánica y Eléctrica, México

Javier García-Ponce 

Escuela Superior de Ingeniería Mecánica y Eléctrica, México

Arturo Pérez-Cebreros

Escuela Superior de Ingeniería Mecánica y Eléctrica, México

prociencia2750@gmail.com

 

ABSTRACT

Computer vision is one of the fields of Artificial Intelligence that is flourishing because it focuses on the development and improvement of techniques that allow computers to identify, process and classify images, in a way that resembles human vision. This feature makes them an excellent tool for vehicle control systems. For this reason, we developed a system for the recognition of Mexico City license plates using artificial vision techniques, image processing and automatic learning, in order to monitor and speed up response times, when a stolen vehicle is found.

Keywords: Computer vision, machine learning, deep learning, license plates, automatic detection.

 

RESUMEN

La visión artificial es uno de los campos de la Inteligencia Artificial que está en auge debido a que se centra en el desarrollo y mejoramiento de técnicas que permiten a las computadoras identificar, procesar y clasificar las imágenes de una manera similar a lo que hace la visión humana. Esta característica los vuelve una excelente herramienta para los sistemas de control vehicular. Por ello, nosotros desarrollamos un sistema para el reconocimiento de matrículas de la Ciudad de México mediante técnicas de visión artificial, procesamiento de imágenes y aprendizaje automático, con la finalidad de monitorear y agilizar los tiempos de respuesta en caso de encontrar un vehículo robado.

Palabras clave: Visión artificial, aprendizaje automático, aprendizaje profundo, matrículas, detección automática.

 

RESUMO

A visão artificial é um dos campos da Inteligência Artificial que está no auge debido a que se centralize no desarrollo e aprimoramento de técnicas que permite que o computador identifique, processe e classifique as imagens de uma maneira similar a lo que hace a visão humanos. Esta característica dos vuelve é uma excelente ferramenta para os sistemas de controle veicular. Por isso, nosotros desarrollamos um sistema para o reconhecimento de matrículas da Ciudad de México com técnicas de visão artificial, processamento de imagens e aprendizado automático, com a finalidad de monitorear e agilizar os tempos de resposta no caso de encontrar um veículo roubado.

Palavras-chave: Visão computacional, aprendizado de máquina, aprendizado profundo, placas de licença, detecção automática.

 


INTRODUCTION

Nowadays, computer vision has a wide range of applications in object recognition, one of these is vehicle license plate recognition systems (LPR), which seek to recognize license plates in a non-intrusive way; that is, they use devices external to the car to monitor and locate the car (Weihong el al., 2020). The device par excellence for these systems is the camera, which can acquire static or dynamic images; cameras are sometimes enhanced with infrared technology to reduce factors that affect detection, for example environmental conditions or the time of day. Ideally, an LPR system should be optimized with recognition algorithms that allow it to work in real time and under real conditions in the field where it is applied, thus achieving an assertive and instantaneous response to the analysis carried out in a traffic context (Valencia et al., 2020).

Moreover, we have new models for Artificial Intelligence (AI) that can be perceived in different applications, such as voice recognition in vehicles (Prongnuch & Sitjongsataporn, 2020), recognition of traffic signs (Rodríguez et al., 2020; Calero et al., 2021), among others (Virgilio et al., 2020). As a plus, technological advances in hardware, in conjunction with these new AI models, have resulted in the development of new and hybrid algorithms, which seek the solution of different problems such as lighting (Mattoo et al., 2021), angles (Wijaya et al., 2020), real time (Benitez-Garcia et al., 2021),  tracking (López-Monroy et al., 2021), among many others (Janai et al., 2020). 

In recent years, the need to improve security in different contexts of society has led to the development of new computer systems and equipment (Karande & Joshi, 2020). Security through AI is an area of interest, which provides us with a completely new dimension to the intelligent tracking and navigation industry, to include route optimization (Díaz-Parra et al., 2020), vehicle function control (Receveur et al., 2020) and theft reduction (Prasad et al., 2020), among others (Martínez-Cruz et al., 2021).

In Vijayalakshmi et al. (2021) propose a model for vehicle identification and tracking, based on the global positioning system (GPS). For this case, the GPS support device moves continuously with the vehicle and will calculate the coordinates of each position. 

In the same way, Dang et al. (2021) proposes security systems against possible theft. In fact, vehicle manufacturing companies are incorporating different methods to provide better security mechanisms. In particular, vehicle safety can be carried out by implementing GPS and GSM (Global System for Mobile Communication) technologies. Additionally, automatic features have been incorporated to slow down vehicles in school or hospital zones (Sathiskumar et al., 2020; Zade et al., 2018). 

License plate recognition is the system where computer vision technology identifies the license plate number from an image without human intervention (Xiao et al., 2020). Design for this type of system is not an easy task, due to the different styles and characteristics of license plates. Some of the most important characteristics to consider in the recognition and detection of the numbers and letters on license plates include: lighting, quality, angle and the style of the typeface font and numbers on the license plate (Asif et al., 2017).

Automatic vehicle identification has become increasingly important for many applications; for example, parking fees (Farag et al., 2019), payment of tolls (Ahmed et al., 2019), traffic surveillance (Gupta et al., 2020), ticketing (Blythe, 2004), access control (Wijnhoven et al., 2011) , among others. A license plate is a unique feature by which to identify each individual vehicle. Automatic vehicle license plate recognition, as a research area has already been extensively studied, over several decades (Shashirangana et al., 2020).

Although significant progress has been made in license plate recognition techniques during the last decade, it is still a challenging task to recognize license plates from complex images, as a profficient system should function effectively under a variety of conditions, such as on a sunny day or at night, or with different colors and complex backgrounds. Selmi, et al. (2020) indicated that Automatic License Plate Detection and Recognition (ALPR) is employed to make detection and recognition processes more robust and efficient in very challenging environments, necessitating further investigation due to some limitations such as: integrity of numbering systems in countries, where colors, languages, sizes and fonts vary.

 

METHODOLOGY

The process for the automatic recognition of license plates begins with the acquisition of the image from a scene in three dimensions; this image is then processed to improve its quality and eliminate any possible imperfections; the next step is to separate the object of interest from the background, followed by the extraction of the characteristics that describe the object (color, texture and geometry). 

In this investigation, we undertake a comparison of intelligent algorithms for license plate recognition. The aim is to determine which of these algorithms represents the best alternative for achieving optimal identification of Mexico City number plates. 

Our solution using a ten-phase model is shown in Figure 1:


Fig 1 Ten-phase model


Acquisition, the first step is acquisition, so the present work set out to create its own database, made up of 1000 images applying RGB (red, green, blue) color mode to represent vehicle license plates in the metropolitan area. 

Grayscale; grayscale is the representation of an image, where each pixel is drawn using an individual numerical value that represents its luminance, on a scale ranging from black to white. For this group, a filter is applied to simplify this task, by representing the image in a single channel; with values limited to a range between 0 and 255, thus facilitating manipulation of the image. 

Gaussian filter; this image may be affected by noise due to the camera’s possible defects, and to eliminate part of this noise and rectify the image, it is necessary to use a mask, also known in the literature as a kernel. This will scroll through the entire image and perform a mathematical operation to reach a new value, which will be used to label the new image (Mirmozaffari, 2020). 

Hysteresis; for this group, we apply a hysteresis function based on two thresholds. This process intends to reduce the possibility of the appearance of false contours, which means it is implemented in order to mark the likely edges of the image and thus detect areas of interest (Mo et al., 2020).

Threshold; for this group, a process is undertaken in order to reach an optimal level, so as to differentiate objects in the background of an image from the objects in the foreground; a range is calculated for the identification of objects of interest. 

Segmentation; consists of dividing a digital image into several areas (groups of pixels), known as segments. 

Homography, this group is intended to facilitate detection by correcting the angle from which the image is viewed (Lee et al., 2020). 

Bounding box; in this group, the different letters on the license plate are individually divided. Once sorted, letters from this group are placed in order for the identification of letters (Darapaneni et al., 2020). 

Image normalization; in this group, a standard is established for all the images of the ordered letters; in this way all classifiers receive the same input properties. 

Intelligent algorithms; in this group, algorithms (Makkar et al., 2017; Liu et al., 2020) are applied from which the training and tests will be carried out, in order to identify letters on the license plate.

 

RESULTS

Figure 2 shows the number of license plates that match the six characters written on a license plate in Mexico City, as well as five, four and three characters, showing that more than 84% are correct using the SVM algorithm. Likewise, figure 3 shows the number of license plates which match the six letters written on them, as well as five, four and three letters, showing that more than 60% are correct when using the KNN algorithm; so apparently less efficient than using the SVM algorithm.


Fig 2 Number of license plates using SVM

Fig 3 Number of license plates using KNN


 

CONCLUSIONS

The analysis of large quantities of data based on pattern recognition and learning algorithms have helped enormously and opened the door to new possibilities. However, this is just the tip of the iceberg; the tendency is moving towards artificial and autonomous intelligent systems that are continually learning from the data, in order to make optimal analytical decisions. 

Despite the fact that the KNN algorithm is very tolerant to noise, SVM has the highest success rate in terms of detecting Mexico City license plates, with 84% effectiveness; the margin of error for one hundred and fifty-three cases, apparently indicated only one letter incorrectly, meaning five characters were correctly identified and that the letter which was most often confused with the classifier was the letter "T" with number "1". Thus future work will involve evaluating different computer vision techniques, together with a variety of neural networks that employ more complex images.

 

REFERENCIAS BIBLIOGRÁFICAS

[1]      Ahmed, S., Tan, T. M., Mondol, A. M., Alam, Z., Nawal, N., & Uddin, J. (2019). Automated toll collection system based on rfid sensor. In 2019 International Carnahan Conference on Security Technology (ICCST) (pp. 1-3). IEEE.

[2]      Asif, M. R., Chun, Q., Hussain, S., Fareed, M. S., & Khan, S. (2017). Multinational vehicle license plate detection in complex backgrounds. Journal of Visual Communication and Image Representation, 46, 176-186.

[3]      Benitez-Garcia, G., Prudente-Tixteco, L., Castro-Madrid, L. C., Toscano-Medina, R., Olivares-Mercado, J., Sanchez-Perez, G., & Villalba, L. J. G. (2021). Improving real-time hand gesture recognition with semantic segmentation. Sensors, 21(2), 356.

[4]      Blythe, P. T. (2004). Improving public transport ticketing through smart cards. In Proceedings of the Institution of Civil Engineers-Municipal Engineer (Vol. 157, No. 1, pp. 47-54). Thomas Telford Ltd.

[5]      Calero, M. J. F., Sanchez, M. A., Vargas, J., & Ayala, M. J. (2021). Ecuadorian regulatory traffic sign detection by using HOG features and ELM classifier. IEEE Latin America Transactions, 19(4), 634-642.

[6]      Dang, T., Gupta, V., Wadia, D. S., Kohli, P., & Chahal, R. K. (2021, March). FaceIgnition: An automatic anti-theft and keyless solution for vehicles. In 2021 International Conference on Computational Intelligence and Knowledge Economy (ICCIKE) (pp. 248-253). IEEE.

[7]      Darapaneni, N., Mogeraya, K., Mandal, S., Narayanan, A., Siva, P., Paduri, A. R., & Agadi, P. M. (2020). Computer vision based license plate detection for automated vehicle parking management system. In 2020 11th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) (pp. 0800-0805). IEEE.

[8]      Díaz-Parra, O., Ruiz-Vanoye, J. A., Fuentes-Penna, A., Zezzatti, A. O., Cruz-Hernández, E., Estrada-García, S., & Alvarado-Pérez, A. (2020). Editorial for Volume 11 Number 2: Electric School Bus Routing Problem for Smart Cities. International Journal of Combinatorial Optimization Problems and Informatics, 11(2), 1.

[9]      Diaz-Ramirez, V. H., & Juarez-Salazar, R. (2021, August). Multiple object tracking in color scenes using composite-matched filtering with complex constrains. In Optics and Photonics for Information Processing XV (Vol. 11841, p. 118410J). International Society for Optics and Photonics

[10]  Farag, M. S., El Din, M. M., & El Shenbary, H. A. (2019). Parking entrance control using license plate detection and recognition. Indonesian Journal of Electrical Engineering and Computer Science, 15(1), 476-483.

[11]  Gupta, A., Gandhi, C., Katara, V., & Brar, S. (2020). Real-time video monitoring of vehicular traffic and adaptive signal change using Raspberry Pi. In 2020 IEEE Students Conference on Engineering & Systems (SCES) (pp. 1-5). IEEE.

[12]  Janai, J., Güney, F., Behl, A., & Geiger, A. (2020). Computer vision for autonomous vehicles: Problems, datasets and state of the art. Foundations and Trends in Computer Graphics and Vision, 12(1–3), 1-308.

[13]  Karande, J., & Joshi, S. (2020, July). Real-Time Detection of Cyber Attacks on the IoT Devices. In 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT) (pp. 1-6). IEEE.

[14]  Le, H., Liu, F., Zhang, S., & Agarwala, A. (2020). Deep homography estimation for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7652-7661).

[15]  Liu, W., Wei, J., & Meng, Q. (2020). Comparisions on KNN, SVM, BP and the CNN for Handwritten Digit Recognition. In 2020 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA) (pp. 587-590). IEEE.

[16]  López-Monroy, A. P., Elias-Miranda, A. A., Vallejo-Aldana, D., García-Carmona, J. M., & Pérez-Espinosa, H. (2021). Deep Learning for Language and Vision Tasks in Surveillance Applications. Computación y Sistemas, 25(2), 317-328.

[17]  Makkar, T., Kumar, Y., Dubey, A. K., Rocha, Á., & Goyal, A. (2017). Analogizing time complexity of KNN and CNN in recognizing handwritten digits. In 2017 Fourth International Conference on Image Information Processing (ICIIP) (pp. 1-6). IEEE.

[18]  Martínez-Cruz, A., Ramírez-Gutiérrez, K. A., Feregrino-Uribe, C., & Morales-Reyes, A. (2021). Security on in-vehicle communication protocols: Issues, challenges, and future research directions. Computer Communications.

[19]  Mattoo, A., Saxena, K., Kumar, S., & Bagwari, N. (2021). Design of IoT-Based Smart Illumination System in Smart Cities. In Proceedings of 3rd International Conference on Computing Informatics and Networks (pp. 517-528). Springer, Singapore.

[20]  Mirmozaffari, M. (2020). Filtering in image processing. ENG transactions, vol 1. Mo, S., Gan, H., Zhang, R., Yan, Y., & Liu, X. (2020, June). Edge Detection Method Based on Hysteresis Connection and Prediction. In 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) (Vol. 1, pp. 1794-1798). IEEE.

[21]  Prasad, S. K., Rachna, J., Khalaf, O. I., & Le, D. N. (2020). Map matching algorithm: Real time location tracking for smart security application. Telecommunications and Radio Engineering, 79(13).

[22]  Prongnuch, S., & Sitjongsataporn, S. (2020). Thai voice-controlled analysis for car parking assistance in system-on-chip architecture. Advances in Technology Innovation, 5(4), 203.

[23]  Receveur, J. B., Victor, S., & Melchior, P. (2020). Autonomous car decision making and trajectory tracking based on genetic algorithms and fractional potential fields. Intelligent Service Robotics, 13(2), 315-330.

[24]  Rodríguez, R. C., Carlos, C. M., Vergara-Villegas, O. O., & Sánchez, V. G. C. (2020). Detección y clasificación de señales de tráfico mexicanas mediante aprendizaje profundo. Res. Comput. Sci., 149(8), 435-449.

[25]  San Hlaing, N. N., Naing, M., & San Naing, S. (2019). GPS and GSM based vehicle tracking system. International Journal of Trend in Scientific Research and Development (IJTSRD).

[26]  Sathiskumar, S., Navean, G. V., Hari Prakash, R., & Vishnu Praveen, S. (2020). Automatic Vehicle Speed Control System In A Restricted Zone. International Journal Of Scientific & Technology Research, 9(02).

[27]  Selmi, Z., Halima, M. B., Pal, U., & Alimi, M. A. (2020). DELP-DAR system for license plate detection and recognition. Pattern Recognition Letters, 129, 213-223.

[28]  Shashirangana, J., Padmasiri, H., Meedeniya, D., & Perera, C. (2020). Automated license plate recognition: a survey on methods and techniques. IEEE Access, 9, 11203-11225.

[29]  Shukla, A., & Jain, A. (2018). GPS-Based Tracking System for Vehicles Using Google Maps. IUP Journal of Electrical and Electronics Engineering, 11(4), 57-64.

[30]  Valencia, J., Ramirez-Guerrero, T., Castañeda, L., & Toro, M. (2020). Detección de infracciones y matrículas en motocicletas, mediante visión artificial, aplicado a Sistemas Inteligentes de Transporte. Revista Ibérica de Sistemas e Tecnologias de Informação, (37), 1-15.

[31]  Vazquez-Nicolas, J. M., Zamora, E., González-Hernández, I., Lozano, R., & Sossa, H. (2020). PD+ SMC quadrotor control for altitude and crack recognition using deep learning. International Journal of Control, Automation and Systems, 18(4), 834-844. 

[32]  Vijayalakshmi, N., Shrinithi, R., Sanjana, V., & Sowmiya, T. (2021, March). Vehicle Tracking and Locking System Based on GSM and GPS. In 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS) (Vol. 1, pp. 758-761). IEEE.

[33]  Virgilio G, V. R., Sossa, H., & Zamora, E. (2020). Vision-based blind spot warning system by deep neural networks. In Mexican Conference on Pattern Recognition (pp. 185-194). Springer, Cham.

[34]  Weihong, W., & Jiaoyang, T. (2020). Research on license plate recognition algorithms based on deep learning in complex environment. IEEE Access, 8, 91661-91675. 

[35]  Wijaya, K. T., Bharoto, L. Y., Purwanto, A., & Syamsuddin, E. Y. (2020). Vision-based parking assist system with bird-eye surround vision for reverse bay parking maneuver recommendation. In 2020 International Electronics Symposium (IES) (pp. 102-107). IEEE.

[36]  Wijnhoven, R. G., & de With, P. H. (2011). Identity verification using computer vision for automatic garage door opening. IEEE Transactions on Consumer Electronics, 57(2), 906-914.

[37]  Xiao, D., Zhang, L., Li, J., & Li, J. (2021). Robust license plate detection and recognition with automatic rectification. Journal of Electronic Imaging, 30(1), 013002.

[38]  Zade, M. B., & Kulkarni, S. M. (2018). Auto Vehicle Speed Control System-the Review. In 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT) (pp. 1556-1559). IEEE.