YOLO and Blockchain Technology Applied to Intelligent Transportation License Plate Character Recognition for Security

1  Introduction

Due to recent advancements in ITS and blockchain technology, an emphasis on ALPR has emerged in many academic fields [1]. ALPR is seen as being extremely important in many ITS applications. The acquisition of a vehicle image, License Plate (LP) [2] localization, segmentation, character classification [3], standardization, and character analysis are some of the components of a typical ALPR approach. However, maintaining the privacy and verifying data are constant concerns in such a paradigm.

Additionally, smart cities use a variety of sensors of various kinds to gather information about different parts of the city in a commonplace manner for efficient and intelligent governance. The ITS is one of the most crucial elements of any project to develop a smart city [3]. It is necessary to claim that a city will become globally recognized and its residents will enjoy more significant living standards. According to ITS, the architecture’s primary actors are consumers interested in traffic information. They safely exchange their data, which other users then confirm collectively. To get from one place to another, users can opt to use either crowd-sourced, verified data or data produced by the system. Based on the suppliers’ reputation, the data recorded is accurate and unchangeable. Moreover, the privacy issue is mainly focused on readings kept in the ALPR database [4]. It is still being determined what sort of use such data may be subject to as ALPR systems are often marketed to government organizations by private businesses. However, these records can be investigated afterward at the option of whoever owns the property and have access to the database. Nevertheless, each nation has laws governing protecting its inhabitants’ privacy. Generally, these rules guard the public against preserving personal information collected without express agreement. Storing personal data for the public interest, public safety, and other considerations is one of the permitted exceptions [5–7].

Blockchain technology has also been discussed in this study paper. A blockchain is a data structure in which blocks are connected to form a chain. Each block contains information that varies depending on the blockchain. The Ethereum network is one of many blockchains developed over time to meet particular operating requirements. This platform [8] can carry smart contracts and keep them on its blockchain. On the Ethereum platform, contracts are performed in an immutable manner, meaning that no unauthorized changes to the code after it has been produced can prevent them from functioning as intended. On the blockchain, smart contracts are scripted. They are similar to the stored procedures in relational database management systems in that they are automatically started after a transaction is activated [9]. However, they have different addresses and live on the blockchain. Blockchain technology can potentially revolutionize various industries, including the ITS sector. The ITS industry is concerned with enhancing transportation systems’ safety, efficiency, and sustainability. ITS systems require the communication and transfer of sensitive data, making security and privacy critical issues. Blockchain technology’s decentralized, immutable, and secure nature makes it an ideal solution to address these concerns. In ITS, blockchain can be utilized in numerous ways, including vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication, traffic management, and the management of electric vehicle charging stations. For instance, blockchain can create a decentralized V2V network where vehicles can securely exchange information about road conditions, weather, and traffic flow. Blockchain can also manage smart contracts that govern autonomous vehicle operations, such as paying tolls or parking fees.

Furthermore, YOLO is an object detection algorithm that can locate and classify objects in real-time. Using YOLO in ITS can enhance safety, efficiency, and sustainability in several ways. One potential application of YOLO in ITS is a license plate and character recognition for security. License plate recognition is essential for traffic enforcement and tracking stolen vehicles. YOLO can quickly and accurately identify license plates and characters in real-time, making enforcing traffic laws easier and improving public safety. Another potential application of YOLO in ITS is for autonomous driving systems. YOLO can detect and classify objects on the road, such as pedestrians, bicycles, and other vehicles, enabling autonomous vehicles to make real-time decisions based on the detected objects. It can enhance the safety and efficiency of autonomous driving systems by improving the vehicle’s ability to navigate and avoid potential collisions. Moreover, YOLO can be used for traffic monitoring and management, which is critical for reducing congestion and improving traffic flow. By detecting and classifying different types of vehicles, such as cars, buses, and trucks, YOLO can provide valuable data to traffic management systems, allowing them to optimize traffic light timings and improve traffic flow. Furthermore, YOLO can be used for environmental monitoring, including air quality and noise pollution. By identifying and classifying vehicles based on emission levels, YOLO can provide data to environmental monitoring systems, allowing real-time adjustments to reduce pollution levels.

Using one or more images that a camera has taken, ALPR systems automatically recognize a vehicle’s license plate number [10,11]. To identify license holders, they employ object detection, image processing, and pattern recognition techniques. The data is connected to the information stored in databases, enabling further research. ALPR systems detect vehicles, and license plate numbers are used to track them in criminal investigations. Once a suspect vehicle has been located, law enforcement officers can stop it, search it for evidence, and, if necessary, make arrests. However, the data stored in an ALPR system stores large amounts of data, which is kept for upcoming investigations. This data can be used to do a retrospective analysis and pick potential locations of a suspect under investigation based on the movement pattern of their car. Citizens’ privacy has been raised due to the keeping of vehicle data that is not relevant to investigations [12].

The deployment of ALPR systems by law enforcement agencies worldwide has bolstered crime prevention efforts and improved investigations. However, public groups and average citizens have questioned whether they maintain private information tied to recognized license plates. Due to the numerous time entries in the ALPR database and its restricted access, which poses a serious privacy risk, law enforcement agencies have dramatically boosted their use of ALPR devices [13,14].

Fig. 1 depicts the model’s framework, which includes the blockchain framework, the YOLO model, and the license plate recognition system. First, the dataset goes through a clustering stage before going through several convolutional layers. This model is called YOLO. It then enters the model used for detecting and recognizing license plates.

Figure 1: Framework for ALPR using YOLO and blockchain technology

Additionally, the ALPR approach may mistakenly identify similar objects like billboards and traffic signs as license plates [15]. This article suggests a novel method for recognizing automobile license plates based on plate pre-identification and the new model YOLO to address these issues. With the emergence of massive data and the advancement of processing power, including Solid state drive (SSD), faster R-CNN and YOLO [16], deep learning [17] techniques have seen tremendous success in the recognition of targets in recent years. However, these techniques could not locate the license plates correctly, which may impact how accurately license plate character recognition works. The performance of RCNN-based license plate detection systems [18] can be a high recall, but it is sensitive to the general text in the image background. To identify every character in an image, an RCNN classifier is used. After that, a different plate/non-plate RCNN classifier separates license plates from other text types. However, this approach is restricted to inputting images of the same size. An innovative vehicle LPR method is provided to address the abovementioned issues. The YOLO model is first suggested and integrated to recognize license plates more precisely with the K-means++ clustering method [18,19]. A plate pre-identification system is secondarily recommended to decrease wrong license plate placement. The YOLO model is one of the most effective and quick deep-learning object detection methods. On the training set, K-means++ clustering is employed for ALPR to choose the best candidate box sizes and numbers, hence improving license plate recognition accuracy. To further pinpoint the location of the license plate, the YOLO model adds one pass-through layer and blends high-resolution features with low-resolution images.

Meanwhile, the plate pre-identification system can accurately separate roadside objects from license plates. Based on the statistical index of the algorithms for recognizing license plates, some evaluation criteria are applied. A single neural network in the YOLO model processes the entire image. The network then separates the images into various regions, predicting the probabilities and accompanying bounding boxes [20].

The Flowchart for vehicle number plate detection and recognition approaches is depicted in Fig. 2, where it is split into two distinct techniques: number plate detection and number plate recognition. In addition, a YOLO object detector is further separated into Fast YOLO v2 and Fast YOLO v3 as part of the number plate identification method.

Figure 2: Flowchart of the current research work

This study’s main contribution is an ALPR system built on the YOLO object detection CNN, which employs blockchain technology to enhance LPR, and uses this system to identify license plates. Temporal redundancy is also used while analyzing video frames, processing each frame separately before merging the outcomes to produce a more reliable prediction for each vehicle. The suggested system outperforms the existing ALPR results, two datasets, and our proposed UFPR-ALPR model. The following is a summary of this academic paper:

•   A brand-new real-time, end-to-end ALPR system utilizing cutting-edge RCNNs for YOLO object identification.

•   A reliable two-stage character segmentation and recognition strategy use straightforward data augmentation techniques for training data, such as license plates and inverted characters.

•   A publicly available ALPR dataset containing 4500 thoroughly annotated photos focusing on typical and unusual real-world settings demonstrates that our suggested ALPR method produces excellent results in both scenarios.

•   A comparison of the suggested approach, earlier research in the literature, and two UFPR-ALPR dataset systems.

This paper is structured as follows: In Section 2, the relevant literature is skimmed over. In Section 3, the dataset is presented. The methodology will then be discussed in Section 4. Sections 5, 6 and 7 cover the discussion and conclusion of the findings or results.

2  Literature Review

This section provides a quick overview of recent research that applies deep learning techniques to ALRP in the context of blockchain and YOLO technologies in the ITS system [21]. Firstly, the literature review for the ITS system will be discussed, as much effort has been made in blockchain technology for an ITS that uses license plate recognition. Due to vehicle ad-hoc networks’ ability to broadcast messages that must be accepted by other cars nearby, anonymity is built to safeguard the blockchain-based application. Data protection regulations are vital to safeguard people’s privacy when personal data is kept and processed since privacy and trust management are significant concerns. Due to recent improvements in hardware and software technologies, video-based detection systems—a crucial part of ITS—can now provide flexible data collecting [22]. By looking at image processing techniques and extrapolating their results to future requirements for traffic engineering, a structured review is offered to help readers better understand this technology’s approaches and potential.

Next, the effort being made to use blockchain technology for ITS license plate recognition will be discussed. The proposed architecture highlights the privacy aspects of the license plate recognition system [23]. Various challenges, such as complicated surroundings and inconsistent lighting, have hindered previous LPR techniques. Deep learning works reasonably well in terms of object detection. Character segmentation and character recognition are two ALPR processes suggested [24,25].

Another example is the trust environment provided by [26] based on an intelligent vehicle architecture. The three main parts of the system are a communication network, blockchain technology, and mobile cloud computing. Mechanisms for privacy and trust are included in the seven-layer blockchain architecture. According to [27], cars grade messages according to their observations in a reputation system that uses the blockchain. The system comprises four entities: miners, standard vehicles, evil vehicles, and reliable organizations responsible for vehicle registration and capacity certification. The vehicles can transmit traffic data, verify the legitimacy of messages received using the issuer’s reputation, and provide ratings. Miners are chosen from among the vehicles to build and deliver a block containing all the ratings to each car. When a block is authorized, it is reviewed before being added to the blockchain [28].

The authors in [29] detected frontal car views and LPs using a single RCNN setup in a cascade fashion, achieving good recall and precision rates. In this instance, references [30–32] updated RCNNs expressly for LP detection and demonstrated that the enhanced versions outperformed the originals. For LP detection, support vector machines (SVM) and RCNNs was employed by [32], who noted that RCNNs are more efficient in real-time systems. The character segmentation and recognition processes are frequently combined in DL-based ALPR systems. A cropped LP’s characters are identified and segmented using an RCNN [33]. They significantly exceeded the baseline. They correctly segmented more than 99% of the characters by integrating character segmentation and recognition using hidden Markov models (HMMs); in this case, reference [34] achieved high LP recognition accuracy. The most likely LP was identified by using the Viterbi algorithm. This work [35] advocated employing random CNNs to extract features for character recognition, outperforming the usage of image pixels or learning the filter weights using back-propagation by a wide margin. According to Liu and Shi, the character identification process should be a sequence labeling problem.

ITS license plate recognition systems use YOLO technology to expedite processing and improve mobile data analysis [36]. Character identification, number extraction from the plate, and region of interest extraction are the three crucial phases. ALPR is used for traffic surveillance. Utilizing YOLO, which contains 53 model layers in the dataset of 200 images, RCNN is utilized to locate and recognize license plates [37–39]. A technique for extracting the characters from the Indian license plate from a grayscale photograph and segmenting the image was put forth [40,41].

Furthermore, reference [42] gathered over 1530 photos of license plates. The image data were processed using a multi-layered system-based deep learning algorithm. YOLO is used to determine where the license plate is on the car to detect the plate’s digits. Wei-hong has proposed a 96.8% accurate automatic license plate identification system. This project makes use of the YOLO algorithm. It can identify many cars in real-time, such as four vehicles at once in a single scene, thanks to its high frames per second (FPS) rates. The environment can impact some number plate photographs, including illumination, unfavorable weather, traffic, and rainy weather. Using a deep learning model based on YOLO and its variation YOLO-9000, reference [43] developed a method for recognizing license plates considering these considerations.

The accuracy, according to Yousef, is 82 percent, with some fault tolerance. Hudaib utilized YOLO v4 to detect and identify license plates. Number plate identification accuracy was 96.2 percent, while number plate detection accuracy was 95 percent. A YOLO-based method for number plate identification and recognition with a 73 percent accuracy was proposed by [44]. Mustafa, a different person, has described the other convolutional neural network techniques for helmet and license plate recognition, using the YOLO v2 onvolutional neural network to detect license numbers and helmet-less riders. They used texts to represent the pixel-sequence behavior to characterize the character classes. For each class, they picked pixel behaviors. The main objective of the authors is to achieve effective real-time performance. The timing complexity of the performance methodology was therefore evaluated. This work’s method is intriguing because it is based on pixel behavior. It also suggested a model that can detect several license plates simultaneously. This model was built on the YOLO method that was previously mentioned. The model was effective at recognizing license plates in actual traffic situations. To account for photographs taken from various perspectives, the authors in references [45,46] improved the YOLO algorithm for identifying license plates by adding a rotating component to the images.

It can be challenging to fully assess the given methods because many studies either target specific aspects of the ALPR (such as LP detection) or conduct tests on datasets that do not accurately reflect real-world events. Additionally, only some techniques can be used in specific situations since they cannot recognize LPs in real-time. Therefore, each stage’s YOLO object detection RCNNs are used to build a reliable, effective, end-to-end ALPR system. Additionally, data augmentation for character recognition is undertaken because, in some ALPR systems, this step is the bottleneck.

Table 1 lists past paper references with the methodology used and results.

3  Dataset

In this study, the UFPR-ALPR and AOLP databases were combined to recognize the license plate and recognize the characters.

3.1 AOLP

This dataset served as the training set for the created model. Two thousand forty-nine photos of Taiwanese license plates make up this collection. Images acquired from multiple sites with varying traffic densities, timings, and weather conditions served as the samples for processing in the experiment. The dataset is split into three main groups: road patrol, traffic law enforcement, and access control. These categories are all distinct from one another. In AC, the angle at which the photos are taken is a pan of 30 and a tilt of 60, and the distance between the plate and camera is less than 5 m. The inclination is 0° to the ground. If the plate in the image is examined, its orientation is below ten, between 0.20 and 0.25 of the image’s ratio. Six hundred eighty-one photos in total were taken under various circumstances. The remaining 100 photographs are taken for testing, leaving us with 581 images for training. The roadside camera has recorded pictures of the vehicles that have broken the laws in a different category called TLE. There were 757 pictures gathered for this category. The image in the final class, RP, was taken using the camera mounted on the patrol car. Different qualities can be seen in these pictures. They are captured from numerous viewpoints and at different distances. In this category, as many as 2049 images are present. A preview of the dataset is shown in Fig. 3.

Figure 3: Images of the AOLP dataset

3.2 UFPR-ALPR Dataset

UFPR-ALPR is another dataset that was used in the experiment. Four thousand five hundred photos were taken from inside the car as it navigated a dense urban environment’s traffic. About 30 frames were taken in each of the 150 videos from which the images were taken. The frames per second (FPS) rate is 30 FPS. Therefore, one LP is visible in the entire collection’s foreground, consisting of footage from 150 automobiles with 30 photos of each vehicle. Fig. 4 illustrates the dataset’s diversity. The films were recorded using the Huawei P9, GoPro Hero4 Silver, and iPhone 7 Plus. The images were saved as Portable Network Graphics (PNG) files with a 1920 × 1080 pixel resolution. The photos are different even though they were taken at the exact resolution and frame rate because separate cameras were utilized. The difference in camera characteristics, including autofocus, data rate, focal length, and optical image stabilization, is the cause of the disparate photographs. In addition, the repeated camera mounting causes a slight alteration in the camera’s location. Each camera collected up to 1500 pictures, classified into the following categories: 900 photographs of grey LP automobiles, 300 images of red LP automobiles, and 300 images of grey LP motorcycles. The LPs come in various sizes; those for cars are 40 cm by 13 cm, while those for motorcycles are 20 cm by 17 cm.

Figure 4: Images of UFPR-ALPR dataset

3.3 Data Pre-Processing

Data pre-processing is critical in preparing a dataset for training a license plate recognition model. This process involves applying various techniques to ensure the accuracy and consistency of the data. This study pre-processed the UFPR-ALPR and AOLP datasets using the following methods. Firstly, image resizing was performed, where license plate images were resized to a typical size of 416 × 416 pixels to ensure consistency in the input size of the images. Secondly, data augmentation techniques such as horizontal and vertical flipping, rotation, and brightness adjustment were applied to increase the diversity of the training data and reduce overfitting. The grayscale conversion was performed to reduce the number of input channels required for the model. Contrast enhancement techniques such as histogram equalization were applied to improve the contrast and visibility of the license plate images. Data augmentation [48,49] is performed in two different approaches, in addition to training with the characters in the training set.

Additionally, noise removal techniques such as Gaussian blur and median filtering were applied to remove any noise or artifacts in the license plate images. Finally, the pre-processed dataset was split into training, and testing sets with a ratio of 80:20 to evaluate the model’s performance on unseen data. These pre-processing techniques ensure that the license plate recognition model is trained on a high-quality dataset, leading to improved accuracy in the recognition process.

4  Materials and Methods

Information about traffic management, environmental changes, and congestion is gathered and disseminated through a transportation system known as ITS. Communications between vehicles and between vehicles and infrastructure are made possible by ITS in various ways. In addition, ITS includes information from both wired and wireless connections. Infrastructure-to-infrastructure communication typically uses cable technology, while V2V and V2I communication frequently use wireless technology. Each ITS node has an IoT permanently deployed to gather data from multiple sources. IoTs collect information on trip time, position, monitoring, and delays. Real-time connectivity is prioritized by all ITS nodes and vehicles. Vehicles are used to deliver the data to a neighboring data center. Data is processed and then sent back to the affected vehicles as information. While V2I communication employs a continuous air interface to communicate across long and medium distances, V2V communication uses a specific short-range communication channel. Following receipt of the data at the nearby data center, operations for data cleaning, error correction, data synthesis, and additional logical analysis begin. The data that has been analyzed gives insight into both the current condition and the predicted future values.

The security of the created blockchain ledge distribution or smart contract is another objective of this strategy. According to the security analysis, the established smart contract is vulnerable to various threats. The approach used in this work to produce an agreement with minimal complexity favored the suggested architecture in terms of security considerations. Therefore, conducting a more thorough security evaluation is necessary to identify any flaws that can endanger the contract’s proper execution in such conditions. The strategy advocated in this study applies to more than just ALPR systems. Considering cutting-edge technologies and platforms, anyone can adapt the suggested architecture to fit different settings requiring privacy, security, and trust in IoT situations. Blockchains have been used increasingly in IoT environments to ensure the functionality of these systems.

Once the ITS is connected with blockchain technology, it will be straightforward to implement blockchain with the automatic license plate recognition system. The precise positioning of license plate regions is crucial for license plate character identification. However, due to the cameras’ differing vantage points, the typical license plate location systems cannot detect license plates in lighting or learning situations. The Yolo model is one of the quickest and most accurate deep learning methods for object detection. However, there are two significant problems with the YOLO model for recognizing license plates [2,50–55]:

•   Most of the time, YOLO’s model can only identify available license plate regions.

•   The YOLO model, which misidentifies other items on the road as license plates, has a more significant false-positive rate.

4.1 Vehicle and LP Detection

Two CNNs are trained in this step to recognize vehicle recognition in an input picture and LP detection in a recognized vehicle. They are examining Fast-YOLO and YOLO models, which can handle both straightforward (ALOP) and more realistic (UFPR-ALPR) data. For more straightforward scenarios, automated identification of automobiles and their LPs is conceivable. However, carrying out these tasks in more realistic settings required a deeper approach. Therefore, the final convolutional layer’s filters must be adjusted to correspond to the number of classes to use the YOLO model. Eq. (1) is followed by the number of filters determined by:

Filters=(C+5)×A(1)

C and A are variables that are used to calculate the number of filters needed in the final convolutional layer of the YOLO model. C refers to the number of classes the YOLO model must detect. For example, if the YOLO model needs to detect vehicles, pedestrians, and traffic signs, then C would equal 3. A refers to the number of anchor boxes that the YOLO model uses. Anchor boxes are predefined boxes of different sizes and aspect ratios that detect objects of different shapes and sizes in an image. Contrarily, the UFPR-ALPR dataset contains photos of vehicles and motorbikes (two classes), necessitating 35 filters for the vehicle detection task. Using two classes produced better results in our tests. In the YOLO model, the same modifications were implemented. In addition, the necessary margin in the validation set was calculated for all LPs to be entirely within the bounding boxes of the cars discovered by the vehicle detection CNN. It prevents losing LPs when the vehicle is accurately recognized and segmented.

4.2 Character Recognition

The recognition of many characters may be hampered by even a one-pixel mismatch between the ground truth and the prediction since many characters may need to be correctly segmented, contain missing pieces, and are small. The characters and their labels are inputted into these networks during training. The top four layers of the character segmentation CNN for digit recognition were deleted in our experiments since the results were comparable but had a lower computational cost. The character segmentation CNN’s complete architecture is still used for letter recognition (with more classes and fewer instances). The last convolutional layer of the networks for recognizing letters and numbers has 155 and 75 filters, respectively. The segmented characters are grouped according to their horizontal and vertical placements to distinguish between cars and motorcycles. Even in circumstances where the LP is significantly skewed, the first three characters match the letters and the final four the numerals. Like the character segmentation stage, negative images representing characters from other vehicle types are first built. Next, an examination determines whether characters may be flipped horizontally and vertically to create new instances. Fig. 5 below shows the Character (License Plate) Recognition model and Vehicle LP Detection.

Figure 5: Block diagram of the proposed method

4.3 YOLO Model

You Only Look Once is referred to by the acronym YOLO. It is an object detection API that was created using Google Net. GoogleNet has a deeper CNN structure. Before YOLO, different object-detecting methods were developed. R-CNN, a deep learning algorithm, created object identification algorithms like Faster R-CNN. Two stages made up the faster R-CNN for object detection: bounding box detection and bounding box classification. However, YOLO is intended as a one-step object detection architecture, with the primary objective being quicker and better detection.

Given that it is intended to be a one-step process, the YOLO architecture only processes an image once before sending it to RCNN. Furthermore, the approach is more superficial because the area and object area are optional to be detected. In comparison to newly created R-CNN-based algorithms, this makes the procedure much simpler and faster. YOLO and its picture recognition mechanism are displayed in Fig. 6. The primary benefit of YOLO is its quick processing time, which is attained by applying the entire image to a single neural network. Contrary to the typical practice of obtaining the object bounding box first and moving on to object recognition, this method involves labor-intensive image processing that requires checking multiple positions and scales.

Figure 6: Object recognition procedure using YOLO

YOLO first splits the input picture into a grid to anticipate the bounding box within the grid. The object’s probabilistic categorization of the classes is then found by maximizing the confidence in the box. Consequently, the object’s center location coordinates (x, y), height, width (w, h), and confidence probabilities in the object’s category are identified.

The CNN structure of an input image is used to create 7 × 7 grid-based images. The total number of these photos will then determine the size of the final forecast region. The coordinates of the object, the class probability value when an object is located inside the grid and the tag of the class are all included in each grid row. The object’s points (coordinates) help determine the difference between the learning area’s position and the predicted one.

The YOLO model uses the loss function to consider the optimal weight. This type uses its capacity for training to keep track of weight. Loss functions can be classified into three categories: classification loss, localization loss, and confidence loss.

4.3.1 Localization Loss Function

The difference between the objects in a cell and the measured and predicted points of the total cells included in that image is used to determine the localization loss function. In addition, the variance between the measured width and height and the expected width and size of the border can also be used to compute it. Eq. (2) illustrates the localization loss function.

λcoord∑i=0s2∑j=0BIijobj[(xi−x^i)2+(yi−yi^)2]+λcoord∑i=0s2∑j=0BIijobj[(ωi−ωi^)2+(hi−hi^)2](2)

where λcoord is a coefficient that weighs the importance of the localization loss compared to the other loss terms, s is the number of grid cells in the image, B is the number of boundary boxes predicted by YOLO, i, and j are indices for the grid cells and boundary boxes, respectively. Iijobj is an indicator variable if the jth boundary box in the grid cell contains an object and 0 otherwise. xi, yi, ωi, hi are the predicted center coordinates, width, and height of the jth boundary box in the ith grid cell. x^i, yi^, ωi^, hi^ are the ground truth center coordinates, width, and height of the object that the j-th boundary box in the i-th grid cell is supposed to detect.

The difference between the objects in a cell and the measured and predicted points of the total cells in that image is used to determine the confidence loss function. In addition, the variance between the measured width and height and the expected width and height of the border can also be used to compute it. Eq. (3) illustrates the localization loss function.

λcoord∑i=0s2∑j=0BIijobj(Ci−Ci^)2(3)

where Ci is the predicted conditional class probability of the object in the ith grid cell, Ci^ is the object’s ground truth conditional class probability in the ith grid cell. When an object is detected, the conditional class probability is subtracted from it to calculate the classification loss function for each object. The classification loss function is the square of the difference. Eq. (4) is a representation of the classification loss function. A value of 1 is provided when an object is discovered. A value of 0 is presented if nothing is found.

∑i=0s2Iijobj∑cϵclasses(pi(c)−pi(c)^)2(4)

where pi(c) is the predicted probability of the object in the ith grid cell belonging to the class, pi(c)^ is the ground truth probability of the object in the ith grid cell belonging to class c. The total loss function known as the YOLO is calculated using Eq. (5). Three things may be found with it: the type of detected object, the number of objects per grid, the size of the location, and the boundary box. S stands for the grid number. It is seven by nature. B is the number of boundary boxes that YOLO foresaw. As a default, it is 2. C: The number of classes that YOLO identified.

λcoord∑i=0s2∑j=0BIijobj[(xi−x^i)2+(yi−yi^)2]+λcoord∑i=0s2∑j=0BIijobj[(ωi−ωi^)2+(hi−hi^)2]+∑i=0s2∑f=0BIijobj(Ci−Ci^)2+λcoord∑i=0s2∑j=0BIijnoobj(ci−ci^)2+∑i=0s2Iijobj∑cϵclasses(pi(c)−pi(c)^)2(5)

The structure of the neural network, the size of the filter, and the total number of stride parameters are all displayed in Fig. 7. Because YOLO views its detection problem as a regression analysis, no intricate mathematical procedure is ever needed. The YOLO neural network model is used convolutional and areas of maximal pooling. Layer-by-layer, fully linked convolutional neural networks are replaced by it. Despite the adjustment, the model can still forecast the class and position of objects simultaneously.

Figure 7: YOLO system architecture

Additionally, YOLO was trained using input images with a size of 416 × 416. They multiply the neural network and divide the result by 32 to get the output size. The prediction approach is more effective with a single center grid than with four grid cells around the center. YOLO could use anchor boxes to forecast 1000 bounding boxes together with their classes and anticipated objectivity for each anchor box, increasing bounding box forecasts for objects of various forms.

4.3.2 Car Licence Plate Recognition by YOLO

If YOLO is to be further optimized by enhancing the bounding box prediction performances, the shape of the picture in the anchor box must be changed. K-means clustering has been successfully used as an optimized anchor box size. However, if K-means clustering is utilized, a larger anchor box will result in a higher loss than a smaller one, which is why adjustments are required. K-average will be employed since it will have a closer spacing between the data about the anchor box’s dimensions. The capacity to recognize centroids is enhanced by using K-means clustering and determining K, which are the occurrences of centroids. The computation for object detection will inevitably take longer because of the enormous number of anchor boxes needed, slowing down the process.

Conversely, YOLO will base its forecasts on grid cells. As shown in Fig. 7, the bounding box’s dimensions can be predicted using the same method. The correct response will continue to be either 1 or 0 if the center coordinates stay inside the cell.

4.3.3 Character Recognition with YOLO

It takes two steps to recognize characters on a license plate. Each character must first be divided into various categories before being categorized individually. However, this approach has some particular difficulties. The first is that it takes time to complete two steps. Identifying the number plate is only helpful if the characters are correctly split in the first phase. Character recognition becomes an object recognition challenge when one character is used to represent an entire class of objects. For example, there are 35 separate classes based on the letters A through Z and the digits 1 through 9. The number 0 is thought to be the letter O. An OCR network, however, that uses a YOLO network with a less significant modification is currently being developed. Finally, it will consider that the typefaces and license plates vary by country. Character recognition will improve as a result.

4.4 Implementation of Blockchain Technology

This study proposes blockchain technology to solve the privacy and security issues of communicating sensitive user data in ITS systems. The use of blockchain technology can ensure the secure and trustworthy transfer of data between vehicles and infrastructure, as well as among different system nodes. Moreover, blockchain technology can enhance the transparency and accountability of the scenario by providing an immutable and tamper-proof record of all transactions and data transfers. A smart contract is designed and deployed on the blockchain network to integrate technology into the proposed system. The smart contract handles the communication and data transfer between different system nodes, ensuring that all data exchanges are performed securely and with the proper authorization. However, the security of the smart contract is a critical concern in this approach. The smart contract must be designed and implemented with adequate security to prevent potential threats and vulnerabilities. A thorough security evaluation is necessary to identify and address any flaws that may endanger the appropriate execution of the contract.

Implementing blockchain technology in the proposed intelligent vehicle license plate recognition (IVLPR) system involves several steps. First, a blockchain network is created to serve as a decentralized ledger for storing data related to license plate recognition. This network can be public or private, depending on the specific requirements of the IVLPR system. Next, a smart contract is created and deployed on the blockchain network. This smart contract contains the business logic and rules that govern the license plate recognition process, including data validation, storage, and access. The smart contract is designed to be self-executing, meaning it automatically enforces the rules and regulations defined in the contract without human intervention.

The license plate recognition data is collected from various sources, including cameras installed in vehicles and infrastructure, and transmitted to the IVLPR system. Advanced computer vision methods, including OCR and object detection, are used to process and analyze the data to retrieve license plate information. The data is then secured and immutable by being encrypted and kept on the blockchain network via a smart contract.

Authorized parties, such as law enforcement agencies or toll authorities, can access the license plate data stored on the blockchain by interacting with the smart contract. The smart contract provides a secure and transparent method for accessing the data, as all transactions are recorded on the blockchain and can be audited by anyone on the network. Implementing blockchain technology in the proposed IVLPR system provides several benefits, including enhanced security, transparency, and trust. The decentralized nature of the blockchain ensures that the license plate data is secure and tamper-proof. At the same time, the smart contract provides a transparent and self-executing mechanism for governing the license plate recognition process.

5  Results

In this work, we conducted experiments to evaluate the performance of the suggested model using a Python implementation in a Windows 10 environment. The experimental setup involved utilizing a dataset primarily consisting of photographs for license plate detection and character recognition. The model employed for learning was based on YOLO, with the adaptation of OCR-Net to recognize the text on license plates. The experimental flow is depicted in the block diagram shown in Fig. 8.

Figure 8: License plate detection and recognition

The dataset used in this work mainly consisted of photographs for license plate detection and character recognition. However, the specific details about the dataset, such as its size, sources, and any preprocessing steps applied, are not provided in the given text. The training parameters used to train the model were not explicitly mentioned in the provided information. It would be helpful to have information about the specific parameters, such as learning rate, batch size, optimizer, number of epochs, and any regularization techniques used, to understand the training process better. However, the text does mention that the model utilized for learning was based on YOLO, and the OCR-Net was employed to recognize the text on license plates.

To evaluate the experimentation outcomes, a confusion matrix methodology is used. The evaluation process can be displayed in tabular format to assess the effectiveness of the suggested categorization technique. The classification algorithm is taught using supervised learning to handle issues like statistical classification in machine learning. According to Table 2, each row in the predicted classes is represented by an instance of that class, and an example of that class illustrates each column in the actual courses. The fusion matrix utilized in the experiment is displayed in Table 2. Here, TP (True Positive) refers to the number of positive instances correctly classified by the model. TN (True Negative) refers to the number of negative cases the model correctly classified. FP (False Positive) refers to the number of negative instances incorrectly classified as positive by the model. FN (False Negative) refers to the number of positive cases incorrectly classified as negative by the model. TP and TN are generally considered correct predictions, while FP and FN are considered incorrect predictions.

Precision, Recall, Average Precision, and mAP are the only criteria used to evaluate performance for object detection. The accuracy is the ratio of the expected positive condition to the actual positive condition. Eq. (6) depicts precision.

Precision=TPTP+FP(6)

The percentage of projected Condition Positive to actual Condition Positive is known as recall. The recall is represented by Eq. (7).

Recall=TPTP+FN(7)

When recall and precision are excellent, the evaluation yields the best results. Unfortunately, it is not easy to decide whether to evaluate the model based on accuracy or recall to get satisfactory results. In this situation, connected to precision or recall values, the best course of action is to compute the average precision. Each pair of accuracy and recall is acquired whenever the threshold is converted, which aids in computing the average precision. Eq. (8) demonstrates average precision.

Average Precision =∑n(Rn−Rn−1)x∗Pn(8)

Rn is the recall value at the nth threshold, and Rn−1 is the recall value at the (n−1)th threshold. Pn is the precision value at the nth threshold, x is The difference between the recall values at the nth and (n−1)th thresholds.

The experimental findings show that the suggested deep learning model for car license plate recognition generated by modifying the YOLO has an average precision for license plate recognition of 96.4% compared to earlier techniques. Table 3 displays a comparison study between the experimental outcomes and conventional methods. The YOLO license plate recognition results are also shown in Fig. 9.

The experiment was conducted using the UFPR-ALPR dataset, and the results are depicted in below Fig. 8. Images captured inside the car make up this dataset, typically in an urban setting. Both the vehicle and the camera are moving during the acquisition operation. There are few recognition failures for the conventional machine learning picture classification algorithms approach. However, the suggested deep learning model achieves satisfactory results by altering the YOLO, as shown in the picture. Typical methods could not reliably identify the characters because the photographs were taken in varying lighting conditions. But the suggested model produced valuable outcomes.

The average precision for the UFPR-ALPR dataset is shown in Table 3, where it is clear that our suggested model outperformed other methods [56] and [57] with 96.1% precision.

The tests were carried out using a different dataset called AOLP, which comprises license plates for the Taiwan region. The photos in this dataset are front and rear views, varying in location, time, distance, and traffic circumstances. Seven hundred images were captured throughout this experiment to evaluate the model on each run. Table 4 demonstrates that the results are far more encouraging than conventional machine learning algorithms. Fig. 10 displays the outcomes.

Figure 10: AOLP results

The average precision for the ALOP dataset is shown in Table 4, where it is clear that our suggested model performed well with 96.2% precision, while alternative processes [56] and [57] scored, respectively, 89.9% and 92.8%.

6  Discussion

Data security is essential in ITS systems since sensitive user data is communicated online. Privacy issues occasionally arise due to the amount of data generated. This approach assesses the security of the established blockchain ledger distribution or smart contract. It is important to stress that increasingly complicated systems could necessitate complex contracts. By considering cutting-edge technologies and platforms, anyone can modify the suggested architecture to additional settings that require privacy, security, and trust. The LP number on a car may be recognized using ALPR systems, which are automated image processing tools. The license identification is retrieved by these systems using a variety of approaches, including object detection, image processing, and pattern recognition. When an ALPR system detects a vehicle, the license plate number is recognized and compared to vehicle database records of interest in criminological investigations.

This work presented a cutting-edge storage architecture to ensure privacy in license plate recognition systems. The architecture uses YOLO technology, smart contracts, and a private blockchain. With a focus on quicker and better detection, YOLO is created as a one-step object detection architecture. First, the YOLO architecture processes a single image, which then sends the image to CNN. The process is much simpler and faster than newly created R-CNN-based algorithms. The primary concept is to use OCR-Net to alter the YOLA model.

Then, a supervised learning format is used to train the classification algorithm in deep learning. Finally, the evaluation procedure was presented in tabular form to evaluate the suggested classification algorithm’s effectiveness. The dataset used to train the model primarily consists of photos for license plate detection and character recognition, and the learning model was built on the YOLO principle. The proposed techniques in this paper support privacy in several ways. First, blockchain technology and smart contracts ensure that sensitive user data communicated over wireless devices and routes such as radio channels and optical fiber is kept secure and private. Blockchain provides a decentralized and transparent way of storing and sharing data without needing a trusted third party, which helps to protect user privacy. Second, combining the YOLO object detection method and RCNN in the ALPR approach enables real-time object identification and recognition while preserving privacy. This approach accurately locates and identifies license plates without revealing other sensitive information. Finally, the proposed model addresses the challenges of identifying license plates due to shifting lighting conditions. It helps to ensure that these challenges do not compromise users’ privacy.

An effective real-time end-to-end ALPR system is made possible by the potent YOLO object detection CNNs used in this work. Every level of ALPR requires the training of a network, except character recognition, where letters and numbers are recognized separately. Additionally, we generated a dataset for UFPR-ALPR that is freely accessible to the public. It consists of 4500 painstakingly annotated images shot by one hundred and fifty cars while they were driving. Our dataset has a greater diversity of aspects and more than twice as many images as this project’s ALOP dataset. Character segmentation and recognition are currently ALOP systems’ bottleneck stages.

Techniques such as data augmentation to replicate LPs from different vehicle categories and boosting characters with a few examples in the training set were employed to boost recognition rates in both phases. These tactics, albeit straightforward, were crucial to obtaining exceptional outcomes. In the ALOP dataset, our system was able to achieve a complete recognition rate of 96.2%, significantly surpassing earlier results (89.9% and 92.8%) and displaying a little superior performance than other commercial systems (94.09%). Furthermore, only the suggested approach could accurately identify at least six characters in all LPs. The proposed ALOP and two commercial systems were assessed as baselines on the new dataset. Since both commercial systems had recognition rates under 70%, the results showed that the UFPR-ALPR dataset is exceedingly tricky. With an identification percentage of 96.1%, our system did better. However, more is needed for several applications of ALOP in the real world. The results were then assessed using parameters from confusion matrices. To forecast the accuracy of the characters recognized, average precision was determined. Finally, traditional machine learning approaches were used to compare the outcomes further. The results obtained using the suggested methodology outperformed those obtained using traditional machine learning methods in earlier research. A total accuracy of 96.2% was attained compared to some of the other procedures.

The proposed technique has several advantages over previously used technologies. Firstly, it offers improved accuracy, with a recognition rate of 96.2%, surpassing the previous technology. Secondly, the technique enables real-time object identification and recognition by combining a cutting-edge object detection technique with an RCNN built with the tensor flow core open source libraries. This capability allows for faster and more efficient identification of license plates, enhancing the system’s overall performance. Additionally, the proposed model can accurately identify license plate characters, significantly improving over standard methods even in shifting lighting conditions. This ability to identify license plate characters accurately enhances the security and reliability of the system [58].

The proposed technique for intelligent vehicle license plate identification based on YOLO and blockchain technology has demonstrated promise. It does, however, have some restrictions that must be recognized. For starters, the results’ generalizability may be limited due to the limited research on the application of YOLO and blockchain technology specifically for license plate recognition. Additional study is required to validate and expand the findings to other situations and datasets. Second, the proposed technique’s execution necessitates the usage of complicated blockchain and smart contract technology. This complexity can make implementation, maintenance, and integration with existing systems difficult. The method’s feasibility and ease of adoption may change based on the resources and skills available in different circumstances. Finally, the suggested technique’s real-time object identification and recognition capabilities necessitate strong hardware [59]. This may limit its scalability and usability in resource-constrained contexts when access to such technology is limited or prohibitively expensive. The system’s practical practicality and requirements should be considered in various deployment situations. Furthermore, the study is hampered by a lack of specific information regarding the dataset used. While the UFPR-ALPR dataset and recognition rate are presented, key parameters such as dataset size, variety, and sources are not discussed. The lack of such information makes it difficult to thoroughly evaluate the suggested model’s robustness and generalizability [60].

There are some scenarios and conditions in which the proposed method may not work well [61]. One example is the reliance on precise and high-quality image data for license plate identification and recognition. In real-world circumstances, environmental factors such as inadequate lighting, occlusions, or fast-moving vehicles may have an impact on the acquired image quality. Under such difficult conditions, the proposed approach may struggle to reliably identify and recognize license plates. Furthermore, the efficacy of the proposed approach may fluctuate when used to different types of license plates, particularly those with non-standard or atypical designs. Distinct areas or countries’ license plates may have distinct font styles, characters, and forms, which can make proper recognition difficult [62].

Another disadvantage is the reliance on blockchain technology, which adds complexity and expense above traditional approaches. Scalability issues plague blockchain networks, and the suggested approach may suffer similar difficulties when processing a huge volume of license plate data in real-time. The blockchain infrastructure’s scalability constraints must be considered, especially in scenarios with high traffic volume or systems requiring near-instantaneous reaction times.

Furthermore, real-time object identification and recognition utilizing deep learning approaches can be computationally demanding. The performance of the suggested technique may be influenced by the available computational resources, such as processing power, memory, and storage. The approach’s real-time performance may be reduced in resource-constrained contexts or on devices with limited capabilities. It is also critical to recognize the potential privacy risks connected with the proposed solution. Intelligent transportation systems collect and process sensitive user data, such as license plate information. It is critical to have strong data privacy and security procedures in place to handle privacy issues and safeguard user confidentiality. So many researchers are researching on the related topics [59–62]. The study does not go into detail about the privacy precautions that were put in place, and more research is needed to analyze the privacy consequences of the proposed approach.

7  Conclusions

ITS require high data security since they transmit sensitive user data online. Blockchain and smart contract technology have partially addressed privacy issues. Our work is the first to implement a blockchain-based method for ensuring privacy in ALPR systems through the YOLO technique. According to the security analysis, the deployed blockchain is immune to several attacks. The method adopted in this work to create a primary complexity contract was more favorable to the suggested architecture in terms of security concerns. However, it is important to stress that more complicated contracts can be required for more complex architectures. In these situations, it is essential to do a more thorough security analysis to find any flaws that could jeopardize the contract’s correct operation.

Moreover, this study’s YOLO object detection approach leverages conventional blockchain technology to locate and identify the license plate. This study used license plate numbers to apply the deep learning technique. Technology for detecting objects OCR-net is used to recognize the character from the plate, and YOLO is used to detect the object with minimal modification. Two license plate databases, UFRP-ALPR and AOLP, have been used. Confusion matrix parameters were used to assess the outcomes. The average precision was determined to estimate how accurately the characters would be recognized. The works were contrasted further using conventional machine learning methods. Earlier research studies utilizing traditional machine learning approaches outperformed those from the suggested methodology. 96.2% overall accuracy was achieved, which is higher than the other methods.

Further research can explore integrating advanced machine learning techniques with blockchain and smart contract technology to enhance the security and privacy of ALPR systems. Additionally, future studies can investigate using different databases and datasets for license plate detection and character recognition and compare the results with traditional machine-learning approaches. Furthermore, exploring the potential of integrating other sensors, such as lidar and radar, can provide more accurate and reliable data for traffic monitoring and law enforcement.

Acknowledgement: Authors would like to thank Mr. Mohammed Osama research team member for providing necessary information while drafting this manuscript.

Funding Statement: The authors extend their appreciation to the deanship of scientific research at Shaqra University for funding this research work through the Project Number (SU-ANN-202248).

Author Contributions: The authors confirm contribution to the paper as follows: study conception and design: F. Alharbi, R. Alshahrani; data collection: M. Zakariah; analysis and interpretation of results: F. Alharbi, R. Alshahrani, M. Zakariah; draft manuscript preparation: A. Aldweesh, A. A. Alghamdi. All authors reviewed the results and approved the final version of the manuscript.

Availability of Data and Materials: The dataset is available on reasonable request.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

References

1. R. Shashidhar, A. S. Manjunath, R. S. Kumar, M. Roopa and S. B. Puneeth, “Vehicle number plate detection and recognition using YOLO-V3 and OCR method,” in 2021 IEEE Int. Conf. on Mobile Networks and Wireless Communications (ICMNWC), Tumkur, Karnataka, India, pp. 1–5, 2021. [Google Scholar]

2. L. Rayson, E. Severo, A. Zanlorensi and L. S. Oliveira, “A robust real-time automatic license plate recognition based on the YOLO detector,” in Int. Joint Conf. on Neural Networks (IJCNN), Rio de Janeiro, Brazil, pp. 1–10, 2018. [Google Scholar]

3. R. Surendran, O. I. Khalaf and C. Andres, “Deep learning based intelligent industrial fault diagnosis model,” Computers, Materials & Continua, vol. 70, no. 3, pp. 6323–6338, 2022. [Google Scholar]

4. S. Rajendran, O. I. Khalaf, Y. Alotaibi and S. Alghamdi, “MapReduce-based big data classification model using feature subset selection and hyperparameter tuned deep belief network,” Scientific Report, vol. 11, no. 1, pp. 24138, 2021. [Google Scholar]

5. M. Rajalakshmi, V. Saravanan, V. Arunprasad, C. A. T. Romero, O. I. Khalaf et al., “Machine learning for modeling and control of industrial clarifier process,” Intelligent Automation and Soft Computing, vol. 32, no. 1, pp. 339–359, 2022. [Google Scholar]

6. S. Alsubari, S. Nagi, N. Deshmukh, A. A. Alqarni, N. Alsharif et al., “Data analytics for the identification of fake reviews using supervised learning,” Computers, Materials & Continua, vol. 70, no. 2, pp. 3189–3204, 2022. [Google Scholar]

7. T. Bai, K. Sun, S. Deng, D. Li, W. Li et al., “Multi-scale hierarchical sampling change detection using random forest for high-resolution satellite imagery,” International Journal Remote Sensing, vol. 39, no. 21, pp. 7523–7546, 2018. [Google Scholar]

8. G. Suryanarayana, K. Chandran, O. I. Khalaf, Y. Alotaibi, A. Alsufyani et al., “Accurate magnetic resonance image super-resolution using deep networks and Gaussian filtering in the stationary wavelet domain,” IEEE Access, vol. 9, pp. 71406–71417, 2021. [Google Scholar]

9. C. Kavitha, X. Anita and S. Selvan, “Improving the efficiency of speculative execution strategy in Hadoop using amazon elasticate for redis,” Journal of Engineering Science and Technology, vol. 16, no. 6, pp. 4864–4878, 2021. [Google Scholar]

10. M. A. Khan, O. Ibrahim Khalaf, C. Andrés Tavera Romero, M. Sulaiman and M. A. Bakar, “Application of intelligent paradigm through neural networks for the numerical solution of multi-order fractional differential equations,” Computational Intelligent Neuroscience, vol. 2022, pp. 16, 2022. [Google Scholar]

11. Y. Yu, Y. Li, J. Tian and J. Liu, “Blockchain-based solutions to security and privacy issues in the Internet of Things,” IEEE Wireless Communication, vol. 25, no. 6, pp. 12–18, 2018. [Google Scholar]

12. A. Pouraghily, M. N. Islam, S. Kundu and T. Wolf, “Privacy in blockchain-enabled IoT devices,” in 2018 IEEE/ACM Third Int. Conf. on Internet-of-Things Design and Implementation (IoTDI), Orlando, FL, USA, pp. 292–293, 2018. [Google Scholar]

13. T. Andreica and B. Groza, “Secure V2V communication with identity-based cryptography from license plate recognition,” in Sixth Int. Conf. on Internet of Things: Systems, Management, and Security (IOTSMS), Granada, Spain, pp. 366–373, 2019. [Google Scholar]

14. A. Verri Lucca, G. A. Mariano Sborz, V. R. Q. Leithardt, M. Beko, C. Albenes Zeferino et al., “A review of techniques for implementing elliptic curve point multiplication on hardware,” Journal Sensor Actuator Networks, vol. 10, no. 1, pp. 3, 2020. [Google Scholar]

15. N. Drljevic, D. A. Aranda and V. Stantchev, “Perspectives on risks and standards that affect the requirements engineering of blockchain technology,” Computer Standards & Interfaces, vol. 69, pp. 103409, 2020. [Google Scholar]

16. L. Wei, D. Anguelov, D. Erhan and C. Szegedy, “SSD: Single shot MultiBox detector,” In: B. Leibe, J. Matas, N. Sebe, M. Welling (Eds.Computer Vision—ECCV 2016. Amsterdam, Netherlands: Cornell University, pp. 21–37, 2016. [Google Scholar]

17. S. Ren, K. He, R. Girshick and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” arXiv1506.01497, 2015. [Google Scholar]

18. M. K. Alam, A. Ahmed, R. Salih, A. Faiz, S. Al Asmari et al., “Faster RCNN based robust vehicle detection algorithm for identifying and classifying vehicles,” Journal of Real-Time Image Processing, vol. 20, no. 5, pp. 93, 2023. [Google Scholar]

19. A. Lei, H. Cruickshank, Y. Cao, P. Asuquo, C. P. A. Ogah et al., “Blockchain-based dynamic key management for heterogeneous intelligent transportation systems,” IEEE Internet Things Journal, vol. 4, no. 6, pp. 1832–1843, 2017. [Google Scholar]

20. Z. Lu, G. Qu and Z. Liu, “A survey on recent advances in vehicular network security, trust, and privacy,” IEEE Transaction for Intelligent Transportation Systems, vol. 20, no. 2, pp. 760–776, 2018. [Google Scholar]

21. T. A. Butt, R. Iqbal, K. Salah, M. Aloqaily and Y. Jararweh, “Privacy management in social internet of vehicles: Review challenges, and blockchain-based solutions,” IEEE Access, vol. 7, pp. 79694–79713, 2020. [Google Scholar]

22. Z. Yang and L. S. C. Pun-Cheng, “Vehicle detection in intelligent transportation systems and its applications under varying environments: A review,” Image and Vision Computing, vol. 69, pp. 143–154, 2018. [Google Scholar]

23. L. M. Merola and C. Lum, “Emerging surveillance technologies: Privacy and the case of License Plate Recognition (LPR) technology,” Judicature, vol. 96, pp. 119, 2012. [Google Scholar]

24. G. Angel, J. R. vi. Jeny and J. A. Simon, “ALPR for ambiguous character using template matching with fuzzy classifiers: A survey,” International Journal of Computing, vol. 2, no. 6, pp. 234–244, 2013. [Google Scholar]

25. P. Kaur, Y. Kumar, S. Ahmed, A. Alhumam, R. Singla et al., “Automatic license plate recognition system for vehicles using a CNN,” Computers, Materials & Continua, vol. 71, no. 1, pp. 35–50, 2022. [Google Scholar]

26. S. Khazaee, A. Tourani, S. Soroori, A. Shahbahrami and C. Y. Suen, “A real-time license plate detection method using a deep learning approach,” in Pattern Recognition and Artificial Intelligence: Int. Conf., ICPRAI 2020, Zhongshan, China, pp. 425–438, 2020. [Google Scholar]

27. I. Ochôa, L. Calbusch, K. Viecelli, J. De Paz, V. Leithardt et al., “Privacy in the Internet of Things: A study to protect user’s data in LPR systems using blockchain,” in 17th Int. Conf. on Privacy, Security, and Trust, Fredericton, NB, Canada, pp. 1–5, 2019. [Google Scholar]

28. K. Fabian, A. Unterweger and D. Engel, “Implementing a blockchain from scratch: Why, how, and what we learned,” EURASIP Journal on Information Security, vol. 2019, pp. 1–14, 2019. [Google Scholar]

29. R. Praveen and A. Patawari, “Automatic license plate recognition for Indian roads using faster-RCNN,” in 2019 11th Int. Conf. on Advanced Computing (ICoAC), IEEE, pp. 275–281, 2019. [Google Scholar]

30. I. Tariqul and R. I. Rasel, “Real-time bangla license plate recognition system using faster R-CNN and SSD: A deep learning application,” in 2019 IEEE Int. Conf. on Robotics, Automation, Artificial-Intelligence and Internet-of-Things (RAAICON), IEEE, pp. 108–111, 2019. [Google Scholar]

31. T. Chunling and S. Du, “A hierarchical RCNN for vehicle and vehicle license plate detection and recognition,” International Journal of Electrical and Computer Engineering, vol. 12, no. 1, pp. 731, 2022. [Google Scholar]

32. A. N. Palanivel, T. Vigneshwaran, M. Sriv Arappradhan and R. Madhanraj, “Automatic number plate detection in vehicles using faster R-CNN,” in 2020 Int. Conf. on System, Computation, Automation and Networking (ICSCAN), IEEE, pp. 1–6, 2020. [Google Scholar]

33. O. Naaman, A. M. Abdulazeez, A. Sengur and S. G. S. Al-Ali, “Fused faster RCNNs for efficient detection of the license plates,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 19, no. 2, pp. 974–982, 2020. [Google Scholar]

34. L. Pujun, T. Qu, H. Gao and X. Gong, “Driving intention recognition of surrounding vehicles based on a time-sequenced weights hidden Markov model for autonomous driving,” Sensors, vol. 23, no. 21, pp. 8761, 2023. [Google Scholar]

35. S. Nitin, M. Anul Haq, P. K. Dahiya, B. R. Marwah, R. Lalit et al., “Deep learning and SVM-based approach for Indian licence plate character recognition,” Computers, Materials & Continua, vol. 74, no. 1, pp. 881–895, 2023. [Google Scholar]

36. A. M. Anas, A. Wibowo and E. A. Sarwoko, “YOLOv4 algorithm implementation based on darknet and optical character recognition on vehicle license plate detection,” in AIP Conference Proceedings, AIP Publishing, vol. 2683, no. 1, 2023. [Google Scholar]

37. D. Onishchenko, N. Liubchenko and A. Podorozhniak, “License plate recognition system based on mask R-CNN,” Automation of Technological and Business Processes, vol. 15, no. 3, pp. 37–43, 2023. [Google Scholar]

38. H. Nakahara, M. Shimoda and S. Sato, “A demonstration of FPGA-based you only look once version2 (YOLOv2),” in 2018 28th Int. Conf. on Field Programmable Logic and Applications (FPL), Dublin, Ireland, pp. 457–4571, 2018. [Google Scholar]

39. L. Xie, T. Ahmad, L. Jin, Y. Liu and S. Zhang, “A new CNN-based method for multi-directional car license plate detection,” IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 2, pp. 507–517, 2018. [Google Scholar]

40. K. M. Babu and M. V. Raghunadh, “Vehicle number plate detection and recognition using bounding box method,” in Int. Conf. on Advanced Communication Control and Computing Technologies (ICACCCT), Tamil Nadu, India, pp. 106–110, 2016. [Google Scholar]

41. T. S. Anuj, S. S. Pardeshi and A. D. Patange, “Automatic number plate detection using TensorFlow in Indian scenario: An optical character recognition approach,” Materials Today: Proceedings, vol. 72, pp. 1073–1078, 2023. [Google Scholar]

42. J. Shashirangana, H. Padmasiri, D. Meedeniya and C. Perera, “Automated license plate recognition: A survey on methods and techniques,” IEEE Access, vol. 9, pp. 11203–11225, 2020. [Google Scholar]

43. Y. Jamtsho, P. Riyamongkol and R. Waranusast, “Real-time license plate detection for non-helmeted motorcyclists using YOLO,” ICT Express, vol. 7, no. 1, pp. 104–109, 2021. [Google Scholar]

44. E. Dong, Y. Zhu, Y. Ji and S. Du, “An improved convolution neural network for object detection using YOLOv2,” in IEEE Int. Conf. on Mechatronics and Automation (ICMA), Changchun, China, pp. 1184–1188, 2018. [Google Scholar]

45. G. S. R. K. Shira, P. Logesh, R. D. Praba, K. Kavitha and A. Kalaiselvi, “Traffic surveillance system using YOLO algorithm and machine learning,” in 2023 2nd Int. Conf. on Advancements in Electrical, Electronics, Communication, Computing and Automation (ICAECA), IEEE, pp. 1–6, 2023. [Google Scholar]

46. J. Salna, U. A. Reddy, R. C. Lal and R. Vinay, “Traffic rule violation recognition for two wheeler using YOLO algorithm,” in 2023 Second Int. Conf. on Electronics and Renewable Systems (ICEARS), IEEE, pp. 1477–1480, 2023. [Google Scholar]

47. V. Gnanaprakash, N. Kanthimathi and N. Saranya, “Automatic number plate recognition using deep learning,” IOP Conference Series: Materials Science and Engineering, vol. 1084, no. 1, pp. 12027, 2021. [Google Scholar]

48. T. Kumar, S. Gupta and D. S. Kushwaha, “An efficient approach for automatic number plate recognition for low-resolution images,” in Proc. of the Fifth Int. Conf. on Network, Communication and Computing, Kyoto, Japan, pp. 53–57, 2016. [Google Scholar]

49. H. Li, P. Wang and C. Shen, “Toward end-to-end car license plate detection and recognition with deep neural networks,” IEEE Transaction Intelligent Transportation System, vol. 20, no. 3, pp. 1126–1136, 2018. [Google Scholar]

50. C. Rung-Ching, “Automatic license plate recognition via sliding-window darknet-YOLO deep learning,” Image and Vision Computing, vol. 87, pp. 47–56, 2019. [Google Scholar]

51. L. Rayson, L. A. Zanlorensi, G. R. Gonçalves, E. Todt, W. R. Schwartz et al., “An efficient and layout-independent automatic license plate recognition system based on the YOLO detector,” IET Intelligent Transport Systems, vol. 15, no. 4, pp. 483–503, 2021. [Google Scholar]

52. Al-Qudah, Rabiah and C. Y. Suen, “Enhancing YOLO deep networks for the detection of license plates in complex scenes,” in Proc. of the Second Int. Conf. on Data Science, E-Learning and Information Systems, pp. 1–6, 2019. [Google Scholar]

53. R. Shashidhar, A. S. Manjunath, R. S. Kumar, M. Roopa and S. B. Puneeth, “Vehicle number plate detection and recognition using YOLO-V3 and OCR method,” in 2021 IEEE Int. Conf. on Mobile Networks and Wireless Communications (ICMNWC), IEEE, pp. 1–5, 2021. [Google Scholar]

54. Al-Batat Reda, A. Angelopoulou, S. Premkumar, J. Hemanth and E. Kapetanios, “An end-to-end automated license plate recognition system using YOLO based vehicle and license plate detection with vehicle classification,” Sensors, vol. 22, no. 23, pp. 9477, 2022. [Google Scholar] [PubMed]

55. J. Yonten, P. Riyamongkol and R. Waranusast, “Real-time license plate detection for non-helmeted motorcyclist using YOLO,” ICT Express, vol. 7, no. 1, pp. 104–109, 2021. [Google Scholar]

56. B. K. Cho, S. H. Ryu, D. R. Shin and J. I. Jung, “License plate extraction method for identification of vehicle violations at a railway level crossing,” International Journal of Automotion Technology, vol. 12, no. 2, pp. 281–289, 2011. [Google Scholar]

57. H. W. Lim and Y. H. Tay, “Detection of license plate characters in the natural scene with MSER and SIFT unigram classifier,” in IEEE Conf. on Sustainable Utilization and Development in Engineering and Technology, Kuala Lumpur, Malaysia, pp. 95–98, 2010. [Google Scholar]

58. X. He, M. Guo, N. Nedjah, J. Zhang and L. Peng, “Vehicle and pedestrian detection algorithm based on lightweight YOLOv3-promote and semi-precision acceleration,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10, pp. 19760–19771, 2022. [Google Scholar]

59. S. H. Chen, C. W. Wang, I. Tai, K. P. Weng and K. S. Hsieh, “Modified YOLOv4-DenseNet algorithm for detection of ventricular septal defects in ultrasound images,” International Journal of Interactive Multimedia & Artificial Intelligence, vol. 6, no. 7, pp. 101–108, 2021. [Google Scholar]

60. L. Rayson, E. V. Cardoso, D. R. Lucio, V. Estevam and D. Menotti, “On the cross-dataset generalization in license plate recognition,” arXiv preprint arXiv:2201.00267, 2022. [Google Scholar]

61. H. Abdelsalam, Y. K. Chan and V. C. Koo, “A new image enhancement and super resolution technique for license plate recognition,” Heliyon, vol. 7, no. 11, 2021. [Google Scholar]

62. I. Rashedul, Md R. Islam and K. H. Talukder, “An efficient method for extraction and recognition of bangla characters from vehicle license plates,” Multimedia Tools and Applications, vol. 79, pp. 20107–20132, 2020. [Google Scholar]