DOI:10.32604/iasc.2021.013705
| Intelligent Automation & Soft Computing DOI:10.32604/iasc.2021.013705 | |
| Article |
Tyre Inspection through Multi-State Convolutional Neural Networks
1BMIE, Avinashilingam Institute for Home Science and Higher Education for Women, Coimbatore, 641043, India
2Department of Computer Science and Engineering, Sahrdaya College of Engineering and Technology, Kerala, 680684, India
3Department of Computer Science and Engineering, Anna University Regional Campus, Tirunelveli, 627007, India
4School of Information Technology and Engineering, Vellore Institute of Technology, Vellore, 632014, India
5Department of Information Technology, National Engineering College, Kovilpatti, 628503, India
6Department of Mathematics and Computer Science, Faculty of Science, Beirut Arab University, 115020, Lebanon
7Department of Computer Science and Engineering, Soonchunhyang University, Asan, 336745, Korea
*Corresponding Author: Yunyoung Nam. Email: ynam@sch.ac.kr
Received: 18 August 2020; Accepted: 23 October 2020
Abstract: Road accident is a potential risk to the lives of both drivers and passers-by. Many road accidents occur due to the improper condition of the vehicle tyres after long term usage. Thus, tyres need to be inspected and analyzed while manufacturing to avoid serious road problems. However, tyre wear is a multifaceted happening. It normally needs the non-linearly on many limitations, like tyre formation and plan, vehicle category, conditions of the road. Yet, tyre wear has numerous profitable and environmental inferences particularly due to maintenance costs and traffic safety implications. Thus, the risk to calculate tyre wear is therefore of major importance to tyre producers, convoy owners and government. In this paper, we propose a Multi-state Convolution Neural Networks to analyze tyre tread patterns about wear and tear as well as tyre durability. The feature maps are identified from the input image through the Convolution functions that the sub-sampling utilizes for producing the output with the fully connected networks. The quadratic surface uses to perform the preprocessing of tyre images with several Convolutional layers. Through this work, we aim to reduce the economic implications as well as traffic safety implications which happen due to tyre wear. This will serve as a potential solution to tyre wear-related issues.
Keywords: Convolutional neural networks; machine learning; deep learning; tyre wear prediction
In recent days, AI together with deep learning and machine learning invites many researchers to focus on solving numerous numbers of real-world problems. There are multiple ways for implementing machine learning classification for which the algorithms and techniques used are reliant on the problem to be solved and the dataset used. It will take a long time to study and analyze all kinds of machine learning classifiers. Characteristically, it is expensive and period overshadowing to construct a classifier, since this need gathering training information within related methodologies. Choosing an appropriate machine learning method during widespread experiments is fully based on machine learning approaches. On the contrary, Deep learning is assumed to be an artificial intelligence function which is used to improve the performance of the human brain’s function using different learning techniques [1].
Road accident is a potential risk to the lives of both drivers and passers-by. Many road accidents occur due to the improper condition of the vehicle tyres after long term usage. Thus, tyres need to be inspected while manufacturing to avoid serious road problems. This learning spotlights on tire exertion as knowledgeable by illumination vehicles consisting of tyre-related events such as feasts or horizontal tyres, tire or wheel deficit, tire or wheel breakdown, and tire squalor. In recent days, many road accidents happen due to improper maintenance of tyre in our vehicles [2]. Sufficient tyre stride deepness on all tires of a vehicle is extremely necessary to take care of correct hold on the road beneath completely dissimilar road circumstances. In recent days, we can predict the tyre conditions using Machine Learning. The related works of tyre detection are mostly based on the concept of image patches with Gabor filters [3]. These kinds of filters did not identify the high variant image defect but detect well in some kind of standard way of object identification. Despite executing the deep learning methodology, the capability to identify the complex tyre pattern identification will increase the accuracy of the system. In this paper, we propose a methodology based on Convolution Neural Network (CNN) with 5 entities of multi-state CNN. MaxPooling training model is utilized to train the activation function in several training states. The contrast normalization methodology is implemented for preprocessing the tyre defect images so that the high amount of accuracy is maintained in the classification state. Multi-state Convolution neural network is proposed to implement the deep learning methodology concept to train the tyre-related images.
Traffic stream calculation with big data was demonstrated in Lv et al. [4], which focuses on deep learning methodology. Deep construction representation for traffic flow calculation has been discussed by the authors in whom Sparse AutoEncoder (SAE) model has been used to remove traffic flow characteristics. The authors applied a logistic deterioration layer for predicting traffic flow. The Training method was also designed to instruct the desired layers using supervised learning. Finally, BP method was used for fine-tuning the whole network. Experiment has been conducted which consists of directory of presentation to estimate the efficiency. Determination of the structure of a SAE model along with results was shown by the authors.
Deep Neural Network [5] has been discussed on detecting Transportation Modes. Basic concepts and handcrafted features were explained by the authors using a framework of the deep feature learning model. A framework of SAE was discussed by formulating TDF learning algorithm. Point level deep feature learning with trajectory level deep feature learning was defined. Detection methods and its analysis were also made and comparison using linear regression and support vector machine were done for THF, TDF and both. The new approach [6] described the reasons for tyre breakdown on Road Traffic disaster. The author has surveyed for collecting road-related data for which 450 questionnaires were set and given to commercial drivers. It is found that there were 4 major causes of tyre failure such as overinflating, under-inflation, wear and congestion of vehicles.
Human-like independent Vehicle swiftness [7] was designed to manage by Deep strengthening knowledge with dual Q-Learning. The authors discussed incorporated perception model to build the surroundings for an independent vehicle. Agent-environment interaction system was described with their proposed system which is implemented using double Q-learning approach [8]. Road information and GPS position information are used to illustrate the integrated perception approach. Two algorithms were proposed such as an algorithm for general DRL framework and algorithm for Double deep Q-learning based Vehicle swiftness regulator. In Zhang et al. [9], soft computing techniques were exploited to acquire spatial-temporal traffic data within real-time traffic. An innovative deep design to facilitate temporal and dynamical spatial architecture for traffic situation estimating was proposed which is followed by the requirement of supporting spatial and temporal data. A concentration system was introduced into the model to enumerate their connection for which obtained concentration load is used for daily travelling and pathway preparation [10].
Arbitrary Forest HTM Cortical learning method [11] was used for increasing deep learning routine. Average communication error is calculated to evaluate the result in which the performance increases when the error decreases. Firstly, the optimized algorithm is defined for which HTM boundaries are made to accumulate evolutions within spare disseminated illustrations [12]. The authors proposed a Dynamic Learning algorithm to test and predict the result in which the mean absolute percentage error is considered. The method in Bülbül et al. [13] analyzed the position of the accident occurrence and purpose of the risk of an accident by machine learning method. The authors used machine learning algorithms namely Ada_Boost, CART, C4.5, Naive Bayes, One_R (Regression) for the identification of the trouble, obtaining of information, information preprocessing and modelling. A generalization methodology in Gothane et al. [14] was used for preparing parameters, creation of related information, assessing the significance of feature and production of organization rules. The authors collected dataset on road accidents trained with conclusion logic contains 13 attributes which were implemented using WEKA tool and Attribute details were listed in the form of a table. Association model for Apriori algorithm was also used by the authors to conclude the results [15].
The elaborated forecast of the reason for accident and horizontal position on roads has been implemented using the latest methodologies [16]. Demonstrating the stages concerned in mining were discussed which is followed that represents the exertion connected to latest methods was used. Data mining of crash data was presented that includes gathering of road and accidental data, pre-processing of road and accidental data to prepare the required dataset. Smart Road Accident finding and transmission device [17] was utilized for which they concentrated on various reports of road accidents and traffic conditions. In this work, the accident was found by incorporating pulsation and gyroscope sensors where data are sent to the important numbers using GSM component beside with the position recognized by the GPS device. The classifications of road accidents using fuzzy techniques [18] were implemented for which Fuzzy Equivalence Relations are defined. Various definitions and prepositions are used to describe the algorithm [19]. The application of these definitions is made implementable in a fuzzy classification tree of road accidents. Besides these researches, there are some others which can be found in [20].
This section discusses the proposed method using deep learning. Sample tyre images were collected from tyre manufactures using Hi-Fi camera which includes both two-wheelers and four-wheelers, Fig. 1 depicts the diagrammatic representation of our determined method.
Figure 1: Diagrammatic representation of fetching the analysis report using multi-state CNN
Here, the user initially takes some sample tyre images using high-end camera which is given as the input for multi-state CNN. This multi-state Convolutional Neural Network, which is considered to be a category of a deep neural network, will analyze the visual images of various types of tyre wear. This multi-state CNN uses multilayer preceptors to process and predict the images which require minimal preprocessing. Durability prediction and type of wear detection are done using this deep neural network. The analysis report will be given to the user after prediction and detection are completed.
Fig. 2 show various steps involved in tyre wear prediction and detection. This process starts with collecting various vehicles tyre images which are extracted and given to Convolutional neural network. In deep learning, a Convolutional neural network (CNN, or ConvNet) is described to be a subset of deep, feed-forward artificial neural networks, which is used in recognizing apparent images. CNNs apply different multilayer Perceptron which is formulated to entail least preprocessing compared with many other preprocessing techniques. Multi-state CNNs need little preprocessing only when it is being differentiated with various image classification algorithms. This network is made to acquire the filters whereas traditional algorithms were hand-engineered which are considered to be a major advantage.
Figure 2: Tyre wear prediction-deployment
A Multi-state CNN comprises an arrangement of stacked Convolutional layers in which each layer exchanges one measure of activation to another utilizing differentiable capacity. Each layer is intended to have a lot of neurons which have its weights and bias. Every neuron is made to completely associate with all neurons in the previous layers in which neurons in a single layer work freely and would not share any associations. The last completely associated layer is said to be the “output layer” and that speaks to the class scores. In any case, Multi-state CNN architecture comprises of input and three principal kinds of layers: Convolutional Layer, Pooling Layer, and Fully-Connected Layer.
Fig. 3 depicts a typical multi-state CNN which delineates convolution, sub-sampling from the input layer to completely associated output layer. A convolution layer will encounter the output of neurons which are associated with the input by executing a dot product among their weights, and every neuron is required to become familiar with the usefulness through every emphasis through gradient descent based refinement of weights and bias. A multi-state CNN changes a unique picture layer by layer through these neurons from the first pixel esteems to the final class scores. Convolution layer and completely associated layer achieve changes utilizing any parameters that are engaged with every cycle of the deep learning process. Convolutional systems incorporate nearby or worldwide pooling layers, which join the output of neuron bunches at one layer to the following layer and vice versa. For instance, max pooling utilizes the most extreme value from every one of a group of neurons at the earlier layer and in normal pooling contains the normal value from every one of a bunch of neurons at the earlier layer.
Figure 3: Typical multi-state convolutional neural network
Fig. 4 demonstrates the quadratic surface for the machine learning approach.
Figure 4: Quadratic surface for multi-state convolutional neural networks
Dynamic programming might be incorporated with artificial neural systems which are connected to multi-dimensional nonlinear issues, for example, those engaged with vehicle routing and certain resource management. Works that come surprisingly close to reinforcement learning are viewed as certain control issues, games and other decision making tasks. Fig. 5 represents the proposed neural network architecture which consists of an input layer, Convolutional layer with the required number of times and fully connected output layer to perform preprocessing of tyre images using various vehicular nodes. The architecture of the proposed multi-state CNN functions is fully based on AlexNet which is considered to be an advanced version of CNN.
Figure 5: Architecture of proposed multi-state convolutional neural network
AlexNet is being utilized here to improve the execution, which was inculcated statistically costly, yet made achievable because of the usage of GPUs during training. AlexNet is intended to carry eight layers in which the initial five were Convolutional layers. Some of them pursued by max-pooling layers, and the last three were completely connected layers. Non-saturating ReLU initiation function is used in this AlexNet, to produce high training performance over other existing methods. The final layer is deliberated to be a fully connected output layer which produces the high performance and AlexNet architecture is shown in Fig. 6.
Figure 6: Typical architecture of multi-state convolutional neural network
There are two GPUs used in typical Multi-state Convolutional Neural Network architecture in which the first GPU that works on the layer-parts on the other hand, the second one runs the layer-parts at the downside. The GPUs are designed to interconnect only at ascertained layers in which the input of the network is 150,528-dimensional. The kernels of the second, fourth, and fifth Convolutional layers are associated only to those kernels that are portrayed with the previous layer on the same GPU. The third Convolutional layer kernel is incorporated into all kernels that are mapped in the second layer. Fully-connected layers consist of a number of neurons that are attached to all neurons in the previous layer. Fig. 7 demonstrates the diagonal Convolutional neural network model.
Figure 7: Diagonal multi-state CNN
This Detection phase contains a Convolutional neural network as described in the typical architecture of Multi-state CNN. Multi-state CNN was created based on the inspiration from AlexNet. This multi-state CNN finds whether the photo contains a tyre or not. If it does not contain a tyre of appropriate orientation, the module will reject the photo and prompt the user for a new one. Durability prediction multi-state CNN phase predicts the durability of the tyre from the photo. Type of wear detection multi-state CNN phase predicts the class of the tyre from the photo. Through this information, the user can know the cause of tyre wear which is implemented by a multi-state CNN. Analysis report phase combines the output from two previous modules. The transformation parameter is the computer for the proposed work using Eq. (1):
where 
is the transformation parameter value, 
is the output value and P is the pixel value of the input image. For obtaining the value of Gray for the input image, Eq. (2) is used:
where 
demonstrates the value of Gray in the initial state, 
demonstrates the value of Gray after the transformation state, 
is the middle value, 
is the edge value of the input image. The Gaussian filter is used to normalize the contrast value and is computed using Eq. (3).
where 
for the Gaussian filter, 
is the constant value for computing the contrast value, P is the coefficient value. The pixel value for doing the classification is computed using Eq. (4)
where 
is the classification function for obtaining the pixel value, the 5 × 5 value is used for classification.
Here, we implemented the algorithm on OpenCV. Using this method, around 50 tyre images are captured using the camera and given as the input for multi-state CNN. Fig. 8 shows the output of multi-state CNN which shows 100% training accuracy.
Figure 8: Tyre wear prediction–loss curves using multi-state CNN
Fig. 9 shows the output of multi-state CNN which shows 95% test accuracy.
Figure 9: Tyre wear prediction–accuracy curves using multi-state CNN
Fig. 10 demonstrates the testing and the validation of the accuracy of our proposed method whenever the batch size is increased then the inclusion get dropout. Even though it is a minimum amount of modification among the related models, it needs some time to train and evaluating for the accuracy model. The proposed method has the best accuracy compared to the related methods. Therefore, it is constructed to identify the tyre detection using classification with deep learning methods.
Figure 10: Accuracy
The multi-state CNN model has 4 types of channels like 1-net, 3-net, 5-net and 9-net. The accuracy percentage of the proposed work with 9-set is the biggest value of 99.31%, 5-set has the value of 99.18%, 3-set has the value of 98.79% and 1-set has the value of 97.91%, the comparison result is illustrated in Fig. 11.
Figure 11: Accuracy for different type of channels
The variable-sized resolution images range from 50 × 50 pixels to 200 × 200 pixels despite the uncertainty of the tyre defect parameters in the production time. Every dataset is analysed using the rotation, scaling and transformation. The proposed work determines every training image to be of the same size, such that every dataset has the common resolution value of 127 × 127. AlexNet is the initial channel model for our proposed work, the framework of AlexNet is having 12 layers. The output layer consists of 6 neurons according to 6 tyre defects from the dataset. Despite training, the initial filter values are arbitrarily initialized with the distribution range of [−0.05, +0.05]. The ReLU is utilized for the activation function to analyze the features of shape as the input function. The effect of the image classification results based on tyre detection is illustrated in Fig. 12.
Figure 12: Analysis of tyre defects from the dataset
Fig. 13 demonstrates the Recognition Rate for various classification models using Convolutional neural network model.
Figure 13: Recognition rate
Fig. 14 illustrates the Overall accuracy for the classification models while performing the Scaling, rotation, translation and the combinations of both.
Figure 14: Overall accuracy
In this paper, we aim to predict and detect tyre wear which is considered to be a major thread in most of the road accidents that are happening nowadays using multi-state Convolutional neural networks. Tyre tread pattern has been analyzed using various layers of multi-state CNN and predicts the nature of the tyre. In this work, Multi-state CNN was used for processing the tyre images which is done very effectively. There are four major predictions such as Tyre should be replaced immediately; Tyre would be replaced within a month; Tyre could be replaced within three months and No need for Tyre replacement after preprocessing. In the future, it is planned to upload the tyre images in cloud storage for any researcher to perform preprocessing. A mobile app should be made available for the tyre manufacturer and they would provide this service for their customers to give an alert message to change the tyre in time. It will provide an easy and convenient method to assess the condition of the tyre of their vehicle.
Funding Statement: This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ICAN (ICT Challenge and Advanced Network of HRD) program (IITP-2020-0-01832) supervised by the IITP (Institute of Information & Communications Technology Planning & Evaluation) and the Soonchunhyang University Research Fund.
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
1. M. Bianchini and F. Scarselli. (2014). “On the complexity of neural network classifiers: A comparison between shallow and deep architectures,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 8, pp. 1553–1565. [Google Scholar]
2. W. Ren, S. Han, T. Fwa, J. Zhang and Z. He. (2019). “A new laboratory test method for tire-pavement noise,” Measurement, vol. 145, pp. 137–143. [Google Scholar]
3. G. H. Hu. (2015). “Automated defect detection in textured surfaces using optimal elliptical gabor filters,” Optik, vol. 126, no. 14, pp. 1331–1340. [Google Scholar]
4. Y. Lv, Y. Duan, W. Kang, Z. Li and F. Wang. (2015). “Traffic flow prediction with big data: A deep learning approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 2, pp. 865–873. [Google Scholar]
5. H. Wang, G. Liu, J. Duan and L. Zhang. (2017). “Detecting transportation modes using deep neural network,” IEICE Transactions on Information and Systems, vol. 100, no. 5, pp. 1132–1135. [Google Scholar]
6. I. Edunyah. (2016). “Causes of tyre failure on road traffic accident; A case study of takoradi township,” International Journal of Scientific and Research, vol. 6, no. 2, pp. 30–35. [Google Scholar]
7. Y. Zhang, P. Sun, Y. Yin, L. Lin and X. Wang. (2018). “Human-like autonomous vehicle speed control by deep reinforcement learning with double q-learning,” IEEE Intelligent Vehicles Symposium, Changshu, China. [Google Scholar]
8. Y. Wei, Q. Yuan, H. Shen and L. Zhang. (2017). “Boosting the accuracy of multispectral image pansharpening by learning a deep residual network,” IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 10, pp. 1795–1799. [Google Scholar]
9. Y. Zhang, T. Cheng, Y. Ren and K. Xie. (2020). “A novel residual graph convolution deep learning model for short-term network-based traffic forecasting,” International Journal of Geographical Information Science, vol. 34, no. 5, pp. 969–995. [Google Scholar]
10. L. L. Minku, A. P. White and X. Yao. (2010). “The impact of diversity on online ensemble learning in the presence of concept drift,” IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 5, pp. 730–742. [Google Scholar]
11. M. A. Abbas. (2018). “Improving deep learning performance using random forest htm cortical learning algorithm,” in First International Workshop on Deep and Representation Learning (IWDRL), Proc. of the IEEE, Cairo, Egypt, pp. 13–18. [Google Scholar]
12. V. Sze, Y. Chen, T. Yang and J. S. Emer. (2017). “Efficient processing of deep neural networks: A tutorial and survey,” in Proc. of the IEEE, vol. 105, no. 12, pp. 2295–2329. [Google Scholar]
13. H.İ. Bülbül, T. Kaya and Y. Tulgar. (2016). “Analysis for status of the road accident occurance and determination of the risk of accident by machine learning in istanbul,” IEEE Int. Conf. on Machine Learning and Applications (ICMLA), Anaheim, CA, USA, pp. 426–430. [Google Scholar]
14. S. Gothane and M. V. Sarode. (2016). “Analyzing factors, construction of dataset, estimating importance of factor, and generation of association rules for indian road accident,” in Int. Conf. on Advanced Computing (IACC), Bhimavaram, India, pp. 15–18. [Google Scholar]
15. D. Miyashita, S. Kousai, T. Suzuki and J. Deguchi. (2017). “A neuromorphic chip optimized for deep learning and cmos technology with time-domain analog and digital mixed-signal processing,” IEEE Journal of Solid-State Circuits, vol. 52, no. 10, pp. 2679–2689. [Google Scholar]
16. G. Kaur and H. Kaur. (2017). “Prediction of the cause of accident and accident prone location on roads using data mining techniques,” in Int. Conf. on Computing, Communication and Networking Technologies (ICCCNT), Delhi, India, pp. 1–7. [Google Scholar]
17. N. R. Vatti, P. L. Vatti, R. Vatti and C. Garde. (2018). “Smart road accident detection and communication system,” in Int. Conf. on Current Trends towards Converging Technologies (ICCTCT), Coimbatore, India, pp. 1–4. [Google Scholar]
18. A. Katsoukis, A. Konguetsof, L. Iliadis and B. Papadopoulos. (2018). “Classification of road accidents using fuzzy techniques,” Innovations in Intelligent Systems and Applications (INISTA), Thessaloniki, Greece, pp. 1–5. [Google Scholar]
19. Z. Luo, L. Liu, J. Yin, Y. Li and Z. Wu. (2017). “Deep learning of graphs with ngram convolutional neural networks,” IEEE Transactions on Knowledge and Data Engineering, vol. 29, no. 10, pp. 2125–2139. [Google Scholar]
20. J. Ejsmont and W. Owczarzak. (2019). “Engineering method of tire rolling resistance evaluation,” Measurement, vol. 145, pp. 144–149. [Google Scholar]
 | This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |