DOI:10.32604/cmc.2022.025025
| Computers, Materials & Continua DOI:10.32604/cmc.2022.025025 | |
| Article |
IoT and Blockchain-Based Mask Surveillance System for COVID-19 Prevention Using Deep Learning
1Department of Computer Science and Engineering, Mawlana Bhashani Science and Technology University, Tangail, Bangladesh
2Department of Computer Science, Najran University, Najran, 55461, Saudi Arabia
*Corresponding Author: Naif Al Mudawi. Email: naalmudawi@nu.edu.sa
Received: 08 November 2021; Accepted: 18 January 2022
Abstract: On the edge of the worldwide public health crisis, the COVID-19 disease has become a serious headache for its destructive nature on humanity worldwide. Wearing a facial mask can be an effective possible solution to mitigate the spreading of the virus and reduce the death rate. Thus, wearing a face mask in public places such as shopping malls, hotels, restaurants, homes, and offices needs to be enforced. This research work comes up with a solution of mask surveillance system utilizing the mechanism of modern computations like Deep Learning (DL), Internet of things (IoT), and Blockchain. The absence or displacement of the mask will be identified with a raspberry pi, a camera module, and the operations of DL and Machine Learning (ML). The detected information will be sent to the cloud server with the mechanism of IoT for real-time data monitoring. The proposed model also includes a Blockchain-based architecture to secure the transactions of mask detection and create efficient data security, monitoring, and storage from intruders. This research further includes an IoT-based mask detection scheme with signal bulbs, alarms, and notifications in the smartphone. To find the efficacy of the proposed method, a set of experiments has been enumerated and interpreted. This research work finds the highest accuracy of 99.95% in the detection and classification of facial masks. Some related experiments with IoT and Block-chain-based integration have also been performed and calculated the corresponding experimental data accordingly. A System Usability Scale (SUS) has been accomplished to check the satisfaction level of use and found the SUS score of 77%. Further, a comparison among existing solutions on three emergent technologies is included to track the significance of the proposed scheme. However, the proposed system can be an efficient mask surveillance system for COVID-19 and workable in real-time mask detection and classification.
Keywords: Internet of things (IoT); blockchain; deep learning; feature reduction; surveillance system
The Severe Acute Respiratory Syndrome Coronavirus 2(SARS-CoV-2) is the newly arisen infectious disease that originated in Wuhan, China, in December 2019 [1]. Around 170 people are reported as dead, and 7, 711 people were infected by the novel coronavirus in China before declared as a global pandemic. The rapidly spreading of the COVID-19 [2] disease has vastly impacted every aspect of society. Recently, the economic and health sectors have been paralyzed by the massive transmitting of the virus. World Health Organization (WHO) reported that as of July 9, 2021, at this moment in time, around 186,596,118 people had been affected, and over 4,030,961 people have been dead worldwide. Recent evidence suggests that the virus transmits mainly between people in close interaction, where the distance between them is less than 1 m [3]. Respiratory droplets or aerosols containing the virus contaminate people who are in close contact. The virus is inhaled directly into interaction with the nose, mouth, and eyes. There is no antiviral medicine to prevent this disease. Nowadays, clinically approved vaccines can't prevent infections too. Society has encountered complications for human health, the environment, and economic losses. To prevent infection is the key way to protect from the disease, WHO recommended that people should maintain a social distance of at least 2 m [2] and wear face masks appropriately to avoid virus transmission. The world has announced no masks, no service policy [4]. Many service providers also ensure face masks beforehand to enter their premises. Protective masks lower the probability of viruses transmitting to the human respiratory system through infected people's droplets. According to report analysis, if 50% of people wear face masks, only 50% of the population would be attacked by the virus [5]. One of the challenges to using masks in society is the people's health condition and the limit of authorities’ personnel which reflect less monitoring of masks usage. So, there are several methods that some people wear a face mask in different covering positions. The study debated and found that the COVID-19 has increased spread thrown droplets or through the air for more than three hours active. Once 80% of the people wear face masks, the outbreak can be stopped noticeably. However, people would become careless and complacent to wear a face mask, and sometimes they do not wear masks properly, not covering both nose and mouth. Therefore, on the one hand, the detection of face masks is so important, identification of face mask displacement is more significant on the other hand to avoid virus transmission to others. Many of the papers proposed face mask detection approaches. But, this paper presents an approach to prevent the coronavirus from spreading by detecting mask displacement in public areas.
This research adopts a combination of Deep Learning (DL) with Machine Learning (ML) [6,7] Internet of things (IoT) and Blockchain-based system to detect the absence and displacement of a mask to make a way of prevention of COVID-19 especially in the shopping mall, hotels, restaurants, homes, and offices. The DL-based architecture has been utilized by Convolutional Neural Network (CNN). Thus, the contributions of this research work are described as follows:
• The real-time video surveillance will monitor the individuals in public places and detect whether a person is wearing a face mask appropriately or not by using DL with CNN. This operation will be performed with a raspberry and camera module and stored the detected information to work with IoT and Blockchain-based operations.
• After detecting the absence or displacement of the facial mask, the system will transmit the spotted information to the IoT cloud for real-time data monitoring, and analog operations like alarm and notification will occur.
• To ensure an efficient transaction of detected results from IoT devices end to Cloud server, a Blockchain and Software-Defined Networking (SDN) based architecture will optimize the security, safety, and integrity of the proposed solution.
The manuscript is organized into five interconnected sections. In Section 2, the background of the previous study is presented. Section 3 provides the overall proposed methodology and working principles. Section 4 depicts the results of our proposed solution along with relevant discussion. Finally, Section 5 illustrates the conclusion of this manuscript with drawbacks and future works.
This section will discuss the previous study related to our proposed integration with their research gaps. The section is identically classified into two associated parts. First, the related work in mask detection with deep learning and the Internet of Things (IoT) will be presented. The second part will illustrate the related study of COVID-19 and Blockchain. Many great contributors have placed a significant trace in the field of COVID-19 and mask detection with deep learning, IoT, and Blockchain.
2.1 Mask Detection with Deep Learning Along with IoT
The authors of the paper [8] approached a system to monitor people in public whether they are wearing face masks or not. MobileNet and OpenCV were employed as the backbone of their real-time automatic face mask identification system, with Tensorflow and Keras used to train the model. After detecting any person not wearing a mask, the system will capture the person's image and send it to that person and the higher authorities. For advanced detection of face masks, the article [9] discussed using two object detection models, YOLOv3 and faster R-CNN. The authors compared the performance of both the models; i.e., the average precision of YOLOv3 was 55, and Faster R-CNN was 62, whether the inference time was 0.045 s and 0.15 s, respectively. The authors concluded by stating that both models are reasonably accurate. However, it would be prudent to utilize the model with the YOLOv3 algorithm when applying these models to real-world surveillance cameras because it does single-shot detection and has a considerably shorter inference time than Faster-RCNN.
The research [10] developed an IoT-based multi-features Smart Screening and Disinfection Walkthrough Gate (SSDWG) to screen a large number of people at once rapidly. The SSDWG included two primary modules: the first was for disinfecting and measuring the temperature of humans in a non-contact environment using the MLX90614, and the second was for detecting the presence of a person and activating the disinfection spray system using ultrasonic and motion sensors. Using VGG-16, Mo-bileNetV2, Inception v3, ResNet-50, and CNN, the second module used a transfer learning strategy to detect face masks in real-time scenarios and classify persons who wore them properly improperly, or not at all. Using the VGG-16, MobileNetV2, Inception V3, ResNet-50, and CNN, the mask detection and classification module achieved accuracies of 99.81%, 99.6%, 99.46%, 99.22%, and 99.07% for face mask detection and classification in three classes, respectively (FWPM, FWIPM, and FWOM). The authors also classed face masks into two groups (N-95 and Surgical) and used the VGG-16 and MobileNetV2 models to obtain 98.17% and 97.37% accuracy, correspondingly.
The paper [11] had utilized deep, and traditional machine learning suggested a hybrid model for face mask detection. Resnet50 was used for feature extraction, while the authors used decision trees, Support Vector Machine (SVM), and ensemble algorithm for the classification process. The Three face mask datasets which had been selected were the Real-World Masked Face Dataset (RMFD), the Simulated Masked Face Dataset (SMFD), and the Labeled Faces in the Wild (LFW). In RMFD, the SVM classifier achieved 99.64 percent testing accuracy, 99.49 percent in SMFD, and 100 percent in LFW. In this pandemic condition, several image classification and object identification techniques have emerged to achieve great accuracy in the detection of face masks.
The authors of the paper [12] approached a model named SSDMNV2, which used Single Shot Multibox Detector as a face detector and MobilenetV2 architecture as a framework for the classifier to perform real-time mask detection. The proposed model was created with OpenCV Deep Neural Network (DNN), Tensorflow, and Keras, and yielded positive results. This model had an F1 score of 0.93 and an accuracy score of 0.9264. An AI-based smart device consisted of Raspberry Pi and AI model with a camera proposed in the paper [13] to detect whether a person is wearing a mask or not and send an alert message via an integrated mobile app. If no one is present at home, this mobile app identifies if anyone enters the house. The authors used the CNN model with Tensorflow to identify face masks.
The paper [14] presented a new application that classifies people based on their mask-wearing using an autonomous Drone. The data were trained using TensorFlow, OpenCV, and Keras, and the classification was based on the MobileNetV2 architecture. The IoT-controlled autonomous Drone is in charge of closely monitoring and facilitating the movement of people from afar. The authors of the paper [15] used two deep CNN architectures and one CNN-based MobilenetV2 architecture to detect face masks, including Max pooling and Average pooling. Max pooling has a training accuracy of 96.49% and a validation accuracy of 98.67%. On the other hand, average pooling obtained 95.19% training accuracy and 96.23% validation accuracy. The MobileNetV2 architecture has the highest training and validation accuracy, 99.72% and 99.82%, respectively.
Tab. 1 shows the corresponding summary of the literature review. In these papers, the authors only focused on mask detection with two classes: mask & non-mask with deep learning mechanisms. Some authors had adopted IoT besides deep learning to detect and classify masks. But no competitive solution has yet been proposed with the highest accuracy of detection with raspberry pi and classification in three classes: mask, non-mask, and mask displacement or mismatch. Also, a combination of three emergent technologies like Deep Learning, Internet of things (IoT), and Block-chain has not been adopted in a mask surveillance system for COVID-19 prevention.
In paper [16], Sharma et al. created a Blockchain-based solution to deal with the global pandemic. The authors had also discovered and explained nine crucial applications of Blockchain in combating the COVID-19 pandemic threat.
In the paper [17], some creative approaches are being used to tackle the issues brought on by the COVID-19 situation by applying Blockchain. The author offered real-world examples of how Blockchain can be used in the supply chain, finance, healthcare, identifying people, securing credentials, tracing connections, identity management, and data analysis.
The fundamental problems that evolved during the COVID-19 pandemic are discussed in the first part of this paper [18]. After that, the authors investigated whether or not Blockchain can be an essential emergent technology by examining future use cases. Additionally, the authors outline how Blockchain can be applied for each use case and overview how the technology will function.
Xu et al. in paper [19] proposed a Blockchain-based privacy-preserving contact tracing technique for COVID-19 known as BeepTrace. In this work, the authors had revealed that the users need to be isolated from their personal information like home address when using the platform, utilizing Blockchain bridging. The authors also claimed their method had proved improved security and privacy and the advantages of being battery friendly and available to everyone on the Internet.
An exploratory framework has been developed in paper [20] by employing Blockchain as a structure and applying several peer-to-peer, time stamping, and de-centralized storage advantages of Blockchain to examine the feasibility of building a new system to identify and validate unknown incidences of COVID-19 virus. The framework's design will also allow citizens to forecast the risk of COVID-19 disease mainly in the urban areas via a unique P2P-Mobile Application. The authors also claimed that the plan would result in an effective system capable of supporting governments, public health organizations, and people in the decision-making process concerning the detection, prediction, and prevention of infection of COVID-19 disease.
In the study of these, the authors had only adopted the Blockchain technology for privacy-preserving, secured tracing connections, and identity management for the people stuck in the home during the COVID-19 pandemic. But no architectural model of Blockchain has been proposed to work with the mask surveillance system.
3 Methodology and Working Principle
This methodology of the proposed model is classified into four interconnected phases. In the first phase, the system will capture the face of the individuals. For this purpose, a camera module will capture the face with accurate focusing. After that, the captured information will send to the raspberry pi. The scheme will perform detection and live to monitor in the second phase. In this scheme, images will be taken from the camera module, and then the deep learning-based automatic operation is applied to classify facemask status. The detected information will be sent to the cloud for real-time monitoring if the internet connection is available.
In the third phase, a Blockchain-based intelligent monitoring system has been presented to track the secure transaction of detected information from raspberry pi to the IoT could server. Finally, in the fourth phase, an analog operation is performed at the last stage to ensure a complete IoT-enabled system. The operation includes a wireless connection such as Bluetooth with another microcontroller to provide analog notifications to the corresponding users. The phase will also provide an alarm and switch on a signal bulb after detecting the information from 2nd phase. The system will further provide notification and vibration in the smartphone via Bluetooth. Fig. 1 shows the corresponding overall system illustration.
The proposed system utilized a deep learning pipeline to automatically detect masks from the captured images in real-time. Fig. 2 depicts the required pipeline for this purpose.
According to Fig. 2, a training dataset is utilized to enlighten the formation of the mask displacement technique. The training dataset holds three types of images of human faces- (1) Wearing no mask, (2) Wearing the mask, and (3) wearing the mask in the wrong way. Each image of the dataset is resized into 224 × 224 × 3. A pre-trained ResNet-50 CNN [21] model is used by excluding the classifier part to extract the features from each image. In ResNet-50, there exist a total of 50-layer and it consists of 26 million parameters. ResNet-50 joins the input of the Nth layer to the (N + n)th layer; thus, it constructs a deep architecture that uses skip connection to generate information over layers. Fig. 3 presents the general architecture of ResNet-50; like CNN, it consists of convolutional, pooling, fully connected, and other layers associated with a deep network. From ResNet-50, we have gained 2048 features for each image.
Figure 1: Overall system illustration
Figure 2: Automatic mask surveillance detection pipeline from deep learning
Figure 3: Structure of ResNet-50
Including one class value with features, the feature vector has a total of 2049 values for each image. The features of the feature vector are optimized by using two techniques, namely Principal component analysis (PCA) and Linear discriminant analysis (LDA). PCA reserves correlated features by eliminating uncorrelated features. To eliminate uncorrelated features, PCA gets the covariance between features by using the following equation [22]:
LDA is a supervised linear dimensionality reduction technique that finds feature subspace and optimizes class separability. It performs separation by computing linear discriminants that enhance multiple access separability. Mathematically for each class, we can calculate the within-class variance as follows [23]:
To build the ML model, this research uses several classifiers, namely Logistic regression (LR) [24], Support vector machine (SVM) [25], and Random Forest (RF) [26].
To classify data points, LR first sets a weight matrix by a random initialization, and then each feature is multiplied with the weight using the Eq. (3). After finding ‘a’ is passed to Eq. (4), and it is followed for finding cost using (5) for the purpose of iteration, during which weight is updated by using Eqs. (6) and (7). Thus, after completing the training, the model can test new data to predict the unknown classes.
SVM turns lower-dimensional data to higher dimensions using a certain kernel function K (x, x’) and then, using complex iterative approaches, separates the data through a certain hyperplane for classification. Eq. (8) shows the general form of a hyperplane, and Eq. (9) presents the sigmoid function used as SVM kernel in this research.
RF is a Decision tree (DT) classifier-based bagging ensemble model. Formally predictor RF is defined by Eq. (10) which indicates RF formed by a group of DT base trees. In Eq. (10) Θi is the output associated with random variable Θ
Each of the classifiers described above is examined with three feature sets- Original feature set, PCA optimized feature set, and LDA optimized feature set. Based on all analyses, we have observed that the performance of PCA with LR outperforms all processes. Hence LR with PCA is used to build the main model in this research. The selection process of the best model is summarized in Fig. 4.
It has to be noted that for LDA and PCA, we have stored a total of 50 components during feature reduction. The utilization of the feature reduction process during model selection is shown in Fig. 5.
3.1 Safe Monitoring of Cloud Data with Blockchain
This section illustrates how Blockchain technology will secure the transaction of data from the sender end devices to the end-users. In Fig. 6, the proposed architecture of Blockchain has been depicted. In this figure, the proposed model is enriched with five essential elements of Blockchain. Firstly, authentication is responsible for the data authentication retrieved from the IoT devices such as raspberry pi. Secondly, immutability represents the unchanged ability of the data retired from the sender station of the IoT devices [27]. Thus, the intruders will not be able to break down the avail pattern of the data. Thirdly, decentralization & distribution are some of the most outstanding features of Blockchain. No transaction has yet occurred in a centralized form with these features, such as individuals, groups, or organizations, especially in IoT-based data transactions. These will also ensure data transfer control and make a transaction in a distributed manner. Finally, tokenization includes some form of entity that can be stored and removed to confirm the transaction from the sender station [28,29]. To work with data storage and management, this research also places a way of database management system in the cloud for interactively real-time data monitoring and storage.
Figure 4: Model selection approach of the mask displacement system
Figure 5: Mechanism of feature reduction for this system
Figure 6: Proposed blockchain-enabled cloud security system
Fig. 7 shows a block diagram of the layered based architecture of the proposed Blockchain-based scheme [30]. In this figure, the proposed model will utilize a Soft-ware-Defined Networking (SDN) [31] based switch system to work with multiple IoT devices. Suppose a large number of detected information is available in the perception layer. In that case, the system will use an SDN switch to immediately create a gateway with the respective servers in the Edge layer. In this layer, the Blockchain-based operations have been carried to secure available transactions and control the ambiguous transmission of data. After the proper operations and the utilization, the processed data is transmitted to the cloud layer for real-time monitoring at the user end devices and ensuring a proper mask surveillance system.
Figure 7: The layered based architecture of the proposed blockchain system
3.2 IoT Based Operation on Mask Displacement
This section presents how the mask displacement will be performed with IoT, besides deep learning techniques. Fig. 8 shows the corresponding diagram of the proposed mask monitoring system with IoT. In this figure, the camera module is responsible for scanning the facial mask in a public place. The camera module is connected with rasp-berry pi as a microprocessor. When the system finds the decision on the facial mask from the raspberry, the scheme immediately sends the command to another micro-controller attached with raspberry pi to create some analog response like generic alarms and smartphone notifications via a Bluetooth connection. The model also includes three bulbs to produce light signals. In this way, the green bulb will be switched ON if the scheme finds the facial mask in the appropriate position, the blue blub will be ON for mask displacement or mismatch, and the red bulb will be ON if the system cannot detect any mask in the face. These operations will only happen if a Bluetooth connection is available between the raspberry and micro-controller. Again, the model includes a way of real-time data monitoring and storage with the cloud via a reliable internet connection. In case of internet connection failure, no data is transmitted from the raspberry pi to the cloud server.
This section will present the current research work results and relevant discussion. Initially, the section provides the experimental data analysis with deep learning for mask detection and classification. Then the section depicts the experiments with Raspberry Pi and camera module along with analog configuration. After that, the section will include a System Usability Scale (SUS) on some randomly selected individuals about our proposed scheme and interpret the calculated data accordingly. At last, the section will give a comparison among existing solutions with our proposed system.
Figure 8: A block diagram of IoT-based mask monitoring
4.1 Experimental Data Analysis with Deep Learning
There exist three classes in our dataset, namely- mask, no-mask, and mismatch, where mask means a person wearing a mask in the right way, No-mask means a person wearing no mask, and Mismatch means a person wearing the mask in the wrong way. For the first two classes- mask and no-mask we have adopted the images from [32]. The no-mask class in [32] holds 123 number mismatch images; hence, we extracted these images and added them to the mismatch class. To fulfill the mismatch class, we collected our images from various sources; thus, the 123 images mismatch class has 3563 images. Therefore, the final dataset has a total of 10885 images. Among them, 3599, 3723, and 3563 images belong to mask, no-mask, and mismatch class, respectively. To build up and examine our system, we partitioned the dataset into two fixed halves, the first 80% for the training stage and the rest 20% for the testing stage.
The confusion matrix (CM) presents the capability of any classification techniques by representing the right and wrong detection by the system [33]. Tab. 2 shows the CM of our Mask displacement detection system.
Tab. 3 presents the CM as well as the overall accuracy of this scheme when we have performed feature reduction using PCA. From Tab. 3, it is noticed that the peak accuracy of the system with PCA is obtained when we have used LR for classification.
Tab. 4 presents the CM as well as the overall accuracy of this scheme when we have performed feature reduction using LDA. From Tab. 4, it is noticed that the peak accuracy of the system with LDA is obtained when we have used LR for classification.
Tab. 5 presents the CM and the overall accuracy of this scheme when we have classification without feature reduction. From Tab. 5, it is noticed that the peak accuracy of the system without feature reduction is obtained when we have used LR for classification.
Fig. 9 shows the comparison of the misdetection rate of different classifiers for several methods used in our scheme. By analyzing this figure, it is noticed that the lowest misdetection rate of the system is obtained by using PCA with LR and which is 0.05%, and this is the actual outcome of this system. Therefore, the highest accuracy of our scheme is 99.95%.
Figure 9: Performance of different classifiers concerning various feature processing approaches
The maximal accuracy of this research, along with LDA and PCA, is gained by using the LR classifier. In this research, we have utilized 50 components for each feature re-duction approach. Fig. 10 shows the performance of PCA and LDA concerning the different numbers of components by using the LR classifier.
Figure 10: Performance of feature reduction techniques concerning various components
This research has also analyzed two other CNN techniques - AlexNet and Inception v3 on our mentioned dataset. Fig. 11 shows the comparison between these two models and the proposed approach for the model of achieving the best accuracy.
Figure 11: Performance of different CNN techniques on the mentioned dataset
4.2 Experiments with IoT Based Integration
This work has accomplished some experiments with IoT-based embedded systems with the raspberry pi and camera module. The outcomes of the experiment were analog. To perform this experiment, ten samples’ data have been calculated for mask detection and classification with IoT-based integration. In this experiment, Bluetooth and internet connection have played a significant role in creating an analog response and cloud data transmission. This experiment finds the average delay time of 1.4 s to transfer data from raspberry to IoT cloud server after mask detection and classification. Tab. 6 shows the experimental data with IoT-based models, and Fig. 12 illustrates a line chart diagram of cloud data transmission delay. This figure clearly remarks that the unavailability of the internet connection results in null transmission of data from raspberry pi to IoT cloud server.
Figure 12: Line chart of cloud data transmission delay
4.3 Experimental Data with Blockchain
Due to set up the environment to work with Blockchain-based integration. This model has occupied Emulator Mininet-Wifi, Ganache, and Visual Studio Code to build the required environment for the simulation. This research also used Ubuntu (Linux) and Raspbian as Operating System (OS) with Core (TM)-i7 processor, 2.20 GHz CPU, 8GB RAM 1 SSD ROM, and other external memory. The experimental setup for Blockchain has been accomplished with Solidity language along with JavaScript. In the initial experiment, the model has created smart contracts for transmitting data from one block to another. While initializing the transaction, the proposed has hashed the current address to transform a smart contract with the amount of gas consumption from the local Ethereum. Ethereum can control the number of transactions in the cloud with the corresponding smart contracts. Tab. 7 shows the related experimental data analysis with Blockchain from an address of an IoT device such as ‘0x7b39D5FBCbAD9f3A1230245c0c671Fc5348795eD’. Again, Fig. 13 shows the amount Gas consumption per transaction in Ethereum based Blockchain system for IoT Data transactions.
Figure 13: Mined vs. amount of gas consumption for blockchain transaction
4.4 System Usability Scale (SUS)
This research work has performed a System Usability Scale (SUS) on twelve randomly selected individuals about this proposed orientation. To accomplish this experiment, the proposed model has been provided and requested to comment on their experience for testing and prediction. SUS score shows that around 77% of persons marked the proposed solution reliable for interactive mask detection and classification. Approximately 17% did not give any constructive feedback, and the rest of the 16% people doubted our proposed solution. The corresponding results of the experiment have been interpreted in Fig. 14.
4.5 Comparison among the Existing Solutions
Tab. 8 presents the comparison among existing methods and proposed methods based on several parameters. From the investigation of our related work section, we have taken the most compatible works for comparison. From Tab. 8, it is noticeable that the proposed architecture holds several novels and improved techniques in this research field.
Figure 14: Pie chart of system usability scale (SUS)
From the analysis of Tab. 8, we have found that most authors try to establish their model to capture whether a person can wear a mask or not. Still, the authors of the paper [10] have also tried to detect whether a person wears the mask in the wrong way, which we have also tried in our work. Hence Fig. 15 shows the comparison between the paper Hussain et al. and the proposed scheme.
Figure 15: Comparison among proposed scheme and paper [10]
SARS-CoV-2 (Severe Acute Respiratory Syndrome Coronavirus 2) was discovered in China at the end of 2019 and has since spread over the world. Every day the death rate touches its highest peak, and the socio-economical balance of the country to country has been damaged due to COVID-19. According to the medical expert's opinion, wearing a facial can be a possible solution to fight against the destructive unseen enemy. Thus, the paper reflects on the embodiment of a scheme with the help of Deep Learning (DL), the Internet of things (IoT), and Blockchain. The proposed model includes a raspberry pi, a camera module, and Convolutional Neural Networks (CNN) based architecture to detect and classify masks in public places and send the detected results to the cloud for real-time data monitoring with IoT. This model also includes a Blockchain architecture model for moving discovered data from the microprocessor to the cloud to assure dependability, safety, and security. The proposed model further includes an IoT-based architecture to create an analog response like alarm and notification upon mask detection and classification. This also presents experimental data of the proposed solution, such as data analysis with deep learning models, testing with IoT-based architecture, a System Usability Scale (SUS), and an identical comparison among the existing solutions with our proposed system. This study finds the highest accuracy of 99.95% in mask detection and classification. Also, the proposed combined solution tracks out the SUS score of 77%. While working with the proposed solution, several drawbacks have been noticed. Firstly, the system can only detect a single person with a facemask. Though the CNN-based architecture can classify and detect masks, IoT-based orientation becomes unable to respond to multiple detected information at a time. This happens when the model is established on the IoT devices like the raspberry pi. The raspberry has little memory and less computational space than the generic personal computer. Secondly, the proposed model becomes unavailable when multiple cameras have been attached with raspberry pi. Thirdly, the data was moderate to work such a global issue. Finally, the integration of the Blockchain in the conceptual stages, no real-life implementation has yet been performed. In the future, the drawbacks found this research will be resolved, and a complete package of a mask surveillance system for COVID-19 prevention will be presented with the real-time of three emergent technologies such DL, IoT, and Blockchain. However, the research work in this manuscript can be adjustable in real-life mask detection and classification.
Funding Statement: The authors received no specific funding for this study.
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
1. R. Pivonello, R. S. Auriemma, C. Pivonello, A. M. Isidori, G. Corona et al., “Sex disparities in covid-19 severity and outcome: Are men weaker or women stronger?,” Neuroendocrinology, vol. 11, pp. 1066–1085, 2020. [Google Scholar]
2. C. Wang, P. W. Horby, F. G. Hayden and G. F. Gao, “A novel coronavirus outbreak of global health concern,” The Lancet, vol. 395, pp. 470–473, 2020. [Google Scholar]
3. L. Matrajt and T. Leung, “Evaluating the effectiveness of social distancing interventions to delay or flatten the epidemic curve of coronavirus disease,” Emerging Infectious Diseases, vol. 26, pp. 1740, 2020. [Google Scholar]
4. R. Erkhembayar, E. Dickinson, D. Badarch, I. Narula, D. Warburton et al., “Early policy actions and emergency response to the COVID-19 pandemic in Mongolia: Experiences and challenges,” The Lancet Global Health, vol. 8, pp. e1234–e1241, 2020. [Google Scholar]
5. M. Liang, L. Gao, C. Cheng, Q. Zhou, J. P. Uy et al., “Efficacy of face mask in preventing respiratory virus transmission: A systematic review and meta-analysis,” Travel Medicine and Infectious Disease, vol. 36, pp. 11751, 2020. [Google Scholar]
6. S. Chen, C. Zhang, M. Dong, J. Le and M. Rao, “Using ranking-cnn for age estimation,” in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 5183–5192, 2017. [Google Scholar]
7. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 4510–4520, 2018. [Google Scholar]
8. K. Suresh, M. Palangappa and S. Bhuvan, “Face mask detection by using optimistic convolutional neural network,” in 2021 6th Int. Conf. on Inventive Computation Technologies (ICICT), Coimbatore, India, pp. 1084–1089, 2021. [Google Scholar]
9. S. Singh, U. Ahuja, M. Kumar, K. Kumar and M. Sachdeva, “Face mask detection using YOLOv3 and faster R-CNN models: COVID-19 environment,” Multimedia Tools and Applications, vol. 80, pp. 19753–19768, 2021. [Google Scholar]
10. S. Hussain, Y. Yu, M. Ayoub, A. Khan, R. Rehman et al., “IoT and deep learning based approach for rapid screening and face mask detection for infection spread control of COVID-19,” Applied Sciences, vol. 11, pp. 3495, 2021. [Google Scholar]
11. M. Loey, G. Manogaran, M. H. N. Taha and N. E. M. Khalifa, “A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic,” Measurement, vol. 167, pp. 108288, 2021. [Google Scholar]
12. P. Nagrath, R. Jain, A. Madan, R. Arora and J. Hemanth, “SSDMNV2: A real time DNN-based face mask detection system using single shot multibox detector and mobilenetv2,” Sustainable Cities and Society, vol. 66, pp. 102692, 2021. [Google Scholar]
13. K. Baluprithviraj, K. Bharathi, S. Chendhuran and P. Lokeshwaran, “Artificial intelligence based smart door with face mask detection,” in 2021 Int. Conf. on Artificial Intelligence and Smart Systems (ICAIS), Coimbatore, India, pp. 543–548, 2021. [Google Scholar]
14. E. K. Elsayed, A. M. Alsayed, O. M. Salama, A. M. Alnour and H. A. Mohammed, “Deep learning for covid-19 facemask detection using autonomous drone based on IoT,” in 2020 Int. Conf. on Computer, Control, Electrical, and Electronics Engineering (ICCCEEE), Khartoum, Sudan, pp. 1–5, 2020. [Google Scholar]
15. F. J. M. Shamrat, S. Chakraborty, M. M. Billah, M. Al Jubair, M. S. Islam et al., “Face mask detection using convolutional neural network (CNN) to reduce the spread of covid-19,” in 2021 5th Int. Conf. on Trends in Electronics and Informatics (ICOEI), Tirunelveli, India, pp. 1231–1237, 2021. [Google Scholar]
16. A. Sharma, S. Bahl, A. K. Bagha, M. Javaid, D. K. Shukla et al., “Blockchain technology and its applications to combat COVID-19 pandemic,” Research on Biomedical Engineering, vol. 8, pp. 1–8, 2020. [Google Scholar]
17. A. Khurshid, “Applying blockchain technology to address the crisis of trust during the COVID-19 pandemic,” JMIR Medical Informatics, vol. 8, pp. e20477, 2020. [Google Scholar]
18. A. Kalla, T. Hewa, R. A. Mishra, M. Ylianttila and M. Liyanage, “The role of blockchain to fight against COVID-19,” IEEE Engineering Management Review, vol. 48, pp. 85–96, 2020. [Google Scholar]
19. H. Xu, L. Zhang, O. Onireti, Y. Fang, W. J. Buchanan et al., “Beeptrace: Blockchain-enabled privacy-preserving contact tracing for covid-19 pandemic and beyond,” IEEE Internet of Things Journal, vol. 8, pp. 3915–3929, 2020. [Google Scholar]
20. G. Natarajan, “Blockchain-based secure framework for elearning during COVID-19.” Indian Journal of Science and Technology, vol. 13, no. 12, pp. 1328–1341, 2020. [Google Scholar]
21. H. A. Khan, W. Jue, M. Mushtaq and M. U. Mushtaq, “Brain tumor classification in MRI image using convolutional neural network,” Mathematical Biosciences and Engineering, vol. 17, pp. 6203, 2020. [Google Scholar]
22. G. Chao, Y. Luo and W. Ding, “Recent advances in supervised dimension reduction: A survey,” Machine Learning and Knowledge Extraction, vol. 1, pp. 341–358, 2019. [Google Scholar]
23. N. Al Mudawi, N. Beloff and M. White, “Developing a Framework of Critical Factors Affecting the Adoption of Cloud Computing in Government Systems (ACCE-GOV),” In: Arai K. (Eds.) in Intelligent Computing. Lecture Notes in Networks and Systems, Springer, London, UK, vol. 283, pp. 520–538, 2022. [Google Scholar]
24. S. Ambesange, A. Vijayalaxmi, S. Sridevi and B. Yashoda, “Multiple heart diseases prediction using logistic regression with ensemble and hyper parameter tuning techniques,” in 2020 Fourth World Conf. on Smart Trends in Systems, Security and Sustainability (WorldS4), London, UK, pp. 827–832, 2020. [Google Scholar]
25. S. Ding, B. Qi and H. Tan, “An overview on theory and algorithm of support vector machines,” Journal of University of Electronic Science and Technology of China, vol. 40, pp. 2–10, 2011. [Google Scholar]
26. T. Aşuroğlu, K. Açıcı, C. B. Erdaş, M. K. Toprak, H. Erdem et al., “Parkinson's disease monitoring from gait analysis via foot-worn sensors,” Biocybernetics and Biomedical Engineering, vol. 38, no. 3, pp. 760–72, 2018. [Google Scholar]
27. A. Tharwat, T. Gaber, A. Ibrahim and A. E. Hassanien, “Linear discriminant analysis: A detailed tutorial,” AI Communications, vol. 30, pp. 169–190, 2017. [Google Scholar]
28. U. Tariq, A. Ibrahim, T. Ahmad, Y. Bouteraa and A. Elmogy, “Blockchain in internet-of-things: A necessity framework for security, reliability, transparency, immutability and liability,” IET Communications, vol. 13, pp. 3187–3192, 2019. [Google Scholar]
29. L. Kan, Y. Wei, A. H. Muhammad, W. Siyuan, L. C. Gao et al., “A multiple blockchains architecture on inter-blockchain communication,” in 2018 IEEE Int. Conf. on Software Quality, Reliability and Security Companion (QRS-C), Lisbon, Portugal, pp. 139–145, 2018. [Google Scholar]
30. A. Alazeb and P. Brajendra, “Maintaining data integrity in fog computing based critical infrastructure systems.” in 2019 Int. Conf. on Computational Science and Computational Intelligence (CSCI), Las Vegas, The United State, pp. 40–47, 2019. [Google Scholar]
31. A. Rahman, M. J. Islam, Z. Rahman, M. M. Reza, A. Anwar et al., “Distb-condo: Distributed blockchain-based iotsdn model for smart condominium,” IEEE Access, vol. 8, pp. 209594–209609, 2020. [Google Scholar]
32. O. GURAV, “Face mask detection dataset,” 2020. [online]. Available: https://www.kaggle.com/omkargurav/face-mask-dataset?fbclid=IwAR1K4sdOgnpiQu09er_Fwq0db8kcBjDY7oR5JG3z6_GBAhffFwLedFAWuKA. [Google Scholar]
33. A. S. R. S. Rao and J. A. Vazquez, “Identification of COVID-19 can be quicker through artificial intelligence framework using a mobile phone–based survey when cities and towns are under quarantine,” Infection Control and Hospital Epidemiology, vol. 41, no. 7, pp. 1, 2020. [Google Scholar]
 | This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |