A Deep Learning Approach for Landmines Detection Based on Airborne Magnetometry Imaging and Edge Computing

1  Introduction

A landmine can be defined as an explosive device that detonates from approach, contact, or just the presence of a vehicle or an individual. It is a self-contained munition that can be deployed at the surface and sub-surface levels [1]. Landmines are considered indiscriminate weapons, as anyone who triggers the mine becomes its victim. Depending on the intention of the landmine planter, some mined areas can be marked, whereas an alternative technique is to rely on surprise. Military establishments can use it to secure disputed borders and restrict enemy movement at times of war. In such cases, using landmines creates tactical barriers and denies access to critical locations. The extent regions of minefields are marked, and the deployment locations are also recorded carefully. However, like any other weapon, mines can also be used with malicious intent. In such scenarios, the landmine deployment can be utterly random without following the marking and mapping procedure. Once deployed, the landmines can lie dormant for a very long time until triggered or removed.

Based on the target, landmines can be categorized as Antipersonnel Landmines and Antitank Landmines [2]. The Antipersonnel Landmines are used against enemy personnel. These are designed to explode from a person’s presence, proximity, or contact. It also includes Improvised Explosive Devices (IEDs) with those same victim-activated characteristics. Moreover, improvised landmines have been extensively used in recent years, primarily by non-state armed groups [3]. Antipersonnel mines can be further categorized as blast landmines and fragmentation landmines. When triggered, the blast landmines create a heavy explosion, while the fragmentation landmines release fragments through the air. The Antivehicle or Antitank landmines are more powerful and contain more explosives than antipersonnel mines. These mines are triggered by heavy-load targets like artillery tanks and other enemy vehicles.

1.1 Risk Involved with Landmines

Landmines can significantly impact the daily lives of people in mine-affected communities. Landmines were used extensively in World War Two (WWII) [1]. Still, in post-World War II conflicts, they have also been used to render land useless to enemy civilian populations. The influence of landmines outlasts the war as they can remain dormant until triggered for decades. It poses a severe threat to future generations even after the conflict ends. Antipersonnel mines are estimated to contaminate about sixty states and other areas as of 2019. According to the report, children accounted for 43% of all civilian casualties in 2019. The total number of casualties per year due to landmines or Explosive Remnant of War (ERW) during 1999–2021, as recorded in Landmine Monitor Report [3], has been displayed in Fig. 1.

Figure 1: Annual casualties due to landmines/ERW between 1999 and 2021 [3]

Due to the risk involved and the adverse effects caused by landmines, the case for a landmine-free world has become more assertive in recent years. Still, millions of landmines are present in different parts of the world. Removal of the buried landmines is a very challenging task and involves a considerable amount of risk. As a result, various efforts are ongoing to develop new or improve existing technologies that can help to detect and clear landmines.

1.2 Landmine Detection

Demining or mine clearance is the process of removing landmines from a particular area. It involves carefully surveying the target area using various types of sensing devices that detect the presence of landmines. The manual removal of landmines using metal detectors is the most commonly used method of landmine detection. However, there is a risk of accidentally triggering some landmines in the process. Also, manual demining consumes significant time and effort to survey the desired area. Therefore, a lot of research is going on to mitigate these problems. With technological advancements, the size of the sensors used has been reduced significantly. It helps in mounting the sensors over some carrier vehicles that can be used to survey the target area. Some specialized sensor designs are even developed so that the sensing device can be mounted over an Unmanned Aerial Vehicle (UAV) or drone that can help to get the aerial survey of the desired location [4]. It helps to carry out the remote survey much faster while ensuring safety. The sensing technologies used for the detection of landmines include Metal Detector, Ground Penetration Radar (GPR) [5], Thermal or Infrared Images [6], Multispectral and Hyperspectral Images [7], and Magnetometer [8].

This study focuses on employing magnetic field data from a specific region to detect landmines. Magnetometers are specialized instruments that measure various aspects of the magnetic field, including its intensity, magnetic dipole moment, direction, strength, or relative changes at specific locations. Magnetometers have been used for geophysical surveys in various applications like mineral exploration [9]. In geophysical surveys, a magnetometer serves to detect alterations or anomalies in the Earth’s magnetic field. Specific magnetometer designs have been engineered to be compatible with UAVs, enabling aerial platforms to be equipped with this technology [10]. It can be effectively utilized for remote geomagnetic surveys.

Similarly, a geomagnetic survey can sense the magnetic anomaly or distortion caused by landmines in a particular region. The data gathered by the magnetometer is typically collected in the form of magnetic intensity readings at various locations. The collected data can be processed and analyzed using various Artificial Intelligence (AI) techniques to locate the target landmine signature. Some deep learning models have shown remarkable performance in various image processing applications. In this paper, different Convolutional Neural Network (CNN) based models have been explored to detect landmine signatures present in magnetometry images.

1.3 Deep Learning

Deep learning is a sub-branch of machine learning that enables a model or algorithm to make predictions, do classifications, or make certain decisions based on data without explicitly programming the same [11]. The benefits of deep learning include automated feature extraction from the raw data. It significantly reduces the effort required to extract the essential features present in the data that are important for decision-making. Automatic feature extraction is achieved using multiple interconnected layers where low-level features are extracted at lower layers while higher-level features are extracted at deeper ones.

The deep learning models are derived from Artificial Neural Networks (ANN), but their architecture contains intense interconnected layers that enable them to extract very complex features from the data. At each layer, multiple processing units called neurons are connected to the other layers. Each neuron performs some computations on the neuron input, and an activation function is used to generate the final neuron output. The deep neural network consists of multiple layers where each layer can have numerous neurons connected to the neurons of the other layers in different configurations. Specific parameters control the computation at each neuron. The overall network is trained to learn the parameter values that best fit the data. It is done using back-propagation, in which the net loss is used to update the network parameters. The network training is repeated at certain times on the dataset, continuously updating network parameters to minimize network loss.

The researchers from the deep learning community have proposed different network architectures for different domains, resulting in the formation of various popular deep learning models like Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), Generative Adversarial Network (GAN), Deep Belief Network (DBN), etc. [12]. The CNN is an algorithmic model based on deep learning. It has shown remarkable progress in various Computer Vision (CV) related tasks such as image classification, image recognition, object detection, image captioning, etc. [13]. Like any other deep learning algorithm, the objective of CNN is to learn some critical features from the input data that can be in the form of images. In CNN models, three layers are present: Convolutional Layers, Pooling Layers, and Fully Connected layers. The convolutional layers use a kernel or filter of a particular dimension. It is convolved through the input image that transforms the input image and generates feature maps. The pooling layers reduce the spatial dimensions of the input matrix by selecting only the required features. It generates a reduced representation that requires a manageable number of parameters while retaining essential features for computing output. A CNN model can generate the final feature map using various convolutional and pooling layers. It can then be flattened and fed to the fully connected layers to generate the final output. Like any other deep learning model, this output is compared with the expected result to compute loss. The loss is back-propagated to update different model parameters.

The work conducted in this paper aims to explore the use of CNN-based deep learning models to automate landmine detection in magnetometry images. In magnetometry images, the image matrix represents the magnetic map of a particular area. It is obtained from the measurements captured by the magnetometer during the survey. From such images, the CNN-based model can be trained to detect the magnetic signatures of the landmines that represent the magnetic anomalies caused by them. The detected signatures can be easily mapped to the geographical coordinates. It can be helpful for the removal of detected landmines.

1.4 MAGICS for Automating Landmine Detection

The MAGnetometry Imaging based Classification System (MAGICS) for landmine and buried object detection and localization is a system consisting of a multiplicity of components to achieve full automation of the process of detection of buried objects based on robotic airborne magnetic sensing. In this system, Fig. 2, the default operational scenario is perceived as follows:

Figure 2: The default scenario of MAGICS, which plans the mission, classifies and localizes the landmines and reports on results

1.   Operator enters the mission’s parameters via a GUI. The inputs include the coordinates of the search area, the search strategy, and options.

2.   The system calculates the optimal trajectory to carry on the search.

3.   The airborne magnetometer scans the area.

4.   The collected data are preprocessed to produce the heatmap image to be uploaded to the system.

5.   The deep learning model will classify and localize the landmine magnetic signatures and present the results via the GUI.

Here are some factors that highlight the differences and potential advantages of MAGICS:

•   Remote Sensing: It utilizes UAV-based airborne magnetometry, enabling remote sensing and surveying of minefields from a safe distance.

•   Deep Learning-Based Detection: MAGICS employs deep learning models for landmine detection in magnetometry images. Traditional methods often rely on manual data analysis, which can be time-consuming and less accurate.

•   Automation: It is a fully automated system designed to detect and localize landmines autonomously. Traditional methods may require extensive manual labor and expertise.

•   Edge Computing: The incorporation of edge computing in MAGICS allows for real-time analysis of magnetic field data, reducing the need for extensive post-processing and enabling faster decision-making in the field.

The proposed work brings several significant contributions to the field of landmine detection:

•   Developing an autonomous landmine detection framework that integrates magnetometry and deep learning.

•   Using a simulated dataset with diverse target signatures to obtain a more robust and versatile landmine detection system.

•   Utilizing the transfer learning technique to adapt pre-trained models for landmine detection.

In summary, the proposed work contributes to improving the efficiency and accuracy of landmine detection through the integration of magnetometry and deep learning, along with the utilization of a comprehensive simulated dataset. The rest of the paper is structured as follows: Section 1.5 briefly reviews the existing work related to landmine detection. Section 2 discusses the work carried out for landmine detection, involving the detection models and the dataset used. Section 3 evaluates the results obtained based on various parameters. Section 4 concludes the overall work with some future directions.

1.5 Related Work

Various researchers have considered the use of magnetic surveys in various geographical applications. The same has been explored for detecting buried landmines using magnetometer-based imaging. Zhang et al. [14] have considered the detection of sub-surface, Unexplored Ordnance (UXO) using magnetometer and Electromagnetic Induction (EMI) sensors. The likelihood ratio test and Support Vector Machine (SVM) have been used for classifying the target. The work in [15] has utilized a fluxgate sensor array to detect underground explosives based on captured magnetic field measurements. The authors used the K-Nearest Neighbour method to classify explosive and non-explosive materials. Yilmaz et al. have proposed a classification method for passive mine detection based on magnetic survey [16]. The proposed method is based on the magnetic anomaly, measurement height, and soil type.

With sensor design and operation advances, some magnetometers can be mounted over UAVs or drones to carry out magnetic surveys [17]. Yoo et al. [18] have used a magnetometer mounted over a drone to detect the landmines. The detection has been made based on the magnetic anomaly captured in the magnetometry image. Mu et al. [8] have also used UAV-based magnetic surveys to detect near-surface targets automatically. The authors have used a Deep Convolutional Neural Network (DCNN)-based model for target detection in the captured data. In [19], the authors improved the work done in [18] and carried out a drone-based magnetometer survey for mine detection in the demilitarized zone. The low pass filtering has been used to eliminate magnetic swing noise due to pendulum motion. The moving average method has been utilized to eliminate the changes related to the magnetometer heading. The authors have carried out the magnetic exploration in an actual mine removal area.

Ibraheem et al. [20] have presented a phase-based filter that can be used with magnetic field data to find buried UXO accurately. The study in [21] introduced a dual-mode landmine detection system that combines spectroscopic metal detection with GPR. The proposed system employed feature-level sensor fusion based on key features extracted from both sensors. It enhances operator feedback and automates the location of buried, visually obscured objects, including landmines. The work in [22] has explored the application of deep learning and UAVs for the automated detection of scatterable antipersonnel landmines. It leverages multispectral and thermal datasets collected by an automated UAV survey system. The work in [23] was also focused on using a thermal imaging dataset collected using UAVs for landmine detection. It has employed a Multilayer Perceptron (MLP) model to detect the presence of landmines in thermal images collected at varying heights. The work in [24] also tackled the critical issue of UXO detection in conflict-affected regions using thermal imaging and deep learning. The dataset of thermal images containing UXO belonging to different classes has been formulated and utilized for detection. The study in [25] has also proposed a joint detection system that combines a time-domain electromagnetic detection system (TDEM-Cart) and a multi-rotor UAV-based magnetic system (UAVMAG). The cooperative processing of the fused data from both systems has been used to give more accurate positioning results and information, which helps find and tell apart the UXO. The fusion network based on auto-encoder has been formulated in [26] to fuse the visible and infrared images to improve the detection of different types of landmines.

Previous research has explored the utilization of various methods for landmine detection. However, the prior studies often rely on limited field data and involve a relatively small number of targets for detection. In contrast, the proposed work aims to leverage magnetometry data and deep learning models for landmine detection. The approach utilizes a simulated dataset that encompasses many diverse targets to be detected. It provides the ability to test and train detection models under more extensive and varied conditions, thereby enhancing the robustness and effectiveness of landmine detection systems. Deep learning models have become beneficial technologies with extensive applications across several areas. In the Internet of Things (IoT) context, deep learning plays a vital role in enhancing services and system management [27]. It also helps to effectively identify anomalies and extract essential features in time-series Internet of Things (IoT) data [28]. Utilizing CNN-based deep learning methods to analyze and interpret magnetic field data enhances the accuracy and reliability of landmine signature identification. The models can learn complex patterns and characteristics from the data, enabling them to detect small magnetic anomalies linked to landmines effectively.

2  Landmine Detection Using Magnetometry Images

The work carried out in this paper focuses on the application of deep learning models for automating the process of landmine detection. It is similar to the object detection task where, based on an input image, the goal is to detect various objects present in it. In this study, transfer learning is adapted to enhance the landmine detection process using magnetometry images. Transfer learning is a technique where knowledge gained from one task was applied to a different but related task [29]. As part of our study, we used transfer learning to improve the ability to find landmines in magnetometry images by adapting existing deep learning models. It involves the selection of efficient detection models and retraining and fine-tuning them on the simulated dataset. The input and output layers of the models are customized to suit the requirements of the dataset. During fine-tuning, the internal layers of the model are adjusted to learn the features and patterns in the magnetometry images. It involves training the models on the simulated dataset to improve the ability to detect landmine signatures effectively. With this, the models adjust the internal parameters to develop a deep understanding of the unique features and patterns found in magnetometry images, which are those that indicate the presence of landmine traces. In landmine detection, the input is the magnetometry image, and the landmine signatures are the objects the model must detect. For this reason, various CNN-based single-stage object detectors have been used to detect the landmine signatures in magnetometry images.

2.1 Landmine Signature Detection Models

Various CNN-based single-stage object detection models have been selected and trained on the magnetometry image dataset. The objective is to leverage the existing state-of-the-art models for automating the landmine detection process. The general architecture followed by the single-stage object detectors contains the backbone, middle, and head networks. The backbone is responsible for generating the feature map from the input. The middle network, sometimes called the neck, performs the multi-scale feature fusion over the feature map generated by the backbone. The head generates the final outputs, including the bounding box and class predictions for different objects in an image [30]. The single-stage detectors predict a large number of bounding boxes for each image. The predictions are filtered out using Non-Max Suppression (NMS). It considers the highly overlapping boxes and keeps only the bounding box having the highest confidence value of containing the object while suppressing the rest. It eliminates the redundant predictions and outputs a single box for each object.

The models used for landmine detection are Retinanet, YOLOv5, and EfficientDet-D0. All the models are single-stage detectors that have shown promising results for object detection. The models have been customized and trained to detect landmine signatures present in magnetometry images.

2.1.1 RetinaNet

RetinaNet is a single-stage CNN-based model proposed by Lin et al. [31] for object detection. The architecture of RetinaNet has been depicted in Fig. 3. It uses some of the existing convolutional networks like ResNet [32] as the backbone for feature extraction. The Feature Pyramid Network (FPN) [33] has been adopted on top of the backbone network for multi-scale feature fusion in a top-down pathway. The head contains two subnetworks: the Classification subnetwork and the Regression subnetwork. The classification subnetwork determines the presence of the object and its classification for each spatial location. On the other hand, the regression subnetwork fits the bounding box values for each object in the image.

Figure 3: General architecture of RetinaNet

RetinaNet is a one-stage detector model that makes it faster and simpler to detect the objects present in an image. The authors have proposed a novel loss function called Focal Loss, an extension of the cross-entropy function. Using a simple cross-entropy loss function for single-stage detectors suffers from the class imbalance problem. The problem is addressed by novel focal loss used in RetinaNet. It is formed using additional parameters in the cross-entropy loss equation to down-weigh the loss associated with well-classified examples.

FL(pt)=−α(1−pt)γlog⁡(pt)(1)

In Eq. (1), FL is the focal loss calculated for a particular class probability, pt. α is the weighing factor and γ is the modulating factor. For RetinaNet, the authors have used the γ value of 2 and α value of 0.25.

2.1.2 YOLOv5

YOLO stands for You Only Look Once [34]. It represents a family of object detection models that also follows single-stage detection. The YOLO model uses a single network for both classification and regression tasks. Redmon et al. proposed the first YOLO model in 2016 [34]. Since then, various other versions of the YOLO model have been formed by incorporating the essential changes and improving the overall model performance. In our work, the YOLOv5 model has been used for landmine detection. Various improvements in YOLOv5 have enabled it to achieve better detection results on the standard object detection Pascal VOC dataset [35] and Microsoft COCO dataset [36].

The general architecture of YOLOv5 has been drawn in Fig. 4. In YOLOv5 Cross Stage Partial Network (CSPNet) [37] has been used as the backbone network, and Path Aggregation Network (PANet) [38] has been used as the neck network. PANet contains an extra bottom-up path aggregation network on top of FPN. The head of the YOLOv5 generates multi-scale feature maps to handle different object sizes, enabling it to detect small, medium, and large objects more efficiently. Another aspect that makes YOLOv5 different is the use of novel mosaic data augmentation [39]. It includes generating an augmented image from some random crop of four different images. It enhances the detection of small-scale objects in the image. The loss function of YOLOv5 is composed of classification loss, box loss or regression loss, and objectness or confidence loss.

Figure 4: General architecture of YOLOv5

YOLOv5 contains a set of models generated on different scales. The scale represents the model size or depth and, thus, the number of parameters a particular configuration uses. The larger models have better accuracy but require more time to train and infer from the images. We have utilized YOLOv5s and YOLOv5x models to detect landmine signatures in our work.

2.1.3 EfficientDet

Google Research, Brain team has developed the EfficientDet model [40]. EfficientDet has shown remarkable performance for object detection on the Pascal VOC and Microsoft COCO datasets.

Fig. 5 displays the overall architecture of EfficientDet. The authors have used EfficientNet [41] as the backbone, while a novel weighted Bi-directional Feature Pyramid Network (Bi-FPN) has been used for feature fusion. Several optimizations have been proposed in the cross-stage network used in bi-FPN. It includes removing the node with only one input edge, adding extra connections between the same level input and output, and repeating the bidirectional feature fusion layer multiple times. It enables the model to form more high-level feature fusion. The head contains dedicated convolutional network layers for class and box prediction. Like RetinaNet, it also uses the focal loss function, but different parameter values have been used. The authors have considered the γ value of 1.5 and α value of 0.25 for EfficientDet. The novel compound scaling method has generated a family of models of different sizes. The approach jointly scales up all the network dimensions: width, depth, and resolution. The scaling is controlled by a simple compound coefficient, ϕ. In our work, we have used the Efficient-D0 model. The D0 represents the 0 value of ϕ. It is the smallest among all the models in the EfficientDet family. The use of compound scaling simplifies the task of scaling up all the dimensions of the backbone, feature fusion, and head network.

Figure 5: General architecture of EfficientDet

2.2 Dataset Used

In the context of landmines, magnetic signatures are representations of how these buried objects affect the Earth’s magnetic field. Landmines often contain materials that disturb the surrounding magnetic field. This disturbance results in magnetic anomalies or irregularities in the magnetic field intensity. Various studies have revealed the presence of positive anomalies due to buried objects like UXO [8,20,25]. The magnetic field generated by an object is strongest in its close proximity and gradually weakens with increasing distance. It is a fundamental characteristic of magnetometry and plays a pivotal role in shaping the magnetic signature of landmines. In our simulated dataset, the effect is produced by modeling the magnetic signature as a positive circular anomaly. The magnetic field intensity is highest at the center of this anomaly, where the magnetic target (in this case, the landmine) is located. It gradually decreases as we move away from the center or target location. Extensive field studies and research in the area of magnetic inspection have consistently revealed the presence of magnetic anomalies that closely resemble circular patterns for buried objects like UXOs or landmines. Similar patterns can be observed in the real-time magnetic map obtained by the SENSYS magnetometer in field inspection using MagDroneR4 as shown in Fig. 6 [42]. The sample illustrated in the figure is excerpted from a study that utilized the MagDroneR4 UAV-magnetometer in an area containing various types of UXOs. Therefore, the choice to replicate it in the simulated dataset is based on empirical evidence and to ensure that the data closely aligns with real-world scenarios. Through this approach, we have aimed to capture the essence of magnetic signatures of landmines, thereby enhancing the authenticity and utility of our simulated dataset for training and testing landmine detection models.

Figure 6: Cropped section of a magnetic map from a surveyed region using the MagDroneR4 UAV-Magnetometer [42]

While real-world data is invaluable, simulated data is crucial in advancing the development and evaluation of deep learning models for landmine detection. Acquiring real magnetometry images recorded over landmine fields using UAV platforms presents considerable logistical and resource challenges. These challenges include safety concerns, access limitations, and the costs associated with field surveys. Simulated data, in this context, offers a practical and accessible means to conduct extensive testing and research. The simulated dataset used in our study is thoughtfully created to encompass various scenarios. It includes multiple targets, variations in anomalies, and different background conditions. The controlled nature also helps to experiment with various parameters and conditions. It is instrumental in systematic testing and fine-tuning. The availability of a large and diverse dataset has been instrumental in training and validating the deep learning models. It has provided the foundation for model development, enhancing detection accuracy and assessing the models’ generalizability across various scenarios. The diversity is a pivotal strength, providing a controlled yet comprehensive testbed for evaluating the deep learning models. The study can bridge the gap between developing deep learning models and their deployment in actual UAV-based magnetic surveys.

There is also a lack of a publically available dataset of magnetometry landmine images. For this reason, we have used the simulated magnetometry images to train and evaluate various deep learning models. The multiple steps involved in generating simulated magnetometry images have been depicted in Fig. 7. We initiate the generation process for each image in our dataset by creating a background matrix of random numerical values that defines the background magnetic field intensity readings in nanoteslas (nT). These values are initially generated as a matrix, simulating the magnetic field’s characteristics within the surveyed area. To enhance the realism, we applied a mean filter over the matrix. It emulates the natural variations and fluctuations typically observed in magnetic field data. Some random magnetic signatures are added at arbitrary locations to introduce the presence of landmine signatures. These signature values are superimposed onto the background magnetic intensity values, mimicking the magnetic anomalies caused by buried landmines. The resulting matrix effectively represented the magnetic field intensity values obtained from a survey of a specific area using a magnetometer. The process helps to create magnetometry images that closely resemble the real-world conditions encountered during landmine detection missions.

Figure 7: Steps involved in the generation of simulated Magnetometry colormap image

The mean value has been subtracted from all the matrix elements to normalize the data and highlight variations. It effectively produces a matrix illustrating the deviation or variation in the captured magnetic field intensity values from the background. The processed matrix is further transformed into a contour plot or colormap image. It provides a graphical representation of the magnetic field intensity variations, making it easier to visualize the spatial distribution of magnetic anomalies, including potential landmine locations, within the surveyed area. We have employed bounding boxes to precisely locate and annotate the regions associated with each signature (representing potential landmine positions). These bounding boxes enclose the target landmines within the contour plot image. The bounding box coordinates are further processed to ensure consistency and standardization. It involves dividing the box values by the dimensions of the matrix, ensuring that the annotations are relative to the size of the image matrix.

The simulation process is implemented using Python, involving the conversion of matrix values into custom-contour plot images. It resulted in the generation of an extensive dataset comprising 1000 simulated images split into a training set of 802 random images and a validation set consisting of the remaining 198. Notably, the precise locations of landmine signatures within each image are appropriately recorded to facilitate the generation of annotations vital for training the landmine detection models. The aim is to train the models to identify all landmine signatures within an image. The detection task is complicated by the miniature size of landmines that generates relatively subtle magnetic anomalies compared to larger objects. Furthermore, the potential overlap of anomalies from closely situated landmines in a minefield is thoughtfully considered in creating the signature map. While real-time magnetometer data often contains noise, the simulated image dataset operated under the assumption that the images represented pre-processed, noise-free data. This assumption streamlined model training, focusing on the core task of landmine signature detection.

Fig. 8 represents a sample magnetometry image generated using custom colormap and its detected signatures using a trained model. The image generated from the simulated matrix contains mean deducted values representing the magnetic intensity variation at a particular region. The landmine signatures used in our approach are represented as concentric circular regions of varying shape and size. The magnetic intensity is highest at the center and decreases outward. The variation accounts for the fact that different types of landmines may generate magnetic anomalies of different shapes and sizes, depending on their design and orientation. The simplified model serves as a proxy for magnetic anomalies caused by buried landmines. Also landmines are typically smaller in size compared to UXOs and often found in higher densities within a minefield. Consequently, we incorporated magnetic signatures of smaller dimensions with increased density in the simulated dataset to accurately represent real-world conditions. The signatures are superimposed at random locations on the random background matrix to produce the final magnetic map of the minefield. In practical landmine detection scenarios, it’s common to encounter areas with multiple landmines buried at different depths and orientations, resulting in a diverse range of magnetic signatures. We can train the detection models to recognize and differentiate multiple landmines by incorporating variation into the simulated data. It enhances the realism of the training data and contributes to more effective model training and evaluation.

Figure 8: Magnetometry image sample from the simulated dataset and the detection results

While the simulated dataset captures certain aspects of real-world magnetic signatures produced by landmines, it is important to acknowledge the differences. Magnetic anomalies observed in real-world scenarios exhibit variations in their shape, size, and intensity, which can be attributed to many factors, including the type of landmine, its depth, the composition of the soil, and the sensitivity of the magnetometer. The synthetic dataset exhibits some level of symmetry in the magnetic signatures, perhaps limiting its ability to accurately capture the variety and asymmetry commonly observed in real landmine signals. Precise parameters govern the production of synthetic signatures to create idealized representations that may not fully replicate the spontaneous variations observed in real environments. The signatures do not incorporate the presence of noise or interference commonly observed in actual magnetic field data collected from magnetometry surveys conducted in regions affected by landmines.

The dataset for this study is collected through a simulation process designed to mimic real-world magnetometry data acquisition. The steps taken or factors considered to ensure its accuracy and reliability are as follows:

•   Simulation Setup: The dataset is generated using a simulation setup that aims to replicate the conditions of a real magnetometry survey. It includes factors such as the magnetic signatures of landmines and their expected impact on the magnetic field.

•   Controlling Variables: During the simulation, various parameters and variables are controlled to ensure consistency and accuracy. It includes the size, shape, and location of the landmine signatures and the magnetic field properties.

•   Annotations: To evaluate the performance of detection models, annotations are generated for each synthetic image. These annotations include the precise locations and characteristics of the landmine signatures.

•   Data Augmentation: Data augmentation techniques have been applied to increase the dataset’s diversity and improve model generalization. These techniques could include variations in rotation, scaling, or other transformations.

3  Results and Evaluation

Since deep learning models require a high amount of computation, the implementation of different models has been carried out using a GPU machine.

3.1 Experimental Setup

The training and evaluation of all the detection models have been carried out using Google Colaboratory [43]. It is a platform provided by Google to run Python code online in the form of a notebook. The execution of the code written in a notebook takes place on Google’s cloud servers. It helps to leverage Google’s powerful hardware resources, including GPUs and TPUs. In our work, we have used the GPU run-time provided by the platform to implement different CNN-based models for landmine detection using magnetometry images.

We have utilized three different deep learning models: YOLOv5, RetinaNet, and EfficientDet. Each model underwent extensive training to adapt to the task of landmine detection. YOLOv5 was trained for 300 epochs, RetinaNet for 100 epochs, and EfficientDet for 200 epochs. The extensive training duration allowed the models to learn intricate features and patterns present in the magnetic images, enhancing their detection capabilities. The training process has been initilaized with an initial learning rate of 1e-5. The learning rate schedule and optimization techniques are fine-tuned during training to strike a balance between rapid convergence and avoiding local minima. To enhance the robustness, we have applied data augmentation techniques, including random rotations, flips, and scaling, to the training images. The augmentations helped the models generalize better to different orientations and scales of landmines. We closely monitored the model’s performance on the validation set throughout the training process to prevent overfitting. We employed the metric mean Average Precision (mAP) to assess the models’ precision-recall trade-offs and detection accuracy.

3.2 Evaluation Metric

The Microsoft COCO object detection criterion has been used to evaluate the landmine signature detection results obtained from different models. It considers the mean Average Precision (mAP) as the metric for performance evaluation. The mAP value represents the area under the Precision-Recall curve (P-R curve) obtained for all the classes over some Intersection over Union (IOU) threshold. The IOU value represents the ratio of the intersection region and the union region of the true and predicted bounding boxes. It is the percentage overlap between the predicted and true bounding boxes. For a perfect fit, the IOU value will be 1. IOU value is used as a threshold or confidence value in object detection models to calculate true positives. It helps in the determination of the mAP metric that considers both precision and recall estimates.

Eq. (2) represents the general equation for calculating Average Precision (AP). Here, P is the Precision computed as a function of R, i.e., Recall. The integral gives the area under the P-R curve.

AP=∫01P(R)dR(2)

For discrete recall intervals, the Eq. (2) can be written as:

AP=1n∑iP(Ri)(3)

In Eq. (3), Ri represents the Recall intervals for which the Precision values have been considered, and n is the total number of recall intervals. Since multiple classes exist, the final mAP metric is computed as the average of the AP values for all categories.

mAP=1m∑i=1mAPi(4)

The mAP values used for evaluating the performance of different models are mAP at 0.5 and mAP at 0.5:0.95. The first metric represents the mAP value at the IOU threshold value of 0.5. In contrast, the second metric is computed as the average of mAP values at 10 IOU threshold intervals in the range of 0.5 to 0.95. The second metric forms a more general representation of model performance due to the consideration of a wide range of confidence scores.

3.3 Results Obtained for Different Models

Table 1 gives an overview of key configurations for different models that have been trained for landmine detection. The training time per step for YOLOv5s is minimum, while EfficientDet-D0 has the minimum number of parameters. YOLOv5x is the largest model, while the training time for RetinaNet is maximum. The EfficientDet and RetinaNet models take a minimum input size 512 by automatically re-scaling the image. It increases the training time required for these models compared to YOLO models that can be trained directly using 224 image size. All the YOLO models have been trained for 300 epochs, RetinaNet for 100 epochs, and EfficientDet for 200 epochs. Within these epochs, it has been observed that the respective models have reached a point beyond which there was no further improvement in performance. Fig. 9 depicts the obtained mAP values after each iteration for YOLOv5 models. The overall loss concerning training epochs for YOLOv5s and YOLOv5x has been depicted in Fig. 10 while Fig. 11 shows the validation loss. Similarly, Fig. 12 presents the loss for EfficientDet-D0, and Fig. 13 presents the loss for RetinaNet.

Figure 9: Mean Average Precision (mAP) at 0.5:0.95 for different YOLOv5 models

Figure 10: Total training loss for different YOLOv5 models

Figure 11: Total validation loss for different YOLOv5 models

Figure 12: Total training and validation loss for EfficientDet-D0

Figure 13: Total training and validation loss for RetinaNet

Table 2 depicts the performance of various models after training and the inference time requirements for different models. Based on the mAP evaluation metric at 0.5:0.95 threshold, it can be observed that out of all the models, EfficientDet-D0 produced the best results with the mAP value of 0.978. The performance has been obtained by increasing the learning rate to the initial value after 100 epochs. It can lead to a sudden increase in loss but helps the model to reach global optima. YOLOv5x trained with Adam optimization also has a close mAP value of 0.9764. The YOLO models use the learning rate schedule that first decreases and then increases the learning rate, which helps escape local optima. The mAP value for RetinaNet is 0.9622, while for YOLOv5s, it is 0.9345 when trained using Adam optimization.

Although YOLO models have been trained using an original image size of 224, for comparison, inference time for the input image of size 512 has also been considered. It has been evaluated that for input size 512, the inference time is approximately 0.024 s for YOLOv5s and 0.14 s for YOLOv5x. Based on the same, the inference time for YOLOv5s is minimum, followed by EfficientDet-D0, then YOLOV5x, and lastly, RetinaNet. Although the accuracy of YOLOv5s is the lowest among all the models, its small size and minimum detection time make it suitable for time-critical applications.

The use of Adam optimizer over Stochastic Gradient Descent (SGD) significantly improves the performance of the detection model. It can be observed from the YOLO models’ performances that have been trained using both optimizers. EfficientDet-D0 is the model that utilized the minimum number of parameters and achieved maximum performance.

There is a trade-off between the precision of landmine detection and the time needed for training and detection. It needs to be considered while creating effective solutions for practical applications in the real world. As the size and complexity of the detection model grow, the associated parameters also increase, impacting the model’s performance. Larger and more complex models have a better capacity to acquire intricate patterns and features within the data. Our investigations have also indicated that increasing the model size enhances accuracy in detecting landmine signatures, as illustrated in Table 2. However, the improvement in accuracy is accompanied by an increase in the duration of training. As the complexity of the model increases, the training procedure requires more computational resources and time to optimize a greater number of parameters efficiently. The computational complexity can affect the detection speed during the deployment of trained models for real-time detection. In situations where precision is crucial, opting for a larger model can be considered to maximize accuracy. On the other hand, when timely detection is critical, a smaller, more efficient model can be selected to maintain an acceptable level of accuracy while speeding up detection.

3.4 Challenges

There are certain limitations to the performance of MAGICS:

•   Generalization to Real-World Data: While the simulated dataset provides a valuable resource for training and testing the MAGICS, it is important to acknowledge that simulated data may not capture all the complexities and variations in real-world scenarios. Therefore, further validation of real data collected by UAVs over actual minefields is essential.

•   Terrain and Environmental Variability: Real-world environments can include various terrains, vegetation types, and environmental conditions. The performance of the proposed system may vary depending on factors such as soil composition, vegetation density, and weather conditions.

•   Deployment Challenges: Implementing MAGICS in operational settings may require considerations for hardware deployment, power supply, and connectivity. These practical challenges need to be addressed for successful real-world applications.

•   Scalability: Large-scale operations generate vast amounts of data. Efficient data collection, storage, transmission, and analysis become critical. It requires robust data management systems and high-performance computing infrastructure. Also, coordinating logistics and resources in remote or conflict-affected regions can be complex.

4  Conclusion and Future Work

The active use of landmines and existing landmines poses considerable risk, affecting countless lives. Landmine detection needs continuous research and improvements to incorporate various artificial intelligence techniques to automate the process. The MAGICS has been proposed as an automated framework to deal with such a problem. The work done in this paper focused on applying deep learning models for the detection of landmines using magnetometry images. For this, various CNN-based models have been trained using a simulated magnetometer-based image dataset. The results obtained from the introduced models confirm their use for automating landmine detection from magnetometer-captured images. The models have shown satisfactory results with the best performance of EfficientDet-D0 while minimum time requirement by YOLOv5s. Deep learning can significantly automate the landmine detection process and reduce the time and effort required. Moreover, deep learning models can enhance the detection since they can detect those signatures that sometimes are not easy to interpret. It can significantly improve the overall accuracy of landmine detection.

The work done in the paper can be extended by considering some real-time magnetometry images. Then, different deep learning models can be trained using such images in the presence of extreme noise conditions due to sensing. The tuning of various hyper-parameters involved in deep learning models and applying different model configurations can be further explored to improve performance.

Acknowledgement: This research work was funded by Institutional Fund Projects under Grant No (IFPNC-001-611-2020). Therefore, the authors gratefully acknowledge technical and financial support from the Ministry of Education and King Abdelaziz University, Jeddah, Saudi Arabia. The authors would like to thank King Abdulaziz University for its support.

Funding Statement: This research work was funded by Institutional Fund Projects under Grant No (IFPNC-001-611-2020). Therefore, the authors gratefully acknowledge technical and financial support from the Ministry of Education and King Abdelaziz University, Jeddah, Saudi Arabia.

Author Contributions: The authors confirm contribution to the paper as follows: Software, K.K. and A.A.; Validation, B.A.; Formal analysis, A.A. and K.K.; Investigation, A.B. and B.A.; Supervision, A.B. and N.K.; Project administration, A.B. All authors reviewed the results and approved the final version of the manuscript.

Availability of Data and Materials: The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

References

1. International campaign to ban landmine. http://icbl.org (accessed on 12/03/2023) [Google Scholar]

2. Bello, R. (2013). Literature review on landmines and detection methods. Frontiers in Science, 3(1), 27–42. [Google Scholar]

3. Landmine monitor. http://the-monitor.org (accessed on 12/03/2023) [Google Scholar]

4. Šipoš, D., Gleich, D. (2020). A lightweight and low-power uav-borne ground penetrating radar design for landmine detection. Sensors, 20(8), 2234. [Google Scholar] [PubMed]

5. Lameri, S., Lombardi, F., Bestagini, P., Lualdi, M., Tubaro, S. (2017). Landmine detection from GPR data using convolutional neural networks. 2017 25th European Signal Processing Conference (EUSIPCO), pp. 508–512. Kos, Greece, IEEE. [Google Scholar]

6. Kaya, S., Leloglu, U. M. (2017). Buried and surface mine detection from thermal image time series. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(10), 4544–4552. [Google Scholar]

7. Silva, J. S., Guerra, I. F. L., Bioucas-Dias, J., Gasche, T. (2019). Landmine detection using multispectral images. IEEE Sensors Journal, 19(20), 9341–9351. [Google Scholar]

8. Mu, Y., Zhang, X., Xie, W., Zheng, Y. (2020). Automatic detection of near-surface targets for unmanned aerial vehicle (UAV) magnetic survey. Remote Sensing, 12(3), 452. [Google Scholar]

9. Macharet, D. G., Perez-Imaz, H. I., Rezeck, P. A., Potje, G. A., Benyosef, L. C. et al. (2016). Autonomous aeromagnetic surveys using a fluxgate magnetometer. Sensors, 16(12), 2169. [Google Scholar] [PubMed]

10. Shahsavani, H. (2021). An aeromagnetic survey carried out using a rotary-wing uav equipped with a low-cost magneto-inductive sensor. International Journal of Remote Sensing, 42(23), 1–14. [Google Scholar]

11. Dong, S., Wang, P., Abbas, K. (2021). A survey on deep learning and its applications. Computer Science Review, 40, 100379. [Google Scholar]

12. Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y. et al. (2018). A survey on deep learning: Algorithms, techniques, and applications. ACM Computing Surveys, 51(5), 1–36. [Google Scholar]

13. Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E. (2018). Deep learning for computer vision: A brief review. Computational Intelligence and Neuroscience, 2018, 1–13. [Google Scholar]

14. Zhang, Y., Collins, L., Yu, H., Baum, C. E., Carin, L. (2003). Sensing of unexploded ordnance with magnetometer and induction data: Theory and signal processing. IEEE Transactions on Geoscience and Remote Sensing, 41(5), 1005–1015. [Google Scholar]

15. Gürkan, S., Karapınar, M., Doğan, S. (2019). Detection and imaging of underground objects for distinguishing explosives by using a fluxgate sensor array. Applied Sciences, 9(24), 5415. [Google Scholar]

16. Yilmaz, C., Kahraman, H. T., Söyler, S. (2018). Passive mine detection and classification method based on hybrid model. IEEE Access, 6, 47870–47888. [Google Scholar]

17. Zheng, Y., Li, S., Xing, K., Zhang, X. (2021). Unmanned aerial vehicles for magnetic surveys: A review on platform selection and interference suppression. Drones, 5(3), 93. [Google Scholar]

18. Yoo, L. S., Lee, J. H., Ko, S. H., Jung, S. K., Lee, S. H. et al. (2020). A drone fitted with a magnetometer detects landmines. IEEE Geoscience and Remote Sensing Letters, 17(12), 2035–2039. [Google Scholar]

19. Yoo, L. S., Lee, J. H., Lee, Y. K., Jung, S. K., Choi, Y. (2021). Application of a drone magnetometer system to military mine detection in the demilitarized zone. Sensors, 21(9), 3175. [Google Scholar] [PubMed]

20. Ibraheem, I. M., Aladad, H., Alnaser, M. F., Stephenson, R. (2021). IAS: A new novel phase-based filter for detection of unexploded ordnances. Remote Sensing, 13(21), 4345. [Google Scholar]

21. Marsh, A. L., van Verre, W. L., Davidson, J., Gao, X., Peyton, A. J. et al. (2019). Combining electromagnetic spectroscopy and ground-penetrating radar for the detection of anti-personnel landmines. Sensors, 19(15), 3390. [Google Scholar]

22. Baur, J., Steinberg, G., Nikulin, A., Chiu, K., de Smet, T. S. (2020). Applying deep learning to automate uav-based detection of scatterable landmines. Remote Sensing, 12(5), 859. [Google Scholar]

23. Forero-Ramírez, J. C., García, B., Tenorio-Tamayo, H. A., Restrepo-Girón, A. D., Loaiza-Correa, H. et al. (2022). Detection of legbreaker antipersonnel landmines by analysis of aerial thermographic images of the soil. Infrared Physics & Technology, 125, 104307. [Google Scholar]

24. Bajić Jr, M., Potočnik, B. (2023). UAV thermal imaging for unexploded ordnance detection by using deep learning. Remote Sensing, 15(4), 967. [Google Scholar]

25. Mu, Y., Xie, W., Zhang, X. (2021). The joint UAV-borne magnetic detection system and cart-mounted time domain electromagnetic system for uxo detection. Remote Sensing, 13(12), 2343. [Google Scholar]

26. Qiu, Z., Guo, H., Hu, J., Jiang, H., Luo, C. (2023). Joint fusion and detection via deep learning in uav-borne multispectral sensing of scatterable landmine. Sensors, 23(12), 5693. [Google Scholar] [PubMed]

27. Xu, Y., Xiao, W., Yang, X., Li, R., Yin, Y. et al. (2023). Towards effective semantic annotation for mobile and edge services for Internet-of-Things ecosystems. Future Generation Computer Systems, 139, 64–73. [Google Scholar]

28. Gao, H., Qiu, B., Barroso, R. J. D., Hussain, W., Xu, Y. et al. (2022). TSMAE: A novel anomaly detection approach for Internet of Things time series data using memory-augmented autoencoder. IEEE Transactions on Network Science and Engineering, 10(5), 2978–2990. [Google Scholar]

29. Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y. et al. (2020). A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1), 43–76. [Google Scholar]

30. Zaidi, S. S. A., Ansari, M. S., Aslam, A., Kanwal, N., Asghar, M. et al. (2022). A survey of modern deep learning based object detection models. Digital Signal Processing, 126, 103514. [Google Scholar]

31. Lin, T. Y., Goyal, P., Girshick, R., He, K., Dollár, P. (2017). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988. Venice, Italy. [Google Scholar]

32. He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. Las Vegas, Nevada. [Google Scholar]

33. Lin, T. Y., Dollár, P., Girshick, R., He, K., Hariharan, B. et al. (2017). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125. Honolulu, Hawaii. [Google Scholar]

34. Redmon, J., Divvala, S., Girshick, R., Farhadi, A. (2016). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788. Las Vegas, Nevada. [Google Scholar]

35. Everingham, M., Eslami, S. A., van Gool, L., Williams, C. K., Winn, J. et al. (2015). The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision, 111(1), 98–136. [Google Scholar]

36. Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P. et al. (2014). Microsoft COCO: Common objects in context. European Conference on Computer Vision, pp. 740–755. Zurich, Switzerland, Springer. [Google Scholar]

37. Wang, C. Y., Liao, H. Y. M., Wu, Y. H., Chen, P. Y., Hsieh, J. W. et al. (2020). CSPNet: A new backbone that can enhance learning capability of cnn. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 390–391. Seattle, WA, USA. [Google Scholar]

38. Wang, K., Liew, J. H., Zou, Y., Zhou, D., Feng, J. (2019). PANet: Few-shot image semantic segmentation with prototype alignment. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9197–9206. Seoul, Korea. [Google Scholar]

39. Bochkovskiy, A., Wang, C. Y., Liao, H. Y. M. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934. [Google Scholar]

40. Tan, M., Pang, R., Le, Q. V. (2020). Efficientdet: Scalable and efficient object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10781–10790. Seattle, WA, USA. [Google Scholar]

41. Tan, M., Le, Q. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. International Conference on Machine Learning, pp. 6105–6114. California, USA, PMLR. [Google Scholar]

42. Sensys. https://sensysmagnetometer.com (accessed on 12/03/2023) [Google Scholar]

43. Google colaboratory. http://colab.research.google.com/ (accessed on 12/03/2023) [Google Scholar]