Fuzzy clustering theory is widely used in data mining of full-face tunnel boring machine. However, the traditional fuzzy clustering algorithm based on objective function is difficult to effectively cluster functional data. We propose a new Fuzzy clustering algorithm, namely FCM–ANN algorithm. The algorithm replaces the clustering prototype of the FCM algorithm with the predicted value of the artificial neural network. This makes the algorithm not only satisfy the clustering based on the traditional similarity criterion, but also can effectively cluster the functional data. In this paper, we first use the
Cluster analysis belongs to unsupervised pattern recognition and is a multivariate statistical analysis method. It divides an initial sample set into several subsets according to a certain criterion, so as to achieve clustering of sample sets and analyze clustering results. Because in the actual engineering application, the research objects without clear classification boundaries occupy the main position, the fuzzy clustering is mainly used to cluster such objects, that is, the objects can belong to two or more categories at the same time [
There have been many researches on FCM algorithms. Yu et al. [
In the research of FCM type clustering algorithm, the research of clustering prototype has always been an important direction. The initial clustering prototype is a “point” in space, which is only suitable for the detection of hypersphere clustering structures. In order to detect the non-hypersphere clustering structure, Bzedek et al. [
Functional data refers to data obeying a function, and can also be regarded as random observation data of a function in an interval. With the development of data acquisition and storage capabilities, data can be collected in many fields with functional features. Functional data has been applied to many fields such as economics, medicine, meteorology and neuroscience [
Although the above method can achieve clustering of some functional data, it lacks universality in application. Because some of these methods require high accuracy of data, they cannot be implemented with high data noise. Others will ignore certain parameter information, resulting in low classification accuracy.
In the recent period, machine learning methods represented by deep learning have been applied to many fields, such as computer vision, natural language processing, and data mining. Since deep learning has good feature learning capabilities, it is also used as an alternative method for engineering problems. Samaniego et al. [
In the field of fuzzy clustering, there are also many studies that combine neural network methods. ANN simulates a biological neural network and builds a training model from multiple nodes to achieve regression and approximation of complex functions. Xu et al. [
For the clustering problem of functional data, this paper proposes the FCM–ANN algorithm. It obtains an approximate functional model through the ANN training dataset. Then the function model is used as a new clustering prototype to replace the clustering prototype of the traditional FCM algorithm, and participates in the algorithm’s alternating optimization. Finally, the algorithm will obtain the clustering result of the dataset and the corresponding functional data ANN model.
The rest of this paper is organized as follows. In Section 2, we first introduce the FCM algorithm and the artificial neural network algorithm, and then propose the FCM–ANN algorithm. Section 3 introduces the clustering results of applying the algorithm to synthetic dataset experiments and compares it with the traditional FCM algorithm. Section 4 introduces the experimental results of applying the FCM–ANN algorithm to the operation data of the tunnel boring machine (TBM) and compares it with the method of not performing classification modeling. In Section 5 we make some conclusions on this paper.
The FCM algorithm is a fuzzy clustering algorithm based on objective function. For a given dataset
where
The partition matrix is expressed as
Combined with the constraint
The specific steps of the FCM algorithm are as follows:
Set the number of subclusters of fuzzy clustering Calculate (or update) the partition matrix Update clustering prototypes Calculate the discriminant
If the
The algorithm can also start by initializing the fuzzy partition matrix
As a hotspot in recent years, artificial neural network is a multi-disciplinary subject area with a wide range and depth. A neural network is a broad and interconnected network of adaptive simple units whose organization mimics the interactions of the biological nervous system with the real world. The most basic building block in a neural network is a neuron. Each neuron is connected to other neurons, and each neuron transmits information through signals.
In this paper, we use the BP (error BackPropagation) algorithm to construct a neural network, which is the most common neural network learning algorithm by far. For a given training dataset
The structure has
For a
where
In view of the fact that the traditional FCM algorithm can not effectively solve the fuzzy clustering problem of functional data, we propose the FCM–ANN algorithm. In the FCM–ANN algorithm, we replace the clustering prototype in the traditional FCM algorithm with the neural network prediction value. This makes the clustering prototype change to conform to a certain data partitioning function, thus achieving accurate clustering of data.
For functional data, the iterative process of the FCM–ANN algorithm can be expressed as follows:
Set the number of subclusters of fuzzy clustering The clustering result can be obtained according to the partition matrix The prediction model is used for the independent variables Use the predicted output as a clustering prototype, the corresponding Euclidean distance Update the partition matrix Calculate the discriminant:
If the
The FCM–ANN algorithm combines the advantages of both FCM and ANN. FCM makes the iteration always go in the direction of gradient descent. ANN provides accurate prediction results and can approximate any nonlinear function. In theory, the algorithm can achieve fuzzy clustering of arbitrary functional data.
In this section, we construct some synthetic datasets to study the validity of FCM–ANN algorithms and their effect on predictions with the different number of samples, number of attributes, and noise.
The synthetic datasets are created as follows. In each cluster of each dataset, the objects of
Number of attributes | Cluster | Equation | |
---|---|---|---|
2 | 1 | (11.1) | |
2 | (11.2) | ||
3 | 1 | (12.1) | |
2 | (12.2) | ||
4 | 1 | (13.1) | |
2 | (13.2) | ||
5 | 1 | (14.1) | |
2 | (14.2) | ||
6 | 1 | (15.1) | |
2 | (15.2) |
In order to test the clustering performance, we carried out a comparison experiment between the proposed algorithm and the FCM algorithm.
The clustering performance is evaluated in terms of misclassification (
where
We set the fuzzification parameter
Number of samples | Misclassification ( |
||||||
---|---|---|---|---|---|---|---|
FCM | |||||||
100 | 31.80 | 29.87 | 26.07 | 26.77 | 42.00 | ||
Std. | 9.92 | 10.87 | 10.22 | 10.36 | 11.82 | 0 | |
200 | 29.33 | 25.63 | 23.93 | 20.85 | 35.50 | ||
Std. | 11.10 | 8.41 | 12.38 | 12.22 | 12.91 | 0 | |
300 | 28.10 | 25.03 | 22.89 | 18.40 | 39.33 | ||
Std. | 9.94 | 9.39 | 11.59 | 13.13 | 13.41 | 0 | |
400 | 27.91 | 23.69 | 22.31 | 16.80 | 38.50 | ||
Std. | 10.52 | 7.45 | 6.80 | 7.66 | 8.06 | 0 |
Number of attributes | Misclassification ( |
||||||
---|---|---|---|---|---|---|---|
FCM | |||||||
2 | 31.33 | 27.63 | 23.93 | 20.85 | 34.50 | ||
Std. | 11.10 | 8.41 | 12.38 | 12.22 | 9.91 | 0 | |
3 | 31.78 | 28.17 | 26.38 | 22.42 | 46.00 | ||
Std. | 13.40 | 12.08 | 11.15 | 12.75 | 12.23 | 0 | |
4 | 21.13 | 18.63 | 15.60 | 13.20 | 47.00 | ||
Std. | 15.06 | 16.94 | 16.46 | 11.45 | 11.65 | 0 | |
5 | 21.78 | 17.12 | 16.15 | 15.28 | 40.00 | ||
Std. | 12.76 | 13.43 | 12.17 | 13.48 | 11.85 | 0 | |
6 | 30.17 | 28.42 | 25.25 | 23.93 | 36.00 | ||
Std. | 10.42 | 9.62 | 12.00 | 13.50 | 14.94 | 0 |
Misclassification ( |
|||||||
---|---|---|---|---|---|---|---|
Noise | FCM | ||||||
3% | 28.47 | 27.26 | 26.59 | 22.50 | 34.25 | ||
Std. | 11.88 | 13.07 | 12.70 | 13.41 | 14.38 | 0 | |
4% | 29.98 | 27.47 | 28.65 | 23.09 | 36.25 | ||
Std. | 10.73 | 10.27 | 11.37 | 11.06 | 13.87 | 0 | |
5% | 30.12 | 28.53 | 26.00 | 24.25 | 30.50 | ||
Std. | 10.24 | 11.23 | 13.01 | 12.21 | 13.78 | 0 | |
6% | 30.94 | 28.08 | 26.34 | 25.48 | 33.50 | ||
Std. | 11.55 | 13.17 | 12.82 | 12.82 | 13.65 | 0 | |
7% | 31.53 | 28.65 | 28.28 | 27.07 | 33.00 | ||
Std. | 11.95 | 12.32 | 12.69 | 16.23 | 15.21 | 0 |
We use the
The one-sample
The statistic for the one-sample
where
The
Under the condition of the significance level
This hypothesis is tested by calculating the statistic
Number of samples | |||||
---|---|---|---|---|---|
100 | 0.5638 | −0.9876 | 0.8064 | ||
200 | 0.4034 | −0.4654 | −1.8288 | −2.8240 | |
300 | 1.6795 | 0.0002 | −0.9804 | −2.7069 | −2.9315 |
400 | 1.4896 | −0.9469 | −2.1303 | −5.7648 | −5.6257 |
Number of attributes | |||||
2 | 1.6841 | −0.4654 | −1.8288 | −3.6789 | |
3 | 1.4132 | 0.6665 | −1.0897 | −1.9154 | |
4 | −1.3838 | −2.0250 | −3.0753 | −5.8179 | −5.4545 |
5 | −1.3590 | −3.1597 | −3.9161 | −3.8831 | −5.1443 |
6 | 0.1122 | −0.4268 | −0.9552 | ||
Noise | |||||
3% | 1.5729 | 0.9312 | 0.6742 | −1.0040 | −1.1946 |
4% | 1.2952 | 1.7287 | −0.9300 | −1.0794 | |
5% | 1.6928 | 0.4139 | −0.3308 | −0.6331 | |
6% | 1.2594 | 0.5629 | 0.2016 | −0.6431 | |
7% | 1.5954 | 1.3919 | 0.6868 | −0.0248 |
In order to evaluate whether there is a significant difference between the results obtained by the FCM–ANN algorithm and the FCM algorithm, we also used the SPSS software to conduct a Wilcoxon rank-sum test. We compare the results in
Rank sum | |||||
---|---|---|---|---|---|
110 | 105 | 105 | 105 | 105 |
1) By analyzing
2) According to the results of
3) According to the vertical comparison
4) According to the results of
5) According to
We use the average of running time to measure the computation cost of the algorithm. The experimental environment is intel(R) Core (TM) i5-9300H 2.40 GHz CPU, 8 GB RAM, Windows 10 operating system, Python 3.7.0.
Number of attributes | Running time/s | |||||
---|---|---|---|---|---|---|
FCM | ||||||
100 | 0.3271 | 0.3989 | 0.5709 | 0.5559 | 0.5527 | 0.0369 |
200 | 0.4092 | 0.4252 | 0.6779 | 0.6681 | 0.7563 | 0.1057 |
300 | 0.6002 | 0.5838 | 0.5714 | 0.4235 | 0.9569 | 0.0768 |
400 | 0.7380 | 0.8997 | 1.0559 | 1.1190 | 1.2350 | 0.1127 |
Number of attributes | Running time/s | |||||
FCM | ||||||
2 | 0.4092 | 1.4252 | 1.6778 | 2.6681 | 2.7563 | 0.0439 |
3 | 0.7149 | 0.9637 | 0.7444 | 0.9320 | 1.6733 | 0.0469 |
4 | 0.8643 | 1.2287 | 1.2804 | 1.0578 | 1.1262 | 0.7982 |
5 | 1.1038 | 1.1927 | 1.2482 | 1.6605 | 1.7108 | 0.2075 |
5 | 0.9758 | 1.2253 | 1.4003 | 1.8045 | 2.3908 | 0.2558 |
Running time/s | ||||||
Noise | FCM | |||||
3% | 1.4720 | 1.6738 | 1.6185 | 2.3198 | 2.0802 | 0.2922 |
4% | 1.5481 | 2.2992 | 2.7570 | 2.9883 | 2.8625 | 0.2666 |
5% | 1.3907 | 2.4404 | 2.6289 | 3.4468 | 3.7464 | 0.3583 |
6% | 1.2716 | 1.8903 | 2.2232 | 2.9157 | 3.0527 | 0.2733 |
7% | 1.5146 | 1.3511 | 2.3948 | 2.6059 | 2.8353 | 0.3480 |
According to the results in
Then, we concentrate on the convergence analysis of FCM–ANN algorithm. In the Synthetic data sets, we selected data sets with the number of attributes of 2–5 for testing, as shown in
It can be seen that in the case of different number of attributes, the objective function value declines rapidly in the first 6 iterations and then tends to converge gradually. Therefore, it can be further proved that the proposed algorithm is effective.
The tunnel project studied in this paper is located on a metro line in China, with a length of 2,000 meters and a diameter of 6.4 meters. The project adopted earth pressure balance (EPB) shield TBM. This system consists of a cutterhead, chamber, screw conveyor, tail skin and other auxiliary. The operation data adopted in this paper are collected from TBM construction projects, and there are 52 parameters, among which each parameter has different correlation with the tunneling speed. The statistical chart of the datasets used in this paper is listed in Appendix A.
The experiment is divided into two parts. Experiment 1 uses the neural network algorithm to directly train the TBM operation data to establish the prediction model of predicting the tunneling speed. Experiment 2 uses the FCM–ANN algorithm to cluster the TBM operation data firstly, aiming to obtain clustering results and prediction models for each cluster, and then use the model to predict each cluster of data. The two experimental results are compared to verify the practicability of the algorithm.
The same 1200 sets of TBM operation data were used in both experiments. We divide the datasets into three parts: Training set, validation set, and test set. Among them, the test set accounts for 20%, and the remaining 80% of the data is divided into training sets and validation sets. In addition, in order to prevent over-fitting problems, we perform 5-fold cross-validation on the training set and the validation set. The data are randomly divided into 5 equal parts, and each of the 5 equal parts is used as a separate test set, and the remaining 4 parts are used as the training set for building the model for validity verification. We use mean square error (MSE) as an indicator to evaluate the prediction accuracy of the validation set, which is calculated by
1) The steps of Experiment 1 are as follows:
Use the neural network algorithm to train the training set and the validation set, and adopt the cross-validation method to obtain multiple neural network prediction models. Use MSE as an indicator to evaluate the predictive effect of each model, and select the optimal model. Apply the optimal model to the test set for prediction to obtain prediction results of the tunneling speed. Compare the prediction results of the tunneling speed with the target values to evaluate the prediction results of the neural network algorithm.
2) The steps of Experiment 2 are as follows:
Use the FCM–ANN algorithm to train the training set, so as to obtain the clustering results of the training set and their corresponding neural network prediction models. Use the FCM–ANN algorithm to train the validation set, then the clustering results of the validation set can be obtained. According to the clustering results of the validation set, the neural network prediction models obtained by the training set is used to predict each cluster of corresponding data, then the prediction results of the validation set can be obtained. Use the cross-validation method to repeat Steps 1–3, and use MSE as an indicator to evaluate the effect of each prediction, and select the optimal model. Use the FCM–ANN algorithm to train the test set, then the clustering results of the test set can be obtained. According to the clustering results of the test set, the optimal neural network prediction models are used to predict each cluster of corresponding data, then the prediction results of the tunneling speed can be obtained.
Validation MSE | |||||
---|---|---|---|---|---|
Experiments | 1 | 2 | 3 | 4 | 5 |
1 | 319.22 | 255.40 | 230.16 | 355.52 | 193.76 |
43.77 | 20.16 | 26.24 | 31.43 | 42.94 |
As can be seen from
In this section, we use root mean square error (
The coefficient of determination
where
The value range of
In the prediction experiment of the tunneling speed, 8 independent experiments were performed, and the
The results of Experiment 1 and Experiment 2 can be compared by the above figure. As can be seen from
In this paper, we propose the FCM–ANN algorithm for functional data that is difficult to cluster effectively in traditional methods. The FCM–ANN algorithm is based on the FCM algorithm and uses the predicted value of artificial neural network as the clustering prototype to perform the iterative update of the algorithm. We first apply the algorithm to the synthetic datasets and discuss the effects of different number of samples, different number of attributes, and different noise on the clustering results under different membership thresholds. Then the algorithm is applied to the TBM operation data and compared with the method of modeling without clustering. The results show that the FCM–ANN algorithm can accurately and effectively predict tunneling speed. The future work will be mainly focused on replacing existing ANN models with more sophisticated neural network models while improving algorithm theory to make the iterative process more complete.
Parameter | ( |
( |
( |
( |
( |
( |
( |
( |
( |
( |
---|---|---|---|---|---|---|---|---|---|---|
Min. | 712.59 | 28.38 | 28.72 | 13.98 | 71.90 | 148.52 | 20.56 | 34.48 | 31.18 | 58.02 |
1st quartile | 1242.31 | 33.40 | 29.74 | 212.00 | 107.84 | 171.89 | 115.02 | 64.93 | 49.24 | 114.25 |
Mean | 1702.14 | 37.74 | 33.41 | 219.24 | 111.85 | 190.18 | 130.40 | 93.17 | 58.88 | 133.77 |
3rd quartile | 2675.15 | 41.16 | 36.58 | 255.49 | 135.82 | 235.63 | 142.17 | 97.95 | 97.95 | 155.89 |
Max. | 3058.41 | 51.39 | 38.10 | 322.31 | 197.64 | 290.47 | 160.32 | 115.13 | 119.14 | 202.20 |
Parameter | ( |
( |
( |
( |
( |
( |
( |
( |
( |
( |
Min. | 1.32 | 1.70 | 1.63 | 0.00 | 1.74 | 2.13 | 1.59 | 2.16 | 0.00 | 7.12 |
1st quartile | 3.20 | 2.75 | 3.49 | 0.00 | 4.35 | 4.39 | 4.15 | 4.40 | 7.67 | 72.59 |
Mean | 5.33 | 6.18 | 5.77 | 4.27 | 5.74 | 6.84 | 12.57 | 5.57 | 11.94 | 85.71 |
3rd quartile | 14.97 | 14.50 | 14.99 | 12.17 | 14.62 | 14.82 | 15.63 | 15.45 | 16.15 | 112.34 |
Max. | 57.19 | 57.29 | 65.54 | 63.21 | 46.43 | 54.31 | 68.20 | 57.57 | 21.61 | 171.73 |
Parameter | ( |
( |
( |
( |
( |
( |
( |
( |
( |
|
Min. | −4.31 | 0.00 | 1.47 | 1.30 | 1.59 | 1.51 | 1.37 | 0.22 | 30.53 | 0.00 |
1st quartile | −4.09 | 0.02 | 1.68 | 1.48 | 1.76 | 1.67 | 1.60 | 2.15 | 39.84 | 0.00 |
Mean | −4.03 | 0.04 | 1.92 | 1.54 | 2.21 | 2.10 | 1.70 | 2.57 | 41.20 | 0.00 |
3rd quartile | −3.97 | 0.10 | 2.00 | 1.61 | 2.29 | 2.17 | 1.78 | 3.53 | 43.00 | 1.48 |
Max. | −3.84 | 0.14 | 2.25 | 1.80 | 2.53 | 2.34 | 2.04 | 6.39 | 51.24 | 5.11 |
Parameter | ( |
( |
( |
( |
( |
( |
( |
( |
( |
( |
Min. | 0.00 | 0.00 | 0.00 | 1.22 | 1.44 | 1.69 | 1.15 | 0.52 | 1.20 | 0.00 |
1st quartile | 0.00 | 0.00 | 0.00 | 3.18 | 3.97 | 3.82 | 3.26 | 2.28 | 1.37 | 18.39 |
Mean | 0.00 | 0.00 | 0.00 | 12.47 | 5.70 | 5.36 | 5.17 | 2.70 | 1.70 | 25.99 |
3rd quartile | 0.94 | 0.00 | 0.00 | 18.35 | 15.02 | 15.09 | 15.01 | 3.67 | 1.80 | 31.85 |
Max. | 5.12 | 0.00 | 1.49 | 74.67 | 61.05 | 48.54 | 43.46 | 5.89 | 2.04 | 53.75 |
Parameter | ( |
( |
( |
( |
( |
( |
( |
( |
( |
( |
Min. | 0.00 | 314.98 | 335.83 | 306.87 | 291.67 | 0.36 | 22.79 | 4.53 | 0.19 | 10099.52 |
1st quartile | 3.00 | 685.91 | 720.30 | 702.94 | 683.34 | 8.21 | 35.54 | 23.89 | 0.20 | 14692.63 |
Mean | 3.00 | 1086.51 | 1073.90 | 1040.60 | 1050.74 | 10.17 | 49.59 | 45.41 | 7.37 | 15640.34 |
3rd quartile | 3.00 | 1334.75 | 1383.33 | 1368.67 | 1334.47 | 19.48 | 64.42 | 48.42 | 11.75 | 16340.86 |
Max. | 3.00 | 1805.12 | 1828.83 | 1795.89 | 1778.86 | 21.59 | 66.08 | 51.73 | 12.76 | 16967.07 |
Parameter | ( |
( |
||||||||
Min. | 0.50 | 0.55 | ||||||||
1st quartile | 0.64 | 33.67 | ||||||||
Mean | 1.90 | 48.02 | ||||||||
3rd quartile | 2.06 | 60.31 | ||||||||
Max. | 2.12 | 72.64 |