Learning Dual-Domain Calibration and Distance-Driven Correlation Filter: A Probabilistic Perspective for UAV Tracking

1  Introduction

Smart cities [1] are increasingly emerging as the primary outcome of urbanization and information technology integration as social intelligence advances. Six-generation (6G) technology [2] supports intelligent interconnection of human-machine objects and can generate potential application scenarios [3] such as full regional coverage by making full use of full spectrum resources. However, smart cities lack the support of spatial geographic information, and the application scenarios usually require terminal devices to provide more processing power and are more sensitive to transmission delays. However, current devices typically have poor processing power, and the centralized architecture of traditional cloud computing networks may result in poor transmission latency [4]. With the development of the Internet of Things (IoT) [5–10], information data is extracted in a timely manner using big data technologies, and edge computing servers are deployed at the edge of the network through mobile edge computing (MEC) [11–13]. This can effectively solve the problems of sensitive transmission delay and insufficient computing power of mobile terminals [14], and facilitate the provision of sufficient information for smart city construction [15–18]. In MEC networks, UAVs are highly flexible and operable and can dock space networks with ground networks to collaboratively build three-dimensional networks [19]. UAVs are able to dynamically adjust their deployment position in the network by tracking and adaptively responding to changes in the network environment [20]. UAVs’ visual tracking has been used for aerial refueling [21], aerial aircraft tracking [22], and target tracking [23,24]. Although there have been many successful research results in visual tracking, it still remains a difficult point to achieve high robustness and accuracy visual tracking with the influence of environmental factors such as occlusion, target in-plane/out-plane rotation, fast motion, blurring, lighting, and deformation, etc.

Recently, discriminative correlation filters (DCFs) [25] based tracking methods have received widespread attention [26–28]. However, these methods have caused the boundary effect problem with adverse effects on tracking performance while using the circular matrix which is calculated in the frequency domain to improve the model efficiency. Therefore, if it is affected by aberrance in challenging scenes. Compared with ordinary target tracking, the targets in the UAV domain are often very small and the existing DCF methods are definitely sensitive to handle the small targets in this case [29,30]. The existing DCF methods estimate the center position of the target, and the tracking model will continue to degenerate in the sequences [28]. Thus, if the target is small and similar to the background, UAV tracking is more prone to template drift and model degradation during the learning process.

A variety of visual tracking methods based on DCF have been proposed, and most of the successful strategies are applied to solve the above model degradation problem. For example, spatially regularized discriminative correlation filters (SRDCF) [31] and spatial-temporal regularized correlation filters (STRCF) [32] used spatial regularization and space-time regularization to solve boundary effects. In addition, STRCF mitigates the degradation of time filtering by introducing penalty terms in the context of SRDCF, which makes the tracker subject to more noise interference and reduces tracking reliability. And adaptive spatially-regularized correlation filters (ASRCF) [33] used the adaptive spatial regularization method to further improve the estimation ability of object perception and obtain more reliable filter coefficients. Many DCFs based trackers have suppressed the boundary effects by expanding the search area to obtain better tracking effects. The background-aware correlation filter (BACF) [34] used a cropping matrix to extract patches densely from the background and expand the search area at a lower computational cost. Group feature selection method for discriminative correlation filters (GFS-DCF) [35] advocates channel selection in DCF-based tracking and used an effective low-rank approximation [36]. The filter constraint here can smoothly realize the temporal information and adaptively integrate historical information to achieve group feature selection across channels and spatial dimensions. In order to get accurate tracking, a large margin object tracking method with circulant feature maps (LMCF) [37] and an attentional correlation filter network (ACFN) [38], tried to suppress the possible abnormalities through some measures. To solve the aberrance problem of response maps, the aberrance repressed correlation filter (ARCF) [39] added a regular term on the basis of BACF which can suppress the mutation of the response maps and used Euclidean distance to define the direct response approximation between the two response maps. In addition, in order to deal with the problem of boundary discontinuity and sample contamination caused by the cosine window, Li et al. [40] proposed a spatial regularization method to remove the cosine window and further introduced binary and Gaussian shape mask functions to eliminate the boundary problem.

For UAV tracking, there are also some special strategies in the DCF framework. Semantic-aware real-time correlation tracking framework (SARCT) [41] introduced a semantic-aware DCF-based framework for UAV tracking by combing the semantic segmentation module and sharing features for the object detection module and semantic segmentation. Also, the part-based mechanism was also combined with BACF in [42] and the Bayesian inference approach was proposed to achieve a coarse-to-fine strategy. AutoTrack [43] proposed a new method for online automatic adaptive learning of spatiotemporal regularization terms, which uses spatial local response map variants as spatial regularization, so that DCF focuses on the learning of the trusted part of the object, and determines the update processing of the filter according to the global response map variants. Overall, these existing methods share some commonalities. In particular, they often heavily rely on a presumed fixed spatial or temporal filter learning and fixed response map approximation.

In this paper, UAV tracking is sufficiently far away from the object in traditional tracking scenarios that the small target is fundamentally limited by less knowledge used in online learning and a more unbalanced sample problem. While many existing works focus on the sparsity [44] in spatial smooth or temporal smooth, the direct coupling of the several regularization terms used in several works like [34,41–43] is not a good choice, and the online filter learning will be obfuscated by redundancy. Therefore, it is interesting to ask the problem concerned in this work: what can we measure in online learning: probabilistic response fitting or direct response approximation?

From the perspective of actual and observed probability filter distribution in small targets for UAV tracking, the probabilistic response fitting in temporal smooth is more robust than the direct response approximation. Although Euclidean distance is useful, it is limited by the largest-scaled feature that would dominate the others. It is hard to judge the “best” similarity measure or a best-performing measure here. In general, we are the first work to transfer the statistical distance to correlation response fitting in the online tracking model. It also could shed light on the performance and behavior of measures in deep learning works like [45]. Although the insights of this work are the most similar to our work, the main truth is that the distribution assumption is different.

Furthermore, a novel unsupervised method called maximally divergent intervals (MDI) [46] searched for continuous intervals of time and region in a space containing anomalous events instead of analyzing the data point by point and attempted to use KL divergence and unbiased KL divergence to achieve better detection results. Not only that, but MDI also introduced JS divergence to demonstrate the superiority of the model and improve its scalability of the model. Similarly, Jiang et al. in [47] also used KL divergence to search for abnormal blocks in higher-order tensors. At the beginning of signal processing, the KL and JS divergence was used to fit the matching degree of the two distribution signals. Inspired by this, we consider the similarity search between the response maps of the filters as the curve fitting problem between the adjacent frame filters. It should be noted that the original Euclidean distance is not applicable because it is prone to filter under-fitting learning. Actually, the under-fitting problem had recently been mentioned in feature embedding problems like [48]. While it seems to be rather complicated to estimate a probabilistic response fitting by the above KL or JS procedure, the main benefit of estimating the mean and the covariance lies in the inherent regulation properties in our correlation filter response term.

While the KL divergence and JS divergence can achieve a good fitting effect. In addition, since the fitting object in the KL divergence is the response map of the adjacent frame filter to the image block [39], the KL divergence term can ensure that the filter has the highest similarity in the response map in the time domain. Therefore, our model uses a temporal-response calibrated filter based on the KL divergence. Since the KL divergence does not measure the distance between the two distributions in space, the KL divergence can be considered as a measure of the information loss of the response maps in the filter in our algorithm. Then we can find the target with the highest response value in the time domain according to the principle of function loss minimization, which is considered our tracking target. Experiments have shown that trackers based on temporal-response calibrated filters have greatly improved accuracy.

Based on the above-mentioned analysis, we propose a new model named AKLCF. Specifically, we use KL divergence fitting to mitigate the under-fitting of the Euclidean distance. In addition, to ensure the validity of the model, we add model confidence terms to avoid the variations of the model that occurred during the optimization process, which greatly improves the stability of the model. The algorithm flow chart of AKLCF is shown in Fig. 1a. Specifically, the multi-channel features are extracted from the previous frame of an image. We use KL divergence to calculate the similarity between response maps, and then build the learning model of AKLCF. At the same time, we further use the JS divergence to investigate the impact of the similarity measurement method on the tracking effect of related filters and propose the AJSCF model. The algorithm flow chart of AJSCF is shown in Fig. 1b. Notice that AJSCF is different from AKLCF in divergence. Compared with KL divergence, JS divergence solves the problem of asymmetry of KL divergence and avoids the “gradient disappearance” problem in the process of KL divergence fitting, thereby improving the tracking effect of the model.

Figure 1: The algorithm flowcharts of aberrance Kullback-Leibler (KL) divergence correlation filter (AKLCF) and aberrance Jensen-Shannon (JS) divergence correlation filter (AJSCF). The structure of the two flowcharts is the same and the main difference is that the former uses KL divergence to achieve similarity fitting, while the latter uses JS divergence

The UAV tracking will be further influenced by the imbalance of positive and negative samples during training DCF. Although kernelized correlation filter (KCF) [49] gave continuous weighting to samples with different offsets by continuous labeling, and could effectively detect target samples of interest to a certain extent and improve the accuracy of visual tracking, it does not solve the adverse effects of imbalance of positive and negative samples on visual tracking. Therefore, inspired by the collaborative distribution alignment (CDA) model [50], we propose a dual-domain Jensen-Shannon divergence correlation filter (DJSCF) model, which introduces dual-domain weighting into the data items of the loss function, uses a weighting method to suppress the influence of the imbalance of positive and negative samples and combines JS divergence to improve the accuracy of target tracking. The algorithm flow chart of DJSCF is shown in Fig. 2. DJSCF adds the sorting and selection of features on the basis of AJSCF, which is realized by dual-domain weighting, and effectively alleviates the problem of imbalance between samples.

Figure 2: The flow chart of dual-domain Jensen-Shannon divergence correlation filter (DJSCF)

The main contributions of this work are shown as follows:

•   To address the under-fitting of the response approximation in the Euclidean distance domain, we suggest the AKLCF model, which makes use of KL divergence fitting. The distortion brought on by noise can be efficiently reduced by the filter based on temporal response calibration. To further assure the model’s validity, we also include model confidence terms.

•   We introduce JS divergence to propose the AJSCF model, which validates the impact of measurement methods on tracking accuracy and enhances the richness and verifiability of article content, in order to address the asymmetry of KL divergence and the disappearance of gradient.

•   In order to address the issue of sample imbalance, we present a DJSCF model with JS divergence fitting that incorporates two-domain weighting due to two diagonally weighted matrices in the sparse data item. Several tests were carried out to demonstrate the excellence of our DJSCF.

2  Preliminaries

2.1 Distance Measurement

Target tracking mainly uses the tracking algorithm to select the one with the highest similarity to the target object as the target of the next frame, and then obtains the motion trajectory of the target object in the entire video sequence. Therefore, how accurately represent the similarity between the candidate block and the target block is an important factor that determines the accuracy of target tracking. As the research of target tracking and the difficulty of tracking tasks continue to increase, the distance similarity measurement methods used by tracking algorithms are also continuously improved. In image processing and analysis theory, common similarity measurement methods include Euclidean distance, block distance, checkerboard distance, weighted distance, Bart Charlie coefficient, Harsdorf distance, etc. Among them, Euclidean distance is the most applied and simplest.

The most commonly used mathematical expression of the Euclidean distance is the norm, which is mainly divided into four norms. Among them, the target tracking algorithm often uses the L1 norm and the L2 norm to extract sample sparsity characteristics and measure similarity between samples. With the complex changes in the target tracking scene, the Euclidean distance cannot meet the performance requirements of target tracking. Researchers try to apply similarity measurement algorithms in the signal domain to target trackings, such as KL divergence and JS divergence.

When the probability of an event is p (x), its information quantity is −log⁡(p(x)), and the expectation of information quantity is entropy. Relative entropy is also known as KL divergence. If we have two separate probability distributions P (x) and Q (x) for the same random variable x, we can use KL divergence to measure the difference between the two distributions. In machine learning, P is often used to represent the true distribution of the sample, and Q is used to represent the distribution predicted by the model. Then the KL divergence can calculate the difference between the two distributions, that is the loss value.

DKL(p||q)=∑iP(xi)logp(xi)q(xi)(1)

From the KL divergence formula, we can see that the closer the distribution of Q is to P (the more fitting the Q distribution is to P), the smaller the divergence value is, that is, the smaller the loss value is. Because the logarithmic function is convex, the value of KL divergence is non-negative. KL divergence is sometimes called KL distance, but it does not satisfy the property of distance. Because KL divergence is not symmetric and KL divergence does not satisfy triangle inequality. In machine learning, KL divergence can be used to evaluate the gap between label and predictions. In the process of optimization, we only need to pay attention to the cross entropy.

In probability statistics, JS divergence has the same ability to measure the similarity of two probability distributions as KL divergence. Its calculation method is based on KL divergence and inherits the non-negative nature of KL divergence. However, there is one important difference: JS divergence has symmetry. The formula of JS divergence is as follows: set two probability distributions as P and Q, M=0.5×(P+Q).

JS(P1||P2)=12KL(P||M)+12KL(Q||M)(2)

If the two distributions P and Q are far apart and completely do not overlap, then the KL divergence value is meaningless, while the JS divergence value is a constant log⁡2. This has a great impact on the deep learning algorithm based on gradient descent, which means that the gradient is 0, that is, the gradient disappears.

2.2 Dual-Domain Weighting

Due to the differences in background, light, sensor resolution, and other aspects, the data collected by UAVs in different scenes may be in different data domains. There are different numbers of positive samples and negative samples in each domain, and the number of negative samples (background) is far greater than the number of positive samples (target) in single-target tracking. The problem of sample imbalance will lead to the over-fitting of a large sample proportion, and the prediction will tend to classify with more samples. Therefore, there is a strong need to align the number of samples between different domains and train the data to find samples suitable for the target domain. Inspired by the trilateral weighted sparse coding scheme (TWSC) in [51] and the dual-domain collaboration in [50], we use the method of two-field weighting to increase the weight of samples, that is, to add a larger weight to the samples with a smaller number of samples, and to add a smaller weight to the samples with a larger number of samples.

In the model, we introduce two diagonal matrices, wa and wb, as weighted matrices. wa is a block diagonal matrix, with each block having the same diagonal element to describe sample attributes in different channels. wb is used to describe the sample variance in the corresponding patch. The formula can be expressed as:

Lm=12∥wax−wby∥F2(3)

where x is the positive sample, y is the negative sample. Our method also introduces the idea of dual-domain weighting, and the supporting materials show the specific processing operations of wa and wb. In other words, two weight matrices are introduced into the data fidelity term of the sparse coding frame to adaptively represent the statistics of positive and negative samples in each patch of the multi-channel, which can make better use of the sparse prior of the natural image.

3  The Dual-Domain Collaboration and Distances Correlation Filters

In this section, we first introduce the AKLCF and explain the role of KL divergence in the model tracking process. We further make the derivation of the optimization process in detail. Then, we introduce and derive DJSCF in detail, and introduce how dual-domain weighting and JS divergence can jointly improve the accuracy and robustness of target tracking. Considering that our model is based on the similarity of response maps, Fig. 2 shows the response effect graphs of the three methods. The trackers learn both positive samples (red sample) of objects and negative samples (green sample) in the background.

3.1 Aberrance Kullback-Leibler Divergence Correlation Filter (AKLCF) Model

Our AKLCF optimizes the response approximation in Euclidean distance to the KL divergence to achieve the curve fitting of the filters. In addition, the AKLCF increases the model confidence to improve the reliability of the model. Below we detail the construction of model functions. For simplicity, let Mk=∑d=1DBxkd∗fkd,Mk−1=∑d=1DBxk−1d∗fk−1d, where x∈RM×N×D, fk∈RM×N×D, fk−1∈RM×N×D, y∈RM×N. Here, we get the simplified form:

argminf12∥Mk−y∥F2+∑d=1D∥fkd∥22+r2∥Mk[ψp,q]−Mk−1[ψp,q]∥22(4)

Then, we replace the third term of Eq. (4) with the function R(Mk|Mk−1;xk−1) to obtain the loss function of AKLCF:

argminf12∥Mk−y∥F2+λ2∑d=1D∥fkd∥22+R(Mk|Mk−1;xk−1)(5)

As discussed above, in the formula with Eq. (5) add of is replaced by the Kullback-Leibler divergence term. In addition, we add a model confidence term to ensure the reliability of the model. Then we improve the loss functions by minimizing the following unconstraint objective:

argminf12∥Mk−y∥F2+λ2∑d=1D∥fkd∥22+β2Rmc+α2Rkl(6)

where β and α are penalty parameters.β2Rmc is the model confidence term and Rmc=∑d=1D(xk−1d)T∗ϕkd∗xk−1d, where ϕkd is a covariance matrix.

To better reflect the effect of the KL divergence term, we first analyze it in detail to obtain the specific expression of the KL divergence term. Since the KL divergence is a measure of the asymmetry of the difference between the two probability distributions. In our model, the KL divergence is used to measure the difference between the convolution matrix of the current frame and the previous frame, that is:

Rkl=Rkl(N(Mk,Φk)|N(Mk−1,Φk−1))(7)

where ϕk, ϕk−1 are covariance matrixes. Mk=∑d=1DBxkd∗fkd, Mk−1=∑d=1DBxk−1d∗fk−1d are the convolutions of the image matrix and filter coefficients, which denote the response maps from the current and previous frame respectively. k and k−1 represent the current and previous frame, respectively.

For convenience, we assume that the value vectors of filter and image blocks have multivariate Gaussian distributions, then the convolution matrix also has a Gaussian distribution, i.e., M∼N(M0,Φ), where M0 is the mean value vector of the convolution matrix distribution and Φ represents the covariance matrix of distribution. In the previous section, the function Mi=∑d=iDBxd∗fd indicates the convolution prediction of the class label fi and the sample xi in i−th dimension with the given multivariate Gaussian distribution. For better understanding, we regard each value fi in the i−th dimension as the models knowledge about the corresponding feature xi and regard the diagonal entry of covariance matrix Φi,i as the confidence of feature Mi. The smaller the value of Φi,i is, the more confident the learner is in the mean weight value Mi. Expect diagonal values, other covariance terms Φi,j can be took as the correlations between two weight value Mi and Mj for image feature ∑d=iDBxi∗fi and ∑d=jDBxj∗fj. In this way, we model parameter confidence for each element of along all channels with a diagonal Gaussian distribution. Thus, we construct the kullback-leibler divergence term based on the divergence between empirical distribution and probability distribution. As a result, we get the Rkl as following:

Rkl(N(Mk,Φk)∣N(Mk−1,Φk−1))=∑d=1D[logdetΦk−1ddetΦkd+tr((Φk−1−1)dΦkd)+(Mk−Mk−1)T(Φk−1−1)d(Mk−Mk−1)](8)

We substitute the specific expression of the model confidence term and Eq. (8) into Eq. (6) to get:

argminf12∥Mk−y∥F2+β2∑d=1D(xk−1d)T∗Φkd∗xk−1d+λ2∑d=1D∥fkd∥22+α2Rkl(N(Mk,Φk)|N(Mk−1,Φk−1))(9)

In general, the first term of the formula is data fidelity, the second is the space penalty, and the third and fourth terms are our improved contents, namely the model confidence term and the KL divergence term.

3.2 The Dual-Domain Jensen-Shannon Divergence Correlation Filter (DJSCF) Model

DJSCF adds dual-domain weighted matrices and JS divergence in the response map which need to be considered in the optimization process, therefore the construction of the loss function in DJSCF tracker is described in detail below:

argminf12∥∑d=1Dwa(Bxkd∗fkd)−wb⋅y∥F2+λ2∑d=1D∥fkd∥22+γ2∥∑d=1D(Bxk−1d∗fk−1d)[ψp,q]−∑d=1D(Bxkd∗fkd)[ψp,q]∥22(10)

Let Mk=∑d=1DBxkd∗fkd, Mk−1=∑d=1DBxk−1d∗fk−1d. The objective function is converted into:

argminf12∥waMk−wby∥F2+λ2∑d=1D∥fkd∥22+γ2∥(Mk−Mk−1)[ψp,q]∥22(11)

Similar to AKLCF, we use JS divergence and model confidence to improve the third term of the objective function, and get the objective function as follows:

argminf12∥waMk−wby∥F2+λ2∑d=1D∥fkd∥22+R(Mk|Mk−1;xk−1)(12)

It equals to:

argminf12∥waMk−wby∥F2+λ2∑d=1D∥fkd∥22+β2Rmc+α2RJS(13)

Since the JS divergence is proposed based on the KL divergence, we use two covariance matrices (Φkd and Φk−1d) to derive the DJSCF model according to the previous derivation, then RJS=RJS(N(Mk,ϕk)|N(Mk−1,ϕk−1). Finally, we get the loss function as follows:

argminf12∥waMk−wby∥F2+β2∑d=1D(xk−1d)T∗Φkd∗xk−1d+λ2∑d=1D∥fkd∥22+α2RJS(N(Mk,Φk)|N(Mk−1,Φk−1))(14)

The AJSCF model is a version that only replaces the KL divergence of AKLCF with JS divergence. The comparison between AJSCF and AKLCF shows the superiority of JS divergence in similarity fitting. At the same time, the latter DJSCF is obtained by adding dual-domain weighting based on AJSCF. And comparing AJSCF with DJSCF can verify the effect of dual-domain weighting on tracking performance.

3.3 ADMM Decomposition of AKLCF and DJSCF

AKLCF: The model in Eq. (9) is a convex problem and can be minimized with the ADMM algorithm to obtain the global optimal solution. Since Eq. (9) has a convolution calculation in the time domain, Pasival’s theorem is used to facilitate the calculation by converting the problem into the frequency domain. Then we get the objective function in the frequency domain as follows:

ε(fk)=12∥M^k−y^∥F2+β2∑d=1D(X^k−1d)TΦkdX^k−1d+λ2∑d=1D∥fkd∥22+α2Rkl(N(M^k,Φk)|N(M^k−1,Φk−1))(15)

where the superscript ∧ denotes a signal that has been performed discrete Fourier transformation (DFT). In Eq. (15), Mk^=∑d=1DXkd^(ID⊗BT)fkd. Xk is the matrix form of input sample. ID is an identity matrix whose size is D ×D. Operator ⊗ and superscript T indicate respectively Kronecker production and conjugate transpose operation. Similar to the BACF tracker, the alternative direction method of multipliers (ADMM) is used to speed up calculation and achieve a global optimal solution of Eq. (15) for its convexity. Therefore, the augmented Lagrangian form of Eq. (15) can be written as follows:

12∥X^kg^k−y^∥F2+ξ^T(g^k−∑d=1DN(ID⊗BT)fkd)+β2∑d=1D(X^k−1d)TΦkdX^k−1d+μ2∥g^k−∑d=1DN(ID⊗BT)fkd∥22+λ2∑d=1D∥fkd∥22+α2Rkl(N(M^k,Φk)|N(M^k−1,Φk−1))(16)

According to Eq. (8) and Pasival’s theorem, the KL divergence term in the frequency domain is expressed as follows:

Rkl(N(Mk^,Φk)|N(Mk^,Φk−1)=[logdetΦk−1detΦk+tr((Φk−1−1)Φk)+(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)](17)

Substitute Eq. (17) into Eq. (16), and using Pasival’s theorem, then we get the specific optimization function.

ε^(fk,g^k,Φk,ξ^)=12∥X^kg^k−y^∥F2+λ2∥fk∥22+ξ^T(g^k−N(ID⊗BT)fk)+μ2∥g^k−N(ID⊗BT)fk∥22+β2(X^k−1)TΦkX^k−1+α2[logdetΦk−1detΦk+tr(Φk−1−1Φk)+(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)](18)

where μ is a penalty factor. In the current frame, since the response map in the previous frame is already generated, Mk−1^ can be treated as a constant signal, which can simplify the further calculation. The ADMM algorithm is used to divide the optimization function into several sub-problems as follows:

{fk+1∗=argminf{λ2∥fk∥22+ξ^T(g^k−N(ID⊗BT)fk)+μ2∥g^k−N(ID⊗BT)fk∥22}g^k+1∗,Φk=argming,Φ{ξ^T(g^k−N(ID⊗BT)fk)+12∥X^kg^k−y^∥F2+μ2∥g^k−N(ID⊗BT)fk∥22+β2(X^k−1)TΦkX^k−1+α2[logdetΦk−1detΦk+tr(Φk−1−1Φk)+(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)]}(19)

DJSCF: Similar to AKLCF, according to Pasival’s theorem, we get:

ε(fk)=12∥WaX^k(ID⊗BT)fk−Wby^∥F2+λ2∥fk∥22+β2(X^k−1)TΦkX^k−1+α2RJS(N(M^k,Φk)|N(M^k−1,Φk−1))(20)

We get the optimization formulation:

ε(fk,g^k)=12∥WaX^kg^k−Wby^∥F2+λ2∥fk∥22+ζ^T(g^k−N(ID⊗BT)fk)+μ2∥g^k−N(ID⊗BT)fk∥22+β2X^k−1TΦkX^k−1+α2RJS(N(M^k,Φk)|N(M^k−1,Φk−1))(21)

According to the principle of JS divergence and Eq. (8), we get:

RJS(N(M^k,Φk)|N(M^k−1))=12Rkl(N(M^k,Φk)|N(M^k,Φk)+N(M^k−1,Φk−1)2)+12Rkl(N(M^k−1,Φk−1)|N(M^k,Φk)+N(M^k−1,Φk−1)2)=12(logdetΦk−1detΦk+tr(Φk−1−1Φk)+(M^k−M^k−1)TΦk−1−1(M^k−M^k−1))+12(logdetΦkdetΦk−1+tr(Φk−1Φk−1)+(M^k−M^k−1)TΦk−1(M^k−M^k−1))(22)

Then the optimization solution of DJSCF is implemented by ADMM and is divided into the following subproblems:

{fk+1∗=argminf{λ2∥fk∥22+ξ^T(g^k−N(ID⊗BT)fk)+μ2∥g^k−N(ID⊗BT)fk∥22}g^k+1∗,Φk=argming,Φ{12∥WaX^kg^k−Wby^∥F2+ξ^T(g^k−N(ID⊗BT)fk)+μ2∥g^k−N(ID⊗BT)fk∥22+β2(X^k−1)TΦkX^k−1+α2RJS}(23)

3.4 Solution of Sub-Problems of Two Models

From Eqs. (19) and (23), it is easy to find that except for the subproblem gk+1∗, the solutions of other subproblems in the two models are the same. Thus, we talk about the solution integration of the two models and only discuss them separately on subproblem gk+1∗.

Solution to subproblem fk+1∗:

The solution to subproblem fk+1∗ can be easily obtained as follows:

fk+1∗−(λ+μN)−1N(ID⊗BT)ξ^+μN(ID⊗BT)g^k−(λN+μ)−1(ξ+μgk)(24)

where

{gk=1N(ID⊗BT)g^kξ=1N(ID⊗BT)ξ^k(25)

Solution to subproblem: gk+1∗^ and Φk:

In AKLCF model, the augmented Lagrangian form is expressed as follows:

L(g^k+1∗,Φk)=12∥X^kg^k−y^∥F2+ξ^T(g^k−N(ID⊗BT)fk)+μ2∥g^k−N(ID⊗BT)fk∥22+β2(X^k−1)TΦkX^k−1+α2[logdetΦk−1detΦk+tr(Φk−1−1Φk)+(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)](26)

Using the ADMM algorithm, two sub-problems can be obtained, namely:

{L(g^k+1∗)=12∥X^kg^k−y^∥F2+ξ^T(g^k−N(ID⊗BT)fk)+μ2∥g^k−N(ID⊗BT)fk∥22+α2(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)L(Φk)=β2(X^k−1)TΦkX^k−1+α2[logdetΦk−1detΦk+tr(Φk−1−1Φk)+(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)](27)

Unfortunately, since subproblem gk+1∗^ contains Xk^gk^, it would be highly time consuming to solve gk+1∗^ subproblem and it is more difficult than to solve subproblem fk∗^. And the calculation of gk+1∗^ needs to be carried out in every ADMM iteration. Therefore, the sparsity of Xk^ is exploited. Each element of y^, i.e., y^(n),n=1,2,⋯,N, is solely dependent on each xk^(n)=[xt1^(n),xt2^(n),⋯,xtD^(n)]T and gt^(n)=[conj(gt1^(n)),conj(gt2^(n)),⋯,conj(gtD^(n))]T. Operator conj(.) denotes the complex conjugate operation.

The subproblem gk+1∗^ can be thus further divided into N smaller problems as follows solved over n=[1,2,⋯,N]:

g^k+1∗(n)=12∥X^k(n)g^k(n)−y^(n)∥F2+ξ^T(g^k(n)−N(ID⊗BT)f^k(n))+μ2∥g^k(n)−N(ID⊗BT)f^k(n)∥22+α2(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)(28)

Each smaller problem can be efficiently calculated and the solution is presented below:

g^k+1∗=11+α(x^kT(n)x^k(n)+μ1+αID)−1(x^k(n)y^(n)−ξ^(n)+μf^k(n)+∂Rkl∂g^k)(29)

where

{Rkl=logdetΦk−1detΦk+tr(Φk−1−1Φk)+(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)M^k=BX^kg^k(30)

Finally, the solution of gk+1∗ is expressed as follows:

g^k+1∗(n)=11+α(x^k(n)x^kT(n)+μ1+αID)−1(x^k(n)y^(n)−ξ^(n)+μf^k(n)+α2Φk−1−1(x^k(n)g^kT(n)x^k(n)+x^k−1(n)g^k−1(n)))(31)

According to the ADMM algorithm, the ϕk in Eq. (31) appears as ϕk−1, thus we only need to calculate the result of ϕk−1. We first obtain the derivation of ϕk from Eq. (26).

∂L∂Φk=α(−Φk−1+Φk−1−1)+βX^k−1TX^k−1=0(32)

Then the solution of ϕk−1 can be obtained.

Φk−1=Φk−1−1−(Φk−1−1)TX^k−1βαX^k−1TΦk−1−1I+βαX^k−1TΦk−1−1X^k−1(33)

In the actual calculation process, ϕk−1 is directly substituted into the calculation, thus we allow the solution for Φk−1 to produce a full matrix implicitly, and then project it to a diagonal matrix, where the non-zero off-diagonal entries are dropped. Unlike the AKLCF, DJSCF introduces the dual-domain weighted matrices, therefore the solution of gk∗^ involves the solutions of the weighted matrices Wa or Wb. It can be obtained from Eq. (23) as follows:

{L(g^k+1∗)=12∥X^kg^k−y^∥F2+ξ^T(g^k−N(ID⊗BT)fk)+α2[12(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)+12(M^k−M^k−1)TΦk−1(M^k−M^k−1)]+μ2∥g^k−N(ID⊗BT)fk∥22L(Φk)=β2(X^k−1)TΦkX^k−1+α4[logdetΦk−1detΦk+tr(Φk−1−1Φk)+(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)]+α4[logdetΦkdetΦk−1+tr(Φk−1Φk−1)+(M^k−M^k−1)TΦk−1(M^k−M^k−1)](34)

Similar to Eq. (28), the subproblem gk+1∗ of DJSCF also can be divided into N smaller problems as follows:

g^k+1∗(n)=12∥X^k(n)g^k(n)−y^(n)∥F2+ξ^T(g^k(n)−N(ID⊗BT)f^k(n))+μ2∥g^k(n)−N(ID⊗BT)f^k(n)∥22+α4(M^k−M^k−1)TΦk−1−1(M^k−M^k−1)+α4(M^k−M^k−1)TΦk−1(M^k−M^k−1)(35)

After iterative calculation, gk+1∗ and ϕk can be updated as follows:

g^k+1∗(n)=11+α(Wax^kT(n)x^k(n)+μ1+αID)−1[Wbx^k(n)y^(n)−ξ^(n)+μf^k(n)+α4(Φk−1+ΦkΦk−1Φk)(x^kT(n)g^k(n)x^k(n)−x^k−1(n)g^k−1(n))](36)