Computers, Materials & Continua DOI:10.32604/cmc.2022.022371 | |
Article |
Robust Frequency Estimation Under Additive Mixture Noise
1School of Computer and Communication Engineering, University of Science & Technology Beijing, Beijing, 100083, China
2School of Automation Science and Electrical Engineering, Beihang University, Beijing, 100191, China
3School of Information Engineering, Wuhan University of Technology, Wuhan, 430070, China
4Department of Energy and Public Works, Queensland, 4702, Australia
*Corresponding Author: Longting Huang. Email: huanglt08@whut.edu.cn
Received: 05 August 2021; Accepted: 01 December 2021
Abstract: In many applications such as multiuser radar communications and astrophysical imaging processing, the encountered noise is usually described by the finite sum of
Keywords: Frequency estimation; additive cauchy-gaussian noise; voigt profile; metropolis-hastings algorithm; cramér-rao lower bound
Heavy-tailed noise is commonly encountered in a variety of area such as wireless communication and image processing [1–8]. Typical models of impulsive noise are
In this work, the frequency estimation is considered in the presence of the additive Cauchy-Gaussian (ACG) noise [17,18], which is the sum of two variables with one following Cauchy distribution (
Because of the Voigt profile in the target PDF of the ACG noise, we choose the M-H algorithm as the sampling method. To improve the performance of the M-H algorithm, an updating criterion of proposal covariance is devised, with the use of the samples in a batch process. Since we assume all unknown parameters are independent, the proposal covariance is in fact a diagonal matrix. Therefore, the square of difference between the neighbor samples is employed as the diagonal elements of proposal covariance, referred to as the proposal variance. Meanwhile, a batch-mode method is utilized to make the proposal variance more accurate. As the proposal variance is updated only according to the samples in each iteration, this criterion can be extended to any other noise type such as the Gaussian and non-Gaussian processes.
The rest of this paper is organized as follows. We review the MCMC and M-H algorithm in Section 2. The main idea of the developed algorithm is provided in Section 3, where the PDF of the additive impulsive noise and posterior PDF of unknown parameters are also included. Then the Cramér-Rao lower bounds (CRLBs) of all unknown parameters are calculated in Section 4. Computer simulations in Section 5 are given to assess the performance of the proposed scheme. Finally, in Section 6, conclusions are drawn. Moreover, a list of symbols is shown in Tab. 1, which are appeared in the following.
2 Review of MCMC and M-H Algorithm
Before reviewing the M-H algorithm, some basic concepts, such as Markov chain should be introduced [29,30]. By employing several dependent random variables
where the probability of
Then the Markov chain is said to be stable if
with
Typical MCMC algorithms draw samples from the conditional PDF
In the following, the details of the MCMC method are provided in Tab. 2. It is worth to point out that the burn-in period is a term of an MCMC run before convergent to a stationary distribution.
Among typical MCMC methods, M-H algorithm is commonly employed, whose main idea [32] is drawing samples from a proposal distribution with a rejection criterion, instead of sampling from
which determines whether the candidate is accepted or not. It is noted that the proposal distributions are usually chosen as uniform, Gaussian or Student's t processes, which are easier to be sampled. The details of the M-H algorithm can be seen in Tab. 3.
In the M-H algorithm, to prove the stationary, we define a transition kernel [33] as
where
Then we have
According to [33], it can be proven that
In this algorithm, samples obtained in each iteration are closing to each other and can be highly correlated since M-H moves tend to be local moves. Asymptotically, the samples drawn from the Markov chain are all unbiased and all come from the target distribution.
Without loss of generality, the observed data
where
3.1 Posterior of Unknown Parameters
Here we investigate the posterior of unknown parameters. Before that, we first express the PDFs of noise terms
Then the PDF of the mixture noise
where
Let
Furthermore, it is also assumed in [34] that both
where
According to the PDF expression of ACG noise in (13), the conditional PDF of the observation vector
where
Assume that the priors for all unknown parameters
where
Due to the multimodality of the likelihood function, the maximum likelihood estimator cannot be employed and the high computational complexity of the grid search. Furthermore, other typical robust estimators, such as the
Therefore, to estimate parameters accurately, the M-H algorithm is utilized, whose details are provided in Tab. 3. To simplify the sampling process, the multivariate Gaussian distribution is selected as the proposal distribution
where
It is noted that the larger proposal variance will cause a faster convergence but possible oscillation around the correct value. While the smaller values of proposal variance lead to slower convergence but small fluctuation. Therefore, the choice of the proposal variance will significantly influence the performance of the estimator. In this paper, a batch-mode proposal variance selection criterion is developed.
To estimate
where L is also called the length of the batch-mode window. Then the proposal covariance
To state the criterion clearly, the details are also shown in Fig. 1.
To start the algorithm, the initial estimate of
where
Finally, utilize the definition of
Let
where
Due to the complicated integral of
where
To assess the proposed algorithm, several simulations have been conducted. The mean square frequency error (MSFE), referred to as
First of all, the choice of the batch-mode window length L for the proposal covariance matrix is studied. Here the density parameters of ACG noise are set to
Second, the convergence rate of the unknown parameters is investigated. Meanwhile, the burn-in period P can be determined, accordingly. In this test, the density parameters are identical to the previous test. Figs. 4 and 5 indicates the estimates of all unknown parameters in different iteration number l, which are
Finally, the MSFE of the proposed estimator is considered. In this test, all parameters are chosen as the same with the previous test. As there is no finite variance for ACG noise [18], the signal-to-noise is difficult to be defined. Therefore, here
In this paper, with the use of the M-H algorithm, a robust parameter estimator of a single sinusoid has been developed, in the presence of additive Cauchy-Gaussian noise. Meanwhile, a new proposal covariance updating criterion is also devised by employing the squared error of the batch-mode M-H samples. It is shown in simulation results that the developed estimator can attain the CRLB with a stationary M-H chain, indicating the accurate of our scheme. In the future work, the method can be extended to the signals with more complicated models.
Funding Statement: The work was supported by National Natural Science Foundation of China (Grant No. 52075397, 61905184, 61701021) and Fundamental Research Funds for the Central Universities (Grant No. FRF-TP-19-006A3).
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
1. A. M. Zoubir, V. Koivunen, Y. Chakhchoukh and M. Muma, “Robust estimation in signal processing: A tutorial-style treatment of fundamental concepts,” IEEE Signal Processing Magazine, vol. 29, no. 4, pp. 61–80, 2012. [Google Scholar]
2. Z. T. Li, W. Wei, T. Z. Zhang, M. Wang, S. J. Hou et al., “Online multi-expert learning for visual tracking,” IEEE Transactions on Image Processing, vol. 29, pp. 934–946, 2020. [Google Scholar]
3. Z. T. Li, J. Zhang, K. H. Zhang and Z. Y. Li, “Visual tracking with weighted adaptive local sparse appearance model via spatio-temporal context learning,” IEEE Transactions on Image Processing, vol. 27, no. 9, pp. 4478–4489, 2018. [Google Scholar]
4. Y. Fei, “A 2.7 GHz low-phase-noise LC-QVCO using the gate-modulated coupling technique,” Wireless Personal Communications, vol. 86, no. 2, pp. 1–11, 2015. [Google Scholar]
5. Y. Xue, Q. Li and F. Ling, “Teensensor: Gaussian processes for micro-blog based teen's acute and chronic stress detection,” Computer Systems Science and Engineering, vol. 34, no. 3, pp. 151–164, 2019. [Google Scholar]
6. F. Mallouli, “Robust em algorithm for iris segmentation based on mixture of Gaussian distribution,” Intelligent Automation & Soft Computing, vol. 25, no. 2, pp. 243–248, 2019. [Google Scholar]
7. J. Wang, Y. Yang, T. Wang, R. Sherratt and J. Zhang, “Big data service architecture: A survey,” Journal of Internet Technology, vol. 21, no. 2, pp. 393–405, 2020. [Google Scholar]
8. J. Zhang, S. Zhong, T. Wang, H. -C. Chao and J. Wang, “Block chain-based systems and applications: A survey,” Journal of Internet Technology, vol. 21, no. 1, pp. 1–4, 2020. [Google Scholar]
9. T. Zhang, A. Wiesel and M. S. Greco, “Multivariate generalized Gaussian distribution: Convexity and graphical models,” IEEE Transactions on Signal Processing, vol. 61, no. 16, pp. 4141–4148, 2013. [Google Scholar]
10. C. L. Nikias and M. Shao, “Signal processing with fractional lower order moments: Stable processes and their applications,” in Proc. of the IEEE, vol. 81, no. 7, pp. 986–1010, 1993. [Google Scholar]
11. K. Aas and I. H. Haff, “The generalized hyperbolic skew student's t-distribution,” Journal of Financial Econometrics, vol. 4, no. 2, pp. 275–309, 2006. [Google Scholar]
12. G. R. Arce, “NonGaussian models,” in Nonlinear Signal Processing: A Statistical Approach, 1st ed., New York, USA: John Wiley & Sons Inc., pp. 17–42, 2005. [Google Scholar]
13. D. A. Reynolds, “Gaussian mixture models,” in Encyclopedia of Biometrics, 1st ed., New York, USA: Springer-Verlag, pp. 659–663, 2009. [Google Scholar]
14. A. Swami, “Non-Gaussian mixture models for detection and estimation in heavy tailed noise,” in Proc. of Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP)), Istanbul, Turkey, pp. 3802–3805, 2000. [Google Scholar]
15. D. Herranz, E. E. Kuruoglu and L. Toffolatti, “An α-stable approach to the study of the P(D) distribution of unresolved point sources in CMB sky maps,” Astronomy and Astrophysics, vol. 424, no. 3, pp. 1081–1096, 2004. [Google Scholar]
16. J. Ilow, D. Hatzinakos and A. N. Venetsanopoulos, “Performance of FH SS radio networks with interference modeled as a mixture of Gaussian and alpha-stable noise,” IEEE Transactions on Communications, vol. 46, no. 4, pp. 509–520, 1998. [Google Scholar]
17. Y. Chen, E. E. Kuruoglu and H. C. So, “Estimation under additive Cauchy-Gaussian noise using markov chain monte carlo,” in Proc. of IEEE Statistical Signal Processing Workshop (SSP), Gold Coast, Australia, pp. 356–359, 2014. [Google Scholar]
18. Y. Chen, E. E. Kuruoglu, H. C. So, L. -T. Huang and W. -Q. Wang, “Density parameter estimation for additive Cauchy-Gaussian mixture,” in Proc. of IEEE Statistical Signal Processing Workshop (SSP), Gold Coast, Australia, pp. 205–208, 2014. [Google Scholar]
19. F. Kahrari, M. Rezaei and F. Yousefzadeh, “On the multivariate skew-normal Cauchy distribution,” Statistics & Probability Letters, vol. 117, pp. 80–88, 2016. [Google Scholar]
20. F. W. J. Olver, D. M. Lozier and R. F. Boisvert, “Parabolic Cylinder functions,” in NIST Handbook of Mathematical Functions, 1st ed., Cambridge, UK: Cambridge University Press, pp. 167–168, 2010. [Google Scholar]
21. S. M. Kay, “Maximum likelihood estimation,” in Fundamentals of Statistical Signal Processing: Estimation Theory, 1st ed., New Jersey: Prentice-Hall, pp. 157–218, 1993. [Google Scholar]
22. S. Fang, L. Huang, Y. Wan, W. Sun and J. Xu, “Outlier detection for water supply data based on joint auto-encoder,” Computers, Materials & Continua, vol. 64, no. 1, pp. 541–555, 2020. [Google Scholar]
23. F. Wang, Z. Wei and X. Zuo, “Anomaly iot node detection based on local outlier factor and time series,” Computers, Materials & Continua, vol. 64, no. 2, pp. 1063–1073, 2020. [Google Scholar]
24. M. Subzar, A. I. Al-Omari and A. R. A. Alanzi, “The robust regression methods for estimating of finite population mean based on srswor in case of outliers,” Computers, Materials & Continua, vol. 65, no. 1, pp. 125–138, 2020. [Google Scholar]
25. Y. Chen, E. E. Kuruoglu and H. C. So, “Optimum linear regression in additive Cauchy-Gaussian noise,” Signal Processing, vol. 106, no. 1, pp. 312–318, 2015. [Google Scholar]
26. G. L. Bretthorst, “Estimating the parameters,” in Bayesian Spectrum Analysis and Parameter Estimation, 1st ed., New York, USA: Springer-Verlag, pp. 43–54, 1988. [Google Scholar]
27. C. Bishop, “Sampling methods,” in Pattern Recognition and Machine Learning, 1st ed., New York, USA: Springer, pp. 523–558, 2006. [Google Scholar]
28. C. Andrieu, N. D. Freitas, A. Doucet and M. I. Jordan, “An introduction to MCMC for machine learning,” Machine Learning, vol. 50, no. 1–2, pp. 5–43, 2003. [Google Scholar]
29. J. C. Spall, “Estimation via markov chain monte carlo,” IEEE Control Systems, vol. 23, no. 2, pp. 34–45, 2003. [Google Scholar]
30. C. M. Grinstead and J. L. Snell, “Markov chains,” in Introduction to Probability, 2nd ed., Rhode Island, USA: American Mathematical Society, pp. 405–470, 2012. [Google Scholar]
31. C. Robert and G. Casella, “Metropolis-Hastings algorithms,” in Introducing Monte Carlo Methods with R, 1st ed., New York, USA: Springer Verlag, pp. 166–198, 2009. [Google Scholar]
32. S. Chib and E. Greenberg, “Understanding the metropolis-hastings algorithm,” The American Statistician, vol. 49, no. 4, pp. 327–335, 1995. [Google Scholar]
33. A. T. Cemgil, “A tutorial introduction to Monte Carlo methods, Markov chain Monte Carlo and particle filtering,” in Academic Press Library in Signal Processing, Volume 1: Signal Processing Theory and Machine Learning, 1st ed., Oxford, UK: Elsevier Science & Technology, pp. 1065–1114, 2013. [Google Scholar]
34. R. Kohn, M. Smith and D. Chan, “Nonparametric regression using linear combinations of basis functions,” Statistics and Computing, vol. 11, no. 4, pp. 313–322, 2001. [Google Scholar]
35. T. H. Li, “A nonlinear method for robust spectral analysis,” IEEE Transactions on Signal Processing, vol. 58, no. 5, pp. 2466–2474, 2010. [Google Scholar]
36. Y. Li and G. Arce, “A maximum likelihood approach to least absolute deviation regression,” EURASIP Journal on Applied Signal Processing, vol. 2004, no. 12, pp. 1762–1769, 2004. [Google Scholar]
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |