Computers, Materials & Continua DOI:10.32604/cmc.2021.017300 | |
Article |
Guided Intra-Patch Smoothing Graph Filtering for Single-Image Denoising
1College of Internet of Things Engineering, Hohai University, Changzhou, 213022, China
2School of Microelectronics and Control Engineering, Changzhou University, Changzhou, 213022, China
3Department of Electrical Engineering, University of Windsor, Ontario, N9B 3P4, Canada
*Corresponding Author: Yan Zhou. Email: yanzhou@hhu.edu.cn
Received: 26 January 2021; Accepted: 04 March 2021
Abstract: Graph filtering is an important part of graph signal processing and a useful tool for image denoising. Existing graph filtering methods, such as adaptive weighted graph filtering (AWGF), focus on coefficient shrinkage strategies in a graph-frequency domain. However, they seldom consider the image attributes in their graph-filtering procedure. Consequently, the denoising performance of graph filtering is barely comparable with that of other state-of-the-art denoising methods. To fully exploit the image attributes, we propose a guided intra-patch smoothing AWGF (AWGF-GPS) method for single-image denoising. Unlike AWGF, which employs graph topology on patches, AWGF-GPS learns the topology of superpixels by introducing the pixel smoothing attribute of a patch. This operation forces the restored pixels to smoothly evolve in local areas, where both intra- and inter-patch relationships of the image are utilized during patch restoration. Meanwhile, a guided-patch regularizer is incorporated into AWGF-GPS. The guided patch is obtained in advance using a maximum-a-posteriori probability estimator. Because the guided patch is considered as a sketch of a denoised patch, AWGF-GPS can effectively supervise patch restoration during graph filtering to increase the reliability of the denoised patch. Experiments demonstrate that the AWGF-GPS method suitably rebuilds denoising images. It outperforms most state-of-the-art single-image denoising methods and is competitive with certain deep-learning methods. In particular, it has the advantage of managing images with significant noise.
Keywords: Graph filtering; image denoising; MAP estimation; superpixel
Image denoising aims to remove noise from images, thereby benefitting the subsequent analysis and processing of images and videos. Current image denoising methods are divided into two categories: Model- and deep-learning-based methods. Prior to the success of deep learning methods, model-based methods have been used for decades to exploit the intrinsic attributes of images [1–3]. Most of them resolve the denoising problem using the internal prior of the target noisy image without considering other images. Therefore, they are often considered as single-image denoising methods. In contrast, deep-learning-based methods employ the external prior from other image databases to recover denoised images. Thus, they can be considered as data-driven methods that require numerous image data [4–6]. Flexible feature learning strategies are adopted in these methods via various deep networks. Because of the feature analysis of external images instead of the target image, deep-learning-based methods have a higher denoising performance than the model-based methods. However, in this study, we focus on the model-based denoising method assuming that only a single noisy image is provided.
In general, model-based denoising methods involve a filtering process, where the input and output data are noisy and denoised images, respectively. Thus, various filtering methods are used in different domains such as spatial, transform, and learned domains. In earlier studies, spatial-domain methods, such as bilateral filtering [7] and non-local means filtering [8], were used for the direct operation on pixels or patches. A spatial smoothing filter was used to remove noise-like components from the target image. However, since this smoothing procedure is only performed in local areas or patch groups, spatial filtering has a limited ability to exploit the statistical information of the entire image. To overcome this problem, transform-domain filters are presented by considering the directional structures in the images. Wavelet and Curvelet filters have been successfully employed for their adequate structural description of basis [9–11]. A popular denoising method, named block matching and three-dimensional (BM3D) filtering, was proposed using a collaborative spatial-wavelet filter [12,13]. It adopts a spatial filtering result to guide sequential wavelet filtering on similar patches. Unfortunately, none of the aforementioned filters are contented-based due to their fixed filter coefficients. Therefore, more advanced models, such as sparse representation and low-rank representation models [3,14–16], have been deployed in different learned domains in recent years. For example, a trilateral weighted sparse coding (TWSC) method is used to estimate data fidelity based on the sparse representation theory [17]. A low-rank approximation approach with adaptive regularizer learning (ARLLR) [18] is presented to shrink the eigenvalues of patches and is highly successful in image denoising. However, it is not yet considered as a filter-based model. The current study proves that the low-rank model is equivalent to a subspace graph filter [19]. The filtering procedure takes place in a graph subspace supported by the eigenvectors of the patch group.
Graph filtering is an essential component of graph signal processing. Its basic idea is to filter the input signal on the network nodes [20]. Once pixels or patches are chosen according to the nodes, graph filtering can adequately fit image denoising. Certain graph polynomial filtering methods are presented to employ various Laplacian matrix regularizers in the existing denoising model [21–23]. Several adaptive graph filtering methods are also proposed, by applying different coefficient shrinkage strategies in the graph-frequency domain. In a pioneer work, an idea lowpass graph filter was designed using a full shrinkage approach in the high graph-frequency band, achieving denoising performance comparable with that of BM3D [24–26]. Given a patch group, the study proves that the eigenvectors of the Laplacian matrix are a set of graph Fourier bases [27]. Recently, an adaptive weighted graph filtering (AWGF) method introduced an effective shrinkage approach in the entire band [19]. Although it theoretically builds a bridge from the existing low-rank model to graph filtering, its denoising performance is barely comparable with that of low-rank denoising methods. In the traditional AWGF method, the graph filter is prioritized, whereas patch attributes are seldom considered for image denoising.
Motivated by the recent progress, we propose a guided intra-patch smoothing AWGF (AWGF-GPS) method for single-image denoising. Our contributions are twofold. (1) Unlike AWGF that uses graph topology on patches, AWGF-GPS learns the superpixel graph to exploit the pixel smoothing attribute in patches. This operation forces the restored pixels to smoothly evolve in the local area, where both intra- and inter-patch relationships are utilized during patch restoration. (2) A guided-patch regularizer is incorporated into AWGF-GPS. The guided patch is obtained in advance using a maximum-a-posteriori (MAP) probability estimator. By considering the guided patch as a sketch of a denoised patch, AWGF-GPS effectively supervises the patch restoration procedure. Consequently, the reliability of the denoised patch is increased. Experiments demonstrate that our AWGF-GPS method suitably rebuilds denoising images. It outperforms most state-of-the-art model-based methods and is competitive with certain deep-learning methods.
We briefly review the existing AWGF model [19]. Given a noisy patch group,
where
where
Note that the noisy patch group,
We propose the AWGF-GPS method to employ both the self-similarity of patches and the local similarity of intra-patch pixels. Fig. 1 depicts the filtering flowchart. The superpixel is defined as a pixel set sharing the same location on the patches. A graph is learned using these superpixels to strengthen the smooth attributes of the neighboring pixels. Sequentially, the corresponding graph Fourier bases are obtained from the Laplacian matrix during the graph analysis. The guided patches are evaluated based on their corresponding noisy patches via a MAP estimator. Combined with the graph Fourier bases and the guided patches, the AWGF-GPS model is finally implemented to restore the denoised patches.
3.1 Graph Learning on Superpixels
We obtain the superpixels from the noisy patch group,
where
The optimization problem in Eq. (3) can be conveniently solved using the GSP Toolbox [31]. Then, the graph Laplacian matrix is given as
We employ the guided patches as a regularizer in our AWGF-GPS model. Here, a MAP estimator is used to generate the guided patch group,
In Eq. (4), symbols
where
where
where
Unfortunately, the guided group,
where
We estimate the mean patch group,
where the diagonal matrix,
The optimal value of M is further achieved under the noise control of
Using graph Fourier bases and guided patches, we define the AWGF-GPS model as
where
We should mention some aspects of the filtering procedure from Eqs. (13) to (14). Unlike the definition in Eq. (2), the noisy patch group,
To simplify Eq. (13), we transform the model by merging its first and third terms as
Subsequently, we cast Eq. (15) in the trace form as
where
Then, we provide the setting of the prior regularizer,
where
Because the shrinkage coefficient matrix,
where fi is the i-th diagonal entry of
Thus, the AWGF-GPS method completes its coefficient shrinkage procedure.
3.4 AWGF-GPS Denoising Framework
We present the iterative AWGF-GPS denoising framework in Algorithm 1. Given an intermediate noisy image,
First, we obtain the Laplacian matrix,
where
We compare the AWGF-GPS method with several state-of-the-art model-based denoising methods, including BM3D [12], TWSC [17], ARLLR [18], and AWGF [19]. A deep-learning image denoising method, named the fast and flexible denoising convolutional neural network (FFDNET) [5], is adopted to show the gap between the model-and deep-learning-based denoising methods. Moreover, because our method is derived from AWGF, the denoising parameters are also inherited from those in AWGF. However, for the weighted coefficients in Eq. (13), parameters
A denoised image comparison of the noise standard deviation of
The denoised image comparison of the noise standard deviation of
The statistical results of the PSNR and SSIM are listed in Tabs. 1 and 2. We use BM3D as a baseline, for it achieves an acceptable performance for all the images and noise levels. TWSC outperforms BM3D because it is a learned domain method. In the sparse representation model, dictionary learning aims to catch patch features as atoms, whereas sparse coding focuses on patch restoration. However, as the noise level increases, the performance of the TWSC degenerates significantly. In this case, the atoms are distorted by direct feature learning from the noisy image. ARLLR is better than the former methods. The shrinkage approach on the patch group is effective in dealing with noise because the relationship among patches is exploited. Our AWGF-GPS method performs the best among these model-based denoising methods. As previously indicated, its coefficient shrinkage strategy is presented on the superpixel graph, where both attributes, within and among the patches, are considered. The guided patch regularizer further enhances the denoising performance. We also note that our method may suffer from an over-smoothing problem. It is inappropriate to deal with images (e.g., Barbara) that contain strong textures. The AWGF-GPS model attempts to design a lowpass graph filter. Because the components of the texture are centralized in high graph-frequency bands, the AWGF-GPS filter is unsatisfactory. FFDNET is best in terms of the noise standard deviation of
We have proposed a guided intra-patch smoothing graph filtering method for single-image denoising. Unlike the traditional AWGF, which only focuses on coefficient shrinkage, the proposed AWGF-GPS method considers more image attributes for denoising. The similarities between patches and intra-patch pixels are exploited by introducing the superpixel operation. Moreover, the guided patches from the MAP estimator provide a reliable optimal direction for the AWGF-GPS model. This provides an additional way to supervise patch restoration during graph filtering. Experiments have demonstrated that the AWGF-GPS method outperforms several state-of-the-art model-based denoising methods and is comparable with certain deep-learning methods.
Funding Statement: This work is supported by Natural Science Foundation of Jiangsu Province, China [BK20170306] and National Key R&D Program, China [2017YFC0306100]. The initials of authors who received these grants are YZ and JL, respectively. It is also supported by Fundamental Research Funds for Central Universities, China [B200202217] and Changzhou Science and Technology Program, China [CJ20200065]. The initials of author who received these grants are YT.
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
1. X. Yang, Y. Xu, Y. Quan and H. Ji, “Image denoising via sequential ensemble learning,” IEEE Transactions on Image Processing, vol. 29, pp. 5038–5049, 2020. [Google Scholar]
2. K. Jin and S. Wang, “Image denoising based on the asymmetric Gaussian mixture model,” Journal of Internet of Things, vol. 2, no. 1, pp. 1–11, 2020. [Google Scholar]
3. F. Zhang, G. Yang and J. Xue, “Hyperspectral image denoising based on low-rank coefficients and orthonormal dictionary,” Signal Processing, vol. 177, no. 1, pp. 107738, 2020. [Google Scholar]
4. C. Tian, L. Fei, W. Zheng, Y. Xu, W. Zuo et al., “Deep learning on image denoising: An overview,” Neural Networks, vol. 131, no. 11, pp. 251–275, 2020. [Google Scholar]
5. K. Zhang, W. Zuo and L. Zhang, “FFDNet: Toward a fast and flexible solution for CNN-based image denoising,” IEEE Transactions on Image Processing, vol. 27, no. 9, pp. 4608–4622, 2018. [Google Scholar]
6. C. Tian, Y. Xu, Z. Li, W. Zuo, L. Fei et al., “Attention-guided CNN for image denoising,” Neural Networks, vol. 124, no. 1–2, pp. 117–129, 2020. [Google Scholar]
7. G. Wu, S. Luo and Z. Yang, “Optimal weighted bilateral filter with dual-range kernel for Gaussian noise removal,” IET Image Processing, vol. 14, no. 9, pp. 1840–1850, 2020. [Google Scholar]
8. R. Verma and R. Pandey, “Adaptive selection of search region for NLM based image denoising,” Optik, vol. 147, no. 1, pp. 151–162, 2017. [Google Scholar]
9. X. Liu and Y. Chen, “NLTV-Gabor-based models for image decomposition and denoising,” Signal Image and Video Processing, vol. 14, no. 2, pp. 305–313, 2020. [Google Scholar]
10. D. Wang, Y. Xiao and Y. Gao, “Image denoising method based on NSCT bivariate model and variational bayes threshold estimation,” Multimedia Tools and Applications, vol. 78, no. 7, pp. 8927–8941, 2019. [Google Scholar]
11. B. Khawla, R. Said and H. Abdeliah, “A wavelet denoising approach based on unsupervised learning model,” EURASIP Journal on Advances in Signal Processing, vol. 2020, no. 36, pp. 1–26, 2020. [Google Scholar]
12. K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080–2090, 2007. [Google Scholar]
13. A. A. Yahya, J. Tan, B. Su, M. Hu, Y. Wang et al., “BM3D image denoising algorithm based on an adaptive filtering,” Multimedia Tools and Applications, vol. 79, no. 4, pp. 20391–20427, 2020. [Google Scholar]
14. W. Dong, L. Zhang, G. Shi and X. Li, “Nonlocally centralized sparse representation for image restoration,” IEEE Transactions on Image Processing, vol. 22, no. 4, pp. 1620–1630, 2013. [Google Scholar]
15. H. G. Du, “Image denoising algorithm based on nonlocal regularization sparse representation,” IEEE Sensors Journal, vol. 20, no. 20, pp. 11943–11950, 2020. [Google Scholar]
16. Y. Li, G. Gui and X. Cheng, “From group sparse coding to rank minimization: A novel denoising model for low-level image restoration,” Signal Processing, vol. 176, pp. 107655, 2020. [Google Scholar]
17. J. Xu, L. Zhang and D. Zhang, “A trilateral weighted sparse coding scheme for real-world image denoising,” in Proc. European Conf. on Computer Vision, Munich, Germany, pp. 21–38, 2018. [Google Scholar]
18. X. Jia, X. Feng and W. Wang, “Adaptive regularizer learning for low rank approximation with application to image denoising,” in Proc. IEEE Int. Conf. on Image Processing, Seoul, South Korea, pp. 3096–3100, 2016. [Google Scholar]
19. Y. Chen, Y. Tang, L. Zhou, Y. Zhou, J. Zhu et al., “Image denoising with adaptive weighted graph filtering,” Computers, Materials & Continua, vol. 64, no. 2, pp. 1219–1232, 2020. [Google Scholar]
20. A. Ortega, P. Frossard, J. Kovačević, J. M. F. Moura and P. Vandergheynst, “Graph signal processing: Overview, challenges, and applications,” Proceedings of the IEEE, vol. 106, no. 5, pp. 808–828, 2018. [Google Scholar]
21. Y. Fang, H. Zhang and Y. Ren, “Graph regularised sparse NMF factorisation for imagery de-noising,” IET Computer Vision, vol. 12, no. 4, pp. 466–475, 2018. [Google Scholar]
22. X. Zeng, W. Bian, W. Liu, J. Shen and D. Tao, “Dictionary pair learning on Grassmann manifolds for image denoising,” IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 4556–4569, 2015. [Google Scholar]
23. W. Waheed and D. B. H. Tay, “Graph polynomial filter for signal denoising,” IET Signal Processing, vol. 12, no. 3, pp. 301–309, 2018. [Google Scholar]
24. A. C. Yağan and M. T. Özgen, “Spectral graph based vertex-frequency Wiener filtering for image and graph signal denoising,” IEEE Transactions on Signal and Information Processing over Networks, vol. 6, pp. 226–240, 2020. [Google Scholar]
25. Y. Tang, Y. Chen, N. Xu, A. Jiang and Y. Gao, “Image denoising via sparse approximation using eigenvectors of graph Laplacian,” in Proc. Visual Communications and Image Processing, Singapore, pp. 1–4, 2015. [Google Scholar]
26. H. Talebi and P. Milanfar, “Global image denoising,” IEEE Transactions on Image Processing, vol. 23, no. 2, pp. 755–768, 2014. [Google Scholar]
27. F. G. Meyer and X. Shen, “Perturbation of the eigenvectors of the graph Laplacian: Application to image denoising,” Applied and Computational Harmonic Analysis, vol. 36, no. 2, pp. 326–334, 2014. [Google Scholar]
28. Z. Yang, Z. Yang and D. Han, “Alternating direction method of multipliers for sparse and low-rank decomposition based on nonconvex nonsmooth weighted nuclear norm,” IEEE Access, vol. 6, no. 1, pp. 56945–56953, 2018. [Google Scholar]
29. Z. Yang, L. Fan, Y. Yang, Z. Yang and G. Gui, “Generalized singular value thresholding operator based nonconvex low-rank and sparse decomposition for moving object detection,” Journal of the Franklin Institute-Engineering and Applied Mathematics, vol. 356, no. 16, pp. 10138–10154, 2019. [Google Scholar]
30. V. Kalofolias, “How to learn a graph from smooth signals,” Artificial Intelligence and Statistics, vol. 51, pp. 920–929, 2016. [Google Scholar]
31. N. Perraudin, J. Paratte, D. Shuman, L. Martin, V. Kalofolias et al., “GSPBOX: A toolbox for signal processing on graphs,” Eprint Arxiv, vol. 61, no. 7, pp. 1644–1656, 2016. [Google Scholar]
32. M. E. Helou and S. Süsstrunk, “Blind universal bayesian image denoising with gaussian noise level learning,” IEEE Transactions on Image Processing, vol. 29, pp. 4885–4897, 2020. [Google Scholar]
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |