Open Access iconOpen Access

ARTICLE

Robust Symmetry Prediction with Multi-Modal Feature Fusion for Partial Shapes

Junhua Xi1, Kouquan Zheng1, Yifan Zhong2, Longjiang Li3, Zhiping Cai1,*, Jinjing Chen4

1 National University of Defense Technology, Changsha, Hunan, China
2 Jiangxi University of Finance and Economics, Jiangxi, China
3 Unit 78111 of Chinese People’s Liberation Army, Chengdu, Sichuan, China
4 Sungkyunkwan University, Korea

* Corresponding Author: Zhiping Cai. Email: email

Intelligent Automation & Soft Computing 2023, 35(3), 3099-3111. https://doi.org/10.32604/iasc.2023.030298

Abstract

In geometry processing, symmetry research benefits from global geometric features of complete shapes, but the shape of an object captured in real-world applications is often incomplete due to the limited sensor resolution, single viewpoint, and occlusion. Different from the existing works predicting symmetry from the complete shape, we propose a learning approach for symmetry prediction based on a single RGB-D image. Instead of directly predicting the symmetry from incomplete shapes, our method consists of two modules, i.e., the multi-modal feature fusion module and the detection-by-reconstruction module. Firstly, we build a channel-transformer network (CTN) to extract cross-fusion features from the RGB-D as the multi-modal feature fusion module, which helps us aggregate features from the color and the depth separately. Then, our self-reconstruction network based on a 3D variational auto-encoder (3D-VAE) takes the global geometric features as input, followed by a prediction symmetry network to detect the symmetry. Our experiments are conducted on three public datasets: ShapeNet, YCB, and ScanNet, we demonstrate that our method can produce reliable and accurate results.

Keywords

Symmetry prediction; multi-modal feature fusion; partial shapes

Cite This Article

APA Style
Xi, J., Zheng, K., Zhong, Y., Li, L., Cai, Z. et al. (2023). Robust Symmetry Prediction with Multi-Modal Feature Fusion for Partial Shapes. Intelligent Automation & Soft Computing, 35(3), 3099–3111. https://doi.org/10.32604/iasc.2023.030298
Vancouver Style
Xi J, Zheng K, Zhong Y, Li L, Cai Z, Chen J. Robust Symmetry Prediction with Multi-Modal Feature Fusion for Partial Shapes. Intell Automat Soft Comput. 2023;35(3):3099–3111. https://doi.org/10.32604/iasc.2023.030298
IEEE Style
J. Xi, K. Zheng, Y. Zhong, L. Li, Z. Cai, and J. Chen, “Robust Symmetry Prediction with Multi-Modal Feature Fusion for Partial Shapes,” Intell. Automat. Soft Comput., vol. 35, no. 3, pp. 3099–3111, 2023. https://doi.org/10.32604/iasc.2023.030298



cc Copyright © 2023 The Author(s). Published by Tech Science Press.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1068

    View

  • 804

    Download

  • 0

    Like

Share Link