Open Access
ARTICLE
Interactive Restoration of Three-Dimensional Implicit Surface with Irregular Parts
1 Graduate School of Information Science and Engineering, Ritsumeikan University, Shiga, 525-8577, Japan
2 College of Information Science and Engineering, Ritsumeikan University, Shiga, 525-8577, Japan
* Corresponding Author: Jiayu Ren. Email:
Computer Modeling in Engineering & Sciences 2023, 136(3), 2111-2125. https://doi.org/10.32604/cmes.2023.025970
Received 08 August 2022; Accepted 08 November 2022; Issue published 09 March 2023
Abstract
Implicit surface generation based on the interpolation of surface points is one of the well-known modeling methods in the area of computer graphics. Several methods for the implicit surface reconstruction from surface points have been proposed on the basis of radial basis functions, a weighted sum of local functions, splines, wavelets, and combinations of them. However, if the surface points contain errors or are sparsely distributed, irregular components, such as curvature-shaped redundant bulges and unexpectedly generated high-frequency components, are commonly seen. This paper presents a framework for restoring irregular components generated on and around surfaces. Users are assumed to specify local masks that cover irregular components and parameters that determine the degree of restoration. The algorithm in this paper removes the defects based on the user-specific masks and parameters. Experiments have shown that the proposed methods can effectively remove redundant protrusions and jaggy noise.Keywords
Implicit representation is a common method for defining three-dimensional shapes and has advantages in many applications, such as shape modeling in computer graphics, physics-based simulation, material science, and medical imaging. One important application of implicit representation is shape modeling based on constructive solid geometry, skeletal modeling, and blending, among others [1,2]. Three-dimensional shapes can be effectively structured as combinations of primitives using algebraic operations. Another application of implicit representation is the shape modeling of obstacles for particle-based fluid simulation [3,4]. Implicit representation helps to efficiently estimate boundary force to fluid particles when the particles approach obstacles. It is also useful for shape modeling of complex materials, such as composite porous and molecular surfaces [5–7]. In these applications, such structures of complex materials and molecules are generated procedurally and defined as isosurfaces of scalar fields.
An implicit surface reconstruction from a set of surface points is a typical problem in computer graphics and several methods for the automatic generation of scalar fields have been proposed since the 1990s. Section 2 provides a brief review of the implicit surface reconstruction methods. However, as mentioned in several articles [8–12], the quality of reconstructed surfaces is sometimes poor depending on the density and accuracy of the given surface points. Defects, such as missing points, noise, and nonuniform sampling, often result in generated shapes with irregular components, such as redundant protrusions and noisy-liked jaggy components. Fig. 1 gives two examples of irregular implicit surfaces.
In this paper, we provide solutions for restoring irregular implicit surfaces. We focus on two major surface irregularities: redundant protrusions and jaggy noise. We assume that users specify masks in three-dimensional space and that restoration is performed only in the masked region. If a mask is specified to cover a redundant protrusion on a surface, our method updates the masked region by replacing the long protruding part with a round or flat-shaped surface. Users can control the roundness/flatness of the restored portion of the surface. The new scaler field within the mask is generated as the least-square solution of Laplace’s equation with both Dirichlet and Neumann conditions. The Dirichlet boundary condition is used to guarantee continuity at the border of the mask, while the Neumann boundary condition is used to control the roundness/flatness. Similarly, users can specify the region of restoration of jaggy noise by a mask that covers the region. Furthermore, users can control the degree of noise reduction by a parameter. Noise reduction is performed through low-resolution resampling within the mask. The final scaler field is generated by dense spline interpolation of the entire domain, with the spline ensuring continuity.
Shape representation can be performed in various ways. Traditionally, shapes can be represented by enumerating small geometric elements and allowing them to cover the surface, a technique known as explicit representation [15,16]. The advantage of this approach is that the surface can be extended arbitrarily based on the users’ needs. Another prospective approach for representing a shape is to find an appropriate field defined by the function
Existing work on implicit representations can be divided into two categories, namely, traditional mathematical methods and newly proposed learning-based methods. A typical traditional method is to generate the field
More recently, deep learning-based approaches have gained more popularity. In those approaches, SDFs [25–28] or occupancy fields [29–32] are learned via machine learning methods. In [25,33], estimated projection vectors are obtained using deep learning methods, such as regression forests [34]. The sign can be calculated using the projection vector and surface normal. Finally, the signed projection vectors can be used to approximate SDFs. Moreover, the surfaces can be represented with decision boundaries retrieved from neural networks [35]. In this approach, an occupancy network is constructed, and the probability of either taking zero or one at an arbitrary point is output. The occupancy field consists of zeros and ones based on the higher probability at the corresponding location. The shape is represented by the decision boundary that separates zeros and ones. Different forms of the expected shape, such as point clouds, predefined models, or images, can be used as input. A state-of-the-art survey is provided in [36].
Although some methods for creating implicit shapes have been proposed, the problem of irregular parts is a typical issue, as mentioned in Section 1. Our approach is to restore irregular surfaces a posteriori according to the interactive operations of users. There are some methods for modifying implicit surfaces interactively. An implicit surface manipulation method, known as WarpCurves [37], has been proposed. This method allows users to modify surfaces by manipulating user-specific curves on surfaces. Another approach for deforming implicit surfaces is free-form deformation [38]. In this method, the field is updated according to control points or curves specified on surfaces. An interactive manipulation approach based on typical mathematical operations, such as Boolean, blending, twisting, and bending, has also been proposed [39]. Various deformations can be performed as a combination of simple operations and real-time rendering. Our method focuses on restoration of unexpectedly generated shapes and is designed to effectively remove irregular parts, including redundant protrusions and jaggy noise, using simple operations. The restoration level can also be controlled by user-specific parameters. The geometric information that users should provide is only masks that cover the irregular parts. The next section discusses the reasons for each type of issue.
One of the common challenges in model creation is irregular parts that are unexpectedly generated, including curvature shaped redundant bulges and unexpectedly generated high frequency components. Defects such as missing points or unevenly displaced normal vectors may cause noisy or redundant problems. In this case, the interpolation may be misguided, resulting in either large curvatures or high-frequency components. As a result, redundant protruding parts or noisy parts are produced. Figs. 3 and 4 show some examples.
This section presents implicit surface restoration methods that remove unexpectedly generated redundant protrusions and jaggy noise. Assuming a scalar field,
where
4.1 Correction of Redundant Parts
The strategy of removing redundant parts is illustrated in Fig. 6. Assuming
P1: the field is as smooth as possible.
P2: the value is equivalent to
P3: the value decays rapidly from positive to negative at the edge penetrated by the base.
The property P3 is to control the speed of decay. In this approach, it is assumed that some protruded surfaces are unexpectedly generated as the result of limitations on scanning environment. It means that users can specify the flatness of the restored surfaces after removing the protrusions by controlling the speed of the decay.
To obtain a field that mostly satisfies the three properties, we propose the following equations:
where
In some special situations, the surface unexpectedly extends toward the border of
For practical computations, the above-mentioned formulation should be appropriately discretized. We apply a finite difference scheme to obtain the discrete linear least-square equations. Assuming the domain
where
where the notation
where
The uniqueness of the least-square solution is critical. As a result, the solution is guaranteed to be unique, as described below. According to Eq. (4), the matrix
Another issue is the choice of least-square solvers. Typically, singular value decompositions (SVD), QR and conjugate gradient least squares (CGLS) are used to solve linear least square problems [40]. In this study, the matrix is sparse and might be quite large depending on the size of the mask and the resolution of the discretization. The algorithms of SVD and QR use dense matrices and are inappropriate for users because they require a significant amount of computational time and memory space. However, the CGLS matches our case because the algorithm is designed for sparse and large matrices. Therefore, we adopt CGLS as the least-square solver throughout this paper.
All restored discrete values in Eq. (3) can be obtained by solving the least-square problem. Finally, the restored field,
where
Assuming the shape,
where
Currently, we have
where
This section presents some experimental examples to evaluate the proposed methods. Three-dimensional restoration examples are provided, using methods discussed in Section 4 and extended to three-dimensional formulations. Fig. 12 shows the data points of “Venus,” “Nefertiti,” and “Happy Buddha.” The implicit surfaces are generated using the grid of polynomial approach [13,16].
Fig. 13 shows restoration examples of removing redundant parts. The redundant part is considered as the protruding part at the bottom of the statue “Venus,” and the user-specified mask covers this area. With different settings of the gradient
Figs. 14 and 15 show restoration examples of partial and overall de-noise, respectively. The resolution of coarse sampling influences the intensity of de-noise, and setting a smaller number of sampling density results in a larger intensity of de-noise. However, the proposed method works well only when there are only a few smaller noisy parts, as shown in Fig. 14, in which case only a smaller mask is required to make corrections. If a de-noise process across the entire domain is required, the proposed method is less robust because it is difficult to control the intensity of de-noise to provide a well-balanced setting capable of removing noisy parts while preserving other details. As a result, there is a trade-off between the details of restoration and the intensity of de-noise. This limitation is vital in the case shown in Fig. 15.
This paper presents many methods to correct three-dimensional implicit surfaces with irregular components, such as redundant protruding parts and jaggy noise. Sections 1 and 2 introduce implicit surfaces and some state-of-the-art generation methods. Section 3 offers observations of irregular parts and their respective causes. Section 4 introduces the proposed correction methods. Experiments in Section 5 demonstrates that the proposed approaches can produce effective corrections, subject to some limitations that will be described further below.
In most situations, users have little knowledge regarding the properties of the field f(x). Good restoration requires the use of appropriate parameters, such as
Moreover, the proposed methods output the final result,
Funding Statement: This work was supported by JSPS KAKENHI Grant No. 21K11928.
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
References
1. Schmidt, R., Wyvill, B., Galin, E. (2005). Interactive implicit modeling with hierarchical spatial caching. International Conference on Shape Modeling and Applications 2005 (SMI' 05), pp. 104–113. https://doi.org/10.1109/SMI.2005.25 [Google Scholar] [CrossRef]
2. Marschner, S., Shirley, P., Ashikhmin, M., Gleicher, M., Hoffman, N. et al. (2015). Fundamentals of computer graphics. 4th edition. USA: A K Peters/CRC Press. [Google Scholar]
3. Kanetsuki, Y., Nakata, S. (2015). Moving particle semi-implicit method for fluid simulation with implicitly defined deforming obstacles. Journal of Advanced Simulation in Science and Engineering, 2(1), 63–75. https://doi.org/10.15748/jasse.2.63 [Google Scholar] [CrossRef]
4. Kanetsuki, Y., Nakata, S. (2020). Artificial force free boundaries: Particle-based fluid simulation with implicit surfaces. In: Computational and experimental simulations in engineering, pp. 207–222. Springer, Cham, Switzerland. https://doi.org/10.1007/978-3-030-27053-7_20 [Google Scholar] [CrossRef]
5. Sonon, B., François, B., Massart, T. J. (2015). An advanced approach for the generation of complex cellular material representative volume elements using distance fields and level sets. Computational Mechanics, 56(2), 221–242. https://doi.org/10.1007/s00466-015-1168-8 [Google Scholar] [CrossRef]
6. Yang, N., Wang, S., Gao, L., Men, Y., Zhang, C. (2017). Building implicit-surface-based composite porous architectures. Composite Structures, 173(4), 35–43. https://doi.org/10.1016/j.compstruct.2017.04.004 [Google Scholar] [CrossRef]
7. Parulek, J., Viola, I. (2012). Implicit representation of molecular surfaces. 2012 IEEE Pacific Visualization Symposium, pp. 217–224. Songdo. [Google Scholar]
8. Liu, Y., Song, Y., Yang, Z., Deng, J. (2017). Implicit surface reconstruction with total variation regularization. Computer Aided Geometric Design, 52(5), 135–153. https://doi.org/10.1016/j.cagd.2017.02.005 [Google Scholar] [CrossRef]
9. Kazhdan, M., Bolitho, M., Hoppe, H. (2006). Passion surface reconstruction. Symposium on Geometry Processing, pp. 61–70. Cagliari. https://doi.org/10.2312/SGP/SGP06/061-070 [Google Scholar] [CrossRef]
10. Wang, J., Yu, Z., Zhu, W., Cao, J. (2013). Feature-preserving surface reconstruction from unoriented, noisy point data. Computer Graphics Forum, 32(1), 164–176. https://doi.org/10.1111/cgf.12006 [Google Scholar] [CrossRef]
11. Liu, S., Wang, C. C. L. (2012). Quasi-interpolation for surface reconstruction from scattered data with radial basis function. Computer Aided Geometric Design, 27(7), 435–447. https://doi.org/10.1016/j.cagd.2012.03.011 [Google Scholar] [CrossRef]
12. Drake, K. P., Fuselier, E. J., Wright, G. B. (2022). Implicit surface reconstruction with a curl-free radial basis function partition of unity method. arXiv: 2101.05940. [Google Scholar]
13. Nakata, S., Aoyama, S., Makino, R., Hasegawa, K., Tanaka, S. (2012). Real-time isosurface rendering of smooth fields. Journal of Visualization, 15(2), 179–187. https://doi.org/10.1007/s12650-011-0119-5 [Google Scholar] [CrossRef]
14. Itoh, T., Nakata, S. (2015). Fast generation of smooth implicit surface based on piecewise polynomial. Computer Modeling in Engineering & Sciences, 107(3), 187–199. https://doi.org/10.3970/cmes.2015.107.187 [Google Scholar] [CrossRef]
15. Vemuri, B., Bolle, R. (1991). On three-dimensional surface reconstruction methods. IEEE Transactions on Pattern Analysis & Machine Intelligence, 13(1), 1–13. https://doi.org/10.1109/34.67626 [Google Scholar] [CrossRef]
16. Poursaeed, O., Fisher, M., Aigerman, N., Kim, V. G. (2020). Coupling explicit and implicit surface representations for generative 3D modeling. In: Lecture notes in computer science, vol. 12355, pp. 667–683. Springer, Cham, Switzerland. https://doi.org/10.1007/978-3-030-58607-2_39 [Google Scholar] [CrossRef]
17. Turk, G., O'Brien, J. F. (2002). Modelling with implicit surfaces that interpolate. ACM Transactions on Graphics, 21(4), 855–873. https://doi.org/10.1145/571647.571650 [Google Scholar] [CrossRef]
18. Carr, J. C., Beatson, R. K., Cherrie, J. B., Mitchell, T. J., Fright, W. R. et al. (2001). Reconstruction and representation of 3D objects with radial basis functions. Proceedings of the 28th annual conference on Computer Graphics and interactive techniques (SIGGRAPH ’01), pp. 67–76. Los Angeles. [Google Scholar]
19. Zeng, Y., Zhu, Y. (2022). Implicit surface reconstruction based on a new interpolation/approximation radial basis function. Computer Aided Geometric Design, 92(4), 102062. https://doi.org/10.1016/j.cagd.2021.102062 [Google Scholar] [CrossRef]
20. Ohtake, Y., Alexander, B., Alexa, M., Turk, G., Seidel, H. P. (2003). Multi-level partition of unity implicits. ACM Transactions on Graphics, 22(3), 463–470. https://doi.org/10.1145/882262.882293 [Google Scholar] [CrossRef]
21. Kazhdan, M. (2005). Reconstruction of solid models from oriented point sets. Proceedings of the Third Eurographics Symposium on Geometry Processing (SIGGRAPH ’05), pp. 73–82. Vienna. [Google Scholar]
22. Manson, J., Petrova, G., Schaefer, S. (2008). Streaming surface reconstruction using wavelets. Computer Graphics Forum, 27(5), 1411–1420. https://doi.org/10.1111/j.1467-8659.2008.01281.x [Google Scholar] [CrossRef]
23. Berger, M., Levine, J. A., Nonato, L. G., Taubin, G., Silva, C. T. (2013). A benchmark for surface reconstruction. ACM Transaction on Graphics, 32(2), 1–17. https://doi.org/10.1145/2451236.2451246 [Google Scholar] [CrossRef]
24. Berger, M., Tagliasacchi, A., Seversky, L. M., Alliez, P., Guennebaud, G. et al. (2017). A survey of surface reconstruction from point clouds. Computer Graphics Forum, 36(1), 301–329. https://doi.org/10.1111/cgf.12802 [Google Scholar] [CrossRef]
25. Erler, P., Guerrero, P., Ohrhallinger, S., Mitra, N. J., Wimmer, M. (2020). Points2Surf: Learning implicit surfaces from point clouds. European Conference on Computer Vision, pp. 108–124. Glasgow. [Google Scholar]
26. Liu, S., Guo, H., Pan, H., Wang, P., Tong, S. et al. (2021). Deep implicit moving least-squares functions for 3D reconstruction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1788–1797. [Google Scholar]
27. Ma, B., Han, Z., Liu, Y., Zwicker, M. (2021). Neural-pull: Learning signed distance functions from point clouds by learning to pull space onto surfaces. Proceedings of the 38th International Conference on Machine Learning, pp. 7246–7257. [Google Scholar]
28. Ma, B., Liu, Y., Han, Z. (2022). Reconstructing surfaces for sparse point clouds with on-surface priors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6315–6325. New Orleans. [Google Scholar]
29. Jia, M., Kyan, M. (2010). Learning occupancy function from point clouds for surface reconstruction. arXiv: 11378. [Google Scholar]
30. Martel, J. N. P., Lindell, D. B., Lin, C. Z., Chan, E. R., Monteiro, M. et al. (2021). ACORN: Adaptive coordinate networks for neural scene representation. arXiv: 2105.02788. [Google Scholar]
31. Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., Geiger, A. (2020). Convolutional occupancy networks. European Conference on Computer Vision, pp. 523–540. Glasgow. [Google Scholar]
32. Takikawa, T., Litalien, J., Yin, K., Kreis, K., Loop, C. et al. (2021). Neural geometric level of detail: Real-time rendering with implicit 3D shapes. arXiv: 2101.10994. [Google Scholar]
33. Ladický, L., Saurer, O., Jeong, S., Maninchedda, F., Pollefeys, M. (2017). From point clouds to mesh using regression. 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3913–3922. Venice. [Google Scholar]
34. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324 [Google Scholar] [CrossRef]
35. Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A. (2019). Occupancy networks: Learning 3D reconstruction in function space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4460–4470. Long Beach. [Google Scholar]
36. Huang, Z., Wen, Y., Wang, Z., Ren, J., Jia, K. (2022). Surface reconstruction from point clouds: A survey and a benchmark. arXiv: 2205.02413. [Google Scholar]
37. Sugihara, M., Wyvill, B., Schmidt, R. (2010). WarpCurves: A tool for explicit manipulation of implicit surfaces. Computers & Graphics, 34(4), 282–291. https://doi.org/10.1016/j.cag.2010.03.008 [Google Scholar] [CrossRef]
38. Eyiyurekli, M., Breen, D. (2010). Interactive free-form level-set surface-editing operators. Computers & Graphics, 34(5), 621–638. https://doi.org/10.1016/j.cag.2010.06.006 [Google Scholar] [CrossRef]
39. Reiner, T., Mückl, G., Dachsbacher, C. (2011). Interactive modeling of implicit surfaces using a direct visualization approach with signed distance functions. Computers & Graphics, 35(3), 596–603. https://doi.org/10.1016/j.cag.2011.03.010 [Google Scholar] [CrossRef]
40. Ascher, U. M., Greif, C. (2011). A first course in numerical methods. USA: Society for Industrial and Applied Mathematics. [Google Scholar]
41. Kim, D. B., Kang, E. C., Lee, K. H., Pajarola, R. B. (2006). Framework for adaptive sampling of point-based surfaces using geometry and color attributes. Proceedings of the International Conference on Computational Science, pp. 371–374. Reading, UK. [Google Scholar]
Cite This Article
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.