Implicit Filtering for Learning Neural Signed Distance Functions from 3D Point Clouds

This section will briefly describe the concepts we used in our implicit filtering. We focus on the SDF $f:\bm{R}^{3}\rightarrow\bm{R}$ inferred from the point cloud $\bm{P}=\{\bm{p}_{i}|\bm{p}_{i}\in\bm{R}^{3}\}_{i=1}^{N}$ without ground truth signed distances and normals. $f$ predicts a signed distance $s\in\bm{R}$ for an arbitrary query point $\bm{q}$, as formulated by $s=f_{\theta}(\bm{q})$, where $\theta$ denotes the parameters of the neural network.

The level set $\mathcal{S}_{d}$ of SDF is defined as a set of continuous query points with the same signed distance $d$, formulated as $\mathcal{S}_{d}=\{\bm{q}|f_{\theta}(\bm{q})=d\}$. The goal of our implicit filtering is to smooth each level set with geometry details. Then we can extract the zero level set as a mesh by running the marching cubes algorithm [27].

Filtering for 2D images replaces the intensity of each pixel with the weighted intensity values from nearby pixels. Different from images, the resolution of implicit fields is infinite and we need to find the neighborhood on each level set for filtering. By minimizing the following loss function,

we can approximate that all points in $\bm{P}$ are located on level set $\mathcal{S}_{0}$, which makes it feasible to find neighbor points on $\mathcal{S}_{0}$. For a given point $\bm{\bar{p}}$ on $\mathcal{S_{\text{0}}}$, one simple strategy of filtering is to average positions of neighbor points $\mathcal{N}(\bm{\bar{p}},\mathcal{S}_{0})\subset\bm{P}$ on $\mathcal{S}_{0}$ by a Gaussian filter based on relative positions as follows:

where the Gaussian function $\phi$ is defined as $\phi(||\bm{\bar{p}}-\bm{p}_{j}||)=\text{exp}\left(-\frac{||\bm{\bar{p}}-\bm{p}_{j}||^{2}}{\sigma_{p}^{2}}\right).$

However, as depicted in Fig. 2, it is evident that this weighted mean position yields excessively smooth surfaces, causing sharp features and details to be further obscured. To keep the geometric details, our filtering operator suggests measuring the projection distance to the gradient of neighbor points as shown in Fig. 2 and Fig. 3(b). When calculating weights, it is vital to account for both the impact of relative positions and the gradient similarity. Following the principles of bilateral filtering, to compute the filtered point for $\bm{\bar{p}}$, we simply need to minimize the following distance equation:

where the gradient $\bm{n}_{\bar{p}}$, $\bm{n}_{p_{j}}$ and the Gaussian function $\psi$ are defined as $\bm{n}_{\bar{p}}=\frac{\nabla f_{\theta}(\bm{\bar{p}})}{||\nabla f_{\theta}(\bm{\bar{p}})||},\bm{n}_{\bm{p}_{j}}=\frac{\nabla f_{\theta}(\bm{p}_{j})}{||\nabla f_{\theta}(\bm{p}_{j})||},\psi(\bm{n}_{\bar{p}},\bm{n}_{p_{j}})=\text{exp}\left(-\frac{1-\bm{n}_{\bar{p}}^{T}\bm{n}_{p_{j}}}{1-\text{cos}(\sigma_{n})}\right).$

In addition to projection to the gradient $\bm{n}_{p_{j}}$, we observe that the projection distance to $\bm{n}_{\bar{p}}$ can assist in learning a more stable gradient for point $\bar{p}$ which is also adopted in EAR[16]. Taking into account the bidirectional projection, our final bilateral filtering operator can be formulated as follows:

Although similar filtering methods have been widely studied in applications such as point cloud denoising and resampling[48, 16], there are two critical problems when applying these methods in implicit fields:

1. 1.

   Filtering the zero level set needs to sample points on the level set $\mathcal{S}_{0}$, which necessitates the resolution of the equation $f_{\theta}=0$, or the utilization of the marching cubes algorithm [27]. Both methods pose challenges in achieving fast and uniform point sampling. For the randomly sampled point $\bm{q}$ on non-zero level set $\mathcal{S}_{f_{\theta}(\bm{q})}$, we can also not filter this level set since there are no neighbor points on $\mathcal{S}_{f_{\theta}(\bm{q})}$.

2. 2.

   The normals utilized in our filtering are derived from the gradients of the neural network $f_{\theta}$. While the network typically offers reliable gradients, we may find that $\nabla f_{\theta}=0$ is also the optimal solution to the minimum value of Eqs. 3 and 4. This degenerate solution is unexpected, as it implies a scenario where there is no surface when the gradient is zero everywhere.

We will focus on addressing the two issues in the subsequent sections.

Inspired by NeuralPull [4], we can pull a query point to the zero level set by the gradient of the neural network $f_{\theta}$. For a given query point $\bm{q}$ as input, the pulled location $\hat{\bm{q}}$ can be formulated as follows:

The point $\bm{q}$ and $\hat{\bm{q}}$ lie respectively on level set $\mathcal{S}_{f_{\theta}(q)}$ and $\mathcal{S}_{0}$ as illustrate in Fig. 4(b). By adopting the sampling strategy in NeuralPull, we can generate samples $\bm{Q}=\{\bm{q}_{i}|\bm{q}_{i}\in\bm{R}^{3}\}_{i=1}^{M}$ on different level sets near the surface and pull them to $\mathcal{S}_{0}$ by Eq. 5, to obtain $\hat{\bm{Q}}=\{\hat{\bm{q}}_{i}|\hat{\bm{q}}_{i}=\bm{q}_{i}-f_{\theta}(\bm{q}_{i})\nabla f_{\theta}(\bm{q}_{i})/||\nabla f_{\theta}(\bm{q}_{i})||,\bm{q}_{i}\in\bm{Q}\}_{i=1}^{M}$. Hence, we can filter the zero level set by minimizing Eq. 4 across all pulled query points $\hat{\bm{Q}}$, which is equivalent to optimizing the following loss:

where for each $\hat{\bm{q}}\in\hat{\bm{Q}}$, $\mathcal{N}(\hat{\bm{q}},\mathcal{S}_{0})$ denotes finding the neighbors of $\hat{\bm{q}}$ within the input points $\bm{P}$, since $\bm{P}$ is assumed to be located on $\mathcal{S}_{0}$.

This filtering mechanism can be easily extended to non-zero level sets in a similar inverse manner. To be more specific, as for level set $S_{f_{\theta}(\bm{q})}$, the neighbor points for query point $\bm{q}\in\bm{Q}$ are required. These points should lie on the level set $S_{f_{\theta}(\bm{q})}$ same as $\bm{q}$, allowing us to filter the level set $S_{f_{\theta}(\bm{q})}$ using the same filter as described in Eq. 4.

However, obtaining $\mathcal{N}(\bm{q},S_{f_{\theta}(q)})$ in $\bm{P}$ is not feasible, since all input points $\bm{P}$ are situated on the zero level set instead of the $S_{f_{\theta}(\bm{q})}$ level set. To address this issue, we propose a technique for identifying neighbors of $\bm{q}$ on level set $S_{f_{\theta}(\bm{q})}$, by projecting the input points $\bm{P}$ inversely onto the specific level set $S_{f_{\theta}(\bm{q})}$ based on the gradient, as depicted in Fig. 4(b). The projected neighbor points can be represented as in Eq. 7. Filtering across multiple level sets helps to enhance the performance of our method by optimizing the consistency between different level sets within the SDF field, We further showcase this evidence in the ablation study detailed in Section Sec. 4.4.

Based on the above analysis, we can filter the level sets $S_{f_{\theta}(\bm{q})}$ by minimizing Eq. 4 over all sample points $\bm{Q}$ through Eq. 7, equivalent to optimizing the following loss:

It is worth noting that for a fixed query point $\bm{q}$, the pulled query point $\hat{\bm{q}}$ dynamically changes when training the neural network, which results in a time-consuming process to repeatedly conduct neighbor searching for $\hat{\bm{q}}$. To handle this matter, we substitute the $\mathcal{N}(\hat{\bm{q}},\mathcal{S}_{0})$ with $\mathcal{N}(NN(\bm{q}),\mathcal{S}_{0})$, where $NN(\bm{q})$ denotes the nearest point of $\bm{q}$ within the point cloud $\bm{P}$ as shown in Fig. 5. While this substitution may introduce a slight bias for training, it also ensures the neighbor points are close to $\hat{\bm{q}}$, therefore this trade-off between efficiency and accuracy is reasonable.

The other problem of implicit filtering is gradient degeneration. Overfitting the neural network requires the SDF to be geometrically initialized. We can consider the initialized implicit field as the noisy field and apply our filter directly to train the network from the beginning to fit the raw point cloud by removing the ‘noise’. However, if the denoise target is too complex, gradient degeneration will occur during the training process. Therefore, we need to add a constraint to the gradient of the SDF.

There are two ways for training the neural network to pull query points onto the surface based on NeuralPull [4] and CAP-UDF [49]. One is minimizing the distance between the pulled point $\hat{\bm{q}}$ and the nearest point $NN(\bm{q})$ as formulated below:

The other is minimizing the Chamfer distance between moved query points and the raw point cloud:

A stable SDF can be trained by the losses above since they are trying to move the query points to be in the same distribution with the point cloud, which can provide the constraint for our implicit filter. Here we choose $L_{CD}$ since the filtered points are likely not the nearest points and $L_{CD}$ is a more relaxed constraint.

Finally, our loss function is formulated as:

where $\alpha_{1},\alpha_{2}$, and $\alpha_{3}$ is the balance weights for our implicit filtering loss.

We employ a neural network similar to OccNet [31] and the geometric network initialization proposed in SAL[1] with a smaller radius the same as GridPull[10] to learn the SDF. We use the strategy in NeuralPull[4] to sample queries around each point $\bm{p}$ in $\bm{P}$. We set the weight $\alpha_{3}$ to 10 to constrain the learned SDF and $\alpha_{1}$ and $\alpha_{2}$ to 1. The parameters $\sigma_{n},\sigma_{p}$ are set to $15^{\circ},\max_{\bm{p}_{j}\in\mathcal{N}(\bm{\bar{p}},\mathcal{S}_{f_{\theta}(\bar{p})})}(||\bar{\bm{p}}-\bm{p}_{j}||)$ respectively.