An Intrinsic Vector Heat Network

Given a 2-manifold surface $\mathcal{M}$ embedded in $\mathbb{R}^{3}$, the tangent plane $T_{p}^{M}$ for a given point $p\in\mathcal{M}$ is a 2D space that is orthogonal to the surface normal of $\mathcal{M}$ at point $p$. A tangent plane $T_{p}^{M}$ can be defined by the spanning of an arbitrary choice of two orthogonal basis vectors ${\mathbf{e}}_{0},{\mathbf{e}}_{1}$. A 2D tangent vector ${\mathbf{u}}\in T_{p}^{M}$ can then be expressed as a linear combination of bases vectors ${\mathbf{u}}=u_{0}{\mathbf{e}}_{0}+u_{1}{\mathbf{e}}_{1}$. For computational convenience, these tangent vectors ${\mathbf{u}}$ are often represented as a complex number ${\mathbf{u}}=u_{0}+\imath u_{1}\in\mathbb{C}$ of their coefficients $u_{0},u_{1}$ (Knöppel et al., 2013; Vaxman et al., 2016). Note that the coefficients $u_{0},u_{1}$ will change, by a coordinate transformation, depending on the choice of tangent bases ${\mathbf{e}}_{0},{\mathbf{e}}_{1}$ in order to represent the same tangent vector. Since the choice of bases is arbitrary, an important invariance for tangent vector processing is to guarantee that the method is independent of the choice of bases (see Figure 1).

An edge vector ${\mathbf{e}}_{ij}={\mathbf{v}}_{j}-{\mathbf{v}}_{i}\in\mathbb{R}^{3}$ between two adjacent vertices $i,j$ with locations ${\mathbf{v}}_{i},{\mathbf{v}}_{j}\in\mathbb{R}^{3}$ can be expressed as a tangent vector in $T_{i}^{M}$ and $T_{j}^{M}$ via the logarithmic map (sometimes alternately referred to as the exponential map) (Schmidt et al., 2006). A simple way to compute the logarithmic map is to represent the edge vector ${\mathbf{e}}_{ij}$ in the polar coordinate $(l_{ij},\theta_{j})$ of the tangent plane $T_{i}^{M}$, where $l_{ij}=\|{\mathbf{e}}_{ij}\|$ is the edge length and $\theta_{j}\in[0,2\pi)$ denotes the angular coordinate (normalized to $2\pi$) of this edge vector from a (arbitrarily chosen) tangent basis ${\mathbf{e}}_{0}$, see (Knöppel et al., 2013) for more details. As the edge vector ${\mathbf{e}}_{ij}$ exists in both tangent planes $T_{i}^{M},T_{j}^{M}$, one can compute the angular difference between $T_{i}^{M},T_{j}^{M}$ and obtain the coordinate transformation ${\mathbf{r}}_{ij}\in\mathbb{C}$ (a rotation) needed to make sure that $log_{i}({\mathbf{e}}_{ij})\in T_{i}^{M}$ is mapped to $log_{j}({\mathbf{e}}_{ij})\in T_{j}^{M}$ (see inset). Transporting tangent vectors from $T_{i}^{M}$ to $T_{j}^{M}$ following the computed rotation ${\mathbf{r}}_{ij}$ leads to as-parallel-as-possible transport, or parallel transport. Note that only the angle of these vectors varies as we traverse the curved surface; their length remains unchanged. This motivates constructing a network that can preserve these disentangled vector properties.

The heat equation on a tangent vector field $u:\mathcal{M}\rightarrow\mathbb{C}$ can be written as

where $\Delta_{c}$ is the connection Laplacian. This vector heat equation characterizes how a “vector” gets diffused in space. Intuitively, diffusing a single vector heat source will smear it out to its neighborhood with smaller magnitude while maintaining its direction to be as-parallel-as-possible (see Figure 3). This is different from classic heat diffusion where no notion of parallel transport is captured and can cause inconsistent vector directions if one naively applies scalar heat diffusion to each vector field channel independently.

In several applications, one may want to learn a N-way rotational symmetry fields (Palacios & Zhang, 2007) that are invariant under rotation of an integer multiple of $\nicefrac{{2\pi}}{{N}}$ (see Figure 4). For instance, one of our applications in quadrilateral meshing requires to output a 4-Rosy fields in order to mesh a surface with the “cross” pattern on most vertices.

The generalization to N-Rosy fields is straightforward given the complex number representation (subsection 3.1). Since multiplication with (unit) complex numbers represents rotations, raising a complex number to the power of $N$ factors out all the N-ways rotational symmetry (de Goes et al., 2016). Thus, to measure the difference between, e.g., 4-Rosy fields, one simply measures the difference between ${\mathbf{u}}^{4}$ (see section 6).

Starting with a set of tangent vectors ${\mathbf{u}}\in\mathbb{C}^{n\times c^{l}}$ (represented as complex numbers, see subsection 3.1), where $n$ denotes the number of vertices and $c^{l}$ denotes the number of tangent vector fields. We harness information from local neighborhoods by solving the vector heat equation with the implicit Euler integration in Equation 3, inspired by (Sharp et al., 2022). As the implicit Euler requires an expensive step of solving a linear system, we utilize spectral acceleration (Donati et al., 2022) to speed up the process. Specifically, given the solution of the generalized eigenvalue problem of the discrete connection Laplacian ${\mathbf{L}}_{c}$ (see Equation 2),

where $\Phi=\{\Phi_{i}\}\in\mathbb{C}^{n\times k}$ is the stack of $k$ eigenvectors with the lowest frequencies and $\Sigma=\textit{diag}(\lambda_{i})\in\mathbb{C}^{k\times k}$ is a diagonal matrix with corresponding eigenvalues. We use ${\mathbf{M}}$ to denote the mass matrix of vertex areas such that $\Phi_{i}^{\top}{\mathbf{M}}\Phi_{i}=1$. $k$ is a user-defined number to specify how many eigenvalues/eigenvectors are in use. Then the vector diffusion process in Equation 3 can be approximated with

where $\odot$ denotes element-wise multiplication. Such a spectral acceleration replaces linear solves with matrix multiplications, thus is significantly faster for small $k$. In our implementation, we set $k=128$.

Inspired by (Sharp et al., 2022), we treat time-step size $s$ in Equation 5 as trainable parameters. Intuitively, the network learns whether to diffuse the vectors over a small or large local neighborhood. Specifically, each Vector Heat Diffusion module (Figure 2) consists of $m$ trainable time steps. Each time step $s_{i}$ will diffuse the input feature ${\mathbf{X}}^{l}\in\mathbb{C}^{n\times c^{l}}$ at layer $l$ into a set of diffused features ${\mathbf{Y}}^{l}_{i}\in\mathbb{C}^{n\times c^{l}}$ via Equation 5. Thus, a collection of $m$ time steps $[s_{1},\cdots,s_{m}]$ will turn an input feature ${\mathbf{X}}^{l}$ with size $n$ x $c^{l}$ into a set of diffused features ${\mathbf{Y}}^{l}=[{\mathbf{Y}}^{l}_{1},\cdots,{\mathbf{Y}}^{l}_{m}]$ with size $n$ x $m_{c^{l}}$.

After the vector heat diffusion module, we use a vector-valued MLP that consists of a per-vertex linear layer

where ${\mathbf{W}}^{l}\in\mathbb{R}^{mc^{l}\times c^{l+1}}$ is a matrix of size $mc^{l}$-by-$c^{l+1}$ that linearly combine the complex-valued features at each vertex into ${\mathbf{Z}}^{l}\in\mathbb{C}^{n\times c^{l+1}}$. Then we follow the idea presented by (Wiersma et al., 2022) to apply non-linearities $\sigma$ (e.g., ReLU) on the magnitude of each complex feature as

where we use ${\mathbf{Z}}^{l}_{ij}$ to denote the entry corresponding to the $i$th row and the $j$th column in ${\mathbf{Z}}$, $b^{l}_{j}\in\mathbb{R}$ is a bias term for each channel added to the feature norm. In our experiments, we use the ReLU activation – if the complex feature norm $\|{\mathbf{Z}}^{l}_{ij}\|$ is smaller than the bias $b^{l}_{j}$, the complex feature is set to $\mathbf{0}$, otherwise it is unchanged in the output ${\mathbf{X}}^{l+1}$.

In summary, our overall architecture (see Figure 2) consists of several Vector Diffusion Blocks. Each block contains a vector heat diffusion layer (see subsection 4.1) and a vector-valued MLP with two hidden layers (see subsection 4.2). We also have two extra vector MLPs (green, yellow) to adapt to a different number of input/output channels. Our architecture is invariant to choice of tangent bases due to the Vector Heat Diffusion module baking in parallel transport (see subsection 3.2). Furthermore, the vector heat diffusion process is intrinsic, ensuring the architecture is also invariant to rigid transformations and isometries of the underlying shape. These desirable invariances result in a general network architecture for processing tangent vector fields on manifolds.

To demonstrate our method’s effectiveness, we evaluate it on the task of quadrilateral remeshing. Given a triangle mesh $\mathcal{M}$, we use the per-channel gradient of the first $c^{\text{in}}=15$ channels of the Heat Kernel Signature (HKS) (Sun et al., 2009) as input features to our network, giving $c^{\text{in}}$ vector-valued features per vertex. The output is a 4-Rosy cross field defined on each vertex (see subsection 3.4 and Figure 9). We then interpolate the per-vertex cross field onto faces (see subsection 6.1), followed by off-the-shelf algorithms by Bommes et al. (2009) and (Ebke et al., 2013) to turn the per-face cross field into a quadrilateral mesh (see Figure 9).