Abstract

---

The denoising process of PDT. Our framework learns to use diffusion models to transform points into clustered&structured points distributions, such as surface mesh keypoints, skeletal joints and continuous feature lines.

Point-based representations have consistently played a vital role in geometric data structures. Most point cloud learning and processing methods typically leverage the unordered and unconstrained nature to represent the underlying geometry of 3D shapes. However, how to extract meaningful structural information from unstructured point cloud distributions and transform them into semantically meaningful point distributions remains an under-explored problem. We present PDT, a novel framework for point distribution transformation with diffusion models. Given a set of input points, PDT learns to transform the point set from its original geometric distribution into a target distribution that is semantically meaningful. Our method utilizes diffusion models with novel architecture and learning strategy, which effectively correlates the source and the target distribution through a denoising process. Through extensive experiments, we show that our method successfully transforms input point clouds into various forms of structured outputs - ranging from surface-aligned keypoints, and inner sparse joints to continuous feature lines. The results showcase our framework's ability to capture both geometric and semantic features, offering a powerful tool for various 3D geometry processing tasks where structured point distributions are desired.

Overview

---

PDT leverages a diffusion transformer-based architecture to transform Gaussian noise into semantically meaningful point distributions, guided by input reference points. We demonstrate the effectiveness of our approach across three structural representations: surface keypoints for artist-inspired meshes, inner skeletal joints for character rigging, and continuous feature lines for garment analysis.

Application: Remeshing

---

Results & denoising processes of mesh keypoints prediction.

Application: Skeletal Joints Prediction

---

Results & denoising processes of skeletal joints prediction.

Application: Continuous Feature Lines Extraction

---

Results & denoising processes of feature line extraction.

Results Gallery: Remeshing

---

Results Gallery: Skeletal Joints

---

Results Gallery: Feature Lines

---

Core Idea

---

We pair noisy points from Gaussian distribution each with an input point as a per-point reference. Then, our diffusion model is trained to drag and denoise the Gaussian noise into a desired structural points distribution.

Framework

---

Architecture overview of our PDT. The model extracts per-point features from input reference points and associates them with corresponding noisy points through adding its positional encoding features. The combined features and timestep embeddings are processed through a series of DiT layers to learn the distribution transformation.