PDT: Point Distribution Transformation with Diffusion Models

SIGGRAPH 2025 Conference

Jionghao Wang*1, Cheng Lin*2, Yuan Liu3, Rui Xu2, Zhiyang Dou2, Xiaoxiao Long4,
Hao-Xiang Guo5, Taku Komura2, Wenping Wang1, Xin Li1

1Texas A&M University    2The University of Hong Kong   3HKUST   4Nanjing University   5Skywork AI  

Paper (Arxiv)

Code

Abstract

The denoising process of PDT. Our framework learns to use diffusion models to transform points into clustered&structured points distributions, such as surface mesh keypoints, skeletal joints and continuous feature lines.

Point-based representations have consistently played a vital role in geometric data structures. Most point cloud learning and processing methods typically leverage the unordered and unconstrained nature to represent the underlying geometry of 3D shapes. However, how to extract meaningful structural information from unstructured point cloud distributions and transform them into semantically meaningful point distributions remains an under-explored problem. We present PDT, a novel framework for point distribution transformation with diffusion models. Given a set of input points, PDT learns to transform the point set from its original geometric distribution into a target distribution that is semantically meaningful. Our method utilizes diffusion models with novel architecture and learning strategy, which effectively correlates the source and the target distribution through a denoising process. Through extensive experiments, we show that our method successfully transforms input point clouds into various forms of structured outputs - ranging from surface-aligned keypoints, and inner sparse joints to continuous feature lines. The results showcase our framework's ability to capture both geometric and semantic features, offering a powerful tool for various 3D geometry processing tasks where structured point distributions are desired.


Overview

PDT leverages a diffusion transformer-based architecture to transform Gaussian noise into semantically meaningful point distributions, guided by input reference points. We demonstrate the effectiveness of our approach across three structural representations: surface keypoints for artist-inspired meshes, inner skeletal joints for character rigging, and continuous feature lines for garment analysis.


Application: Remeshing

Results & denoising processes of mesh keypoints prediction.


Application: Skeletal Joints Prediction

Results & denoising processes of skeletal joints prediction.


Application: Continuous Feature Lines Extraction

Results & denoising processes of feature line extraction.


Results Gallery: Remeshing


Results Gallery: Skeletal Joints


Results Gallery: Feature Lines


Core Idea

We pair noisy points from Gaussian distribution each with an input point as a per-point reference. Then, our diffusion model is trained to drag and denoise the Gaussian noise into a desired structural points distribution.


Framework

Architecture overview of our PDT. The model extracts per-point features from input reference points and associates them with corresponding noisy points through adding its positional encoding features. The combined features and timestep embeddings are processed through a series of DiT layers to learn the distribution transformation.


Citation

@misc{,
}