BeatDance: Generating Beat-Consistent 3D Dance with
Hierarchical Spatial-Temporal Modeling

Xiaojian Shen1, Dahu Shi2, Jianrong Zhang3, Hai Li1, Hongwei Zhao1,
Dawei Zhang4, Yunzhi Zhuge5, Zhiliang Wu2,†, Guanghui Yue6, Wei Zhou7
1Jilin University    2Zhejiang University    3University of Technology Sydney    4Zhejiang Normal University
5Dalian University of Technology    6Shenzhen University    7Cardiff University    †Corresponding Author
Paper (Under Review) GitHub Repo

Abstract

Generating realistic 3D dance from music is a challenging task that requires accurate synchronization with musical rhythms while capturing the spatial complexity of human motion. Although existing methods can generate physically plausible dance motions, they often struggle to achieve precise alignment with music, such as the beat. To address this limitation, we propose a novel diffusion-based framework, BeatDance, with two components: 1) We present a Hierarchical Decoupled Attention (HDA) module, which first disentangles the learning of human pose and temporal dynamics. A hierarchical structure is then employed to capture both short-term and long-term dependencies, thereby enhancing spatial-temporal modeling. 2) We adopt cycle-consistent learning by introducing an auxiliary dance-to-music module. During training, discrepancies between the reconstructed and original music induce a stronger loss signal, effectively encouraging the consistency property between the music and dance motion. Extensive experimental results demonstrate that our proposed approach outperforms recent competitive methods on two benchmark datasets.

Method Overview

Method Overview

Figure 1: Overview of the BeatDance framework. Hierarchical Decoupled Attention (HDA) employs a hierarchical design to capture both short-term and long-term music-dance dependencies, building upon a Decoupled Attention (DA) that splits the modeling into spatial and temporal branches. Additionally, our cycle-consistent learning mechanism uses an auxiliary Dance-to-Music (D2M) model to reconstruct the source music features from the generated dance, thereby enforcing a tighter music-dance alignment.

Visual Comparisons

💡 Click Play/Pause or drag timeline to sync. 🔊 Please unmute for music!

Citation

@article{BeatDance2025,
    title={BeatDance: Generating Beat-Consistent 3D Dance with Hierarchical Spatial-Temporal Modeling},
    author={Shen, Xiaojian and Shi, Dahu and Zhang, Jianrong and Li, Hai and Zhao, Hongwei and Zhang, Dawei and Zhuge, Yunzhi and Wu, Zhiliang and Yue Guanghui and Zhou, Wei},
    journal={Under Review},
    year={2025}
}