MADM: Bridging Mamba and Autoregressive Diffusion for Text-to-Motion Generation

Generating human motion from text presents three primary challenges. First, human motion sequences often span hundreds of frames, making long-sequence modeling computationally inefficient and demanding. Second, the complex many-to-many mapping between language and motion complicates precise semantic alignment. Third, achieving global consistency alone does not ensure local spatial-temporal dependencies, resulting in motions that lack local smoothness and realistic spatial structure. To address these challenges, we propose a Mamba-based autoregressive diffusion model (MADM). First, we devise a token reordering strategy into Mamba to construct Masked Autoregressive Mamba, enabling efficient bidirectional context modeling under the masked autoregressive paradigm. Second, we design a Semantic-Aware Spatial Augmentation module that fuses textual semantics with motion features and employs lightweight convolutions to capture short-term dependencies, thereby enhancing semantic alignment, spatial awareness, and local smoothness. Finally, we incorporate a Diffusion Head into the autoregressive framework, allowing MADM to directly model the continuous motion space and avoid the vector quantization loss. Experimental results on the HumanML3D and KIT-ML datasets demonstrate that MADM achieves state-of-the-art performance with superior efficiency. On HumanML3D, it attains a Top-1 score of 0.514 and an FID of 0.052, highlighting its advantages in motion quality and semantic alignment. Notably, benefited from the linear complexity of Mamba, MADM reduces the computational cost to 8.973G FLOPs, compared to 29.569G for the Transformer baseline.

We compare MADM with state-of-the-art methods on the HumanML3D dataset. As shown in the chart below, our model achieves a superior trade-off between motion quality (FID) and semantic alignment (Top-1 Precision).