DSAQuant: Denoising Stage-Aligned Quantization-Aware Training Framework for Video Generation

Background and Context

Video diffusion models (VDMs) have achieved remarkable performance in text-to-video generation, capable of producing high-quality and coherent dynamic visual content. However, the massive parameter counts and computational complexity of these models present significant memory and compute challenges for practical deployment. Quantization-Aware Training (QAT) has emerged as a promising technique to compress and accelerate generative models by simulating quantization errors during the training phase, thereby avoiding increased inference runtime overhead.

Despite existing QAT methods performing adequately in preserving prompt semantics, overall layout, and coarse motion, applying them to video generation often results in severe degradation of visual details, texture fidelity, and image clarity. This phenomenon not only compromises the visual quality of generated content but also limits the potential of quantization techniques in high-end video generation models. Consequently, achieving efficient quantization without sacrificing detail quality has become a critical scientific problem in the field of video generation.

Deep Analysis

The root cause of this degradation lies in the design flaws of traditional quantization pipelines, which employ timestep-agnostic strategies that ignore the phased nature of the video denoising process. Video diffusion models do not denoise uniformly; early steps are primarily responsible for establishing global structure and motion trajectories, while mid-to-late steps focus on refining local appearance and high-frequency details. To address this, the authors propose DSAQuant (Denoising-Stage-Aligned Quantization-aware training), a framework that aligns quantization with the denoising stages. During training, DSAQuant introduces a denoising-stage-guided supervision mechanism. In early denoising steps, it retains teacher distillation to stabilize structural planning and prevent structural collapse due to quantization. In later steps, it shifts to objective-driven optimization to enhance detail reconstruction. This phased supervision allows the model to dynamically adjust its tolerance for quantization error and optimization direction based on the specific task requirements of each denoising stage.

During inference, DSAQuant implements a denoising-stage gating guidance strategy. Traditional Classifier-Free Guidance (CFG) enhances semantic alignment but can amplify high-frequency errors introduced by quantization in the final denoising steps, leading to artifacts. DSAQuant dynamically disables CFG in the last few denoising steps to prevent the accumulation and amplification of quantization errors, thereby improving the clarity and detail quality of the final output. Extensive experiments on the Wan and CogVideoX model families, evaluated under extreme W4A4 (4-bit weight, 4-bit activation) and W3A3 (3-bit weight, 3-bit activation) settings, demonstrate that DSAQuant consistently outperforms existing SOTA QAT baselines. Notably, under aggressive W3A3 quantization, DSAQuant improved the VBench average score by up to 6.60 points while maintaining strong text-video alignment capabilities. Ablation studies confirmed the critical roles of both the denoising-stage-guided supervision and the gating guidance strategy.

Industry Impact

The introduction of DSAQuant holds profound implications for both the open-source community and industrial deployment of video generation. First, it provides a new paradigm for significantly enhancing the performance of quantized models without modifying the model architecture, offering valuable reference ideas for the quantization optimization of other generative models. Second, by substantially reducing the computational and memory requirements of models without compromising visual quality, DSAQuant makes it possible to deploy high-quality video generation models on edge devices or in resource-constrained environments. This significantly expands the application scenarios of video generation technology. Furthermore, this work emphasizes the importance of understanding internal model mechanisms, such as the differences between denoising stages, for optimizing model performance. It encourages subsequent research to focus more on the deep connections between model behavior and optimization strategies.

As video generation technology is widely applied in entertainment, education, design, and other fields, the fine-grained quantization technology represented by DSAQuant will become one of the key infrastructures for driving industry cost reduction and efficiency improvement, and achieving large-scale deployment. It lays a solid foundation for building a more efficient and inclusive AI video generation ecosystem. The ability to maintain high fidelity at low bit-widths like W3A3 is particularly significant for reducing inference costs, which is a major barrier to scaling video AI services. By enabling high-quality generation on less powerful hardware, DSAQuant democratizes access to advanced video creation tools, allowing for real-time generation and interactive applications that were previously computationally prohibitive.

Outlook

Looking forward, the success of DSAQuant suggests that future quantization research must move beyond static, timestep-agnostic approaches. The explicit modeling of the denoising process's phased characteristics appears to be a crucial direction for advancing model compression in diffusion-based generative AI. As video models continue to grow in size and complexity, techniques that dynamically adjust training and inference parameters based on the semantic or structural needs of different generation stages will likely become standard.

The integration of such adaptive quantization strategies with other efficiency techniques, such as sparse attention or distilled architectures, could further push the boundaries of what is possible on consumer-grade hardware. The significant improvement in VBench scores under W3A3 quantization indicates that aggressive compression is no longer a trade-off for quality but can be managed through intelligent training frameworks. This paves the way for more sustainable and accessible video generation technologies, reducing the environmental and economic costs associated with training and running large-scale diffusion models. The community is now poised to explore similar stage-aligned optimizations for other modalities, potentially revolutionizing how we approach efficiency in generative AI across text, image, and video domains.