Comparing PEFT Variants in Single-Patient Dysarthria ASR: An Empirical Study of the LoRA Family on Whisper and Qwen3

Background and Context

Automatic Speech Recognition (ASR) systems face significant challenges when adapting to patients with dysarthria, a speech disorder often resulting from neurological conditions such as stroke. Unlike typical speech patterns, dysarthric speech exhibits high individual specificity, rendering generic, pre-trained models ineffective for clinical applications. Consequently, single-patient adapter architectures have emerged as the preferred solution in production environments, allowing models to specialize for individual users without retraining the entire network. Despite the widespread adoption of Parameter-Efficient Fine-Tuning (PEFT) techniques in large language models, there has been a lack of systematic research comparing the efficacy of different PEFT variants in this specific, data-scarce domain.

This study addresses that gap by conducting a rigorous single-speaker case study comparing seven mainstream PEFT variants: LoRA, QLoRA, AdaLoRA, DoRA, LoHA, VeRA, and VB-LoRA. The research aims to determine which fine-tuning strategy offers the optimal balance between recognition accuracy and computational cost in resource-constrained clinical settings. By evaluating these methods on production-grade benchmarks, the study provides data-driven insights into how different adapter structures influence model performance. The primary objective is to identify a lightweight yet highly accurate approach that can be deployed in real-world medical scenarios where storage and processing power are limited.

Deep Analysis

The experimental framework utilized two representative base models: a Hungarian fine-tuned Whisper-large-v3 and a multilingual Qwen3-ASR-1.7B checkpoint. The subject was a Hungarian male patient with severe post-stroke dysarthria, whose dataset comprised 409 utterances classified as severely dysarthric through auditory-perceptual clinical assessment. The study implemented a detailed fine-tuning pipeline for all seven PEFT methods, incorporating attention projection adapters to enhance the capture of patient-specific speech features. To ensure statistical validity, the researchers employed paired Bootstrap tests across three random seeds, providing a robust evaluation of each method's performance.

Results from the Whisper and Qwen3 benchmarks revealed distinct performance hierarchies among the PEFT variants. Standard LoRA and DoRA demonstrated no statistically significant difference (p>0.5), achieving Character Error Rates (CER) of 13.86% and 13.90% on Whisper, and 28.10% and 28.33% on Qwen3, respectively. Given their comparable performance, LoRA was selected as the baseline due to its lower complexity and cost. Surprisingly, the widely used 4-bit (NF4) QLoRA performed the worst across all seeds and benchmarks, with CERs of 14.56% and 30.09%, failing to deliver the expected memory savings at this scale. Other variants like LoHA, VeRA, VB-LoRA, and AdaLoRA also failed to surpass LoRA, although LoHA achieved a 18.6% relative CER reduction on Whisper.

When compared to full fine-tuning, which yielded the highest accuracy with a CER of 11.43%, LoRA proved remarkably efficient. A LoRA adapter targeting only the feed-forward blocks, weighing just 115 MB, achieved a performance only 0.66 percentage points worse than full fine-tuning. Crucially, this represented a storage cost of merely 3.7% of the full model. This finding strongly supports LoRA as the superior choice for balancing precision and efficiency. Furthermore, a six-point registration grid experiment quantified the impact of data volume, revealing that just five minutes of patient audio could capture 45.6% of the performance gain observed between zero-shot and 30-minute full-data scenarios. This suggests that rapid deployment is feasible with minimal data collection.

Industry Impact

The findings of this study offer critical engineering guidance for the production deployment of ASR systems for dysarthric patients. In healthcare environments characterized by limited resources and scarce patient data, the ability to achieve near-full-tuning performance with a 3.7% storage overhead significantly lowers the barrier to entry for personalized ASR solutions. This efficiency enables the integration of advanced speech recognition into edge devices and resource-constrained medical terminals, facilitating real-time assistance for patients with severe communication impairments. The study effectively validates the "small model + strong adapter" technical route, demonstrating that high performance does not always require massive computational investments.

Additionally, the study serves as a cautionary note regarding the application of popular PEFT technologies like QLoRA in low-resource scenarios. The poor performance of 4-bit QLoRA indicates that quantization and fine-tuning effectiveness do not always align, emphasizing the need for careful trade-off analysis between memory compression and accuracy. For the open-source community, the commitment to releasing training scripts and recipes will foster reproducibility and encourage further exploration of optimal adaptation strategies. This transparency is vital for advancing the field and ensuring that future developments are built on robust, empirically validated foundations.

Outlook

While this research focuses on a single speaker and language, its rigorous experimental design and quantitative analysis framework lay the groundwork for future multi-lingual and multi-patient studies. The demonstrated efficacy of LoRA in capturing significant performance gains with minimal data suggests that scalable, personalized ASR systems are increasingly viable. As the technology matures, it holds great promise for broader applications in neuro-rehabilitation and assistive communication, potentially improving the quality of life for millions of individuals with speech disorders. Future work will likely expand upon these findings to address diverse linguistic contexts and varying degrees of speech impairment, further refining the balance between model complexity and clinical utility.

The study also highlights the importance of attention projection adapters in reducing character error rates, pointing to specific architectural enhancements that yield tangible benefits. As the field moves forward, the integration of these insights into standard PEFT workflows could lead to more adaptive and responsive ASR systems. The empirical evidence provided here not only supports current deployment practices but also sets a benchmark for evaluating new fine-tuning methods in specialized domains. Ultimately, the goal is to create inclusive speech technologies that are both powerful and accessible, ensuring that advancements in AI benefit those with the greatest need for communication support.