HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-Tuned LLM with Quantization
Su Y, Yan N, Deng Y, Dohler M, Schober R (2026)
Publication Type: Journal article
Publication year: 2026
Journal
Book Volume: 25
Pages Range: 20054-20070
Abstract
Federated fine-Tuning of pre-Trained Large Language Models (LLMs) enables task-specific adaptation across diverse datasets while preserving privacy. However, challenges such as high computational and memory demands, heterogeneous client resources, bandwidth constraints, and ineffective global aggregation hinder its efficiency. To address these issues, we propose HAFLQ (Heterogeneous Adaptive Federated LoRA Fine-Tuned LLM with Quantization), featuring four technical innovations. First, a salience-driven adaptive quantization scheme ranks transformer blocks by Hessian-based importance scores, applying block-wise quantization (e.g., INT8 for low-salience blocks, FP32 for critical ones) tailored to each client's GPU capacity. Second, an importance-based parameter management mechanism enables clients with limited resources to selectively freeze low-importance LoRA rank-1 matrices (identified via gradient-sensitivity metrics) while training only critical ones, avoiding truncation-induced performance loss. Third, bandwidth-Adaptive communication quantization allocates higher precision (32/16-bit) to important parameters and lower precision (8/4-bit) to less critical ones based on wireless channel conditions, prioritizing transmission of high-impact rank-1 matrices. Fourth, adaptive rank-1 matrix-level aggregation weights client contributions by Frobenius norm and aggregates only updated matrices, preventing information dilution from zero-padding approaches. Experiments on text classification tasks show that HAFLQ outperforms existing heterogeneous federated learning baselines in convergence speed and accuracy under bandwidth-sufficient conditions. Under bandwidth-constrained scenarios, HAFLQ reduces memory usage by 31% via adaptive LLM quantization and communication cost by 49% via adaptive communication quantization, while the complete framework achieves 50% relative accuracy improvement over the zero-padding aggregation baseline (from 59% to 89%).
Authors with CRIS profile
Involved external institutions
United Kingdom (GB)
Sweden (SE)How to cite
APA:
Su, Y., Yan, N., Deng, Y., Dohler, M., & Schober, R. (2026). HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-Tuned LLM with Quantization. IEEE Transactions on Wireless Communications, 25, 20054-20070. https://doi.org/10.1109/TWC.2026.3711528
MLA:
Su, Yang, et al. "HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-Tuned LLM with Quantization." IEEE Transactions on Wireless Communications 25 (2026): 20054-20070.
BibTeX: Download