CF-LoRA: Decoupled Factor Aggregation and Adaptation-Aware Client Clustering for Federated LoRA Fine-Tuning

CF-LoRA (arXiv:2609.36986v1, September 29, 2026) is a clustered federated fine-tuning framework designed to resolve structural aggregation mismatch and statistical collaboration mismatch in heterogeneous federated LoRA settings.

The method employs a three-stage decoupled aggregation scheme: Global A Aggregation: It learns a globally shared $A$ factor while retaining locally personalized $B_i$ factors, avoiding spurious cross-client products that occur when independently averaging both factors. Adaptation-Aware Clustering: It clusters clients based on the cosine similarity of their learned $B_i$ factors to group those with similar adaptation patterns. * Intra-Cluster B Aggregation: It freezes the shared $A$ factor and aggregates only the $B$ factors within each cluster, enabling beneficial knowledge sharing while reducing negative transfer from heterogeneous data.

This approach achieves higher average accuracy across language and vision benchmarks compared to existing baselines while communicating only one LoRA factor per optimization round.

CF-LoRA addresses two core failure modes of federated LoRA fine-tuning under heterogeneous client data. First, a structural aggregation mismatch arises when independently trained LoRA factors are averaged directly: because LoRA updates are low-rank factor pairs, naïve averaging of the factors does not necessarily correspond to a meaningful aggregate low-rank adapter and can degrade the effective update. Second, a statistical collaboration mismatch occurs when a single global model or adapter is enforced across clients whose data distributions, tasks, or adaptation directions differ substantially, causing negative transfer.

The paper proposes CF-LoRA, a federated LoRA framework that separates these two issues. Its decoupled factor aggregation component is designed to combine LoRA factors in a way that better preserves the structure of low-rank adaptation, rather than treating each factor as an ordinary parameter matrix to be averaged. Its adaptation-aware client clustering component groups clients based on similarity in their learned adaptations, allowing collaboration to occur among clients that are statistically compatible while reducing harmful interference from dissimilar clients.

The significance of the work is that it targets a practical weakness of current federated parameter-efficient fine-tuning: LoRA makes distributed adaptation efficient and private, but standard federated averaging can still be poorly matched to low-rank updates and non-IID client populations. By improving both how adapters are aggregated and which clients are aggregated together, CF-LoRA aims to make federated LoRA more robust, more faithful to client-specific adaptations, and more effective for real-world deployments where data heterogeneity is the norm rather than the exception.