Standard federated learning yields global models that degrade regional performance on heterogeneous retail search data, while existing personalized FL methods catastrophically collapse on modern transformers.
RegionFed is an architecture-robust federated learning framework that resolves the performance degradation of standard FL and the catastrophic collapse of parameter-level personalized FL on modern transformers by operating entirely at the gradient level. It uses the $\ell_2$ conflict between regional and global gradients as a unified signal to diagnose heterogeneity, route regions to the most efficient personalization strategy, and adaptively control personalization strength without requiring code changes for different architectures like T5 or RoBERTa.
Key advantages include: Transformer Compatibility: Unlike parameter-level methods (e.g., SCAFFOLD, pFedMe) which collapse to below 10% accuracy on T5 models due to tied embeddings and LayerNorm interactions, RegionFed maintains stable convergence. High Performance: It achieves 92.27% accuracy on Amazon ESCI and Reviews datasets, closing the gap to privacy-violating centralized upper bounds while providing differential privacy. Regional Personalization: By personalizing at the region level rather than the individual user level, it pools sufficient data for stable gradient estimates and captures regional semantic differences (e.g., distinct vocabularies or product preferences). Efficiency and Scalability: RegionFed demonstrates $O(1/\sqrt{T})$ convergence, scales better than standard FL under high heterogeneity, and outperforms architecture-specific baselines like FedTP in both accuracy and regional fairness.
RegionFed addresses a practical failure mode of federated learning in retail search: when customer query data is highly heterogeneous across regions, a single globally aggregated query-understanding model can underperform region-specific models, even though it is supposed to benefit from pooled data. In retail, “query understanding” includes tasks such as intent detection, category prediction, entity recognition, synonym expansion, and query rewriting, all of which are sensitive to local product catalogs, language use, seasonal behavior, and customer expectations. The paper frames this as a personalization problem in which standard federated averaging treats regional differences as noise or nuisance heterogeneity, thereby inducing negative transfer for regions whose query distributions differ from the global majority.
Its central contribution is a federated framework that treats region as a first-class personalization axis rather than merely a source of non-IID client data. Rather than relying only on a shared global transformer, RegionFed couples cross-region federated learning with region-conditioned adaptation, allowing a shared query-understanding backbone to be specialized for local retail contexts. The paper also emphasizes the instability of existing personalized federated learning methods when applied to modern transformer architectures, where aggressive local fine-tuning or poorly constrained personalization can cause catastrophic collapse. RegionFed is therefore positioned as a more stable, region-aware alternative that preserves the benefits of federated learning while avoiding the degradation of local relevance.
This matters because retail search systems are often privacy-sensitive and operationally fragmented: individual retailers or regional stores may not be able to share raw query logs, yet each region still needs highly relevant search behavior. A one-size-fits-all global model is insufficient for heterogeneous markets, while purely local models miss the statistical strength of federated aggregation. By targeting the specific interaction between retail query understanding, regional heterogeneity, and transformer-based personalization, the paper contributes a more deployable path toward privacy-preserving, region-adaptive search intelligence.