Momentum-Guided Federated Split Distillation for Personalized Temporal Edge Intelligence

The paper Momentum-Guided Federated Split Distillation for Personalized Temporal Edge Intelligence (arXiv:2609.31159v1), submitted on September 25, 2026, introduces a framework for efficient edge intelligence using two novel components: TeRR-SAtt and AMGF.

• TeRR-SAtt is a temporal reservoir student attention design that combines fixed reservoir representations with a lightweight temporal student and personalized output modules.
• AMGF is an anticipatory momentum-guided fusion mechanism that clusters clients via learning momentum to derive specialized teacher updates.

Evaluated on real-world smart-building data, the framework reduces edge training latency by 65.50%, inference latency by 44.70%, training memory usage by 18.40%, and inference CPU usage by 33.10% compared to baselines. Additionally, AMGF improves local learning by up to 35.31% in RMSE over global updates. The work is published in ECML PKDD 2026.

The paper addresses personalized temporal edge intelligence under the practical constraints of federated learning: edge devices generate time-dependent, often non-stationary data, but they cannot easily upload raw streams to a central server due to privacy, bandwidth, and latency concerns. The proposed framework, Momentum-Guided Federated Split Distillation, is designed to train collaborative models across edge clients while preserving local adaptation. The core idea is to use split distillation to distribute learning responsibilities between clients and a coordinating server or teacher model, so that edge devices can learn useful temporal representations without exposing raw data or requiring large, fully trainable models locally. The “momentum-guided” aspect is intended to stabilize federated optimization in the presence of heterogeneous clients, non-IID temporal distributions, and possible drift over time.

A key architectural contribution is TeRR-SAtt, described as a temporal reservoir student attention design. This design combines three elements: fixed reservoir representations, a lightweight temporal student, and personalized output modules. The fixed reservoir component appears to provide a low-cost, largely untrained temporal feature space that captures temporal structure without the overhead of training a deep temporal backbone. The lightweight student then refines or attends over these features in a computationally efficient way, while the personalized output modules allow each edge client to maintain local behavior-specific predictions. The abstract also introduces AMG, which appears to be the momentum-guided mechanism supporting the federated update or coordination process, likely aimed at improving convergence and robustness across heterogeneous temporal edge workloads.

This work matters because it targets a common but difficult regime in edge AI: temporal data that must be processed locally, adapted to individual devices or users, and coordinated across a network without centralized raw-data training. By combining reservoir-style temporal encoding, split distillation, personalization, and momentum-guided federated updates, the framework aims to reduce communication and compute costs while still supporting autonomy and client-specific behavior. For applications such as sensor networks, wearable devices, industrial monitoring, or other latency-sensitive edge systems, the approach is relevant because it balances efficiency, privacy, personalization, and temporal adaptability—a difficult tradeoff in federated edge intelligence.