P-GADMM: Parallel Group-Based ADMM for Asynchronous Optimization in Heterogeneous Edge Networks

P-GADMM (Parallel Group-Based ADMM) is a distributed optimization algorithm designed for heterogeneous edge networks that mitigates the efficiency loss of synchronous ADMM caused by straggler clients. Unlike traditional methods that group clients by data or topology, P-GADMM uses a computation-aware grouping strategy based on client processing capabilities and local data sizes to minimize speed variation within groups.

The algorithm combines edge-level aggregation with bounded asynchronous coordination at the cloud, allowing faster groups to participate in global updates without waiting for slower ones. This design reduces synchronization delay and idle time while controlling stale information through an explicit delay threshold. Experiments on MNIST and CIFAR10 demonstrate that P-GADMM reduces wall-clock training time compared to baselines like Asynch-ADMM and GADMM, while maintaining comparable final accuracy under both IID and non-IID settings.

P-GADMM addresses a synchronization bottleneck in distributed optimization for heterogeneous edge networks. In standard synchronous ADMM, every global iteration requires all clients or groups to complete their local updates before the system can proceed, which causes fast devices to idle while waiting for slower ones. The paper proposes a parallel, group-based variant of ADMM that organizes participants into groups and allows updates to be performed asynchronously and in parallel, thereby reducing both communication overhead and sensitivity to stragglers.

The key insight is to combine two ideas that are often treated separately: group aggregation to reduce the number of messages exchanged, and asynchronous execution to avoid global barriers. By letting groups update when they are ready rather than forcing all groups to synchronize at each round, P-GADMM is better suited to edge environments where compute capability, data availability, and network latency vary significantly across devices. The work is positioned as an extension of group-based ADMM methods, which mainly target communication reduction, toward a more robust design for temporally heterogeneous settings.

This matters because many edge and federated optimization workloads are limited not just by the number of communication rounds, but by the slowest participant in each round. An asynchronous group-based ADMM approach can improve wall-clock efficiency and throughput while preserving the distributed structure of ADMM, making it relevant for large-scale edge learning, inference optimization, and other distributed systems where devices are nonuniform and communication is costly.