Heterogeneous Data \& Expensive Communication- Layer-wised Aggregation

Introduction

Instead of collaboratively train only one global for all clients, personalized federated learning (pFL) mechanisms are proposed to allow each client to train a customized model to adapt to their own data distribution. Researches over the past few years have applied the weighted aggregation manner to produce personalized models, where the weights are determined by calibrating the distance of the entire model parameters or loss values, and have yet to consider the layer-level impacts to the aggregation process, leading to lagged model convergence and inadequate personalization over non-IID datasets. We design a novel pFL training framework dubbed Layer-wised Personalized Federated learning (pFedLA) that can discern the importance of each layer from different clients, and thus is able to optimize the personalized model aggregation for clients with heterogeneous data.

Reference:

[4]. Layer-wised Model Aggregation for Personalized Federated Learning. CVPR, CCF-A