Research

Federated learning faces huge challenges from model overfitting due to the lack of data and statistical diversity among clients. To address these challenges, this paper proposes a novel personalized federated learning method via Bayesian variational inference named pFedBayes. To alleviate the overfitting, weight uncertainty is introduced to neural networks for clients and the server. To achieve personalization, each client updates its local distribution parameters by balancing its construction error over private data and its KL divergence with global distribution from the server. Theoretical analysis gives an upper bound of averaged generalization error and illustrates that the convergence rate of the generalization error is minimax optimal up to a logarithmic factor. Experiments show that the proposed method outperforms other advanced personalized methods on personalized models, e.g., pFedBayes respectively outperforms other SOTA algorithms on MNIST, FMNIST and CIFAR-10 under non-i.i.d. limited data. 

Internet of Things networks are large-scale distributed systems consisting of a massive number of simple devices communicating, typically, over a shared wireless medium. This new paradigm requires novel ways of coordinating access to limited communication resources without introducing unreasonable delays. Herein, the optimal design of a remote estimation system with $n$ sensors communicating with a fusion center via a collision channel of limited capacity k < n is considered.  In particular, for independent and identically distributed observations with a symmetric probability density function, we show that the problem of minimizing the mean-squared error with respect to a threshold strategy is quasi-convex. When coordination among sensors via a local communication network is available, the on-line learning of possibly unknown parameters of the probabilistic model is possible, enabling each sensor to optimize its own threshold autonomously. We propose two strategies for remote estimation with local communication: One strategy swiftly reaches the performance of the optimal decentralized threshold policy, whereas a second strategy approaches the performance of the optimal centralized scheme with a slower convergence rate. A hybrid scheme that combines the best of both approaches is proposed, offering fast convergence and excellent performance.

This research considers the problem of recovering a structured signal from a relatively small number of noisy measurements with the aid of a similar signal which is known beforehand. We propose a new approach to integrate prior information into the standard recovery procedure by maximizing the correlation between the prior knowledge and the desired signal. We then establish performance guarantees (in terms of the number of measurements) for the proposed method under sub-Gaussian measurements. Specific structured signals including sparse vectors, block-sparse vectors, and low-rank matrices are also analyzed. Furthermore, we present an interesting geometrical interpretation for the proposed procedure. Our results demonstrate that if prior information is good enough, then the proposed approach can (remarkably) outperform the standard recovery procedure.

Covariance matrix estimation concerns the problem of estimating the covariance matrix from a collection of samples, which is of extreme importance in many applications. Classical results have shown that O(n) samples are sufficient to accurately estimate the covariance matrix from n -dimensional independent Gaussian samples. However, in many practical applications, the received signal samples might be correlated, which makes the classical analysis inapplicable. In this paper, we develop a non-asymptotic analysis for the covariance matrix estimation from linearly-correlated Gaussian samples. Our theoretical results show that the error bounds are determined by the signal dimension n , the sample size m , and the shape parameter of the distribution of the correlated sample covariance matrix. Particularly, when the shape parameter is a class of Toeplitz matrices (which is of great practical interest), O(n) samples are also sufficient to faithfully estimate the covariance matrix from correlated samples. Simulations are provided to verify the correctness of the theoretical results.