Bayesian Inference with Nonlinear Generative Models: Comments on Secure Learning

Bereyhi A, Loureiro B, Krzakala F, Müller R, Schulz-Baldes H (2023)

Publication Type: Journal article

Publication year: 2023

Journal

IEEE Transactions on Information Theory Institute of Electrical and Electronics Engineers (IEEE)

Pages Range: 1-1

DOI: 10.1109/TIT.2023.3325187

Abstract

Unlike the classical linear model, nonlinear generative models have been addressed sparsely in the literature of statistical learning. This work aims to shed light on these models and their secrecy potential. To this end, we invoke the replica method to derive the asymptotic normalized cross entropy in an inverse probability problem whose generative model is described by a Gaussian random field with a generic covariance function. Our derivations further demonstrate the asymptotic statistical decoupling of the Bayesian estimator and specify the decoupled setting for a given nonlinear model. The replica solution depicts that strictly nonlinear models establish an all-or-nothing phase transition: there exists a critical load at which the optimal Bayesian inference changes from perfect to an uncorrelated learning. Based on this finding, we design a new secure coding scheme which achieves the secrecy capacity of the wiretap channel. This interesting result implies that strictly nonlinear generative models are perfectly secured without any secure coding. We justify this latter statement through the analysis of an illustrative model for perfectly secure and reliable inference.

Authors with CRIS profile

Ralf Müller Professur für Informationsübertragung Hermann Schulz-Baldes Professur für Mathematik (Mathematische Physik)

Involved external institutions

University of Toronto

Canada (CA) École Normale Supérieure - PSL (ENS)

France (FR) École Polytechnique Fédérale de Lausanne (EPFL)

Switzerland (CH)

How to cite

APA:

Bereyhi, A., Loureiro, B., Krzakala, F., Müller, R., & Schulz-Baldes, H. (2023). Bayesian Inference with Nonlinear Generative Models: Comments on Secure Learning. IEEE Transactions on Information Theory, 1-1. https://dx.doi.org/10.1109/TIT.2023.3325187

MLA:

Bereyhi, Ali, et al. "Bayesian Inference with Nonlinear Generative Models: Comments on Secure Learning." IEEE Transactions on Information Theory (2023): 1-1.

BibTeX: Download