On Empirical Bayes Variational Autoencoder: An Excess Risk Bound

Rong Tang , Yun Yang

[Proceedings link] [PDF]

Session: Generalization and PAC-Learning 1 (B)

Session Chair: Dylan Foster

Poster: Poster Session 3

Abstract: In this paper, we consider variational autoencoders (VAE) via empirical Bayes estimation, referred to as Empirical Bayes Variational Autoencoders (EBVAE), which is a general framework including popular VAE methods as special cases. Despite the widespread use of VAE, its theoretical aspects are less explored in the literature. Motivated by this, we establish a general theoretical framework for analyzing the excess risk associated with EBVAE under the setting of density estimation, covering both parametric and nonparametric cases, through the lens of M-estimation. As an application, we analyze the excess risk of the commonly-used EBVAE with Gaussian models and highlight the importance of covariance matrices of Gaussian encoders and decoders in obtaining a good statistical guarantee, shedding light on the empirical observations reported in the literature.

Summary presentation

Full presentation