CS Colloquium: Probabilistic Deep Learning and Black Box Variational Inference

Scientists and scholars across many fields seek to answer questions in their respective disciplines using large data sets. One approach to answering such questions is to use probabilistic generative models. Generative models help scientists express domain knowledge, uncover hidden structure, and form predictions. In this talk, I present my work on making generative modeling more expressive and easier to use. First, I present a multi-layer probabilistic model called deep exponential families (DEFs). Deep exponential families uncover coarse-to-fine hidden structure. These models can be used as components of larger models to solve applied problems, such as in recommendation systems or medical diagnosis. Though expressive, DEFs come with an analytical challenge---scientists need to compute the hidden structure given observed data, i.e., posterior inference. Using classical methods for inference in DEFs is tedious and impractical for non-experts. Thus, in the second part of the talk, I will describe my work on black box variational inference (BBVI). BBVI is an optimization based algorithm to approximate the posterior. BBVI expands the reach of variational inference to new models, improves the fidelity of the approximation, and allows for new types of variational inference. We study BBVI in the context of DEFs to fit complex models of text and medical records. Black box variational methods make probabilistic generative models and Bayesian deep learning more accessible to the broader scientific community.

Bio:
Rajesh Ranganath is a PhD candidate in the Computer Science Department at Princeton University. He works on easy-to-use, flexible machine learning methods with David Blei and on machine learning for medicine with collaborators at the Columbia University Medical Center. He obtained his BS and MS from Stanford University in computer science. Rajesh has won several awards and fellowships including the NDSEG graduate fellowship and the Porter Ogden Jacobus Fellowship, the highest honor for doctoral students at Princeton University.