Dustin Tran

Research Scientist at Google
trandustin@google.com
Blog

I am a research scientist at Google and a fourth-year Ph.D. student, where I am advised by David Blei and Andrew Gelman. I am broadly interested in advancing science and intelligence, and where the ideas involve probability, programs, and/or neural nets.

I lead development of Edward, a probabilistic programming language in TensorFlow, and contribute to TensorFlow Probability. I used to be on the Stan development team. Previously, I was a Statistics Ph.D. student at Harvard before transferring to Columbia, where I worked with Edo Airoldi and also spent time at the Harvard Intelligent Probabilistic Systems group.

Recently, I have been giving the following talk:

• An Overview of Edward: A Probabilistic Programming System Slides
• Why Aren't You Using Probabilistic Programming? Slides (An opinionated version)

Curriculum Vitae

Publications

Preprints

Some of my work is available as preprints on arXiv.

TensorFlow Distributions
A backend for efficient, composable manipulation of probability distributions.
Joshua V. Dillon, Ian Langmore, Dustin Tran, Eugene Brevdo, Srinivas Vasudevan, Dave Moore, Brian Patton, Alex Alemi, Matt Hoffman, Rif A. Saurous

Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data
How to distribute inference with massive data sets and how to combine inferences from many data sets.
Andrew Gelman, Aki Vehtari, Pasi Jylänki, Tuomas Sivula, Dustin Tran, Swupnil Sahai, Paul Blomstedt, John P. Cunningham, David Schiminovich, Christian Robert

Edward: A library for probabilistic modeling, inference, and criticism
Everything and anything about probabilistic models.
Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, David M. Blei

Model criticism for Bayesian causal inference
How to validate inferences from causal models.
Dustin Tran, Francisco J. R. Ruiz, Susan Athey, David M. Blei

Stochastic gradient descent methods for estimation with large data sets
Fast and statistically efficient algorithms for generalized linear models and M-estimation.
Dustin Tran, Panos Toulis, Edoardo M. Airoldi
Journal of Statistical Software, To appear

2018

Implicit causal models for genome-wide association studies
Generative models applied to causality in genomics.
Dustin Tran, David M. Blei
International Conference on Learning Representations, 2018

Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches
How to make weight perturbations in evolution strategies and variational BNNs as mini-batch-friendly as activation perturbations in dropout and batch norm.
Yeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, Roger Grosse
International Conference on Learning Representations, 2018

2017

Hierarchical implicit models and likelihood-free variational inference
Combining the idea of implicit densities with hierarchical Bayesian modeling and deep neural networks.
Dustin Tran, Rajesh Ranganath, David M. Blei
Neural Information Processing Systems, 2017

Variational inference via $\chi$-upper bound minimization
Overdispersed approximations and upper bounding the model evidence.
Adji B. Dieng, Dustin Tran, Rajesh Ranganath, John Paisley, David M. Blei
Neural Information Processing Systems, 2017

Comment, "Fast approximate inference for arbitrarily large semiparametric regression models via message passing"
The role of message passing in automated inference.
Dustin Tran, David M. Blei
Journal of the American Statistical Association, 112(517):156–158, 2017

Automatic differentiation variational inference
An automated tool for black box variational inference, available in Stan.
Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, David M. Blei
Journal of Machine Learning Research, 18(14):1–45, 2017

Deep probabilistic programming
How to build a language with rich compositionality for modeling and inference.
Dustin Tran, Matthew D. Hoffman, Rif A. Saurous, Eugene Brevdo, Kevin Murphy, David M. Blei
International Conference on Learning Representations, 2017

2016

Operator variational inference
How to formalize computational and statistical tradeoffs in variational inference.
Rajesh Ranganath, Jaan Altosaar, Dustin Tran, and David M. Blei
Neural Information Processing Systems, 2016

Hierarchical variational models
A Bayesian formalism for constructing expressive variational families.
Rajesh Ranganath, Dustin Tran, David M. Blei
International Conference on Machine Learning, 2016

Spectral M-estimation with application to hidden Markov models
Applying M-estimation for sample efficiency and robustness in moment-based estimators.
Dustin Tran, Minjae Kim, Finale Doshi-Velez
Artificial Intelligence and Statistics, 2016

Towards stability and optimality in stochastic gradient descent
A stochastic gradient method combining numerical stability and statistical efficiency.
Panos Toulis, Dustin Tran, Edoardo M. Airoldi
Artificial Intelligence and Statistics, 2016

The variational Gaussian process
A powerful variational model that can universally approximate any posterior.
Dustin Tran, Rajesh Ranganath, David M. Blei
International Conference on Learning Representations, 2016

2015

Copula variational inference
Posterior approximations using copulas, which find meaningful dependence between latent variables.
Dustin Tran, David M. Blei, Edoardo M. Airoldi
Neural Information Processing Systems, 2015

Software

TensorFlow Probability

TensorFlow Probability is a library for probabilistic reasoning and statistical analysis in TensorFlow. As part of the TensorFlow ecosystem, TensorFlow Probability provides integration of probabilistic methods with deep networks, gradient-based inference via automatic differentiation, and scalability to large datasets and models via hardware acceleration (e.g., GPUs) and distributed computation.

Edward

Edward is a Python library for probabilistic modeling, inference, and criticism. It is a testbed for fast experimentation and research with probabilistic models, ranging from classical hierarchical models on small data sets to complex deep probabilistic models on large data sets. (See Edward2 in TensorFlow Probability for latest developments.)

Blog