My primary research involves creating optimization techniques for inference and parameter recovery in the areas of variational inference and numerical algebraic geometry.
Challenges in Variational Inference:
When using normalizing flows, a method in variational inference, to approximate probability distributions, challenges arise when the distribution has a complicated geometric shape or underlying model that is costly to evaluate. When there are multiple modes or highly correlated parameters, normalizing flow may not accurately capture the distribution. For example, in multimodal cases, the approximated distribution may not capture all of the modes. Additionally, when a posterior distribution has an underlying model that is costly to evaluate, the problem may become intractable due to computational cost. These issues can be resolved using different optimization techniques, which I discuss below from my research.
Figure 1
(a) bimodal probability distribution from closed form expression
(b) bimodal posterior distribution from ODE model for HIV dynamics
(c) highly-correlated marginal posterior from ODE model for blood flow
AdaAnn: Adaptive Annealing Scheduler for Probability Density Approximation
When probability distributions have multiple modes or highly correlated parameters, it can be difficult for to accurately approximation the distribution without additional optimization techniques. Our adaptive annealing scheduler AdaAnn helps to accurately capture the correct distribution while increasing computational efficiency compared to other common annealing schedulers. The annealing scheduler creates a sequence of intermediate distributions that are easier to approximate throughout the optimization. The corresponding figures illustrates how the step size of the scheduler continues to adapt throughout the entire annealing schedule.
LINFA: A Python Library for Variational Inference with Normalizing Flow and Annealing
LINFA is a publicly available Python package that is for posterior distributions with an underlying model that is computationally expensive to evaluate and when the distribution has a complicated shape, such as multiple modes or highly correlated parameters. This package incorporates a surrogate model with our adaptive annealing scheduler AdaAnn to decrease computational cost and increase accuracy. The corresponding figure, with highly correlated parameters and an underlying ODE model, was approximated with LINFA.
More details can be found in my papers
Emma R. Cobian 