Differentiable programming across the PDE/ML divide

Differentiable programming is a concept which is attracting a lot of attention in the machine learning community. It’s usually defined as a programming model which enables inverse problems to be solved through algorithmic differentiation (AD). This immediately suggests a link with simulation science, in which inverse problems are pervasive, but it’s a definition of what differentiable programming does, rather than what it is. Here I’ll present a definition of differentiable programming which is equally applicable to simulation frameworks such as Firedrake and FEniCS, and to machine learning frameworks such as PyTorch and TensorFlow. I’ll argue that differentiable programming provides a design concept that enables us to create frameworks for inverse simulation problems that reach outside just solving PDEs and incorporate external data and non-PDE operators such as neural networks. I’ll demonstrate both of these capabilities in differentiable programming extensions to Firedrake.

Professor Ham's research interests centre on the development and composition of high level abstractions for scientific computation, particularly geophysical fluids. His interests span computational and computer science and include both new numerical schemes and novel approaches to their implementation.

Download Slides