David Duvenaud
Computer scientist pursues generative models in deep learning
David Duvenaud, assistant professor in computer science at the University of Toronto, has been awarded a 2022 Sloan Research Fellowship in support of his research in deep learning.
He says the goal of his research is to build generative models, as opposed to simply decision-making classifiers or predictors, because generative models can, in principle, answer any question about the domain, in addition to classification or predicting.
“For example, in a medical setting, we could build a classifier that predicts the probability of a disease given all the data we’ve seen about the patient,” says Dr. Duvenaud. “But if we instead built a generative model, we could ask for the joint probability of more than one disease at a time or ask to see what representative data for patients with the disease looks like. This helps us check the quality of the model, and also lets us make more sophisticated decisions.
He says a generative model also usually makes it easier to incorporate many kinds of data.
Dr. Duvenaud plans to use funds from the Sloan Fellowship, which is an unrestricted research gift, to support his group’s development of “deep, natively continuous-time models.”
“The standard approach to handling data that comes through time, such as medical records, has been to divide time into discrete bins, and somehow average all the data together within a bin,” he says. “This is usually pretty fast and simple. But it unfortunately requires that we somehow squash the data into a pre-specified format before the model can even look at it. Instead, we’re building models that can directly ingest data in a much rawer format. This requires a bit more thought into the model setup, and making them generative also requires an extra step, called ‘approximate inference.’ But I’d argue that we were previously pushing much of that complexity onto the users of these methods by forcing them to do their own pre-processing. In a sense, we’re trying to automate more of the scientific modeling pipeline.”
On winning the Sloan Fellowship, Dr. Duvenaud says he had “made my peace two years ago with never getting it, since I had aged out of eligibility. But they changed the rules last year to allow anyone pre-tenure to apply. Mainly I am thankful to my letter-writers, who I felt bad about asking for time-consuming favours.”
He says these kinds of awards are useful for “giving non-insiders a rough idea of who is seen as productive and impactful in a field.”
