About Me

That's me:




The research problems I am most interested in tend to fall under the umbrella of "computational sustainability" - using computational tools to figure out how to manage the Earth's resources so that we as humans do our best at the game of long-term thriving. I believe that the reinforcement learning formalism is particularly well suited for modelling these types of problems, because it involves an explicit trade-off between immediate and long-term rewards. Coincidentally, I just finished a computer science PhD focused on reinforcement learning - isn't it nice when things just work out like that?

My PhD work has focused on investigating the properties of value function estimators in batch reinforcement learning. This involves using mathematics and simulations in order to analyze methods for predicting the total reward obtained under some intervention policy. The math is mostly statistics (looking at estimator biases, variances, and consistency), and the simulations have been based on natural resource management models (one of the Alaskan halibut, and another of the Canadian mallard).

Currently, I am involved in the management of the natural resources inside the Earth's crust - I recently started a postdoc in mining engineering as part of McGill's COSMO Lab. This should be a good opportunity to learn more about modelling systems where humans and nature interact, and to contribute to better mining practices in the process.

I started my PhD at McGill's Reasoning and Learning Lab in 2007, under the supervision of Doina Precup. A few years after, Joelle Pineau has become my co-supervisor. They are a great team to work with. Previously, I completed an MSc in computer science at the University of Alberta under the supervision of Rich Sutton and Vadim Bulitko, and a BSc (also in computer science, but with a solid dose of mathematics) from the University of Bucharest.

Outside my PhD, I read a lot about energy, natural resources, policy and human interactions. I don't read many novels anymore, since there is plenty of fiction to be found in media articles. Speaking of media, one of the projects I am involved in is a blog called Staring Down the Barrel.