Past Seminars: 2021

Abstract: OneCup AI uses computer vision to uniquely identify and monitor livestock in rural locations. The development of this technology includes solving problems that don’t occur in other AI & ML industries as we also need to consider lack of connectivity, outdoor installation and the unpredictability of livestock. This traditional sector is slow to adopt new technologies due to a lack of knowledge as to how these proposed systems work and what ROI they can provide. As well, new technology can often be cost-prohibitive to a producer operating on slim margins. OneCup’s AI system BETSY, or Bovine Expert Tracking and Surveillance not only meets the easy to use requirements of a rancher but does so with state of the art machine and deep learning. We bring to the table a novel use of AI in a traditional industry such as livestock management. Developing the system to utilize all the new technology available to us, we leverage that and listen to producers to determine the best future functionality of BETSY.

Presenter Bio: Geoffrey has a distinguished career leveraging cutting edge technologies with several startup exits. Accolades include Entrepreneur of the Year and Associate Founder, Singularity University. He holds a Master's degree in Computer Science from GeorgiaTech, specializing in Artificial Intelligence and Deep Learning. A lifelong learner actively seeking opportunities to deepen his knowledge in a range of subjects.

Abstract: Throughout the COVID-19 pandemic, countries relied on a variety of ad-hoc border control protocols to allow for non-essential travel while safeguarding public health: from quarantining all travellers to restricting entry from select nations based on population-level epidemiological metrics such as cases, deaths or testing positivity rates. Here we report the design and performance of a reinforcement learning system, nicknamed ‘Eva’. In the summer of 2020, Eva was deployed across all Greek borders to limit the influx of asymptomatic travellers infected with SARS-CoV-2, and to inform border policies through real-time estimates of COVID-19 prevalence. In contrast to country-wide protocols, Eva allocated Greece’s limited testing resources based upon incoming travellers’ demographic information and testing results from previous travellers. By comparing Eva’s performance against modelled counterfactual scenarios, we show that Eva identified 1.85 times as many asymptomatic, infected travellers as random surveillance testing, with up to 2-4 times as many during peak travel, and 1.25-1.45 times as many asymptomatic, infected travellers as testing policies that only utilize epidemiological metrics. We demonstrate that this latter benefit arises, at least partially, because population-level epidemiological metrics had limited predictive value for the actual prevalence of SARS-CoV-2 among asymptomatic travellers and exhibited strong country-specific idiosyncrasies in the summer of 2020. Our results raise serious concerns on the effectiveness of country-agnostic internationally proposed border control policies that are based on population-level epidemiological metrics. Instead, our work represents a successful example of the potential of reinforcement learning and real-time data for safeguarding public health.

Presenter Bio: Hamsa Bastani is an Assistant Professor of Operations, Information, and Decisions at the Wharton School, University of Pennsylvania. Her research focuses on developing novel machine learning algorithms for data-driven decision-making, with applications to healthcare operations, social good, and revenue management. She designs methods for sequential decision-making, transfer learning and human-in-the-loop analytics. Her applied work uses large-scale, novel data sources to inform policy around impactful societal problems. Her work has received several recognitions, including the Wagner Prize for Excellence in Practice (2021), the Pierskalla Award for the best paper in healthcare (2016, 2019, 2021), the Behavioral OM Best Paper Award (2021), as well as first place in the George Nicholson and MSOM student paper competitions (2016).

Abstract: Pathology is the study of the causes and effects of disease. A pathologist is a medical professional who evaluates patient samples to determine whether or not they have a particular disease. In many serious diseases such as cancer, a pathologist’s evaluation is considered the gold standard or ground truth and has a profound impact on a patient’s life. Unfortunately, various studies in the community have found that discordance amongst pathologists is common leading to errors in diagnoses which can lead to worse patient outcomes. At PathAI, we have demonstrated the ability of using artificial intelligence to aid pathologists in making better diagnoses and as well in the discovery of new biomarkers that can be used to predict response to potential life saving treatments. In this talk, I will focus on a couple of areas that we have worked on, including cancer and liver disease.

Presenter Bio: Aditya completed his PhD in machine learning and computer vision at MIT. He completed his MS at Stanford in 2011 and BS at Caltech in 2009. In his research he developed new methods for an array of applications in computer vision, including eye-tracking, prediction of image memorability, and visualization of deep networks. He is the recipient of a Facebook Fellowship and his work has been widely covered by various media outlets including BBC, The New York Times and The Washington Post. He has published over 30 papers in the fields of deep learning, computer vision and neuroscience.

Abstract: MEDO.ai is one of Edmonton's many startup success stories, offering an AI platform that makes obtaining and interpreting ultrasound images faster, more reliable and accessible to all. However, many are unaware of the academic origins of the technology – let alone how Medo transitioned from academic advancement into a commercial success. In this special Edmonton Startup Week edition of the AI Seminar, join Amii Fellow and Canada CIFAR AI Chair Jacob Jaremko, also a clinician-scientist and co-founder of MEDO.ai, as he walks us through the company's origins and the steps the team took to launch what has grown into a key member of the Edmonton startup community.

Presenter Bio: Fellow and Canada CIFAR AI Chair at Amii, Jacob Jaremko is a clinician-scientist, currently an Associate Professor in the Faculty of Medicine at the University of Alberta, a practicing Pediatric and Musculoskeletal radiologist and partner at Medical Imaging Consultants, and co-founder of MEDO.ai. His research has focused on the influence of anatomy & childhood development of joints on the development of adult osteoarthritis. Seeking to develop objective imaging biomarkers of disease, he has generated 3D ultrasound tools for assessment of infant hip dysplasia, and semi-quantitative MRI scoring systems for arthritis.

Medical images are becoming easier to acquire than to interpret reliably. Dr. Jaremko is increasingly focused on automating medical image analysis, particularly in ultrasound, using artificial intelligence. Tools from natural image and video processing can be adapted to medical image analysis, with special attention to problems such as small training data sets and extreme class imbalance. A handheld ultrasound probe with AI image interpretation could ultimately be used by clinicians at any point of care -- becoming the 21st-century stethoscope!

Abstract: Digital Advertising is growing at about $60B per year globally and not showing any signs of slowing down. LoKnow Inc. is right in the middle of this industry and working on some of the most interesting opportunities, many of them with AI and ML possibilities. Gain some insight from the executive team and the technology team during an intro to the challenges in this industry and LoKnow’s vision of having everyone benefit from, and love the vast world of digital advertising.

Presenter Bios: Tyler Hanson wishfully describes himself as a “senior millennial,” but he is unavoidably the office dad. Whether he’s dispensing wisdom or busting guts with his subtle wit, he’s always happy to lend his guidance to the LoKnow team. A “Knower” since day one, he served on LoKnow’s Board of Directors since the company’s founding before stepping into the role of President in 2020. An mechanical engineer and proud alumnus of the University of Alberta, Tyler is no stranger to the boardroom. He likes to give back to his alma mater by volunteering his time serving on both the U of A Senate and as the President of the Alumni Association.

Nolan McMahon is our resident machine learning specialist, but he himself is a learning machine. Graduating from the University of Calgary with an honours degree in physics, Nolan discovered his passion for machine learning when he got to test the efficacy of quantum computers. He spends a lot of his time studying and reading university-level textbooks on physics and electrical engineering. When he really wants to relax, Nolan plays computer games that tickle his strategy-loving nature. An avid learner, Nolan thinks of his university experience as learning how to learn, and is fascinated by teaching. He particularly looks up to physicist Richard Feynman and his easy-to-follow approach to teaching difficult concepts. Along with his interest in electrical engineering, he would someday like to delve deeper into those two disciplines.

Abstract: Normalizing Flows are a class of generative model which have been growing in popularity recently. NFs are a highly expressive form of generative model which allow for efficient sampling, exact likelihood computation and are quickly becoming competitive with GANs. This talk will provide a brief review of NFs and present a selection of results from my group on both theoretical aspects of flows and their practical application. In particular, I will: show theoretical results around the tail behavior of typical NF architectures; show an application of NFs to modeling realistic image noise in camera systems; show how NFs can be successfully applied to high resolution signals; and show how NFs can be used to build more expressive but analytically tractable stochastic processes.

Presenter Bio: Marcus Brubaker is an Assistant Professor at York University, a Faculty Affiliate of the Vector Institute and an Adjunct Professor at the University of Toronto. From 2018-2020 he worked as Research Director of Borealis AI, a machine learning research lab founded by the Royal Bank of Canada. He is an Associate Editor for IET Computer Vision and regularly serves as an area chair for machine learning and computer vision conferences including ECCV 2018, WACV 2019, UAI 2019, AAAI 2021 and CVPR 2021. Previously he worked on problems in electron cryomicroscopy (cryo-EM), human motion estimation, autonomous driving and Monte Carlo methods. His current research is focused on generative models and he continues to explore methods in cryo-EM.

Abstract: Recent work has shown that sparse representations -- where only a small percentage of units are active -- can significantly reduce interference. Those works, however, relied on relatively complex regularization or meta-learning approaches, that have only been used offline in a pre-training phase. In this work, we pursue a direction that achieves sparsity by design, rather than by learning. Specifically, we design an activation function that produces sparse representations deterministically by construction, and so is more amenable to online training. The idea relies on the simple approach of binning, but overcomes the two key limitations of binning: zero gradients for the flat regions almost everywhere, and lost precision -- reduced discrimination -- due to coarse aggregation. We introduce a Fuzzy Tiling Activation (FTA) that provides non-negligible gradients and produces overlap between bins that improves discrimination. We first show that FTA is robust under covariate shift in a synthetic online supervised learning problem, where we can vary the level of correlation and drift. Then we move to the deep reinforcement learning setting and investigate both value-based and policy gradient algorithms that use neural networks with FTAs, in classic discrete control and Mujoco continuous control environments. We show that algorithms equipped with FTAs are able to learn a stable policy faster without needing target networks on most domains.

Presenter Bio: Yangchen is a PhD candidate at the University of Alberta, working with Martha White and Amir-massoud Farahmand (University of Toronto). He is broadly interested in ML, RL, deep learning. More info about Yangchen can be found on his website: https://yannickycpan.github.io/yangchenpan/

Abstract: If AI or ML is the steak in your sandwich, then data wrangling and data visualization are the bread that allows you to eat it. Together, these enabling skills make your models consumable. In this talk, we will propose a structured approach to data wrangling and look at different data visualization techniques to ensure accurate, accessible insights.

Presenter Bio: Daniel Haight is the President and co-founder of Darkhorse Analytics. He is a Certified Analytics Professional and an award-winning lecturer at the University of Alberta School of Business. His work has spanned healthcare, energy, marketing, professional sports, and transportation. He started his career at Mercer Management Consulting in Toronto advising senior management and jet-setting around the continent. Subsequently, he nearly made millions of dollars in a small Internet startup. Instead, he enjoyed the magnificent failure of the dotcom bust. Along the way, he started a used car dealership, purchased a second-hand trampoline for fifteen dollars, recorded a rock video, and fathered three children. His current work focuses on predictive analytics and data visualization. His goal is to help managers make better decisions by combining their experience with the power of analytics. His even bigger goal is to design a company where Monday mornings are even more exciting than Friday afternoons.

Abstract: Heuristic search algorithms such as A* use a heuristic function to guide its search by focusing on states that are estimated to be closer to a goal state. In this talk we will explore the use of a policy, i.e., a probability distribution over actions, for solving single-agent deterministic problems. I will start by describing Levin tree search, an algorithm that uses a policy and offers guarantees on the number of nodes it needs to expand to solve state-space search problems. We will then discuss Policy-Guided Heuristic Search (PHS), a search algorithm that uses both a policy and a heuristic function to guide its search. PHS also offers guarantees on the number of nodes it needs to expand to solve search problems. I will then present empirical results showing the advantages of policy-guided tree search algorithms, especially when it is difficult to learn effective heuristic functions to guide the search.

Presenter Bio: Levi is an Assistant Professor in the Department of Computing Science at the University of Alberta and a Professor on leave from the Universidade Federal de Viçosa in Brazil. He received his Ph.D. in Computing Science in 2013 from the University of Alberta, studying under the supervision of Robert Holte (Amii Fellow and founding researcher) and Sandra Zilles. Levi has co-authored more than 45 refereed papers at venues such as the International Joint Conference on Artificial Intelligence (IJCAI), the conference for the Association for the Advancement of Artificial Intelligence (AAAI) and the Neural Information Processing Systems (NeurIPS) Conference. He has also served as a Senior Program Committee Member for IJCAI (where he received the honour of being the Distinguished Program Committee Member in 2018 and 2019) and AAAI and Program Committee Member for the Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE).

Abstract: Work has changed for good. We’re collaborating digitally more than ever. This brings new challenges for leaders to successfully create productive, positive and inclusive workplaces. But while digital communities and online collaboration platforms present new challenges, they also present unique opportunities to build culture in ways that didn’t exist before. Machine learning can power an automated culture building program by measuring sentiment, natural language and microaggressions on an anonymized, aggregated level and provide leaders, researchers and advocates with critical and actionable insights about their digital communities and team cultures.

Presenter Bio: Lana Cuthbertson is the CEO and Founder of Areto Labs, a B2B SaaS social enterprise startup that builds technology to make digital communities more positive and inclusive. Her career has taken her from journalism to communications and marketing to product and program management, allowing her to build expertise in technology, digital communications, leadership, innovation, change and knowledge management and writing. She has worked in the technology, financial services, pharmaceuticals, post-secondary and nonprofit industries. She is a passionate advocate for gender equality. She created an A.I. based Twitter bot to encourage women in politics called @ParityBOT, which she ran during elections in Canada, New Zealand and the U.S. and published results at NeurIPS 2019 at the AI for Social Good workshop. She co-founded and chaired ParityYEG in 2018, and chaired the Alberta North chapter of Equal Voice from 2015 to 2017. She was named a Top 40 Under 40 by Avenue Edmonton in 2018.

Abstract: RUNWITHIT started in 2014, using AI-based modelling to address system complexity and risk, building synthetic realities around global digital systems to explore unprecedented futures. Work included modelling around variety of communications, digital, AI, geospatial and IoT systems, and a modelling platform emerged capable of modelling tens of millions of diverse entities to explore emergent behaviour. This has lead to RWI’s current role in supplying rapid Single Synthetic Environments based on an extensive, collaborative catalogue for multiple sectors. One example is energy. As threats to energy grids increase, connecting measures to ensure grid readiness, energy security, and resilience becomes critical. The additional pressures of electrification, decentralization, climate change, and cyber attacks, demand more adaptive scenario planning, mitigating technology and education. Applications of artificial intelligence-based modelling can make these complex futures more accessible to all stakeholders, such as a Single Synthetic Environment, a modelling approach that creates a living digital twin. These accurate geospatial environments include hyper-localized models of the people and businesses, the infrastructure, technology and policies, and then enable any future scenario to play forward. The models don’t rely solely on historical data or precedent; they produce the data required to compare and optimize different options based on quantified impacts and outcomes. This data can range from calculating psychosocial responses in customers such as equity, engagement, adoption, trust, and willingness to pay to measuring demand-side shifts created by new patterns of life, technology adoption, and climate shifts. For High Impact Low Frequency events, Single Synthetic Environments provide insight into the impact of investment choices, expose cascading vulnerabilities, and furnish opportunities to innovate resilience measures. Tomorrow is not like yesterday, and there are incredible opportunities to improve the forward radar of energy transition, mobility and resilience assisted by scenario-based artificial intelligence.

Presenter Bio: Myrna Bittner is the CEO and Co-Founder of RWI Synthetics (2014), an AI-based modelling company that creates Single Synthetic Environments (SSE) for disrupted sectors. SSE's are live, geospatially accurate cities or regions, complete with their people, activity, policies, infrastructure used to calculate the impacts of all kinds of opportunities and risks, existing and anticipated, under any conceivable scenarios. The technology of RWI originates from a previous venture of Myrna's in the ‘90s, NeuralVR, a neural net and 3D visualization research company focused on organizing, aggregating, and visualizing unstructured documents. In 2019, RWI created its first SSE of a city for the Keynote at the IoT World Expo in Silicon Valley. In 2020, Myrna and her team were a part of the Incubatenergy® Labs Challenge, creating a POC of a dual-disaster for EPRI and Phoenix-based utility, Salt River Projects. Building on this, in 2021, EPRI and RWI partnered on a project showcased at AFWERX, reimagining energy for the USAF with Synthetic Bases. RWI is also a finalist in Toyota Mobility Foundation’s City Architecture of Tomorrow Challenge, reimagining the future of mobility in Kuala Lumpur. RWI is a women-led, Certified Aboriginal Business, passionate about diversity and representation in their team, process, and technology, working with clients to ensure that intersectionality, diversity, equity, and inclusion are incorporated into the design of better futures.

Abstract: Edge computing is a promising paradigm that brings servers closer to users, leading to lower latencies and enabling latency-sensitive applications such as cloud gaming, virtual/augmented reality, telepresence, and telecollaboration. Due to the high number of possible edge servers and incoming user requests, the optimum choice of user-server matching has become a difficult challenge, especially in the 5G era where the network can offer very low latencies. In this paper, we introduce the problem of fair server selection as not only complying with an application’s latency threshold but also reducing the variance of the latency among users in the same session. Due to the dynamic and rapidly evolving nature of such an environment and the capacity limitation of the servers, we propose as solution a Reinforcement Learning (RL) method in the form of a Quadruple Q-Learning model with action suppression, Qvalue normalization, and a reward function that minimizes the variance of the latency. Our evaluations in the context of a cloud gaming application show that, compared to a existing methods, our proposed method not only better meets the application’s latency threshold but is also more fair with a reduction of up to 35% in the standard deviation of the latencies experienced by users.

Presenter Bio: Alaa Eddin Alchalabi is an AI enthusiast, futurist, an educator, an artist, and an emerging AI scholar with flair for research and passion for teaching. Alaa is especially passionate about the recent trends in applied AI research and Multimedia Systems, and has been involved in collaborative projects on an international caliber such as projects funded by the European Consortium (Erasmus+), the Turkish TUBITAK agency, the Canadian research council (NSERC), Vector Institute, and the US based the National Science Foundation's (NSF). The AI/Multimedia projects he worked on range from Multi-authentication systems, AI-powered games, Brain-controlled applications, and edge cloud architectures. Alaa is currently pursuing his PhD in Electrical and Computer Engineering from the University of Ottawa, and holds the prestigious Ontario Trillium Award. Alaa is currently working in Swarmio Media as a research scientist.

*this presentation was pre-recorded; the recording is linked above*

Abstract: This is the story of my assorted attempts to do some good with machine learning. Through its telling, I’ll highlight several models of organizing social good efforts, describe half a dozen social good problems that would benefit from our community's attention, and present both resources and challenges for those looking to do some good with ML.

Presenter Bio: Lester Mackey is a Principal Researcher at Microsoft Research, where he develops machine learning methods, models, and theory for large-scale learning tasks driven by applications from healthcare, climate forecasting, and the social good. Lester moved to Microsoft from Stanford University, where he was an assistant professor of Statistics and (by courtesy) of Computer Science. He earned his PhD in Computer Science and MA in Statistics from UC Berkeley and his BSE in Computer Science from Princeton University. He co-organized the second place team in the Netflix Prize competition for collaborative filtering, won the Prize4Life ALS disease progression prediction challenge, won prizes for temperature and precipitation forecasting in the yearlong real-time Subseasonal Climate Forecast Rodeo, and received best paper and best student paper awards from the ACM Conference on Programming Language Design and Implementation and the International Conference on Machine Learning.

Abstract: Vertebral fracture diagnosis is difficult even for experts radiologists, but is integral to the care of patients with osteoporosis. Features are often subtle leading to underdiagnosis but may also be mimicked by normal variants leading to overdiagnosis, a "perfect storm." We discuss our experience with convolutional neural networks (CNNs) for automating vertebral fracture recognition from low-resolution DXA images. Recently, we have applied transfer learning and active learning to address different scan modes and generalization to other cohorts.

Presenter Bio: Dr. Leslie is Professor of Medicine and Radiology at the University of Manitoba with 500 peer-reviewed publications. His research interests are in fracture risk assessment, osteoporosis testing and other nuclear diagnostic techniques. Barret Monchka is a data analyst experienced in working with complex health data. Barret has a Bachelor of Computer Science and is currently completing an MSc in Community Health Sciences at the University of Manitoba.

Abstract: As the use of machine learning in safety-critical domains becomes widespread, the importance of proactively addressing sources of failure and evaluating model reliability has increased. Achieving this, however, can be difficult because the performance and reliability of ML models are vulnerable to being overly dependent on the "context" (i.e., artifacts specific to the training dataset) on which the model was trained. In this talk, we will overview work in this area over the last five years and describe in more detail two example state-of-the-art approaches tackling challenges in building safe and reliable AI. The first describes causally-inspired learning algorithms which allow model developers to specify potentially problematic changes in context and then learn models which are guaranteed to be stable to these shifts. The second tackles monitoring for safety: to be able to evaluate the stability of a model to changes in setting or population, typically requires applying the model to a large number of independent datasets. Since the cost of collecting such datasets is often prohibitive, we will describe a distributionally robust framework for evaluating this type of robustness using the fixed, available evaluation data. This talk will be jointly presented by Prof. Suchi Saria and Adarsh Subbaswamy.

Presenter Bio: Dr. Suchi Saria is the John C. Malone Associate professor of computer science and statistics at the Whiting School of Engineering and of health policy at the Bloomberg School of Public Health. She is also the founding Research Director of the Malone Center for Engineering in Healthcare at Hopkins and the founder of Bayesian Health, which leverages state-of-the-art machine learning and behavior change expertise to unlock improved patient care outcomes at scale by providing real-time precise, patient-specific, and actionable insights at the point of care. Recently, Dr. Saria won a grant from the FDA and is collaborating with them in the development of frameworks for the evaluation of safety and reliability of AI. She was named by IEEE Intelligent Systems as Artificial Intelligence’s “10 to Watch” (2015), MIT Technology Review’s ‘35 Innovators under 35’ (2017), World Economic Forum’s Young Global Leader (2018), DARPA Young Faculty Awardee (2016) and a Sloan Research Fellow (2018). She was invited to join the National Academy of Engineering’s Frontiers of Engineering (2017) and the National Academy of Medicine’s Emerging Leaders in Health and Medicine (2018). She has given over 250 invited talks and is on the editorial board of the Journal of Machine Learning Research.

Abstract: A successful approach to playing two-player, zero-sum games has been to deploy a static artifact resembling a Nash equilibrium, which has led work in artificial intelligence to focus on computing such artifacts. This approach is less sound and has been less successful in multi-player, general-sum games. We suggest instead to field learning algorithms that ensure strong performance in hindsight relative to "deviations", i.e., pre-defined behavior modifications. A society of such "hindsight rational" agents converges toward mediated equilibrium, a traditional notion of equilibrium based on average correlated play rather than factored behavior, in contrast to Nash equilibrium. We re-examine deviation types and mediated equilibria in extensive-form games to gain a more complete understanding and resolve past misconceptions. We introduce a new deviation type that has implicitly formed the basis for the counterfactual regret minimization (CFR) algorithm. We show hints of performance improvements that can be gained by only changing CFR's target deviation set in both zero-sum and non-zero-sum games.

Presenter Bio: Dustin is a Ph.D. candidate at the University of Alberta and the Alberta Machine Intelligence Institute (Amii) working with Professor Michael Bowling. He works on multi-agent learning and scaleable, dependable learning algorithms. He is a coauthor of [DeepStack](https://www.deepstack.ai) and he created [Cepheus's public match interface](http://poker-play.srv.ualberta.ca/). He completed a B.Sc. and M.Sc. in computing science at the University of Alberta where his M.Sc was also supervised by Michael Bowling. As an undergraduate, he worked with the Computer Poker Research Group (CPRG) to create an [open-source web interface to play against poker bots](https://github.com/dmorrill10/acpc_poker_gui_client) and to develop the 1st-place 3-player Kuhn poker entry in the 2014 Annual Computer Poker Competition (ACPC).

Abstract: The vast majority of computer science literature in privacy can be broadly divided into two categories--differential, where the idea is to ensure that participation of an entity or an individual does not change the outcome significantly and inferential, where we are trying to bound the inferences an adversary can make based on auxiliary information.

In this talk, I will present two new case-studies, one in each framework. The first looks at a form of inferential privacy that allows more fine-grained control in a local setting than the individual level. The second looks at privacy against adversaries who have bounded learning capacity, and has ties to the theory of generative adversarial networks.

Joint work with Joseph Geumlek, Jacob Imola and Ashwin Macchanavajhhala.

Presenter Bio: Kamalika Chaudhuri received a Bachelor of Technology degree in Computer Science and Engineering in 2002 from Indian Institute of Technology, Kanpur, and a PhD in Computer Science from University of California at Berkeley in 2007. After a postdoc at the Information Theory and Applications Center at UC San Diego, she joined the CSE department at UC San Diego as an assistant professor. She received an NSF CAREER Award in 2013 and a Hellman Faculty Fellowship in 2012.

Kamalika's research interests are in the foundations of trustworthy machine learning, which includes problems such as learning from sensitive data while preserving privacy, learning under sampling bias, and in the presence of an adversary. She is particularly interested in privacy-preserving machine learning, which addresses how to learn good models and predictors from sensitive data, while preserving the privacy of individuals.

Abstract: Markov decision processes (MDPs) is a minimalist framework that is designed to capture the most important aspects of decision making under uncertainty, a problem of major practical interest and thoroughly studies in reinforcement learning. The unfortunate price of the minimalist approach is that MDPs lack structure and as such planning and learning in MDPs with combinatorial-sized state and action spaces is strongly intractable: Bellman's curse of dimensionality is here to stay in the worst-case. However, apparently already Bellman and his co-workers realized as early as in the 1960s that for many problem of practical interest, the optimal value function of an MDP is well approximated by just using a few basis functions that are standardly used in numerical calculations. As knowing the optimal value function is essentially equivalent to knowing how to act optimally, one hopes that there will be some algorithms that can efficiently compute the few approximating coefficients. If this is possible, we can think of the algorithm as computing the value function in a compressed space. However, until recently not much has been known about whether these compressed computations are possible and when. In this talk, I will discuss a few recent results (some positive, some negative) that are concerned with these compressed computations and conclude with some open problems.

Presenter Bio: Csaba Szepesvári is a Canada CIFAR AI Chair, the team-lead for the “Foundations” team at DeepMind and a Professor of Computing Science at the University of Alberta. He earned his PhD in 1999 from Jozsef Attila University, in Szeged, Hungary. In addition to regularly publishing at top tier journals and conferences, he has (co-)authored three books. Currently, he serves as the action editor of the Journal of Machine Learning Research and as an associate editor of the Mathematics of Operations Research journal, in addition to serving regularly on program committees of various machine learning and AI conferences. Dr. Szepesvári's main interest is developing principled, learning-based approaches to artificial intelligence (AI). He is the co-inventor of UCT, an influential Monte-Carlo tree search algorithm, a variant of which was used in the AlphaGo program which, in a landmark game, defeated the top Go professional Lee Sedol in 2016, ten years of the invention of UCT. In 2020, Dr. Szepesvári co-founded the weekly “Reinforcement Learning Theory virtual seminar series”, which showcases top theoretical work in the area of reinforcement learning with speakers and which is open to attendees from all over the world.

Abstract: In 2016, Karen rolled out of bed with a groan. Realizing that the aging process was only going to get stronger, she set out to find a solution for her aches and pains. Crossfit left her sore for days and yoga caused her to pull more muscles more than stretch them. She decided to embrace her passion for technology and develop her own stretching app, one that would methodically get your body moving without the pain of traditional exercise. Working alongside a local physiotherapist, Karen figured out how to move every joint in every direction, providing lubrication to connective tissue and improving overall blood circulation. After two and a half years of trial and error, WeStretch was released to the Apple Store and Google Play. Since WeStretch is a stretching app with an animated virtual trainer, it was designed using Unity software alongside various algorithms to generate unique and customized stretching routines using artificial intelligence. In the time it has been on the market, WeStretch has been able to release updates that methodically work on targeting specific problem areas of the body, helping athletes improve their sport with customized warmups and cool downs, and it has routines that will help anyone improve their overall functional strength without equipment or going to the gym. (To see why functional strength is important, try going from standing to sitting to lying down three times in a row!) As a fun addition, WeStretch has added the “Alberta Tour” which has Ada, your virtual trainer, stretching in front of some of the iconic landmarks of Alberta, such as the Three Sisters Mountain or Pinto McBean. Ultimately, Karen’s lessons with app creation are to just get started, fail fast then fail faster, and don’t be scared to embrace change.

Presenter Bio: Karen Willsey is a U of A alumni, graduating with a Bachelor of Science, specializing in comp-sci in 1994. She was also a competitive gymnast on the U of A team, passing the Elite Canada exam and competing across Canada and the United States for the university. Karen has been married to her husband, Kevin, for 27 years and they have six children together, four of which have studied at the U of A, and two under the age of 15. Growing up with an entrepreneurial background, Karen went from managing convenience stores to designing custom software for her father’s trucking company, to managing and owning residential and commercial real estate. Designing apps is her new passion and WeStretch is just her starting point.

Abstract: Efficiently navigating a superpressure balloon in the stratosphere requires the integration of a multitude of cues, such as wind speed and solar elevation, and the process is complicated by forecast errors and sparse wind measurements. Coupled with the need to make decisions in real time, these factors rule out the use of conventional control techniques. This talk describes the use of reinforcement learning to create a high-performing flight controller for Loon superpressure balloons. Our algorithm uses data augmentation and a self-correcting design to overcome the key technical challenge of reinforcement learning from imperfect data, which has proved to be a major obstacle to its application to physical systems. We deployed our controller to station Loon balloons at multiple locations across the globe, including a 39-day controlled experiment over the Pacific Ocean. Analyses show that the controller outperforms Loon’s previous algorithm and is robust to the natural diversity in stratospheric winds. These results demonstrate that reinforcement learning is an effective solution to real-world autonomous control problems in which neither conventional methods nor human intervention suffice, offering clues about what may be needed to create artificially intelligent agents that continuously interact with real, dynamic environments.

Bio: Marlos C. Machado is a research scientist at DeepMind Alberta. His research interests lie broadly in artificial Intelligence and particularly focus on reinforcement learning. He received his B.Sc. and M.Sc. from Universidade Federal de Minas Gerais, in Brazil, and his Ph.D. from the University of Alberta, where he introduced the idea of temporally-extended exploration through options. He was a research scientist at Google Brain from 2019 to 2021, during which time he made major contributions to reinforcement learning, in particular the introduction of an operator view of policy gradient methods and the application of deep reinforcement learning to control Loon’s stratospheric balloons. Marlos C. Machado is also an adjunct professor at the University of Alberta.

Abstract: Autonomous mobile robots are becoming pervasive in everyday life, and hybrid approaches that merge traditional control theory and modern data-driven methods are becoming increasingly important. In the first half of seminar, we begin with a discussion of safety verification methods, and their computational and practical challenges. In the second half, we examine connections between optimal control and reinforcement learning, and between optimal control and visual navigation.

Bio: Mo Chen is an Assistant Professor in the School of Computing Science at Simon Fraser University, Burnaby, BC, Canada, where he directs the Multi-Agent Robotic Systems Lab. He completed his PhD in the Electrical Engineering and Computer Sciences Department at the University of California, Berkeley with Claire Tomlin in 2017, and received his BASc in Engineering Physics from the University of British Columbia in 2011. From 2017 to 2018, Mo was a postdoctoral researcher in the Aeronautics and Astronautics Department at Stanford University with Marco Pavone. His research interests include multi-agent systems, safety-critical systems, reinforcement learning, and human-robot interactions.