Past Seminars: 2023

YouTube link

Abstract:
The self-attention mechanism in the transformer architecture is capable of capturing long-range dependencies and it is the main reason behind its effectiveness in processing sequential data. Nevertheless, despite their success, transformers have two significant drawbacks that still limit their broader applicability: (1) In order to remember past information, the self-attention mechanism requires access to the whole history to be provided as context. (2) The inference cost in transformers is expensive. In this paper we introduce recurrent alternatives to the transformer self-attention mechanism that offer a context-independent inference cost, leverage long-range dependencies effectively, and perform well in practice. We evaluate our approaches in reinforcement learning problems where the aforementioned computational limitations make the application of transformers nearly infeasible. We quantify the impact of the different components of our architecture in a diagnostic environment and assess performance gains in 2D and 3D pixel-based partially-observable environments. When compared to a state-of-the-art architecture, GTrXL, inference in our approach is at least 40% cheaper while reducing memory use in more than 50%. Our approach either performs similarly or better than GTrXL, improving more than 37% upon GTrXL performance on harder tasks.

Presenter Bio:
Subhojeet Pramanik works in deep learning and reinforcement learning. His expertise lies in building scalable algorithms, specifically in designing efficient transformer architectures. He completed his MSc at the University of Alberta under the supervision of Adam White and Marlos C. Machado. During his masters, he worked in representation learning in the partially observable reinforcement learning setting. He was also an intern at the Huawei Research Lab in Edmonton working on applying reinforcement learning techniques to deep learning operator fusion. Previously, Subhojeet worked at IBM Cloud for two years as an ML Engineer in topics such as deep learning and NLP.  

YouTube link

Abstract:
This talk will present several challenges and opportunities within the area of team sports analytics and key research areas within multiagent systems (MAS). We specifically consider invasion games, where players invade the opposing team’s territory and can interact anywhere on a playing surface (ice hockey or soccer). We discuss how MAS is well-equipped to study invasion games and will benefit both MAS and sports analytics fields. We highlight topics along two axes: short-term strategy (coaching) and long-term team planning (management).

Presenter Bio:
Dr. David Radke is a Senior Research Scientist with the Chicago Blackhawks of the National Hockey League (NHL). He holds a PhD from the University of Waterloo where he was advised by Dr. Kate Larson and Dr. Tim Brecht. His research areas include artificial intelligence (AI) and ice hockey analytics, specifically focusing on the areas of multiagent systems and reinforcement learning. He has published several papers at several top AI conferences, including IJCAI and AAMAS. He has also published papers at LINHAC, a hockey analytics conference, where he received a best paper award in 2022 for his work with NHL tracking data. Dr. Radke's graduate work was supported by the Vector Institute computing resources and NSERC, Ontario Graduate Scholarship, and Type 1 Cheriton awards.

YouTube link

Abstract:
It is widely believed that model-based reinforcement learning can improve sample efficiency by synthesizing imagined experience which generalizes beyond the data. However, learned value functions also generalize. Why should we expect model generalization to be inherently better? In this talk, I address this question with a simple theoretical result. I also present extensive empirical results which demonstrate that the intuition behind the theory leads to significant practical benefits in environments with underlying structure that allows learned models to generalize.

Presenter Bio:
Kenny Young is a PhD student at the University of Alberta, supervised by Richard Sutton. He has worked on a variety of topics in and around reinforcement learning, including developing the popular MinAtar testbed. Most recently he is interested in model-based reinforcement learning as a means to build agents that are capable of learning, planning and thinking about their world.

YouTube link

Abstract:
This talk develops the idea that reinforcement learning can solve the problems of sports analytics. I focus on ranking player performance. We measure the value of a player’s actions by how much they increase their team’s chance of success (i.e., scoring the next goal). I  describe deep reinforcement learning techniques for building success probability models from sports data.  The resulting action values and player rankings are illustrated with data from the National Hockey League comprising over 3M events.

Presenter Bio:
Dr. Oliver Schulte is a Professor in the School of Computing Science at Simon Fraser University, Vancouver, Canada. He received his Ph.D. from Carnegie Mellon University in 1997. His current research focuses on machine learning for structured data, such as sports events, networks, and relational databases. He has given several invited talks on sports analytics and spent two years at Sportlogiq, a sports analytics company. He has published papers in leading AI and machine learning venues on a variety of topics, including sports analytics, learning Bayesian networks, game theory, and scientific discovery. While he has won some nice awards, his biggest claim to fame may be a draw against chess world champion Gary Kasparov.

YouTube link

Abstract:
Learning from experience and continual adaptation to changing environments is crucial for intelligent robots to solve an open-ended sequence of increasingly complex tasks. In this talk, I’ll outline some oft-ignored, practical challenges of continual learning on real-world robots and address two such issues: (i) How to specify reinforcement learning tasks?, and (ii) How to set up a real-time learning agent? Our findings helped us produce the first demonstration of pixel-based control by real-time model-free learning on four different kinds of real robots from scratch in just a few hours.

Presenter Bio:
Gautham Vasan is a PhD student at the University of Alberta, advised by Dr. Rupam Mahmood. He is interested in building machines with human-like intelligence. His research focuses on policy gradient methods, real-time learning architectures and temporal abstraction in reinforcement learning. At Kindred Systems Inc, he worked on developing deep reinforcement learning techniques for an automated put wall robot, known as SORT, which efficiently identifies apparel items, picks them, places them, and sorts them into complete end-customer orders. His website: https://gauthamvasan.github.io/

YouTube link

Abstract:
The development of tools for cancer imaging is dependent on the availability of public imaging datasets, many of which do not include annotations or derived features. We have generated AI-derived annotations for lung cancer imaging collections within the National Cancer Institute Imaging Data Commons (IDC), where the largest collection contains over 26,000 patients. Our work demonstrates how one can harmonize their data and make it publicly available along with code, to promote reproducibility and transparency of AI workflows.

Presenter Bio:
Dr. Deepa Krishnaswamy is a postdoctoral research fellow at Brigham and Women’s Hospital, working with Dr. Andrey Fedorov in the Surgical Planning Laboratory. She completed her PhD at the University of Alberta under the supervision of Dr. Kumaradevan Punithakumar and Dr. Michelle Noga, in the field of registration and segmentation of the left ventricle in ultrasound volumes. Over the past two years of her postdoc, she has focused on developing publicly available, and reproducible methods, for enhancing lung cancer imaging collections in NCI’s Imaging Data Commons. Additionally, she has developed and contributed to a 3DSlicer plugin for multi-parametric MRI annotation of the prostate. Her other interests include prostate data curation, deep learning approaches for MRI segmentation, and open-source tools for visualization. 

YouTube link

Abstract:
In this seminar presented by the Alberta Machine Intelligence Institute (Amii) and Technology Alberta, Ali Salma, CMO of Lawtiq.com, shows that in the traditional investment landscape, investors face challenges in finding the right opportunities because of the role of intermediaries and the limitations of human judgment. This talk details the AI Advantage at Lawtiq.com, including an introduction to Lawtiq.com and its mission, how AI is integrated into Lawtiq.com's platform, and the benefits of AI-driven investment matching. These benefits include efficiency, accuracy, and personalization.

Presenter Bio:
Ali Salman is a problem solver, growth strategist, and fractional CMO who has worked with Remax, WFG, Subway, Manpower, YMCA, The Brick, Coca-Cola, Columbia University, Maxwell, Mucho Burrito, KIA, Honda, Samsung and other Fortune 500 and INC 5000 companies. Currently he heads Lawtiq.com

YouTube link coming soon!

Abstract:
Robotic Swarms are groups consisting of a large number of simpler robots, which don’t have many individual capabilities, but together can perform various types of tasks.  Oftentimes inspired by their biological counterparts, robotic swarms normally rely on distributed and simple actions, which combined result in complex and emergent behaviors. In this talk, we will present some of the works that we have been developing at VeRLab - UFMG in the areas of navigation, segregation, transportation, localization and mapping with robotics swarms.

Presenter Bio:
Luiz Chaimowicz is a Full Professor in the Computer Science Department at the Universidade Federal de Minas Gerais (UFMG) - Brazil. He received his Ph.D. degree in Computer Science from UFMG in 2002, and from 2003 to 2004, he held a Postdoctoral Research appointment in Robotics with the GRASP Laboratory – University of Pennsylvania. Professor Chaimowicz co-directs UFMG’s Computer Vision and Robotics Laboratory (VeRLab) and the Multidisciplinary Research Laboratory in Digital Games. His research encompasses various aspects of artificial intelligence applied to mobile robotics and digital games. In addition to his research and teaching, he has held different administrative roles at UFMG, including serving as the Chair of the Department of Computer Science from 2021 to 2023, Computer Science Graduate Program Coordinator from 2014 to 2016, and Undergraduate Program Coordinator from 2009 to 2011.

YouTube link

Abstract:
Learning from experience means remaining adaptive and responsive to errors over time. However, gradient-based deep learning can fail dramatically in the continual, online setting. In this work we address this shortcoming by combining two meta-learning methods: the purely online Partition Tree Weighting (PTW) mixture-of-experts algorithm, and a novel variant of the Model-Agnostic Meta-Learning (MAML) initialization-learning procedure. We demonstrate our approach, Replay-MAML PTW, in a piecewise stationary classification task. In this continual, online setting, Replay-MAML PTW matches and even outperforms an augmented learner that is allowed to pre-train offline and is given explicit notification when the task changes. Replay-MAML PTW thus provides a base learner with the benefits of offline training, explicit task sampling, and boundary notification, all for a O(log2 (t)) increase in computation and memory. This makes deep learning more viable for fully online, task-agnostic continual learning, which is at the heart of general intelligence.

Presenter Bio:
Anna Koop recently started as a Research Engineer at Google DeepMind in Montreal, having just about completed her PhD on experiential learning for AI, supervised by Michael Bowling at the University of Alberta. She's been a teacher, programmer, consultant, manager and student in almost every permutation, including Managing Director of the Applied Machine Learning Team at Amii. She is fascinated by learning, can't stay away from research, and is passionately curious about nearly everything.

YouTube link

Abstract:
Reinforcement Learning from Human Feedback (RLHF) has emerged as the central alignment technique used to finetune state-of-the-art AI systems such as GPT-4, Claude, Bard, and Llama-2. Given RLHF's status as the default industry alignment technique, there is a need to carefully study how we got here and what challenges persist in today's state of the art.  We review open challenges and fundamental limitations with RLHF with a focus on applications in large language models. Technical progress in some respects is tractable, and this should be seen as a cause for concerted work and optimism. However, other problems with RLHF cannot fully be solved and instead must be avoided or compensated for with non-RLHF approaches.

Presenter Bio:
Stephen "Casp" Casper is a Ph.D student at MIT in Computer Science (EECS) in the Algorithmic Alignment Group advised by Dylan Hadfield-Menell. His research focuses include interpreting, diagnosing, debugging, and auditing deep learning systems.

YouTube link

Abstract:
When players engage with parts of a video game they do not enjoy, one proposed solution is to adapt the game to the player’s preference. In industry, this adaptation is called “AI directors”, but these are uncommon, domain-specific, and rule-based. In this talk, I will present Nightingale, a reinforcement learning-based AI director developed for the industry game. My team evaluated the effectiveness of this AI director, but found inconclusive evidence. I will present the results and their implications for future AI directors.

Presenter Bio:
Kristen Yu is a PhD Candidate at the University of Alberta supervised by Nathan Sturtevant and co-supervised by Matthew Guzdial. Her research focuses on applying AI techniques to video game design to create a better player experience. 

YouTube link

Abstract:
This talk presents the approach to grounding concepts to vision using text descriptions that focus on visual attributes. The research includes two studies. The first explores visual word sense disambiguation, a task that involves selecting the image that best represents the contextual meaning of a word. The second study focuses on producing visual descriptions for arbitrary, concrete concepts for use in downstream tasks such as zero-shot image classification and generation. Both studies show that conditioning a pre-trained large language model (LLM) with lexico-semantic knowledge empirically improves visual descriptions, thus confirming the utility of LLM-produced visual descriptions in grounding lexical concepts to the visual domain.

Presenter Bio:
Mike Ogezi is completing his MSc at the University of Alberta, focusing on the intersection of natural language processing and computer vision (supervisor: Dr Greg Kondrak). He is particularly interested in enhancing large language models with visual knowledge. Before his MSc, he was a software engineer working on optimal order matching for cryptocurrency trading. In his downtime, he enjoys playing board and video games.

YouTube link

Abstract:
The end-to-end learning ability of self-driving vehicles has achieved significant milestones over the last decade owing to rapid advances in deep learning and computer vision algorithms. However, as autonomous driving technology is a safety-critical application of artificial intelligence (AI), road accidents and established regulatory principles necessitate the need for the explainability of intelligent action choices for self-driving vehicles. To facilitate interpretability of decision-making in autonomous driving, we present a Visual Question Answering (VQA) framework, which explains self-driving actions with question-answering-based causal reasoning. To do so, we first collect driving videos in a simulation environment using reinforcement learning (RL) and extract consecutive frames from this log data uniformly for five selected action categories. Further, we manually annotate the extracted frames using question-answer pairs as justifications for the actions chosen in each scenario. Finally, we evaluate the correctness of the VQA-predicted answers for actions on unseen driving scenes. The empirical results suggest that the VQA mechanism can provide support to interpret real-time decisions of autonomous vehicles and help enhance overall driving safety.

Presenter Bio:
Shahin Atakishiyev received a BSc Computer Engineering degree from Qafqaz University, Azerbaijan, in June 2015 and an MSc Computer Engineering degree with a specialization in Software Engineering and Intelligent Systems from the University of Alberta, Canada, in January 2018. Currently, he is a PhD student in the Department of Computing Science at the University of Alberta under the supervision of Prof. Randy Goebel. Shahin's research interests include Safe, Ethical, and Explainable Artificial Intelligence and its applications to real-world problems.

Abstract:
This talk laments some challenges and pitfalls in the design of reinforcement learning experiments. We will draw parallels between animal learning research and RL and reflect on how animal learning research can influence our own empirical strategies. We will dive into the difference between an agent and an algorithm, and discuss how this pedantic nuance changes how we might analyze our data. Finally, we will explore a common empirical methodology where increasing the number of seeds makes us increasingly confidently incorrect.

Presenter Bio:
Andy Patterson is a PhD student at the University of Alberta working with Martha White on reliable reinforcement learning methods. His focus on reliability has led to studying robust statistics for RL, using gradient-TD strategies in deep RL learning systems, and most recently revisiting commonplace empirical methodologies for evaluating RL methods.

YouTube link coming soon!

Abstract:
Complex reasoning problems contain states that vary in the computational cost required to determine a good action plan. Taking advantage of this property, we present Adaptive Subgoal Search, a trainable search method that plans using high-level steps, adjusted to the local complexity of the environment. Adaptive Subgoal Search benefits from the efficiency of planning with longer subgoals and the fine control with the shorter ones, and thus scales well to difficult planning problems.

Presenter Bio:
Michał Zawalski is a PhD student at the University of Warsaw. His primary research interests revolve around reinforcement learning, planning algorithms, goal-conditioned problems, and multi-agent RL. He's especially interested in finding simple yet effective algorithms.

Michal Tyrolski is currently a student researcher at the University of Warsaw and a Machine Learning Researcher at DeepFlare. He holds a degree from the same university and is currently engaged in work on planning in reinforcement learning in Piotr Miłos' research lab. Michal's areas of interest encompass both meta and model-based reinforcement learning. In the past, he has worked at Nvidia, Microsoft, and Samsung. Since 2020 he has been one of the organizers of the ML in PL Conference, the biggest research-oriented machine learning conference in Poland.

YouTube link

Abstract:
Recognizing dialog acts of users is an essential component in building successful conversational agents. In this work, we propose a dialog act (DA) classifier for two of our open domain conversational agents. For this, we curated a high-quality, multi-domain dataset with ∼24k user utterances labelled into 8 suitable DAs. Our fine-tuned BERT-based model outperforms the baseline SVM classifier by achieving state-of-the-art accuracy on the proposed dataset. Moreover, it generalizes well on unseen data. To address the issue of data scarcity when training DA classifiers, we implemented different data augmentation techniques and compared their performance. Our extensive experiments show that, in a simulated low data regime with only 10 examples per label, methods as simple as synonym replacement can double the size of the existing training data and boost accuracy of our DA classifier by ∼8%. Lastly, we demonstrate how our proposed classifier and augmentation techniques can be adapted to effectively detect dialog acts in languages other than English.

Presenter Bio:
Maliha is a recent Masters graduate from the University of Alberta in the field of Natural Language Processing. She has extensively worked with Large Language Models for detecting user dialog acts in chatbots. Moreover, to tackle the scarcity of labelled training data, she has successfully looked into data augmentation techniques that artificially generate new data to improve model performance. When she is not busy scrolling through LinkedIn looking for a new full-time position as an ML Developer, you can find her singing at a karaoke bar or playing at a board games café. Currently, she is building data pipelines and doing prompt engineering as an Associate ML Developer at AltaML. 

YouTube link

Abstract:
A default assumption in reinforcement learning (RL) and optimal control is that observations arrive at discrete time points on a fixed clock cycle. Yet, many applications involve continuous-time systems where the time discretization, in principle, can be managed. The impact of time discretization on RL methods has not been fully characterized in existing theory, but a more detailed analysis of its effect could reveal opportunities for improving data-efficiency. 

In this talk, I will discuss our work that provides a key initial step to bridge this gap. We analyze Monte-Carlo policy evaluation for Linear Quadratic Regulator (LQR) systems and uncover a fundamental trade-off between approximation and statistical error in value estimation. Importantly, these two errors behave differently to time discretization, leading to an optimal choice of temporal resolution for a given data budget. These findings show that managing the temporal resolution can provably improve policy evaluation efficiency in LQR systems with finite data. Empirically, we demonstrate the trade-off in numerical simulations of LQR instances and standard RL benchmarks for non-linear continuous control. 

Presenter Bio:
Zichen (Vincent) Zhang is a Ph.D. student in Computing Science at University of Alberta, working with Prof. Dale Schuurmans and Prof. Martin Jagersand. He is interested in machine learning and its applications on robotics perception and control. Currently he’s working toward developing efficient algorithms for continuous control. 

YouTube link

Abstract:
Off-policy learning from multistep returns is crucial for sample-efficient reinforcement learning, but counteracting off-policy bias without exacerbating variance is challenging. Classically, off-policy bias is corrected in a per-decision manner: past temporal-difference errors are re-weighted by the instantaneous Importance Sampling (IS) ratio after each action via eligibility traces. Many off-policy algorithms rely on this mechanism, along with differing protocols for cutting the IS ratios to combat the variance of the IS estimator. Unfortunately, once a trace has been fully cut, the effect cannot be reversed. This has led to the development of credit-assignment strategies that account for multiple past experiences at a time. These trajectory-aware methods have not been extensively analyzed, and their theoretical justification remains uncertain.

In this talk, I propose a unifying framework for per-decision and trajectory-aware methods, and establish the first general convergence conditions for trajectory awareness in the tabular setting. I also introduce a new algorithm called Recency-Bounded Importance Sampling (RBIS), which leverages trajectory awareness to perform robustly across hyperparameters in several off-policy control tasks. 

Presenter Bio:
Brett Daley is a Ph.D. student at the University of Alberta working with Marlos C. Machado and Martha White. His research interest is temporal credit assignment for off-policy prediction and control with function approximation. Brett previously earned his M.Sc. in Computer Science from Northeastern University, M.Sc. in Management and Global Affairs from Tsinghua University through the Schwarzman Scholars program, and B.Sc. in Electrical and Computer Engineering from Northeastern University. 

https://www.youtube.com/watch?v=DwQWbDvrRAQ&list=PLKlhhkvvU8-aKj4cWVY4SsOiz4dfKmiUl&index=10

Abstract:
Off-policy policy evaluation has been a critical and challenging problem in reinforcement learning, and Temporal-Difference (TD) learning is one of the most important approaches for addressing it. There has been significant interest in searching for consistent off-policy TD algorithms that are guaranteed to find the on-policy TD fixed point. Notably, Full Importance-Sampling TD is the only existing consistent off-policy TD method under general linear function approximation but, unfortunately, has a high variance and is scarcely practical. This notorious high variance issue motivates the introduction of Emphatic TD, which tames down the variance but has a biased fixed point. Inspired by these two methods, we propose a new consistent algorithm called Average Emphatic TD (AETD) with a transient bias, which strikes a balance between bias and variance. Further, we unify AETD with several existing algorithms and obtain a new family of consistent algorithms called Consistent Emphatic TD (CETD), which can control a smooth bias-variance trade-off by varying the speed at which the transient bias fades. Through theoretical analysis and experiments on a didactic example, we validate the consistency of CETD. Moreover, we show that CETD converges faster to the lowest error in a complex task with a high variance.  

Presenter Bio:
Jiamin is an M.Sc. student at the University of Alberta working with Rupam Mahmood. His research interests lie in artificial intelligence, especially reinforcement learning. Currently, he is focusing on off-policy policy evaluation. 

https://www.youtube.com/watch?v=Dy6K6sM3teE&list=PLKlhhkvvU8-aKj4cWVY4SsOiz4dfKmiUl&index=8

Abstract:
This session explores the unique AI challenges faced during the development of Inflexion Studio’s debut title, Nightingale, a co-op survival-crafting experience in which players venture through portals where adventure and mystery await across a myriad of beautiful and increasingly-dangerous procedurally generated realms.

Arta Seify and Nathan Sturtevant will describe the range of approaches crafted during development to solve challenges in pathfinding and populating the realms with creatures. Learn about the systems devised to support efficient pathfinding across large realms with a changing path network, creature terrain-type preferences, pathfinding for creatures of vastly different sizes, and procedurally filling the realms with wildlife such that they “feel alive”.

Presenter Bio:
Arta Seify has been working as an AI developer on Nightingale since 2021. Previously, he worked as an engine programmer on Minecraft: Bedrock, and before that, was a graduate student at the University of Alberta.

Nathan Sturtevant is a professor at the University of Alberta working on research in AI and games, is the director of the Alberta Machine Intelligence Institute, and a consultant with Inflexion Games. He previously wrote the pathfinding engine in Dragon Age: Origins, in addition to his own game-related research and development projects.

https://www.youtube.com/watch?v=5lyY2eBhcZA&list=PLKlhhkvvU8-aKj4cWVY4SsOiz4dfKmiUl&index=5

Abstract:
Knowledge distillation (KD) is the process of transferring knowledge from a large model to a small one. It has gained increasing attention in the natural language processing community, driven by the demands of compressing ever-growing language models. In this work, we propose an f-distill framework, which formulates sequential knowledge distillation as minimizing a generalized f-divergence function. We propose four distilling variants under our framework and show that existing SeqKD and ENGINE approaches are approximations of our f-distill methods. We further derive step-wise decomposition for our f-distill, reducing intractable sequence-level divergence to word-level losses that can be computed in a tractable manner. Experiments across four datasets show that our methods outperform existing KD approaches, and that our symmetric distilling losses can better force the student to learn from the teacher distribution. 

Presenter Bio:
Yuqiao is a first-year Ph.D. student at the Department of Computing Science, University of Alberta, working with Dr. Lili Mou. In his Master's program, he explored dialogue systems for diverse response generation. He focused on the one-to-many mapping phenomenon in the dialogue task, where a dialogue context may have many plausible responses. Yuqiao addressed the one-to-many phenomenon using mixture models and proposed a novel variant of the expectation-maximization algorithm for latent variable training. Currently, Yuqiao is interested in fundamental issues in natural language generation, such as the one-to-many phenomenon, label bias, and exposure bias. Yuqiao has papers published at LREC'22 and ICLR'23. 

https://www.youtube.com/watch?v=x65q1BrYwmY&list=PLKlhhkvvU8-aKj4cWVY4SsOiz4dfKmiUl&index=7

Abstract:
Recently we have seen many successful applications of Reinforcement Learning (RL).
It is natural to extend the scope of RL to vision-based real-time learning of robotic control tasks.
However, a vision-based real-time robotic RL agent faces some practical issues oft-ignored in conventional RL research.
The first issue we investigated is that robots deployed in the real world are usually tethered to a resource-limited computer, while vision-based RL algorithms are expensive.
The second issue we investigated is how to design an effective reward function that is independent of domain knowledge. 

Presenter Bio:
Yan Wang is a Master's student in the Department of Computing Science at the University of Alberta, supervised by Rupam Mahmood. 

He is interested in applying RL methods to practical questions, especially robotic control tasks.

He likes playing video games, board games, and reading stories about UFOs.

https://www.youtube.com/watch?v=7BExMjBM8ts&list=PLKlhhkvvU8-aKj4cWVY4SsOiz4dfKmiUl&index=4

Abstract:
It is now a standard for neural network representations to be trained on large, publicly available datasets, and used for new problems. The reasons for why neural network representations have been so successful for transfer, however, are still not fully understood. We show that, after training, neural network representations align their top singular vectors to the targets. We investigate this representation alignment phenomenon in a variety of neural network architectures and find that (a) alignment emerges across a variety of different architectures and optimizers, with more alignment arising from depth (b) alignment increases for layers closer to the output and (c) existing high-performance deep CNNs exhibit high levels of alignment. We then highlight why alignment between the top singular vectors and the targets can speed up learning and show in a classic synthetic transfer problem that representation alignment correlates with positive and negative transfer to similar and dissimilar tasks. 

Presenter Bio:
Ehsan Imani is a PhD student at the University of Alberta. He received an MSc degree in computer science from the University of Alberta in 2019. Afterwards, he worked as a Research Assistant at the University of Alberta and later at Huawei Noah's Ark Lab. His research interests are Representation Learning and Reinforcement Learning. 

https://www.youtube.com/watch?v=shsevcJ3zPI&list=PLKlhhkvvU8-aKj4cWVY4SsOiz4dfKmiUl&index=1

Abstract:
In ten years, Drivewyze has grown from startup to managing the largest software deployment in trucking history. This presentation will introduce Drivewyze, business and technical challenges unique to the industry, solutions, and scaling experiences. Finally some opportunities in AI/ML will be introduced. 

Presenter Bio:
Mr. Malkin is a technical leader with more than 30 years of experience building teams and delivering top quality software projects, including past experience at IBM taking solutions to scale (globalization, enterprise scalability, and security). Mr. Malkin is the chief designer and architect for Drivewyze including back‐office services and client applications. He has a dozen US software patents pending or awarded. 

https://www.youtube.com/watch?v=oA_XLqh4Das&list=PLKlhhkvvU8-aKj4cWVY4SsOiz4dfKmiUl&index=3

Abstract:
Modern deep-learning systems are specialized to problem settings in which training occurs once and then never again, as opposed to continual-learning settings in which training occurs continually. If deep-learning systems are applied in a continual learning setting, then it is well known that they may fail catastrophically to remember earlier examples. More fundamental, but less well known, is that they may also lose their ability to adapt to new data, a phenomenon called \textit{loss of plasticity}. We show loss of plasticity using the MNIST and ImageNet datasets repurposed for continual learning as sequences of tasks. In ImageNet, binary classification performance dropped from 89% correct on an early task to 77%, or to about the level of a linear network, on the 2000th task. Such loss of plasticity occurred with a wide range of deep network architectures, optimizers, and activation functions, and was not eased by batch normalization or dropout. In our experiments, loss of plasticity was correlated with the proliferation of dead units, units with very large weights, and more generally with a loss of unit diversity. Loss of plasticity was substantially eased by L2-regularization, particularly when combined with weight perturbation (Shrink and Perturb). We show that plasticity can be fully maintained by a new algorithm---called \emph{continual backpropagation}---which is just like conventional backpropagation except that a small fraction of less-used units are re-initialized after each example. This continual injection of diversity appears to maintain plasticity indefinitely in deep networks. 

Presenter Bio:
Shibhansh is a Ph.D. student at the University of Alberta, working with Dr. Richard Sutton and Dr. Rupam Mahmood. He received his B. Tech. in Computer Science and Engineering from the Indian Institute of Technology Kanpur. He aims to understand the computational principles that give rise to intelligence. To this end, his research focuses on various aspects of Continual learning, Deep learning, and Reinforcement learning.