Welcome to Spring 2022
ESE 650: Learning in Robotics
Term: Spring 2022
Location: ANNS 110
Time: Tuesday & Thursday, 1:45 PM - 3:15 PM
Course Instructors
Course Description
A robot is a machine that senses its environment using sensors, interacts with this environment using actuators to perform a given task and does so efficiently using previous experience of performing similar tasks. In ESE 650, we will cover the fundamentals of these three aspects of robotics: perception, planning, and learning. A tentative list of topics includes state estimation (EKF, UKF, Particle Filters, visual-inertial odometry), control and planning (LQR, MDPs, sampling-based planning), reinforcement learning (policy gradients, Q-learning, Imitation Learning, Offline RL) and some miscellaneous topics (meta-learning and formal verification). The coursework will have both applied and theoretical aspects. This is a graduate level class; undergraduates will need permission from the instructor to enroll. Some experience with, or appreciation of, robotics is recommended.
Course Syllabus
Module 1: State Estimation
Background on probability
Topics: Markov chains, Hidden Markov Models, Kalman Filter, Extended and Unscented Kalman Filter, particle filters, occupancy grids
Module 2: Control and Planning
Background on linear control, dynamic programming
Topics: Markov Decision Processes, Value and Policy Iteration, Bellman equation, Linear Quadratic Regulator, Linear Quadratic Gaussian, sampling-based motion planning
Module 3: Reinforcement Learning
Background on deep learning and optimization
Topics: Imitation Learning, Policy gradient, Q-Learning, Inverse RL, Model-based RL, Offline RL
Prerequisites:
Required
Proficiency in programming (ENGR 105, CIS 110, CIS 120 or equivalent). All assignments will be based on Python but if you have used a similar language like MATLAB before, you should be able to pick up Python easily. Recitation sessions will provide preparatory material.
Probability (ESE 301, STAT 430, CIS 261, ENM 503, ESE 530 or equivalent)
Linear Algebra (Math 312, EAS 205 or equivalent)
Recommended
Machine Learning or Data Analysis (ESE 305, ESE 402, ESE 545, CIS 519/520, or a first machine learning course)
(Soft recommendation) Optimization (ESE 304, ESE 504, ESE 605)
Textbook References:
Detailed instructor notes in addition to supplementary reading material from the following books.
(Thrun) “Probabilistic Robotics” by Sebastian Thrun, Wolfram Burgard and Dieter Fox. PDF
(Barfoot) “State Estimation for Robotics” by Tim Barfoot. PDF
(Lavalle) “Planning Algorithms” by Steve Lavalle. PDF
(Sutton) “Reinforcement Learning: An Introduction” by Richard Sutton and Andrew Barto PDF
(d2l) Dive into Deep Learning by Aston Zhang, Zack Lipton, Mu Li and Alex Smola available at https://d2l.ai is a good reference to read about deep learning.
(Russell) “Artificial Intelligence: A Modern Approach” by Stuart Russell and Peter Norvig. PDF
The following books contain some advanced material. You can use it for your own reference and to brush up fundamentals of machine learning and optimization.
“Pattern Recognition and Machine Learning” by Christopher Bishop. PDF
“An Invitation to 3-D Vision: From Images to Models“ by Yi Ma, Stefano Soatto, Jana Kosecka, Shankar Sastry. PDF
“Reinforcement Learning and Optimal Control” by Dmitri Bertsekas. Material
“Feedback Systems: An Introduction for Scientists and Engineers” by Karl Johan Astrom and Richard M. Murray, PDF
(Advanced) “Linear Systems Theory” by João P. Hespanha. Website
(Fairly advanced) “Stochastic Models, Information Theory, and Lie Groups, Volume 1: Classical Results and Geometric Methods” by Gregory Chirikjian. PDF
Grading Policy:
4 homeworks (60% in total)
Mid-term exam (20%)
Final project (teams of 3) (20%)
Each student will have 10 “late days” to use during the semester which can be used to submit the deliverables later than they are due. Deliverables that are submitted late after exhausting this quota will result in 50% credit deduction per day. Late days cannot be used for the final project submission.
Canvas: https://canvas.upenn.edu/courses/1626578
Ed: edstem.org/us/courses/16341/discussion/
Lecture Notes (2021): https://pratikac.github.io/pub/21_ese650.pdf
Teaching Assistants
Code of Academic Integrity
University of Pennsylvania's CIS/ESE department encourages collaboration among graduate students. However, it is important to recognize the distinction between collaboration and cheating, which is prohibited and carries serious consequences. Cheating may be defined as using or attempting to use unauthorized assistance, material, or study aids in academic work or examinations. Some examples of cheating are: collaborating on a take-home exam or homework unless explicitly allowed; copying homework; handing in someone else's work as your own; and plagiarism. Any student suspected of cheating will be reported to the Office of Student Conduct.
You are encouraged to collaborate with your peers for solving problems in the homework, reading books and curating other instructional materials to improve your understanding of the concepts taught in the class. While doing so, you might generate code/pseudo-code/solutions for the homeworks/project. When you begin to write your submission you should keep aside all these materials (including your friends) and do things “from scratch”. In short, everything you write/code and submit should be your own work done independently.
You should disclose all collaborations in your submission at the top. If you came across some code as a part of your homework/project you must mention it.
Collaboration is different from cheating. The latter will have serious consequences. Cheating is defined as attempting, abetting or using unauthorized assistance (knowledgeable friend who is not taking the class) or material (e.g., online code). Some examples of cheating are: copying someone else’s work for homework/exams, handing in someone else's work as your own or handing in stuff from the Internet as your own work. These will not be tolerated. Your score for that particular homework or exam will be zeroed out if found guilty, you will be penalized one letter grade and this incident will be reported to the university.
Inclusive Environment
If you have any special needs that affect your coursework/attendance, please talk to the instructor and I will do my best to accommodate them. We are all adults here and we should together create an open and inclusive academic environment. This includes being respectful while interacting with your peers, being receptive to their ideas/thoughts, and, in general, maintaining a scholarly discourse both inside and outside the classroom. Transgressions will be reported to the university.