Syllabus

This course is a rigorous introduction to modern machine learning concepts, techniques, and algorithms. Topics include regression, classification, unsupervised and supervised learning, reinforcement learning, kernels, support vector machines, clustering, Bayesian methods, and more. Problem sets emphasize taking theory into practice, by gaining a thorough mathematical understanding of the machine learning methods, coding them from scratch, and applying them to data sets. 

Course Prerequisites

This is a graduate course that is also open to upper-level undergraduates. The prerequisites are the following:

•  CAS CS 365 Foundations of Data Science  (required for BU undergraduates)

OR

• Linear algebra (CAS CS 132 or MA 242 or equivalent) 

• Multivariate Calculus (eg, CAS MA 225) including partial derivatives

• Probability/Statistics (CAS CS 237 or MA 381 or 581 or equivalent)

• Comfort with programming in Python (eg, CAS CS 111 and 112, or equivalent)

You should be very comfortable with programming in Python and be able to both write and understand code quickly. Intro to Optimization (CAS CS 507) is also highly recommended as a precursor to this course.

Alternative Machine Learning Courses

• CAS CS 599 Applied Machine Learning covers similar algorithms but focuses more on applying them to problems, rather than the mathematical principles behind them

• Introduction to Machine Learning (ENG EC 414) is more suitable for undergraduates

• CAS CS 506 Computational Tools for Data Science 

• https://www.coursera.org/learn/machine-learning/ Andrew Ng’s basic intro to Machine Learning course on Coursera, can be taken as a precursor to this course

• MA 751  Statistical Machine Learning is similar to this course

Textbooks

The required textbook for the course is

• Bishop, C. M. Pattern Recognition and Machine Learning.

Other recommended supplemental textbooks on general machine learning:

• Duda, R.O., Hart, P.E. and Stork, D.G. Pattern Classification. Wiley-Interscience. 2nd Edition. 2001.

• Theodoridis, S. and Koutroumbas, K. Pattern Recognition. Edition 4. Academic Press, 2008.

• Russell, S. and Norvig, N. Artificial Intelligence: A Modern Approach. Prentice Hall Series in Artificial Intelligence. 2003.

• Hastie, T., Tibshirani, R. and Friedman, J. The Elements of Statistical Learning. Springer. 2001.

• Koller, D. and Friedman, N. Probabilistic Graphical Models. MIT Press. 2009.

• Ian Goodfellow, Yoshua Bengio, Aaron Courville. Deep Learning.

• Kevin Patrick Murphy. Probabilistic Machine Learning.

Recommended background reading on matrix calculus 

• Reference reading on matrix calculus and linear algebra can be found here (new link)

• Matrix derivatives cheat sheet

• Useful tool http://www.matrixcalculus.org/ 

Deliverables/Graded Work

The main graded work for the course is the midterm, final, problem sets and participation. There will be six problem sets (ps1-ps6), each consisting of written and programming problems meant to prepare students for the quizzes and two exams. Problem sets will be graded based on percent completed by due date. Note: there are no projects in this course. The course grade consists of the following:

• Problem Sets (graded on effort) 20%

• Quizzes 20%

• Midterm 25%

• Final 25%

• Participation 10% (according to engagements in classes and Piazza; read the following for more details)

Class Participation

A part of the grade is based on class participation, either in person or online. In-person participation means asking and answering questions in class and/or in the TA's or instructors office hours. Since the class size is large, do introduce yourself to the course staff to make sure we know your name! Online participation means asking and answering questions on Piazza. 

Note that Piazza is highly catered to getting you help quickly and efficiently from classmates and instructors. Rather than emailing questions to the teaching staff, we ask you to please post all your questions on Piazza. For private questions like re-grading requests please select just the “Instructors” in the “Post to:” field of the Piazza note. 

Software/Compute Resources

Programming assignments will be developed in the Python programming language, using mostly Jupyter Notebooks. No special computing resources are required besides a regular laptop. GPUs are not necessary.

Students may also have access to the SCC computing cluster, please request this from the instructors. See SCC Quick Start Guide for further information on getting started with the SCC. 

Another resource you may want to use is Google’s Colab, see https://colab.research.google.com/notebooks/welcome.ipynb

Help! My laptop broke!

If you find yourself in a situation where you need a back up machine, say, due to a broken laptop, the BU Common at the Mugar Library offers access to Microsoft Windows 10 virtual machines. The lab is co-located with the IT Help Center and the Print Center and hosts a broad selection of software including Python and Jupyter. Also note that the IT Help Center is an authorized repair center for most major laptop brands, and can often perform many common types of repairs within one day.  They also offer temporary loaners for certain models of laptops while they are working on your machine.

Course Policies

Late and Missed Work

Late problem sets will incur the following penalties

• 20% off per day, up to 2 days (e.g. 100/100 points on due date, 80/100 points 1 day late, 60/100 2 days late)

No extensions will be granted. However, we understand that unforeseen events can happen. For this reason, we will drop the lowest of all the problem sets from your final grade. In order to be fair, this policy will apply to everyone. Exceptions to these policies can only be made in cases of significant medical or family emergencies, with proper documentation. In such cases please send a private message to the instructors via Piazza.

Copyright Notice

All course materials, including lecture slides/notes, videos/screenshots, homeworks, solutions, and solution code are copyrighted by Prof. Saenko. All rights are reserved. Other than for personal use by registered students, the course materials or any portion thereof may not be reproduced or used in any manner whatsoever (including posting on a public forum or website) without the express written permission of the authors. Acting otherwise would go against the ethical code of conduct expected of students at Boston University and from a legal standpoint it may even constitute a violation of copyright law.

Academic Honesty Policy

The instructors take academic honesty very seriously. Cheating, plagiarism and other misconduct will be subject to grading penalties up to failing the course.  Students enrolled in the course are responsible for familiarizing themselves with the detailed BU policy, available here. In particular, plagiarism is defined as follows, and applies to all written materials and software, including material found online. Collaboration on homework only in the form of discussion is allowed, but should be acknowledged and you should always come up with your own solution rather than copy others’ solutions (which is defined as plagiarism).

Plagiarism: Representing the work of another as one’s own. Plagiarism includes but is not limited to the following: copying the answers of another student on an examination, copying or restating the work or ideas of another person or persons in any oral or written work (printed or electronic) without citing the appropriate source, and collaborating with someone else in an academic endeavor without acknowledging his or her contribution. Plagiarism can consist of acts of commission-appropriating the words or ideas of another-or omission failing to acknowledge/document/credit the source or creator of words or ideas (see below for a detailed definition of plagiarism). It also includes colluding with someone else in an academic endeavor without acknowledging his or her contribution, using audio or video footage that comes from another source (including work done by another student) without permission and acknowledgement of that source. 

Prohibited behaviors include: 

• copying all or part of someone else's work, even if you subsequently modify it; this includes cases in which someone tells you what you should write for your solution 

• viewing all or part of someone else's work

• showing all or part of your work to another student 

• consulting solutions from past semesters, or those found online or in books 

• posting your work or our solutions where others can view it (e.g., online). 

Incidents of academic misconduct will be reported to the Academic Conduct Committee (ACC). The ACC may suspend/expel students found guilty of misconduct. At a minimum, students who engage in misconduct will have their final grade reduced by one letter grade (e.g., from a B to a C).

Accommodations for Students with Disabilities

If you need accommodations such as extra time on exams or note-taking services due to a disability, please first contact Disability & Access Services to obtain an accommodation letter. You can then send the letter to the instructors via a private message on Piazza. Please see https://www.bu.edu/disability/incoming-students/. Please see us in office hours if you have any further questions.

Religious Observance

Students are permitted to be absent from class, including classes involving examinations, labs, excursions, and other special events, for purposes of religious observance.  In-class, take-home and lab assignments, and other work shall be made up in consultation with the student’s instructors. More details on BU’s religious observance policy are available here.

COVID-19

There is no remote component of this class, students are expected to attend in person. If you miss class due to COVID-19 or any other health related absence, please check the website for lecture recordings and other posted materials. All students are expected to follow university guidelines. For a detailed description of official BU policies regarding COVID, please visit https://www.bu.edu/covid/.