Computational Optimization Methods and Algorithms

Previous year notes are attached below.

Pre-requisites: 

Mathematics-I, Mathematics-II, Programming (C/C++/MATLAB)

Course Syllabus: 

[Unit-1] Introduction: Scope and philosophy of computational optimization; review of calculus (sets, relations, functions, continuity, differentiability, analysis, vector algebra, matrices, norms, sequence, Taylor series, gradients); various types of optimizations – continuous vs discrete, constrained vs unconstrained, global vs local, deterministic vs stochastic, and linear vs non-linear; review of theory of computations and complexity of algorithms, polynomial-hard and non-polynomial hard problems.  

[Unit-2] Calculus of Optimization: Classical methods of optimization for differentiable functions for both single and multi-variable functions, necessary and sufficient conditions for extremum, Hessian matrix, saddle point; multi-variable optimization with equality constraints, Lagrange multiplier method; multi-variable optimization with inequality constraints, directions of search, usable-feasible directions, Karush-Kuhn-Tucker (KKT) condition, regular point; computational challenges with classical optimization methods. 

[Unit-3] Convex Optimization: Fundamentals of convex optimization, convex sets, convex function and their properties; convex quadratic programming problems; steepest descent direction method, conjugate gradient direction (Fletcher-Reeves) method, quasi-Newton's method, Davidon-Fletcher-Powell (DFP) method, Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm and its variants. 

[Unit-4] Mathematical Programming: Introduction to linear programming problems (LPP) and its formulations, graphical methods, simplex method, duality in optimization problems, dual simplex method, computational considerations of simplex method; introductory integer linear programming (ILP), branch-bound algorithm, applications of ILP and computational consideration. 

Course Outcomes: 

The present course will provide a computational perspective of the optimization theory and algorithmic implementation. By attending the course, students will be able to understand 

1. Computational complexity of optimization problems.

2. Various classical methods for optimization problems where objectives and constraints (equality and inequality) are differentiable functions.

3. Convex optimization algorithms and their application in various engineering fields. 

4. Algorithms to solve various mathematical programming problems.   

Recommended Books: 

Text Books:

1. H A Taha, Operations Research: An Introduction, Pearson Prentice Hall, 8th Edition, 2007.

2. S. Chandra, Jayadev, A. Mehra, Numerical Optimization with Applications, Narosa Publishing,  2nd Edition, 2016. 

Reference Books:

1. J. Nocedal, S. J. Wright, Numerical optimization, Springer, 2nd Edition, 2000.

2. S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press, 1st Edition, 2004. 

Evaluation Scheme: 

Students will be evaluated on a scale of 100 = (0.75*E1+0.5*E2+0.5*E3) where

E1 - 2 Open Book Exams of 50 Marks each for 2 Hr duration

E2 - 1 Poster Presentation on Self-Learning Topic of 25 Marks for 15 Mins

E3 - Creativity Test (Real-Time Problem/ Debate / Game) of 25 Marks 

Note: A minimum of 20 marks are required to pass the course. Also, Highest grade (A) will be allotted to students scoring 85+. The grading will be relative-scaled.

Exam/Test Schedule: 

1. First Open-book exam will be on February 28, 2025, from 02:00 pm to 04:00 pm.

2. Presentation on self-learning topics 1-6 will be held on February 08, 2025. 

3. Presentation on self-learning topics 7-9 will be held on February 15, 2025.

4. Abstract (word limit 150) on real-time application topic to be submitted by February 23, 2025.

5. Presentation on self-learning topics 10-14 will be held on March 29, 2025.

6. Presentation on self-learning topics 15-19 will be held on April 05, 2025.

7. Report (max 4 pages) on real-time application topic to be submitted by April 13, 2025.

8. Presentation on self-learning topics 20-21 will be held on April 21, 2025.

9. Second Open-book exam will be on April 29, 2025, from 02:30 pm to 04:30 pm.