Aims

• To introduce the basic concept and applications of Pattern Recognition.
• To introduce the concept of statistical pattern recognition.
• To develop an approach to the modelling of pattern recognition systems.

Objectives

By the end of the course students should be able to:

• Understand the key concepts of modern pattern recognition theory  and their application in pattern recognition systems.
• Understand the  different type of pattern recognition methods and to compare them.

Content

• Machine Perception

• Pattern Recognition Systems 

• Sensing 

• Learning and Adaptation 

• Supervised Learning 

• Unsupervised Learning 

• Reinforcement Learning
  

• Bayesian Decision Theory

  •  Minimum-Error-Rate Classification 

  • Classifiers, Discriminants, and Decision Surfaces

•  Maximum Likelihood and Bayesian Estimation

•  Bayesian Estimation
• Bayesian Parameter Estimation: Gaussian Case
• Bayesian Parameter Estimation: General Theory 
• Sufficient Statistics and the Exponential Family 
• Problems of Dimensionality
• Component Analysis and Discriminants 
• Principal Component Analysis (PCA)
• Fisher Linear Discriminant 
• Multiple Discriminant Analysis
  

•  Nonparametric Techniques

•  Density Estimation
• Parzen Windows
• Probabilistic Neural Networks (PNNs)
• k_n--Nearest-Neighbor Estimation 
• The Nearest-Neighbor Rule 

•  Linear Discriminant Functions

• Linear Discriminant Functions and Decision Surfaces 
• Generalized Linear Discriminant Functions 
• Minimizing the Perceptron Criterion Function
• Relaxation Procedures
• Nonseparable Behavior 
• Minimum Squared-Error Procedures 
• Minimum Squared-Error and the Pseudoinverse 
• Relation to Fisher's Linear Discriminant 
• The Widrow-Hoff Procedure 
• Stochastic Approximation Methods 
• The Ho-Kashyap Procedures 
• The Descent Procedure 
• Linear Programming Algorithms 
• Support Vector Machines (SVM)
  
  

• Nonmetric Methods

• Decision Trees 
• CART 
• Recognition with Strings 
• Grammatical Methods
  
  

• Algorithm-Independent Machine Learning

• Lack of Inherent Superiority of Any Classifier 
• Bias and Variance 
• Resampling for Estimating Statistics 
• Resampling for classifier design 
• Estimating and Comparing Classifiers 
• Combining Classifiers
  
  

•  Unsupervised Learning and Clustering

• Mixture Densities and Identifiability 
• Maximum-Likelihood Estimates 
• Application to Normal Mixtures 
• Data Description and Clustering 
• Criterion Functions for Clustering 
• Iterative Optimization 
• Hierarchical Clustering 
• The problem of validity
• On-line clustering
• Graph-Theoretic Methods 
• Principal Component Analysis (PCA) 
• Nonlinear Component Analysis (NLCA) 
• Independent Component Analysis (ICA) 
• Dimensional Representations and Multidimensional Scaling (MDS) 
• Self-Organizing Feature Maps 
• Clustering and Dimensionality Reduction
  
  

Official Syllabus

 Not available.

Core Text

•  Richard O. Duda, Peter E. Hart, David G. Stork, Pattern Classification, Second Edition,ISBN: 0-471-05669-3, 680 pages, November 2000, £79.50 / €112.50

 

Secondary Texts

• K. Fukunaga, Statistical Pattern Recognition. Academic Press, 1990. 2nd Edition.

Assessment

•  Final examination  
•  Half Term examination 
•  Coursework
• Projects 

Exercises 

• Exercise1
• Exercise 2
• Exercise 3
• Exercise 4 
• Exercise 5