Class Timing:
Self-study Timing:
Quiz:
Assignment Due Date:
Project Due Date:
Assessment
Course Objectives
Learn the theories and techniques behind Web search engines.
Get hands on project experience by developing real-world applications, such as intelligent tools for improving search accuracy from user feedback, email spam detection, or scientific literature organization and mining.
Learn tools and techniques to do cutting-edge research in the area of information retrieval or text mining.
Open the door to the amazing job opportunities in Search Technology and E-commerce companies such as Google, Microsoft, Yahoo! and Amazon.
Learning Outcomes
LO1: Learn to write code for text indexing and retrieval
LO2: Learn to evaluate information retrieval systems
LO3: Learn to analyze textual and semi-structured data sets
LO4: Learn about text similarity measure
LO5: Understanding about search engine
Module 1: Overview of text retrieval systems
The nature of unstructured and semi-structured text. Inverted index and Boolean queries.
Text encoding: tokenization, stemming, stop words, phrases, index optimization, Index compression: lexicon compression and postings lists compression, Gap encoding, gamma codes, Zipf’s Law, Index construction, Postings size estimation, dynamic indexing, positional indexes, n-gram indexes, real-world issues.
Module 2: Retrieval models, implementation, and evaluation
Boolean, vector space, TF-IDF, Okapi, probabilistic, language modeling, latent semantic indexing, Vector space scoring, The cosine measure, Efficiency considerations, Document length normalization, Relevance feedback and query expansion, Rocchio.
Evaluating search engines, User happiness, precision, recall, F-measure. Creating test collections: kappa measure, interjudge agreement.
Module 3: Text classification and clustering
Introduction to text classification, Naive Bayes models, Spam filtering, Vector space classification using hyperplanes, centroids, k Nearest Neighbors, Support vector machine classifiers, Kernel functions, Boosting.
Clustering versus classification, Partitioning methods, k-means clustering, Mixture of gaussian model, Hierarchical agglomerative clustering, Clustering terms using documents.
Module 4: Web Information Retrieval
Hypertext, web crawling, search engines, ranking, link analysis, PageRank, HITS.
XML retrieval, semantic web.
Module 5: IR applications and Advanced Topics
Summarization, Topic detection and tracking, Personalization, Question answering, Cross language informtion retrieval.
Information extraction, Question answering, Opinion summarization, Social Network. Recommender Systems.
Lecture 1: Overview; Foundations of Information Retrieval; Source: CS54701 Information Retrieval by Clifton
Lecture 2: Indexing and Querying; Boolean Retrieval; Source: Chapter 1 & 2, Manning, Raghavan and Schütze
Lecture 3: Text Encoding; Source: Chapter 2, Manning, Raghavan and Schütze
Lecture 4: Dictionaries and Tolerant Retrieval; Source: Chapter 3, Manning, Raghavan and Schütze
Lecture 5: Index Construction; Source: Chapter 4, Manning, Raghavan and Schütze
Lecture 6: Index Compression; Source: Chapter 5, Manning, Raghavan and Schütze
Lecture 7: Scoring, Term weighting; Vector Space Model; Source: Chapter 6, Manning, Raghavan and Schütze
Lecture 8: Computing Scores; Source: Chapter 7, Manning, Raghavan and Schütze
Lecture 9: Evaluation in Information Retrieval; Source: Chapter 8, Manning, Raghavan and Schütze
Lecture 10: Relevance Feedback and Query Expansion; Source: Chapter 9, Manning, Raghavan and Schütze
Lecture 11: XML Retrieval; Source: Chapter 10, Manning, Raghavan and Schütze
Lecture 12: Probabilistic Information Retrieval; Source: Chapter 11, Manning, Raghavan and Schütze
Lecture 13: Language Models; Source: Chapter 12, Manning, Raghavan and Schütze
Lecture 14: Text Classification; Source: Chapter 13, Manning, Raghavan and Schütze
Lecture 15: Distributed Representations; Source: Chapter 14, Manning, Raghavan and Schütze
Lecture 16: Learning Ranking; Source: Chapter 15, Manning, Raghavan and Schütze
Lecture 17: Link Analysis; Source: Chapter ?, Manning, Raghavan and Schütze
Lecture 18: Crawling; Source: Chapter ?, Manning, Raghavan and Schütze
Lecture 19: Web QA; Source: Chapter ?, Manning, Raghavan and Schütze
Lecture 20: Personalization; Source: Chapter ?, Manning, Raghavan and Schütze
Self-Study 1: Web Crawling
Self-Study 2: Indexing and Searching using pySolr
Self-Study 3: Boolean Retrieval using Python
Self-Study 4: Vector Space Retrieval using Python
Self-Study 5: Probabilistic Retrieval using Python
Self-Study 6: Classification in Retrieval using Python
Introduction to Information Retrieval, by C. Manning, P. Raghavan, and H. Schütze (Cambridge University Press, 2008).
Search Engines: Information Retrieval in Practice. Croft, W. Bruce; Metzler, Donald; Strohman, Trevor. Addison Wesley (2008)
Information Retrieval: Implementing and Evaluating Search Engines, Stefan Buettcher, Charles L. A. Clarke, Gordon V. Cormack. MIT Press. (2010)
Modern Information Retrieval, Ricardo Baeza-Yates and Berthier Ribeiro-Neto, Addison-Wesley, (1999)
Quiz 24%
Assignments 20%
Project 40% (Implementation 10%, Knowledge 10%, Analysis 20%)
Attendance 16%