Course Title: Topics in Artificial Intelligence: Reading the Classics


Credit Hours: 2 (Reading course, 2-3 contact hours/week) In-person @NIAS Discussion Room (1st Floor)



Course Description:

Artificial Intelligence (AI) has captured the imagination of the public to such alarming proportions that a holistic understanding of the underlying mathematical and philosophical principles is often neglected. For the serious researcher, it is therefore indispensable to go back to reading, assimilating, questioning, debating and critically engaging with the classics - original papers written in the field of AI (starting from Alan Turing’s famous 1950 paper) to modern developments, especially the recent advances in neuroscience inspired AI. This course will attempt the impossible task - of engaging with the breadth and depth of AI - giving equal importance to all the different facets of AI - historical, mathematical, philosophical, technological, cultural and societal aspects.



Learning Objectives:

The primary objective is to familiarize the student with key ideas, principles and methods in Artificial Intelligence, its origins, intersections with neuroscience, ongoing debates, and some of the critical issues with current AI research trends. As much as possible, original papers/articles/essays/ book chapters/excerpts etc. will be used as reading material in this entire course.



Pre-requisites for registration:

Comfort with mathematical and philosophical thinking, and scientific rigor is expected. Interest in reading original papers in AI (sometimes reading outside the comfort zone). MATLAB/Python or any equivalent programming language would be required for some of the assignments. Auditing is not allowed for this course.



Expected Student Workload:

This is not a lecture style course but purely a reading course with 2-2½ contact hours/week (discussion on the papers-for-the-week and in-class interactions) with remaining ~3 hours/week of preparation, reading papers, books and term paper preparation. The student is expected to read the research papers/material for the week ahead of the class and the nature of the class will be more in the style of interaction (no lectures!). For every topic, the student is expected to submit a short essay (and/or a computer program). The student will be required to work towards a term paper (research based) - and submit a report at the end of the course and a final in-class presentation.


Course Duration:

August-December 2022.


Topics for Discussion

The origin and history of AI, Turing Test, John Searle’s Chinese room argument, Perceptron, XOR problem, Multi-Layer Perceptron, Backpropagation, Universal Approximation Theorem, No Free Lunch Theorem, Maximum Margin Classifier/Support Vector Machines, Convolutional Neural Networks, Long Short Term Memory; Generative Adversarial Nets; Language models, word2vec, language understanding by generative pre-training; Reinforcement learning; Neuroscience inspired AI: MothNet, ChaosNet/Neurochaos Learning; Causality and AI, Singularity, AI Ethics, Dark side of AI, AI & Society, AI, Brain and Consciousness debate.


Basis for Final Grades

In class participation/discussions/presentations: 50%

Term paper final report: 30%

Term paper final presentation (in-class): 20%




Reading Material for the Sessions

Session 1:

  • Turing, A. M. (1950). Computing Machinery and Intelligence. Mind, Vol. 59 (236): 433–460.

  • Searle, J. R. (1980). Minds, Brains and Programs, Behavioral and Brain Sciences, 3: 417–57.



Session 2:

  • Kathpalia, A., & Nagaraj, N. (2021). Measuring causality. Resonance, 26(2), 191-210.

  • Nagaraj, N. (2022). Can Machines Think Causally? Unpublished (under review).

  • Pearl, J. and Mackenzie, D. (2018). The book of why: the new science of cause and effect. Basic books.



Session 3:

  • McCulloch, W.S. and Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5(4), pp.115-133.

  • Kleene, S. C. (1956). Representation of events in nerve nets and finite automata. Automata studies, 34, 3-41.

  • URL: https://towardsdatascience.com/mcculloch-pitts-model-5fdf65ac5dd1




Session 4 & 5:

  • Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6), p.386.

  • Boser, B.E., Guyon, I.M. and Vapnik, V.N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on Computational learning theory (pp. 144-152).

  • Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297.

  • URLs

· https://news.cornell.edu/stories/2019/09/professors-perceptron-paved-way-ai-60-years-too-soon

· https://www.cs.cornell.edu/courses/cs4780/2018fa/lectures/lecturenote03.html

· https://www.cs.cornell.edu/courses/cs4780/2018fa/lectures/lecturenote09.html



Session 6:

  • Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., ... & Steinberg, D. (2008). Top 10 algorithms in data mining. Knowledge and information systems, 14(1), 1-37.

  • URL: k-means clustering (Basira labs)

https://www.youtube.com/watch?v=H_Xa80PiIJQ

  • URL: kNN (Kilian Weinberger)

https://www.youtube.com/watch?v=oymtGlGdT-k



Session 7:

  • Quinlan, J. R. (1986). Induction of decision trees. Machine learning, 1(1), 81-106.

  • Ho, T. K. (1995, August). Random decision forests. In Proceedings of 3rd international conference on document analysis and recognition (Vol. 1, pp. 278-282). IEEE.

  • URL: Random Forest Algorithm Clearly Explained!

https://www.youtube.com/watch?v=v6VJ2RO66Ag

  • URL: Decision Tree Classification Clearly Explained!

https://www.youtube.com/watch?v=ZVR2Way4nwQ


Session 8:

  • Shlens, J. (2014). A tutorial on principal component analysis. arXiv preprint arXiv:1404.1100.

  • Strang, G. (2006). Linear algebra and its applications. Belmont, CA: Thomson, Brooks/Cole.

  • Smith, L. I. (2002). A tutorial on Principal Components Analysis (Computer Science Technical Report No. OUCS-2002-12). Retrieved from http://hdl.handle.net/10523/7534



Session 9:

  • Holland, J. H. (1992). Genetic algorithms. Scientific american, 267(1), 66-73.

  • Whitley, D. (1994). A genetic algorithm tutorial. Statistics and computing, 4(2), 65-85.

  • Golberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning. Addion wesley, 1989(102), 36.

      • URLs:

· A very good example of "Vanilla Genetic Algorithm":

https://www.youtube.com/watch?v=MacVqujSXWE

· A more deeper technical video is by Prof. Winston (MIT OCW lectures)

https://www.youtube.com/watch?v=kHyNqSnzP8Y

· Machine Intelligence - Lecture 18 (Evolutionary Algorithms)

https://www.youtube.com/watch?v=3-NiZPbkr7A

· A.I. learns to play | Neural Network + Genetic Algorithm

https://www.youtube.com/watch?v=rxzioiG-Vnk

· NPTEL lectures - by Prof. Balaji of IITM, he works out an example in detail:

https://www.youtube.com/watch?v=Z_8MpZeMdD4&list=LL&index=4

https://www.youtube.com/watch?v=uI5viW4r5ic&list=LL&index=2



Session 10:



Session 11:

  • Rumelhart, D.E., Hinton, G.E. and Williams, R.J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), pp.533-536.


  • URLs:

· Caltech's Machine Learning Course - CS 156 by Professor Yaser Abu-Mostafa

https://www.youtube.com/watch?v=Ih5Mr93E-2c&list=PLMBPC5xOll7T0Tv6ZC33XVUkamluL9swr&index=8

· https://www.youtube.com/watch?v=8d6jf7s6_Qs

· https://www.youtube.com/watch?v=Ilg3gGewQ5U&list=PLMBPC5xOll7T0Tv6ZC33XVUkamluL9swr&index=3



Session 12:

  • LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4), 541-551.

  • Stabinger, S., Rodríguez-Sánchez, A., & Piater, J. (2016, September). 25 years of cnns: Can we compare to human abstraction capabilities? In International conference on artificial neural networks (pp. 380-387). Springer, Cham.

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90.

  • URL: MIT 6.S191: Convolutional Neural Networks by Alexander Amini



Others:

  • Tegmark, M. (2017). Life 3.0: Being human in the age of artificial intelligence. Vintage.

  • Mitchell, M. (2019). Artificial intelligence: A guide for thinking humans. Penguin UK.




Could not discuss the following, but would have desired to:

  • Cybenko, G., (1989). Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems, 2(4), pp.303-314.

  • Minsky, M. and Papert, S.A., (2017). Perceptrons. Reissue of the 1988 Expanded Edition with a new foreword by Léon Bottou: An Introduction to Computational Geometry. MIT press.

  • Wolpert, D.H., (1996). The lack of a priori distinctions between learning algorithms. Neural computation, 8(7), pp.1341-1390.

  • Flagel, L., Brandvain, Y. and Schrider, D.R. (2019). The unreasonable effectiveness of convolutional neural networks in population genetic inference. Molecular biology and evolution, 36(2), pp.220-238.

  • Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), pp.1735-1780.

  • Subramanian, A., Chitlangia, S. and Baths, V. (2022). Reinforcement learning and its connections with neuroscience and psychology. Neural Networks, 145, pp.271-287.

  • Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.

  • Radford, A., Narasimhan, K., Salimans, T. and Sutskever, I. (2018). Improving language understanding by generative pre-training. URL: https://openai.com/blog/language-unsupervised/

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.

  • Hassabis, D., Kumaran, D., Summerfield, C. and Botvinick, M. (2017). Neuroscience-inspired artificial intelligence. Neuron, 95(2), pp.245-258.

  • Delahunt, C.B. and Kutz, J.N. (2019). Putting a bug in ML: The moth olfactory network learns to read MNIST. Neural Networks, 118, pp.54-64.

  • Balakrishnan, H.N., Kathpalia, A., Saha, S. and Nagaraj, N. (2019). ChaosNet: A chaos based artificial neural network architecture for classification. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(11), p.113125.

  • Balakrishnan, H.N., and Nagaraj, N. (2021). When noise meets chaos: Stochastic resonance in neurochaos learning. Neural Networks, 143, pp.425-435.

  • Balakrishnan, H.N., and Aditi Kathpalia and Nithin Nagaraj. (2022). Causality Preserving Chaotic Transformation and Classification using Neurochaos Learning. In Proceedings of Advances in Neural Information Processing Systems, NeurIPS 2022 (Eds.: Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho).