Resources

2.1 - Architecture and Functioning

Proposed Readings:

• Alammar, J. (n.d.). A Gentle Introduction to Transformers. Retrieved from http://jalammar.github.io/illustrated-transformer/

Proposed Media:

• ColdFusion. (2019). Transformers - The AI that Changed Everything. YouTube. https://www.youtube.com/watch?v=MX-3O9wCNjo

2.2 - Techniques Used in Language Models

Proposed Readings:

• Alammar, J. (n.d.). The Illustrated Transformer. Retrieved from http://nlp.seas.harvard.edu/2018/04/03/attention.html

Proposed Media:

• Welch Labs. (2014). Neural Networks Demystified [Video]. YouTube. https://www.youtube.com/watch?v=bxe2T-V8XRs

2.3 - Prompt Based Generation

Proposed Readings:

• Jurafsky, D., & Martin, J. H. (2019). Speech and Language Processing (3rd ed.). Pearson. (Chapter 24: Neural Network Language Models)

• OpenAI. (2021). How ChatGPT Works - The Magic of Language Models. Retrieved from https://openai.com/blog/how-chatgpt-works/

Proposed Media:

• CodeEmporium. (2021). How Language Models Work - Explained! [Video]. YouTube. https://www.youtube.com/watch?v=rURRYi2VPiU

2.4 - Pre-Training and Fine-Tuning for Performance

Proposed Readings:

• Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press. (Chapter 15: Representation Learning)

• Hugging Face. (n.d.). Illustrated Guide to Pretrained Language Models. Retrieved from https://huggingface.co/blog/how-to-train

Proposed Media:

• Deeplizard. (2019). Transfer Learning and Pretrained Models [Video]. YouTube. https://www.youtube.com/watch?v=yofjFQddwHE