teaching‎ > ‎

Profilierungsmodul II

Profilierungsmodul II: Deep Learning for Natural Language Processing

Zeit: Mi. 9-11 (VL), Mi. 11-12 (Ü)
Raum: 131
Start: 19.10.2016


Aufteilung für die Bewertung:

  • Übungsblätter 40%
  • Implementierung 20%
  • Schriftliche Ausarbeitung 20% (Zu Abschnitten des Reports Namen des jeweiligen Autors angeben.)
  • Präsentation 20%
Abgabetermine:
  • Implementierungen: 31.1.2017
  • Schriftliche Ausarbeitungen: 21.2.2017

The topics of this lecture are the foundations of deep learning, with a particular focus on practical aspects and applications to natural language processing and knowledge representation.


Topics:
  • Foundations of machine learning
    • Loss functions
    • linear regression
    • logistic regression
    • gradient-based optimization
    • neural networks and backpropagation
  • Deep learning tools
    • Numpy
    • Theano
    • Keras
    • Tensorflow
  • Applications
    • Relation extraction
    • Knowledge graph embeddings
    • Practical projects (NLP related, to be agreed on during the course)

Presentations:


  • Relation Pattern Correspondences with CNNs

    • Daria Palchik, Elena Neuburg, Marina Sedinkina

    • Presenation: 1.2.

  • Relation Pattern Correspondences with LSTMS; Relation Classification with Recursive CNNs

    • Dan Lüdtke, Diego Martin Arroyo, Karl-Heinz Krachenfels

    • Presentation: 1.2.

  • Word2Vec Comparison: Binary Hierarchical Softmax

    • Nicolai Ruhnau, Stefan Schweter

    • Presentation: 1.2.

  • Word2Vec Comparison: Two-Level Hierarchical Softmax

    • Eduard Saller, Kevin Falkner

    • Presentation: 8.2

  • Relation Prediction without Tagged Candidates

    • Costanza Conforti, Nina Pörner, Raphael Höps

    • Presentation: 8.2.

  • Irony and Sarcasm Prediction with LSTM and CNN

    • Isabell Krammer, Jennifer Ling, Fedor Uvarov

    • Presentation: 8.2.
  • Language Modeling for Armenian

    • Christian Meyer, Tsovinar Sekhleyan

    • Presentation: 8.2


Course Material:

 date slides homework materials
 Oct. 19, 2016
 Machine learning overview I (pdf)
 (pdf) 
 Oct. 26, 2016
 Machine learning overview IIa
 Introduction to Numpy Ia
 (pdf) 
 Nov. 2, 2016
 Machine learning overview II (pdf)
 Introduction to Numpy I (pdf)
 (pdf)
 (gitlab)
 (ipynb)
 
 Nov. 9, 2016
 Machine learning overview III (pdf)
 Introduction to Theano Ia
 (ipynb) 
 Nov. 15, 2016
 Introduction to Theano I / CNNs (pdf)
 (pdf)MNIST example code (github)
 Nov. 23, 2016
 Introduction to Theano II / RNNs (pdf) (ipynb)
 (data)
 
 Nov. 30, 2016
 Introduction to Keras / Hyper-parameters (pdf)
  Hyper-parameter optimization example script:
 hyperopt.py
 Dec. 7, 2016
 Knowledge Base Population (pdf)
  
 Dec. 14, 2016
 Word2Vec (pdf)
 Proposed project topics (pdf)
  Mikolov et al.: Distributed representations of words and phrases and their compositionality. (pdf)
 Mikolov et al.: Efficient Estimation of Word Representations in Vector Space. (pdf)
 Goldberg & Levy: Word2Vec Explained. (pdf)
 Dec. 21, 2016
 Project Topics Q&A
  Intro to Tensorflow (Stanford, pdf)
 Jan. 11, 2017
 Tensorflow  
 Jan. 18, 2017 Neural Attention
  
 Jan. 25, 2017 Neural Machine Translation
  
 Feb. 1, 2017 Project Presentations
  
 Feb. 8, 2017 Project Presentations
  

More materials:
Bengio: Practical recommendations ... (arXiv)
Socher: Neural Tips and Tricks (pdf)
ċ
atis.json.tar.gz
(123k)
Benjamin Roth,
Nov 24, 2016, 4:13 AM
Ċ
Benjamin Roth,
Dec 7, 2016, 12:21 AM
Ċ
Benjamin Roth,
Oct 18, 2016, 11:26 PM
Ċ
Benjamin Roth,
Oct 25, 2016, 12:08 PM
Ċ
Benjamin Roth,
Nov 15, 2016, 3:35 PM
ċ
hyperopt.py
(4k)
Benjamin Roth,
Nov 30, 2016, 5:04 AM
Ċ
Benjamin Roth,
Nov 30, 2016, 4:17 AM
Ċ
Benjamin Roth,
Oct 18, 2016, 11:27 PM
Ċ
Benjamin Roth,
Oct 25, 2016, 12:43 PM
Ċ
Benjamin Roth,
Nov 9, 2016, 12:17 AM
Ċ
Benjamin Roth,
Oct 26, 2016, 6:20 AM
ċ
numpy.ipynb
(103k)
Benjamin Roth,
Nov 2, 2016, 4:04 AM
Ċ
numpy.pdf
(19k)
Benjamin Roth,
Nov 2, 2016, 4:04 AM
Ċ
Benjamin Roth,
Oct 25, 2016, 12:35 PM
Ċ
Benjamin Roth,
Oct 26, 2016, 6:23 AM
Ċ
Benjamin Roth,
Dec 14, 2016, 12:22 AM
ċ
theano_I.ipynb
(11k)
Benjamin Roth,
Nov 9, 2016, 3:30 AM
ċ
theano_II.ipynb
(19k)
Benjamin Roth,
Nov 24, 2016, 4:13 AM
Ċ
Benjamin Roth,
Nov 15, 2016, 10:53 AM
Ċ
Benjamin Roth,
Nov 15, 2016, 12:36 PM
Ċ
Benjamin Roth,
Dec 14, 2016, 3:55 AM
Comments