Profilierungsmodul II: Deep Learning for Natural Language Processing Zeit: Mi. 9-11 (VL), Mi. 11-12 (Ü)
Raum: 131
Start: 19.10.2016
Aufteilung für die Bewertung: - Übungsblätter 40%
- Implementierung 20%
- Schriftliche Ausarbeitung 20% (Zu Abschnitten des Reports Namen des jeweiligen Autors angeben.)
- Präsentation 20%
Abgabetermine:
- Implementierungen: 31.1.2017
- Schriftliche Ausarbeitungen: 21.2.2017
The
topics of this lecture are the foundations of deep learning, with a
particular focus on practical aspects and applications to natural
language processing and knowledge representation.
Topics:
- Foundations of machine learning
- Loss functions
- linear regression
- logistic regression
- gradient-based optimization
- neural networks and backpropagation
- Deep learning tools
- Numpy
- Theano
- Keras
- Tensorflow
- Applications
- Relation extraction
- Knowledge graph embeddings
- Practical projects (NLP related, to be agreed on during the course)
Presentations: Relation Pattern Correspondences with CNNs Relation Pattern Correspondences with LSTMS; Relation Classification with Recursive CNNs Word2Vec Comparison: Binary Hierarchical Softmax Word2Vec Comparison: Two-Level Hierarchical Softmax Relation Prediction without Tagged Candidates
Irony and Sarcasm Prediction with LSTM and CNN Language Modeling for Armenian
Course Material: date | slides | homework | materials | Oct. 19, 2016
| Machine learning overview I (pdf)
| (pdf) | | Oct. 26, 2016
| Machine learning overview IIa Introduction to Numpy Ia
| (pdf) | | Nov. 2, 2016
| Machine learning overview II (pdf) Introduction to Numpy I (pdf)
| (pdf) (gitlab) (ipynb)
| | Nov. 9, 2016
| Machine learning overview III (pdf) Introduction to Theano Ia
| (ipynb) | | Nov. 15, 2016
| Introduction to Theano I / CNNs (pdf)
| (pdf) | MNIST example code (github)
| Nov. 23, 2016
| Introduction to Theano II / RNNs (pdf) | (ipynb) (data)
| | Nov. 30, 2016
| Introduction to Keras / Hyper-parameters (pdf)
| | Hyper-parameter optimization example script: hyperopt.py | Dec. 7, 2016
| Knowledge Base Population (pdf)
| | | Dec. 14, 2016
| Word2Vec (pdf) Proposed project topics (pdf)
| | Mikolov et al.: Distributed representations of words and phrases and their compositionality. (pdf) Mikolov et al.: Efficient Estimation of Word Representations in Vector Space. (pdf) Goldberg & Levy: Word2Vec Explained. (pdf)
| Dec. 21, 2016
| Project Topics Q&A
| | Intro to Tensorflow (Stanford, pdf)
| Jan. 11, 2017
| Tensorflow | | | Jan. 18, 2017 | Neural Attention
| | | Jan. 25, 2017 | Neural Machine Translation
| | | Feb. 1, 2017 | Project Presentations
| | | Feb. 8, 2017 | Project Presentations
| | |
More materials:
Bengio: Practical recommendations ... ( arXiv) Socher: Neural Tips and Tricks ( pdf) |
 Updating...
Benjamin Roth, Nov 24, 2016, 4:13 AM
Ċ Benjamin Roth, Dec 7, 2016, 12:21 AM
Ċ Benjamin Roth, Oct 18, 2016, 11:26 PM
Ċ Benjamin Roth, Oct 25, 2016, 12:08 PM
Ċ Benjamin Roth, Nov 15, 2016, 3:35 PM
Benjamin Roth, Nov 30, 2016, 5:04 AM
Ċ Benjamin Roth, Nov 30, 2016, 4:17 AM
Ċ Benjamin Roth, Oct 18, 2016, 11:27 PM
Ċ Benjamin Roth, Oct 25, 2016, 12:43 PM
Ċ Benjamin Roth, Nov 9, 2016, 12:17 AM
Ċ Benjamin Roth, Oct 26, 2016, 6:20 AM
Benjamin Roth, Nov 2, 2016, 4:04 AM
Ċ Benjamin Roth, Nov 2, 2016, 4:04 AM
Ċ Benjamin Roth, Oct 25, 2016, 12:35 PM
Ċ Benjamin Roth, Oct 26, 2016, 6:23 AM
Ċ Benjamin Roth, Dec 14, 2016, 12:22 AM
Benjamin Roth, Nov 9, 2016, 3:30 AM
Benjamin Roth, Nov 24, 2016, 4:13 AM
Ċ Benjamin Roth, Nov 15, 2016, 10:53 AM
Ċ Benjamin Roth, Nov 15, 2016, 12:36 PM
Ċ Benjamin Roth, Dec 14, 2016, 3:55 AM
|