| Instructor: | Tatjana Scheffler |
| TA: | Edit Szügyi |
| Time: | Tuesdays & Fridays, 10 a.m.-12 |
| Place: | Golm, building 14, room 009 |
| Modules: | CSBM1 (M.Sc. Cognitive Systems) |
| Moodle: | Please register on the course's Moodle site. |
To be admitted to the module exam, you need to pass the course. For this, we will grade the best two assignments out of each half of the semester (i.e., the best 2 from the first 3 + the best 2 from the second 3). At least 250 points in total (out of 400) in these 4 assignments are needed to pass the course.
The grade will be based on a collaborative final project, to be completed during the semester break. There are four graded deliverables for this project:
The grade will be composed equally from these four parts. Details will be discussed in class.
This class is the graduate-level introduction to computational linguistics, a first-year class in the MSc Cognitive Systems. The purpose of this class is to introduce the important concepts, models and methods used in natural language processing (NLP). After the successful completion of this course, students should be able to (i) read and understand the scientific literature in the area of computational linguistics and (ii) start implementing their own NLP projects.
We will cover the following topics:
Readings in J/M are marked for the third edition unless marked otherwise.
| Date | Topic | Readings | Assignments |
|---|---|---|---|
| T 10/16 | Introduction | ||
| F 10/19 | Review: Probability Theory | Harald Goldstein, A short introduction to probability and related concepts And/Or: Manning/Schütze, chapter 2.1 Optional: Kevin Murphy, Binomial and multinomial distributions |
|
| T 10/23 | N-grams | Jurafsky/Martin, chapters 3.1-3.2 A video about Zipf's law |
A1 released |
| F 10/26 | Smoothing language models | Jurafsky/Martin, chapters 3.4-3.7 Opt.: Manning/Schütze, chapters 6.2-6.3 Chen/Goodman, 1998 |
|
| T 30/10 | Classification | Jurafsky/Martin, chapter 4 | |
| F 11/2 | Classification (2) | Jurafsky/Martin, chapter 5 | A1 due, A2 released |
| T 11/6 | Part of speech tagging | Jurafsky/Martin (2nd ed.!), chapters 5.1-5.5, 6.1-6.5; J/M (3rd ed.) ch. 8.1-8.4 | |
| F 11/9 | discussion of A1 (Edit) | be prepared to present your solutions! | |
| T 11/13 | HMM Training | Jurafsky/Martin, rest of chapter 8 (-8.4) + Appendix A HMM spreadsheet, Eisner's ice cream example Eisner's paper explaining how to work with the spreadsheet |
|
| F 11/16 | Context free grammars, CKY parsing | Jurafsky/Martin, chapter 11 (-11.2), J/M chapter 10 as background further reading: Santorini/Kroch, Online syntax textbook CKY animation |
A2 due, A3 released |
| T 11/20 | PCFGs | Jurafsky/Martin, chapters 12.1-12.5, 12.7 | |
| F 11/23 | discussion of A2 | be prepared to present your solutions! | |
| T 11/27 | dies academicus (no class) | ||
| F 11/30 | Training PCFGs | Jurafsky/Martin, chapter 12 Manning/Schütze, chapter 11 Michael Collins, The inside-outside algorithm. |
A3 due, A4 released |
| T 12/4 | Advanced PCFG models | Mark Johnson (1998), PCFG
Models of Linguistic Tree Representations (esp. on parent
annotations) Michael Collins, Lexicalized PCFGs Dan Klein/Chris Manning (2003), Accurate unlexicalized parsing |
|
| F 12/7 | discussion of A3 | be prepared to present your solutions! | |
| T 12/11 | Dependency parsing | McDonald/Pereira/Ribarov/Hajic (2005),
Non-projective
Dependency Parsing using Spanning Tree Algorithms Joakim Nivre (2008), Algorithms for Deterministic Incremental Dependency Parsing |
|
| F 12/14 | Statistical machine translation: Alignments | Jurafsky/Martin (2nd ed.), ch. 25 (through 25.6) Adam Lopez, Word Alignment and the Expectation-Maximization Algorithm tutorial (try here) http://mt-class.org/ MT Talks |
A4 due |
| T 12/18 | Phrase-based machine translation | Jurafsky/Martin (2nd ed.), ch. 25 | |
| T 12/18, 12:30pm | discussion of A3 | be prepared to present your solutions! | |
| 12/24-30 | no class (winter break) | ||
| 12/31-1/6 | no class (winter break) | ||
| T 1/8 | Syntax-based machine translation | David Chiang, Hierarchical phrase-based translation. Computational Linguistics, 2007. | A5 distributed |
| F 1/11 | Semantic parsing | Zettlemoyer/Collins, Learning to Map Sentences to
Logical Form. Structured Classification with Probabilistic
Categorial Grammars. 2005 Wong/Mooney, Learning for Semantic Parsing with Statistical Machine Translation. HLT-NAACL, 2006 Opt.: Mark Steedman, A very short introduction to CCG. 1996 |
|
| T 1/15 | Lexical semantics | Jurafsky/Martin, ch. 6 (+ Appendix C) Further reading: Mitchell/Lapata (2008), Vector-based models of semantic composition; Baroni/Zamparelli (2010), Nouns are vectors, adjectives are matrices |
|
| F 1/18 | Speech recognition + synthesis | Jurafsky/Martin (2nd ed.), ch. 8,9 Links for further reading/system samples: "Human parity" speech recognition Language log on human parity speech recognition Google speech API web demo S. Germesin on using the speech API remotely MARY TTS (DFKI) WaveNet: Generating raw audio Deep Learning in Speech Synthesis Expressive and emotional synthetic speech |
A5 due, A6 released |
| T 1/22 | LDA | David Blei, Probabilistic topic models. 2012 Links for further reading: Steyvers/Griffiths, Probabilistic Topic Models. 2007. Darling, A Theoretical and Practical Implementation Tutorial on Topic Modeling and Gibbs Sampling. Online Topic Modelling Tutorial and Demo for LDA |
|
| F 1/25 | discussion of A5 | be prepared to present your solutions! | |
| T 1/29 | Intro to deep neural networks | Richard Socher, Recursive Deep Learning for Natural Language
Processing and Computer Vision (Chapter 2). 2014 Further reading (for intuitions): http://karpathy.github.io/neuralnets/, http://neuralnetworksanddeeplearning.com/chap4.html |
|
| F 2/1 | presentations of final projects, final discussion | all | A6 due |
| 2/4-10 | no class |
Most computational linguists own both of these books. We will assign weekly readings, so you should ensure you get your own copy or have access to the copies that are available in the university library.