| Instructor: | Tatjana Scheffler |
| TA: | Jens Johannsmeier |
| Time: | Tuesdays, 12-2 p.m. Fridays, 10 a.m.-12 |
| Place: | Golm, building 14, room 009 |
| Modules: | CSBM1 (M.Sc. Cognitive Systems) |
| Moodle: | Please register on the course's Moodle site. |
The grade will be based on a final project, to be completed during the semester break. The grade takes the following criteria into account: proposal and its presentation, difficulty/creativity, correctness/performance, code readability/documentation.
This class is the graduate-level introduction to computational linguistics, a first-year class in the MSc Cognitive Systems. The purpose of this class is to introduce the important concepts, models and methods used in natural language processing (NLP). After the successful completion of this course, students should be able to (i) read and understand the scientific literature in the area of computational linguistics and (ii) start implementing their own NLP projects.
We will cover the following topics:
| Date | Topic | Readings | Assignments |
|---|---|---|---|
| T 10/18 | Introduction | ||
| F 10/21 | Review: Probability Theory | Harald Goldstein, A short introduction to probability and related concepts And/Or: Manning/Schütze, chapter 2.1 Optional: Kevin Murphy, Binomial and multinomial distributions |
A1 released |
| T 10/25 | practical concerns (TA) | notes for assignment | |
| F 10/28 | N-grams | Jurafsky/Martin, chapters 4.1-4.2 A video about Zipf's law |
|
| T 11/1 | Smoothing | Jurafsky/Martin, chapters 4.5-4.7 Opt.: Manning/Schütze, chapters 6.2-6.3 |
|
| F 11/4 | Part of Speech Tagging | Jurafsky/Martin, chapters 5.1-5.5, 6.1-6.4 | A1 due, A2 released |
| T 11/8 | HMM Training | Jurafsky/Martin, chapters 5.7-5.8, 6.5 HMM spreadsheet, Eisner's ice cream example |
|
| F 11/11 | discussion of A1 | be prepared to present your solutions! | |
| T 11/15 | Context-Free Grammars & Parsing (CKY) | Jurafsky/Martin, chapter 13 (-13.4), J/M chapter 12 as background |
|
| F 11/18 | Probabilistic Context-Free Grammars | Jurafsky/Martin, chapters 14.1-14.5, 14.7 | A2 due, A3 released |
| T 11/22 | Training PCFGs | Jurafsky/Martin, chapter 14.3 Manning/Schütze, chapter 11 Michael Collins, The inside-outside algorithm. |
|
| F 11/25 | discussion of A2 | be prepared to present your solutions! | |
| T 11/29 | Advanced PCFG models | Mark Johnson (1998), PCFG
Models of Linguistic Tree Representations (esp. on parent
annotations) Michael Collins, Lexicalized PCFGs Dan Klein/Chris Manning (2003), Accurate unlexicalized parsing |
|
| F 12/2 | Dependency parsing | McDonald/Pereira/Ribarov/Hajic (2005),
Non-projective
Dependency Parsing using Spanning Tree Algorithms Joakim Nivre (2008), Algorithms for Deterministic Incremental Dependency Parsing |
A3 due, A4 released |
| T 12/6 | Speech recognition | Jurafsky/Martin, ch. 9 Links for further reading: "Human parity" speech recognition Language log on human parity speech recognition Google speech API web demo S. Germesin on using the speech API remotely |
|
| F 12/9 | discussion of A3 | be prepared to present your
solutions! Joshua Goodman, Semiring Parsing. Computational Linguistics, 1999. |
|
| T 12/13 | Speech synthesis | Jurafsky/Martin, ch. 8 | |
| F 12/16 | no class | A4 due | |
| 12/19-23 | no class (winter break) | ||
| 12/26-30 | no class (winter break) | ||
| T 1/3 | Statistical machine translation: alignments | Jurafsky/Martin, ch. 25 (through 25.6) Adam Lopez, Word Alignment and the Expectation-Maximization Algorithm tutorial (try here) http://mt-class.org/ MT Talks |
final project guidelines |
| F 1/6 | discussion of A4 | be prepared to present your solutions! | A5 released |
| T 1/10 | Phrase-based machine translation | Jurafsky/Martin, ch. 25 | |
| F 1/13 | Syntax-based machine translation | David Chiang, Hierarchical phrase-based translation. Computational Linguistics, 2007. | project proposal due! |
| T 1/17 | Semantic parsing | Zettlemoyer/Collins, Learning to Map Sentences to
Logical Form. Structured Classification with Probabilistic
Categorial Grammars. 2005 Wong/Mooney, Learning for Semantic Parsing with Statistical Machine Translation. HLT-NAACL, 2006 Opt.: Mark Steedman, A very short introduction to CCG. 1996 |
|
| F 1/20 | Lexical semantics | Jurafsky/Martin, chs. 19 + 20 Further reading: Mitchell/Lapata (2008), Vector-based models of semantic composition; Baroni/Zamparelli (2010), Nouns are vectors, adjectives are matrices |
A5 due, A6 released |
| T 1/24 | LDA | David Blei, Probabilistic topic models. 2012 | |
| F 1/27 | discussion of A5 | be prepared to present your solutions! | |
| T 1/31 | NLP for discourse | Webber/Egg/Kordoni, Discourse
Structure and Language Technology. 2012. (accessible from the
U Potsdam network) We'll discuss the following papers on Bayesian pragmatics for discourse relations: Yung et al., 2016a, Yung et al., 2016b |
|
| F 2/3 | no class (instructor away) | A6 due | |
| T 2/7 | Intro to deep neural networks | Richard Socher, Recursive Deep Learning for Natural Language
Processing and Computer Vision (Chapter 2). 2014 Further reading (for intuitions): http://karpathy.github.io/neuralnets/, http://neuralnetworksanddeeplearning.com/chap4.html |
|
| F 2/10 | discussion of A6 + presentations of final projects | 10-minute presentations of ideas, approaches, preliminary results |
Most computational linguists own both of these books. We will assign weekly readings, so you should ensure you get your own copy or have access to the copies that are available in the university library.