CSI5180: Topics in Artificial Intelligence

Natural Language Processing, A Statistical Approach

Instructor

Nathalie Japkowicz

Office: STE 5029
E-mail: nat@site.uottawa.ca
Telephone: 562-5800 ext. 6693 (Note: e-mail is more reliable)

Meeting Times and Locations

Time:
Monday, 12:30pm-2:00pm;
Thurdsay, 2:00pm-3:30pm
Location: JJL 140C

Office Hours and Locations

Times: Wednesday, 11-1
Location: STE 5029

Overview

Natural Language Processing (NLP) is the subfield of Artificial Intelligence concerned with building computer systems such as natural language interfaces to databases or the World-Wide Web, automatic machine-translation systems, text analysis systems, speech understanding systems, or computer-aided instruction systems.

Until recently, NLP was mainly approached by rule-based or symbolic methods. In the past few years, however, statistical methods have been given a lot of attention as they seem to address many of the bottlenecks encountered by the symbolic methods.

This course will focus mainly on statistical approaches. In particulat, we will concentrate on approaches such as n-gram models, markov models, probabilitic context free grammars. If time permits, we will consider applications such as statistical alignment and machine translation, clustering, information retrieval and text categorization.

Pre-Requisites

Students should have reasonable exposure to Artificial Intelligence and some programming experience in a high level language.

Evaluation

Students will be evaluated on:

A series of 4 written and programming assignments (40%)
One in-class Presentation (10%)
Seven research paper critics [individual or in group](10%)
A Final Project (40%)

Required Textbooks

Foundations of Statistical Natural Language Processing, by Chris Manning and Hinrich Schutze, MIT Press.

Timetable

Monday, September 22: Assignment # 1 handed out (Due on October 9)
Thursday, October 9: Assignment # 2 handed out (Due on October 30)
Thursday, October 30: Assignment # 3 handed out (Due on November 13)
Thursday, November 6: Project Proposal Due
Thursday, November 13: Assignment # 4 handed out (Due on November 27)
Monday, December 1: Projects Due, Project Presentation

Assignments

Assignment 1, postscript , PDF (Now available)
Assignment 2 postscript , PDF (Now available)
Assignment 3, postscript , PDF (Now available)
Assignment 4, postscript , PDF (Now available)

Course Support:

Useful Links:

Syllabus

(click on "Powerpoint File" to retrieve lecture slides)

Week	Topics	Readings
Week 1: September 4	Preliminaries Powerpoint File Introduction to Statistical NLP Powerpoint File
Week 2: September 8-11	Linguistics Essentials Powerpoint File Mathematical Foundations I: Probability Theory Powerpoint File	Chap. 1, 2 & 3
Week 3: September 15-18	Mathematical Foundations II: Information Theory Powerpoint File Corpus Based Work Powerpoint File	Chap. 2 & 4
Week 4: September 22-25	Collocations Powerpoint File	Chap. 5 Collocation Dictionary Optimization Using WordNet and k-Nearest Neighbor Learning by Yuseop Kim , Byoung-Tak Zhang , Yung Taek Kim in Machine Translation, Volume 16, Issue 2, June 2001.
Week 5: September 29 - October 2	Statistical Inference Powerpoint File	Chap. 6 A Loss Function Analysis for the Classification Methods in Text Categorization Support vector machines for spam categorization by Drucker, Wu, and Vapnik, IEE TNN, Vol 10, No. 5, 1999.
Week 6: October 6-9	Word Sense Disambiguation Powerpoint File	Chap. 7 & Word Sense Acquisition from Comparable Bilingual Corpora by Hiroyuki Kaji in HLT-NAACL 2003 Exploiting Parallel Texts for Word Sense Disambiguation: An Empirical Study by Hwee Tou Ng, Bin Wang and Yee Seng Chan in ACL 2003
Week 7: October 13-16	Lexical Acquistion Powerpoint File	Chap. 8 Improving Subcategorization Acquisition Using Word Sense Disambiguation by Anna Korhonen and Judita Preiss in ACL 2003 Syntactic Features and Word Similarity for Supervised Metonymy Resolution by Malvina Nissim and Katja Markert in ACL 2003
Week 8: October 20-23	MidTerm Week (Review + Exam)	Chaps. 1, 2, 3, 4, 5, 6, 7 & 8
Week 9: October 27-30	Markov Models Powerpoint File	Chap. 9 Automatic Labeling of Semantic Roles by Daniel Gildea and Daniel Jurafsky in Computational Linguistics, Volume 28, Issue 3, September 2003. Another Paper
Week 10: November 3-6	Part-of-Speech Tagging	Chap. 10 A SNoW Based Supertagger with applications to NP Chunking, by Libin Chen and Aarvind Joshi in ACL'03. Detecting Errors in Part-of-Speech Annotations by Markus Dickinson and Detmar Meurers in EACL'03.
Week 11: November 10-13	Probabilistic Context Free Grammars Powerpoint File	Chap. 9 & 11
Week 12: November 17-20	Probabilistic Parsing	Chap. 12
Week 13: November 24-27	Statistical Alignment & Machine Translation; Powerpoint File	Chap. 13 Word Reordering and a DP-based Search in Statistical Machine Translation by Christoph Tillmann and Hermann Ney in Computational Linguistics, Vol 29, Issue 1, March 2003. Minimum Error-rate training in Statistical Machine Translation by Franz Josef Och in ACL'03.

CSI5180: Topics in Artificial Intelligence Natural Language Processing, A Statistical Approach