CmpE 591 Empirical Methods in Artificial Intelligence

Spring 2004

Goals In this course we discuss empirical methods for experiment design and statistical analysis.

Instructor Ethem Alpaydin, Professor, alpaydin AT boun.edu.tr
Textbook Paul R. Cohen, Empirical Methods for Artificial Intelligence, The MIT Press, 1995 and extra material (see below)

Discussion leader, topic by week:

• Oya: (Cohen, Ch 2) Exploratory Data Analysis
• Itir: (Cohen, Ch 3) Basic Issues in Experiment Design
• Ersin: (Cohen, Ch 4) Hypothesis Testing and Estimation
• Onur: (Cohen, Ch 5) Computer-Intensive Statistical Methods
• Baris: (Cohen, Ch 6) Performance Assessment
• Kemal: (Cohen, Ch 7) Explaining Performance: Interactions and Dependencies
• Cetin: (Cohen, Ch 8) Modeling
• Arzucan: (Cohen, Ch 9) Tactics for Generalization
• Levent: (Witten, Frank Data Mining Morgan Kaufmann Ch 5) "Credibility: Evaluating what's been learned"
• Ali: Dietterich "Approximate statistical tests for comparing supervised classification learning algorithms" Neural Computation 1998, Alpaydin "Combined 5x2 cv F test for comparing supervised classification learning algorithms" Neural Computation 1999
• Alp: Adams, Hand "Improving the practice of classifier performance assessment" Neural Computation 2000, Salzberg, "On comparing classifiers: A critique of current research and methods" Data Mining and Knowledge Discovery
• Burak: (Conover Practical Nonparametric Statistics 3rd Ed Wiley Secs 2.5, 3.1,3.2) Some tests based on the binomial distribution
• Aydin: (Conover Secs 3.3-5) Sign test
• Bugra: (Conover Secs 4.1-4) Contingency tables
• Utku: (Conover Secs 4.5-8) Goodness of fit tests