Evaluation 1: Fundamental Empirical Techniques and Caveats

Tutorial proceedings

This tutorial will introduce UM researchers to the fundamental techniques of empirical evaluation of user modeling systems. The target audience of this tutorial are new UM researchers who want to learn how to evaluate their UM system. No background in statistics is assumed. Topics will include experiment design, running experiments, and experiment analysis, especially statistical techniques for analysis.

After this tutorial, attendees will be well prepared to participate in the follow-on afternoon tutorial, "Evaluation 2: Formative Evaluation Methods for Adaptive Systems" where attendees will have a chance to apply what they have learned in this tutorial to their own specific evaluation problems.

Lecture notes will be made available during the tutorial and afterwards on the website.

Date and time

• Tuesday, June 26 2007, Morning session

Organizer and Contact Person

David N. Chin Information & Computer Sciences, University of Hawaii, USA David Chin is an expert in the empirical evaluation of user modeling systems with two previous UM Conference tutorials on evaluation. He has served on the editorial board of the international journal, User Modeling and User-Adapted Interaction, since its inception in 1990, guest edited two special issues of the journal on empirical evaluation (Volume 12 issues 2-3), served on the organizing committees of the 2001, 2003, and 2005 UM Conference Workshops on Empirical Evaluation of Adaptive Systems and written a UMUAI review article on empirical evaluation. David N. Chin Univ of Hawaii, Dept of ICS 1680 East West Rd, POST 371 USA +1 808 956 8162 chin@hawaii.edu