Presenter:  Jared Smith
Presentation type:  Talk
Presentation date/time:  7/28  9:50-10:15
 
Effects of Misspecification in Hierarchical modeling
 
Jared Smith, University of California, Irvine
 
A number of recent papers within the cognitive modeling literature have proposed the application of hierarchical modeling to take into account subject variability (e.g., DeCarlo, 2002, Psychological Review; Karabatsos & Batchelder, 2003 Psychometrika; Klauer, 2006, Psychometrika; Lee & Webb, 2005, Psychonomic Bulletin & Review; Rouder & Lu, 2005, Psychometrika; Rouder, Sun, Speckman, Lu, & Zhou, 2003, Psychonomic Bulletin & Review). One limitation of these methods is that they require making approximately accurate hierarchical assumptions concerning the distribution of a model's parameters over successive subject observations. These assumptions may be misspecified even if the base model is correctly specified. The purpose of this paper is to examine the consequences of misspecification at the hierarchical level with simulations and analysis of existing data sets. In particular, it is demonstrated that the application of finite mixture models may lead to deceptive results if the underlying distribution of the parameters is continuous and unimodal. Moreover, results may be problematic in the case where individual differences are modeled with unimodal distributions, when the true data generating distribution is multimodal (e.g. a finite mixture models). It is argued that hierarchical modeling provides a powerful method to account for individual differences, but that researchers should take care in interpreting the fitted hierarchical distributions without appropriate model checking, especially for the hierarchical assumptions.