Presenter:  Woojae Kim
Presentation type:  Poster
Presentation date/time:  7/26  5:30-6:30
 
Model Selection with Data under Individual Differences
 
Woojae Kim, Indiana University
Richard Shiffrin, Indiana University
 
Hierarchical modeling has been demonstrated as a good way of modeling data with individual differences (Rouder and Lu, 2004; Navarro et al., 2006). In a situation where the size of data available from each subject is small and individual differences clearly exist, hierarchical modeling provides far more accurate model estimation than modeling either individual or averaged (or aggregate) data. Can we expect the same kind of benefit from hierarchical modeling for the model selection problem? That is, does a model selection judgment made with hierarchical models represent a better decision than that with models of individual or aggregate data in a situation like the above? The present study investigates this question. By taking models from different modeling areas and employing simulation approaches, this study evaluates the decision performance of model selection with hierarchical models, in comparison to model selection with models of individual and aggregate data. Predictive accuracy, which is operationalized by the discrepancy of the selected model from the true, generating model, is used as a criterion for the decision performance. The simulation design includes the variation of sample size within a subject and the different degrees of individual differences. The results demonstrate that hierarchical modeling provides better decision making for model selection.