Analyzing the data of individuals has several advantages over analyzing the data combined across the individuals (we term this 'group analysis'): Grouping can distort the form of data, and different individuals might perform the task using different processes and parameters. We investigate the possibility that group analysis might still be useful, and might even outperform individual analysis, when there is a large amount of measurement noise, such as might occur for an experiment with few trials per condition. We employ a simulation technique in which data are generated from each of two known models (e.g., the exponential and power laws of forgetting), each with parameter variation across simulated individuals. We examine how well the generating model and its competitor each fare in fitting (both sets of) the data, using both individual and group analysis. To assess accuracy, we need a method for selecting a model based on the fits. We compare the results of selecting models based on maximum likelihood, AIC, BIC, minimum description length, Bayesian model selection, cross validation, generalization, and predictive validation. We looked at a wide range of comparison models, subject parameters, number of subjects, and trials per condition, and found cases where group analysis is a more accurate method of model selection. |