Dome, L., & Wills, A. J. (2025). g-Distance: On the Comparison of Model and Human Heterogeneity. Psychological Review, 132(3), 632–655. https://doi.org/10.1037/rev0000550
Models are often evaluated when their behavior is at its closest to a single, sometimes averaged, set of empirical results, but this evaluation neglects the fact that both model and human behavior can be heterogeneous. Here, we develop a measure, g-distance, which considers model adequacy as the extent to which models exhibit a similar range of behaviors to the humans they model. We define g as the combination of two easily interpretable dimensions of model adequacy: accommodation and excess flexibility. We apply this measure to five models of an irrational learning effect, the inverse base-rate effect (IBRE). g-distance identifies two models, a neural network with rapid attentional shifts (NNRAS) and a dissimilarity-similarity generalized context model (DGCM18), that outperform the previously most supported model (EXIT). We show that this conclusion holds for a wide range of beliefs about the relative importance of excess flexibility and accommodation. We further show that a pre-existing metric, the Bayesian Information Criterion (BIC), misidentifies a known-poor model of the IBRE as the most adequate model. Along the way, we discover that some of the models accommodate human behavior in ways that seem unintuitive from an informal understanding of their operation, thus underlining the importance of formal expression of theories. We discuss the implications of our findings for model evaluation generally, and for models of the inverse base-rate effect in particular, and end by suggesting future avenues of research in computational modeling.
@article{dome2025gdistance,title={g-{{Distance}}: {{On}} the Comparison of Model and Human Heterogeneity.},shorttitle={g-{{Distance}}},author={Dome, Lenard and Wills, Andy J.},year={2025},journal={Psychological Review},volume={132},number={3},pages={632--655},publisher={American Psychological Association},issn={0033-295X},doi={10.1037/rev0000550},urldate={2025-04-16},langid={english},}
Dome, L., & Wills, A. (2025). Better generalization through distraction? Concurrent load reduces the size of the inverse base-rate effect. Psychonomic Bulletin & Review. https://doi.org/https://doi.org/10.3758/s13423-025-02661-1
The inverse base-rate effect (IBRE) is an irrational phenomenon in predictive learning. It occurs when people try to generalize what they have experienced to novel and ambiguous events. This irrational generalization manifests as a preference for rare, unlikely outcomes in the face of ambiguity. At least two formal mathematical models of this irrational preference (EXIT, NNRAS) lead to a counter-intuitive prediction: the effect reduces under concurrent load. We tested this prediction across two experiments (N1 = 72, M1age = 20.12; N2 = 160, M2age = 20.88). We confirm the prediction, but only when participants were under an obvious time constraint. This empirical confirmation is as surprising as the prediction itself—irrationality reduces under increased task demands. Further, our data are more consistent with the NNRAS model than with EXIT, the most prominent model of the IBRE to date.
@article{dome2025better,title={Better generalization through distraction? Concurrent load reduces the size of the inverse base-rate effect},journal={Psychonomic Bulletin & Review.},author={Dome, Lenard and Wills, Andy},year={2025},doi={https://doi.org/10.3758/s13423-025-02661-1},}
news
2025.04.16.
PAPER ALERT New Metric, New Rankings: Psychological Model Evaluation Rethinked. Check out our new paper, where we introduce g-distance to evaluate psychological models by comparing the range of behaviours they exhibit to the heterogeneity observed in human behaviour. Assessing not only how well models produce observed behaviours, but also how well they reject unobserved ones. Check it out at Psychological Review or read it here!
2025.02.26.
PAPER ALERT Surprisingly, irrational generalization decreases as task demands rise—but only for ambiguous items! The only model capturing all data trends: rapid, competitive attentional shifts—a simpler alternative to the title-holder, EXIT. Check out our new paper in Psychonomic Bulletin & Review or read it here!
PAPER ALERT Dome and Wills (2023) Errorless irrationality: removing error-driven components from the inverse base-rate effect paradigm. In Proceedings of the Annual Meeting of the Cognitive Science Society 45 (45) 237-243