Cognitive Models for Machine Theory of Mind

Some of the required characteristics for a true machine theory of mind (MToM) include the ability to (1) reproduce the full diversity of human thought and behavior, (2) develop a personalized model of an individual with very limited data, and (3) provide an explanation for behavioral predictions grounded in the cognitive processes of the individual. We propose that a certain class of cognitive models provide an approach that is well suited to meeting those requirements. Being grounded in a mechanistic framework like a cognitive architecture such as ACT-R naturally fulfills the third requirement by mapping behavior to cognitive mechanisms. Exploiting a modeling paradigm such as instance-based learning accounts for the first requirement by reflecting variations in individual experience into a diversity of behavior. Mechanisms such as knowledge tracing and model tracing allow a specific run of the cognitive model to be aligned with a given individual behavior trace, fulfilling the second requirement. We illustrate these principles with a cognitive model of decision-making in a search and rescue task in the Minecraft simulation environment. We demonstrate that cognitive models personalized to individual human players can provide the MToM capability to optimize artificial intelligence agents by diagnosing the underlying causes of observed human behavior, projecting the future effects of potential interventions, and managing the adaptive process of shaping human behavior. Examples of the inputs provided by such analytic cognitive agents include predictions of cognitive load, probability of error, estimates of player self-efficacy, and trust calibration. Finally, we discuss implications for future research and applications to collective human-machine intelligence.