Facial emotion recognition is vital for human social behavior. During the COVID-19 pandemic, face masks were widely adopted for viral mitigation and remain crucial public health tools. However, questions persist about their impact on emotion recognition and neural processing, especially in children, parents, and young adults. We developed the Masked Affective and Social Cognition task, featuring masked and unmasked faces displaying fear, sadness, and anger. We recruited three racial and ethnically diverse samples: 119 college students, 30 children who entered school age at the beginning of the pandemic, and 31 fathers of the aforementioned children. Of the latter two groups, 41 participants (n = 23 fathers, 18 children) did the Masked Affective and Social Cognition task during a neuroimaging scan, while the remaining 20 participants (n = 8 fathers, 12 children) who were not eligible for scanning completed the task during their lab visit. Behaviorally, we found that participants recognized emotions less accurately when viewing masked faces and also found an interaction of emotion by condition, such that accuracy was particularly compromised by sad masked faces. Neurally, masked faces elicited greater activation in the posterior cingulate, insula, and fusiform gyrus. Anterior insula and inferior frontal gyrus activation were driven by sad, masked faces. These results were consistent across age groups. Among fathers, activation to sad masked faces was associated with stress and depression. Overall, our findings did not depend on previous mask exposure or timing of participation during the pandemic. These results have implications for understanding face emotion recognition, empathy, and socioemotional neurodevelopment. (PsycInfo Database Record (c) 2024 APA, all rights reserved).