Exploring Relevant Features for EEG-Based Investigation of Sound Perception in Naturalistic Soundscapes

A comprehensive analysis of everyday sound perception can be achieved using Electroencephalography (EEG) with the concurrent acquisition of information about the environment. While extensive research has been dedicated to speech perception, the complexities of auditory perception within everyday environments, specifically the types of information and the key features to extract, remain less explored. Our study aims to systematically investigate the relevance of different feature categories: discrete sound-identity markers, general cognitive state information, and acoustic representations, including discrete sound onset, the envelope, and mel-spectrogram. Using continuous data analysis, we contrast different features in terms of their predictive power for unseen data and thus their distinct contributions to explaining neural data. For this, we analyse data from a complex audio-visual motor task using a naturalistic soundscape. The results demonstrated that the feature sets that explain the most neural variability were a combination of highly detailed acoustic features with a comprehensive description of specific sound onsets. Furthermore, it showed that established features can be applied to complex soundscapes. Crucially, the outcome hinged on excluding periods devoid of sound onsets in the analysis in the case of the discrete features. Our study highlights the importance to comprehensively describe the soundscape, using acoustic and nonacoustic aspects, to fully understand the dynamics of sound perception in complex situations. This approach can serve as a foundation for future studies aiming to investigate sound perception in natural settings.Significance Statement This study is an important step in our broader research endeavor, which aims to understand sound perception in everyday life. Although conducted in a stationary setting, this research provides foundational insights into necessary environmental information to obtain to understand concurrent neural responses. We delved into the analysis of various acoustic features, sound-identity labeling, and cognitive information, with the goal of refining neural models related to sound perception. Our findings particularly highlight the need for a thorough analysis and description of complex soundscapes. Our study provides key considerations for future research in sound perception across various contexts, from laboratory settings to mobile EEG technologies, and paves the way for investigations into more naturalistic environments, advancing the field of auditory neuroscience.