Exploring Domain-Specific Enhancements for a Neural Foley Synthesizer
Authors:
Ashwin Pillay,
Sage Betko,
Ari Liloia,
Hao Chen,
Ankit Shah
Abstract:
Foley sound synthesis refers to the creation of authentic, diegetic sound effects for media, such as film or radio. In this study, we construct a neural Foley synthesizer capable of generating mono-audio clips across seven predefined categories. Our approach introduces multiple enhancements to existing models in the text-to-audio domain, with the goal of enriching the diversity and acoustic charac…
▽ More
Foley sound synthesis refers to the creation of authentic, diegetic sound effects for media, such as film or radio. In this study, we construct a neural Foley synthesizer capable of generating mono-audio clips across seven predefined categories. Our approach introduces multiple enhancements to existing models in the text-to-audio domain, with the goal of enriching the diversity and acoustic characteristics of the generated foleys. Notably, we utilize a pre-trained encoder that retains acoustical and musical attributes in intermediate embeddings, implement class-conditioning to enhance differentiability among foley classes in their intermediate representations, and devise an innovative transformer-based architecture for optimizing self-attention computations on very large inputs without compromising valuable information. Subsequent to implementation, we present intermediate outcomes that surpass the baseline, discuss practical challenges encountered in achieving optimal results, and outline potential pathways for further research.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
Structured randomness: Jamming of soft discs and pins
Authors:
Prairie Wentworth-Nice,
Sean A. Ridout,
Brian Jenike,
Ari Liloia,
Amy L. Graves
Abstract:
Simulations are used to find the zero temperature jamming threshold, $φ_j$, for soft, bidisperse disks in the presence of small fixed particles, or "pins", arranged in a lattice. The presence of pins leads, as one expects, to a decrease in $φ_j$. Structural properties of the system near the jamming threshold are calculated as a function of the pin density. While the correlation length exponent rem…
▽ More
Simulations are used to find the zero temperature jamming threshold, $φ_j$, for soft, bidisperse disks in the presence of small fixed particles, or "pins", arranged in a lattice. The presence of pins leads, as one expects, to a decrease in $φ_j$. Structural properties of the system near the jamming threshold are calculated as a function of the pin density. While the correlation length exponent remains $ν= 1/2$ at low pin densities, the system is mechanically stable with more bonds, yet fewer contacts than the Maxwell criterion implies in the absence of pins. In addition, as pin density increases, novel bond orientational order and long-range spatial order appear, which are correlated with the square symmetry of the pin lattice.
△ Less
Submitted 29 April, 2020; v1 submitted 9 April, 2020;
originally announced April 2020.
