Showing 1–50 of 103 results for author: Yin, H

A Superdirective Beamforming Approach based on MultiTransUNet-GAN

Authors: Yali Zhang, Haifan Yin, Liangcheng Han

Abstract: In traditional multiple-input multiple-output (MIMO) communication systems, the antenna spacing is often no smaller than half a wavelength. However, by exploiting the coupling between more closely-spaced antennas, a superdirective array may achieve a much higher beamforming gain than traditional MIMO. In this paper, we present a novel utilization of neural networks in the context of superdirective… ▽ More In traditional multiple-input multiple-output (MIMO) communication systems, the antenna spacing is often no smaller than half a wavelength. However, by exploiting the coupling between more closely-spaced antennas, a superdirective array may achieve a much higher beamforming gain than traditional MIMO. In this paper, we present a novel utilization of neural networks in the context of superdirective arrays. Specifically, a new model called MultiTransUNet-GAN is proposed, which aims to forecast the excitation coefficients to achieve ``superdirectivity" or ``super-gain" in the compact uniform linear or planar antenna arrays. In this model, we integrate a multi-level guided attention and a multi-scale skip connection. Furthermore, generative adversarial networks are integrated into our model. To improve the prediction accuracy and convergence speed of our model, we introduce the warm up aided cosine learning rate (LR) schedule during the model training, and the objective function is improved by incorporating the normalized mean squared error (NMSE) between the generated value and the actual value. Simulations demonstrate that the array directivity and array gain achieved by our model exhibit a strong agreement with the theoretical values. Overall, it shows the advantage of enhanced precision over the existing models, and a reduced requirement for measurement and the computation of the excitation coefficients. △ Less

Submitted 27 August, 2024; v1 submitted 24 August, 2024; originally announced August 2024.

Comments: 12 pages, 11 figures, 6 tables, to appear in IEEE Trans. Commun

Transforming Time-Varying to Static Channels: The Power of Fluid Antenna Mobility

Authors: Weidong Li, Haifan Yin, Fanpo Fu, Yandi Cao, Merouane Debbah

Abstract: This paper addresses the mobility problem with the assistance of fluid antenna (FA) on the user equipment (UE) side. We propose a matrix pencil-based moving port (MPMP) prediction method, which may transform the time-varying channel to a static channel by timely sliding the liquid. Different from the existing channel prediction method, we design a moving port selection method, which is the first a… ▽ More This paper addresses the mobility problem with the assistance of fluid antenna (FA) on the user equipment (UE) side. We propose a matrix pencil-based moving port (MPMP) prediction method, which may transform the time-varying channel to a static channel by timely sliding the liquid. Different from the existing channel prediction method, we design a moving port selection method, which is the first attempt to transform the channel prediction to the port prediction by exploiting the movability of FA. Theoretical analysis shows that for the line-ofsight (LoS) channel, the prediction error of our proposed MPMP method may converge to zero, as the number of BS antennas and the port density of the FA are large enough. For a multi-path channel, we also derive the upper and lower bounds of the prediction error when the number of paths is large enough. When the UEs move at a speed of 60 or 120 km/h, simulation results show that, with the assistance of FA, our proposed MPMP method performs better than the existing channel prediction method. △ Less

Submitted 9 August, 2024; v1 submitted 8 August, 2024; originally announced August 2024.

Mixstyle based Domain Generalization for Sound Event Detection with Heterogeneous Training Data

Authors: Yang Xiao, Han Yin, Jisheng Bai, Rohan Kumar Das

Abstract: This work explores domain generalization (DG) for sound event detection (SED), advancing adaptability towards real-world scenarios. Our approach employs a mean-teacher framework with domain generalization to integrate heterogeneous training data, while preserving the SED model performance across the datasets. Specifically, we first apply mixstyle to the frequency dimension to adapt the mel-spectro… ▽ More This work explores domain generalization (DG) for sound event detection (SED), advancing adaptability towards real-world scenarios. Our approach employs a mean-teacher framework with domain generalization to integrate heterogeneous training data, while preserving the SED model performance across the datasets. Specifically, we first apply mixstyle to the frequency dimension to adapt the mel-spectrograms from different domains. Next, we use the adaptive residual normalization method to generalize features across multiple domains by applying instance normalization in the frequency dimension. Lastly, we use the sound event bounding boxes method for post-processing. Our approach integrates features from bidirectional encoder representations from audio transformers and a convolutional recurrent neural network. We evaluate the proposed approach on DCASE 2024 Challenge Task 4 dataset, measuring polyphonic SED score (PSDS) on the DESED dataset and macro-average pAUC on the MAESTRO dataset. The results indicate that the proposed DG-based method improves both PSDS and macro-average pAUC compared to the challenge baseline. △ Less

Submitted 29 August, 2024; v1 submitted 4 July, 2024; originally announced July 2024.

Comments: Submitted to ICASSP 2025

FMSG-JLESS Submission for DCASE 2024 Task4 on Sound Event Detection with Heterogeneous Training Dataset and Potentially Missing Labels

Authors: Yang Xiao, Han Yin, Jisheng Bai, Rohan Kumar Das

Abstract: This report presents the systems developed and submitted by Fortemedia Singapore (FMSG) and Joint Laboratory of Environmental Sound Sensing (JLESS) for DCASE 2024 Task 4. The task focuses on recognizing event classes and their time boundaries, given that multiple events can be present and may overlap in an audio recording. The novelty this year is a dataset with two sources, making it challenging… ▽ More This report presents the systems developed and submitted by Fortemedia Singapore (FMSG) and Joint Laboratory of Environmental Sound Sensing (JLESS) for DCASE 2024 Task 4. The task focuses on recognizing event classes and their time boundaries, given that multiple events can be present and may overlap in an audio recording. The novelty this year is a dataset with two sources, making it challenging to achieve good performance without knowing the source of the audio clips during evaluation. To address this, we propose a sound event detection method using domain generalization. Our approach integrates features from bidirectional encoder representations from audio transformers and a convolutional recurrent neural network. We focus on three main strategies to improve our method. First, we apply mixstyle to the frequency dimension to adapt the mel-spectrograms from different domains. Second, we consider training loss of our model specific to each datasets for their corresponding classes. This independent learning framework helps the model extract domain-specific features effectively. Lastly, we use the sound event bounding boxes method for post-processing. Our proposed method shows superior macro-average pAUC and polyphonic SED score performance on the DCASE 2024 Challenge Task 4 validation dataset and public evaluation dataset. △ Less

Submitted 28 June, 2024; originally announced July 2024.

Comments: Technical report for DCASE 2024 Challenge Task 4

Single-Codec: Single-Codebook Speech Codec towards High-Performance Speech Generation

Authors: Hanzhao Li, Liumeng Xue, Haohan Guo, Xinfa Zhu, Yuanjun Lv, Lei Xie, Yunlin Chen, Hao Yin, Zhifei Li

Abstract: The multi-codebook speech codec enables the application of large language models (LLM) in TTS but bottlenecks efficiency and robustness due to multi-sequence prediction. To avoid this obstacle, we propose Single-Codec, a single-codebook single-sequence codec, which employs a disentangled VQ-VAE to decouple speech into a time-invariant embedding and a phonetically-rich discrete sequence. Furthermor… ▽ More The multi-codebook speech codec enables the application of large language models (LLM) in TTS but bottlenecks efficiency and robustness due to multi-sequence prediction. To avoid this obstacle, we propose Single-Codec, a single-codebook single-sequence codec, which employs a disentangled VQ-VAE to decouple speech into a time-invariant embedding and a phonetically-rich discrete sequence. Furthermore, the encoder is enhanced with 1) contextual modeling with a BLSTM module to exploit the temporal information, 2) a hybrid sampling module to alleviate distortion from upsampling and downsampling, and 3) a resampling module to encourage discrete units to carry more phonetic information. Compared with multi-codebook codecs, e.g., EnCodec and TiCodec, Single-Codec demonstrates higher reconstruction quality with a lower bandwidth of only 304bps. The effectiveness of Single-Code is further validated by LLM-TTS experiments, showing improved naturalness and intelligibility. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: Accepted by Interspeech 2024

A DAFT Based Unified Waveform Design Framework for High-Mobility Communications

Authors: Xingyao Zhang, Haoran Yin, Yanqun Tang, Yu Zhou, Yuqing Liu, Jinming Du, Yipeng Ding

Abstract: With the increasing demand for multi-carrier communication in high-mobility scenarios, it is urgent to design new multi-carrier communication waveforms that can resist large delay-Doppler spreads. Various multi-carrier waveforms in the transform domain were proposed for the fast time-varying channels, including orthogonal time frequency space (OTFS), orthogonal chirp division multiplexing (OCDM),… ▽ More With the increasing demand for multi-carrier communication in high-mobility scenarios, it is urgent to design new multi-carrier communication waveforms that can resist large delay-Doppler spreads. Various multi-carrier waveforms in the transform domain were proposed for the fast time-varying channels, including orthogonal time frequency space (OTFS), orthogonal chirp division multiplexing (OCDM), and affine frequency division multiplexing (AFDM). Among these, the AFDM is a strong candidate for its low implementation complexity and ability to achieve optimal diversity. This paper unifies the waveforms based on the discrete affine Fourier transform (DAFT) by using the chirp slope factor "k" in the time-frequency representation to construct a unified design framework for high-mobility communications. The design framework is employed to verify that the bit error rate performance of the DAFT-based waveform can be enhanced when the signal-to-noise ratio (SNR) is sufficiently high by adjusting the chirp slope factor "k". △ Less

Submitted 4 June, 2024; originally announced June 2024.

Dual Hyperspectral Mamba for Efficient Spectral Compressive Imaging

Authors: Jiahua Dong, Hui Yin, Hongliu Li, Wenbo Li, Yulun Zhang, Salman Khan, Fahad Shahbaz Khan

Abstract: Deep unfolding methods have made impressive progress in restoring 3D hyperspectral images (HSIs) from 2D measurements through convolution neural networks or Transformers in spectral compressive imaging. However, they cannot efficiently capture long-range dependencies using global receptive fields, which significantly limits their performance in HSI reconstruction. Moreover, these methods may suffe… ▽ More Deep unfolding methods have made impressive progress in restoring 3D hyperspectral images (HSIs) from 2D measurements through convolution neural networks or Transformers in spectral compressive imaging. However, they cannot efficiently capture long-range dependencies using global receptive fields, which significantly limits their performance in HSI reconstruction. Moreover, these methods may suffer from local context neglect if we directly utilize Mamba to unfold a 2D feature map as a 1D sequence for modeling global long-range dependencies. To address these challenges, we propose a novel Dual Hyperspectral Mamba (DHM) to explore both global long-range dependencies and local contexts for efficient HSI reconstruction. After learning informative parameters to estimate degradation patterns of the CASSI system, we use them to scale the linear projection and offer noise level for the denoiser (i.e., our proposed DHM). Specifically, our DHM consists of multiple dual hyperspectral S4 blocks (DHSBs) to restore original HSIs. Particularly, each DHSB contains a global hyperspectral S4 block (GHSB) to model long-range dependencies across the entire high-resolution HSIs using global receptive fields, and a local hyperspectral S4 block (LHSB) to address local context neglect by establishing structured state-space sequence (S4) models within local windows. Experiments verify the benefits of our DHM for HSI reconstruction. The source codes and models will be available at https://github.com/JiahuaDong/DHM. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Comments: 13 pages, 6 figures

High-Precision Positioning with Continuous Delay and Doppler Shift using AFT-MC Waveforms

Authors: Cong Yi, Haoran Yin, Xianjie Lu, Yanqun Tang

Abstract: This paper explores a novel integrated localization and communication (ILAC) system using the affine Fourier transform multicarrier (AFT-MC) waveform. Specifically, we consider a multiple-input multiple-output (MIMO) AFT-MC system with ILAC and derive a continuous delay and Doppler shift channel matrix model. Based on the derived signal model, we develop a two-step algorithm with low complexity fo… ▽ More This paper explores a novel integrated localization and communication (ILAC) system using the affine Fourier transform multicarrier (AFT-MC) waveform. Specifically, we consider a multiple-input multiple-output (MIMO) AFT-MC system with ILAC and derive a continuous delay and Doppler shift channel matrix model. Based on the derived signal model, we develop a two-step algorithm with low complexity for estimating channel parameters. Furthermore, we derive the Cramér-Rao lower bound (CRLB) of location estimation as the fundamental limit of localization. Finally, we provide some insights about the AFT-MC parameters by explaining the impact of the parameters on localization performance. Simulation results demonstrate that the AFT-MC waveform is able to provide significant localization performance improvement compared to orthogonal frequency division multiplexing (OFDM) while achieving the CRLB of location estimation. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Fusing Pretrained ViTs with TCNet for Enhanced EEG Regression

Authors: Eric Modesitt, Haicheng Yin, Williams Huang Wang, Brian Lu

Abstract: The task of Electroencephalogram (EEG) analysis is paramount to the development of Brain-Computer Interfaces (BCIs). However, to reach the goal of developing robust, useful BCIs depends heavily on the speed and the accuracy at which BCIs can understand neural dynamics. In response to that goal, this paper details the integration of pre-trained Vision Transformers (ViTs) with Temporal Convolutional… ▽ More The task of Electroencephalogram (EEG) analysis is paramount to the development of Brain-Computer Interfaces (BCIs). However, to reach the goal of developing robust, useful BCIs depends heavily on the speed and the accuracy at which BCIs can understand neural dynamics. In response to that goal, this paper details the integration of pre-trained Vision Transformers (ViTs) with Temporal Convolutional Networks (TCNet) to enhance the precision of EEG regression. The core of this approach lies in harnessing the sequential data processing strengths of ViTs along with the superior feature extraction capabilities of TCNet, to significantly improve EEG analysis accuracy. In addition, we analyze the importance of how to construct optimal patches for the attention mechanism to analyze, balancing both speed and accuracy tradeoffs. Our results showcase a substantial improvement in regression accuracy, as evidenced by the reduction of Root Mean Square Error (RMSE) from 55.4 to 51.8 on EEGEyeNet's Absolute Position Task, outperforming existing state-of-the-art models. Without sacrificing performance, we increase the speed of this model by an order of magnitude (up to 4.32x faster). This breakthrough not only sets a new benchmark in EEG regression analysis but also opens new avenues for future research in the integration of transformer architectures with specialized feature extraction methods for diverse EEG datasets. △ Less

Submitted 7 August, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Comments: Accepted HCI International 2024

GI-Free Pilot-Aided Channel Estimation for Affine Frequency Division Multiplexing Systems

Authors: Yu Zhou, Haoran Yin, Nanhao Zhou, Yanqun Tang, Xiaoying Zhang, Weijie Yuan

Abstract: The recently developed affine frequency division multiplexing (AFDM) can achieve full diversity in doubly selective channels, providing a comprehensive sparse representation of the delay-Doppler domain channel. Thus, accurate channel estimation is feasible by using just one pilot symbol. However, traditional AFDM channel estimation schemes necessitate the use of guard intervals (GI) to mitigate da… ▽ More The recently developed affine frequency division multiplexing (AFDM) can achieve full diversity in doubly selective channels, providing a comprehensive sparse representation of the delay-Doppler domain channel. Thus, accurate channel estimation is feasible by using just one pilot symbol. However, traditional AFDM channel estimation schemes necessitate the use of guard intervals (GI) to mitigate data-pilot interference, leading to spectral efficiency degradation. In this paper, we propose a GI-free pilot-aided channel estimation algorithm for AFDM systems, which improves spectral efficiency significantly. To mitigate the interference between the pilot and data symbols caused by the absence of GI, we perform joint interference cancellation, channel estimation, and signal detection iterately. Simulation results show that the bit error rate (BER) performance of the proposed method can approach the ideal case with perfect channel estimation. △ Less

Submitted 1 April, 2024; originally announced April 2024.

Modeling Analog Dynamic Range Compressors using Deep Learning and State-space Models

Authors: Hanzhi Yin, Gang Cheng, Christian J. Steinmetz, Ruibin Yuan, Richard M. Stern, Roger B. Dannenberg

Abstract: We describe a novel approach for developing realistic digital models of dynamic range compressors for digital audio production by analyzing their analog prototypes. While realistic digital dynamic compressors are potentially useful for many applications, the design process is challenging because the compressors operate nonlinearly over long time scales. Our approach is based on the structured stat… ▽ More We describe a novel approach for developing realistic digital models of dynamic range compressors for digital audio production by analyzing their analog prototypes. While realistic digital dynamic compressors are potentially useful for many applications, the design process is challenging because the compressors operate nonlinearly over long time scales. Our approach is based on the structured state space sequence model (S4), as implementing the state-space model (SSM) has proven to be efficient at learning long-range dependencies and is promising for modeling dynamic range compressors. We present in this paper a deep learning model with S4 layers to model the Teletronix LA-2A analog dynamic range compressor. The model is causal, executes efficiently in real time, and achieves roughly the same quality as previous deep-learning models but with fewer parameters. △ Less

Submitted 24 March, 2024; originally announced March 2024.

LM2D: Lyrics- and Music-Driven Dance Synthesis

Authors: Wenjie Yin, Xuejiao Zhao, Yi Yu, Hang Yin, Danica Kragic, Mårten Björkman

Abstract: Dance typically involves professional choreography with complex movements that follow a musical rhythm and can also be influenced by lyrical content. The integration of lyrics in addition to the auditory dimension, enriches the foundational tone and makes motion generation more amenable to its semantic meanings. However, existing dance synthesis methods tend to model motions only conditioned on au… ▽ More Dance typically involves professional choreography with complex movements that follow a musical rhythm and can also be influenced by lyrical content. The integration of lyrics in addition to the auditory dimension, enriches the foundational tone and makes motion generation more amenable to its semantic meanings. However, existing dance synthesis methods tend to model motions only conditioned on audio signals. In this work, we make two contributions to bridge this gap. First, we propose LM2D, a novel probabilistic architecture that incorporates a multimodal diffusion model with consistency distillation, designed to create dance conditioned on both music and lyrics in one diffusion generation step. Second, we introduce the first 3D dance-motion dataset that encompasses both music and lyrics, obtained with pose estimation technologies. We evaluate our model against music-only baseline models with objective metrics and human evaluations, including dancers and choreographers. The results demonstrate LM2D is able to produce realistic and diverse dance matching both lyrics and music. A video summary can be accessed at: https://youtu.be/4XCgvYookvA. △ Less

Submitted 14 March, 2024; originally announced March 2024.

Description on IEEE ICME 2024 Grand Challenge: Semi-supervised Acoustic Scene Classification under Domain Shift

Authors: Jisheng Bai, Mou Wang, Haohe Liu, Han Yin, Yafei Jia, Siwei Huang, Yutong Du, Dongzhe Zhang, Dongyuan Shi, Woon-Seng Gan, Mark D. Plumbley, Susanto Rahardja, Bin Xiang, Jianfeng Chen

Abstract: Acoustic scene classification (ASC) is a crucial research problem in computational auditory scene analysis, and it aims to recognize the unique acoustic characteristics of an environment. One of the challenges of the ASC task is the domain shift between training and testing data. Since 2018, ASC challenges have focused on the generalization of ASC models across different recording devices. Althoug… ▽ More Acoustic scene classification (ASC) is a crucial research problem in computational auditory scene analysis, and it aims to recognize the unique acoustic characteristics of an environment. One of the challenges of the ASC task is the domain shift between training and testing data. Since 2018, ASC challenges have focused on the generalization of ASC models across different recording devices. Although this task, in recent years, has achieved substantial progress in device generalization, the challenge of domain shift between different geographical regions, involving discrepancies such as time, space, culture, and language, remains insufficiently explored at present. In addition, considering the abundance of unlabeled acoustic scene data in the real world, it is important to study the possible ways to utilize these unlabelled data. Therefore, we introduce the task Semi-supervised Acoustic Scene Classification under Domain Shift in the ICME 2024 Grand Challenge. We encourage participants to innovate with semi-supervised learning techniques, aiming to develop more robust ASC models under domain shift. △ Less

Submitted 28 February, 2024; v1 submitted 4 February, 2024; originally announced February 2024.

Finite-Precision Arithmetic Transceiver for Massive MIMO Systems

Authors: Yiming Fang, Li Chen, Yunfei Chen, Huarui Yin

Abstract: Efficient implementation of massive multiple-input-multiple-output (MIMO) transceivers is essential for the next-generation wireless networks. To reduce the high computational complexity of the massive MIMO transceiver, in this paper, we propose a new massive MIMO architecture using finite-precision arithmetic. First, we conduct the rounding error analysis and derive the lower bound of the achieva… ▽ More Efficient implementation of massive multiple-input-multiple-output (MIMO) transceivers is essential for the next-generation wireless networks. To reduce the high computational complexity of the massive MIMO transceiver, in this paper, we propose a new massive MIMO architecture using finite-precision arithmetic. First, we conduct the rounding error analysis and derive the lower bound of the achievable rate for single-input-multiple-output (SIMO) using maximal ratio combining (MRC) and multiple-input-single-output (MISO) systems using maximal ratio transmission (MRT) with finite-precision arithmetic. Then, considering the multi-user scenario, the rounding error analysis of zero-forcing (ZF) detection and precoding is derived by using the normal equations (NE) method. The corresponding lower bounds of the achievable sum rate are also derived and asymptotic analyses are presented. Built upon insights from these analyses and lower bounds, we propose a mixed-precision architecture for massive MIMO systems to offset performance gaps due to finite-precision arithmetic. The corresponding analysis of rounding errors and computational costs is obtained. Simulation results validate the derived bounds and underscore the superiority of the proposed mixed-precision architecture to the conventional structure. △ Less

Submitted 25 March, 2024; v1 submitted 24 January, 2024; originally announced January 2024.

Comments: 16 pages, 8 figures. Submitted to IEEE JSAC for possible publication

Sub-band and Full-band Interactive U-Net with DPRNN for Demixing Cross-talk Stereo Music

Authors: Han Yin, Mou Wang, Jisheng Bai, Dongyuan Shi, Woon-Seng Gan, Jianfeng Chen

Abstract: This paper presents a detailed description of our proposed methods for the ICASSP 2024 Cadenza Challenge. Experimental results show that the proposed system can achieve better performance than official baselines. This paper presents a detailed description of our proposed methods for the ICASSP 2024 Cadenza Challenge. Experimental results show that the proposed system can achieve better performance than official baselines. △ Less

Submitted 10 January, 2024; originally announced January 2024.

Comments: Submitted to ICASSP 2024

Joint Channel Estimation and Data Recovery for Millimeter Massive MIMO: Using Pilot to Capture Principal Components

Authors: Shusen Cai, Li Chen, Yunfei Chen, Huarui Yin, Weidong Wang

Abstract: Channel state information (CSI) is important to reap the full benefits of millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems. The traditional channel estimation methods using pilot frames (PF) lead to excessive overhead. To reduce the demand for PF, data frames (DF) can be adopted for joint channel estimation and data recovery. However, the computational complexity of t… ▽ More Channel state information (CSI) is important to reap the full benefits of millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems. The traditional channel estimation methods using pilot frames (PF) lead to excessive overhead. To reduce the demand for PF, data frames (DF) can be adopted for joint channel estimation and data recovery. However, the computational complexity of the DF-based methods is prohibitively high. To reduce the computational complexity, we propose a joint channel estimation and data recovery (JCD) method assisted by a small number of PF for mmWave massive MIMO systems. The proposed method has two stages. In Stage 1, differing from the traditional PF-based methods, the proposed PF-assisted method is utilized to capture the angle of arrival (AoA) of principal components (PC) of channels. In Stage 2, JCD is designed for parallel implementation based on the multi-user decoupling strategy. The theoretical analysis demonstrates that the PF-assisted JCD method can achieve equivalent performance to the Bayesian-optimal DF-based method, while greatly reducing the computational complexity. Simulation results are also presented to validate the analytical results. △ Less

Submitted 3 January, 2024; originally announced January 2024.

Comments: 16 pages,11 figures,submitted to IEEE transactions on communications

A Low-Complexity Range Estimation with Adjusted Affine Frequency Division Multiplexing Waveform

Authors: Jiajun Zhu, Yanqun Tang, Xizhang Wei, Haoran Yin, Jinming Du, Zhengpeng Wang, Yuqinng Liu

Abstract: Affine frequency division multiplexing (AFDM) is a recently proposed communication waveform for time-varying channel scenarios. As a chirp-based multicarrier modulation technique it can not only satisfy the needs of multiple scenarios in future mobile communication networks but also achieve good performance in radar sensing by adjusting the built-in parameters, making it a promising air interface… ▽ More Affine frequency division multiplexing (AFDM) is a recently proposed communication waveform for time-varying channel scenarios. As a chirp-based multicarrier modulation technique it can not only satisfy the needs of multiple scenarios in future mobile communication networks but also achieve good performance in radar sensing by adjusting the built-in parameters, making it a promising air interface waveform in integrated sensing and communication (ISAC) applications. In this paper, we investigate an AFDM-based radar system and analyze the radar ambiguity function of AFDM with different built-in parameters, based on which we find an AFDM waveform with the specific parameter c2 owns the near-optimal time-domain ambiguity function. Then a low-complexity algorithm based on matched filtering for high-resolution target range estimation is proposed for this specific AFDM waveform. Through simulation and analysis, the specific AFDM waveform has near-optimal range estimation performance with the proposed low-complexity algorithm while having the same bit error rate (BER) performance as orthogonal time frequency space (OTFS) using simple linear minimum mean square error (LMMSE) equalizer. △ Less

Submitted 29 December, 2023; v1 submitted 18 December, 2023; originally announced December 2023.

Comments: The paper has been submitted to IEEE WCNC 2024 WS-13: Mobile Sensing-Communication-Computation Synergy for 6G Internet of Things

Output contraction analysis of nonlinear systems

Authors: Hao Yin, Bayu Jayawardhana, Stephan Trenn

Abstract: This paper introduce the notion of output contraction that expands the contraction notion to the time-varying nonlinear systems with output. It pertains to the systems' property that any pair of outputs from the system converge to each other exponentially. This concept exhibits a more expansive nature when contrasted with another generalized contraction framework known as partial contraction. The… ▽ More This paper introduce the notion of output contraction that expands the contraction notion to the time-varying nonlinear systems with output. It pertains to the systems' property that any pair of outputs from the system converge to each other exponentially. This concept exhibits a more expansive nature when contrasted with another generalized contraction framework known as partial contraction. The first result establishes a connection between the output contraction of a time-varying system and the output exponential stability of its variational system. Subsequently, we derive a sufficient condition for achieving output contraction in time-varying systems by applying the output contraction Lyapunov criterion. Finally, we apply the results to analyze the output exponential stability of nonlinear time-invariant systems. △ Less

Submitted 11 December, 2023; originally announced December 2023.

Contraction analysis of time-varying DAE systems via auxiliary ODE systems

Authors: Hao Yin, Bayu Jayawardhana, Stephan Trenn

Abstract: This paper studies the contraction property of time-varying differential-algebraic equation (DAE) systems by embedding them to higher-dimension ordinary differential equation (ODE) systems. The first result pertains to the equivalence of the contraction of a DAE system and the uniform global exponential stability (UGES) of its variational DAE system. Such equivalence inherits the well-known proper… ▽ More This paper studies the contraction property of time-varying differential-algebraic equation (DAE) systems by embedding them to higher-dimension ordinary differential equation (ODE) systems. The first result pertains to the equivalence of the contraction of a DAE system and the uniform global exponential stability (UGES) of its variational DAE system. Such equivalence inherits the well-known property of contracting ODE systems on a specific manifold. Subsequently, we construct an auxiliary ODE system from a DAE system whose trajectories encapsulate those of the corresponding variational DAE system. Using the auxiliary ODE system, a sufficient condition for contraction of the time-varying DAE system is established by using matrix measure which allows us to estimate an lower bound on the parameters of the auxiliary system. Finally, we apply the results to analyze the stability of time-invariant DAE systems, and to design observers for time-varying ODE systems. △ Less

Submitted 11 December, 2023; originally announced December 2023.

Error Performance of Coded AFDM Systems in Doubly Selective Channels

Authors: Haoran Yin

Abstract: Affine frequency division multiplexing (AFDM) is a strong candidate for the sixth-generation wireless network thanks to its strong resilience to delay-Doppler spreads. In this letter, we investigate the error performance of coded AFDM systems in doubly selective channels. We first study the conditional pairwise-error probability (PEP) of AFDM system and derive its conditional coding gain. Then, we… ▽ More Affine frequency division multiplexing (AFDM) is a strong candidate for the sixth-generation wireless network thanks to its strong resilience to delay-Doppler spreads. In this letter, we investigate the error performance of coded AFDM systems in doubly selective channels. We first study the conditional pairwise-error probability (PEP) of AFDM system and derive its conditional coding gain. Then, we show that there is a fundamental trade-off between the diversity gain and the coding gain of AFDM system, namely the coding gain declines with a descending speed with respect to the number of separable paths, while the diversity gain increases linearly. Moreover, we propose a near-optimal turbo decoder based on the sum-product algorithm for coded AFDM systems to improve its error performance. Simulation results verify our analyses and the effectiveness of the proposed turbo decoder, showing that AFDM outperforms orthogonal frequency division multiplexing (OFDM) and orthogonal time frequency space (OTFS) in both coded and uncoded cases over high-mobility channels. △ Less

Submitted 27 November, 2023; originally announced November 2023.

Interactive Dual-Conformer with Scene-Inspired Mask for Soft Sound Event Detection

Authors: Han Yin, Jisheng Bai, Mou Wang, Dongyuan Shi, Woon-Seng Gan, Jianfeng Chen

Abstract: Traditional binary hard labels for sound event detection (SED) lack details about the complexity and variability of sound event distributions. Recently, a novel annotation workflow is proposed to generate fine-grained non-binary soft labels, resulting in a new real-life dataset named MAESTRO Real for SED. In this paper, we first propose an interactive dual-conformer (IDC) module, in which a cross-… ▽ More Traditional binary hard labels for sound event detection (SED) lack details about the complexity and variability of sound event distributions. Recently, a novel annotation workflow is proposed to generate fine-grained non-binary soft labels, resulting in a new real-life dataset named MAESTRO Real for SED. In this paper, we first propose an interactive dual-conformer (IDC) module, in which a cross-interaction mechanism is applied to effectively exploit the information from soft labels. In addition, a novel scene-inspired mask (SIM) based on soft labels is incorporated for more precise SED predictions. The SIM is initially generated through a statistical approach, referred as SIM-V1. However, the fixed artificial mask may mismatch the SED model, resulting in limited effectiveness. Therefore, we further propose SIM-V2, which employs a word embedding model for adaptive SIM estimation. Experimental results show that the proposed IDC module can effectively utilize the information from soft labels, and the integration of SIM-V1 can further improve the accuracy. In addition, the impact of different word embedding dimensions on SIM-V2 is explored, and the results show that the appropriate dimension can enable SIM-V2 achieve superior performance than SIM-V1. In DCASE 2023 Challenge Task4B, the proposed system achieved the top ranking performance on the evaluation dataset of MAESTRO Real. △ Less

Submitted 7 December, 2023; v1 submitted 23 November, 2023; originally announced November 2023.

Comments: to be improved (unfinished)

AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learning

Authors: Jisheng Bai, Han Yin, Mou Wang, Dongyuan Shi, Woon-Seng Gan, Jianfeng Chen, Susanto Rahardja

Abstract: Previous studies in automated audio captioning have faced difficulties in accurately capturing the complete temporal details of acoustic scenes and events within long audio sequences. This paper presents AudioLog, a large language models (LLMs)-powered audio logging system with hybrid token-semantic contrastive learning. Specifically, we propose to fine-tune the pre-trained hierarchical token-sema… ▽ More Previous studies in automated audio captioning have faced difficulties in accurately capturing the complete temporal details of acoustic scenes and events within long audio sequences. This paper presents AudioLog, a large language models (LLMs)-powered audio logging system with hybrid token-semantic contrastive learning. Specifically, we propose to fine-tune the pre-trained hierarchical token-semantic audio Transformer by incorporating contrastive learning between hybrid acoustic representations. We then leverage LLMs to generate audio logs that summarize textual descriptions of the acoustic environment. Finally, we evaluate the AudioLog system on two datasets with both scene and event annotations. Experiments show that the proposed system achieves exceptional performance in acoustic scene classification and sound event detection, surpassing existing methods in the field. Further analysis of the prompts to LLMs demonstrates that AudioLog can effectively summarize long audio sequences. To the best of our knowledge, this approach is the first attempt to leverage LLMs for summarizing long audio sequences. △ Less

Submitted 4 January, 2024; v1 submitted 21 November, 2023; originally announced November 2023.

PromptSpeaker: Speaker Generation Based on Text Descriptions

Authors: Yongmao Zhang, Guanghou Liu, Yi Lei, Yunlin Chen, Hao Yin, Lei Xie, Zhifei Li

Abstract: Recently, text-guided content generation has received extensive attention. In this work, we explore the possibility of text description-based speaker generation, i.e., using text prompts to control the speaker generation process. Specifically, we propose PromptSpeaker, a text-guided speaker generation system. PromptSpeaker consists of a prompt encoder, a zero-shot VITS, and a Glow model, where the… ▽ More Recently, text-guided content generation has received extensive attention. In this work, we explore the possibility of text description-based speaker generation, i.e., using text prompts to control the speaker generation process. Specifically, we propose PromptSpeaker, a text-guided speaker generation system. PromptSpeaker consists of a prompt encoder, a zero-shot VITS, and a Glow model, where the prompt encoder predicts a prior distribution based on the text description and samples from this distribution to obtain a semantic representation. The Glow model subsequently converts the semantic representation into a speaker representation, and the zero-shot VITS finally synthesizes the speaker's voice based on the speaker representation. We verify that PromptSpeaker can generate speakers new from the training set by objective metrics, and the synthetic speaker voice has reasonable subjective matching quality with the speaker prompt. △ Less

Submitted 8 October, 2023; originally announced October 2023.

Comments: Accepted to ASRU 2023

Music- and Lyrics-driven Dance Synthesis

Authors: Wenjie Yin, Qingyuan Yao, Yi Yu, Hang Yin, Danica Kragic, Mårten Björkman

Abstract: Lyrics often convey information about the songs that are beyond the auditory dimension, enriching the semantic meaning of movements and musical themes. Such insights are important in the dance choreography domain. However, most existing dance synthesis methods mainly focus on music-to-dance generation, without considering the semantic information. To complement it, we introduce JustLMD, a new mult… ▽ More Lyrics often convey information about the songs that are beyond the auditory dimension, enriching the semantic meaning of movements and musical themes. Such insights are important in the dance choreography domain. However, most existing dance synthesis methods mainly focus on music-to-dance generation, without considering the semantic information. To complement it, we introduce JustLMD, a new multimodal dataset of 3D dance motion with music and lyrics. To the best of our knowledge, this is the first dataset with triplet information including dance motion, music, and lyrics. Additionally, we showcase a cross-modal diffusion-based network designed to generate 3D dance motion conditioned on music and lyrics. The proposed JustLMD dataset encompasses 4.6 hours of 3D dance motion in 1867 sequences, accompanied by musical tracks and their corresponding English lyrics. △ Less

Submitted 30 September, 2023; originally announced October 2023.

Multi-user passive beamforming in RIS-aided communications and experimental validations

Authors: Zhibo Zhou, Haifan Yin, Li Tan, Ruikun Zhang, Kai Wang, Yingzhuang Liu

Abstract: Reconfigurable intelligent surface (RIS) is a promising technology for future wireless communications due to its capability of optimizing the propagation environments. Nevertheless, in literature, there are few prototypes serving multiple users. In this paper, we propose a whole flow of channel estimation and beamforming design for RIS, and set up an RIS-aided multi-user system for experimental va… ▽ More Reconfigurable intelligent surface (RIS) is a promising technology for future wireless communications due to its capability of optimizing the propagation environments. Nevertheless, in literature, there are few prototypes serving multiple users. In this paper, we propose a whole flow of channel estimation and beamforming design for RIS, and set up an RIS-aided multi-user system for experimental validations. Specifically, we combine a channel sparsification step with generalized approximate message passing (GAMP) algorithm, and propose to generate the measurement matrix as Rademacher distribution to obtain the channel state information (CSI). To generate the reflection coefficients with the aim of maximizing the spectral efficiency, we propose a quadratic transform-based low-rank multi-user beamforming (QTLM) algorithm. Our proposed algorithms exploit the sparsity and low-rank properties of the channel, which has the advantages of light calculation and fast convergence. Based on the universal software radio peripheral devices, we built a complete testbed working at 5.8GHz and implemented all the proposed algorithms to verify the possibility of RIS assisting multi-user systems. Experimental results show that the system has obtained an average spectral efficiency increase of 13.48bps/Hz, with respective received power gains of 26.6dB and 17.5dB for two users, compared with the case when RIS is powered-off. △ Less

Submitted 11 May, 2024; v1 submitted 17 September, 2023; originally announced September 2023.

Comments: 11 pages, 8 figures, 2 tables. This paper has been accepted by IEEE Transactions on Communications

Low-complexity eigenvector prediction-based precoding matrix prediction in massive MIMO with mobility

Authors: Ziao Qin, Haifan Yin, Weidong Li

Abstract: In practical massive multiple-input multiple-output (MIMO) systems, the precoding matrix is often obtained from the eigenvectors of channel matrices and is challenging to update in time due to finite computation resources at the base station, especially in mobile scenarios. In order to reduce the precoding complexity while enhancing the spectral efficiency (SE), a novel precoding matrix prediction… ▽ More In practical massive multiple-input multiple-output (MIMO) systems, the precoding matrix is often obtained from the eigenvectors of channel matrices and is challenging to update in time due to finite computation resources at the base station, especially in mobile scenarios. In order to reduce the precoding complexity while enhancing the spectral efficiency (SE), a novel precoding matrix prediction method based on the eigenvector prediction (EGVP) is proposed. The basic idea is to decompose the periodic uplink channel eigenvector samples into a linear combination of the channel state information (CSI) and channel weights. We further prove that the channel weights can be interpolated by an exponential model corresponding to the Doppler characteristics of the CSI. A fast matrix pencil prediction (FMPP) method is also devised to predict the CSI. We also prove that our scheme achieves asymptotically error-free precoder prediction with a distinct complexity advantage. Simulation results show that under the perfect non-delayed CSI, the proposed EGVP method reduces floating point operations by 80\% without losing SE performance compared to the traditional full-time precoding scheme. In more realistic cases with CSI delays, the proposed EGVP-FMPP scheme has clear SE performance gains compared to the precoding scheme widely used in current communication systems. △ Less

Submitted 30 June, 2024; v1 submitted 24 August, 2023; originally announced August 2023.

Comments: 13pages, 8 figures, 1 table, journal

Channel sensing for holographic interference surfaces based on the principle of interferometry

Authors: Jindiao Huang, Yuyao Wu, Haifan Yin, Yuhao Zhang, Ruikun Zhang

Abstract: The Holographic Interference Surface (HIS) provides a new paradigm for building a more cost-effective wireless communication architecture. In this paper, we derive the principles of holographic interference theory for electromagnetic wave reception and transmission, whereby the optical holography is extended to communication holography and a channel sensing architecture for holographic interferenc… ▽ More The Holographic Interference Surface (HIS) provides a new paradigm for building a more cost-effective wireless communication architecture. In this paper, we derive the principles of holographic interference theory for electromagnetic wave reception and transmission, whereby the optical holography is extended to communication holography and a channel sensing architecture for holographic interference surfaces is established. Unlike the traditional pilot-based channel estimation approaches, the proposed architecture circumvents the complicated processes like filtering, analog to digital conversion (ADC), down conversion. Instead, it relies on interfering the object waves with a pre-designed reference wave, and therefore reduces the hardware complexity and requires less time-frequency resources for channel estimation. To address the self-interference problem in the holographic recording process, we propose a phase shifting-based interference suppression (PSIS) method according to the structural characteristics of communication hologram and interference composition. We then propose a Prony-based multi-user channel segmentation (PMCS) algorithm to acquire the channel state information (CSI). Our theoretical analysis shows that the estimation error of the PMCS algorithm converges to zero when the number of HIS units is large enough. Simulation results show that under the holographic architecture, our proposed algorithm can accurately estimate the CSI in multi-user scenarios. △ Less

Submitted 18 December, 2023; v1 submitted 20 August, 2023; originally announced August 2023.

Prototyping and real-world field trials of RIS-aided wireless communications

Authors: Xilong Pei, Haifan Yin, Li Tan, Lin Cao, Taorui Yang

Abstract: Reconfigurable intelligent surface (RIS) is a promising technology that has the potential to change the way we interact with the wireless propagating environment. In this paper, we design and fabricate an RIS system that can be used in the fifth generation (5G) mobile communication networks. We also propose a practical two-step spatial-oversampling codebook algorithm for the beamforming of RIS, wh… ▽ More Reconfigurable intelligent surface (RIS) is a promising technology that has the potential to change the way we interact with the wireless propagating environment. In this paper, we design and fabricate an RIS system that can be used in the fifth generation (5G) mobile communication networks. We also propose a practical two-step spatial-oversampling codebook algorithm for the beamforming of RIS, which is based on the spatial structure of the wireless channel. This algorithm has much lower complexity compared to the two-dimensional full-space searching-based codebook, yet with only negligible performance loss. Then, a series of experiments are conducted with the fabricated RIS systems, covering the office, corridor, and outdoor environments, in order to verified the effectiveness of RIS in both laboratory and current 5G commercial networks. In the office and corridor scenarios, the 5.8 GHz RIS provided a 10-20 dB power gain at the receiver. In the outdoor test, over 35 dB power gain was observed with RIS compared to the non-deployment case. However, in commercial 5G networks, the 2.6 GHz RIS improved indoor signal strength by only 4-7 dB. The experimental results indicate that RIS achieves higher power gain when transceivers are equipped with directional antennas instead of omni-directional antennas. △ Less

Submitted 6 August, 2023; originally announced August 2023.

Comments: 10 pages, 21 figures

Leveraging Optical Communication Fiber and AI for Distributed Water Pipe Leak Detection

Authors: Huan Wu, Huan-Feng Duan, Wallace W. L. Lai, Kun Zhu, Xin Cheng, Hao Yin, Bin Zhou, Chun-Cheung Lai, Chao Lu, Xiaoli Ding

Abstract: Detecting leaks in water networks is a costly challenge. This article introduces a practical solution: the integration of optical network with water networks for efficient leak detection. Our approach uses a fiber-optic cable to measure vibrations, enabling accurate leak identification and localization by an intelligent algorithm. We also propose a method to access leak severity for prioritized re… ▽ More Detecting leaks in water networks is a costly challenge. This article introduces a practical solution: the integration of optical network with water networks for efficient leak detection. Our approach uses a fiber-optic cable to measure vibrations, enabling accurate leak identification and localization by an intelligent algorithm. We also propose a method to access leak severity for prioritized repairs. Our solution detects even small leaks with flow rates as low as 0.027 L/s. It offers a cost-effective way to improve leak detection, enhance water management, and increase operational efficiency. △ Less

Submitted 28 July, 2023; originally announced July 2023.

Comments: Accepted

Journal ref: IEEE Communications Magazine, 2023

A Phase-Coded Time-Domain Interleaved OTFS Waveform with Improved Ambiguity Function

Authors: Jiajun Zhu, Yanqun Tang, Chao Yang, Chi Zhang, Haoran Yin, Jiaojiao Xiong, Yuhua Chen

Abstract: Integrated sensing and communication (ISAC) is a significant application scenario in future wireless communication networks, and sensing capability of a waveform is always evaluated by the ambiguity function. To enhance the sensing performance of the orthogonal time frequency space (OTFS) waveform, we propose a novel time-domain interleaved cyclic-shifted P4-coded OTFS (TICP4-OTFS) with improved a… ▽ More Integrated sensing and communication (ISAC) is a significant application scenario in future wireless communication networks, and sensing capability of a waveform is always evaluated by the ambiguity function. To enhance the sensing performance of the orthogonal time frequency space (OTFS) waveform, we propose a novel time-domain interleaved cyclic-shifted P4-coded OTFS (TICP4-OTFS) with improved ambiguity function. TICP4-OTFS can achieve superior autocorrelation features in both the time and frequency domains by exploiting the multicarrier-like form of OTFS after interleaved and the favorable autocorrelation attributes of the P4 code. Furthermore, we present the vectorized formulation of TICP4-OTFS modulation as well as its signal structure in each domain. Numerical simulations show that our proposed TICP4-OTFS waveform outperforms OTFS with a narrower mainlobe as well as lower and more distant sidelobes in terms of delay and Doppler-dimensional ambiguity functions, and an instance of range estimation using pulse compression is illustrated to exhibit the proposed waveform\u2019s greater resolution. Besides, TICP4-OTFS achieves better performance of bit error rate for communication in low signal-to-noise ratio (SNR) scenarios. △ Less

Submitted 23 September, 2023; v1 submitted 26 July, 2023; originally announced July 2023.

Comments: This paper has been accepted by 2023 IEEE Globecom Workshops (GC Wkshps): Workshop on Integrated Sensing and Communications for Internet of Things

NTIRE 2023 Quality Assessment of Video Enhancement Challenge

Authors: Xiaohong Liu, Xiongkuo Min, Wei Sun, Yulun Zhang, Kai Zhang, Radu Timofte, Guangtao Zhai, Yixuan Gao, Yuqin Cao, Tengchuan Kou, Yunlong Dong, Ziheng Jia, Yilin Li, Wei Wu, Shuming Hu, Sibin Deng, Pengxiang Xiao, Ying Chen, Kai Li, Kai Zhao, Kun Yuan, Ming Sun, Heng Cong, Hao Wang, Lingzhi Fu , et al. (47 additional authors not shown)

Abstract: This paper reports on the NTIRE 2023 Quality Assessment of Video Enhancement Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2023. This challenge is to address a major challenge in the field of video processing, namely, video quality assessment (VQA) for enhanced videos. The challenge uses the VQA Dataset for Perceptual… ▽ More This paper reports on the NTIRE 2023 Quality Assessment of Video Enhancement Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2023. This challenge is to address a major challenge in the field of video processing, namely, video quality assessment (VQA) for enhanced videos. The challenge uses the VQA Dataset for Perceptual Video Enhancement (VDPVE), which has a total of 1211 enhanced videos, including 600 videos with color, brightness, and contrast enhancements, 310 videos with deblurring, and 301 deshaked videos. The challenge has a total of 167 registered participants. 61 participating teams submitted their prediction results during the development phase, with a total of 3168 submissions. A total of 176 submissions were submitted by 37 participating teams during the final testing phase. Finally, 19 participating teams submitted their models and fact sheets, and detailed the methods they used. Some methods have achieved better results than baseline methods, and the winning methods have demonstrated superior prediction performance. △ Less

Submitted 18 July, 2023; originally announced July 2023.

Superdirectivity-enhanced wireless communications: A multi-user perspective

Authors: Liangcheng Han, Haifan Yin

Abstract: Superdirective array may achieve an array gain proportional to the square of the number of antennas $M^2$. In the early studies of superdirectivity, little research has been done from wireless communication point of view. To leverage superdirectivity for enhancing the spectral efficiency, this paper investigates multi-user communication systems with superdirective arrays. We first propose a field-… ▽ More Superdirective array may achieve an array gain proportional to the square of the number of antennas $M^2$. In the early studies of superdirectivity, little research has been done from wireless communication point of view. To leverage superdirectivity for enhancing the spectral efficiency, this paper investigates multi-user communication systems with superdirective arrays. We first propose a field-coupling-aware (FCA) multi-user channel estimation method, which takes into account the antenna coupling effects. Aiming to maximize the power gain of the target user, we propose multi-user multipath superdirective precoding (SP) as an extension of our prior work on coupling-based superdirective beamforming. Furthermore, to reduce the inter-user interference, we propose interference-nulling superdirective precoding (INSP) as the optimal solution to maximize user power gains while eliminating interference. Then, by taking the ohmic loss into consideration, we further propose a regularized interference-nulling superdirective precoding (RINSP) method. Finally, we discuss the well-known narrow directivity bandwidth issue, and find that it is not a fundamental problem of superdirective arrays in multi-carrier communication systems. Simulation results show our proposed methods outperform the state-of-the-art methods significantly. Interestingly, in the multi-user scenario, an 18-antenna superdirective array can achieve up to a 9-fold increase of spectral efficiency compared to traditional multiple-input multiple-output (MIMO), while simultaneously reducing the array aperture by half. △ Less

Submitted 12 July, 2023; originally announced July 2023.

Comments: 11 pages, 8 figures

On the Effectiveness of Speech Self-supervised Learning for Music

Authors: Yinghao Ma, Ruibin Yuan, Yizhi Li, Ge Zhang, Xingran Chen, Hanzhi Yin, Chenghua Lin, Emmanouil Benetos, Anton Ragni, Norbert Gyenge, Ruibo Liu, Gus Xia, Roger Dannenberg, Yike Guo, Jie Fu

Abstract: Self-supervised learning (SSL) has shown promising results in various speech and natural language processing applications. However, its efficacy in music information retrieval (MIR) still remains largely unexplored. While previous SSL models pre-trained on music recordings may have been mostly closed-sourced, recent speech models such as wav2vec2.0 have shown promise in music modelling. Neverthele… ▽ More Self-supervised learning (SSL) has shown promising results in various speech and natural language processing applications. However, its efficacy in music information retrieval (MIR) still remains largely unexplored. While previous SSL models pre-trained on music recordings may have been mostly closed-sourced, recent speech models such as wav2vec2.0 have shown promise in music modelling. Nevertheless, research exploring the effectiveness of applying speech SSL models to music recordings has been limited. We explore the music adaption of SSL with two distinctive speech-related models, data2vec1.0 and Hubert, and refer to them as music2vec and musicHuBERT, respectively. We train $12$ SSL models with 95M parameters under various pre-training configurations and systematically evaluate the MIR task performances with 13 different MIR tasks. Our findings suggest that training with music data can generally improve performance on MIR tasks, even when models are trained using paradigms designed for speech. However, we identify the limitations of such existing speech-oriented designs, especially in modelling polyphonic information. Based on the experimental results, empirical suggestions are also given for designing future musical SSL strategies and paradigms. △ Less

Submitted 11 July, 2023; originally announced July 2023.

RIS with insufficient phase shifting capability: Modeling, beamforming, and experimental validations

Authors: Lin Cao, Haifan Yin, Li Tan, Xilong Pei

Abstract: Most research works on reconfigurable intelligent surfaces (RIS) rely on idealized models of the reflection coefficients, i.e., uniform reflection amplitude for any phase and sufficient phase shifting capability. In practice however, such models are oversimplified. This paper introduces a realistic reflection coefficient model for RIS based on measurements. The reflection coefficients are modeled… ▽ More Most research works on reconfigurable intelligent surfaces (RIS) rely on idealized models of the reflection coefficients, i.e., uniform reflection amplitude for any phase and sufficient phase shifting capability. In practice however, such models are oversimplified. This paper introduces a realistic reflection coefficient model for RIS based on measurements. The reflection coefficients are modeled as discrete complex values that have non-uniform amplitudes and suffer from insufficient phase shift capability. We then propose a group-based query algorithm that takes the imperfect coefficients into consideration while calculating the reflection coefficients. We analyze the performance of the proposed algorithm, and derive the closed-form expressions to characterize the received power of an RIS-aided wireless communication system. The performance gains of the proposed algorithm are confirmed in simulations. Furthermore, we validate the proposed theoretical results by experiments with our fabricated RIS prototype systems. The simulation and measurement results match well with the theoretical analysis. △ Less

Submitted 16 April, 2024; v1 submitted 5 July, 2023; originally announced July 2023.

Comments: 13 pages, 11 figures

A genetic algorithm based superdirective beamforming method under excitation power range constraints

Authors: Jingcheng Xie, Haifan Yin, Liangcheng Han

Abstract: The array gain of a superdirective antenna array can be proportional to the square of the number of antennas. However, the realization of the so-called superdirectivity entails accurate calculation and application of the excitations. Moreover, the excitations require a large dynamic power range, especially when the antenna spacing is smaller. In this paper, we derive the closed-form solution for t… ▽ More The array gain of a superdirective antenna array can be proportional to the square of the number of antennas. However, the realization of the so-called superdirectivity entails accurate calculation and application of the excitations. Moreover, the excitations require a large dynamic power range, especially when the antenna spacing is smaller. In this paper, we derive the closed-form solution for the beamforming vector to achieve superdirectivity. We show that the solution only relies on the data of the array electric field, which is available in measurements or simulations. In order to alleviate the high requirement of the power range, we propose a genetic algorithm based approach with a certain excitation range constraint. Full-wave electromagnetic simulations show that compared with the traditional beamforming method, our proposed method achieves greater directivity and narrower beamwidth with the given range constraints. △ Less

Submitted 5 July, 2023; originally announced July 2023.

Comments: 5 pages, 6 figures

MARBLE: Music Audio Representation Benchmark for Universal Evaluation

Authors: Ruibin Yuan, Yinghao Ma, Yizhi Li, Ge Zhang, Xingran Chen, Hanzhi Yin, Le Zhuo, Yiqi Liu, Jiawen Huang, Zeyue Tian, Binyue Deng, Ningzhi Wang, Chenghua Lin, Emmanouil Benetos, Anton Ragni, Norbert Gyenge, Roger Dannenberg, Wenhu Chen, Gus Xia, Wei Xue, Si Liu, Shi Wang, Ruibo Liu, Yike Guo, Jie Fu

Abstract: In the era of extensive intersection between art and Artificial Intelligence (AI), such as image generation and fiction co-creation, AI for music remains relatively nascent, particularly in music understanding. This is evident in the limited work on deep music representations, the scarcity of large-scale datasets, and the absence of a universal and community-driven benchmark. To address this issue… ▽ More In the era of extensive intersection between art and Artificial Intelligence (AI), such as image generation and fiction co-creation, AI for music remains relatively nascent, particularly in music understanding. This is evident in the limited work on deep music representations, the scarcity of large-scale datasets, and the absence of a universal and community-driven benchmark. To address this issue, we introduce the Music Audio Representation Benchmark for universaL Evaluation, termed MARBLE. It aims to provide a benchmark for various Music Information Retrieval (MIR) tasks by defining a comprehensive taxonomy with four hierarchy levels, including acoustic, performance, score, and high-level description. We then establish a unified protocol based on 14 tasks on 8 public-available datasets, providing a fair and standard assessment of representations of all open-sourced pre-trained models developed on music recordings as baselines. Besides, MARBLE offers an easy-to-use, extendable, and reproducible suite for the community, with a clear statement on copyright issues on datasets. Results suggest recently proposed large-scale pre-trained musical language models perform the best in most tasks, with room for further improvement. The leaderboard and toolkit repository are published at https://marble-bm.shef.ac.uk to promote future music AI research. △ Less

Submitted 23 November, 2023; v1 submitted 18 June, 2023; originally announced June 2023.

Comments: camera-ready version for NeurIPS 2023

Performance of Graph Database Management Systems as route planning solutions for different data and usage characteristics

Authors: Karin Festl, Patrick Promitzer, Daniel Watzenig, Huilin Yin

Abstract: Graph databases have grown in popularity in recent years as they are able to efficiently store and query complex relationships between data. Incidentally, navigation data and road networks can be processed, sampled or modified efficiently when stored as a graph. As a result, graph databases are a solution for solving route planning tasks that comes more and more to the attention of developers of a… ▽ More Graph databases have grown in popularity in recent years as they are able to efficiently store and query complex relationships between data. Incidentally, navigation data and road networks can be processed, sampled or modified efficiently when stored as a graph. As a result, graph databases are a solution for solving route planning tasks that comes more and more to the attention of developers of autonomous vehicles. To achieve a computational performance that enables the realization of route planning on large road networks or for a great number of agents concurrently, several aspects need to be considered in the design of the solution. Based on a concrete use case for centralized route planning, we discuss the characteristics and properties of a use case that can significantly influence the computational effort or efficiency of the database management system. Subsequently we evaluate the performance of both Neo4j and ArangoDB depending on these properties. With these results, it is not only possible to choose the most suitable database system but also to improve the resulting performance by addressing relevant aspects in the design of the application. △ Less

Submitted 12 June, 2023; originally announced June 2023.

Comments: Submitted to IEEE IAVVC 2023

Convolutional Recurrent Neural Network with Attention for 3D Speech Enhancement

Authors: Han Yin, Jisheng Bai, Mou Wang, Siwei Huang, Yafei Jia, Jianfeng Chen

Abstract: 3D speech enhancement can effectively improve the auditory experience and plays a crucial role in augmented reality technology. However, traditional convolutional-based speech enhancement methods have limitations in extracting dynamic voice information. In this paper, we incorporate a dual-path recurrent neural network block into the U-Net to iteratively extract dynamic audio information in both t… ▽ More 3D speech enhancement can effectively improve the auditory experience and plays a crucial role in augmented reality technology. However, traditional convolutional-based speech enhancement methods have limitations in extracting dynamic voice information. In this paper, we incorporate a dual-path recurrent neural network block into the U-Net to iteratively extract dynamic audio information in both the time and frequency domains. And an attention mechanism is proposed to fuse the original signal, reference signal, and generated masks. Moreover, we introduce a loss function to simultaneously optimize the network in the time-frequency and time domains. Experimental results show that our system outperforms the state-of-the-art systems on the dataset of ICASSP L3DAS23 challenge. △ Less

Submitted 19 November, 2023; v1 submitted 8 June, 2023; originally announced June 2023.

Comments: Published on IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC 2023)

MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

Authors: Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghao Xiao, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge, Roger Dannenberg, Ruibo Liu, Wenhu Chen, Gus Xia, Yemin Shi, Wenhao Huang, Zili Wang, Yike Guo, Jie Fu

Abstract: Self-supervised learning (SSL) has recently emerged as a promising paradigm for training generalisable models on large-scale data in the fields of vision, text, and speech. Although SSL has been proven effective in speech and audio, its application to music audio has yet to be thoroughly explored. This is partially due to the distinctive challenges associated with modelling musical knowledge, part… ▽ More Self-supervised learning (SSL) has recently emerged as a promising paradigm for training generalisable models on large-scale data in the fields of vision, text, and speech. Although SSL has been proven effective in speech and audio, its application to music audio has yet to be thoroughly explored. This is partially due to the distinctive challenges associated with modelling musical knowledge, particularly tonal and pitched characteristics of music. To address this research gap, we propose an acoustic Music undERstanding model with large-scale self-supervised Training (MERT), which incorporates teacher models to provide pseudo labels in the masked language modelling (MLM) style acoustic pre-training. In our exploration, we identified an effective combination of teacher models, which outperforms conventional speech and audio approaches in terms of performance. This combination includes an acoustic teacher based on Residual Vector Quantisation - Variational AutoEncoder (RVQ-VAE) and a musical teacher based on the Constant-Q Transform (CQT). Furthermore, we explore a wide range of settings to overcome the instability in acoustic language model pre-training, which allows our designed paradigm to scale from 95M to 330M parameters. Experimental results indicate that our model can generalise and perform well on 14 music understanding tasks and attain state-of-the-art (SOTA) overall scores. △ Less

Submitted 22 April, 2024; v1 submitted 31 May, 2023; originally announced June 2023.

Comments: accepted by ICLR 2024

Locate and Beamform: Two-dimensional Locating All-neural Beamformer for Multi-channel Speech Separation

Authors: Yanjie Fu, Meng Ge, Honglong Wang, Nan Li, Haoran Yin, Longbiao Wang, Gaoyan Zhang, Jianwu Dang, Chengyun Deng, Fei Wang

Abstract: Recently, stunning improvements on multi-channel speech separation have been achieved by neural beamformers when direction information is available. However, most of them neglect to utilize speaker's 2-dimensional (2D) location cues contained in mixture signal, which limits the performance when two sources come from close directions. In this paper, we propose an end-to-end beamforming network for… ▽ More Recently, stunning improvements on multi-channel speech separation have been achieved by neural beamformers when direction information is available. However, most of them neglect to utilize speaker's 2-dimensional (2D) location cues contained in mixture signal, which limits the performance when two sources come from close directions. In this paper, we propose an end-to-end beamforming network for 2D location guided speech separation merely given mixture signal. It first estimates discriminable direction and 2D location cues, which imply directions the sources come from in multi views of microphones and their 2D coordinates. These cues are then integrated into location-aware neural beamformer, thus allowing accurate reconstruction of two sources' speech signals. Experiments show that our proposed model not only achieves a comprehensive decent improvement compared to baseline systems, but avoids inferior performance on spatial overlapping cases. △ Less

Submitted 2 June, 2023; v1 submitted 18 May, 2023; originally announced May 2023.

Comments: Accepted by Interspeech 2023. arXiv admin note: substantial text overlap with arXiv:2212.03401

An Overview of Resource Allocation in Integrated Sensing and Communication

Authors: Jinming Du, Yanqun Tang, Xizhang Wei, Jiaojiao Xiong, Jiajun Zhu, Haoran Yin, Chi Zhang, Haibo Chen

Abstract: Integrated sensing and communication (ISAC) is considered as a promising solution for improving spectrum efficiency and relieving wireless spectrum congestion. This paper systematically introduces the evolutionary path of ISAC technologies, then sorts out and summarizes the current research status of ISAC resource allocation. From the perspective of different integrated levels of ISAC, we introduc… ▽ More Integrated sensing and communication (ISAC) is considered as a promising solution for improving spectrum efficiency and relieving wireless spectrum congestion. This paper systematically introduces the evolutionary path of ISAC technologies, then sorts out and summarizes the current research status of ISAC resource allocation. From the perspective of different integrated levels of ISAC, we introduce and elaborate the research progress of resource allocation in different stages, namely, resource separated, orthogonal, converged, and collaborative stages. In addition, we give in-depth consideration to propose a new resource allocation framework from a multi-granularity perspective. Finally, we demonstrate the feasibility of our proposed framework with a case of full-duplex ISAC system. △ Less

Submitted 15 May, 2023; originally announced May 2023.

Comments: 6 pages,4 figures,conference

A manifold learning-based CSI feedback framework for FDD massive MIMO

Authors: Yandi Cao, Haifan Yin, Ziao Qin, Weidong Li, Weimin Wu, Mérouane Debbah

Abstract: Massive multi-input multi-output (MIMO) in Frequency Division Duplex (FDD) mode suffers from heavy feedback overhead for Channel State Information (CSI). In this paper, a novel manifold learning-based CSI feedback framework (MLCF) is proposed to reduce the feedback and improve the spectral efficiency for FDD massive MIMO. Manifold learning (ML) is an effective method for dimensionality reduction.… ▽ More Massive multi-input multi-output (MIMO) in Frequency Division Duplex (FDD) mode suffers from heavy feedback overhead for Channel State Information (CSI). In this paper, a novel manifold learning-based CSI feedback framework (MLCF) is proposed to reduce the feedback and improve the spectral efficiency for FDD massive MIMO. Manifold learning (ML) is an effective method for dimensionality reduction. However, most ML algorithms focus only on data compression, and lack the corresponding recovery methods. Moreover, the computational complexity is high when dealing with incremental data. Considering to utilize the intrinsic manifold structure where the CSI samples reside, we propose a landmark selection algorithm to describe the topological skeleton of this manifold. Based on the learned skeleton, the local patch of the incremental CSI on the manifold can be easily determined by its nearest landmarks. This motivates us to propose an incremental CSI compression and reconstruction scheme by keeping the local geometric relationships with landmarks invariant. We theoretically prove the convergence of the proposed landmark selection algorithm. Meanwhile, the upper bound on the error of approximating CSI with landmarks is derived. Simulation results under an industrial channel model of 3GPP demonstrate that the proposed MLCF outperforms existing deep learning based algorithms. △ Less

Submitted 23 August, 2024; v1 submitted 27 April, 2023; originally announced April 2023.

Comments: 14 pages, 7 figures, 2 tables, to appear in IEEE Tans.Commun

Fast QTMT Partition for VVC Intra Coding Using U-Net Framework

Authors: Zhao Zan, Leilei Huang, ShuShi Chen, Xiantao Zhang, Zhenghui Zhao, Haibing Yin, Yibo Fan

Abstract: Versatile Video Coding (VVC) has significantly increased encoding efficiency at the expense of numerous complex coding tools, particularly the flexible Quad-Tree plus Multi-type Tree (QTMT) block partition. This paper proposes a deep learning-based algorithm applied in fast QTMT partition for VVC intra coding. Our solution greatly reduces encoding time by early termination of less-likely intra pre… ▽ More Versatile Video Coding (VVC) has significantly increased encoding efficiency at the expense of numerous complex coding tools, particularly the flexible Quad-Tree plus Multi-type Tree (QTMT) block partition. This paper proposes a deep learning-based algorithm applied in fast QTMT partition for VVC intra coding. Our solution greatly reduces encoding time by early termination of less-likely intra prediction and partitions with negligible BD-BR increase. Firstly, a redesigned U-Net is recommended as the network's fundamental framework. Next, we design a Quality Parameter (QP) fusion network to regulate the effect of QPs on the partition results. Finally, we adopt a refined post-processing strategy to better balance encoding performance and complexity. Experimental results demonstrate that our solution outperforms the state-of-the-art works with a complexity reduction of 44.74% to 68.76% and a BD-BR increase of 0.60% to 2.33%. △ Less

Submitted 6 April, 2023; originally announced April 2023.

RIS-aided Wireless Communications: Can RIS Beat Metal Plate?

Authors: Jiangfeng Hu, Haifan Yin, Li Tan, Lin Cao, Xilong Pei

Abstract: Reconfigurable Intelligent Surface (RIS) has recently been regarded as a paradigm-shifting technology beyond 5G, for its flexibility on smartly adjusting the response to the impinging electromagnetic (EM) waves. Usually, RIS can be implemented by properly reconfiguring the adjustable parameters of each RIS unit to align the signal phase on the receiver side. And it is believed that the phase align… ▽ More Reconfigurable Intelligent Surface (RIS) has recently been regarded as a paradigm-shifting technology beyond 5G, for its flexibility on smartly adjusting the response to the impinging electromagnetic (EM) waves. Usually, RIS can be implemented by properly reconfiguring the adjustable parameters of each RIS unit to align the signal phase on the receiver side. And it is believed that the phase alignment can be also mechanically achieved by a metal plate with the same physical size. However, we found in the prototype experiments that, a well-rotated metal plate can only approximately perform as well as RIS under limited conditions, although its scattering efficiency is relatively higher. When it comes to the case of spherical wave impinging, RIS outperforms the metal plate even beyond the receiving near-field regions. We analyze this phenomenon with wave optics theory and propose explicit scattering models for both the metal plate and RIS in general scenarios. Finally, the models are validated by simulations and field measurements. △ Less

Submitted 6 March, 2023; originally announced March 2023.

Comments: 5 pages, 5 figures

Overview and Performance Analysis of Various Waveforms in High Mobility Scenarios

Authors: Yu Zhou, Haoran Yin, Jiaojiao Xiong, Shiyu Song, Jiajun Zhu, Jinming Du, Haibo Chen, Yanqun Tang

Abstract: In the high-mobility scenarios of next-generation wireless communication systems (beyond 5G/6G), the performance of orthogonal frequency division multiplexing (OFDM) deteriorates drastically due to the loss of orthogonality between the subcarriers caused by large Doppler frequency shifts. Various emerging waveforms have been proposed for fast time-varying channels with excellent results. In this p… ▽ More In the high-mobility scenarios of next-generation wireless communication systems (beyond 5G/6G), the performance of orthogonal frequency division multiplexing (OFDM) deteriorates drastically due to the loss of orthogonality between the subcarriers caused by large Doppler frequency shifts. Various emerging waveforms have been proposed for fast time-varying channels with excellent results. In this paper, we classify these waveforms from the perspective of their modulation domain and establish a unified framework to provide a comprehensive system structure comparison. Then we analyze bit error rate (BER) performance of each waveform in doubly selective channels. Through the discussions on their complexity and compatibility with OFDM systems, we finally give the candidate waveform suggestions. △ Less

Submitted 27 February, 2023; originally announced February 2023.

Cyclic Delay-Doppler Shift: A Simple Transmit Diversity Technique for Delay-Doppler Waveforms in Doubly Selective Channels

Authors: Haoran Yin, Jiaojiao Xiong, Yu Zhou, Chi Zhang, Di Zhang, Xizhang Wei, Yanqun Tang

Abstract: Delay-Doppler waveform design has been considered as a promising solution to achieve reliable communication under high-mobility channels for the space-air-ground-integrated networks (SAGIN). In this paper, we introduce the cyclic delay-Doppler shift (CDDS) technique for delay-Doppler waveforms to extract transmit diversity in doubly selective channels. Two simple CDDS schemes, named time-domain CD… ▽ More Delay-Doppler waveform design has been considered as a promising solution to achieve reliable communication under high-mobility channels for the space-air-ground-integrated networks (SAGIN). In this paper, we introduce the cyclic delay-Doppler shift (CDDS) technique for delay-Doppler waveforms to extract transmit diversity in doubly selective channels. Two simple CDDS schemes, named time-domain CDDS (TD-CDDS) and modulation-domain CDDS (MD-CDDS), are proposed in the setting of multiple-input multiple-output (MIMO). We demonstrate the applications of CDDS on two representative delay-Doppler waveforms, namely orthogonal time frequency space (OTFS) and affine frequency division multiplexing (AFDM), by deriving their corresponding CDDS matrices. Furthermore, we prove theoretically and experimentally that CDDS can provide OTFS and AFDM with full transmit diversity gain on most occasions. △ Less

Submitted 13 June, 2023; v1 submitted 22 February, 2023; originally announced February 2023.

A review of codebooks for CSI feedback in 5G new radio and beyond

Authors: Ziao Qin, Haifan Yin

Abstract: Codebooks have been indispensable for wireless communication standard since the first release of the Long-Term Evolution in 2009. They offer an efficient way to acquire the channel state information (CSI) for multiple antenna systems. Nowadays, a codebook is not limited to a set of pre-defined precoders, it refers to a CSI feedback framework, which is more and more sophisticated. In this paper, we… ▽ More Codebooks have been indispensable for wireless communication standard since the first release of the Long-Term Evolution in 2009. They offer an efficient way to acquire the channel state information (CSI) for multiple antenna systems. Nowadays, a codebook is not limited to a set of pre-defined precoders, it refers to a CSI feedback framework, which is more and more sophisticated. In this paper, we review the codebooks in 5G New Radio (NR) standards. The codebook timeline and the evolution trend are shown. Each codebook is elaborated with its motivation, the corresponding feedback mechanism, and the format of the precoding matrix indicator. Some insights are given to help grasp the underlying reasons and intuitions of these codebooks. Finally, we point out some unresolved challenges of the codebooks for future evolution of the standards. In general, this paper provides a comprehensive review of the codebooks in 5G NR and aims to help researchers understand the CSI feedback schemes from a standard and industrial perspective. △ Less

Submitted 13 June, 2023; v1 submitted 17 February, 2023; originally announced February 2023.

Comments: 11pages, 7 figures, 1 table, magzine review

Stress Flow Guided Non-Planar Print Trajectory Optimization for Additive Manufacturing of Anisotropic Polymers

Authors: Xavier Guidetti, Efe C. Balta, Yannick Nagel, Hang Yin, Alisa Rupenyan, John Lygeros

Abstract: When manufacturing parts using material extrusion additive manufacturing and anisotropic polymers, the mechanical properties of a manufactured component are strongly dependent on the print trajectory orientation. We conduct non-planar slicing and optimize the print trajectories to maximize the alignment between the material deposition direction and the stress flow induced by a predefined load case… ▽ More When manufacturing parts using material extrusion additive manufacturing and anisotropic polymers, the mechanical properties of a manufactured component are strongly dependent on the print trajectory orientation. We conduct non-planar slicing and optimize the print trajectories to maximize the alignment between the material deposition direction and the stress flow induced by a predefined load case. The trajectory optimization framework considers manufacturability constraints in the form of uniform layer height and line spacing. We demonstrate the method by manufacturing a load bearing mechanical bracket using a 5-axis 3D printer and a liquid crystal polymer material. The failure strength and stiffness of the optimized bracket are improved by a factor of 44 and 6 respectively when compared with conventional printing. △ Less

Submitted 12 June, 2023; v1 submitted 12 January, 2023; originally announced January 2023.

Comments: Accepted and published on Elsevier Additive Manufacturing

Journal ref: Additive Manufacturing, Volume 72, 2023, 103628

MIMO-DBnet: Multi-channel Input and Multiple Outputs DOA-aware Beamforming Network for Speech Separation

Authors: Yanjie Fu, Haoran Yin, Meng Ge, Longbiao Wang, Gaoyan Zhang, Jianwu Dang, Chengyun Deng, Fei Wang

Abstract: Recently, many deep learning based beamformers have been proposed for multi-channel speech separation. Nevertheless, most of them rely on extra cues known in advance, such as speaker feature, face image or directional information. In this paper, we propose an end-to-end beamforming network for direction guided speech separation given merely the mixture signal, namely MIMO-DBnet. Specifically, we d… ▽ More Recently, many deep learning based beamformers have been proposed for multi-channel speech separation. Nevertheless, most of them rely on extra cues known in advance, such as speaker feature, face image or directional information. In this paper, we propose an end-to-end beamforming network for direction guided speech separation given merely the mixture signal, namely MIMO-DBnet. Specifically, we design a multi-channel input and multiple outputs architecture to predict the direction-of-arrival based embeddings and beamforming weights for each source. The precisely estimated directional embedding provides quite effective spatial discrimination guidance for the neural beamformer to offset the effect of phase wrapping, thus allowing more accurate reconstruction of two sources' speech signals. Experiments show that our proposed MIMO-DBnet not only achieves a comprehensive decent improvement compared to baseline systems, but also maintain the performance on high frequency bands when phase wrapping occurs. △ Less

Submitted 6 December, 2022; originally announced December 2022.

Comments: Submitted to ICASSP 2023

MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learning

Authors: Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Chenghua Lin, Xingran Chen, Anton Ragni, Hanzhi Yin, Zhijie Hu, Haoyu He, Emmanouil Benetos, Norbert Gyenge, Ruibo Liu, Jie Fu

Abstract: The deep learning community has witnessed an exponentially growing interest in self-supervised learning (SSL). However, it still remains unexplored how to build a framework for learning useful representations of raw music waveforms in a self-supervised manner. In this work, we design Music2Vec, a framework exploring different SSL algorithmic components and tricks for music audio recordings. Our mo… ▽ More The deep learning community has witnessed an exponentially growing interest in self-supervised learning (SSL). However, it still remains unexplored how to build a framework for learning useful representations of raw music waveforms in a self-supervised manner. In this work, we design Music2Vec, a framework exploring different SSL algorithmic components and tricks for music audio recordings. Our model achieves comparable results to the state-of-the-art (SOTA) music SSL model Jukebox, despite being significantly smaller with less than 2% of parameters of the latter. The model will be released on Huggingface(Please refer to: https://huggingface.co/m-a-p/music2vec-v1) △ Less

Submitted 5 December, 2022; originally announced December 2022.