Showing 201–250 of 460 results for author: Yi, J

Convergence Rate of Accelerated Average Consensus with Local Node Memory: Optimization and Analytic Solutions

Authors: Jing-Wen Yi, Li Chai, Jingxin Zhang

Abstract: Previous researches have shown that adding local memory can accelerate the consensus. It is natural to ask questions like what is the fastest rate achievable by the $M$-tap memory acceleration, and what are the corresponding control parameters. This paper introduces a set of effective and previously unused techniques to analyze the convergence rate of accelerated consensus with $M$-tap memory of l… ▽ More Previous researches have shown that adding local memory can accelerate the consensus. It is natural to ask questions like what is the fastest rate achievable by the $M$-tap memory acceleration, and what are the corresponding control parameters. This paper introduces a set of effective and previously unused techniques to analyze the convergence rate of accelerated consensus with $M$-tap memory of local nodes and to design the control protocols. These effective techniques, including the Kharitonov stability theorem, the Routh stability criterion and the robust stability margin, have led to the following new results: 1) the direct link between the convergence rate and the control parameters; 2) explicit formulas of the optimal convergence rate and the corresponding optimal control parameters for $M \leq 2$ on a given graph; 3) the optimal worst-case convergence rate and the corresponding optimal control parameters for the memory $M \geq 1$ on a set of uncertain graphs. We show that the acceleration with the memory $M = 1$ provides the optimal convergence rate in the sense of worst-case performance. Several numerical examples are given to demonstrate the validity and performance of the theoretical results. △ Less

Submitted 10 December, 2021; v1 submitted 18 October, 2021; originally announced October 2021.

Comments: 30 pages, 2 figures

Exploring Universal Intrinsic Task Subspace via Prompt Tuning

Authors: Yujia Qin, Xiaozhi Wang, Yusheng Su, Yankai Lin, Ning Ding, Jing Yi, Weize Chen, Zhiyuan Liu, Juanzi Li, Lei Hou, Peng Li, Maosong Sun, Jie Zhou

Abstract: Why can pre-trained language models (PLMs) learn universal representations and effectively adapt to broad NLP tasks differing a lot superficially? In this work, we empirically find evidence indicating that the adaptations of PLMs to various few-shot tasks can be reparameterized as optimizing only a few free parameters in a unified low-dimensional intrinsic task subspace, which may help us understa… ▽ More Why can pre-trained language models (PLMs) learn universal representations and effectively adapt to broad NLP tasks differing a lot superficially? In this work, we empirically find evidence indicating that the adaptations of PLMs to various few-shot tasks can be reparameterized as optimizing only a few free parameters in a unified low-dimensional intrinsic task subspace, which may help us understand why PLMs could easily adapt to various NLP tasks with small-scale data. To find such a subspace and examine its universality, we propose an analysis pipeline called intrinsic prompt tuning (IPT). Specifically, we resort to the recent success of prompt tuning and decompose the soft prompts of multiple NLP tasks into the same low-dimensional nonlinear subspace, then we learn to adapt the PLM to unseen data or tasks by only tuning parameters in this subspace. In the experiments, we study diverse few-shot NLP tasks and surprisingly find that in a 250-dimensional subspace found with 100 tasks, by only tuning 250 free parameters, we can recover 97% and 83% of the full prompt tuning performance for 100 seen tasks (using different training data) and 20 unseen tasks, respectively, showing great generalization ability of the found intrinsic task subspace. Besides being an analysis tool, IPT could further help us improve the prompt tuning stability. △ Less

Submitted 25 November, 2022; v1 submitted 15 October, 2021; originally announced October 2021.

Comments: Previously accepted by Findings of ACL 2022 and Findings of EMNLP 2022

Optimal Needle Placement for Prostate Rotating-Shield Brachytherapy (RSBT)

Authors: Jirong Yi, Quentin E. Adams, Karolyn M. Hopfensperger, Ryan T. Flynn, Yusung Kim, John M. Buatti, Weiyu Xu, Xiaodong Wu

Abstract: Purpose: To present an efficient NEEdle Position Optimization (NEEPO) algorithm for prostate rotating shield brachytherapy (RSBT). With RSBT, the increased flexibility beyond conventional high-dose-rate brachytherapy (HDR-BT) due to the partially shielded radiation source has been shown by Adams et al. in 2020 to enable improved urethra sparing (23.1%), enhanced dose escalation (29.9%), or both, w… ▽ More Purpose: To present an efficient NEEdle Position Optimization (NEEPO) algorithm for prostate rotating shield brachytherapy (RSBT). With RSBT, the increased flexibility beyond conventional high-dose-rate brachytherapy (HDR-BT) due to the partially shielded radiation source has been shown by Adams et al. in 2020 to enable improved urethra sparing (23.1%), enhanced dose escalation (29.9%), or both, with 20 needles without NEEPO-optimized positions. Within this regime of improved dosimetry, we propose in this work that the benefits of RSBT can be maintained while also reducing the number of needles needed for the delivery. The goal of NEEPO is to provide the capability to further increase the dosimetric benefit of RSBT and to minimize the number of needles needed to satisfy a dosimetric goal. Methods: The NEEPO algorithm generates a needle pool for a given patient and then iteratively constructs a subset of needles from the pool based on relative needle importance as determined by total dwell times within needles. The NEEPO algorithm is based on a convex optimization formulation using a quadratic dosimetric penalty function, dwell time regularization by total variation, and a block sparsity regularization term to enable iterative removal of low-importance needles. RSBT treatment plans for 26 patients were generated using single fraction prescriptions with both dose escalation and urethra sparing goals, and compared to baseline HDR-BT treatment plans. △ Less

Submitted 14 October, 2021; originally announced October 2021.

Comments: 12 pages

Adversarial Attack across Datasets

Authors: Yunxiao Qin, Yuanhao Xiong, Jinfeng Yi, Lihong Cao, Cho-Jui Hsieh

Abstract: Existing transfer attack methods commonly assume that the attacker knows the training set (e.g., the label set, the input size) of the black-box victim models, which is usually unrealistic because in some cases the attacker cannot know this information. In this paper, we define a Generalized Transferable Attack (GTA) problem where the attacker doesn't know this information and is acquired to attac… ▽ More Existing transfer attack methods commonly assume that the attacker knows the training set (e.g., the label set, the input size) of the black-box victim models, which is usually unrealistic because in some cases the attacker cannot know this information. In this paper, we define a Generalized Transferable Attack (GTA) problem where the attacker doesn't know this information and is acquired to attack any randomly encountered images that may come from unknown datasets. To solve the GTA problem, we propose a novel Image Classification Eraser (ICE) that trains a particular attacker to erase classification information of any images from arbitrary datasets. Experiments on several datasets demonstrate that ICE greatly outperforms existing transfer attacks on GTA, and show that ICE uses similar texture-like noises to perturb different images from different datasets. Moreover, fast fourier transformation analysis indicates that the main components in each ICE noise are three sine waves for the R, G, and B image channels. Inspired by this interesting finding, we then design a novel Sine Attack (SA) method to optimize the three sine waves. Experiments show that SA performs comparably to ICE, indicating that the three sine waves are effective and enough to break DNNs under the GTA setting. △ Less

Submitted 25 July, 2022; v1 submitted 12 October, 2021; originally announced October 2021.

Trustworthy AI: From Principles to Practices

Authors: Bo Li, Peng Qi, Bo Liu, Shuai Di, Jingen Liu, Jiquan Pei, Jinfeng Yi, Bowen Zhou

Abstract: The rapid development of Artificial Intelligence (AI) technology has enabled the deployment of various systems based on it. However, many current AI systems are found vulnerable to imperceptible attacks, biased against underrepresented groups, lacking in user privacy protection. These shortcomings degrade user experience and erode people's trust in all AI systems. In this review, we provide AI pra… ▽ More The rapid development of Artificial Intelligence (AI) technology has enabled the deployment of various systems based on it. However, many current AI systems are found vulnerable to imperceptible attacks, biased against underrepresented groups, lacking in user privacy protection. These shortcomings degrade user experience and erode people's trust in all AI systems. In this review, we provide AI practitioners with a comprehensive guide for building trustworthy AI systems. We first introduce the theoretical framework of important aspects of AI trustworthiness, including robustness, generalization, explainability, transparency, reproducibility, fairness, privacy preservation, and accountability. To unify currently available but fragmented approaches toward trustworthy AI, we organize them in a systematic approach that considers the entire lifecycle of AI systems, ranging from data acquisition to model development, to system development and deployment, finally to continuous monitoring and governance. In this framework, we offer concrete action items for practitioners and societal stakeholders (e.g., researchers, engineers, and regulators) to improve AI trustworthiness. Finally, we identify key opportunities and challenges for the future development of trustworthy AI systems, where we identify the need for a paradigm shift toward comprehensively trustworthy AI systems. △ Less

Submitted 26 May, 2022; v1 submitted 3 October, 2021; originally announced October 2021.

Efficient-FedRec: Efficient Federated Learning Framework for Privacy-Preserving News Recommendation

Authors: Jingwei Yi, Fangzhao Wu, Chuhan Wu, Ruixuan Liu, Guangzhong Sun, Xing Xie

Abstract: News recommendation is critical for personalized news access. Most existing news recommendation methods rely on centralized storage of users' historical news click behavior data, which may lead to privacy concerns and hazards. Federated Learning is a privacy-preserving framework for multiple clients to collaboratively train models without sharing their private data. However, the computation and co… ▽ More News recommendation is critical for personalized news access. Most existing news recommendation methods rely on centralized storage of users' historical news click behavior data, which may lead to privacy concerns and hazards. Federated Learning is a privacy-preserving framework for multiple clients to collaboratively train models without sharing their private data. However, the computation and communication cost of directly learning many existing news recommendation models in a federated way are unacceptable for user clients. In this paper, we propose an efficient federated learning framework for privacy-preserving news recommendation. Instead of training and communicating the whole model, we decompose the news recommendation model into a large news model maintained in the server and a light-weight user model shared on both server and clients, where news representations and user model are communicated between server and clients. More specifically, the clients request the user model and news representations from the server, and send their locally computed gradients to the server for aggregation. The server updates its global user model with the aggregated gradients, and further updates its news model to infer updated news representations. Since the local gradients may contain private information, we propose a secure aggregation method to aggregate gradients in a privacy-preserving way. Experiments on two real-world datasets show that our method can reduce the computation and communication cost on clients while keep promising model performance. △ Less

Submitted 30 May, 2023; v1 submitted 12 September, 2021; originally announced September 2021.

Journal ref: EMNLP 2021;page 2814-2824

Training Meta-Surrogate Model for Transferable Adversarial Attack

Authors: Yunxiao Qin, Yuanhao Xiong, Jinfeng Yi, Cho-Jui Hsieh

Abstract: We consider adversarial attacks to a black-box model when no queries are allowed. In this setting, many methods directly attack surrogate models and transfer the obtained adversarial examples to fool the target model. Plenty of previous works investigated what kind of attacks to the surrogate model can generate more transferable adversarial examples, but their performances are still limited due to… ▽ More We consider adversarial attacks to a black-box model when no queries are allowed. In this setting, many methods directly attack surrogate models and transfer the obtained adversarial examples to fool the target model. Plenty of previous works investigated what kind of attacks to the surrogate model can generate more transferable adversarial examples, but their performances are still limited due to the mismatches between surrogate models and the target model. In this paper, we tackle this problem from a novel angle -- instead of using the original surrogate models, can we obtain a Meta-Surrogate Model (MSM) such that attacks to this model can be easier transferred to other models? We show that this goal can be mathematically formulated as a well-posed (bi-level-like) optimization problem and design a differentiable attacker to make training feasible. Given one or a set of surrogate models, our method can thus obtain an MSM such that adversarial examples generated on MSM enjoy eximious transferability. Comprehensive experiments on Cifar-10 and ImageNet demonstrate that by attacking the MSM, we can obtain stronger transferable adversarial examples to fool black-box models including adversarially trained ones, with much higher success rates than existing methods. The proposed method reveals significant security challenges of deep models and is promising to be served as a state-of-the-art benchmark for evaluating the robustness of deep models in the black-box setting. △ Less

Submitted 7 September, 2021; v1 submitted 4 September, 2021; originally announced September 2021.

Comments: 15 pages, 7 figures

Spin-Triplet Superconductivity in K2Cr3As3

Authors: J. Yang, J. Luo, C. J. Yi, Y. G. Shi, Y. Zhou, Guo-qing Zheng

Abstract: A spin-triplet superconductor can harbor Majorana bound states that can be used in topological quantum computing. Recently, K2Cr3As3 and its variants with critical temperature Tc as high as 8 K have emerged as a new class of superconductors with ferromagnetic spin fluctuations. Here we report a discovery in K2Cr3As3 single crystal that, the spin susceptibility measured by 75As Knight shift below T… ▽ More A spin-triplet superconductor can harbor Majorana bound states that can be used in topological quantum computing. Recently, K2Cr3As3 and its variants with critical temperature Tc as high as 8 K have emerged as a new class of superconductors with ferromagnetic spin fluctuations. Here we report a discovery in K2Cr3As3 single crystal that, the spin susceptibility measured by 75As Knight shift below Tc is unchanged with the magnetic field H0 applied in the ab plane, but vanishes toward zero temperature when H0 is along the c axis, which unambiguously establishes this compound as a spin-triplet superconductor described by a vector order-parameter d parallel to the c axis. Combining with points-nodal gap we show that K2Cr3As3 is a new platform for the study of topological superconductivity and its possible technical application. △ Less

Submitted 5 January, 2022; v1 submitted 3 September, 2021; originally announced September 2021.

Journal ref: Sci. Adv. 7, eabl4432 (2021)

Study of $D_{s}^{+} \rightarrow K^{+} K^{-} π^{+}$ decay

Authors: Zhong-Yu Wang, Jing-Yu Yi, Zhi-Feng Sun, C. W. Xiao

Abstract: We investigate the $D_{s}^{+} \rightarrow K^{+} K^{-} π^{+}$ decay theoretically with the final state interactions, which is based on the chiral unitary approach and takes into account the external and internal $W$-emission mechanisms at the quark level. Only considering three resonances contributions, the $f_0(980)$ in $S$-wave, the $\bar {K}^{*}(892)^{0}$ and $φ(1020)$ in $P$-wave, one can make… ▽ More We investigate the $D_{s}^{+} \rightarrow K^{+} K^{-} π^{+}$ decay theoretically with the final state interactions, which is based on the chiral unitary approach and takes into account the external and internal $W$-emission mechanisms at the quark level. Only considering three resonances contributions, the $f_0(980)$ in $S$-wave, the $\bar {K}^{*}(892)^{0}$ and $φ(1020)$ in $P$-wave, one can make a good description of the recent experimental data from BESIII Collaboration, where the contribution from $S$-wave is found to be small. Besides, we also make a calculation of the corresponding branching fractions, which are consistent with the results of BESIII Collaboration and Particle Data Group. △ Less

Submitted 7 September, 2021; v1 submitted 31 August, 2021; originally announced September 2021.

Comments: More discussions were made. Welcome for comments

Face Photo-Sketch Recognition Using Bidirectional Collaborative Synthesis Network

Authors: Seho Bae, Nizam Ud Din, Hyunkyu Park, Juneho Yi

Abstract: This research features a deep-learning based framework to address the problem of matching a given face sketch image against a face photo database. The problem of photo-sketch matching is challenging because 1) there is large modality gap between photo and sketch, and 2) the number of paired training samples is insufficient to train deep learning based networks. To circumvent the problem of large m… ▽ More This research features a deep-learning based framework to address the problem of matching a given face sketch image against a face photo database. The problem of photo-sketch matching is challenging because 1) there is large modality gap between photo and sketch, and 2) the number of paired training samples is insufficient to train deep learning based networks. To circumvent the problem of large modality gap, our approach is to use an intermediate latent space between the two modalities. We effectively align the distributions of the two modalities in this latent space by employing a bidirectional (photo -> sketch and sketch -> photo) collaborative synthesis network. A StyleGAN-like architecture is utilized to make the intermediate latent space be equipped with rich representation power. To resolve the problem of insufficient training samples, we introduce a three-step training scheme. Extensive evaluation on public composite face sketch database confirms superior performance of our method compared to existing state-of-the-art methods. The proposed methodology can be employed in matching other modality pairs. △ Less

Submitted 22 August, 2021; originally announced August 2021.

Fractal Geometry of the Valleys of the Parabolic Anderson Equation

Authors: Promit Ghosal, Jaeyun Yi

Abstract: We study the macroscopic fractal properties of the deep valleys of the solution of the $(1+1)$-dimensional parabolic Anderson equation $${\partial \over \partial t}u(t,x) =\frac{1}{2} {\partial^2 \over \partial x^2} u(t,x) + u(t,x)\dot{W}(t,x),t>0, x\in {\bf R},\quad u(0,x) \equiv u_0(x),x\in {\bf R}, $$ where $\dot{W}$ is the time-space white noise and… ▽ More We study the macroscopic fractal properties of the deep valleys of the solution of the $(1+1)$-dimensional parabolic Anderson equation $${\partial \over \partial t}u(t,x) =\frac{1}{2} {\partial^2 \over \partial x^2} u(t,x) + u(t,x)\dot{W}(t,x),t>0, x\in {\bf R},\quad u(0,x) \equiv u_0(x),x\in {\bf R}, $$ where $\dot{W}$ is the time-space white noise and $0<\inf_{x\in {\bf R}} u_0(x)\leq \sup_{x\in {\bf R}} u_0(x)<\infty.$ Unlike the macroscopic multifractality of the tall peaks, we show that valleys of the parabolic Anderson equation are macroscopically monofractal. In fact, the macroscopic Hausdorff dimension (introduced by Barlow and Taylor [J. Phys. A 22 (1989) 2621--2628; Proc. Lond. Math. Soc. (3) 64 (1992) 125--152]) of the valleys undergoes a phase transition at a point which does not depend on the initial data. The key tool of our proof is a lower bound to the lower tail probability of the parabolic Anderson equation. Such lower bound is obtained for the first time in this paper and will be derived by utilizing the connection between the parabolic Anderson equation and the Kardar-Parisi-Zhang equation. Our techniques of proving this lower bound can be extended to other models in the KPZ universality class including the KPZ fixed point. △ Less

Submitted 10 August, 2021; v1 submitted 9 August, 2021; originally announced August 2021.

Comments: 19 pages

MSC Class: 60H15; 35R60; 60K37

Pure Exploration and Regret Minimization in Matching Bandits

Authors: Flore Sentenac, Jialin Yi, Clément Calauzènes, Vianney Perchet, Milan Vojnovic

Abstract: Finding an optimal matching in a weighted graph is a standard combinatorial problem. We consider its semi-bandit version where either a pair or a full matching is sampled sequentially. We prove that it is possible to leverage a rank-1 assumption on the adjacency matrix to reduce the sample complexity and the regret of off-the-shelf algorithms up to reaching a linear dependency in the number of ver… ▽ More Finding an optimal matching in a weighted graph is a standard combinatorial problem. We consider its semi-bandit version where either a pair or a full matching is sampled sequentially. We prove that it is possible to leverage a rank-1 assumption on the adjacency matrix to reduce the sample complexity and the regret of off-the-shelf algorithms up to reaching a linear dependency in the number of vertices (up to poly log terms). △ Less

Submitted 31 July, 2021; originally announced August 2021.

Journal ref: Proceedings of the 38th International Conference on Machine Learning, PMLR 139, 2021

Quasistatic work processes: When slowness implies certainty

Authors: Juyeon Yi, Peter Talkner

Abstract: Two approaches are outlined to characterize the fluctuation behavior of work applied to a system by a slow change of a parameter. One approach uses the adiabatic theorems of quantum and classical mechanics, the other one is based on the behavior of the correlations of the generalized coordinate that is conjugate to the changed parameter. Criteria are obtained under which the work done on small the… ▽ More Two approaches are outlined to characterize the fluctuation behavior of work applied to a system by a slow change of a parameter. One approach uses the adiabatic theorems of quantum and classical mechanics, the other one is based on the behavior of the correlations of the generalized coordinate that is conjugate to the changed parameter. Criteria are obtained under which the work done on small thermally isolated as well as on open systems ceases to fluctuate in a quasistatic process. △ Less

Submitted 9 January, 2022; v1 submitted 28 July, 2021; originally announced July 2021.

Comments: 5 pages

Journal ref: Phys. Rev. E 104, L062102 (2021)

Information retrieval and eigenstates coalescence in a non-Hermitian quantum system with anti-$\mathcal{PT}$ symmetry

Authors: Liangyu Ding, Kaiye Shi, Yuxin Wang, Qiuxin Zhang, Chenhao Zhu, Ludan Zhang, Jiaqi Yi, Shuaining Zhang, Xiang Zhang, Wei Zhang

Abstract: Non-Hermitian systems with parity-time reversal ($\mathcal{PT}$) or anti-$\mathcal{PT}$ symmetry have attracted a wide range of interest owing to their unique characteristics and counterintuitive phenomena. One of the most extraordinary features is the presence of an exception point (EP), across which a phase transition with spontaneously broken $\mathcal{PT}$ symmetry takes place. We implement a… ▽ More Non-Hermitian systems with parity-time reversal ($\mathcal{PT}$) or anti-$\mathcal{PT}$ symmetry have attracted a wide range of interest owing to their unique characteristics and counterintuitive phenomena. One of the most extraordinary features is the presence of an exception point (EP), across which a phase transition with spontaneously broken $\mathcal{PT}$ symmetry takes place. We implement a Floquet Hamiltonian of a single qubit with anti-$\mathcal{PT}$ symmetry by periodically driving a dissipative quantum system of a single trapped ion. With stroboscopic emission and quantum state tomography, we obtain the time evolution of density matrix for an arbitrary initial state, and directly demonstrate information retrieval, eigenstates coalescence, and topological energy spectra as unique features of non-Hermitian systems. △ Less

Submitted 27 July, 2021; originally announced July 2021.

Comments: 6 pages, 5 figures

Study of pion vector form factor and its contribution to the muon $(g-2)$

Authors: Jing-Yu Yi, Zhong-Yu Wang, C. W. Xiao

Abstract: In the present work, we investigate several theoretical models of the pion vector form factor and aim at getting the best fit for the two-pion cross sections to reduce the uncertainties of the calculation of two-pion contribution to the muon anomalous magnetic moment. Combined with a polynomial description to the pion vector form factor, we obtain the best fit from the Gounaris-Sakurai (or Kühn-Sa… ▽ More In the present work, we investigate several theoretical models of the pion vector form factor and aim at getting the best fit for the two-pion cross sections to reduce the uncertainties of the calculation of two-pion contribution to the muon anomalous magnetic moment. Combined with a polynomial description to the pion vector form factor, we obtain the best fit from the Gounaris-Sakurai (or Kühn-Santamaria) model for the experimental data up to 1 GeV. By product, the branching ratio of $ω\to ππ$ can be extracted as $\text{Br}(ω\toππ)=(1.52\pm 0.06) \%$, which is consistent with the one of Particle Data Group. With the best fit of the data, we obtain the muon anomalous magnetic moment from two-pion contribution as $a_μ^{\mathrm{HVP,\; LO}}(π^{+} π^{-} \leq 1\ \text{GeV}) = (497.76\pm3.15) \times 10^{-10}$. Our results are consistent with the other works. △ Less

Submitted 28 December, 2021; v1 submitted 20 July, 2021; originally announced July 2021.

Comments: Updated with more references. More discussions can be referred to the published one

Cross-modal Variational Auto-encoder for Content-based Micro-video Background Music Recommendation

Authors: Jing Yi, Yaochen Zhu, Jiayi Xie, Zhenzhong Chen

Abstract: In this paper, we propose a cross-modal variational auto-encoder (CMVAE) for content-based micro-video background music recommendation. CMVAE is a hierarchical Bayesian generative model that matches relevant background music to a micro-video by projecting these two multimodal inputs into a shared low-dimensional latent space, where the alignment of two corresponding embeddings of a matched video-m… ▽ More In this paper, we propose a cross-modal variational auto-encoder (CMVAE) for content-based micro-video background music recommendation. CMVAE is a hierarchical Bayesian generative model that matches relevant background music to a micro-video by projecting these two multimodal inputs into a shared low-dimensional latent space, where the alignment of two corresponding embeddings of a matched video-music pair is achieved by cross-generation. Moreover, the multimodal information is fused by the product-of-experts (PoE) principle, where the semantic information in visual and textual modalities of the micro-video are weighted according to their variance estimations such that the modality with a lower noise level is given more weights. Therefore, the micro-video latent variables contain less irrelevant information that results in a more robust model generalization. Furthermore, we establish a large-scale content-based micro-video background music recommendation dataset, TT-150k, composed of approximately 3,000 different background music clips associated to 150,000 micro-videos from different users. Extensive experiments on the established TT-150k dataset demonstrate the effectiveness of the proposed method. A qualitative assessment of CMVAE by visualizing some recommendation results is also included. △ Less

Submitted 11 December, 2022; v1 submitted 15 July, 2021; originally announced July 2021.

Verifix: Verified Repair of Programming Assignments

Authors: Umair Z. Ahmed, Zhiyu Fan, Jooyong Yi, Omar I. Al-Bataineh, Abhik Roychoudhury

Abstract: Automated feedback generation for introductory programming assignments is useful for programming education. Most works try to generate feedback to correct a student program by comparing its behavior with an instructor's reference program on selected tests. In this work, our aim is to generate verifiably correct program repairs as student feedback. The student assignment is aligned and composed wit… ▽ More Automated feedback generation for introductory programming assignments is useful for programming education. Most works try to generate feedback to correct a student program by comparing its behavior with an instructor's reference program on selected tests. In this work, our aim is to generate verifiably correct program repairs as student feedback. The student assignment is aligned and composed with a reference solution in terms of control flow, and differences in data variables are automatically summarized via predicates to relate the variable names. Failed verification attempts for the equivalence of the two programs are exploited to obtain a collection of maxSMT queries, whose solutions point to repairs of the student assignment. We have conducted experiments on student assignments curated from a widely deployed intelligent tutoring system. Our results indicate that we can generate verified feedback in up to 58% of the assignments. More importantly, our system indicates when it is able to generate a verified feedback, which is then usable by novice students with high confidence. △ Less

Submitted 30 June, 2021; originally announced June 2021.

Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy

Authors: Xinwei Zhang, Xiangyi Chen, Mingyi Hong, Zhiwei Steven Wu, Jinfeng Yi

Abstract: Providing privacy protection has been one of the primary motivations of Federated Learning (FL). Recently, there has been a line of work on incorporating the formal privacy notion of differential privacy with FL. To guarantee the client-level differential privacy in FL algorithms, the clients' transmitted model updates have to be clipped before adding privacy noise. Such clipping operation is subs… ▽ More Providing privacy protection has been one of the primary motivations of Federated Learning (FL). Recently, there has been a line of work on incorporating the formal privacy notion of differential privacy with FL. To guarantee the client-level differential privacy in FL algorithms, the clients' transmitted model updates have to be clipped before adding privacy noise. Such clipping operation is substantially different from its counterpart of gradient clipping in the centralized differentially private SGD and has not been well-understood. In this paper, we first empirically demonstrate that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity when training neural networks, which is partly because the clients' updates become similar for several popular deep architectures. Based on this key observation, we provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates. To the best of our knowledge, this is the first work that rigorously investigates theoretical and empirical issues regarding the clipping operation in FL algorithms. △ Less

Submitted 25 June, 2021; originally announced June 2021.

Towards Heterogeneous Clients with Elastic Federated Learning

Authors: Zichen Ma, Yu Lu, Zihan Lu, Wenye Li, Jinfeng Yi, Shuguang Cui

Abstract: Federated learning involves training machine learning models over devices or data silos, such as edge processors or data warehouses, while keeping the data local. Training in heterogeneous and potentially massive networks introduces bias into the system, which is originated from the non-IID data and the low participation rate in reality. In this paper, we propose Elastic Federated Learning (EFL),… ▽ More Federated learning involves training machine learning models over devices or data silos, such as edge processors or data warehouses, while keeping the data local. Training in heterogeneous and potentially massive networks introduces bias into the system, which is originated from the non-IID data and the low participation rate in reality. In this paper, we propose Elastic Federated Learning (EFL), an unbiased algorithm to tackle the heterogeneity in the system, which makes the most informative parameters less volatile during training, and utilizes the incomplete local updates. It is an efficient and effective algorithm that compresses both upstream and downstream communications. Theoretically, the algorithm has convergence guarantee when training on the non-IID data at the low participation rate. Empirical experiments corroborate the competitive performance of EFL framework on the robustness and the efficiency. △ Less

Submitted 17 June, 2021; originally announced June 2021.

Comments: Under Review

Object-Guided Instance Segmentation With Auxiliary Feature Refinement for Biological Images

Authors: Jingru Yi, Pengxiang Wu, Hui Tang, Bo Liu, Qiaoying Huang, Hui Qu, Lianyi Han, Wei Fan, Daniel J. Hoeppner, Dimitris N. Metaxas

Abstract: Instance segmentation is of great importance for many biological applications, such as study of neural cell interactions, plant phenotyping, and quantitatively measuring how cells react to drug treatment. In this paper, we propose a novel box-based instance segmentation method. Box-based instance segmentation methods capture objects via bounding boxes and then perform individual segmentation withi… ▽ More Instance segmentation is of great importance for many biological applications, such as study of neural cell interactions, plant phenotyping, and quantitatively measuring how cells react to drug treatment. In this paper, we propose a novel box-based instance segmentation method. Box-based instance segmentation methods capture objects via bounding boxes and then perform individual segmentation within each bounding box region. However, existing methods can hardly differentiate the target from its neighboring objects within the same bounding box region due to their similar textures and low-contrast boundaries. To deal with this problem, in this paper, we propose an object-guided instance segmentation method. Our method first detects the center points of the objects, from which the bounding box parameters are then predicted. To perform segmentation, an object-guided coarse-to-fine segmentation branch is built along with the detection branch. The segmentation branch reuses the object features as guidance to separate target object from the neighboring ones within the same bounding box region. To further improve the segmentation quality, we design an auxiliary feature refinement module that densely samples and refines point-wise features in the boundary regions. Experimental results on three biological image datasets demonstrate the advantages of our method. The code will be available at https://github.com/yijingru/ObjGuided-Instance-Segmentation. △ Less

Submitted 14 June, 2021; originally announced June 2021.

Comments: Accepted in TMI

Leveraging Tripartite Interaction Information from Live Stream E-Commerce for Improving Product Recommendation

Authors: Sanshi Yu, Zhuoxuan Jiang, Dong-Dong Chen, Shanshan Feng, Dongsheng Li, Qi Liu, Jinfeng Yi

Abstract: Recently, a new form of online shopping becomes more and more popular, which combines live streaming with E-Commerce activity. The streamers introduce products and interact with their audiences, and hence greatly improve the performance of selling products. Despite of the successful applications in industries, the live stream E-commerce has not been well studied in the data science community. To f… ▽ More Recently, a new form of online shopping becomes more and more popular, which combines live streaming with E-Commerce activity. The streamers introduce products and interact with their audiences, and hence greatly improve the performance of selling products. Despite of the successful applications in industries, the live stream E-commerce has not been well studied in the data science community. To fill this gap, we investigate this brand-new scenario and collect a real-world Live Stream E-Commerce (LSEC) dataset. Different from conventional E-commerce activities, the streamers play a pivotal role in the LSEC events. Hence, the key is to make full use of rich interaction information among streamers, users, and products. We first conduct data analysis on the tripartite interaction data and quantify the streamer's influence on users' purchase behavior. Based on the analysis results, we model the tripartite information as a heterogeneous graph, which can be decomposed to multiple bipartite graphs in order to better capture the influence. We propose a novel Live Stream E-Commerce Graph Neural Network framework (LSEC-GNN) to learn the node representations of each bipartite graph, and further design a multi-task learning approach to improve product recommendation. Extensive experiments on two real-world datasets with different scales show that our method can significantly outperform various baseline approaches. △ Less

Submitted 7 June, 2021; originally announced June 2021.

Comments: To appear in KDD'21 ADS

Knowledge Inheritance for Pre-trained Language Models

Authors: Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Abstract: Recent explorations of large-scale pre-trained language models (PLMs) have revealed the power of PLMs with huge amounts of parameters, setting off a wave of training ever-larger PLMs. However, it requires tremendous computational resources to train a large-scale PLM, which may be practically unaffordable. In addition, existing large-scale PLMs are mainly trained from scratch individually, ignoring… ▽ More Recent explorations of large-scale pre-trained language models (PLMs) have revealed the power of PLMs with huge amounts of parameters, setting off a wave of training ever-larger PLMs. However, it requires tremendous computational resources to train a large-scale PLM, which may be practically unaffordable. In addition, existing large-scale PLMs are mainly trained from scratch individually, ignoring that many well-trained PLMs are available. To this end, we explore the question how could existing PLMs benefit training large-scale PLMs in future. Specifically, we introduce a pre-training framework named "knowledge inheritance" (KI) and explore how could knowledge distillation serve as auxiliary supervision during pre-training to efficiently learn larger PLMs. Experimental results demonstrate the superiority of KI in training efficiency. We also conduct empirical analyses to explore the effects of teacher PLMs' pre-training settings, including model architecture, pre-training data, etc. Finally, we show that KI could be applied to domain adaptation and knowledge transfer. △ Less

Submitted 26 April, 2022; v1 submitted 28 May, 2021; originally announced May 2021.

Comments: NAACL 2022

Deep learning in biomedical optics

Authors: Lei Tian, Brady Hunt, Muyinatu A. Lediju Bell, Ji Yi, Jason T. Smith, Marien Ochoa, Xavier Intes, Nicholas J. Durr

Abstract: This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each… ▽ More This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. △ Less

Submitted 23 May, 2021; originally announced May 2021.

A Simple yet Universal Strategy for Online Convex Optimization

Authors: Lijun Zhang, Guanghui Wang, Jinfeng Yi, Tianbao Yang

Abstract: Recently, several universal methods have been proposed for online convex optimization, and attain minimax rates for multiple types of convex functions simultaneously. However, they need to design and optimize one surrogate loss for each type of functions, which makes it difficult to exploit the structure of the problem and utilize the vast amount of existing algorithms. In this paper, we propose a… ▽ More Recently, several universal methods have been proposed for online convex optimization, and attain minimax rates for multiple types of convex functions simultaneously. However, they need to design and optimize one surrogate loss for each type of functions, which makes it difficult to exploit the structure of the problem and utilize the vast amount of existing algorithms. In this paper, we propose a simple strategy for universal online convex optimization, which avoids these limitations. The key idea is to construct a set of experts to process the original online functions, and deploy a meta-algorithm over the \emph{linearized} losses to aggregate predictions from experts. Specifically, we choose Adapt-ML-Prod to track the best expert, because it has a second-order bound and can be used to leverage strong convexity and exponential concavity. In this way, we can plug in off-the-shelf online solvers as black-box experts to deliver problem-dependent regret bounds. Furthermore, our strategy inherits the theoretical guarantee of any expert designed for strongly convex functions and exponentially concave functions, up to a double logarithmic factor. For general convex functions, it maintains the minimax optimality and also achieves a small-loss bound. △ Less

Submitted 14 May, 2021; v1 submitted 8 May, 2021; originally announced May 2021.

Efficient Click-Through Rate Prediction for Developing Countries via Tabular Learning

Authors: Joonyoung Yi, Buru Chang

Abstract: Despite the rapid growth of online advertisement in developing countries, existing highly over-parameterized Click-Through Rate (CTR) prediction models are difficult to be deployed due to the limited computing resources. In this paper, by bridging the relationship between CTR prediction task and tabular learning, we present that tabular learning models are more efficient and effective in CTR predi… ▽ More Despite the rapid growth of online advertisement in developing countries, existing highly over-parameterized Click-Through Rate (CTR) prediction models are difficult to be deployed due to the limited computing resources. In this paper, by bridging the relationship between CTR prediction task and tabular learning, we present that tabular learning models are more efficient and effective in CTR prediction than over-parameterized CTR prediction models. Extensive experiments on eight public CTR prediction datasets show that tabular learning models outperform twelve state-of-the-art CTR prediction models. Furthermore, compared to over-parameterized CTR prediction models, tabular learning models can be fast trained without expensive computing resources including high-performance GPUs. Finally, through an A/B test on an actual online application, we show that tabular learning models improve not only offline performance but also the CTR of real users. △ Less

Submitted 15 April, 2021; originally announced April 2021.

Comments: ICLR 2021 Workshop (PML4DC), 8 pages, 2 figures

DebiasedRec: Bias-aware User Modeling and Click Prediction for Personalized News Recommendation

Authors: Jingwei Yi, Fangzhao Wu, Chuhan Wu, Qifei Li, Guangzhong Sun, Xing Xie

Abstract: News recommendation is critical for personalized news access. Existing news recommendation methods usually infer users' personal interest based on their historical clicked news, and train the news recommendation models by predicting future news clicks. A core assumption behind these methods is that news click behaviors can indicate user interest. However, in practical scenarios, beyond the relevan… ▽ More News recommendation is critical for personalized news access. Existing news recommendation methods usually infer users' personal interest based on their historical clicked news, and train the news recommendation models by predicting future news clicks. A core assumption behind these methods is that news click behaviors can indicate user interest. However, in practical scenarios, beyond the relevance between user interest and news content, the news click behaviors may also be affected by other factors, such as the bias of news presentation in the online platform. For example, news with higher positions and larger sizes are usually more likely to be clicked. The bias of clicked news may bring noises to user interest modeling and model training, which may hurt the performance of the news recommendation model. In this paper, we propose a bias-aware personalized news recommendation method named DebiasRec, which can handle the bias information for more accurate user interest inference and model training. The core of our method includes a bias representation module, a bias-aware user modeling module, and a bias-aware click prediction module. The bias representation module is used to model different kinds of news bias and their interactions to capture their joint effect on click behaviors. The bias-aware user modeling module aims to infer users' debiased interest from the clicked news articles by using their bias information to calibrate the interest model. The bias-aware click prediction module is used to train a debiased news recommendation model from the biased click behaviors, where the click score is decomposed into a preference score indicating user's interest in the news content and a news bias score inferred from its different bias features. Experiments on two real-world datasets show that our method can effectively improve the performance of news recommendation. △ Less

Submitted 15 April, 2021; originally announced April 2021.

Comments: 10 pages, 10 figures

Continual Learning for Fake Audio Detection

Authors: Haoxin Ma, Jiangyan Yi, Jianhua Tao, Ye Bai, Zhengkun Tian, Chenglong Wang

Abstract: Fake audio attack becomes a major threat to the speaker verification system. Although current detection approaches have achieved promising results on dataset-specific scenarios, they encounter difficulties on unseen spoofing data. Fine-tuning and retraining from scratch have been applied to incorporate new data. However, fine-tuning leads to performance degradation on previous data. Retraining tak… ▽ More Fake audio attack becomes a major threat to the speaker verification system. Although current detection approaches have achieved promising results on dataset-specific scenarios, they encounter difficulties on unseen spoofing data. Fine-tuning and retraining from scratch have been applied to incorporate new data. However, fine-tuning leads to performance degradation on previous data. Retraining takes a lot of time and computation resources. Besides, previous data are unavailable due to privacy in some situations. To solve the above problems, this paper proposes detecting fake without forgetting, a continual-learning-based method, to make the model learn new spoofing attacks incrementally. A knowledge distillation loss is introduced to loss function to preserve the memory of original model. Supposing the distribution of genuine voice is consistent among different scenarios, an extra embedding similarity loss is used as another constraint to further do a positive sample alignment. Experiments are conducted on the ASVspoof2019 dataset. The results show that our proposed method outperforms fine-tuning by the relative reduction of average equal error rate up to 81.62%. △ Less

Submitted 15 April, 2021; originally announced April 2021.

Comments: 5 pages, conference

Journal ref: Proc. Interspeech 2021, 886-890

Gradient-based Adversarial Deep Modulation Classification with Data-driven Subsampling

Authors: Jinho Yi, Aly El Gamal

Abstract: Automatic modulation classification can be a core component for intelligent spectrally efficient wireless communication networks, and deep learning techniques have recently been shown to deliver superior performance to conventional model-based strategies, particularly when distinguishing between a large number of modulation types. However, such deep learning techniques have also been recently show… ▽ More Automatic modulation classification can be a core component for intelligent spectrally efficient wireless communication networks, and deep learning techniques have recently been shown to deliver superior performance to conventional model-based strategies, particularly when distinguishing between a large number of modulation types. However, such deep learning techniques have also been recently shown to be vulnerable to gradient-based adversarial attacks that rely on subtle input perturbations, which would be particularly feasible in a wireless setting via jamming. One such potent attack is the one known as the Carlini-Wagner attack, which we consider in this work. We further consider a data-driven subsampling setting, where several recently introduced deep-learning-based algorithms are employed to select a subset of samples that lead to reducing the final classifier's training time with minimal loss in accuracy. In this setting, the attacker has to make an assumption about the employed subsampling strategy, in order to calculate the loss gradient. Based on state of the art techniques available to both the attacker and defender, we evaluate best strategies under various assumptions on the knowledge of the other party's strategy. Interestingly, in presence of knowledgeable attackers, we identify computational cost reduction opportunities for the defender with no or minimal loss in performance. △ Less

Submitted 3 April, 2021; originally announced April 2021.

Comments: 5 pages, 5 figures

Half-Truth: A Partially Fake Audio Detection Dataset

Authors: Jiangyan Yi, Ye Bai, Jianhua Tao, Haoxin Ma, Zhengkun Tian, Chenglong Wang, Tao Wang, Ruibo Fu

Abstract: Diverse promising datasets have been designed to hold back the development of fake audio detection, such as ASVspoof databases. However, previous datasets ignore an attacking situation, in which the hacker hides some small fake clips in real speech audio. This poses a serious threat since that it is difficult to distinguish the small fake clip from the whole speech utterance. Therefore, this paper… ▽ More Diverse promising datasets have been designed to hold back the development of fake audio detection, such as ASVspoof databases. However, previous datasets ignore an attacking situation, in which the hacker hides some small fake clips in real speech audio. This poses a serious threat since that it is difficult to distinguish the small fake clip from the whole speech utterance. Therefore, this paper develops such a dataset for half-truth audio detection (HAD). Partially fake audio in the HAD dataset involves only changing a few words in an utterance.The audio of the words is generated with the very latest state-of-the-art speech synthesis technology. We can not only detect fake uttrances but also localize manipulated regions in a speech using this dataset. Some benchmark results are presented on this dataset. The results show that partially fake audio presents much more challenging than fully fake audio for fake audio detection. The HAD dataset is publicly available: https://zenodo.org/records/10377492. △ Less

Submitted 15 December, 2023; v1 submitted 8 April, 2021; originally announced April 2021.

Comments: accepted by Interspeech 2021

FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularization

Authors: Zhengkun Tian, Jiangyan Yi, Ye Bai, Jianhua Tao, Shuai Zhang, Zhengqi Wen

Abstract: Transducer-based models, such as RNN-Transducer and transformer-transducer, have achieved great success in speech recognition. A typical transducer model decodes the output sequence conditioned on the current acoustic state and previously predicted tokens step by step. Statistically, The number of blank tokens in the prediction results accounts for nearly 90\% of all tokens. It takes a lot of comp… ▽ More Transducer-based models, such as RNN-Transducer and transformer-transducer, have achieved great success in speech recognition. A typical transducer model decodes the output sequence conditioned on the current acoustic state and previously predicted tokens step by step. Statistically, The number of blank tokens in the prediction results accounts for nearly 90\% of all tokens. It takes a lot of computation and time to predict the blank tokens, but only the non-blank tokens will appear in the final output sequence. Therefore, we propose a method named fast-skip regularization, which tries to align the blank position predicted by a transducer with that predicted by a CTC model. During the inference, the transducer model can predict the blank tokens in advance by a simple CTC project layer without many complicated forward calculations of the transducer decoder and then skip them, which will reduce the computation and improve the inference speed greatly. All experiments are conducted on a public Chinese mandarin dataset AISHELL-1. The results show that the fast-skip regularization can indeed help the transducer model learn the blank position alignments. Besides, the inference with fast-skip can be speeded up nearly 4 times with only a little performance degradation. △ Less

Submitted 6 April, 2021; originally announced April 2021.

Comments: Submitted to INTERSPEECH2021

Solving Large Scale Quadratic Constrained Basis Pursuit

Authors: Jirong Yi

Abstract: Inspired by alternating direction method of multipliers and the idea of operator splitting, we propose a efficient algorithm for solving large-scale quadratically constrained basis pursuit. Experimental results show that the proposed algorithm can achieve 50~~100 times speedup when compared with the baseline interior point algorithm implemented in CVX. Inspired by alternating direction method of multipliers and the idea of operator splitting, we propose a efficient algorithm for solving large-scale quadratically constrained basis pursuit. Experimental results show that the proposed algorithm can achieve 50~~100 times speedup when compared with the baseline interior point algorithm implemented in CVX. △ Less

Submitted 2 April, 2021; originally announced April 2021.

Comments: 5 pages, 1 figure

TSNAT: Two-Step Non-Autoregressvie Transformer Models for Speech Recognition

Authors: Zhengkun Tian, Jiangyan Yi, Jianhua Tao, Ye Bai, Shuai Zhang, Zhengqi Wen, Xuefei Liu

Abstract: The autoregressive (AR) models, such as attention-based encoder-decoder models and RNN-Transducer, have achieved great success in speech recognition. They predict the output sequence conditioned on the previous tokens and acoustic encoded states, which is inefficient on GPUs. The non-autoregressive (NAR) models can get rid of the temporal dependency between the output tokens and predict the entire… ▽ More The autoregressive (AR) models, such as attention-based encoder-decoder models and RNN-Transducer, have achieved great success in speech recognition. They predict the output sequence conditioned on the previous tokens and acoustic encoded states, which is inefficient on GPUs. The non-autoregressive (NAR) models can get rid of the temporal dependency between the output tokens and predict the entire output tokens in at least one step. However, the NAR model still faces two major problems. On the one hand, there is still a great gap in performance between the NAR models and the advanced AR models. On the other hand, it's difficult for most of the NAR models to train and converge. To address these two problems, we propose a new model named the two-step non-autoregressive transformer(TSNAT), which improves the performance and accelerating the convergence of the NAR model by learning prior knowledge from a parameters-sharing AR model. Furthermore, we introduce the two-stage method into the inference process, which improves the model performance greatly. All the experiments are conducted on a public Chinese mandarin dataset ASIEHLL-1. The results show that the TSNAT can achieve a competitive performance with the AR model and outperform many complicated NAR models. △ Less

Submitted 3 April, 2021; originally announced April 2021.

Comments: Submitted to Interspeech2021

Fast Certified Robust Training with Short Warmup

Authors: Zhouxing Shi, Yihan Wang, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh

Abstract: Recently, bound propagation based certified robust training methods have been proposed for training neural networks with certifiable robustness guarantees. Despite that state-of-the-art (SOTA) methods including interval bound propagation (IBP) and CROWN-IBP have per-batch training complexity similar to standard neural network training, they usually use a long warmup schedule with hundreds or thous… ▽ More Recently, bound propagation based certified robust training methods have been proposed for training neural networks with certifiable robustness guarantees. Despite that state-of-the-art (SOTA) methods including interval bound propagation (IBP) and CROWN-IBP have per-batch training complexity similar to standard neural network training, they usually use a long warmup schedule with hundreds or thousands epochs to reach SOTA performance and are thus still costly. In this paper, we identify two important issues in existing methods, namely exploded bounds at initialization, and the imbalance in ReLU activation states and improve IBP training. These two issues make certified training difficult and unstable, and thereby long warmup schedules were needed in prior works. To mitigate these issues and conduct faster certified training with shorter warmup, we propose three improvements based on IBP training: 1) We derive a new weight initialization method for IBP training; 2) We propose to fully add Batch Normalization (BN) to each layer in the model, since we find BN can reduce the imbalance in ReLU activation states; 3) We also design regularization to explicitly tighten certified bounds and balance ReLU activation states during wamrup. We are able to obtain 65.03% verified error on CIFAR-10 ($ε=\frac{8}{255}$) and 82.36% verified error on TinyImageNet ($ε=\frac{1}{255}$) using very short training schedules (160 and 80 total epochs, respectively), outperforming literature SOTA trained with hundreds or thousands epochs under the same network architecture. The code is available at https://github.com/shizhouxing/Fast-Certified-Robust-Training. △ Less

Submitted 27 October, 2021; v1 submitted 31 March, 2021; originally announced March 2021.

Comments: NeurIPS 2021

On the Adversarial Robustness of Vision Transformers

Authors: Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin-Yu Chen, Cho-Jui Hsieh

Abstract: Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision. This work provides a comprehensive study on the robustness of vision transformers (ViTs) against adversarial perturbations. Tested on various white-box and transfer attack settings, we find that ViTs possess better adversarial robustness whe… ▽ More Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision. This work provides a comprehensive study on the robustness of vision transformers (ViTs) against adversarial perturbations. Tested on various white-box and transfer attack settings, we find that ViTs possess better adversarial robustness when compared with MLP-Mixer and convolutional neural networks (CNNs) including ConvNeXt, and this observation also holds for certified robustness. Through frequency analysis and feature visualization, we summarize the following main observations contributing to the improved robustness of ViTs: 1) Features learned by ViTs contain less high-frequency patterns that have spurious correlation, which helps explain why ViTs are less sensitive to high-frequency perturbations than CNNs and MLP-Mixer, and there is a high correlation between how much the model learns high-frequency features and its robustness against different frequency-based perturbations. 2) Introducing convolutional or tokens-to-token blocks for learning high-frequency features in ViTs can improve classification accuracy but at the cost of adversarial robustness. 3) Modern CNN designs that borrow techniques from ViTs including activation function, layer norm, larger kernel size to imitate the global attention, and patchify the images as inputs, etc., could help bridge the performance gap between ViTs and CNNs not only in terms of performance, but also certified and empirical adversarial robustness. Moreover, we show adversarial training is also applicable to ViT for training robust models, and sharpness-aware minimization can also help improve robustness, while pre-training with clean images on larger datasets does not significantly improve adversarial robustness. △ Less

Submitted 2 November, 2022; v1 submitted 29 March, 2021; originally announced March 2021.

Comments: Published in Transactions on Machine Learning Research (TMLR). Codes available at https://github.com/RulinShao/on-the-adversarial-robustness-of-visual-transformer

BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation

Authors: Jungbeom Lee, Jihun Yi, Chaehun Shin, Sungroh Yoon

Abstract: Weakly supervised segmentation methods using bounding box annotations focus on obtaining a pixel-level mask from each box containing an object. Existing methods typically depend on a class-agnostic mask generator, which operates on the low-level information intrinsic to an image. In this work, we utilize higher-level information from the behavior of a trained object detector, by seeking the smalle… ▽ More Weakly supervised segmentation methods using bounding box annotations focus on obtaining a pixel-level mask from each box containing an object. Existing methods typically depend on a class-agnostic mask generator, which operates on the low-level information intrinsic to an image. In this work, we utilize higher-level information from the behavior of a trained object detector, by seeking the smallest areas of the image from which the object detector produces almost the same result as it does from the whole image. These areas constitute a bounding-box attribution map (BBAM), which identifies the target object in its bounding box and thus serves as pseudo ground-truth for weakly supervised semantic and instance segmentation. This approach significantly outperforms recent comparable techniques on both the PASCAL VOC and MS COCO benchmarks in weakly supervised semantic and instance segmentation. In addition, we provide a detailed analysis of our method, offering deeper insight into the behavior of the BBAM. △ Less

Submitted 16 March, 2021; originally announced March 2021.

Comments: CVPR 2021

Femtosecond field-driven on-chip unidirectional electronic currents in nonadiabatic tunnelling regime

Authors: Liping Shi, Ihar Babushkin, Anton Husakou, Oliver Melchert, Bettina Frank, Juemin Yi, Gustav Wetzel, Ayhan Demircan, Christoph Lienau, Harald Giessen, Misha Ivanov, Uwe Morgner, Milutin Kovacev

Abstract: Recently, asymmetric plasmonic nanojunctions [Karnetzky et. al., Nature Comm. 2471, 9 (2018)] have shown promise as on-chip electronic devices to convert femtosecond optical pulses to current bursts, with a bandwidth of multi-terahertz scale, although yet at low temperatures and pressures. Such nanoscale devices are of great interest for novel ultrafast electronics and opto-electronic applications… ▽ More Recently, asymmetric plasmonic nanojunctions [Karnetzky et. al., Nature Comm. 2471, 9 (2018)] have shown promise as on-chip electronic devices to convert femtosecond optical pulses to current bursts, with a bandwidth of multi-terahertz scale, although yet at low temperatures and pressures. Such nanoscale devices are of great interest for novel ultrafast electronics and opto-electronic applications. Here, we operate the device in air and at room temperature, revealing the mechanisms of photoemission from plasmonic nanojunctions, and the fundamental limitations on the speed of optical-to-electronic conversion. Inter-cycle interference of coherent electronic wavepackets results in a complex energy electron distribution and birth of multiphoton effects. This energy structure, as well as reshaping of the wavepackets during their propagation from one tip to the other, determine the ultrafast dynamics of the current. We show that, up to some level of approximation, the electron flight time is well-determined by the mean ponderomotive velocity in the driving field. △ Less

Submitted 8 March, 2021; v1 submitted 4 March, 2021; originally announced March 2021.

Comments: references updated

Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT

Authors: Ye Bai, Jiangyan Yi, Jianhua Tao, Zhengkun Tian, Zhengqi Wen, Shuai Zhang

Abstract: Attention-based encoder-decoder (AED) models have achieved promising performance in speech recognition. However, because the decoder predicts text tokens (such as characters or words) in an autoregressive manner, it is difficult for an AED model to predict all tokens in parallel. This makes the inference speed relatively slow. We believe that because the encoder already captures the whole speech u… ▽ More Attention-based encoder-decoder (AED) models have achieved promising performance in speech recognition. However, because the decoder predicts text tokens (such as characters or words) in an autoregressive manner, it is difficult for an AED model to predict all tokens in parallel. This makes the inference speed relatively slow. We believe that because the encoder already captures the whole speech utterance, which has the token-level relationship implicitly, we can predict a token without explicitly autoregressive language modeling. When the prediction of a token does not rely on other tokens, the parallel prediction of all tokens in the sequence is realizable. Based on this idea, we propose a non-autoregressive speech recognition model called LASO (Listen Attentively, and Spell Once). The model consists of an encoder, a decoder, and a position dependent summarizer (PDS). The three modules are based on basic attention blocks. The encoder extracts high-level representations from the speech. The PDS uses positional encodings corresponding to tokens to convert the acoustic representations into token-level representations. The decoder further captures token-level relationships with the self-attention mechanism. At last, the probability distribution on the vocabulary is computed for each token position. Therefore, speech recognition is re-formulated as a position-wise classification problem. Further, we propose a cross-modal transfer learning method to refine semantics from a large-scale pre-trained language model BERT for improving the performance. △ Less

Submitted 29 August, 2021; v1 submitted 15 February, 2021; originally announced February 2021.

Comments: 14 pages, 7 figures

Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training

Authors: Lue Tao, Lei Feng, Jinfeng Yi, Sheng-Jun Huang, Songcan Chen

Abstract: Delusive attacks aim to substantially deteriorate the test accuracy of the learning model by slightly perturbing the features of correctly labeled training examples. By formalizing this malicious attack as finding the worst-case training data within a specific $\infty$-Wasserstein ball, we show that minimizing adversarial risk on the perturbed data is equivalent to optimizing an upper bound of nat… ▽ More Delusive attacks aim to substantially deteriorate the test accuracy of the learning model by slightly perturbing the features of correctly labeled training examples. By formalizing this malicious attack as finding the worst-case training data within a specific $\infty$-Wasserstein ball, we show that minimizing adversarial risk on the perturbed data is equivalent to optimizing an upper bound of natural risk on the original data. This implies that adversarial training can serve as a principled defense against delusive attacks. Thus, the test accuracy decreased by delusive attacks can be largely recovered by adversarial training. To further understand the internal mechanism of the defense, we disclose that adversarial training can resist the delusive perturbations by preventing the learner from overly relying on non-robust features in a natural setting. Finally, we complement our theoretical findings with a set of experiments on popular benchmark datasets, which show that the defense withstands six different practical attacks. Both theoretical and empirical results vote for adversarial training when confronted with delusive adversaries. △ Less

Submitted 13 December, 2021; v1 submitted 9 February, 2021; originally announced February 2021.

Comments: NeurIPS 2021

Superconductivity in centrosymmetric topological superconductor candidate TaC

Authors: D. Yu. Yan, M. Yang, C. X. Wang, P. B. Song, C. J. Yi, Y. G. Shi

Abstract: We report the synthesis and physical properties of the single crystals of TaC, which are proposed to hold topological band structure as a topological superconductor (TSC) candidate. Magnetization, resistivity and specific heat measurements are performed and indicate that TaC is bulk superconductor with critical temperature of 10.3 K. TaC is a strongly coupled type-II superconductor and the superco… ▽ More We report the synthesis and physical properties of the single crystals of TaC, which are proposed to hold topological band structure as a topological superconductor (TSC) candidate. Magnetization, resistivity and specific heat measurements are performed and indicate that TaC is bulk superconductor with critical temperature of 10.3 K. TaC is a strongly coupled type-II superconductor and the superconducting state can be well described by s-wave Bardeen-Cooper-Schrieffer (BCS) theory with a single gap. The upper critical field (Hc2) of TaC shows linear temperature dependence, which is quite different from most conventional superconductors and isostructural NbC, which is proposed to manifest topological nodal-loops or type-II Dirac points as well as superconductivity. Our results suggest that TaC would be a new candidate for further research of TSCs. △ Less

Submitted 3 February, 2021; originally announced February 2021.

Comments: 23 pages, 5 figures, 2 tables

Macroscopic multi-fractality of Gaussian random fields and linear SPDEs with colored noise

Authors: Jaeyun Yi

Abstract: We consider the linear stochastic heat and wave equations with generalized Gaussian noise that is white in time and spatially correlated. Under the assumption that the homogeneous spatial correlation $f$ satisfies some mild conditions, we show that the solutions to the linear stochastic heat and wave equations exhibit tall peaks in macroscopic scales, which means they are macroscopically multi-fra… ▽ More We consider the linear stochastic heat and wave equations with generalized Gaussian noise that is white in time and spatially correlated. Under the assumption that the homogeneous spatial correlation $f$ satisfies some mild conditions, we show that the solutions to the linear stochastic heat and wave equations exhibit tall peaks in macroscopic scales, which means they are macroscopically multi-fractal. We compute the macroscopic Hausdorff dimension of the peaks for Gaussian random fields with vanishing correlation and then apply this result to the solution of the linear stochastic heat and wave equations. We also study the spatio-temporal multi-fractality of the linear stochastic heat and wave equations. Our result is an extension of Khoshnevisan, Kim, and Xiao \cite{KKX,KKX2} and Kim \cite{K} to a more general class of the linear stochastic partial differential equations and Gaussian random fields. △ Less

Submitted 23 January, 2021; originally announced January 2021.

Comments: 19pages

MSC Class: 60H15; 60G15; 60K37; 35R60

Removing Undesirable Feature Contributions Using Out-of-Distribution Data

Authors: Saehyung Lee, Changhwa Park, Hyungyu Lee, Jihun Yi, Jonghyun Lee, Sungroh Yoon

Abstract: Several data augmentation methods deploy unlabeled-in-distribution (UID) data to bridge the gap between the training and inference of neural networks. However, these methods have clear limitations in terms of availability of UID data and dependence of algorithms on pseudo-labels. Herein, we propose a data augmentation method to improve generalization in both adversarial and standard learning by us… ▽ More Several data augmentation methods deploy unlabeled-in-distribution (UID) data to bridge the gap between the training and inference of neural networks. However, these methods have clear limitations in terms of availability of UID data and dependence of algorithms on pseudo-labels. Herein, we propose a data augmentation method to improve generalization in both adversarial and standard learning by using out-of-distribution (OOD) data that are devoid of the abovementioned issues. We show how to improve generalization theoretically using OOD data in each learning scenario and complement our theoretical analysis with experiments on CIFAR-10, CIFAR-100, and a subset of ImageNet. The results indicate that undesirable features are shared even among image data that seem to have little correlation from a human point of view. We also present the advantages of the proposed method through comparison with other data augmentation methods, which can be used in the absence of UID data. Furthermore, we demonstrate that the proposed method can further improve the existing state-of-the-art adversarial training. △ Less

Submitted 20 November, 2021; v1 submitted 17 January, 2021; originally announced January 2021.

Comments: Published as a conference paper at ICLR 2021

HIVE-Net: Centerline-Aware HIerarchical View-Ensemble Convolutional Network for Mitochondria Segmentation in EM Images

Authors: Zhimin Yuan, Xiaofen Ma, Jiajin Yi, Zhengrong Luo, Jialin Peng

Abstract: Semantic segmentation of electron microscopy (EM) is an essential step to efficiently obtain reliable morphological statistics. Despite the great success achieved using deep convolutional neural networks (CNNs), they still produce coarse segmentations with lots of discontinuities and false positives for mitochondria segmentation. In this study, we introduce a centerline-aware multitask network by… ▽ More Semantic segmentation of electron microscopy (EM) is an essential step to efficiently obtain reliable morphological statistics. Despite the great success achieved using deep convolutional neural networks (CNNs), they still produce coarse segmentations with lots of discontinuities and false positives for mitochondria segmentation. In this study, we introduce a centerline-aware multitask network by utilizing centerline as an intrinsic shape cue of mitochondria to regularize the segmentation. Since the application of 3D CNNs on large medical volumes is usually hindered by their substantial computational cost and storage overhead, we introduce a novel hierarchical view-ensemble convolution (HVEC), a simple alternative of 3D convolution to learn 3D spatial contexts using more efficient 2D convolutions. The HVEC enables both decomposing and sharing multi-view information, leading to increased learning capacity. Extensive validation results on two challenging benchmarks show that, the proposed method performs favorably against the state-of-the-art methods in accuracy and visual quality but with a greatly reduced model size. Moreover, the proposed model also shows significantly improved generalization ability, especially when training with quite limited amount of training data. △ Less

Submitted 8 January, 2021; originally announced January 2021.

Comments: Accepted by CPMB

Robust Text CAPTCHAs Using Adversarial Examples

Authors: Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin-Yu Chen, Cho-Jui Hsieh

Abstract: CAPTCHA (Completely Automated Public Truing test to tell Computers and Humans Apart) is a widely used technology to distinguish real users and automated users such as bots. However, the advance of AI technologies weakens many CAPTCHA tests and can induce security concerns. In this paper, we propose a user-friendly text-based CAPTCHA generation method named Robust Text CAPTCHA (RTC). At the first s… ▽ More CAPTCHA (Completely Automated Public Truing test to tell Computers and Humans Apart) is a widely used technology to distinguish real users and automated users such as bots. However, the advance of AI technologies weakens many CAPTCHA tests and can induce security concerns. In this paper, we propose a user-friendly text-based CAPTCHA generation method named Robust Text CAPTCHA (RTC). At the first stage, the foregrounds and backgrounds are constructed with randomly sampled font and background images, which are then synthesized into identifiable pseudo adversarial CAPTCHAs. At the second stage, we design and apply a highly transferable adversarial attack for text CAPTCHAs to better obstruct CAPTCHA solvers. Our experiments cover comprehensive models including shallow models such as KNN, SVM and random forest, various deep neural networks and OCR models. Experiments show that our CAPTCHAs have a failure rate lower than one millionth in general and high usability. They are also robust against various defensive techniques that attackers may employ, including adversarial training, data pre-processing and manual tagging. △ Less

Submitted 7 January, 2021; originally announced January 2021.

With False Friends Like These, Who Can Notice Mistakes?

Authors: Lue Tao, Lei Feng, Jinfeng Yi, Songcan Chen

Abstract: Adversarial examples crafted by an explicit adversary have attracted significant attention in machine learning. However, the security risk posed by a potential false friend has been largely overlooked. In this paper, we unveil the threat of hypocritical examples -- inputs that are originally misclassified yet perturbed by a false friend to force correct predictions. While such perturbed examples s… ▽ More Adversarial examples crafted by an explicit adversary have attracted significant attention in machine learning. However, the security risk posed by a potential false friend has been largely overlooked. In this paper, we unveil the threat of hypocritical examples -- inputs that are originally misclassified yet perturbed by a false friend to force correct predictions. While such perturbed examples seem harmless, we point out for the first time that they could be maliciously used to conceal the mistakes of a substandard (i.e., not as good as required) model during an evaluation. Once a deployer trusts the hypocritical performance and applies the "well-performed" model in real-world applications, unexpected failures may happen even in benign environments. More seriously, this security risk seems to be pervasive: we find that many types of substandard models are vulnerable to hypocritical examples across multiple datasets. Furthermore, we provide the first attempt to characterize the threat with a metric called hypocritical risk and try to circumvent it via several countermeasures. Results demonstrate the effectiveness of the countermeasures, while the risk remains non-negligible even after adaptive robust training. △ Less

Submitted 13 December, 2021; v1 submitted 29 December, 2020; originally announced December 2020.

Comments: AAAI 2022

On the Limitations of Denoising Strategies as Adversarial Defenses

Authors: Zhonghan Niu, Zhaoxi Chen, Linyi Li, Yubin Yang, Bo Li, Jinfeng Yi

Abstract: As adversarial attacks against machine learning models have raised increasing concerns, many denoising-based defense approaches have been proposed. In this paper, we summarize and analyze the defense strategies in the form of symmetric transformation via data denoising and reconstruction (denoted as $F+$ inverse $F$, $F-IF$ Framework). In particular, we categorize these denoising strategies from t… ▽ More As adversarial attacks against machine learning models have raised increasing concerns, many denoising-based defense approaches have been proposed. In this paper, we summarize and analyze the defense strategies in the form of symmetric transformation via data denoising and reconstruction (denoted as $F+$ inverse $F$, $F-IF$ Framework). In particular, we categorize these denoising strategies from three aspects (i.e. denoising in the spatial domain, frequency domain, and latent space, respectively). Typically, defense is performed on the entire adversarial example, both image and perturbation are modified, making it difficult to tell how it defends against the perturbations. To evaluate the robustness of these denoising strategies intuitively, we directly apply them to defend against adversarial noise itself (assuming we have obtained all of it), which saving us from sacrificing benign accuracy. Surprisingly, our experimental results show that even if most of the perturbations in each dimension is eliminated, it is still difficult to obtain satisfactory robustness. Based on the above findings and analyses, we propose the adaptive compression strategy for different frequency bands in the feature domain to improve the robustness. Our experiment results show that the adaptive compression strategies enable the model to better suppress adversarial perturbations, and improve robustness compared with existing denoising strategies. △ Less

Submitted 16 December, 2020; originally announced December 2020.

Gate-controlled magnetic phase transition in a van der Waals magnet Fe$_5$GeTe$_2$

Authors: Cheng Tan, Wen-Qiang Xie, Guolin Zheng, Nuriyah Aloufi, Sultan Albarakati, Meri Algarni, Jiangpeng Song, James Partridge, Dimitrie Culcer, Xiaolin Wang, Jiabao Yi, Yimin Xiong, Mingliang Tian, Yu-Jun Zhao, Lan Wang

Abstract: Magnetic van der Waals (vdW) materials, including ferromagnets (FM) and antiferromagnets (AFM), have given access to the investigation of magnetism in two-dimensional (2D) limit and attracted broad interests recently. However, most of them are semiconducting or insulating and the vdW itinerant magnets, especially vdW itinerant AFM, are very rare. Here, we studied the anomalous Hall effect of a vdW… ▽ More Magnetic van der Waals (vdW) materials, including ferromagnets (FM) and antiferromagnets (AFM), have given access to the investigation of magnetism in two-dimensional (2D) limit and attracted broad interests recently. However, most of them are semiconducting or insulating and the vdW itinerant magnets, especially vdW itinerant AFM, are very rare. Here, we studied the anomalous Hall effect of a vdW itinerant magnet Fe$_5$GeTe$_2$ (F5GT) with various thicknesses down to 6.8 nm (two unit cells). Despite the robust ferromagnetic ground state in thin-layer F5GT, however, we show that the electron doping implemented by a protonic gate can eventually induce a magnetic phase transition from FM to AFM. Realization of an antiferromagnetic phase in F5GT highlights its promising applications in high-temperature antiferromagnetic vdW devices and heterostructures. △ Less

Submitted 1 December, 2020; originally announced December 2020.

Comments: 27 pages

A two-dimensional electron gas based on a 5s oxide with high room-temperature mobility and strain sensitivity

Authors: Zexin Feng, Peixin Qin, Yali Yang, Han Yan, Huixin Guo, Xiaoning Wang, Xiaorong Zhou, Yuyan Han, Jiabao Yi, Dongchen Qi, Xiaojiang Yu, Mark B. H. Breese, Xin Zhang, Haojiang Wu, Hongyu Chen, Hongjun Xiangb, Chengbao Jiang, Zhiqi Liu

Abstract: The coupling of optical and electronic degrees of freedom together with quantum confinement in low-dimensional electron systems is particularly interesting for achieving exotic functionalities in strongly correlated oxide electronics. Recently, high room-temperature mobility has been achieved for a large bandgap transparent oxide - BaSnO$_3$ upon extrinsic La or Sb doping, which has excited signif… ▽ More The coupling of optical and electronic degrees of freedom together with quantum confinement in low-dimensional electron systems is particularly interesting for achieving exotic functionalities in strongly correlated oxide electronics. Recently, high room-temperature mobility has been achieved for a large bandgap transparent oxide - BaSnO$_3$ upon extrinsic La or Sb doping, which has excited significant research attention. In this work, we report the observation of room-temperature ferromagnetism in BaSnO$_3$ thin films and the realization of a two-dimensional electron gas (2DEG) on the surface of transparent BaSnO$_3$ via oxygen vacancy creation, which exhibits a high carrier density of $\sim 7.72*10^{14} /{\rm cm}^2$ and a high room-temperature mobility of ~18 cm$^2$/V/s. Such a 2DEG is rather sensitive to strain and a less than 0.1% in-plane biaxial compressive strain leads to a giant resistance enhancement of 350% (more than 540 kOhm/Square) at room temperature. Thus, this work creates a new path to exploring the physics of low-dimensional oxide electronics and devices applicable at room temperature. △ Less

Submitted 1 January, 2021; v1 submitted 25 November, 2020; originally announced November 2020.

Comments: 28 pages, 7 figures

Journal ref: Acta Materialia 204, 116516 (2021)

Deep Time Delay Neural Network for Speech Enhancement with Full Data Learning

Authors: Cunhang Fan, Bin Liu, Jianhua Tao, Jiangyan Yi, Zhengqi Wen, Leichao Song

Abstract: Recurrent neural networks (RNNs) have shown significant improvements in recent years for speech enhancement. However, the model complexity and inference time cost of RNNs are much higher than deep feed-forward neural networks (DNNs). Therefore, these limit the applications of speech enhancement. This paper proposes a deep time delay neural network (TDNN) for speech enhancement with full data learn… ▽ More Recurrent neural networks (RNNs) have shown significant improvements in recent years for speech enhancement. However, the model complexity and inference time cost of RNNs are much higher than deep feed-forward neural networks (DNNs). Therefore, these limit the applications of speech enhancement. This paper proposes a deep time delay neural network (TDNN) for speech enhancement with full data learning. The TDNN has excellent potential for capturing long range temporal contexts, which utilizes a modular and incremental design. Besides, the TDNN preserves the feed-forward structure so that its inference cost is comparable to standard DNN. To make full use of the training data, we propose a full data learning method for speech enhancement. More specifically, we not only use the noisy-to-clean (input-to-target) to train the enhanced model, but also the clean-to-clean and noise-to-silence data. Therefore, all of the training data can be used to train the enhanced model. Our experiments are conducted on TIMIT dataset. Experimental results show that our proposed method could achieve a better performance than DNN and comparable even better performance than BLSTM. Meanwhile, compared with the BLSTM, the proposed method drastically reduce the inference time. △ Less

Submitted 11 November, 2020; originally announced November 2020.

Comments: Accepted by ISCSLP 2021

Gated Recurrent Fusion with Joint Training Framework for Robust End-to-End Speech Recognition

Authors: Cunhang Fan, Jiangyan Yi, Jianhua Tao, Zhengkun Tian, Bin Liu, Zhengqi Wen

Abstract: The joint training framework for speech enhancement and recognition methods have obtained quite good performances for robust end-to-end automatic speech recognition (ASR). However, these methods only utilize the enhanced feature as the input of the speech recognition component, which are affected by the speech distortion problem. In order to address this problem, this paper proposes a gated recurr… ▽ More The joint training framework for speech enhancement and recognition methods have obtained quite good performances for robust end-to-end automatic speech recognition (ASR). However, these methods only utilize the enhanced feature as the input of the speech recognition component, which are affected by the speech distortion problem. In order to address this problem, this paper proposes a gated recurrent fusion (GRF) method with joint training framework for robust end-to-end ASR. The GRF algorithm is used to dynamically combine the noisy and enhanced features. Therefore, the GRF can not only remove the noise signals from the enhanced features, but also learn the raw fine structures from the noisy features so that it can alleviate the speech distortion. The proposed method consists of speech enhancement, GRF and speech recognition. Firstly, the mask based speech enhancement network is applied to enhance the input speech. Secondly, the GRF is applied to address the speech distortion problem. Thirdly, to improve the performance of ASR, the state-of-the-art speech transformer algorithm is used as the speech recognition component. Finally, the joint training framework is utilized to optimize these three components, simultaneously. Our experiments are conducted on an open-source Mandarin speech corpus called AISHELL-1. Experimental results show that the proposed method achieves the relative character error rate (CER) reduction of 10.04\% over the conventional joint enhancement and transformer method only using the enhanced features. Especially for the low signal-to-noise ratio (0 dB), our proposed method can achieves better performances with 12.67\% CER reduction, which suggests the potential of our proposed method. △ Less

Submitted 9 November, 2020; originally announced November 2020.

Comments: Accepted by IEEE/ACM Transactions on Audio, Speech, and Language Processing

Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System

Authors: Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, Xiangnan He

Abstract: The general aim of the recommender system is to provide personalized suggestions to users, which is opposed to suggesting popular items. However, the normal training paradigm, i.e., fitting a recommender model to recover the user behavior data with pointwise or pairwise loss, makes the model biased towards popular items. This results in the terrible Matthew effect, making popular items be more fre… ▽ More The general aim of the recommender system is to provide personalized suggestions to users, which is opposed to suggesting popular items. However, the normal training paradigm, i.e., fitting a recommender model to recover the user behavior data with pointwise or pairwise loss, makes the model biased towards popular items. This results in the terrible Matthew effect, making popular items be more frequently recommended and become even more popular. Existing work addresses this issue with Inverse Propensity Weighting (IPW), which decreases the impact of popular items on the training and increases the impact of long-tail items. Although theoretically sound, IPW methods are highly sensitive to the weighting strategy, which is notoriously difficult to tune. In this work, we explore the popularity bias issue from a novel and fundamental perspective -- cause-effect. We identify that popularity bias lies in the direct effect from the item node to the ranking score, such that an item's intrinsic property is the cause of mistakenly assigning it a higher ranking score. To eliminate popularity bias, it is essential to answer the counterfactual question that what the ranking score would be if the model only uses item property. To this end, we formulate a causal graph to describe the important cause-effect relations in the recommendation process. During training, we perform multi-task learning to achieve the contribution of each cause; during testing, we perform counterfactual inference to remove the effect of item popularity. Remarkably, our solution amends the learning process of recommendation which is agnostic to a wide range of models -- it can be easily implemented in existing methods. We demonstrate it on Matrix Factorization (MF) and LightGCN [20]. Experiments on five real-world datasets demonstrate the effectiveness of our method. △ Less

Submitted 7 June, 2021; v1 submitted 29 October, 2020; originally announced October 2020.

Comments: To Appear in SIGKDD 2021