Showing 151–200 of 516 results for author: Fang, H

A Novel Modular, Reconfigurable Battery Energy Storage System: Design, Control, and Experimentation

Authors: Amir Farakhor, Di Wu, Yebin Wang, Huazhen Fang

Abstract: This paper presents a novel modular, reconfigurable battery energy storage system. The proposed design is characterized by a tight integration of reconfigurable power switches and DC/DC converters. This characteristic enables isolation of faulty cells from the system and allows fine power control for individual cells toward optimal system-level performance. An optimal power management approach is… ▽ More This paper presents a novel modular, reconfigurable battery energy storage system. The proposed design is characterized by a tight integration of reconfigurable power switches and DC/DC converters. This characteristic enables isolation of faulty cells from the system and allows fine power control for individual cells toward optimal system-level performance. An optimal power management approach is developed to extensively exploit the merits of the proposed design. Based on receding-horizon convex optimization, this approach aims to minimize the total power losses in charging/discharging while allocating the power in line with each cell's condition to achieve state-of-charge (SoC) and temperature balancing. By appropriate design, the approach manages to regulate the power of a cell across its full SoC range and guarantees the feasibility of the optimization problem. We perform extensive simulations and further develop a lab-scale prototype to validate the proposed system design and power management approach. △ Less

Submitted 12 January, 2023; originally announced January 2023.

Comments: This work is published in the IEEE Transactions on Transportation Electrification

A Brain-inspired Memory Transformation based Differentiable Neural Computer for Reasoning-based Question Answering

Authors: Yao Liang, Hongjian Fang, Yi Zeng, Feifei Zhao

Abstract: Reasoning and question answering as a basic cognitive function for humans, is nevertheless a great challenge for current artificial intelligence. Although the Differentiable Neural Computer (DNC) model could solve such problems to a certain extent, the development is still limited by its high algorithm complexity, slow convergence speed, and poor test robustness. Inspired by the learning and memor… ▽ More Reasoning and question answering as a basic cognitive function for humans, is nevertheless a great challenge for current artificial intelligence. Although the Differentiable Neural Computer (DNC) model could solve such problems to a certain extent, the development is still limited by its high algorithm complexity, slow convergence speed, and poor test robustness. Inspired by the learning and memory mechanism of the brain, this paper proposed a Memory Transformation based Differentiable Neural Computer (MT-DNC) model. MT-DNC incorporates working memory and long-term memory into DNC, and realizes the autonomous transformation of acquired experience between working memory and long-term memory, thereby helping to effectively extract acquired knowledge to improve reasoning ability. Experimental results on bAbI question answering task demonstrated that our proposed method achieves superior performance and faster convergence speed compared to other existing DNN and DNC models. Ablation studies also indicated that the memory transformation from working memory to long-term memory plays essential role in improving the robustness and stability of reasoning. This work explores how brain-inspired memory transformation can be integrated and applied to complex intelligent dialogue and reasoning systems. △ Less

Submitted 7 January, 2023; originally announced January 2023.

Tracing the Origin of Adversarial Attack for Forensic Investigation and Deterrence

Authors: Han Fang, Jiyi Zhang, Yupeng Qiu, Ke Xu, Chengfang Fang, Ee-Chien Chang

Abstract: Deep neural networks are vulnerable to adversarial attacks. In this paper, we take the role of investigators who want to trace the attack and identify the source, that is, the particular model which the adversarial examples are generated from. Techniques derived would aid forensic investigation of attack incidents and serve as deterrence to potential attacks. We consider the buyers-seller setting… ▽ More Deep neural networks are vulnerable to adversarial attacks. In this paper, we take the role of investigators who want to trace the attack and identify the source, that is, the particular model which the adversarial examples are generated from. Techniques derived would aid forensic investigation of attack incidents and serve as deterrence to potential attacks. We consider the buyers-seller setting where a machine learning model is to be distributed to various buyers and each buyer receives a slightly different copy with same functionality. A malicious buyer generates adversarial examples from a particular copy $\mathcal{M}_i$ and uses them to attack other copies. From these adversarial examples, the investigator wants to identify the source $\mathcal{M}_i$. To address this problem, we propose a two-stage separate-and-trace framework. The model separation stage generates multiple copies of a model for a same classification task. This process injects unique characteristics into each copy so that adversarial examples generated have distinct and traceable features. We give a parallel structure which embeds a ``tracer'' in each copy, and a noise-sensitive training loss to achieve this goal. The tracing stage takes in adversarial examples and a few candidate models, and identifies the likely source. Based on the unique features induced by the noise-sensitive loss function, we could effectively trace the potential adversarial copy by considering the output logits from each tracer. Empirical results show that it is possible to trace the origin of the adversarial example and the mechanism can be applied to a wide range of architectures and datasets. △ Less

Submitted 30 December, 2022; originally announced January 2023.

Surveillance Face Anti-spoofing

Authors: Hao Fang, Ajian Liu, Jun Wan, Sergio Escalera, Chenxu Zhao, Xu Zhang, Stan Z. Li, Zhen Lei

Abstract: Face Anti-spoofing (FAS) is essential to secure face recognition systems from various physical attacks. However, recent research generally focuses on short-distance applications (i.e., phone unlocking) while lacking consideration of long-distance scenes (i.e., surveillance security checks). In order to promote relevant research and fill this gap in the community, we collect a large-scale Surveilla… ▽ More Face Anti-spoofing (FAS) is essential to secure face recognition systems from various physical attacks. However, recent research generally focuses on short-distance applications (i.e., phone unlocking) while lacking consideration of long-distance scenes (i.e., surveillance security checks). In order to promote relevant research and fill this gap in the community, we collect a large-scale Surveillance High-Fidelity Mask (SuHiFiMask) dataset captured under 40 surveillance scenes, which has 101 subjects from different age groups with 232 3D attacks (high-fidelity masks), 200 2D attacks (posters, portraits, and screens), and 2 adversarial attacks. In this scene, low image resolution and noise interference are new challenges faced in surveillance FAS. Together with the SuHiFiMask dataset, we propose a Contrastive Quality-Invariance Learning (CQIL) network to alleviate the performance degradation caused by image quality from three aspects: (1) An Image Quality Variable module (IQV) is introduced to recover image information associated with discrimination by combining the super-resolution network. (2) Using generated sample pairs to simulate quality variance distributions to help contrastive learning strategies obtain robust feature representation under quality variation. (3) A Separate Quality Network (SQN) is designed to learn discriminative features independent of image quality. Finally, a large number of experiments verify the quality of the SuHiFiMask dataset and the superiority of the proposed CQIL. △ Less

Submitted 3 January, 2023; originally announced January 2023.

Comments: 15 pages, 9 figures

Harmonic analysis on the fourfold cover of the space of ordered triangles I: the invariant differentials

Authors: Hanlong Fang, Xiaocheng Li, Yunfeng Zhang

Abstract: Denote by $SL_n(\mathbb R)$ the group of $n\times n$ real matrices with determinant one, $A$ the subgroup consisting of the diagonal matrices with positive entries, and $SL_n(\mathbb R)/A$ the manifold of left cosets $gA$, $g\in SL_n(\mathbb R)$. In this paper, we will be concerned with the harmonic analysis on the homogeneous space $SL_n(\mathbb R)/A$ when $n=3$. In particular, we provide explici… ▽ More Denote by $SL_n(\mathbb R)$ the group of $n\times n$ real matrices with determinant one, $A$ the subgroup consisting of the diagonal matrices with positive entries, and $SL_n(\mathbb R)/A$ the manifold of left cosets $gA$, $g\in SL_n(\mathbb R)$. In this paper, we will be concerned with the harmonic analysis on the homogeneous space $SL_n(\mathbb R)/A$ when $n=3$. In particular, we provide explicit generators and their relations for the algebra of the invariant differential operators. Then we prove that some of the non-central generators are essentially self-adjoint. △ Less

Submitted 2 January, 2023; originally announced January 2023.

MSC Class: 43A85; 22E46; 22E30

AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains

Authors: Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, Cewu Lu

Abstract: As the basis for prehensile manipulation, it is vital to enable robots to grasp as robustly as humans. Our innate grasping system is prompt, accurate, flexible, and continuous across spatial and temporal domains. Few existing methods cover all these properties for robot grasping. In this paper, we propose AnyGrasp for grasp perception to enable robots these abilities using a parallel gripper. Spec… ▽ More As the basis for prehensile manipulation, it is vital to enable robots to grasp as robustly as humans. Our innate grasping system is prompt, accurate, flexible, and continuous across spatial and temporal domains. Few existing methods cover all these properties for robot grasping. In this paper, we propose AnyGrasp for grasp perception to enable robots these abilities using a parallel gripper. Specifically, we develop a dense supervision strategy with real perception and analytic labels in the spatial-temporal domain. Additional awareness of objects' center-of-mass is incorporated into the learning process to help improve grasping stability. Utilization of grasp correspondence across observations enables dynamic grasp tracking. Our model can efficiently generate accurate, 7-DoF, dense, and temporally-smooth grasp poses and works robustly against large depth-sensing noise. Using AnyGrasp, we achieve a 93.3% success rate when clearing bins with over 300 unseen objects, which is on par with human subjects under controlled conditions. Over 900 mean-picks-per-hour is reported on a single-arm system. For dynamic grasping, we demonstrate catching swimming robot fish in the water. Our project page is at https://graspnet.net/anygrasp.html △ Less

Submitted 6 June, 2023; v1 submitted 16 December, 2022; originally announced December 2022.

Comments: Paper accepted to T-RO. Project page is at https://graspnet.net/anygrasp.html

A Generic Reinforced Explainable Framework with Knowledge Graph for Session-based Recommendation

Authors: Huizi Wu, Hui Fang, Zhu Sun, Cong Geng, Xinyu Kong, Yew-Soon Ong

Abstract: Session-based recommendation (SR) has gained increasing attention in recent years. Quite a great amount of studies have been devoted to designing complex algorithms to improve recommendation performance, where deep learning methods account for the majority. However, most of these methods are black-box ones and ignore to provide moderate explanations to facilitate users' understanding, which thus m… ▽ More Session-based recommendation (SR) has gained increasing attention in recent years. Quite a great amount of studies have been devoted to designing complex algorithms to improve recommendation performance, where deep learning methods account for the majority. However, most of these methods are black-box ones and ignore to provide moderate explanations to facilitate users' understanding, which thus might lead to lowered user satisfaction and reduced system revenues. Therefore, in our study, we propose a generic Reinforced Explainable framework with Knowledge graph for Session-based recommendation (i.e., REKS), which strives to improve the existing black-box SR models (denoted as non-explainable ones) with Markov decision process. In particular, we construct a knowledge graph with session behaviors and treat SR models as part of the policy network of Markov decision process. Based on our particularly designed state vector, reward strategy, and loss function, the reinforcement learning (RL)-based framework not only achieves improved recommendation accuracy, but also provides appropriate explanations at the same time. Finally, we instantiate the REKS in five representative, state-of-the-art SR models (i.e., GRU4REC, NARM, SR-GNN, GCSAN, BERT4REC), whereby extensive experiments towards these methods on four datasets demonstrate the effectiveness of our framework on both recommendation and explanation tasks. △ Less

Submitted 15 December, 2022; v1 submitted 13 December, 2022; originally announced December 2022.

Comments: 13 pages, 10 figures,accepted by ICDE 2023

Estimator: An Effective and Scalable Framework for Transportation Mode Classification over Trajectories

Authors: Danlei Hu, Ziquan Fang, Hanxi Fang, Tianyi Li, Chunhui Shen, Lu Chen, Yunjun Gao

Abstract: Transportation mode classification, the process of predicting the class labels of moving objects transportation modes, has been widely applied to a variety of real world applications, such as traffic management, urban computing, and behavior study. However, existing studies of transportation mode classification typically extract the explicit features of trajectory data but fail to capture the impl… ▽ More Transportation mode classification, the process of predicting the class labels of moving objects transportation modes, has been widely applied to a variety of real world applications, such as traffic management, urban computing, and behavior study. However, existing studies of transportation mode classification typically extract the explicit features of trajectory data but fail to capture the implicit features that affect the classification performance. In addition, most of the existing studies also prefer to apply RNN-based models to embed trajectories, which is only suitable for classifying small-scale data. To tackle the above challenges, we propose an effective and scalable framework for transportation mode classification over GPS trajectories, abbreviated Estimator. Estimator is established on a developed CNN-TCN architecture, which is capable of leveraging the spatial and temporal hidden features of trajectories to achieve high effectiveness and efficiency. Estimator partitions the entire traffic space into disjointed spatial regions according to traffic conditions, which enhances the scalability significantly and thus enables parallel transportation classification. Extensive experiments using eight public real-life datasets offer evidence that Estimator i) achieves superior model effectiveness (i.e., 99% Accuracy and 0.98 F1-score), which outperforms state-of-the-arts substantially; ii) exhibits prominent model efficiency, and obtains 7-40x speedups up over state-of-the-arts learning-based methods; and iii) shows high model scalability and robustness that enables large-scale classification analytics. △ Less

Submitted 11 December, 2022; originally announced December 2022.

Comments: 12 pages, 8 figures

Uncertainty Estimation in Deep Speech Enhancement Using Complex Gaussian Mixture Models

Authors: Huajian Fang, Timo Gerkmann

Abstract: Single-channel deep speech enhancement approaches often estimate a single multiplicative mask to extract clean speech without a measure of its accuracy. Instead, in this work, we propose to quantify the uncertainty associated with clean speech estimates in neural network-based speech enhancement. Predictive uncertainty is typically categorized into aleatoric uncertainty and epistemic uncertainty.… ▽ More Single-channel deep speech enhancement approaches often estimate a single multiplicative mask to extract clean speech without a measure of its accuracy. Instead, in this work, we propose to quantify the uncertainty associated with clean speech estimates in neural network-based speech enhancement. Predictive uncertainty is typically categorized into aleatoric uncertainty and epistemic uncertainty. The former accounts for the inherent uncertainty in data and the latter corresponds to the model uncertainty. Aiming for robust clean speech estimation and efficient predictive uncertainty quantification, we propose to integrate statistical complex Gaussian mixture models (CGMMs) into a deep speech enhancement framework. More specifically, we model the dependency between input and output stochastically by means of a conditional probability density and train a neural network to map the noisy input to the full posterior distribution of clean speech, modeled as a mixture of multiple complex Gaussian components. Experimental results on different datasets show that the proposed algorithm effectively captures predictive uncertainty and that combining powerful statistical models and deep learning also delivers a superior speech enhancement performance. △ Less

Submitted 15 May, 2023; v1 submitted 9 December, 2022; originally announced December 2022.

Comments: ©2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Journal ref: ICASSP 2023 - IEEE International Conference on Acoustics, Speech and Signal Processing

DeAR: A Deep-learning-based Audio Re-recording Resilient Watermarking

Authors: Chang Liu, Jie Zhang, Han Fang, Zehua Ma, Weiming Zhang, Nenghai Yu

Abstract: Audio watermarking is widely used for leaking source tracing. The robustness of the watermark determines the traceability of the algorithm. With the development of digital technology, audio re-recording (AR) has become an efficient and covert means to steal secrets. AR process could drastically destroy the watermark signal while preserving the original information. This puts forward a new requirem… ▽ More Audio watermarking is widely used for leaking source tracing. The robustness of the watermark determines the traceability of the algorithm. With the development of digital technology, audio re-recording (AR) has become an efficient and covert means to steal secrets. AR process could drastically destroy the watermark signal while preserving the original information. This puts forward a new requirement for audio watermarking at this stage, that is, to be robust to AR distortions. Unfortunately, none of the existing algorithms can effectively resist AR attacks due to the complexity of the AR process. To address this limitation, this paper proposes DeAR, a deep-learning-based audio re-recording resistant watermarking. Inspired by DNN-based image watermarking, we pioneer a deep learning framework for audio carriers, based on which the watermark signal can be effectively embedded and extracted. Meanwhile, in order to resist the AR attack, we delicately analyze the distortions that occurred in the AR process and design the corresponding distortion layer to cooperate with the proposed watermarking framework. Extensive experiments show that the proposed algorithm can resist not only common electronic channel distortions but also AR distortions. Under the premise of high-quality embedding (SNR=25.86dB), in the case of a common re-recording distance (20cm), the algorithm can effectively achieve an average bit recovery accuracy of 98.55%. △ Less

Submitted 3 April, 2023; v1 submitted 5 December, 2022; originally announced December 2022.

Comments: Accepted by AAAI2023

Multi-rater Prism: Learning self-calibrated medical image segmentation from multiple raters

Authors: Junde Wu, Huihui Fang, Yehui Yang, Yuanpei Liu, Jing Gao, Lixin Duan, Weihua Yang, Yanwu Xu

Abstract: In medical image segmentation, it is often necessary to collect opinions from multiple experts to make the final decision. This clinical routine helps to mitigate individual bias. But when data is multiply annotated, standard deep learning models are often not applicable. In this paper, we propose a novel neural network framework, called Multi-Rater Prism (MrPrism) to learn the medical image segme… ▽ More In medical image segmentation, it is often necessary to collect opinions from multiple experts to make the final decision. This clinical routine helps to mitigate individual bias. But when data is multiply annotated, standard deep learning models are often not applicable. In this paper, we propose a novel neural network framework, called Multi-Rater Prism (MrPrism) to learn the medical image segmentation from multiple labels. Inspired by the iterative half-quadratic optimization, the proposed MrPrism will combine the multi-rater confidences assignment task and calibrated segmentation task in a recurrent manner. In this recurrent process, MrPrism can learn inter-observer variability taking into account the image semantic properties, and finally converges to a self-calibrated segmentation result reflecting the inter-observer agreement. Specifically, we propose Converging Prism (ConP) and Diverging Prism (DivP) to process the two tasks iteratively. ConP learns calibrated segmentation based on the multi-rater confidence maps estimated by DivP. DivP generates multi-rater confidence maps based on the segmentation masks estimated by ConP. The experimental results show that by recurrently running ConP and DivP, the two tasks can achieve mutual improvement. The final converged segmentation result of MrPrism outperforms state-of-the-art (SOTA) strategies on a wide range of medical image segmentation tasks. △ Less

Submitted 1 December, 2022; originally announced December 2022.

Decentralized Matrix Factorization with Heterogeneous Differential Privacy

Authors: Wentao Hu, Hui Fang

Abstract: Conventional matrix factorization relies on centralized collection of users' data for recommendation, which might introduce an increased risk of privacy leakage especially when the recommender is untrusted. Existing differentially private matrix factorization methods either assume the recommender is trusted, or can only provide a uniform level of privacy protection for all users and items with unt… ▽ More Conventional matrix factorization relies on centralized collection of users' data for recommendation, which might introduce an increased risk of privacy leakage especially when the recommender is untrusted. Existing differentially private matrix factorization methods either assume the recommender is trusted, or can only provide a uniform level of privacy protection for all users and items with untrusted recommender. In this paper, we propose a novel Heterogeneous Differentially Private Matrix Factorization algorithm (denoted as HDPMF) for untrusted recommender. To the best of our knowledge, we are the first to achieve heterogeneous differential privacy for decentralized matrix factorization in untrusted recommender scenario. Specifically, our framework uses modified stretching mechanism with an innovative rescaling scheme to achieve better trade off between privacy and accuracy. Meanwhile, by allocating privacy budget properly, we can capture homogeneous privacy preference within a user/item but heterogeneous privacy preference across different users/items. Theoretical analysis confirms that HDPMF renders rigorous privacy guarantee, and exhaustive experiments demonstrate its superiority especially in strong privacy guarantee, high dimension model and sparse dataset scenario. △ Less

Submitted 16 September, 2023; v1 submitted 1 December, 2022; originally announced December 2022.

Comments: Accepted by the 22nd IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom-2023)

ExpNet: A unified network for Expert-Level Classification

Authors: Junde Wu, Huihui Fang, Yehui Yang, Yu Zhang, Haoyi Xiong, Huazhu Fu, Yanwu Xu

Abstract: Different from the general visual classification, some classification tasks are more challenging as they need the professional categories of the images. In the paper, we call them expert-level classification. Previous fine-grained vision classification (FGVC) has made many efforts on some of its specific sub-tasks. However, they are difficult to expand to the general cases which rely on the compre… ▽ More Different from the general visual classification, some classification tasks are more challenging as they need the professional categories of the images. In the paper, we call them expert-level classification. Previous fine-grained vision classification (FGVC) has made many efforts on some of its specific sub-tasks. However, they are difficult to expand to the general cases which rely on the comprehensive analysis of part-global correlation and the hierarchical features interaction. In this paper, we propose Expert Network (ExpNet) to address the unique challenges of expert-level classification through a unified network. In ExpNet, we hierarchically decouple the part and context features and individually process them using a novel attentive mechanism, called Gaze-Shift. In each stage, Gaze-Shift produces a focal-part feature for the subsequent abstraction and memorizes a context-related embedding. Then we fuse the final focal embedding with all memorized context-related embedding to make the prediction. Such an architecture realizes the dual-track processing of partial and global information and hierarchical feature interactions. We conduct the experiments over three representative expert-level classification tasks: FGVC, disease classification, and artwork attributes classification. In these experiments, superior performance of our ExpNet is observed comparing to the state-of-the-arts in a wide range of fields, indicating the effectiveness and generalization of our ExpNet. The code will be made publicly available. △ Less

Submitted 29 November, 2022; originally announced November 2022.

Newly discovered Ca II absorbers in the early universe: statistics, element abundances and dust

Authors: Hannah Fang, Iona Xia, Jian Ge, Kevin Willis, Yinan Zhao

Abstract: We report discoveries of 165 new quasar Ca II absorbers from the Sloan Digital Sky Survey (SDSS) Data Release 7 and 12. Our Ca II rest frame equivalent width distribution supports the weak and strong subpopulations, split at ${W}^{\lambda3934}_{0}=0.7$Å. Comparison of both populations' dust depletion shows clear consistency for weak absorber association with halo-type gas in the Milky Way (MW) whi… ▽ More We report discoveries of 165 new quasar Ca II absorbers from the Sloan Digital Sky Survey (SDSS) Data Release 7 and 12. Our Ca II rest frame equivalent width distribution supports the weak and strong subpopulations, split at ${W}^{\lambda3934}_{0}=0.7$Å. Comparison of both populations' dust depletion shows clear consistency for weak absorber association with halo-type gas in the Milky Way (MW) while strong absorbers have environments consistent with halo and disc-type gas. We probed our high redshift Ca II absorbers for 2175Å dust bumps, discovering 12 2175Å dust absorbers (2DAs). This clearly shows that some Ca II absorbers follow the Large Magellanic Cloud (LMC) extinction law rather than the Small Magellanic Cloud extinction law. About 33% of our strong Ca II absorbers exhibit the 2175Å dust bump while only 6% of weak Ca II absorbers show this bump. 2DA detection further supports the theory that strong Ca II absorbers are associated with disk components and are dustier than the weak population. Comparing average Ca II absorber dust depletion patterns to that of Damped Lyα Absorbers (DLAs), Mg II absorbers, and 2DAs shows that Ca II absorbers generally have environments with more dust than DLAs and Mg II absorbers, but less dust than 2DAs. Comparing 2175Å dust bump strengths from different samples and also the MW and LMC, the bump strength appears to grow stronger as the redshift decreases, indicating dust growth and the global chemical enrichment of galaxies in the universe over time. △ Less

Submitted 23 November, 2022; originally announced November 2022.

Comments: 17 pages, 24 figures, 5 tables; Acccepted for publication in the Monthly Notices of the Royal Astronomical Society (MNRAS)

Open-Domain Conversational Question Answering with Historical Answers

Authors: Hung-Chieh Fang, Kuo-Han Hung, Chao-Wei Huang, Yun-Nung Chen

Abstract: Open-domain conversational question answering can be viewed as two tasks: passage retrieval and conversational question answering, where the former relies on selecting candidate passages from a large corpus and the latter requires better understanding of a question with contexts to predict the answers. This paper proposes ConvADR-QA that leverages historical answers to boost retrieval performance… ▽ More Open-domain conversational question answering can be viewed as two tasks: passage retrieval and conversational question answering, where the former relies on selecting candidate passages from a large corpus and the latter requires better understanding of a question with contexts to predict the answers. This paper proposes ConvADR-QA that leverages historical answers to boost retrieval performance and further achieves better answering performance. In our proposed framework, the retrievers use a teacher-student framework to reduce noises from previous turns. Our experiments on the benchmark dataset, OR-QuAC, demonstrate that our model outperforms existing baselines in both extractive and generative reader settings, well justifying the effectiveness of historical answers for open-domain conversational question answering. △ Less

Submitted 17 November, 2022; originally announced November 2022.

Comments: AACL-IJCNLP 2022

Discovering A Variety of Objects in Spatio-Temporal Human-Object Interactions

Authors: Yong-Lu Li, Hongwei Fan, Zuoyu Qiu, Yiming Dou, Liang Xu, Hao-Shu Fang, Peiyang Guo, Haisheng Su, Dongliang Wang, Wei Wu, Cewu Lu

Abstract: Spatio-temporal Human-Object Interaction (ST-HOI) detection aims at detecting HOIs from videos, which is crucial for activity understanding. In daily HOIs, humans often interact with a variety of objects, e.g., holding and touching dozens of household items in cleaning. However, existing whole body-object interaction video benchmarks usually provide limited object classes. Here, we introduce a new… ▽ More Spatio-temporal Human-Object Interaction (ST-HOI) detection aims at detecting HOIs from videos, which is crucial for activity understanding. In daily HOIs, humans often interact with a variety of objects, e.g., holding and touching dozens of household items in cleaning. However, existing whole body-object interaction video benchmarks usually provide limited object classes. Here, we introduce a new benchmark based on AVA: Discovering Interacted Objects (DIO) including 51 interactions and 1,000+ objects. Accordingly, an ST-HOI learning task is proposed expecting vision systems to track human actors, detect interactions and simultaneously discover interacted objects. Even though today's detectors/trackers excel in object detection/tracking tasks, they perform unsatisfied to localize diverse/unseen objects in DIO. This profoundly reveals the limitation of current vision systems and poses a great challenge. Thus, how to leverage spatio-temporal cues to address object discovery is explored, and a Hierarchical Probe Network (HPN) is devised to discover interacted objects utilizing hierarchical spatio-temporal human/context cues. In extensive experiments, HPN demonstrates impressive performance. Data and code are available at https://github.com/DirtyHarryLYL/HAKE-AVA. △ Less

Submitted 17 November, 2022; v1 submitted 14 November, 2022; originally announced November 2022.

Comments: Techniqual report. A part of the HAKE project. Project: https://github.com/DirtyHarryLYL/HAKE-AVA

Search for boosted keV-MeV light dark matter particles from evaporating primordial black holes at the CDEX-10 experiment

Authors: Z. H. Zhang, L. T. Yang, Q. Yue, K. J. Kang, Y. J. Li, H. P. An, Greeshma C., J. P. Chang, Y. H. Chen, J. P. Cheng, W. H. Dai, Z. Deng, C. H. Fang, X. P. Geng, H. Gong, Q. J. Guo, X. Y. Guo, L. He, S. M. He, J. W. Hu, H. X. Huang, T. C. Huang, H. T. Jia, X. Jiang, S. Karmakar , et al. (59 additional authors not shown)

Abstract: We present novel constraints on boosted light dark matter particles (denoted as ``$χ$'') from evaporating primordial black holes (PBHs) using 205.4 kg$\cdot$day data from the China Jinping Underground Laboratory's CDEX-10 p-type point contact germanium detector with a 160 eVee analysis threshold. $χ$ from PBHs with masses ranging from 1$\times$10$^{15}$ g to 7$\times$10$^{16}$ g are searched in th… ▽ More We present novel constraints on boosted light dark matter particles (denoted as ``$χ$'') from evaporating primordial black holes (PBHs) using 205.4 kg$\cdot$day data from the China Jinping Underground Laboratory's CDEX-10 p-type point contact germanium detector with a 160 eVee analysis threshold. $χ$ from PBHs with masses ranging from 1$\times$10$^{15}$ g to 7$\times$10$^{16}$ g are searched in this work. In the presence of PBH abundance compatible with present bounds, our result excludes the $χ$-nucleon elastic-scattering cross section region from 3.4$\times$10$^{-32}$ cm$^{2}$ to 2.3$\times$10$^{-29}$ cm$^{2}$ for $χ$ of 1 keV to 24 MeV from PBHs with masses of 5$\times$10$^{15}$ g, as well as from 1.1$\times$10$^{-28}$ cm$^{2}$ to 7.6$\times$10$^{-28}$ cm$^{2}$ for $χ$ of 1 keV to 0.6 MeV from PBHs with masses of 7$\times$10$^{16}$ g. If the $χ$-nucleon elastic-scattering cross section can be determined in the future, the abundance of PBHs may be severely constrained by $χ$ evaporation. With the lower threshold (160 eVee) of the CDEX-10 experiment compared to the previously used experiments, this work allows for a better reach at soft spectra produced by heavier PBHs, which demonstrates the vast potential of such a technical route to pursue $χ$ from larger PBHs with a low threshold. △ Less

Submitted 7 September, 2023; v1 submitted 14 November, 2022; originally announced November 2022.

Comments: 8 pages, 6 figures. Version updated to match PRD version

Journal ref: Phys. Rev. D 109, 052006 (2023)

Watermarking in Secure Federated Learning: A Verification Framework Based on Client-Side Backdooring

Authors: Wenyuan Yang, Shuo Shao, Yue Yang, Xiyao Liu, Ximeng Liu, Zhihua Xia, Gerald Schaefer, Hui Fang

Abstract: Federated learning (FL) allows multiple participants to collaboratively build deep learning (DL) models without directly sharing data. Consequently, the issue of copyright protection in FL becomes important since unreliable participants may gain access to the jointly trained model. Application of homomorphic encryption (HE) in secure FL framework prevents the central server from accessing plaintex… ▽ More Federated learning (FL) allows multiple participants to collaboratively build deep learning (DL) models without directly sharing data. Consequently, the issue of copyright protection in FL becomes important since unreliable participants may gain access to the jointly trained model. Application of homomorphic encryption (HE) in secure FL framework prevents the central server from accessing plaintext models. Thus, it is no longer feasible to embed the watermark at the central server using existing watermarking schemes. In this paper, we propose a novel client-side FL watermarking scheme to tackle the copyright protection issue in secure FL with HE. To our best knowledge, it is the first scheme to embed the watermark to models under the Secure FL environment. We design a black-box watermarking scheme based on client-side backdooring to embed a pre-designed trigger set into an FL model by a gradient-enhanced embedding method. Additionally, we propose a trigger set construction mechanism to ensure the watermark cannot be forged. Experimental results demonstrate that our proposed scheme delivers outstanding protection performance and robustness against various watermark removal attacks and ambiguity attack. △ Less

Submitted 3 March, 2023; v1 submitted 14 November, 2022; originally announced November 2022.

BattX: An Equivalent Circuit Model for Lithium-Ion Batteries Over Broad Current Ranges

Authors: Nikhil Biju, Huazhen Fang

Abstract: Advanced battery management is to lithium-ion battery systems as the brain is to the human body. Its performance rests on the use of battery models that are both fast and accurate. However, mainstream equivalent circuit models and electrochemical models have yet to meet this need well, due to struggle with either predictive accuracy or computational complexity. This problem has acquired urgency as… ▽ More Advanced battery management is to lithium-ion battery systems as the brain is to the human body. Its performance rests on the use of battery models that are both fast and accurate. However, mainstream equivalent circuit models and electrochemical models have yet to meet this need well, due to struggle with either predictive accuracy or computational complexity. This problem has acquired urgency as some emerging battery applications running across broad current ranges, e.g., electric vertical take-off and landing aircraft, can hardly find usable models from the literature. Motivated to address the problem, we develop an innovative model in this study. Called BattX, the model is an equivalent circuit model but draws comparisons to a single particle model with electrolyte and thermal dynamics, thus combining their respective merits to be computationally efficient, accurate, and physically interpretable. The model design pivots on leveraging multiple circuits to approximate major electrochemical and physical processes in charging/discharging. Given the model, we develop a multipronged approach to design experiments and identify its parameters in groups from experimental data. Experimental validation proves that the BattX model is capable of accurate voltage prediction for charging/discharging across low to high C-rates. △ Less

Submitted 10 November, 2022; originally announced November 2022.

Comments: 24 pages, 13 figures, 2 tables, and appendix

AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time

Authors: Hao-Shu Fang, Jiefeng Li, Hongyang Tang, Chao Xu, Haoyi Zhu, Yuliang Xiu, Yong-Lu Li, Cewu Lu

Abstract: Accurate whole-body multi-person pose estimation and tracking is an important yet challenging topic in computer vision. To capture the subtle actions of humans for complex behavior analysis, whole-body pose estimation including the face, body, hand and foot is essential over conventional body-only pose estimation. In this paper, we present AlphaPose, a system that can perform accurate whole-body p… ▽ More Accurate whole-body multi-person pose estimation and tracking is an important yet challenging topic in computer vision. To capture the subtle actions of humans for complex behavior analysis, whole-body pose estimation including the face, body, hand and foot is essential over conventional body-only pose estimation. In this paper, we present AlphaPose, a system that can perform accurate whole-body pose estimation and tracking jointly while running in realtime. To this end, we propose several new techniques: Symmetric Integral Keypoint Regression (SIKR) for fast and fine localization, Parametric Pose Non-Maximum-Suppression (P-NMS) for eliminating redundant human detections and Pose Aware Identity Embedding for jointly pose estimation and tracking. During training, we resort to Part-Guided Proposal Generator (PGPG) and multi-domain knowledge distillation to further improve the accuracy. Our method is able to localize whole-body keypoints accurately and tracks humans simultaneously given inaccurate bounding boxes and redundant detections. We show a significant improvement over current state-of-the-art methods in both speed and accuracy on COCO-wholebody, COCO, PoseTrack, and our proposed Halpe-FullBody pose estimation dataset. Our model, source codes and dataset are made publicly available at https://github.com/MVIG-SJTU/AlphaPose. △ Less

Submitted 7 November, 2022; originally announced November 2022.

Comments: Documents for AlphaPose, accepted to TPAMI

A Transformer-Based Substitute Recommendation Model Incorporating Weakly Supervised Customer Behavior Data

Authors: Wenting Ye, Hongfei Yang, Shuai Zhao, Haoyang Fang, Xingjian Shi, Naveen Neppalli

Abstract: The substitute-based recommendation is widely used in E-commerce to provide better alternatives to customers. However, existing research typically uses the customer behavior signals like co-view and view-but-purchase-another to capture the substitute relationship. Despite its intuitive soundness, we find that such an approach might ignore the functionality and characteristics of products. In this… ▽ More The substitute-based recommendation is widely used in E-commerce to provide better alternatives to customers. However, existing research typically uses the customer behavior signals like co-view and view-but-purchase-another to capture the substitute relationship. Despite its intuitive soundness, we find that such an approach might ignore the functionality and characteristics of products. In this paper, we adapt substitute recommendation into language matching problem by taking product title description as model input to consider product functionality. We design a new transformation method to de-noise the signals derived from production data. In addition, we consider multilingual support from the engineering point of view. Our proposed end-to-end transformer-based model achieves both successes from offline and online experiments. The proposed model has been deployed in a large-scale E-commerce website for 11 marketplaces in 6 languages. Our proposed model is demonstrated to increase revenue by 19% based on an online A/B experiment. △ Less

Submitted 8 April, 2023; v1 submitted 4 November, 2022; originally announced November 2022.

Comments: 6 pages, 3 figures, 5 tables, accepted in 2023 SIGIR Industry track (SIGIR Symposium on IR in Practice, SIRIP)

Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler

Authors: Yifang Chen, Karthik Sankararaman, Alessandro Lazaric, Matteo Pirotta, Dmytro Karamshuk, Qifan Wang, Karishma Mandyam, Sinong Wang, Han Fang

Abstract: Active learning with strong and weak labelers considers a practical setting where we have access to both costly but accurate strong labelers and inaccurate but cheap predictions provided by weak labelers. We study this problem in the streaming setting, where decisions must be taken \textit{online}. We design a novel algorithmic template, Weak Labeler Active Cover (WL-AC), that is able to robustly… ▽ More Active learning with strong and weak labelers considers a practical setting where we have access to both costly but accurate strong labelers and inaccurate but cheap predictions provided by weak labelers. We study this problem in the streaming setting, where decisions must be taken \textit{online}. We design a novel algorithmic template, Weak Labeler Active Cover (WL-AC), that is able to robustly leverage the lower quality weak labelers to reduce the query complexity while retaining the desired level of accuracy. Prior active learning algorithms with access to weak labelers learn a difference classifier which predicts where the weak labels differ from strong labelers; this requires the strong assumption of realizability of the difference classifier (Zhang and Chaudhuri,2015). WL-AC bypasses this \textit{realizability} assumption and thus is applicable to many real-world scenarios such as random corrupted weak labels and high dimensional family of difference classifiers (\textit{e.g.,} deep neural nets). Moreover, WL-AC cleverly trades off evaluating the quality with full exploitation of weak labelers, which allows to convert any active learning strategy to one that can leverage weak labelers. We provide an instantiation of this template that achieves the optimal query complexity for any given weak labeler, without knowing its accuracy a-priori. Empirically, we propose an instantiation of the WL-AC template that can be efficiently implemented for large-scale models (\textit{e.g}., deep neural nets) and show its effectiveness on the corrupted-MNIST dataset by significantly reducing the number of labels while keeping the same accuracy as in passive learning. △ Less

Submitted 3 November, 2022; originally announced November 2022.

MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model

Authors: Junde Wu, Rao Fu, Huihui Fang, Yu Zhang, Yehui Yang, Haoyi Xiong, Huiying Liu, Yanwu Xu

Abstract: Diffusion probabilistic model (DPM) recently becomes one of the hottest topic in computer vision. Its image generation application such as Imagen, Latent Diffusion Models and Stable Diffusion have shown impressive generation capabilities, which aroused extensive discussion in the community. Many recent studies also found it is useful in many other vision tasks, like image deblurring, super-resolut… ▽ More Diffusion probabilistic model (DPM) recently becomes one of the hottest topic in computer vision. Its image generation application such as Imagen, Latent Diffusion Models and Stable Diffusion have shown impressive generation capabilities, which aroused extensive discussion in the community. Many recent studies also found it is useful in many other vision tasks, like image deblurring, super-resolution and anomaly detection. Inspired by the success of DPM, we propose the first DPM based model toward general medical image segmentation tasks, which we named MedSegDiff. In order to enhance the step-wise regional attention in DPM for the medical image segmentation, we propose dynamic conditional encoding, which establishes the state-adaptive conditions for each sampling step. We further propose Feature Frequency Parser (FF-Parser), to eliminate the negative effect of high-frequency noise component in this process. We verify MedSegDiff on three medical segmentation tasks with different image modalities, which are optic cup segmentation over fundus images, brain tumor segmentation over MRI images and thyroid nodule segmentation over ultrasound images. The experimental results show that MedSegDiff outperforms state-of-the-art (SOTA) methods with considerable performance gap, indicating the generalization and effectiveness of the proposed model. Our code is released at https://github.com/WuJunde/MedSegDiff. △ Less

Submitted 14 January, 2023; v1 submitted 1 November, 2022; originally announced November 2022.

Comments: Code is released at: https://github.com/WuJunde/MedSegDiff

SO(3)-Knot States and the Volume Conjecture

Authors: Honghuai Fang

Abstract: We study the alternating subspace of holomorphic sections of a special prequantum line bundle over SU(2)-character variety of torus, and show that it is isomorphic to the projective representation of mapping class group of peripheral torus given by the SO(3) Witten-Chern-Simons theory. We conjecture that the large r asymptotics of $L^2$-norm of SO(3)-knot states via geometric quantization capture… ▽ More We study the alternating subspace of holomorphic sections of a special prequantum line bundle over SU(2)-character variety of torus, and show that it is isomorphic to the projective representation of mapping class group of peripheral torus given by the SO(3) Witten-Chern-Simons theory. We conjecture that the large r asymptotics of $L^2$-norm of SO(3)-knot states via geometric quantization capture the simplicial volume of knot complements. △ Less

Submitted 1 November, 2022; originally announced November 2022.

A Faithful Deep Sensitivity Estimation for Accelerated Magnetic Resonance Imaging

Authors: Zi Wang, Haoming Fang, Chen Qian, Boxuan Shi, Lijun Bao, Liuhong Zhu, Jianjun Zhou, Wenping Wei, Jianzhong Lin, Di Guo, Xiaobo Qu

Abstract: Magnetic resonance imaging (MRI) is an essential diagnostic tool that suffers from prolonged scan time. To alleviate this limitation, advanced fast MRI technology attracts extensive research interests. Recent deep learning has shown its great potential in improving image quality and reconstruction speed. Faithful coil sensitivity estimation is vital for MRI reconstruction. However, most deep learn… ▽ More Magnetic resonance imaging (MRI) is an essential diagnostic tool that suffers from prolonged scan time. To alleviate this limitation, advanced fast MRI technology attracts extensive research interests. Recent deep learning has shown its great potential in improving image quality and reconstruction speed. Faithful coil sensitivity estimation is vital for MRI reconstruction. However, most deep learning methods still rely on pre-estimated sensitivity maps and ignore their inaccuracy, resulting in the significant quality degradation of reconstructed images. In this work, we propose a Joint Deep Sensitivity estimation and Image reconstruction network, called JDSI. During the image artifacts removal, it gradually provides more faithful sensitivity maps with high-frequency information, leading to improved image reconstructions. To understand the behavior of the network, the mutual promotion of sensitivity estimation and image reconstruction is revealed through the visualization of network intermediate results. Results on in vivo datasets and radiologist reader study demonstrate that, for both calibration-based and calibrationless reconstruction, the proposed JDSI achieves the state-of-the-art performance visually and quantitatively, especially when the acceleration factor is high. Additionally, JDSI owns nice robustness to patients and autocalibration signals. △ Less

Submitted 24 December, 2023; v1 submitted 23 October, 2022; originally announced October 2022.

Comments: 12 pages, 13 figures, 7 tables

Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering

Authors: Hao Cheng, Hao Fang, Xiaodong Liu, Jianfeng Gao

Abstract: Given its effectiveness on knowledge-intensive natural language processing tasks, dense retrieval models have become increasingly popular. Specifically, the de-facto architecture for open-domain question answering uses two isomorphic encoders that are initialized from the same pretrained model but separately parameterized for questions and passages. This bi-encoder architecture is parameter-ineffi… ▽ More Given its effectiveness on knowledge-intensive natural language processing tasks, dense retrieval models have become increasingly popular. Specifically, the de-facto architecture for open-domain question answering uses two isomorphic encoders that are initialized from the same pretrained model but separately parameterized for questions and passages. This bi-encoder architecture is parameter-inefficient in that there is no parameter sharing between encoders. Further, recent studies show that such dense retrievers underperform BM25 in various settings. We thus propose a new architecture, Task-aware Specialization for dense Retrieval (TASER), which enables parameter sharing by interleaving shared and specialized blocks in a single encoder. Our experiments on five question answering datasets show that TASER can achieve superior accuracy, surpassing BM25, while using about 60% of the parameters as bi-encoder dense retrievers. In out-of-domain evaluations, TASER is also empirically more robust than bi-encoder dense retrievers. Our code is available at https://github.com/microsoft/taser. △ Less

Submitted 22 May, 2023; v1 submitted 11 October, 2022; originally announced October 2022.

X-NeRF: Explicit Neural Radiance Field for Multi-Scene 360$^{\circ} $ Insufficient RGB-D Views

Authors: Haoyi Zhu, Hao-Shu Fang, Cewu Lu

Abstract: Neural Radiance Fields (NeRFs), despite their outstanding performance on novel view synthesis, often need dense input views. Many papers train one model for each scene respectively and few of them explore incorporating multi-modal data into this problem. In this paper, we focus on a rarely discussed but important setting: can we train one model that can represent multiple scenes, with 360… ▽ More Neural Radiance Fields (NeRFs), despite their outstanding performance on novel view synthesis, often need dense input views. Many papers train one model for each scene respectively and few of them explore incorporating multi-modal data into this problem. In this paper, we focus on a rarely discussed but important setting: can we train one model that can represent multiple scenes, with 360$^\circ $ insufficient views and RGB-D images? We refer insufficient views to few extremely sparse and almost non-overlapping views. To deal with it, X-NeRF, a fully explicit approach which learns a general scene completion process instead of a coordinate-based mapping, is proposed. Given a few insufficient RGB-D input views, X-NeRF first transforms them to a sparse point cloud tensor and then applies a 3D sparse generative Convolutional Neural Network (CNN) to complete it to an explicit radiance field whose volumetric rendering can be conducted fast without running networks during inference. To avoid overfitting, besides common rendering loss, we apply perceptual loss as well as view augmentation through random rotation on point clouds. The proposed methodology significantly out-performs previous implicit methods in our setting, indicating the great potential of proposed problem and approach. Codes and data are available at https://github.com/HaoyiZhu/XNeRF. △ Less

Submitted 11 October, 2022; originally announced October 2022.

Search for exotic interactions of solar neutrinos in the CDEX-10 experiment

Authors: X. P. Geng, L. T. Yang, Q. Yue, K. J. Kang, Y. J. Li, H. P. An, Greeshma C., J. P. Chang, Y. H. Chen, J. P. Cheng, W. H. Dai, Z. Deng, C. H. Fang, H. Gong, Q. J. Guo, X. Y. Guo, L. He, S. M. He, J. W. Hu, H. X. Huang, T. C. Huang, H. T. Jia, X. Jiang, S. Karmakar, H. B. Li , et al. (60 additional authors not shown)

Abstract: We investigate exotic neutrino interactions using the 205.4 kg$\cdot$day dataset from the CDEX-10 experiment at the China Jinping Underground Laboratory. New constraints on the mass and couplings of new gauge bosons are presented. Two nonstandard neutrino interactions are considered: a $U(1)_{B-L}$ gauge-boson-induced interaction between an active neutrino and electron/nucleus, and a dark-photon-i… ▽ More We investigate exotic neutrino interactions using the 205.4 kg$\cdot$day dataset from the CDEX-10 experiment at the China Jinping Underground Laboratory. New constraints on the mass and couplings of new gauge bosons are presented. Two nonstandard neutrino interactions are considered: a $U(1)_{B-L}$ gauge-boson-induced interaction between an active neutrino and electron/nucleus, and a dark-photon-induced interaction between a sterile neutrino and electron/nucleus via kinetic mixing with a photon. This work probes an unexplored parameter space involving sterile neutrino coupling with a dark photon. New laboratory limits are derived on dark photon masses below $1~{\rm eV}/c^{2}$ at some benchmark values of $Δm_{41}^{2}$ and $g^{\prime2}{\rm{sin}}^{2}2θ_{14}$. △ Less

Submitted 2 June, 2023; v1 submitted 4 October, 2022; originally announced October 2022.

Comments: 6 pages, 4 figures. Version updated to match PRD version

Journal ref: Phys. Rev. D 107, 112002 (2023)

Unified Control Framework for Real-Time Interception and Obstacle Avoidance of Fast-Moving Objects with Diffusion Variational Autoencoder

Authors: Apan Dastider, Hao Fang, Mingjie Lin

Abstract: Real-time interception of fast-moving objects by robotic arms in dynamic environments poses a formidable challenge due to the need for rapid reaction times, often within milliseconds, amidst dynamic obstacles. This paper introduces a unified control framework to address the above challenge by simultaneously intercepting dynamic objects and avoiding moving obstacles. Central to our approach is usin… ▽ More Real-time interception of fast-moving objects by robotic arms in dynamic environments poses a formidable challenge due to the need for rapid reaction times, often within milliseconds, amidst dynamic obstacles. This paper introduces a unified control framework to address the above challenge by simultaneously intercepting dynamic objects and avoiding moving obstacles. Central to our approach is using diffusion-based variational autoencoder for motion planning to perform both object interception and obstacle avoidance. We begin by encoding the high-dimensional temporal information from streaming events into a two-dimensional latent manifold, enabling the discrimination between safe and colliding trajectories, culminating in the construction of an offline densely connected trajectory graph. Subsequently, we employ an extended Kalman filter to achieve precise real-time tracking of the moving object. Leveraging a graph-traversing strategy on the established offline dense graph, we generate encoded robotic motor control commands. Finally, we decode these commands to enable real-time motion of robotic motors, ensuring effective obstacle avoidance and high interception accuracy of fast-moving objects. Experimental validation on both computer simulations and autonomous 7-DoF robotic arms demonstrates the efficacy of our proposed framework. Results indicate the capability of the robotic manipulator to navigate around multiple obstacles of varying sizes and shapes while successfully intercepting fast-moving objects thrown from different angles by hand. Complete video demonstrations of our experiments can be found in https://sites.google.com/view/multirobotskill/home. △ Less

Submitted 2 April, 2024; v1 submitted 27 September, 2022; originally announced September 2022.

Comments: Submitted to IEEE International Conference on Intelligent Robots and Systems (IROS) 2024

Learning to screen Glaucoma like the ophthalmologists

Authors: Junde Wu, Huihui Fang, Fei Li, Huazhu Fu, Yanwu Xu

Abstract: GAMMA Challenge is organized to encourage the AI models to screen the glaucoma from a combination of 2D fundus image and 3D optical coherence tomography volume, like the ophthalmologists. GAMMA Challenge is organized to encourage the AI models to screen the glaucoma from a combination of 2D fundus image and 3D optical coherence tomography volume, like the ophthalmologists. △ Less

Submitted 23 September, 2022; originally announced September 2022.

The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding

Authors: Hao Fang, Anusha Balakrishnan, Harsh Jhamtani, John Bufe, Jean Crawford, Jayant Krishnamurthy, Adam Pauls, Jason Eisner, Jacob Andreas, Dan Klein

Abstract: In a real-world dialogue system, generated text must be truthful and informative while remaining fluent and adhering to a prescribed style. Satisfying these constraints simultaneously is difficult for the two predominant paradigms in language generation: neural language modeling and rule-based generation. We describe a hybrid architecture for dialogue response generation that combines the strength… ▽ More In a real-world dialogue system, generated text must be truthful and informative while remaining fluent and adhering to a prescribed style. Satisfying these constraints simultaneously is difficult for the two predominant paradigms in language generation: neural language modeling and rule-based generation. We describe a hybrid architecture for dialogue response generation that combines the strengths of both paradigms. The first component of this architecture is a rule-based content selection model defined using a new formal framework called dataflow transduction, which uses declarative rules to transduce a dialogue agent's actions and their results (represented as dataflow graphs) into context-free grammars representing the space of contextually acceptable responses. The second component is a constrained decoding procedure that uses these grammars to constrain the output of a neural language model, which selects fluent utterances. Our experiments show that this system outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness. △ Less

Submitted 26 May, 2023; v1 submitted 16 September, 2022; originally announced September 2022.

Comments: Findings of ACL 2023

Attacking the Spike: On the Transferability and Security of Spiking Neural Networks to Adversarial Examples

Authors: Nuo Xu, Kaleel Mahmood, Haowen Fang, Ethan Rathbun, Caiwen Ding, Wujie Wen

Abstract: Spiking neural networks (SNNs) have attracted much attention for their high energy efficiency and for recent advances in their classification performance. However, unlike traditional deep learning approaches, the analysis and study of the robustness of SNNs to adversarial examples remain relatively underdeveloped. In this work, we focus on advancing the adversarial attack side of SNNs and make thr… ▽ More Spiking neural networks (SNNs) have attracted much attention for their high energy efficiency and for recent advances in their classification performance. However, unlike traditional deep learning approaches, the analysis and study of the robustness of SNNs to adversarial examples remain relatively underdeveloped. In this work, we focus on advancing the adversarial attack side of SNNs and make three major contributions. First, we show that successful white-box adversarial attacks on SNNs are highly dependent on the underlying surrogate gradient technique, even in the case of adversarially trained SNNs. Second, using the best surrogate gradient technique, we analyze the transferability of adversarial attacks on SNNs and other state-of-the-art architectures like Vision Transformers (ViTs) and Big Transfer Convolutional Neural Networks (CNNs). We demonstrate that the adversarial examples created by non-SNN architectures are not misclassified often by SNNs. Third, due to the lack of an ubiquitous white-box attack that is effective across both the SNN and CNN/ViT domains, we develop a new white-box attack, the Auto Self-Attention Gradient Attack (Auto-SAGA). Our novel attack generates adversarial examples capable of fooling both SNN and non-SNN models simultaneously. Auto-SAGA is as much as $91.1\%$ more effective on SNN/ViT model ensembles and provides a $3\times$ boost in attack effectiveness on adversarially trained SNN ensembles compared to conventional white-box attacks like Auto-PGD. Our experiments and analyses are broad and rigorous covering three datasets (CIFAR-10, CIFAR-100 and ImageNet), five different white-box attacks and nineteen classifier models (seven for each CIFAR dataset and five models for ImageNet). △ Less

Submitted 13 October, 2023; v1 submitted 7 September, 2022; originally announced September 2022.

Exotic Dark Matter Search with CDEX-10 Experiment at China's Jinping Underground Laboratory

Authors: W. H. Dai, L. P. Jia, H. Ma, Q. Yue, K. J. Kang, Y. J. Li, H. P. An, Greeshma C., J. P. Chang, Y. H. Chen, J. P. Cheng, Z. Deng, C. H. Fang, X. P. Geng, H. Gong, Q. J. Guo, X. Y. Guo, L. He, S. M. He, J. W. Hu, H. X. Huang, T. C. Huang, H. T. Jia, X. Jiang, S. Karmakar , et al. (59 additional authors not shown)

Abstract: A search for exotic dark matter (DM) in the sub-GeV mass range has been conducted using 205 kg$\cdot$day data taken from a p-type point contact germanium detector of CDEX-10 experiment at China Jinping underground laboratory. New low-mass dark matter searching channels, neutral current fermionic DM absorption ($χ+A\rightarrow ν+A$) and DM-nucleus 3$\rightarrow$2 scattering ($χ+χ+A\rightarrow φ+A$)… ▽ More A search for exotic dark matter (DM) in the sub-GeV mass range has been conducted using 205 kg$\cdot$day data taken from a p-type point contact germanium detector of CDEX-10 experiment at China Jinping underground laboratory. New low-mass dark matter searching channels, neutral current fermionic DM absorption ($χ+A\rightarrow ν+A$) and DM-nucleus 3$\rightarrow$2 scattering ($χ+χ+A\rightarrow φ+A$), have been analyzed with an energy threshold of 160 eVee. No significant signal was found. Thus new limits on the DM-nucleon interaction cross section are set for both models at sub-GeV DM mass region. A cross section limit for the fermionic DM absorption is set to be $\rm 2.5\times 10^{-46} cm^2$(90\% C.L.) at DM mass of 10 MeV/c$^2$. For the DM-nucleus 3$\rightarrow$2 scattering scenario, limits are extended to DM mass of 5 MeV/c$^2$ and 14 MeV/c$^2$ for the massless dark photon and bound DM final state, respectively. △ Less

Submitted 23 November, 2022; v1 submitted 2 September, 2022; originally announced September 2022.

Comments: 5 pages, 7 figures

Journal ref: Phys. Rev. Lett. 129, 221802, 2022

Transformers with Learnable Activation Functions

Authors: Haishuo Fang, Ji-Ung Lee, Nafise Sadat Moosavi, Iryna Gurevych

Abstract: Activation functions can have a significant impact on reducing the topological complexity of input data and therefore improve the performance of the model. Selecting a suitable activation function is an essential step in neural model design. However, the choice of activation function is seldom discussed or explored in Transformer-based language models. Their activation functions are chosen beforeh… ▽ More Activation functions can have a significant impact on reducing the topological complexity of input data and therefore improve the performance of the model. Selecting a suitable activation function is an essential step in neural model design. However, the choice of activation function is seldom discussed or explored in Transformer-based language models. Their activation functions are chosen beforehand and then remain fixed from pre-training to fine-tuning. As a result, the inductive biases they imposed on models cannot be adjusted during this long life cycle. Moreover, subsequently developed models (e.g., RoBERTa, BART, and GPT-3) often follow up prior work (e.g., BERT) to use the same activation function without justification. In this paper, we investigate the effectiveness of using Rational Activation Function (RAF), a learnable activation function, in the Transformer architecture. In contrast to conventional, predefined activation functions, RAFs can adaptively learn optimal activation functions during training according to input data. Our experiments show the RAF-based Transformer (RAFT) achieves a lower validation perplexity than a vanilla BERT with the GELU function. We further evaluate RAFT on downstream tasks in low- and full-data settings. Our results show that RAFT outperforms the counterpart model across the majority of tasks and settings. For instance, RAFT outperforms vanilla BERT on the GLUE benchmark by 5.71 points on average in low-data scenario (where 100 training examples are available) and by 2.05 points on SQuAD in full-data setting. Analysis of the shapes of learned RAFs further unveils that they substantially vary between different layers of the pre-trained model and mostly look very different from conventional activation functions. RAFT opens a new research direction for analyzing and interpreting pre-trained models according to the learned activation functions. △ Less

Submitted 14 February, 2023; v1 submitted 30 August, 2022; originally announced August 2022.

Comments: Accepted by EACL2023 findings

Understanding Diversity in Session-Based Recommendation

Authors: Qing Yin, Hui Fang, Zhu Sun, Yew-Soon Ong

Abstract: Current session-based recommender systems (SBRSs) mainly focus on maximizing recommendation accuracy, while few studies have been devoted to improve diversity beyond accuracy. Meanwhile, it is unclear how the accuracy-oriented SBRSs perform in terms of diversity. Besides, the asserted "trade-off" relationship between accuracy and diversity has been increasingly questioned in the literature. Toward… ▽ More Current session-based recommender systems (SBRSs) mainly focus on maximizing recommendation accuracy, while few studies have been devoted to improve diversity beyond accuracy. Meanwhile, it is unclear how the accuracy-oriented SBRSs perform in terms of diversity. Besides, the asserted "trade-off" relationship between accuracy and diversity has been increasingly questioned in the literature. Towards the aforementioned issues, we conduct a holistic study to particularly examine the recommendation performance of representative SBRSs w.r.t. both accuracy and diversity, striving for better understanding the diversity-related issues for SBRSs and providing guidance on designing diversified SBRSs. Particularly, for a fair and thorough comparison, we deliberately select state-of-the-art non-neural, deep neural, and diversified SBRSs, by covering more scenarios with appropriate experimental setups, e.g., representative datasets, evaluation metrics, and hyper-parameter optimization technique. Our empirical results unveil that: 1) non-diversified methods can also obtain satisfying performance on diversity, which might even surpass diversified ones; and 2) the relationship between accuracy and diversity is quite complex. Besides the "trade-off" relationship, they might be positively correlated with each other, that is, having a same-trend (win-win or lose-lose) relationship, which varies across different methods and datasets. Additionally, we further identify three possible influential factors on diversity in SBRSs (i.e., granularity of item categorization, session diversity of datasets, and length of recommendation lists). △ Less

Submitted 31 May, 2023; v1 submitted 29 August, 2022; originally announced August 2022.

Towards Calibrated Hyper-Sphere Representation via Distribution Overlap Coefficient for Long-tailed Learning

Authors: Hualiang Wang, Siming Fu, Xiaoxuan He, Hangxiang Fang, Zuozhu Liu, Haoji Hu

Abstract: Long-tailed learning aims to tackle the crucial challenge that head classes dominate the training procedure under severe class imbalance in real-world scenarios. However, little attention has been given to how to quantify the dominance severity of head classes in the representation space. Motivated by this, we generalize the cosine-based classifiers to a von Mises-Fisher (vMF) mixture model, denot… ▽ More Long-tailed learning aims to tackle the crucial challenge that head classes dominate the training procedure under severe class imbalance in real-world scenarios. However, little attention has been given to how to quantify the dominance severity of head classes in the representation space. Motivated by this, we generalize the cosine-based classifiers to a von Mises-Fisher (vMF) mixture model, denoted as vMF classifier, which enables to quantitatively measure representation quality upon the hyper-sphere space via calculating distribution overlap coefficient. To our knowledge, this is the first work to measure representation quality of classifiers and features from the perspective of distribution overlap coefficient. On top of it, we formulate the inter-class discrepancy and class-feature consistency loss terms to alleviate the interference among the classifier weights and align features with classifier weights. Furthermore, a novel post-training calibration algorithm is devised to zero-costly boost the performance via inter-class overlap coefficients. Our method outperforms previous work with a large margin and achieves state-of-the-art performance on long-tailed image classification, semantic segmentation, and instance segmentation tasks (e.g., we achieve 55.0\% overall accuracy with ResNetXt-50 in ImageNet-LT). Our code is available at https://github.com/VipaiLab/vMF\_OP. △ Less

Submitted 21 August, 2022; originally announced August 2022.

Journal ref: ECCV2022

Calibrate the inter-observer segmentation uncertainty via diagnosis-first principle

Authors: Junde Wu, Huihui Fang, Hoayi Xiong, Lixin Duan, Mingkui Tan, Weihua Yang, Huiying Liu, Yanwu Xu

Abstract: On the medical images, many of the tissues/lesions may be ambiguous. That is why the medical segmentation is typically annotated by a group of clinical experts to mitigate the personal bias. However, this clinical routine also brings new challenges to the application of machine learning algorithms. Without a definite ground-truth, it will be difficult to train and evaluate the deep learning models… ▽ More On the medical images, many of the tissues/lesions may be ambiguous. That is why the medical segmentation is typically annotated by a group of clinical experts to mitigate the personal bias. However, this clinical routine also brings new challenges to the application of machine learning algorithms. Without a definite ground-truth, it will be difficult to train and evaluate the deep learning models. When the annotations are collected from different graders, a common choice is majority vote. However such a strategy ignores the difference between the grader expertness. In this paper, we consider the task of predicting the segmentation with the calibrated inter-observer uncertainty. We note that in clinical practice, the medical image segmentation is usually used to assist the disease diagnosis. Inspired by this observation, we propose diagnosis-first principle, which is to take disease diagnosis as the criterion to calibrate the inter-observer segmentation uncertainty. Following this idea, a framework named Diagnosis First segmentation Framework (DiFF) is proposed to estimate diagnosis-first segmentation from the raw images.Specifically, DiFF will first learn to fuse the multi-rater segmentation labels to a single ground-truth which could maximize the disease diagnosis performance. We dubbed the fused ground-truth as Diagnosis First Ground-truth (DF-GT).Then, we further propose Take and Give Modelto segment DF-GT from the raw image. We verify the effectiveness of DiFF on three different medical segmentation tasks: OD/OC segmentation on fundus images, thyroid nodule segmentation on ultrasound images, and skin lesion segmentation on dermoscopic images. Experimental results show that the proposed DiFF is able to significantly facilitate the corresponding disease diagnosis, which outperforms previous state-of-the-art multi-rater learning methods. △ Less

Submitted 5 August, 2022; originally announced August 2022.

Comments: arXiv admin note: text overlap with arXiv:2202.06505

Self-organized Polygon Formation Control based on Distributed Estimation

Authors: Qingkai Yang, Fan Xiao, Jingshuo Lyu, Bo Zhou, Hao Fang

Abstract: This paper studies the problem of controlling a multi-robot system to achieve a polygon formation in a self-organized manner. Different from the typical formation control strategies where robots are steered to satisfy the predefined control variables, such as pairwise distances, relative positions and bearings, the foremost idea of this paper is to achieve polygon formations by injecting control i… ▽ More This paper studies the problem of controlling a multi-robot system to achieve a polygon formation in a self-organized manner. Different from the typical formation control strategies where robots are steered to satisfy the predefined control variables, such as pairwise distances, relative positions and bearings, the foremost idea of this paper is to achieve polygon formations by injecting control inputs randomly to a few robots (say, vertex robots) of the group, and the rest follow the simple principles of moving towards the midpoint of their two nearest neighbors in the ring graph without any external inputs. In our problem, a fleet of robots is initially distributed in the plane. The socalled vertex robots take the responsibility of determining the geometric shape of the entire formation and its overall size, while the others move so as to minimize the differences with two direct neighbors. In the first step, each vertex robot estimates the number of robots in its associated chain. Two types of control inputs that serve for the estimation are designed using the measurements from the latest and the last two time instants respectively. In the second step, the self-organized formation control law is proposed where only vertex robots receive external information. Comparisons between the two estimation strategies are carried out in terms of the convergence speed and robustness. The effectiveness of the whole control framework is further validated in both simulation and physical experiments. △ Less

Submitted 2 April, 2023; v1 submitted 29 July, 2022; originally announced July 2022.

Dataset and Evaluation algorithm design for GOALS Challenge

Authors: Huihui Fang, Fei Li, Huazhu Fu, Junde Wu, Xiulan Zhang, Yanwu Xu

Abstract: Glaucoma causes irreversible vision loss due to damage to the optic nerve, and there is no cure for glaucoma.OCT imaging modality is an essential technique for assessing glaucomatous damage since it aids in quantifying fundus structures. To promote the research of AI technology in the field of OCT-assisted diagnosis of glaucoma, we held a Glaucoma OCT Analysis and Layer Segmentation (GOALS) Challe… ▽ More Glaucoma causes irreversible vision loss due to damage to the optic nerve, and there is no cure for glaucoma.OCT imaging modality is an essential technique for assessing glaucomatous damage since it aids in quantifying fundus structures. To promote the research of AI technology in the field of OCT-assisted diagnosis of glaucoma, we held a Glaucoma OCT Analysis and Layer Segmentation (GOALS) Challenge in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2022 to provide data and corresponding annotations for researchers studying layer segmentation from OCT images and the classification of glaucoma. This paper describes the released 300 circumpapillary OCT images, the baselines of the two sub-tasks, and the evaluation methodology. The GOALS Challenge is accessible at https://aistudio.baidu.com/aistudio/competition/detail/230. △ Less

Submitted 28 July, 2022; originally announced July 2022.

Comments: 8 pages, 3 figures, OMIA9 (MICCAI 2022) workshop

Model Predictive Control of Nonlinear Latent Force Models: A Scenario-Based Approach

Authors: Thomas Woodruff, Iman Askari, Guanghui Wang, Huazhen Fang

Abstract: Control of nonlinear uncertain systems is a common challenge in the robotics field. Nonlinear latent force models, which incorporate latent uncertainty characterized as Gaussian processes, carry the promise of representing such systems effectively, and we focus on the control design for them in this work. To enable the design, we adopt the state-space representation of a Gaussian process to recast… ▽ More Control of nonlinear uncertain systems is a common challenge in the robotics field. Nonlinear latent force models, which incorporate latent uncertainty characterized as Gaussian processes, carry the promise of representing such systems effectively, and we focus on the control design for them in this work. To enable the design, we adopt the state-space representation of a Gaussian process to recast the nonlinear latent force model and thus build the ability to predict the future state and uncertainty concurrently. Using this feature, a stochastic model predictive control problem is formulated. To derive a computational algorithm for the problem, we use the scenario-based approach to formulate a deterministic approximation of the stochastic optimization. We evaluate the resultant scenario-based model predictive control approach through a simulation study based on motion planning of an autonomous vehicle, which shows much effectiveness. The proposed approach can find prospective use in various other robotics applications. △ Less

Submitted 27 July, 2022; originally announced July 2022.

Journal ref: 2021 IEEE International Conference on Robotics and Automation (ICRA)

BrainCog: A Spiking Neural Network based Brain-inspired Cognitive Intelligence Engine for Brain-inspired AI and Brain Simulation

Authors: Yi Zeng, Dongcheng Zhao, Feifei Zhao, Guobin Shen, Yiting Dong, Enmeng Lu, Qian Zhang, Yinqian Sun, Qian Liang, Yuxuan Zhao, Zhuoya Zhao, Hongjian Fang, Yuwei Wang, Yang Li, Xin Liu, Chengcheng Du, Qingqun Kong, Zizhe Ruan, Weida Bi

Abstract: Spiking neural networks (SNNs) have attracted extensive attentions in Brain-inspired Artificial Intelligence and computational neuroscience. They can be used to simulate biological information processing in the brain at multiple scales. More importantly, SNNs serve as an appropriate level of abstraction to bring inspirations from brain and cognition to Artificial Intelligence. In this paper, we pr… ▽ More Spiking neural networks (SNNs) have attracted extensive attentions in Brain-inspired Artificial Intelligence and computational neuroscience. They can be used to simulate biological information processing in the brain at multiple scales. More importantly, SNNs serve as an appropriate level of abstraction to bring inspirations from brain and cognition to Artificial Intelligence. In this paper, we present the Brain-inspired Cognitive Intelligence Engine (BrainCog) for creating brain-inspired AI and brain simulation models. BrainCog incorporates different types of spiking neuron models, learning rules, brain areas, etc., as essential modules provided by the platform. Based on these easy-to-use modules, BrainCog supports various brain-inspired cognitive functions, including Perception and Learning, Decision Making, Knowledge Representation and Reasoning, Motor Control, and Social Cognition. These brain-inspired AI models have been effectively validated on various supervised, unsupervised, and reinforcement learning tasks, and they can be used to enable AI models to be with multiple brain-inspired cognitive functions. For brain simulation, BrainCog realizes the function simulation of decision-making, working memory, the structure simulation of the Neural Circuit, and whole brain structure simulation of Mouse brain, Macaque brain, and Human brain. An AI engine named BORN is developed based on BrainCog, and it demonstrates how the components of BrainCog can be integrated and used to build AI models and applications. To enable the scientific quest to decode the nature of biological intelligence and create AI, BrainCog aims to provide essential and easy-to-use building blocks, and infrastructural support to develop brain-inspired spiking neural network based AI, and to simulate the cognitive brains at multiple scales. The online repository of BrainCog can be found at https://github.com/braincog-x. △ Less

Submitted 11 July, 2023; v1 submitted 18 July, 2022; originally announced July 2022.

Comments: This paper was accepted by Patterns. The accepted version can be seen at https://www.cell.com/patterns/fulltext/S2666-3899(23)00144-7

Brain-inspired Graph Spiking Neural Networks for Commonsense Knowledge Representation and Reasoning

Authors: Hongjian Fang, Yi Zeng, Jianbo Tang, Yuwei Wang, Yao Liang, Xin Liu

Abstract: How neural networks in the human brain represent commonsense knowledge, and complete related reasoning tasks is an important research topic in neuroscience, cognitive science, psychology, and artificial intelligence. Although the traditional artificial neural network using fixed-length vectors to represent symbols has gained good performance in some specific tasks, it is still a black box that lac… ▽ More How neural networks in the human brain represent commonsense knowledge, and complete related reasoning tasks is an important research topic in neuroscience, cognitive science, psychology, and artificial intelligence. Although the traditional artificial neural network using fixed-length vectors to represent symbols has gained good performance in some specific tasks, it is still a black box that lacks interpretability, far from how humans perceive the world. Inspired by the grandmother-cell hypothesis in neuroscience, this work investigates how population encoding and spiking timing-dependent plasticity (STDP) mechanisms can be integrated into the learning of spiking neural networks, and how a population of neurons can represent a symbol via guiding the completion of sequential firing between different neuron populations. The neuron populations of different communities together constitute the entire commonsense knowledge graph, forming a giant graph spiking neural network. Moreover, we introduced the Reward-modulated spiking timing-dependent plasticity (R-STDP) mechanism to simulate the biological reinforcement learning process and completed the related reasoning tasks accordingly, achieving comparable accuracy and faster convergence speed than the graph convolutional artificial neural networks. For the fields of neuroscience and cognitive science, the work in this paper provided the foundation of computational modeling for further exploration of the way the human brain represents commonsense knowledge. For the field of artificial intelligence, this paper indicated the exploration direction for realizing a more robust and interpretable neural network by constructing a commonsense knowledge representation and reasoning spiking neural networks with solid biological plausibility. △ Less

Submitted 11 July, 2022; originally announced July 2022.

High-Order Leader-Follower Tracking Control under Limited Information Availability

Authors: Chuan Yan, Tao Yang, Huazhen Fang

Abstract: Limited information availability represents a fundamental challenge for control of multi-agent systems, since an agent often lacks sensing capabilities to measure certain states of its own and can exchange data only with its neighbors. The challenge becomes even greater when agents are governed by high-order dynamics. The present work is motivated to conduct control design for linear and nonlinear… ▽ More Limited information availability represents a fundamental challenge for control of multi-agent systems, since an agent often lacks sensing capabilities to measure certain states of its own and can exchange data only with its neighbors. The challenge becomes even greater when agents are governed by high-order dynamics. The present work is motivated to conduct control design for linear and nonlinear high-order leader-follower multi-agent systems in a context where only the first state of an agent is measured. To address this open challenge, we develop novel distributed observers to enable followers to reconstruct unmeasured or unknown quantities about themselves and the leader and on such a basis, build observer-based tracking control approaches. We analyze the convergence properties of the proposed approaches and validate their performance through simulation. △ Less

Submitted 11 July, 2022; originally announced July 2022.

De-END: Decoder-driven Watermarking Network

Authors: Han Fang, Zhaoyang Jia, Yupeng Qiu, Jiyi Zhang, Weiming Zhang, Ee-Chien Chang

Abstract: With recent advances in machine learning, researchers are now able to solve traditional problems with new solutions. In the area of digital watermarking, deep-learning-based watermarking technique is being extensively studied. Most existing approaches adopt a similar encoder-driven scheme which we name END (Encoder-NoiseLayer-Decoder) architecture. In this paper, we revamp the architecture and cre… ▽ More With recent advances in machine learning, researchers are now able to solve traditional problems with new solutions. In the area of digital watermarking, deep-learning-based watermarking technique is being extensively studied. Most existing approaches adopt a similar encoder-driven scheme which we name END (Encoder-NoiseLayer-Decoder) architecture. In this paper, we revamp the architecture and creatively design a decoder-driven watermarking network dubbed De-END which greatly outperforms the existing END-based methods. The motivation for designing De-END originated from the potential drawback we discovered in END architecture: The encoder may embed redundant features that are not necessary for decoding, limiting the performance of the whole network. We conducted a detailed analysis and found that such limitations are caused by unsatisfactory coupling between the encoder and decoder in END. De-END addresses such drawbacks by adopting a Decoder-Encoder-Noiselayer-Decoder architecture. In De-END, the host image is firstly processed by the decoder to generate a latent feature map instead of being directly fed into the encoder. This latent feature map is concatenated to the original watermark message and then processed by the encoder. This change in design is crucial as it makes the feature of encoder and decoder directly shared thus the encoder and decoder are better coupled. We conducted extensive experiments and the results show that this framework outperforms the existing state-of-the-art (SOTA) END-based deep learning watermarking both in visual quality and robustness. On the premise of the same decoder structure, the visual quality (measured by PSNR) of De-END improves by 1.6dB (45.16dB to 46.84dB), and extraction accuracy after JPEG compression (QF=50) distortion outperforms more than 4% (94.9% to 99.1%). △ Less

Submitted 26 June, 2022; originally announced June 2022.

Unseen Object 6D Pose Estimation: A Benchmark and Baselines

Authors: Minghao Gou, Haolin Pan, Hao-Shu Fang, Ziyuan Liu, Cewu Lu, Ping Tan

Abstract: Estimating the 6D pose for unseen objects is in great demand for many real-world applications. However, current state-of-the-art pose estimation methods can only handle objects that are previously trained. In this paper, we propose a new task that enables and facilitates algorithms to estimate the 6D pose estimation of novel objects during testing. We collect a dataset with both real and synthetic… ▽ More Estimating the 6D pose for unseen objects is in great demand for many real-world applications. However, current state-of-the-art pose estimation methods can only handle objects that are previously trained. In this paper, we propose a new task that enables and facilitates algorithms to estimate the 6D pose estimation of novel objects during testing. We collect a dataset with both real and synthetic images and up to 48 unseen objects in the test set. In the mean while, we propose a new metric named Infimum ADD (IADD) which is an invariant measurement for objects with different types of pose ambiguity. A two-stage baseline solution for this task is also provided. By training an end-to-end 3D correspondences network, our method finds corresponding points between an unseen object and a partial view RGBD image accurately and efficiently. It then calculates the 6D pose from the correspondences using an algorithm robust to object symmetry. Extensive experiments show that our method outperforms several intuitive baselines and thus verify its effectiveness. All the data, code and models will be made publicly available. Project page: www.graspnet.net/unseen6d △ Less

Submitted 23 June, 2022; originally announced June 2022.

DaisyRec 2.0: Benchmarking Recommendation for Rigorous Evaluation

Authors: Zhu Sun, Hui Fang, Jie Yang, Xinghua Qu, Hongyang Liu, Di Yu, Yew-Soon Ong, Jie Zhang

Abstract: Recently, one critical issue looms large in the field of recommender systems -- there are no effective benchmarks for rigorous evaluation -- which consequently leads to unreproducible evaluation and unfair comparison. We, therefore, conduct studies from the perspectives of practical theory and experiments, aiming at benchmarking recommendation for rigorous evaluation. Regarding the theoretical stu… ▽ More Recently, one critical issue looms large in the field of recommender systems -- there are no effective benchmarks for rigorous evaluation -- which consequently leads to unreproducible evaluation and unfair comparison. We, therefore, conduct studies from the perspectives of practical theory and experiments, aiming at benchmarking recommendation for rigorous evaluation. Regarding the theoretical study, a series of hyper-factors affecting recommendation performance throughout the whole evaluation chain are systematically summarized and analyzed via an exhaustive review on 141 papers published at eight top-tier conferences within 2017-2020. We then classify them into model-independent and model-dependent hyper-factors, and different modes of rigorous evaluation are defined and discussed in-depth accordingly. For the experimental study, we release DaisyRec 2.0 library by integrating these hyper-factors to perform rigorous evaluation, whereby a holistic empirical study is conducted to unveil the impacts of different hyper-factors on recommendation performance. Supported by the theoretical and experimental studies, we finally create benchmarks for rigorous evaluation by proposing standardized procedures and providing performance of ten state-of-the-arts across six evaluation metrics on six datasets as a reference for later study. Overall, our work sheds light on the issues in recommendation evaluation, provides potential solutions for rigorous evaluation, and lays foundation for further investigation. △ Less

Submitted 22 June, 2022; originally announced June 2022.

Comments: DaisyRec-v2.0 contains a Python toolkit developed for benchmarking top-N recommendation task. Our code is available at https://github.com/recsys-benchmark/DaisyRec-v2.0

SeATrans: Learning Segmentation-Assisted diagnosis model via Transformer

Authors: Junde Wu, Huihui Fang, Fangxin Shang, Dalu Yang, Zhaowei Wang, Jing Gao, Yehui Yang, Yanwu Xu

Abstract: Clinically, the accurate annotation of lesions/tissues can significantly facilitate the disease diagnosis. For example, the segmentation of optic disc/cup (OD/OC) on fundus image would facilitate the glaucoma diagnosis, the segmentation of skin lesions on dermoscopic images is helpful to the melanoma diagnosis, etc. With the advancement of deep learning techniques, a wide range of methods proved t… ▽ More Clinically, the accurate annotation of lesions/tissues can significantly facilitate the disease diagnosis. For example, the segmentation of optic disc/cup (OD/OC) on fundus image would facilitate the glaucoma diagnosis, the segmentation of skin lesions on dermoscopic images is helpful to the melanoma diagnosis, etc. With the advancement of deep learning techniques, a wide range of methods proved the lesions/tissues segmentation can also facilitate the automated disease diagnosis models. However, existing methods are limited in the sense that they can only capture static regional correlations in the images. Inspired by the global and dynamic nature of Vision Transformer, in this paper, we propose Segmentation-Assisted diagnosis Transformer (SeATrans) to transfer the segmentation knowledge to the disease diagnosis network. Specifically, we first propose an asymmetric multi-scale interaction strategy to correlate each single low-level diagnosis feature with multi-scale segmentation features. Then, an effective strategy called SeA-block is adopted to vitalize diagnosis feature via correlated segmentation features. To model the segmentation-diagnosis interaction, SeA-block first embeds the diagnosis feature based on the segmentation information via the encoder, and then transfers the embedding back to the diagnosis feature space by a decoder. Experimental results demonstrate that SeATrans surpasses a wide range of state-of-the-art (SOTA) segmentation-assisted diagnosis methods on several disease diagnosis tasks. △ Less

Submitted 22 June, 2022; v1 submitted 12 June, 2022; originally announced June 2022.

Learning self-calibrated optic disc and cup segmentation from multi-rater annotations

Authors: Junde Wu, Huihui Fang, Fangxin Shang, Zhaowei Wang, Dalu Yang, Wenshuo Zhou, Yehui Yang, Yanwu Xu

Abstract: The segmentation of optic disc(OD) and optic cup(OC) from fundus images is an important fundamental task for glaucoma diagnosis. In the clinical practice, it is often necessary to collect opinions from multiple experts to obtain the final OD/OC annotation. This clinical routine helps to mitigate the individual bias. But when data is multiply annotated, standard deep learning models will be inappli… ▽ More The segmentation of optic disc(OD) and optic cup(OC) from fundus images is an important fundamental task for glaucoma diagnosis. In the clinical practice, it is often necessary to collect opinions from multiple experts to obtain the final OD/OC annotation. This clinical routine helps to mitigate the individual bias. But when data is multiply annotated, standard deep learning models will be inapplicable. In this paper, we propose a novel neural network framework to learn OD/OC segmentation from multi-rater annotations. The segmentation results are self-calibrated through the iterative optimization of multi-rater expertness estimation and calibrated OD/OC segmentation. In this way, the proposed method can realize a mutual improvement of both tasks and finally obtain a refined segmentation result. Specifically, we propose Diverging Model(DivM) and Converging Model(ConM) to process the two tasks respectively. ConM segments the raw image based on the multi-rater expertness map provided by DivM. DivM generates multi-rater expertness map from the segmentation mask provided by ConM. The experiment results show that by recurrently running ConM and DivM, the results can be self-calibrated so as to outperform a range of state-of-the-art(SOTA) multi-rater segmentation methods. △ Less

Submitted 14 June, 2022; v1 submitted 10 June, 2022; originally announced June 2022.

Constraints on Sub-GeV Dark Matter--Electron Scattering from the CDEX-10 Experiment

Authors: Z. Y. Zhang, L. T. Yang, Q. Yue, K. J. Kang, Y. J. Li, M. Agartioglu, H. P. An, J. P. Chang, Y. H. Chen, J. P. Cheng, W. H. Dai, Z. Deng, C. H. Fang, X. P. Geng, H. Gong, Q. J. Guo, X. Y. Guo, L. He, S. M. He, J. W. Hu, H. X. Huang, T. C. Huang, H. T. Jia, X. Jiang, H. B. Li , et al. (60 additional authors not shown)

Abstract: We present improved germanium-based constraints on sub-GeV dark matter via dark matter--electron ($χ$-$e$) scattering using the 205.4 kg$\cdot$day dataset from the CDEX-10 experiment. Using a novel calculation technique, we attain predicted $χ$-$e$ scattering spectra observable in high-purity germanium detectors. In the heavy mediator scenario, our results achieve 3 orders of magnitude of improvem… ▽ More We present improved germanium-based constraints on sub-GeV dark matter via dark matter--electron ($χ$-$e$) scattering using the 205.4 kg$\cdot$day dataset from the CDEX-10 experiment. Using a novel calculation technique, we attain predicted $χ$-$e$ scattering spectra observable in high-purity germanium detectors. In the heavy mediator scenario, our results achieve 3 orders of magnitude of improvement for $m_χ$ larger than 80 MeV/c$^2$ compared to previous germanium-based $χ$-$e$ results. We also present the most stringent $χ$-$e$ cross-section limit to date among experiments using solid-state detectors for $m_χ$ larger than 90 MeV/c$^2$ with heavy mediators and $m_χ$ larger than 100 MeV/c$^2$ with electric dipole coupling. The result proves the feasibility and demonstrates the vast potential of a new $χ$-$e$ detection method with high-purity germanium detectors in ultralow radioactive background. △ Less

Submitted 21 November, 2022; v1 submitted 8 June, 2022; originally announced June 2022.

Comments: 6 pages, 3 figures. Version updated to match PRL version

Journal ref: Phys. Rev. Lett. 129, 221301 (2022)

A simple method to reprogram the binding specificity of DNA-coated colloids that crystallize

Authors: Pepijn G. Moerman, Huang Fang, Thomas E. Videbæk, W. Benjamin Rogers, Rebecca Schulman

Abstract: DNA-coated colloids can crystallize into a multitude of lattices, ranging from face-centered cubic to diamond and thereby contribute to our understanding of crystallization and open avenues to producing structures with useful photonic properties. Despite the broad potential design space of DNA-coated colloids, the design cycle for synthesizing DNA-coated particles is slow: preparing a particle wit… ▽ More DNA-coated colloids can crystallize into a multitude of lattices, ranging from face-centered cubic to diamond and thereby contribute to our understanding of crystallization and open avenues to producing structures with useful photonic properties. Despite the broad potential design space of DNA-coated colloids, the design cycle for synthesizing DNA-coated particles is slow: preparing a particle with a new type of DNA sequence takes more than one day and requires custom-made and chemically modified DNA that typically takes the supplier over a month to synthesize. Here, we introduce a method to generate particles with custom sequences from a single feed stock in under an hour at ambient conditions. Our method appends new DNA domains onto the DNA grafted to colloidal particles based on a template that takes the supplier less than a week to produce. The resultant particles crystallize as readily and at the same temperature as those produced via direct chemical synthesis. Moreover, we show that particles coated with a single sequence can be converted into a variety of building blocks with differing specificities by appending different DNA sequences to them. This approach to DNA-coated particle preparation will make it practical to identify optimal and complex particle sequence designs and to expand the use of DNA-coated colloids to a much broader range of investigators and commercial entities. △ Less

Submitted 2 June, 2022; originally announced June 2022.

Comments: manuscript 9 pages 6 figures. SI, 12 pages 6 figures