Showing 1–12 of 12 results for author: Bi, G

ToMBench: Benchmarking Theory of Mind in Large Language Models

Authors: Zhuang Chen, Jincenzi Wu, Jinfeng Zhou, Bosi Wen, Guanqun Bi, Gongyao Jiang, Yaru Cao, Mengting Hu, Yunghwei Lai, Zexuan Xiong, Minlie Huang

Abstract: Theory of Mind (ToM) is the cognitive capability to perceive and ascribe mental states to oneself and others. Recent research has sparked a debate over whether large language models (LLMs) exhibit a form of ToM. However, existing ToM evaluations are hindered by challenges such as constrained scope, subjective judgment, and unintended contamination, yielding inadequate assessments. To address this… ▽ More Theory of Mind (ToM) is the cognitive capability to perceive and ascribe mental states to oneself and others. Recent research has sparked a debate over whether large language models (LLMs) exhibit a form of ToM. However, existing ToM evaluations are hindered by challenges such as constrained scope, subjective judgment, and unintended contamination, yielding inadequate assessments. To address this gap, we introduce ToMBench with three key characteristics: a systematic evaluation framework encompassing 8 tasks and 31 abilities in social cognition, a multiple-choice question format to support automated and unbiased evaluation, and a build-from-scratch bilingual inventory to strictly avoid data leakage. Based on ToMBench, we conduct extensive experiments to evaluate the ToM performance of 10 popular LLMs across tasks and abilities. We find that even the most advanced LLMs like GPT-4 lag behind human performance by over 10% points, indicating that LLMs have not achieved a human-level theory of mind yet. Our aim with ToMBench is to enable an efficient and effective evaluation of LLMs' ToM capabilities, thereby facilitating the development of LLMs with inherent social intelligence. △ Less

Submitted 22 February, 2024; originally announced February 2024.

Comments: Under review

A Group Fairness Lens for Large Language Models

Authors: Guanqun Bi, Lei Shen, Yuqiang Xie, Yanan Cao, Tiangang Zhu, Xiaodong He

Abstract: The rapid advancement of large language models has revolutionized various applications but also raised crucial concerns about their potential to perpetuate biases and unfairness when deployed in social media contexts. Evaluating LLMs' potential biases and fairness has become crucial, as existing methods rely on limited prompts focusing on just a few groups, lacking a comprehensive categorical pers… ▽ More The rapid advancement of large language models has revolutionized various applications but also raised crucial concerns about their potential to perpetuate biases and unfairness when deployed in social media contexts. Evaluating LLMs' potential biases and fairness has become crucial, as existing methods rely on limited prompts focusing on just a few groups, lacking a comprehensive categorical perspective. In this paper, we propose evaluating LLM biases from a group fairness lens using a novel hierarchical schema characterizing diverse social groups. Specifically, we construct a dataset, GFair, encapsulating target-attribute combinations across multiple dimensions. In addition, we introduce statement organization, a new open-ended text generation task, to uncover complex biases in LLMs. Extensive evaluations of popular LLMs reveal inherent safety concerns. To mitigate the biases of LLM from a group fairness perspective, we pioneer a novel chain-of-thought method GF-Think to mitigate biases of LLMs from a group fairness perspective. Experimental results demonstrate its efficacy in mitigating bias in LLMs to achieve fairness. △ Less

Submitted 24 December, 2023; originally announced December 2023.

Comments: Work in progress

DiffusEmp: A Diffusion Model-Based Framework with Multi-Grained Control for Empathetic Response Generation

Authors: Guanqun Bi, Lei Shen, Yanan Cao, Meng Chen, Yuqiang Xie, Zheng Lin, Xiaodong He

Abstract: Empathy is a crucial factor in open-domain conversations, which naturally shows one's caring and understanding to others. Though several methods have been proposed to generate empathetic responses, existing works often lead to monotonous empathy that refers to generic and safe expressions. In this paper, we propose to use explicit control to guide the empathy expression and design a framework Diff… ▽ More Empathy is a crucial factor in open-domain conversations, which naturally shows one's caring and understanding to others. Though several methods have been proposed to generate empathetic responses, existing works often lead to monotonous empathy that refers to generic and safe expressions. In this paper, we propose to use explicit control to guide the empathy expression and design a framework DiffusEmp based on conditional diffusion language model to unify the utilization of dialogue context and attribute-oriented control signals. Specifically, communication mechanism, intent, and semantic frame are imported as multi-grained signals that control the empathy realization from coarse to fine levels. We then design a specific masking strategy to reflect the relationship between multi-grained signals and response tokens, and integrate it into the diffusion model to influence the generative process. Experimental results on a benchmark dataset EmpatheticDialogue show that our framework outperforms competitive baselines in terms of controllability, informativeness, and diversity without the loss of context-relatedness. △ Less

Submitted 9 July, 2023; v1 submitted 2 June, 2023; originally announced June 2023.

Comments: accepted by ACL 2023 main conference (Oral)

Psychology-guided Controllable Story Generation

Authors: Yuqiang Xie, Yue Hu, Yunpeng Li, Guanqun Bi, Luxi Xing, Wei Peng

Abstract: Controllable story generation is a challenging task in the field of NLP, which has attracted increasing research interest in recent years. However, most existing works generate a whole story conditioned on the appointed keywords or emotions, ignoring the psychological changes of the protagonist. Inspired by psychology theories, we introduce global psychological state chains, which include the need… ▽ More Controllable story generation is a challenging task in the field of NLP, which has attracted increasing research interest in recent years. However, most existing works generate a whole story conditioned on the appointed keywords or emotions, ignoring the psychological changes of the protagonist. Inspired by psychology theories, we introduce global psychological state chains, which include the needs and emotions of the protagonists, to help a story generation system create more controllable and well-planned stories. In this paper, we propose a Psychology-guIded Controllable Story Generation System (PICS) to generate stories that adhere to the given leading context and desired psychological state chains for the protagonist. Specifically, psychological state trackers are employed to memorize the protagonist's local psychological states to capture their inner temporal relationships. In addition, psychological state planners are adopted to gain the protagonist's global psychological states for story planning. Eventually, a psychology controller is designed to integrate the local and global psychological states into the story context representation for composing psychology-guided stories. Automatic and manual evaluations demonstrate that PICS outperforms baselines, and each part of PICS shows effectiveness for writing stories with more consistent psychological changes. △ Less

Submitted 13 October, 2022; originally announced October 2022.

Comments: Accepted by COLING 2022

COMMA: Modeling Relationship among Motivations, Emotions and Actions in Language-based Human Activities

Authors: Yuqiang Xie, Yue Hu, Wei Peng, Guanqun Bi, Luxi Xing

Abstract: Motivations, emotions, and actions are inter-related essential factors in human activities. While motivations and emotions have long been considered at the core of exploring how people take actions in human activities, there has been relatively little research supporting analyzing the relationship between human mental states and actions. We present the first study that investigates the viability o… ▽ More Motivations, emotions, and actions are inter-related essential factors in human activities. While motivations and emotions have long been considered at the core of exploring how people take actions in human activities, there has been relatively little research supporting analyzing the relationship between human mental states and actions. We present the first study that investigates the viability of modeling motivations, emotions, and actions in language-based human activities, named COMMA (Cognitive Framework of Human Activities). Guided by COMMA, we define three natural language processing tasks (emotion understanding, motivation understanding and conditioned action generation), and build a challenging dataset Hail through automatically extracting samples from Story Commonsense. Experimental results on NLP applications prove the effectiveness of modeling the relationship. Furthermore, our models inspired by COMMA can better reveal the essential relationship among motivations, emotions and actions than existing methods. △ Less

Submitted 14 September, 2022; originally announced September 2022.

Comments: Accepted to COLING 2022

How Does Knowledge Graph Embedding Extrapolate to Unseen Data: A Semantic Evidence View

Authors: Ren Li, Yanan Cao, Qiannan Zhu, Guanqun Bi, Fang Fang, Yi Liu, Qian Li

Abstract: Knowledge Graph Embedding (KGE) aims to learn representations for entities and relations. Most KGE models have gained great success, especially on extrapolation scenarios. Specifically, given an unseen triple (h, r, t), a trained model can still correctly predict t from (h, r, ?), or h from (?, r, t), such extrapolation ability is impressive. However, most existing KGE works focus on the design of… ▽ More Knowledge Graph Embedding (KGE) aims to learn representations for entities and relations. Most KGE models have gained great success, especially on extrapolation scenarios. Specifically, given an unseen triple (h, r, t), a trained model can still correctly predict t from (h, r, ?), or h from (?, r, t), such extrapolation ability is impressive. However, most existing KGE works focus on the design of delicate triple modeling function, which mainly tells us how to measure the plausibility of observed triples, but offers limited explanation of why the methods can extrapolate to unseen data, and what are the important factors to help KGE extrapolate. Therefore in this work, we attempt to study the KGE extrapolation of two problems: 1. How does KGE extrapolate to unseen data? 2. How to design the KGE model with better extrapolation ability? For the problem 1, we first discuss the impact factors for extrapolation and from relation, entity and triple level respectively, propose three Semantic Evidences (SEs), which can be observed from train set and provide important semantic information for extrapolation. Then we verify the effectiveness of SEs through extensive experiments on several typical KGE methods. For the problem 2, to make better use of the three levels of SE, we propose a novel GNN-based KGE model, called Semantic Evidence aware Graph Neural Network (SE-GNN). In SE-GNN, each level of SE is modeled explicitly by the corresponding neighbor pattern, and merged sufficiently by the multi-layer aggregation, which contributes to obtaining more extrapolative knowledge representation. Finally, through extensive experiments on FB15k-237 and WN18RR datasets, we show that SE-GNN achieves state-of-the-art performance on Knowledge Graph Completion task and performs a better extrapolation ability. Our code is available at https://github.com/renli1024/SE-GNN. △ Less

Submitted 12 May, 2022; v1 submitted 24 September, 2021; originally announced September 2021.

Comments: Accepted by AAAI'22

An Evolutional Algorithm for Automatic 2D Layer Segmentation in Laser-aided Additive Manufacturing

Authors: N. Liu, K. Ren, W. Zhang, Y. F. Zhang, Y. X. Chew, J. Y. H. Fuh, G. J. Bi

Abstract: Toolpath planning is an important task in laser aided additive manufacturing (LAAM) and other direct energy deposition (DED) processes. The deposition toolpaths for complex geometries with slender structures can be further optimized by partitioning the sliced 2D layers into sub-regions, and enable the design of appropriate infill toolpaths for different sub-regions. However, reported approaches fo… ▽ More Toolpath planning is an important task in laser aided additive manufacturing (LAAM) and other direct energy deposition (DED) processes. The deposition toolpaths for complex geometries with slender structures can be further optimized by partitioning the sliced 2D layers into sub-regions, and enable the design of appropriate infill toolpaths for different sub-regions. However, reported approaches for 2D layer segmentation generally require manual operations that are tedious and time-consuming. To increase segmentation efficiency, this paper proposes an autonomous approach based on evolutional computation for 2D layer segmentation. The algorithm works in an identify-and-segment manner. Specifically, the largest quasi-quadrilateral is identified and segmented from the target layer iteratively. Results from case studies have validated the effectiveness and efficacy of the developed algorithm. To further improve its performance, a roughing-finishing strategy is proposed. Via multi-processing, the strategy can remarkably increase the solution variety without affecting solution quality and search time, thus providing great application potential in LAAM toolpath planning. To the best of the authors knowledge, this work is the first to address automatic 2D layer segmentation problem in LAAM process. Therefore, it may be a valuable supplement to the state of the art in this area. △ Less

Submitted 26 June, 2020; v1 submitted 12 June, 2020; originally announced June 2020.

A Range-Doppler-Angle Estimation Method for Passive Bistatic Radar

Authors: Liangtian Wan, Xianpeng Wang, Guoan Bi

Abstract: In this paper, an effective target detection and localization method is proposed for a passive bistatic radar (PBR) system. The PBR system consists of a commercial FM radio station, which is a non-cooperative illuminator of opportunity (IO), referred to as the transmitter antenna and multiple surveillance antennas that form an antenna array, e.g., uniform linear array (ULA). Unlike other literatur… ▽ More In this paper, an effective target detection and localization method is proposed for a passive bistatic radar (PBR) system. The PBR system consists of a commercial FM radio station, which is a non-cooperative illuminator of opportunity (IO), referred to as the transmitter antenna and multiple surveillance antennas that form an antenna array, e.g., uniform linear array (ULA). Unlike other literatures where the reference signal is received by a directional antenna, here, the reference signal (direct path) is estimated by beamforming method. Then a modified extensive cancellation algorithm (MECA) based on (least squares) LS method is proposed to solve the disturbance cancellation. After cancelling the disturbance, the matched filter (MF) and LS methods are used for range-Doppler estimation of targets, and then the angles of targets are estimated based on beamforming method. The proposed method is suitable for an antenna array. Simulation results are presented to illustrate the superiority of the proposed MECA disturbance cancellation method and parameter estimation method. △ Less

Submitted 28 October, 2017; originally announced October 2017.

Comments: 5 pages, 7 figures

Towards Cooperation by Carrier Aggregation in Heterogeneous Networks: A Hierarchical Game Approach

Authors: Pu Yuan, Yong Xiao, Guoan Bi, Liren Zhang

Abstract: This paper studies the resource allocation problem for a heterogeneous network (HetNet) in which the spectrum owned by a macro-cell operator (MCO) can be shared by both unlicensed users (UUs) and licensed users (LUs). We formulate a novel hierarchical game theoretic framework to jointly optimize the transmit powers and sub-band allocations of the UUs as well as the pricing strategies of the MCO. I… ▽ More This paper studies the resource allocation problem for a heterogeneous network (HetNet) in which the spectrum owned by a macro-cell operator (MCO) can be shared by both unlicensed users (UUs) and licensed users (LUs). We formulate a novel hierarchical game theoretic framework to jointly optimize the transmit powers and sub-band allocations of the UUs as well as the pricing strategies of the MCO. In our framework, an overlapping coalition formation (OCF) game has been introduced to model the cooperative behaviors of the UUs. We then integrate this OCF game into a Stackelberg game-based hierarchical framework. We prove that the core of our proposed OCF game is non-empty and introduce an optimal sub-band allocation scheme for UUs. A simple distributed algorithm is proposed for UUs to autonomously form optimal coalition formation structure. The Stackelberg Equilibrium (SE) of the proposed hierarchical game is derived and its uniqueness and optimality have been proved. A distributed joint optimization algorithm is also proposed to approach the SE of the game with limited information exchanges between the MCO and the UU. △ Less

Submitted 13 November, 2015; originally announced November 2015.

Comments: 13 pages journal papaer

Sequence Design for Cognitive CDMA Communications under Arbitrary Spectrum Hole Constraint

Authors: Su Hu, Zilong Liu, Yong Liang Guan, Wenhui Xiong, Guoan Bi, Shaoqian Li

Abstract: To support interference-free quasi-synchronous code-division multiple-access (QS-CDMA) communication with low spectral density profile in a cognitive radio (CR) network, it is desirable to design a set of CDMA spreading sequences with zero-correlation zone (ZCZ) property. However, traditional ZCZ sequences (which assume the availability of the entire spectral band) cannot be used because their ort… ▽ More To support interference-free quasi-synchronous code-division multiple-access (QS-CDMA) communication with low spectral density profile in a cognitive radio (CR) network, it is desirable to design a set of CDMA spreading sequences with zero-correlation zone (ZCZ) property. However, traditional ZCZ sequences (which assume the availability of the entire spectral band) cannot be used because their orthogonality will be destroyed by the spectrum hole constraint in a CR channel. To date, analytical construction of ZCZ CR sequences remains open. Taking advantage of the Kronecker sequence property, a novel family of sequences (called "quasi-ZCZ" CR sequences) which displays zero cross-correlation and near-zero auto-correlation zone property under arbitrary spectrum hole constraint is presented in this paper. Furthermore, a novel algorithm is proposed to jointly optimize the peak-to-average power ratio (PAPR) and the periodic auto-correlations of the proposed quasi-ZCZ CR sequences. Simulations show that they give rise to single-user bit-error-rate performance in CR-CDMA systems which outperform traditional non-contiguous multicarrier CDMA and transform domain communication systems; they also lead to CR-CDMA systems which are more resilient than non-contiguous OFDM systems to spectrum sensing mismatch, due to the wideband spreading. △ Less

Submitted 4 October, 2014; originally announced October 2014.

Comments: 13 pages,10 figures,Accepted by IEEE Journal on Selected Areas in Communications (JSAC)--Special Issue:Cognitive Radio Nov, 2014

TDCS-based Cognitive Radio Networks with Multiuser Interference Avoidance

Authors: Su H, Guoan Bi, Yong Liang Guan, Shaoqian Li

Abstract: For overlay cognitive radio networks (CRNs), transform domain communication system (TDCS) has been proposed to support multiuser communications through spectrum bin nulling and frequency domain spreading. In TDCS-based CRNs, each user is assigned a specific pseudorandom spreading sequence. However, the existence of multiuser interference (MUI) is one of main concerns, due to the non-zero cross-cor… ▽ More For overlay cognitive radio networks (CRNs), transform domain communication system (TDCS) has been proposed to support multiuser communications through spectrum bin nulling and frequency domain spreading. In TDCS-based CRNs, each user is assigned a specific pseudorandom spreading sequence. However, the existence of multiuser interference (MUI) is one of main concerns, due to the non-zero cross-correlations between any pair of TDCS signals. In this paper, a novel framework of TDCS-based CRNs with the joint design of sequences and modulation schemes is presented to realize MUI avoidance. With the uncertainty of spectrum sensing results in CRNs, we first introduce a unique sequence design through two-dimensional time-frequency synthesis and obtain a class of almost perfect sequences. That is, periodic auto-correlation and cross-correlations are identically zero for most circular shifts. These correlation properties are further exploited in conjunction with a specially-designed cyclic code shift keying in order to achieve the advantage of MUI avoidance. Numerical results demonstrate that the proposed TDCS-based CRNs are considered as preferable candidates for decentralized networks against the near-far problem. △ Less

Submitted 20 November, 2013; originally announced November 2013.

Comments: to be appeared in IEEE Transaction on Communications, 2014

Journal ref: IEEE Transactions on Communications 61(12): 4828-4835, 2013

Cluster-based Transform Domain Communication Systems for High Spectrum Efficiency

Authors: Su Hu, Yong Liang Guan, Guoan Bi, Shaoqian Li

Abstract: This paper presents a cluster-based transform domain communication system (TDCS) to improve spectrum efficiency. Unlike the utilities of clusters in orthogonal frequency division multiplex (OFDM) systems, the cluster-based TDCS framework divides entire unoccupied spectrum bins into $L$ clusters, where each one represents a data steam independently, to achieve $L$ times of spectrum efficiency compa… ▽ More This paper presents a cluster-based transform domain communication system (TDCS) to improve spectrum efficiency. Unlike the utilities of clusters in orthogonal frequency division multiplex (OFDM) systems, the cluster-based TDCS framework divides entire unoccupied spectrum bins into $L$ clusters, where each one represents a data steam independently, to achieve $L$ times of spectrum efficiency compared to that of the traditional one. Among various schemes of spectrum bin spacing and allocation, the TDCS with random allocation scheme appears to be an ideal candidate to significantly improve spectrum efficiency without seriously degrading power efficiency. In multipath fading channel, the coded TDCS with random allocation scheme achieves robust BER performance due to a large degree of frequency diversity. Furthermore, our study shows that the smaller spectrum bin spacing should be configured for the cluster-based TDCS to achieve higher spectrum efficiency and more robust BER performance. △ Less

Submitted 15 December, 2012; originally announced December 2012.

Comments: 15 pages, 9 figures, Accepted for publication in IET Communications