-
EmoBench: Evaluating the Emotional Intelligence of Large Language Models
Authors:
Sahand Sabour,
Siyang Liu,
Zheyuan Zhang,
June M. Liu,
Jinfeng Zhou,
Alvionna S. Sunaryo,
Juanzi Li,
Tatia M. C. Lee,
Rada Mihalcea,
Minlie Huang
Abstract:
Recent advances in Large Language Models (LLMs) have highlighted the need for robust, comprehensive, and challenging benchmarks. Yet, research on evaluating their Emotional Intelligence (EI) is considerably limited. Existing benchmarks have two major shortcomings: first, they mainly focus on emotion recognition, neglecting essential EI capabilities such as emotion regulation and thought facilitati…
▽ More
Recent advances in Large Language Models (LLMs) have highlighted the need for robust, comprehensive, and challenging benchmarks. Yet, research on evaluating their Emotional Intelligence (EI) is considerably limited. Existing benchmarks have two major shortcomings: first, they mainly focus on emotion recognition, neglecting essential EI capabilities such as emotion regulation and thought facilitation through emotion understanding; second, they are primarily constructed from existing datasets, which include frequent patterns, explicit information, and annotation errors, leading to unreliable evaluation. We propose EmoBench, a benchmark that draws upon established psychological theories and proposes a comprehensive definition for machine EI, including Emotional Understanding and Emotional Application. EmoBench includes a set of 400 hand-crafted questions in English and Chinese, which are meticulously designed to require thorough reasoning and understanding. Our findings reveal a considerable gap between the EI of existing LLMs and the average human, highlighting a promising direction for future research. Our code and data are publicly available at https://github.com/Sahandfer/EmoBench.
△ Less
Submitted 17 July, 2024; v1 submitted 19 February, 2024;
originally announced February 2024.
-
CUAB: Convolutional Uncertainty Attention Block Enhanced the Chest X-ray Image Analysis
Authors:
Chi-Shiang Wang,
Fang-Yi Su,
Tsung-Lu Michael Lee,
Yi-Shan Tsai,
Jung-Hsien Chiang
Abstract:
In recent years, convolutional neural networks (CNNs) have been successfully implemented to various image recognition applications, such as medical image analysis, object detection, and image segmentation. Many studies and applications have been working on improving the performance of CNN algorithms and models. The strategies that aim to improve the performance of CNNs can be grouped into three ma…
▽ More
In recent years, convolutional neural networks (CNNs) have been successfully implemented to various image recognition applications, such as medical image analysis, object detection, and image segmentation. Many studies and applications have been working on improving the performance of CNN algorithms and models. The strategies that aim to improve the performance of CNNs can be grouped into three major approaches: (1) deeper and wider network architecture, (2) automatic architecture search, and (3) convolutional attention block. Unlike approaches (1) and (2), the convolutional attention block approach is more flexible with lower cost. It enhances the CNN performance by extracting more efficient features. However, the existing attention blocks focus on enhancing the significant features, which lose some potential features in the uncertainty information. Inspired by the test time augmentation and test-time dropout approaches, we developed a novel convolutional uncertainty attention block (CUAB) that can leverage the uncertainty information to improve CNN-based models. The proposed module discovers potential information from the uncertain regions on feature maps in computer vision tasks. It is a flexible functional attention block that can be applied to any position in the convolutional block in CNN models. We evaluated the CUAB with notable backbone models, ResNet and ResNeXt, on a medical image segmentation task. The CUAB achieved a dice score of 73% and 84% in pneumonia and pneumothorax segmentation, respectively, thereby outperforming the original model and other notable attention approaches. The results demonstrated that the CUAB can efficiently utilize the uncertainty information to improve the model performance.
△ Less
Submitted 4 May, 2021;
originally announced May 2021.
-
Efficient Cloth Simulation using Miniature Cloth and Upscaling Deep Neural Networks
Authors:
Tae Min Lee,
Young Jin Oh,
In-Kwon Lee
Abstract:
Cloth simulation requires a fast and stable method for interactively and realistically visualizing fabric materials using computer graphics. We propose an efficient cloth simulation method using miniature cloth simulation and upscaling Deep Neural Networks (DNN). The upscaling DNNs generate the target cloth simulation from the results of physically-based simulations of a miniature cloth that has s…
▽ More
Cloth simulation requires a fast and stable method for interactively and realistically visualizing fabric materials using computer graphics. We propose an efficient cloth simulation method using miniature cloth simulation and upscaling Deep Neural Networks (DNN). The upscaling DNNs generate the target cloth simulation from the results of physically-based simulations of a miniature cloth that has similar physical properties to those of the target cloth. We have verified the utility of the proposed method through experiments, and the results demonstrate that it is possible to generate fast and stable cloth simulations under various conditions.
△ Less
Submitted 8 July, 2019;
originally announced July 2019.
-
Hierarchical Cloth Simulation using Deep Neural Networks
Authors:
Young Jin Oh,
Tae Min Lee,
In-Kwon Lee
Abstract:
Fast and reliable physically-based simulation techniques are essential for providing flexible visual effects for computer graphics content. In this paper, we propose a fast and reliable hierarchical cloth simulation method, which combines conventional physically-based simulation with deep neural networks (DNN). Simulations of the coarsest level of the hierarchical model are calculated using conven…
▽ More
Fast and reliable physically-based simulation techniques are essential for providing flexible visual effects for computer graphics content. In this paper, we propose a fast and reliable hierarchical cloth simulation method, which combines conventional physically-based simulation with deep neural networks (DNN). Simulations of the coarsest level of the hierarchical model are calculated using conventional physically-based simulations, and more detailed levels are generated by inference using DNN models. We demonstrate that our method generates reliable and fast cloth simulation results through experiments under various conditions.
△ Less
Submitted 9 February, 2018;
originally announced February 2018.
-
Condensedly: comprehending article contents through condensed texts
Authors:
Chao-Hsuan Ke,
Tsung-Lu Michael Lee,
Jung-Hsien Chiang
Abstract:
Summary: Abstracts in biomedical articles can provide a quick overview of the articles but detailed information cannot be obtained without reading full-text contents. Full-text articles certainly generate more information and contents; however, accessing full-text documents is usually time consuming. Condensedly is a web-based application, which provides readers an easy and efficient way to access…
▽ More
Summary: Abstracts in biomedical articles can provide a quick overview of the articles but detailed information cannot be obtained without reading full-text contents. Full-text articles certainly generate more information and contents; however, accessing full-text documents is usually time consuming. Condensedly is a web-based application, which provides readers an easy and efficient way to access full-text paragraphs using sentences in abstracts as fishing bait to retrieve the big fish reside in full-text. Condensedly is based on the paragraph ranking algorithm, which evaluates and ranks full-text paragraphs based on their association scores with sentences in abstracts.
Availability: http://140.116.247.185/~research/Condensedly
△ Less
Submitted 29 December, 2016;
originally announced December 2016.
