Showing 51–100 of 280 results for author: Lam, W

Constraining the reflective properties of WASP-178b using Cheops photometry

Authors: I. Pagano, G. Scandariato, V. Singh, M. Lendl, D. Queloz, A. E. Simon, S. G. Sousa, A. Brandeker, A. Collier Cameron, S. Sulis, V. Van Grootel, T. G. Wilson, Y. Alibert, R. Alonso, G. Anglada, T. Bárczy, D. Barrado Navascues, S. C. C. Barros, W. Baumjohann, M. Beck, T. Beck, W. Benz, N. Billot, X. Bonfils, L. Borsato , et al. (57 additional authors not shown)

Abstract: Multiwavelength photometry of the secondary eclipses of extrasolar planets is able to disentangle the reflected and thermally emitted light radiated from the planetary dayside. This leads to the measurement of the planetary geometric albedo $A_g$, which is an indicator of the presence of clouds in the atmosphere, and the recirculation efficiency $ε$, which quantifies the energy transport within th… ▽ More Multiwavelength photometry of the secondary eclipses of extrasolar planets is able to disentangle the reflected and thermally emitted light radiated from the planetary dayside. This leads to the measurement of the planetary geometric albedo $A_g$, which is an indicator of the presence of clouds in the atmosphere, and the recirculation efficiency $ε$, which quantifies the energy transport within the atmosphere. In this work we aim to measure $A_g$ and $ε$ for the planet WASP-178 b, a highly irradiated giant planet with an estimated equilibrium temperature of 2450 K.} We analyzed archival spectra and the light curves collected by Cheops and Tess to characterize the host WASP-178, refine the ephemeris of the system and measure the eclipse depth in the passbands of the two respective telescopes. We measured a marginally significant eclipse depth of 70$\pm$40 ppm in the Tess passband and statistically significant depth of 70$\pm$20 ppm in the Cheops passband. Combining the eclipse depth measurement in the Cheops (lambda_eff=6300 AA) and Tess (lambda_eff=8000 AA) passbands we constrained the dayside brightness temperature of WASP-178 b in the 2250-2800 K interval. The geometric albedo 0.1<$\rm A_g$<0.35 is in general agreement with the picture of poorly reflective giant planets, while the recirculation efficiency $ε>$0.7 makes WASP-178 b an interesting laboratory to test the current heat recirculation models. △ Less

Submitted 16 September, 2023; originally announced September 2023.

Comments: Accepted by Astronomy and Astrophysics on 31/08/2023

Distributional Inclusion Hypothesis and Quantifications: Probing for Hypernymy in Functional Distributional Semantics

Authors: Chun Hei Lo, Wai Lam, Hong Cheng, Guy Emerson

Abstract: Functional Distributional Semantics (FDS) models the meaning of words by truth-conditional functions. This provides a natural representation for hypernymy but no guarantee that it can be learnt when FDS models are trained on a corpus. In this paper, we probe into FDS models and study the representations learnt, drawing connections between quantifications, the Distributional Inclusion Hypothesis (D… ▽ More Functional Distributional Semantics (FDS) models the meaning of words by truth-conditional functions. This provides a natural representation for hypernymy but no guarantee that it can be learnt when FDS models are trained on a corpus. In this paper, we probe into FDS models and study the representations learnt, drawing connections between quantifications, the Distributional Inclusion Hypothesis (DIH), and the variational-autoencoding objective of FDS model training. Using synthetic data sets, we reveal that FDS models learn hypernymy on a restricted class of corpus that strictly follows the DIH. We further introduce a training objective that both enables hypernymy learning under the reverse of the DIH and improves hypernymy detection from real corpora. △ Less

Submitted 10 February, 2024; v1 submitted 15 September, 2023; originally announced September 2023.

Comments: 12 pages

Nuclear ground-state properties probed by the relativistic Hartree-Bogoliubov approach

Authors: Zi Xin Liu, Yi Hua Lam, Ning Lu, Peter Ring

Abstract: Using the relativistic Hartree-Bogoliubov framework with separable pairing force coupled with the latest covariant density functionals, i.e., PC-L3R, PC-X, DD-PCX, and DD-MEX, we systematically explore the ground-state properties of all isotopes of Z=8-110. These properties consist of the binding energies, one- and two-neutron separation energies ($S_\mathrm{n}$ and $S_\mathrm{2n}$), root-mean-squ… ▽ More Using the relativistic Hartree-Bogoliubov framework with separable pairing force coupled with the latest covariant density functionals, i.e., PC-L3R, PC-X, DD-PCX, and DD-MEX, we systematically explore the ground-state properties of all isotopes of Z=8-110. These properties consist of the binding energies, one- and two-neutron separation energies ($S_\mathrm{n}$ and $S_\mathrm{2n}$), root-mean-square radius of matter, of neutron, of proton, and of charge distributions, Fermi surfaces, ground-state spins and parities. We then predict the edges of nuclear landscape and bound nuclei for the isotopic chains from oxygen (Z=8) to darmstadtium (Z=110) based on these latest covariant density functionals. The number of bound nuclei predicted by PC-L3R, PC-X, DD-PCX, and DD-MEX, are 9004, 9162, 6799, and 7112, respectively. The root-mean-square deviations of $S_\mathrm{n}$ ($S_\mathrm{2n}$) yielded from PC-L3R, PCX, DD-PCX, and DD-MEX are 0.962 (1.300) MeV, 0.920 (1.483) MeV, 0.993 (1.753) MeV, and 1.010 (1.544) MeV, respectively. The root-mean-square deviations of charge radius distributions of comparing the available experimental values with the theoretical counterparts resulted from PC-L3R, PC-X, DD-PCX, and DD-MEX are 0.035 fm, 0.037 fm, 0.035 fm, and 0.034 fm, respectively. We notice pronounced differences between the empirical and theoretical root-mean-square radii of neutron at nuclei near the neutron drip line of the Mg, Ca, and Kr isotopic chains, suggesting the possible existence of the halo or giant halo phenomena. △ Less

Submitted 18 January, 2024; v1 submitted 12 September, 2023; originally announced September 2023.

Comments: Finalized version, 472 pages (arXiv version), 11 main figures (75 subfigures, colorblind-friendly colors), 6 tables, accepted on 18 December 2023 and to appear at Atomic Data and Nuclear Data Tables, data availability: https://github.com/lamyihua/nuclear-ground-state-properties

Journal ref: Atomic Data and Nuclear Data Tables 156 (2024) 101635

EPA: Easy Prompt Augmentation on Large Language Models via Multiple Sources and Multiple Targets

Authors: Hongyuan Lu, Wai Lam

Abstract: Large language models (LLMs) have shown promising performance on various NLP tasks via task prompting. And their performance can be further improved by appending task demonstrations to the head of the prompt. And usually, a better performance can be achieved with more demonstrations. However, asking the users to write the demonstrations can be cumbersome. As a simple yet cost-effective workaround,… ▽ More Large language models (LLMs) have shown promising performance on various NLP tasks via task prompting. And their performance can be further improved by appending task demonstrations to the head of the prompt. And usually, a better performance can be achieved with more demonstrations. However, asking the users to write the demonstrations can be cumbersome. As a simple yet cost-effective workaround, this paper proposes a novel method called EPA (\textbf{E}asy \textbf{P}rompt \textbf{A}ugmentation)\footnote{While this paper considers augmenting prompts via demonstrations, we name it EPA as the name EDA is already taken by a well-known NLP method \citep{wei-zou-2019-eda}.} that effectively minimizes user efforts in writing demonstrations while improving the model performance at the same time. EPA achieves these goals by automatically augmenting the demonstrations with multiple sources/targets, where each of them paraphrases each other. This is well motivated as augmenting data via paraphrasing effectively improves neural language models. EPA thus employs paraphrasing as an augmentation method for in-context learning. Extensive experiments indicate that EPA effectively improves both NLU and NLG tasks, covering from natural language inference to machine translation in translating tens of languages.\footnote{Code and data will be released upon publication.} △ Less

Submitted 9 September, 2023; originally announced September 2023.

Exceptional behavior in critical first-passage percolation and random sums

Authors: Michael Damron, Jack Hanson, David Harper, Wai-Kit Lam

Abstract: We study first-passage percolation (FPP) on the square lattice. The model is defined using i.i.d. nonnegative random edge-weights $(t_e)$ associated to the nearest neighbor edges of $\mathbb{Z}^2$. The passage time between vertices $x$ and $y$, $T(x,y)$, is the minimal total weight of any lattice path from $x$ to $y$. The growth rate of $T(x,y)$ depends on the value of $F(0) = \mathbb{P}(t_e=0)$:… ▽ More We study first-passage percolation (FPP) on the square lattice. The model is defined using i.i.d. nonnegative random edge-weights $(t_e)$ associated to the nearest neighbor edges of $\mathbb{Z}^2$. The passage time between vertices $x$ and $y$, $T(x,y)$, is the minimal total weight of any lattice path from $x$ to $y$. The growth rate of $T(x,y)$ depends on the value of $F(0) = \mathbb{P}(t_e=0)$: if $F(0) < 1/2$ then $T(x,y)$ grows linearly in $|x-y|$, but if $F(0) > 1/2$ then it is stochastically bounded. In the critical case, where $F(0) = 1/2$, $T(x,y)$ can be bounded or unbounded depending on the behavior of the distribution function $F$ of $t_e$ near 0. In this paper, we consider the critical case in which $T(x,y)$ is unbounded and prove the existence of an incipient infinite cluster (IIC) type measure, constructed by conditioning the environment on the event that the passage time from $0$ to a far distance remains bounded. This IIC measure is a natural candidate for the distribution of the weights at a typical exceptional time in dynamical FPP. A major part of the analysis involves characterizing the limiting behavior of independent nonnegative random variables conditioned to have small sum. We give conditions on random variables that ensure that such limits are trivial, and several examples that exhibit nontrivial limits. △ Less

Submitted 19 August, 2023; originally announced August 2023.

Comments: 28 pages, 2 figures

MSC Class: 60K35; 82B43

CHEOPS and TESS view of the ultra-short period super-Earth TOI-561 b

Authors: J. A. Patel, J. A. Egger, T. G. Wilson, V. Bourrier, L. Carone, M. Beck, D. Ehrenreich, S. G. Sousa, W. Benz, A. Brandeker, A. Deline, Y. Alibert, K. W. F. Lam, M. Lendl, R. Alonso, G. Anglada, T. Bárczy, D. Barrado, S. C. C. Barros, W. Baumjohann, T. Beck, N. Billot, X. Bonfils, C. Broeg, M. -D. Busch , et al. (53 additional authors not shown)

Abstract: Ultra-short period planets (USPs) are a unique class of super-Earths with an orbital period of less than a day and hence subject to intense radiation from their host star. While most of them are consistent with bare rocks, some show evidence of a heavyweight envelope, which could be a water layer or a secondary metal-rich atmosphere sustained by an outgassing surface. Much remains to be learned ab… ▽ More Ultra-short period planets (USPs) are a unique class of super-Earths with an orbital period of less than a day and hence subject to intense radiation from their host star. While most of them are consistent with bare rocks, some show evidence of a heavyweight envelope, which could be a water layer or a secondary metal-rich atmosphere sustained by an outgassing surface. Much remains to be learned about the nature of USPs. The prime goal of the present work is to study the bulk planetary properties and atmosphere of TOI-561b, through the study of its transits and occultations. We obtained ultra-precise transit photometry of TOI-561b with CHEOPS and performed a joint analysis of this data with four TESS sectors. Our analysis of TOI-561b transit photometry put strong constraints on its properties, especially on its radius, Rp=1.42 +/- 0.02 R_Earth (at ~2% error). The internal structure modelling of the planet shows that the observations are consistent with negligible H/He atmosphere, however requiring other lighter materials, in addition to pure iron core and silicate mantle to explain the observed density. We find that this can be explained by the inclusion of a water layer in our model. We searched for variability in the measured Rp/R* over time to trace changes in the structure of the planetary envelope but none found within the data precision. In addition to the transit event, we tentatively detect occultation signal in the TESS data with an eclipse depth of ~27 +/- 11 ppm. Using the models of outgassed atmospheres from the literature we find that the thermal emission from the planet can mostly explain the observation. Based on this, we predict that NIR/MIR observations with JWST should be able to detect silicate species in the atmosphere of the planet. This could also reveal important clues about the planetary interior and help disentangle planet formation and evolution models. △ Less

Submitted 16 August, 2023; originally announced August 2023.

Comments: 17 pages, 10 + 3 figures, 4 tables, accepted for publication in A&A (abstract abbreviated)

Journal ref: A&A 679, A92 (2023)

Company for the ultra-high density, ultra-short period sub-Earth GJ 367 b: discovery of two additional low-mass planets at 11.5 and 34 days

Authors: Elisa Goffo, Davide Gandolfi, Jo Ann Egger, Alexander J. Mustill, Simon H. Albrecht, Teruyuki Hirano, Oleg Kochukhov, Nicola Astudillo-Defru, Oscar Barragan, Luisa M. Serrano, Artie P. Hatzes, Yann Alibert, Eike Guenther, Fei Dai, Kristine W. F. Lam, Szilárd Csizmadia, Alexis M. S. Smith, Luca Fossati, Rafael Luque, Florian Rodler, Mark L. Winther, Jakob L. Rørsted, Javier Alarcon, Xavier Bonfils, William D. Cochran , et al. (16 additional authors not shown)

Abstract: GJ 367 is a bright (V $\approx$ 10.2) M1 V star that has been recently found to host a transiting ultra-short period sub-Earth on a 7.7 hr orbit. With the aim of improving the planetary mass and radius and unveiling the inner architecture of the system, we performed an intensive radial velocity follow-up campaign with the HARPS spectrograph -- collecting 371 high-precision measurements over a base… ▽ More GJ 367 is a bright (V $\approx$ 10.2) M1 V star that has been recently found to host a transiting ultra-short period sub-Earth on a 7.7 hr orbit. With the aim of improving the planetary mass and radius and unveiling the inner architecture of the system, we performed an intensive radial velocity follow-up campaign with the HARPS spectrograph -- collecting 371 high-precision measurements over a baseline of nearly 3 years -- and combined our Doppler measurements with new TESS observations from sectors 35 and 36. We found that GJ 367 b has a mass of $M_\mathrm{b}$ = 0.633 $\pm$ 0.050 M$_{\oplus}$ and a radius of $R_\mathrm{b}$ = 0.699 $\pm$ 0.024 R$_{\oplus}$, corresponding to precisions of 8% and 3.4%, respectively. This implies a planetary bulk density of $ρ_\mathrm{b}$ = 10.2 $\pm$ 1.3 g cm$^{-3}$, i.e., 85% higher than Earth's density. We revealed the presence of two additional non transiting low-mass companions with orbital periods of $\sim$11.5 and 34 days and minimum masses of $M_\mathrm{c}\sin{i_\mathrm{c}}$ = 4.13 $\pm$ 0.36 M$_{\oplus}$ and $M_\mathrm{d}\sin{i_\mathrm{d}}$ = 6.03 $\pm$ 0.49 M$_{\oplus}$, respectively, which lie close to the 3:1 mean motion commensurability. GJ 367 b joins the small class of high-density planets, namely the class of super-Mercuries, being the densest ultra-short period small planet known to date. Thanks to our precise mass and radius estimates, we explored the potential internal composition and structure of GJ 367 b, and found that it is expected to have an iron core with a mass fraction of 0.91$^{+0.07}_{-0.23}$. How this iron core is formed and how such a high density is reached is still not clear, and we discuss the possible pathways of formation of such a small ultra-dense planet. △ Less

Submitted 18 July, 2023; originally announced July 2023.

Comments: 28 pages, 11 figures. Accepted for publication in ApJL

Journal ref: ApJL 955 L3 (2023)

The News Delivery Channel Recommendation Based on Granular Neural Network

Authors: Lin Wu, Rui Li, Jiaxuan Liu, Wong-Hing Lam

Abstract: With the continuous maturation and expansion of neural network technology, deep neural networks have been widely utilized as the fundamental building blocks of deep learning in a variety of applications, including speech recognition, machine translation, image processing, and the creation of recommendation systems. Therefore, many real-world complex problems can be solved by the deep learning tech… ▽ More With the continuous maturation and expansion of neural network technology, deep neural networks have been widely utilized as the fundamental building blocks of deep learning in a variety of applications, including speech recognition, machine translation, image processing, and the creation of recommendation systems. Therefore, many real-world complex problems can be solved by the deep learning techniques. As is known, traditional news recommendation systems mostly employ techniques based on collaborative filtering and deep learning, but the performance of these algorithms is constrained by the sparsity of the data and the scalability of the approaches. In this paper, we propose a recommendation model using granular neural network model to recommend news to appropriate channels by analyzing the properties of news. Specifically, a specified neural network serves as the foundation for the granular neural network that the model is considered to be build. Different information granularities are attributed to various types of news material, and different information granularities are released between networks in various ways. When processing data, granular output is created, which is compared to the interval values pre-set on various platforms and used to quantify the analysis's effectiveness. The analysis results could help the media to match the proper news in depth, maximize the public attention of the news and the utilization of media resources. △ Less

Submitted 30 May, 2023; originally announced June 2023.

A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers

Authors: Xiaoyan Zhao, Yang Deng, Min Yang, Lingzhi Wang, Rui Zhang, Hong Cheng, Wai Lam, Ying Shen, Ruifeng Xu

Abstract: Relation extraction (RE) involves identifying the relations between entities from underlying content. RE serves as the foundation for many natural language processing (NLP) and information retrieval applications, such as knowledge graph completion and question answering. In recent years, deep neural networks have dominated the field of RE and made noticeable progress. Subsequently, the large pre-t… ▽ More Relation extraction (RE) involves identifying the relations between entities from underlying content. RE serves as the foundation for many natural language processing (NLP) and information retrieval applications, such as knowledge graph completion and question answering. In recent years, deep neural networks have dominated the field of RE and made noticeable progress. Subsequently, the large pre-trained language models have taken the state-of-the-art RE to a new level. This survey provides a comprehensive review of existing deep learning techniques for RE. First, we introduce RE resources, including datasets and evaluation metrics. Second, we propose a new taxonomy to categorize existing works from three perspectives, i.e., text representation, context encoding, and triplet prediction. Third, we discuss several important challenges faced by RE and summarize potential techniques to tackle these challenges. Finally, we outline some promising future directions and prospects in this field. This survey is expected to facilitate researchers' collaborative efforts to address the challenges of real-world RE systems. △ Less

Submitted 24 June, 2024; v1 submitted 3 June, 2023; originally announced June 2023.

Research on Multilingual News Clustering Based on Cross-Language Word Embeddings

Authors: Lin Wu, Rui Li, Wong-Hing Lam

Abstract: Classifying the same event reported by different countries is of significant importance for public opinion control and intelligence gathering. Due to the diverse types of news, relying solely on transla-tors would be costly and inefficient, while depending solely on translation systems would incur considerable performance overheads in invoking translation interfaces and storing translated texts. T… ▽ More Classifying the same event reported by different countries is of significant importance for public opinion control and intelligence gathering. Due to the diverse types of news, relying solely on transla-tors would be costly and inefficient, while depending solely on translation systems would incur considerable performance overheads in invoking translation interfaces and storing translated texts. To address this issue, we mainly focus on the clustering problem of cross-lingual news. To be specific, we use a combination of sentence vector representations of news headlines in a mixed semantic space and the topic probability distributions of news content to represent a news article. In the training of cross-lingual models, we employ knowledge distillation techniques to fit two semantic spaces into a mixed semantic space. We abandon traditional static clustering methods like K-Means and AGNES in favor of the incremental clustering algorithm Single-Pass, which we further modify to better suit cross-lingual news clustering scenarios. Our main contributions are as follows: (1) We adopt the English standard BERT as the teacher model and XLM-Roberta as the student model, training a cross-lingual model through knowledge distillation that can represent sentence-level bilingual texts in both Chinese and English. (2) We use the LDA topic model to represent news as a combina-tion of cross-lingual vectors for headlines and topic probability distributions for con-tent, introducing concepts such as topic similarity to address the cross-lingual issue in news content representation. (3) We adapt the Single-Pass clustering algorithm for the news context to make it more applicable. Our optimizations of Single-Pass include ad-justing the distance algorithm between samples and clusters, adding cluster merging operations, and incorporating a news time parameter. △ Less

Submitted 30 May, 2023; originally announced May 2023.

TOI-1416: A system with a super-Earth planet with a 1.07d period

Authors: H. J. Deeg, I. Y. Georgieva, G. Nowak, C. M. Persson, B. L. Cale, F. Murgas, E. Pallé, D. Godoy Rivera, F. Dai, D. R. Ciardi, J. M. Akana Murphy, P. G. Beck, C. J. Burke, J. Cabrera, I. Carleo, W. D. Cochran, K. A. Collins, Sz. Csizmadia, M. El Mufti, M. Fridlund, A. Fukui, D. Gandolfi, R. A. García, E. W. Guenther, P. Guerra , et al. (27 additional authors not shown)

Abstract: TOI 1416 (BD+42 2504, HIP 70705) is a V=10 late G or early K-type dwarf star with transits detected by TESS. Radial velocities verify the presence of the transiting planet TOI-1416 b, with a period of 1.07d, a mass of $3.48 M_{Earth}$ and a radius of $1.62 R_{Earth}$, implying a slightly sub-Earth density of $4.50$ g cm$^{-3}$. The RV data also further indicate a tentative planet c with a period o… ▽ More TOI 1416 (BD+42 2504, HIP 70705) is a V=10 late G or early K-type dwarf star with transits detected by TESS. Radial velocities verify the presence of the transiting planet TOI-1416 b, with a period of 1.07d, a mass of $3.48 M_{Earth}$ and a radius of $1.62 R_{Earth}$, implying a slightly sub-Earth density of $4.50$ g cm$^{-3}$. The RV data also further indicate a tentative planet c with a period of 27.4 or 29.5 days, whose nature cannot be verified due to strong suspicions about contamination by a signal related to the Moon's synodic period of 29.53 days. The near-USP (Ultra Short Period) planet TOI-1416 b is a typical representative of a short-period and hot ($T_{eq} \approx$ 1570 K) super-Earth like planet. A planet model of an interior of molten magma containing a significant fraction of dissolved water provides a plausible explanation for its composition, and its atmosphere could be suitable for transmission spectroscopy with JWST. The position of TOI-1416 b within the radius-period distribution corroborates that USPs with periods of less than one day do not form any special group of planets. Rather, this implies that USPs belong to a continuous distribution of super-Earth like planets with periods ranging from the shortest known ones up to ~ 30 days, whose period-radius distribution is delimitated against larger radii by the Neptune desert and by the period-radius valley that separates super-Earths from sub-Neptune planets. In the abundance of small-short periodic planets against period, a plateau between periods of 0.6 to 1.4 days has however become notable that is compatible with the low-eccentricity formation channel. For the Neptune desert, its lower limits required a revision due to the increasing population of short period planets and new limits are provided. These limits are also given in terms of the planets' insolation and effective temperatures. △ Less

Submitted 29 May, 2023; originally announced May 2023.

Comments: 31 pages, 31 figures, 8 tables, accepted for publication in A&A

Journal ref: A&A 677, A12 (2023)

Diverse and Expressive Speech Prosody Prediction with Denoising Diffusion Probabilistic Model

Authors: Xiang Li, Songxiang Liu, Max W. Y. Lam, Zhiyong Wu, Chao Weng, Helen Meng

Abstract: Expressive human speech generally abounds with rich and flexible speech prosody variations. The speech prosody predictors in existing expressive speech synthesis methods mostly produce deterministic predictions, which are learned by directly minimizing the norm of prosody prediction error. Its unimodal nature leads to a mismatch with ground truth distribution and harms the model's ability in makin… ▽ More Expressive human speech generally abounds with rich and flexible speech prosody variations. The speech prosody predictors in existing expressive speech synthesis methods mostly produce deterministic predictions, which are learned by directly minimizing the norm of prosody prediction error. Its unimodal nature leads to a mismatch with ground truth distribution and harms the model's ability in making diverse predictions. Thus, we propose a novel prosody predictor based on the denoising diffusion probabilistic model to take advantage of its high-quality generative modeling and training stability. Experiment results confirm that the proposed prosody predictor outperforms the deterministic baseline on both the expressiveness and diversity of prediction results with even fewer network parameters. △ Less

Submitted 7 October, 2023; v1 submitted 26 May, 2023; originally announced May 2023.

Comments: Proceedings of Interspeech 2023 (doi: 10.21437/Interspeech.2023-715), demo site at https://thuhcsi.github.io/interspeech2023-DiffVar/

Efficient Neural Music Generation

Authors: Max W. Y. Lam, Qiao Tian, Tang Li, Zongyu Yin, Siyuan Feng, Ming Tu, Yuliang Ji, Rui Xia, Mingbo Ma, Xuchen Song, Jitong Chen, Yuping Wang, Yuxuan Wang

Abstract: Recent progress in music generation has been remarkably advanced by the state-of-the-art MusicLM, which comprises a hierarchy of three LMs, respectively, for semantic, coarse acoustic, and fine acoustic modelings. Yet, sampling with the MusicLM requires processing through these LMs one by one to obtain the fine-grained acoustic tokens, making it computationally expensive and prohibitive for a real… ▽ More Recent progress in music generation has been remarkably advanced by the state-of-the-art MusicLM, which comprises a hierarchy of three LMs, respectively, for semantic, coarse acoustic, and fine acoustic modelings. Yet, sampling with the MusicLM requires processing through these LMs one by one to obtain the fine-grained acoustic tokens, making it computationally expensive and prohibitive for a real-time generation. Efficient music generation with a quality on par with MusicLM remains a significant challenge. In this paper, we present MeLoDy (M for music; L for LM; D for diffusion), an LM-guided diffusion model that generates music audios of state-of-the-art quality meanwhile reducing 95.7% or 99.6% forward passes in MusicLM, respectively, for sampling 10s or 30s music. MeLoDy inherits the highest-level LM from MusicLM for semantic modeling, and applies a novel dual-path diffusion (DPD) model and an audio VAE-GAN to efficiently decode the conditioning semantic tokens into waveform. DPD is proposed to simultaneously model the coarse and fine acoustics by incorporating the semantic information into segments of latents effectively via cross-attention at each denoising step. Our experimental results suggest the superiority of MeLoDy, not only in its practical advantages on sampling speed and infinitely continuable generation, but also in its state-of-the-art musicality, audio quality, and text correlation. Our samples are available at https://Efficient-MeLoDy.github.io/. △ Less

Submitted 25 May, 2023; originally announced May 2023.

Enhancing Grammatical Error Correction Systems with Explanations

Authors: Yuejiao Fei, Leyang Cui, Sen Yang, Wai Lam, Zhenzhong Lan, Shuming Shi

Abstract: Grammatical error correction systems improve written communication by detecting and correcting language mistakes. To help language learners better understand why the GEC system makes a certain correction, the causes of errors (evidence words) and the corresponding error types are two key factors. To enhance GEC systems with explanations, we introduce EXPECT, a large dataset annotated with evidence… ▽ More Grammatical error correction systems improve written communication by detecting and correcting language mistakes. To help language learners better understand why the GEC system makes a certain correction, the causes of errors (evidence words) and the corresponding error types are two key factors. To enhance GEC systems with explanations, we introduce EXPECT, a large dataset annotated with evidence words and grammatical error types. We propose several baselines and analysis to understand this task. Furthermore, human evaluation verifies our explainable GEC system's explanations can assist second-language learners in determining whether to accept a correction suggestion and in understanding the associated grammar rule. △ Less

Submitted 10 June, 2023; v1 submitted 24 May, 2023; originally announced May 2023.

Comments: 9 pages, 7 figures, accepted to the main conference of ACL 2023

mPMR: A Multilingual Pre-trained Machine Reader at Scale

Authors: Weiwen Xu, Xin Li, Wai Lam, Lidong Bing

Abstract: We present multilingual Pre-trained Machine Reader (mPMR), a novel method for multilingual machine reading comprehension (MRC)-style pre-training. mPMR aims to guide multilingual pre-trained language models (mPLMs) to perform natural language understanding (NLU) including both sequence classification and span extraction in multiple languages. To achieve cross-lingual generalization when only sourc… ▽ More We present multilingual Pre-trained Machine Reader (mPMR), a novel method for multilingual machine reading comprehension (MRC)-style pre-training. mPMR aims to guide multilingual pre-trained language models (mPLMs) to perform natural language understanding (NLU) including both sequence classification and span extraction in multiple languages. To achieve cross-lingual generalization when only source-language fine-tuning data is available, existing mPLMs solely transfer NLU capability from a source language to target languages. In contrast, mPMR allows the direct inheritance of multilingual NLU capability from the MRC-style pre-training to downstream tasks. Therefore, mPMR acquires better NLU capability for target languages. mPMR also provides a unified solver for tackling cross-lingual span extraction and sequence classification, thereby enabling the extraction of rationales to explain the sentence-pair classification process. △ Less

Submitted 22 May, 2023; originally announced May 2023.

Comments: To appear at ACL 2023 main conference

A Frustratingly Simple Decoding Method for Neural Text Generation

Authors: Haoran Yang, Deng Cai, Huayang Li, Wei Bi, Wai Lam, Shuming Shi

Abstract: We introduce a frustratingly simple, super efficient and surprisingly effective decoding method, which we call Frustratingly Simple Decoding (FSD), for neural text generation. The idea behind FSD is straightforward: we build an anti-LM based on previously generated text and use this anti-LM to penalize future generation of what has been generated. The anti-LM can be implemented as simple as an n-g… ▽ More We introduce a frustratingly simple, super efficient and surprisingly effective decoding method, which we call Frustratingly Simple Decoding (FSD), for neural text generation. The idea behind FSD is straightforward: we build an anti-LM based on previously generated text and use this anti-LM to penalize future generation of what has been generated. The anti-LM can be implemented as simple as an n-gram language model or a vectorized variant. In this way, FSD introduces no extra model parameters and negligible computational overhead (FSD can be as fast as greedy search). Despite the simplicity, FSD is surprisingly effective; Experiments show that FSD can outperform the canonical methods to date (i.e., nucleus sampling) as well as several strong baselines that were proposed recently. △ Less

Submitted 27 February, 2024; v1 submitted 21 May, 2023; originally announced May 2023.

Comments: LREC-Coling 2024

Chain-of-Symbol Prompting Elicits Planning in Large Langauge Models

Authors: Hanxu Hu, Hongyuan Lu, Huajian Zhang, Yun-Ze Song, Wai Lam, Yue Zhang

Abstract: In this paper, we take the initiative to investigate the performance of LLMs on complex planning tasks that require LLMs to understand a virtual spatial environment simulated via natural language and act correspondingly in text. We propose a benchmark named Natural Language Planning and Action (Natala) composed of a set of novel tasks: Brick World, NLVR-based Manipulations, and Natural Language Na… ▽ More In this paper, we take the initiative to investigate the performance of LLMs on complex planning tasks that require LLMs to understand a virtual spatial environment simulated via natural language and act correspondingly in text. We propose a benchmark named Natural Language Planning and Action (Natala) composed of a set of novel tasks: Brick World, NLVR-based Manipulations, and Natural Language Navigation. We found that current popular LLMs such as ChatGPT still lack abilities in complex planning. This arises a question -- do the LLMs have a good understanding of the environments described in natural language, or maybe other alternatives such as symbolic representations are neater and hence better to be understood by LLMs? To this end, we propose a novel method called CoS (Chain-of-Symbol Prompting) that represents the complex environments with condensed symbolic spatial representations during the chained intermediate thinking steps. CoS is easy to use and does not need additional training on LLMs. Extensive experiments indicate that CoS clearly surpasses the performance of the Chain-of-Thought (CoT) Prompting in all three planning tasks with even fewer tokens used in the inputs compared with CoT on ChatGPT and InstructGPT. The performance gain is strong, by up to 60.8% accuracy (from 31.8% to 92.6%) on Brick World for ChatGPT. CoS also reduces the number of tokens in the prompt obviously, by up to 65.8% of the tokens (from 407 to 139) for the intermediate steps from demonstrations on Brick World. Code and data available at: https://github.com/hanxuhu/chain-of-symbol-planning △ Less

Submitted 5 August, 2024; v1 submitted 17 May, 2023; originally announced May 2023.

Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversations

Authors: Yang Deng, Wenxuan Zhang, Yifei Yuan, Wai Lam

Abstract: Unlike empathetic dialogues, the system in emotional support conversations (ESC) is expected to not only convey empathy for comforting the help-seeker, but also proactively assist in exploring and addressing their problems during the conversation. In this work, we study the problem of mixed-initiative ESC where the user and system can both take the initiative in leading the conversation. Specifica… ▽ More Unlike empathetic dialogues, the system in emotional support conversations (ESC) is expected to not only convey empathy for comforting the help-seeker, but also proactively assist in exploring and addressing their problems during the conversation. In this work, we study the problem of mixed-initiative ESC where the user and system can both take the initiative in leading the conversation. Specifically, we conduct a novel analysis on mixed-initiative ESC systems with a tailor-designed schema that divides utterances into different types with speaker roles and initiative types. Four emotional support metrics are proposed to evaluate the mixed-initiative interactions. The analysis reveals the necessity and challenges of building mixed-initiative ESC systems. In the light of this, we propose a knowledge-enhanced mixed-initiative framework (KEMI) for ESC, which retrieves actual case knowledge from a large-scale mental health knowledge graph for generating mixed-initiative responses. Experimental results on two ESC datasets show the superiority of KEMI in both content-preserving evaluation and mixed initiative related analyses. △ Less

Submitted 17 May, 2023; originally announced May 2023.

Comments: Accepted by ACL 2023 main conference

Easy-to-Hard Learning for Information Extraction

Authors: Chang Gao, Wenxuan Zhang, Wai Lam, Lidong Bing

Abstract: Information extraction (IE) systems aim to automatically extract structured information, such as named entities, relations between entities, and events, from unstructured texts. While most existing work addresses a particular IE task, universally modeling various IE tasks with one model has achieved great success recently. Despite their success, they employ a one-stage learning strategy, i.e., dir… ▽ More Information extraction (IE) systems aim to automatically extract structured information, such as named entities, relations between entities, and events, from unstructured texts. While most existing work addresses a particular IE task, universally modeling various IE tasks with one model has achieved great success recently. Despite their success, they employ a one-stage learning strategy, i.e., directly learning to extract the target structure given the input text, which contradicts the human learning process. In this paper, we propose a unified easy-to-hard learning framework consisting of three stages, i.e., the easy stage, the hard stage, and the main stage, for IE by mimicking the human learning process. By breaking down the learning process into multiple stages, our framework facilitates the model to acquire general IE task knowledge and improve its generalization ability. Extensive experiments across four IE tasks demonstrate the effectiveness of our framework. We achieve new state-of-the-art results on 13 out of 17 datasets. Our code is available at \url{https://github.com/DAMO-NLP-SG/IE-E2H}. △ Less

Submitted 19 May, 2023; v1 submitted 16 May, 2023; originally announced May 2023.

Comments: Findings of ACL 2023

Progressive Translation: Improving Domain Robustness of Neural Machine Translation with Intermediate Sequences

Authors: Chaojun Wang, Yang Liu, Wai Lam

Abstract: Previous studies show that intermediate supervision signals benefit various Natural Language Processing tasks. However, it is not clear whether there exist intermediate signals that benefit Neural Machine Translation (NMT). Borrowing techniques from Statistical Machine Translation, we propose intermediate signals which are intermediate sequences from the "source-like" structure to the "target-like… ▽ More Previous studies show that intermediate supervision signals benefit various Natural Language Processing tasks. However, it is not clear whether there exist intermediate signals that benefit Neural Machine Translation (NMT). Borrowing techniques from Statistical Machine Translation, we propose intermediate signals which are intermediate sequences from the "source-like" structure to the "target-like" structure. Such intermediate sequences introduce an inductive bias that reflects a domain-agnostic principle of translation, which reduces spurious correlations that are harmful to out-of-domain generalisation. Furthermore, we introduce a full-permutation multi-task learning to alleviate the spurious causal relations from intermediate sequences to the target, which results from exposure bias. The Minimum Bayes Risk decoding algorithm is used to pick the best candidate translation from all permutations to further improve the performance. Experiments show that the introduced intermediate signals can effectively improve the domain robustness of NMT and reduces the amount of hallucinations on out-of-domain translation. Further analysis shows that our methods are especially promising in low-resource scenarios. △ Less

Submitted 16 May, 2023; originally announced May 2023.

Comments: ACL 2023 (Findings)

Chain-of-Dictionary Prompting Elicits Translation in Large Language Models

Authors: Hongyuan Lu, Haoran Yang, Haoyang Huang, Dongdong Zhang, Wai Lam, Furu Wei

Abstract: Large language models (LLMs) have shown surprisingly good performance in multilingual neural machine translation (MNMT) even when trained without parallel data. Yet, despite the fact that the amount of training data is gigantic, they still struggle with translating rare words, particularly for low-resource languages. Even worse, it is usually unrealistic to retrieve relevant demonstrations for in-… ▽ More Large language models (LLMs) have shown surprisingly good performance in multilingual neural machine translation (MNMT) even when trained without parallel data. Yet, despite the fact that the amount of training data is gigantic, they still struggle with translating rare words, particularly for low-resource languages. Even worse, it is usually unrealistic to retrieve relevant demonstrations for in-context learning with low-resource languages on LLMs, which restricts the practical use of LLMs for translation -- how should we mitigate this problem? To this end, we present a novel method, CoD, which augments LLMs with prior knowledge with the chains of multilingual dictionaries for a subset of input words to elicit translation abilities for LLMs. Extensive experiments indicate that augmenting ChatGPT with CoD elicits large gains by up to 13x chrF++ points for MNMT (3.08 to 42.63 for English to Serbian written in Cyrillic script) on FLORES-200 full devtest set. We further demonstrate the importance of chaining the multilingual dictionaries, as well as the superiority of CoD to few-shot demonstration for low-resource languages. △ Less

Submitted 17 August, 2024; v1 submitted 11 May, 2023; originally announced May 2023.

A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects

Authors: Yang Deng, Wenqiang Lei, Wai Lam, Tat-Seng Chua

Abstract: Proactive dialogue systems, related to a wide range of real-world conversational applications, equip the conversational agent with the capability of leading the conversation direction towards achieving pre-defined targets or fulfilling certain goals from the system side. It is empowered by advanced techniques to progress to more complicated tasks that require strategical and motivational interacti… ▽ More Proactive dialogue systems, related to a wide range of real-world conversational applications, equip the conversational agent with the capability of leading the conversation direction towards achieving pre-defined targets or fulfilling certain goals from the system side. It is empowered by advanced techniques to progress to more complicated tasks that require strategical and motivational interactions. In this survey, we provide a comprehensive overview of the prominent problems and advanced designs for conversational agent's proactivity in different types of dialogues. Furthermore, we discuss challenges that meet the real-world application needs but require a greater research focus in the future. We hope that this first survey of proactive dialogue systems can provide the community with a quick access and an overall picture to this practical problem, and stimulate more progresses on conversational AI to the next level. △ Less

Submitted 9 May, 2023; v1 submitted 4 May, 2023; originally announced May 2023.

Comments: Accepted by IJCAI 2023 Survey Track

TOI-733 b: a planet in the small-planet radius valley orbiting a Sun-like star

Authors: Iskra Y. Georgieva, Carina M. Persson, Elisa Goffo, Lorena Acuña, Artyom Aguichine, Luisa M. Serrano, Kristine W. F. Lam, Davide Gandolfi, Karen A. Collins, Steven B. Howell, Fei Dai, Malcolm Fridlund, Judith Korth, Magali Deleuil, Oscar Barragán, William D. Cochran, Szilárd Csizmadia, Hans J. Deeg, Eike Guenther, Artie P. Hatzes, Jon M. Jenkins, John Livingston, Rafael Luque, Olivier Mousis, Hannah L. M. Osborne , et al. (18 additional authors not shown)

Abstract: We report the discovery of a hot ($T_{\rm eq}$ $\approx$ 1055 K) planet in the small planet radius valley transiting the Sun-like star TOI-733, as part of the KESPRINT follow-up program of TESS planets carried out with the HARPS spectrograph. TESS photometry from sectors 9 and 36 yields an orbital period of $P_{\rm orb}$ = $4.884765 _{ - 2.4e-5 } ^ { + 1.9e-5 }$ days and a radius of… ▽ More We report the discovery of a hot ($T_{\rm eq}$ $\approx$ 1055 K) planet in the small planet radius valley transiting the Sun-like star TOI-733, as part of the KESPRINT follow-up program of TESS planets carried out with the HARPS spectrograph. TESS photometry from sectors 9 and 36 yields an orbital period of $P_{\rm orb}$ = $4.884765 _{ - 2.4e-5 } ^ { + 1.9e-5 }$ days and a radius of $R_{\mathrm{p}}$ = $1.992 _{ - 0.090 } ^ { + 0.085 }$ $R_{\oplus}$. Multi-dimensional Gaussian process modelling of the radial velocity measurements from HARPS and activity indicators, gives a semi-amplitude of $K$ = $2.23 \pm 0.26 $ m s$^{-1}$, translating into a planet mass of $M_{\mathrm{p}}$ = $5.72 _{ - 0.68 } ^ { + 0.70 }$ $M_{\oplus}$. These parameters imply that the planet is of moderate density ($ρ_\mathrm{p}$ = $3.98 _{ - 0.66 } ^ { + 0.77 }$ g cm$^{-3}$) and place it in the transition region between rocky and volatile-rich planets with H/He-dominated envelopes on the mass-radius diagram. Combining these with stellar parameters and abundances, we calculate planet interior and atmosphere models, which in turn suggest that TOI-733 b has a volatile-enriched, most likely secondary outer envelope, and may represent a highly irradiated ocean world - one of only a few such planets around G-type stars that are well-characterised. △ Less

Submitted 26 April, 2023; v1 submitted 13 April, 2023; originally announced April 2023.

Comments: Accepted for publication in A&A

Journal ref: A&A 674, A117 (2023)

Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder

Authors: Zihao Fu, Wai Lam, Qian Yu, Anthony Man-Cho So, Shengding Hu, Zhiyuan Liu, Nigel Collier

Abstract: The sequence-to-sequence (seq2seq) task aims at generating the target sequence based on the given input source sequence. Traditionally, most of the seq2seq task is resolved by the Encoder-Decoder framework which requires an encoder to encode the source sequence and a decoder to generate the target text. Recently, a bunch of new approaches have emerged that apply decoder-only language models direct… ▽ More The sequence-to-sequence (seq2seq) task aims at generating the target sequence based on the given input source sequence. Traditionally, most of the seq2seq task is resolved by the Encoder-Decoder framework which requires an encoder to encode the source sequence and a decoder to generate the target text. Recently, a bunch of new approaches have emerged that apply decoder-only language models directly to the seq2seq task. Despite the significant advancements in applying language models to the seq2seq task, there is still a lack of thorough analysis on the effectiveness of the decoder-only language model architecture. This paper aims to address this gap by conducting a detailed comparison between the encoder-decoder architecture and the decoder-only language model framework through the analysis of a regularized encoder-decoder structure. This structure is designed to replicate all behaviors in the classical decoder-only language model but has an encoder and a decoder making it easier to be compared with the classical encoder-decoder structure. Based on the analysis, we unveil the attention degeneration problem in the language model, namely, as the generation step number grows, less and less attention is focused on the source sequence. To give a quantitative understanding of this problem, we conduct a theoretical sensitivity analysis of the attention output with respect to the source input. Grounded on our analysis, we propose a novel partial attention language model to solve the attention degeneration problem. Experimental results on machine translation, summarization, and data-to-text generation tasks support our analysis and demonstrate the effectiveness of our proposed model. △ Less

Submitted 8 April, 2023; originally announced April 2023.

Causal Razors

Authors: Wai-yin Lam

Abstract: When performing causal discovery, assumptions have to be made on how the true causal mechanism corresponds to the underlying joint probability distribution. These assumptions are labeled as causal razors in this work. We review numerous causal razors that appeared in the literature, and offer a comprehensive logical comparison of them. In particular, we scrutinize an unpopular causal razor, namely… ▽ More When performing causal discovery, assumptions have to be made on how the true causal mechanism corresponds to the underlying joint probability distribution. These assumptions are labeled as causal razors in this work. We review numerous causal razors that appeared in the literature, and offer a comprehensive logical comparison of them. In particular, we scrutinize an unpopular causal razor, namely parameter minimality, in multinomial causal models and its logical relations with other well-studied causal razors. Our logical result poses a dilemma in selecting a reasonable scoring criterion for score-based casual search algorithms. △ Less

Submitted 7 August, 2023; v1 submitted 20 February, 2023; originally announced February 2023.

Comments: 29 pages for the main paper. 14 pages for the supplementary materials

Product Question Answering in E-Commerce: A Survey

Authors: Yang Deng, Wenxuan Zhang, Qian Yu, Wai Lam

Abstract: Product question answering (PQA), aiming to automatically provide instant responses to customer's questions in E-Commerce platforms, has drawn increasing attention in recent years. Compared with typical QA problems, PQA exhibits unique challenges such as the subjectivity and reliability of user-generated contents in E-commerce platforms. Therefore, various problem settings and novel methods have b… ▽ More Product question answering (PQA), aiming to automatically provide instant responses to customer's questions in E-Commerce platforms, has drawn increasing attention in recent years. Compared with typical QA problems, PQA exhibits unique challenges such as the subjectivity and reliability of user-generated contents in E-commerce platforms. Therefore, various problem settings and novel methods have been proposed to capture these special characteristics. In this paper, we aim to systematically review existing research efforts on PQA. Specifically, we categorize PQA studies into four problem settings in terms of the form of provided answers. We analyze the pros and cons, as well as present existing datasets and evaluation protocols for each setting. We further summarize the most significant challenges that characterize PQA from general QA applications and discuss their corresponding solutions. Finally, we conclude this paper by providing the prospect on several future directions. △ Less

Submitted 3 May, 2023; v1 submitted 16 February, 2023; originally announced February 2023.

Comments: Accepted by ACL 2023 main conference

Universality of Superconcentration in the Sherrington-Kirkpatrick Model

Authors: Wei-Kuo Chen, Wai-Kit Lam

Abstract: We study the universality of superconcentration for the free energy in the Sherrington-Kirkpatrick (SK) model. In arXiv:0907.3381, Chatterjee showed that when the system consists of $N$ spins and Gaussian disorders, the variance of this quantity is superconcentrated by establishing an upper bound of order $N/\log{N}$, in contrast to the $O(N)$ bound obtained from the Gaussian-Poincaré inequality.… ▽ More We study the universality of superconcentration for the free energy in the Sherrington-Kirkpatrick (SK) model. In arXiv:0907.3381, Chatterjee showed that when the system consists of $N$ spins and Gaussian disorders, the variance of this quantity is superconcentrated by establishing an upper bound of order $N/\log{N}$, in contrast to the $O(N)$ bound obtained from the Gaussian-Poincaré inequality. In this paper, we show that superconcentration indeed holds for any choice of centered disorders with finite third moment, where the upper bound is expressed in terms of an auxiliary nondecreasing function $f$ that arises in the representation of the disorder as $f(g)$ for $g$ standard normal. Under an additional regularity assumption on $f$, we further show that the variance is of order at most $N/\log{N}$. △ Less

Submitted 8 February, 2023; originally announced February 2023.

Comments: 19 pages

ChatGPT and Software Testing Education: Promises & Perils

Authors: Sajed Jalil, Suzzana Rafi, Thomas D. LaToza, Kevin Moran, Wing Lam

Abstract: Over the past decade, predictive language modeling for code has proven to be a valuable tool for enabling new forms of automation for developers. More recently, we have seen the advent of general purpose "large language models", based on neural transformer architectures, that have been trained on massive datasets of human written text spanning code and natural language. However, despite the demons… ▽ More Over the past decade, predictive language modeling for code has proven to be a valuable tool for enabling new forms of automation for developers. More recently, we have seen the advent of general purpose "large language models", based on neural transformer architectures, that have been trained on massive datasets of human written text spanning code and natural language. However, despite the demonstrated representational power of such models, interacting with them has historically been constrained to specific task settings, limiting their general applicability. Many of these limitations were recently overcome with the introduction of ChatGPT, a language model created by OpenAI and trained to operate as a conversational agent, enabling it to answer questions and respond to a wide variety of commands from end users. The introduction of models, such as ChatGPT, has already spurred fervent discussion from educators, ranging from fear that students could use these AI tools to circumvent learning, to excitement about the new types of learning opportunities that they might unlock. However, given the nascent nature of these tools, we currently lack fundamental knowledge related to how well they perform in different educational settings, and the potential promise (or danger) that they might pose to traditional forms of instruction. As such, in this paper, we examine how well ChatGPT performs when tasked with answering common questions in a popular software testing curriculum. Our findings indicate that ChatGPT can provide correct or partially correct answers in 55.6% of cases, provide correct or partially correct explanations of answers in 53.0% of cases, and that prompting the tool in a shared question context leads to a marginally higher rate of correct responses. Based on these findings, we discuss the potential promises and perils related to the use of ChatGPT by students and instructors. △ Less

Submitted 11 March, 2023; v1 submitted 7 February, 2023; originally announced February 2023.

Comments: 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 8 pages, 2 tables, 6 figures

ACM Class: D.2.5

Journal ref: 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

The low density, hot Jupiter TOI-640 b is on a polar orbit

Authors: Emil Knudstrup, Simon H. Albrecht, Davide Gandolfi, Marcus L. Marcussen, Elisa Goffo, Luisa M. Serrano, Fei Dai, Seth Redfield, Teruyuki Hirano, Szilárd Csizmadia, William D. Cochran, Hans J. Deeg, Malcolm Fridlund, Kristine W. F. Lam, John H. Livingston, Rafael Luque, Norio Narita, Enric Palle, Carina M. Persson, Vincent Van Eylen

Abstract: TOI-640 b is a hot, puffy Jupiter with a mass of $0.57 \pm 0.02$ M$_{\rm J}$ and radius of $1.72 \pm 0.05$ R$_{\rm J}$, orbiting a slightly evolved F-type star with a separation of $6.33^{+0.07}_{-0.06}$ R$_\star$. Through spectroscopic in-transit observations made with the HARPS spectrograph, we measured the Rossiter-McLaughlin effect, analysing both in-transit radial velocities and the distortio… ▽ More TOI-640 b is a hot, puffy Jupiter with a mass of $0.57 \pm 0.02$ M$_{\rm J}$ and radius of $1.72 \pm 0.05$ R$_{\rm J}$, orbiting a slightly evolved F-type star with a separation of $6.33^{+0.07}_{-0.06}$ R$_\star$. Through spectroscopic in-transit observations made with the HARPS spectrograph, we measured the Rossiter-McLaughlin effect, analysing both in-transit radial velocities and the distortion of the stellar spectral lines. From these observations, we find the host star to have a projected obliquity of $λ=184\pm3^\circ$. From the TESS light curve, we measured the stellar rotation period, allowing us to determine the stellar inclination, $i_\star=23^{+3\circ}_{-2}$, meaning we are viewing the star pole-on. Combining this with the orbital inclination allowed us to calculate the host star obliquity, $ψ=104\pm2^\circ$. TOI-640 b joins a group of planets orbiting over stellar poles within the range $80^\circ-125^\circ$. The origin of this orbital configuration is not well understood. △ Less

Submitted 3 February, 2023; originally announced February 2023.

Comments: 15 pages, 12 figures, accepted for publication in A&A, in press

Journal ref: A&A 671, A164 (2023)

An Earth-sized Planet around an M5 Dwarf Star at 22 pc

Authors: Teruyuki Hirano, Fei Dai, John H. Livingston, Yui Kasagi, Norio Narita, Hiroyuki Tako Ishikawa, Sascha Grziwa, Kristine W. F. Lam, Kohei Miyakawa, Luisa M. Serrano, Yuji Matsumoto, Eiichiro Kokubo, Tadahiro Kimura, Masahiro Ikoma, Joshua N. Winn, John P. Wisniewski, Hiroki Harakawa, Huan-Yu Teng, William D. Cochran, Akihiko Fukui, Davide Gandolfi, Eike W. Guenther, Yasunori Hori, Kai Ikuta, Kiyoe Kawauchi , et al. (24 additional authors not shown)

Abstract: We report on the discovery of an Earth-sized transiting planet ($R_p=1.015\pm0.051\,R_\oplus$) in a $P=4.02$ day orbit around K2-415 (EPIC 211414619), an M5V star at 22 pc. The planet candidate was first identified by analyzing the light curve data by the K2 mission, and is here shown to exist in the most recent data from TESS. Combining the light curves with the data secured by our follow-up obse… ▽ More We report on the discovery of an Earth-sized transiting planet ($R_p=1.015\pm0.051\,R_\oplus$) in a $P=4.02$ day orbit around K2-415 (EPIC 211414619), an M5V star at 22 pc. The planet candidate was first identified by analyzing the light curve data by the K2 mission, and is here shown to exist in the most recent data from TESS. Combining the light curves with the data secured by our follow-up observations including high-resolution imaging and near infrared spectroscopy with IRD, we rule out false positive scenarios, finding a low false positive probability of $2\times 10^{-4}$. Based on IRD's radial velocities of K2-415, which were sparsely taken over three years, we obtain the planet mass of $3.0\pm 2.7\,M_\oplus$ ($M_p<7.5\,M_\oplus$ at $95\,\%$ confidence) for K2-415b. Being one of the lowest mass stars ($\approx 0.16\,M_\odot$) known to host an Earth-sized transiting planet, K2-415 will be an interesting target for further follow-up observations, including additional radial velocity monitoring and transit spectroscopy. △ Less

Submitted 1 February, 2023; originally announced February 2023.

Comments: 20 pages, 12 figures, accepted for publication in AJ

Advancing Multilingual Pre-training: TRIP Triangular Document-level Pre-training for Multilingual Language Models

Authors: Hongyuan Lu, Haoyang Huang, Shuming Ma, Dongdong Zhang, Wai Lam, Furu Wei

Abstract: Despite the success of multilingual sequence-to-sequence pre-training, most existing approaches rely on document-level monolingual corpora in many different languages, sentence-level bilingual corpora,\footnote{In this paper, we use `bilingual corpora' to denote parallel corpora with `bilingual translation pairs' in many different language pairs, each consisting of two sentences/documents with the… ▽ More Despite the success of multilingual sequence-to-sequence pre-training, most existing approaches rely on document-level monolingual corpora in many different languages, sentence-level bilingual corpora,\footnote{In this paper, we use `bilingual corpora' to denote parallel corpora with `bilingual translation pairs' in many different language pairs, each consisting of two sentences/documents with the same meaning written in different languages. We use `trilingual corpora' to denote parallel corpora with `trilingual translation pairs' in many different language combinations, each consisting of three sentences/documents.} and sometimes synthetic document-level bilingual corpora. This hampers the performance with cross-lingual document-level tasks such as document-level translation. Therefore, we propose to mine and leverage document-level trilingual parallel corpora to improve sequence-to-sequence multilingual pre-training. We present \textbf{Tri}angular Document-level \textbf{P}re-training (\textbf{TRIP}), which is the first in the field to accelerate the conventional monolingual and bilingual objectives into a trilingual objective with a novel method called Grafting. Experiments show that TRIP achieves several strong state-of-the-art (SOTA) scores on three multilingual document-level machine translation benchmarks and one cross-lingual abstractive summarization benchmark, including consistent improvements by up to 3.11 d-BLEU points and 8.9 ROUGE-L points. △ Less

Submitted 13 May, 2023; v1 submitted 15 December, 2022; originally announced December 2022.

From Cloze to Comprehension: Retrofitting Pre-trained Masked Language Model to Pre-trained Machine Reader

Authors: Weiwen Xu, Xin Li, Wenxuan Zhang, Meng Zhou, Wai Lam, Luo Si, Lidong Bing

Abstract: We present Pre-trained Machine Reader (PMR), a novel method for retrofitting pre-trained masked language models (MLMs) to pre-trained machine reading comprehension (MRC) models without acquiring labeled data. PMR can resolve the discrepancy between model pre-training and downstream fine-tuning of existing MLMs. To build the proposed PMR, we constructed a large volume of general-purpose and high-qu… ▽ More We present Pre-trained Machine Reader (PMR), a novel method for retrofitting pre-trained masked language models (MLMs) to pre-trained machine reading comprehension (MRC) models without acquiring labeled data. PMR can resolve the discrepancy between model pre-training and downstream fine-tuning of existing MLMs. To build the proposed PMR, we constructed a large volume of general-purpose and high-quality MRC-style training data by using Wikipedia hyperlinks and designed a Wiki Anchor Extraction task to guide the MRC-style pre-training. Apart from its simplicity, PMR effectively solves extraction tasks, such as Extractive Question Answering and Named Entity Recognition. PMR shows tremendous improvements over existing approaches, especially in low-resource scenarios. When applied to the sequence classification task in the MRC formulation, PMR enables the extraction of high-quality rationales to explain the classification process, thereby providing greater prediction explainability. PMR also has the potential to serve as a unified model for tackling various extraction and classification tasks in the MRC formulation. △ Less

Submitted 16 October, 2023; v1 submitted 9 December, 2022; originally announced December 2022.

Comments: Accepted to NeurIPS 2023

Discovery of TOI-1260d and the characterisation of the multi-planet system

Authors: Kristine W. F. Lam, J. Cabrera, M. J. Hooton, Y. Alibert, A. Bonfanti, M. Beck, A. Deline, H. -G. Florén, A. E. Simon, L. Fossati, C. M. Persson, M. Fridlund, S. Salmon, S. Hoyer, H. P. Osborn, T . G. Wilson, I. Y. Georgieva, Gr. Nowak, R. Luque, J. A. Egger, V. Adibekyan R. Alonso, G. Anglada Escudé, T. Bárczy, D. Barrado, S. C. C. Barros , et al. (61 additional authors not shown)

Abstract: We report the discovery of a third planet transiting the star TOI-1260, previously known to host two transiting sub-Neptune planets with orbital periods of 3.127 and 7.493 days, respectively. The nature of the third transiting planet with a 16.6-day orbit is supported by ground-based follow-up observations, including time-series photometry, high-angular resolution images, spectroscopy, and archiva… ▽ More We report the discovery of a third planet transiting the star TOI-1260, previously known to host two transiting sub-Neptune planets with orbital periods of 3.127 and 7.493 days, respectively. The nature of the third transiting planet with a 16.6-day orbit is supported by ground-based follow-up observations, including time-series photometry, high-angular resolution images, spectroscopy, and archival imagery. Precise photometric monitoring with CHEOPS allows to improve the constraints on the parameters of the system, improving our knowledge on their composition. The improved radii of TOI-1260b, TOI-1260c are $2.36 \pm 0.06 \rm R_{\oplus}$, $2.82 \pm 0.08 \rm R_{\oplus}$, respectively while the newly discovered third planet has a radius of $3.09 \pm 0.09 \rm R_{\oplus}$. The radius uncertainties are in the range of 3\%, allowing a precise interpretation of the interior structure of the three planets. Our planet interior composition model suggests that all three planets in the TOI-1260 system contains some fraction of gas. The innermost planet TOI-1260b has most likely lost all of its primordial hydrogen-dominated envelope. Planets c and d were also likely to have experienced significant loss of atmospheric through escape, but to a lesser extent compared to planet b. △ Less

Submitted 8 December, 2022; originally announced December 2022.

Comments: 18 pages, 10 figures, accepted for publication in MNRAS

Radial velocity confirmation of a hot super-Neptune discovered by TESS with a warm Saturn-mass companion

Authors: E. Knudstrup, D. Gandolfi, G. Nowak, C. M. Persson, E. Furlan, J. Livingston, E. Matthews, M. S. Lundkvist, M. L. Winther, J. L. Rørsted, S. H. Albrecht, E. Goffo, I. Carleo, H. J. Deeg, K. A. Collins, N. Narita, H. Isaacson, S. Redfield, F. Dai, T. Hirano, J. M. Akana Murphy, C. Beard, L. A. Buchhave, S. Cary, A. Chontos , et al. (37 additional authors not shown)

Abstract: We report the discovery and confirmation of the planetary system TOI-1288. This late G dwarf harbours two planets: TOI-1288 b and TOI-1288 c. We combine TESS space-borne and ground-based transit photometry with HARPS-N and HIRES high-precision Doppler measurements, which we use to constrain the masses of both planets in the system and the radius of planet b. TOI-1288~b has a period of… ▽ More We report the discovery and confirmation of the planetary system TOI-1288. This late G dwarf harbours two planets: TOI-1288 b and TOI-1288 c. We combine TESS space-borne and ground-based transit photometry with HARPS-N and HIRES high-precision Doppler measurements, which we use to constrain the masses of both planets in the system and the radius of planet b. TOI-1288~b has a period of $2.699835^{+0.000004}_{-0.000003}$ d, a radius of $5.24 \pm 0.09$ R$_\oplus$, and a mass of $42 \pm 3$ M$_\oplus$, making this planet a hot transiting super-Neptune situated right in the Neptunian desert. This desert refers to a paucity of Neptune-sized planets on short period orbits. Our 2.4-year-long Doppler monitoring of TOI-1288 revealed the presence of a Saturn-mass planet on a moderately eccentric orbit ($0.13^{+0.07}_{-0.09}$) with a minimum mass of $84 \pm 7$ M$_\oplus$ and a period of $443^{+11}_{-13}$ d. The 5 sectors worth of TESS data do not cover our expected mid-transit time for TOI-1288 c, and we do not detect a transit for this planet in these sectors. △ Less

Submitted 30 November, 2022; originally announced November 2022.

Comments: 16 pages, 17 figures, under review MNRAS

On the Effectiveness of Parameter-Efficient Fine-Tuning

Authors: Zihao Fu, Haoran Yang, Anthony Man-Cho So, Wai Lam, Lidong Bing, Nigel Collier

Abstract: Fine-tuning pre-trained models has been ubiquitously proven to be effective in a wide range of NLP tasks. However, fine-tuning the whole model is parameter inefficient as it always yields an entirely new model for each task. Currently, many research works propose to only fine-tune a small portion of the parameters while keeping most of the parameters shared across different tasks. These methods ac… ▽ More Fine-tuning pre-trained models has been ubiquitously proven to be effective in a wide range of NLP tasks. However, fine-tuning the whole model is parameter inefficient as it always yields an entirely new model for each task. Currently, many research works propose to only fine-tune a small portion of the parameters while keeping most of the parameters shared across different tasks. These methods achieve surprisingly good performance and are shown to be more stable than their corresponding fully fine-tuned counterparts. However, such kind of methods is still not well understood. Some natural questions arise: How does the parameter sparsity lead to promising performance? Why is the model more stable than the fully fine-tuned models? How to choose the tunable parameters? In this paper, we first categorize the existing methods into random approaches, rule-based approaches, and projection-based approaches based on how they choose which parameters to tune. Then, we show that all of the methods are actually sparse fine-tuned models and conduct a novel theoretical analysis of them. We indicate that the sparsity is actually imposing a regularization on the original model by controlling the upper bound of the stability. Such stability leads to better generalization capability which has been empirically observed in a lot of recent research works. Despite the effectiveness of sparsity grounded by our theory, it still remains an open problem of how to choose the tunable parameters. To better choose the tunable parameters, we propose a novel Second-order Approximation Method (SAM) which approximates the original problem with an analytically solvable optimization function. The tunable parameters are determined by directly optimizing the approximation function. The experimental results show that our proposed SAM model outperforms many strong baseline models and it also verifies our theoretical analysis. △ Less

Submitted 28 November, 2022; originally announced November 2022.

HD 20329b: An ultra-short-period planet around a solar-type star found by TESS

Authors: F. Murgas, G. Nowak, T. Masseron, H. Parviainen, R. Luque, E. Pallé, Judith Korth, I. Carleo, Sz. Csizmadia, E. Esparza-Borges, Ahlam Alqasim, William D. Cochran, Fei Dai, Hans J. Deeg, D. Gandolfi, Elisa Goffo, Petr Kabáth, K. W. F. Lam, John Livingston, Alexandra Muresan, H. L. M. Osborne, Carina M. Persson, L. M. Serrano, Alexis M. S. Smith, Vincent Van Eylen , et al. (23 additional authors not shown)

Abstract: We used TESS light curves and HARPS-N spectrograph radial velocity measurements to establish the physical properties of the transiting exoplanet candidate found around the star HD 20329 (TOI-4524). We performed a joint fit of the light curves and radial velocity time series to measure the mass, radius, and orbital parameters of the candidate. We confirm and characterize HD 20329b, an ultra-short-p… ▽ More We used TESS light curves and HARPS-N spectrograph radial velocity measurements to establish the physical properties of the transiting exoplanet candidate found around the star HD 20329 (TOI-4524). We performed a joint fit of the light curves and radial velocity time series to measure the mass, radius, and orbital parameters of the candidate. We confirm and characterize HD 20329b, an ultra-short-period (USP) planet transiting a solar-type star. The host star (HD 20329, $V = 8.74$ mag, $J = 7.5$ mag) is characterized by its G5 spectral type with $\mathrm{M}_\star= 0.90 \pm 0.05$ M$_\odot$, $\mathrm{R}_\star = 1.13 \pm 0.02$ R$_\odot$, and $\mathrm{T}_{\mathrm{eff}} = 5596 \pm 50$ K; it is located at a distance $d= 63.68 \pm 0.29$ pc. By jointly fitting the available TESS transit light curves and follow-up radial velocity measurements, we find an orbital period of $0.9261 \pm (0.5\times 10^{-4})$ days, a planetary radius of $1.72 \pm 0.07$ $\mathrm{R}_\oplus$, and a mass of $7.42 \pm 1.09$ $\mathrm{M}_\oplus$, implying a mean density of $ρ_\mathrm{p} = 8.06 \pm 1.53$ g cm$^{-3}$. HD 20329b joins the $\sim$30 currently known USP planets with radius and Doppler mass measurements. △ Less

Submitted 7 November, 2022; v1 submitted 4 November, 2022; originally announced November 2022.

Comments: Accepted for publication in A&A, 26 pages

Journal ref: A&A 668, A158 (2022)

HD 191939 revisited: New and refined planet mass determinations, and a new planet in the habitable zone

Authors: J. Orell-Miquel, G. Nowak, F. Murgas, E. Palle, G. Morello, R. Luque, M. Badenas-Agusti, I. Ribas, M. Lafarga, N. Espinoza, J. C. Morales, M. Zechmeister, A. Alqasim, W. D. Cochran, D. Gandolfi, E. Goffo, P. Kabáth, J. Korth, J. Livingston, K. W. F. Lam, A. Muresan, C. M. Persson, V. Van Eylen

Abstract: HD 191939 (TOI-1339) is a nearby (d=54pc), bright (V=9mag), and inactive Sun-like star (G9 V) known to host a multi-planet transiting system. Ground-based spectroscopic observations confirmed the planetary nature of the three transiting sub-Neptunes (HD 191939 b, c, and d) originally detected by TESS and were used to measure the masses for planets b and c with 3$σ$ precision. These previous observ… ▽ More HD 191939 (TOI-1339) is a nearby (d=54pc), bright (V=9mag), and inactive Sun-like star (G9 V) known to host a multi-planet transiting system. Ground-based spectroscopic observations confirmed the planetary nature of the three transiting sub-Neptunes (HD 191939 b, c, and d) originally detected by TESS and were used to measure the masses for planets b and c with 3$σ$ precision. These previous observations also reported the discovery of an additional Saturn-mass planet (HD 191939 e) and evidence for a further, very long-period companion (HD 191939 f). Here, we report the discovery of a new non-transiting planet in the system and a refined mass determination of HD 191939 d. The new planet, HD 191939 g, has a minimum mass of 13.5$\pm$2.0 M$_\oplus$ and a period of about 280 d. This period places the planet within the conservative habitable zone of the host star, and near a 1:3 resonance with HD 191939 e. The compilation of 362 radial velocity measurements with a baseline of 677 days from four different high-resolution spectrographs also allowed us to refine the properties of the previously known planets, including a 4.6$σ$ mass determination for planet d, for which only a 2$σ$ upper limit had been set until now. We confirm the previously suspected low density of HD 191939 d, which makes it an attractive target for attempting atmospheric characterisation. Overall, the planetary system consists of three sub-Neptunes interior to a Saturn-mass and a Uranus-mass planet plus a high-mass long-period companion. This particular configuration has no counterpart in the literature and makes HD 191939 an exceptional multi-planet transiting system with an unusual planet demographic worthy of future observation. △ Less

Submitted 1 November, 2022; originally announced November 2022.

Comments: Accepted for publication in A&A. 20 pages, 8 figures

Journal ref: A&A 669, A40 (2023)

McQueen: a Benchmark for Multimodal Conversational Query Rewrite

Authors: Yifei Yuan, Chen Shi, Runze Wang, Liyi Chen, Feijun Jiang, Yuan You, Wai Lam

Abstract: The task of query rewrite aims to convert an in-context query to its fully-specified version where ellipsis and coreference are completed and referred-back according to the history context. Although much progress has been made, less efforts have been paid to real scenario conversations that involve drawing information from more than one modalities. In this paper, we propose the task of multimodal… ▽ More The task of query rewrite aims to convert an in-context query to its fully-specified version where ellipsis and coreference are completed and referred-back according to the history context. Although much progress has been made, less efforts have been paid to real scenario conversations that involve drawing information from more than one modalities. In this paper, we propose the task of multimodal conversational query rewrite (McQR), which performs query rewrite under the multimodal visual conversation setting. We collect a large-scale dataset named McQueen based on manual annotation, which contains 15k visual conversations and over 80k queries where each one is associated with a fully-specified rewrite version. In addition, for entities appearing in the rewrite, we provide the corresponding image box annotation. We then use the McQueen dataset to benchmark a state-of-the-art method for effectively tackling the McQR task, which is based on a multimodal pre-trained model with pointer generator. Extensive experiments are performed to demonstrate the effectiveness of our model on this task\footnote{The dataset and code of this paper are both available in \url{https://github.com/yfyuan01/MQR} △ Less

Submitted 23 October, 2022; originally announced October 2022.

Comments: Accepted by EMNLP22

Towards Generalizable and Robust Text-to-SQL Parsing

Authors: Chang Gao, Bowen Li, Wenxuan Zhang, Wai Lam, Binhua Li, Fei Huang, Luo Si, Yongbin Li

Abstract: Text-to-SQL parsing tackles the problem of mapping natural language questions to executable SQL queries. In practice, text-to-SQL parsers often encounter various challenging scenarios, requiring them to be generalizable and robust. While most existing work addresses a particular generalization or robustness challenge, we aim to study it in a more comprehensive manner. In specific, we believe that… ▽ More Text-to-SQL parsing tackles the problem of mapping natural language questions to executable SQL queries. In practice, text-to-SQL parsers often encounter various challenging scenarios, requiring them to be generalizable and robust. While most existing work addresses a particular generalization or robustness challenge, we aim to study it in a more comprehensive manner. In specific, we believe that text-to-SQL parsers should be (1) generalizable at three levels of generalization, namely i.i.d., zero-shot, and compositional, and (2) robust against input perturbations. To enhance these capabilities of the parser, we propose a novel TKK framework consisting of Task decomposition, Knowledge acquisition, and Knowledge composition to learn text-to-SQL parsing in stages. By dividing the learning process into multiple stages, our framework improves the parser's ability to acquire general SQL knowledge instead of capturing spurious patterns, making it more generalizable and robust. Experimental results under various generalization and robustness settings show that our framework is effective in all scenarios and achieves state-of-the-art performance on the Spider, SParC, and CoSQL datasets. Code can be found at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/tkk. △ Less

Submitted 23 October, 2022; originally announced October 2022.

Comments: Findings of EMNLP 2022

Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation

Authors: Deng Cai, Xin Li, Jackie Chun-Sing Ho, Lidong Bing, Wai Lam

Abstract: We introduce a new method to improve existing multilingual sentence embeddings with Abstract Meaning Representation (AMR). Compared with the original textual input, AMR is a structured semantic representation that presents the core concepts and relations in a sentence explicitly and unambiguously. It also helps reduce surface variations across different expressions and languages. Unlike most prior… ▽ More We introduce a new method to improve existing multilingual sentence embeddings with Abstract Meaning Representation (AMR). Compared with the original textual input, AMR is a structured semantic representation that presents the core concepts and relations in a sentence explicitly and unambiguously. It also helps reduce surface variations across different expressions and languages. Unlike most prior work that only evaluates the ability to measure semantic similarity, we present a thorough evaluation of existing multilingual sentence embeddings and our improved versions, which include a collection of five transfer tasks in different downstream applications. Experiment results show that retrofitting multilingual sentence embeddings with AMR leads to better state-of-the-art performance on both semantic textual similarity and transfer tasks. Our codebase and evaluation scripts can be found at \url{https://github.com/jcyk/MSE-AMR}. △ Less

Submitted 18 October, 2022; originally announced October 2022.

Comments: EMNLP2022

PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Tasks

Authors: Weiwen Xu, Xin Li, Yang Deng, Wai Lam, Lidong Bing

Abstract: Span identification aims at identifying specific text spans from text input and classifying them into pre-defined categories. Different from previous works that merely leverage the Subordinate (SUB) relation (i.e. if a span is an instance of a certain category) to train models, this paper for the first time explores the Peer (PR) relation, which indicates that two spans are instances of the same c… ▽ More Span identification aims at identifying specific text spans from text input and classifying them into pre-defined categories. Different from previous works that merely leverage the Subordinate (SUB) relation (i.e. if a span is an instance of a certain category) to train models, this paper for the first time explores the Peer (PR) relation, which indicates that two spans are instances of the same category and share similar features. Specifically, a novel Peer Data Augmentation (PeerDA) approach is proposed which employs span pairs with the PR relation as the augmentation data for training. PeerDA has two unique advantages: (1) There are a large number of PR span pairs for augmenting the training data. (2) The augmented data can prevent the trained model from over-fitting the superficial span-category mapping by pushing the model to leverage the span semantics. Experimental results on ten datasets over four diverse tasks across seven domains demonstrate the effectiveness of PeerDA. Notably, PeerDA achieves state-of-the-art results on six of them. △ Less

Submitted 18 May, 2023; v1 submitted 17 October, 2022; originally announced October 2022.

Comments: To appear at ACL 2023 main conference

PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance

Authors: Yang Deng, Wenqiang Lei, Wenxuan Zhang, Wai Lam, Tat-Seng Chua

Abstract: To facilitate conversational question answering (CQA) over hybrid contexts in finance, we present a new dataset, named PACIFIC. Compared with existing CQA datasets, PACIFIC exhibits three key features: (i) proactivity, (ii) numerical reasoning, and (iii) hybrid context of tables and text. A new task is defined accordingly to study Proactive Conversational Question Answering (PCQA), which combines… ▽ More To facilitate conversational question answering (CQA) over hybrid contexts in finance, we present a new dataset, named PACIFIC. Compared with existing CQA datasets, PACIFIC exhibits three key features: (i) proactivity, (ii) numerical reasoning, and (iii) hybrid context of tables and text. A new task is defined accordingly to study Proactive Conversational Question Answering (PCQA), which combines clarification question generation and CQA. In addition, we propose a novel method, namely UniPCQA, to adapt a hybrid format of input and output content in PCQA into the Seq2Seq problem, including the reformulation of the numerical reasoning process as code generation. UniPCQA performs multi-task learning over all sub-tasks in PCQA and incorporates a simple ensemble strategy to alleviate the error propagation issue in the multi-task learning by cross-validating top-$k$ sampled Seq2Seq outputs. We benchmark the PACIFIC dataset with extensive baselines and provide comprehensive evaluations on each sub-task of PCQA. △ Less

Submitted 18 March, 2023; v1 submitted 17 October, 2022; originally announced October 2022.

Comments: Accepted by EMNLP 2022 (main conference)

ConReader: Exploring Implicit Relations in Contracts for Contract Clause Extraction

Authors: Weiwen Xu, Yang Deng, Wenqiang Lei, Wenlong Zhao, Tat-Seng Chua, Wai Lam

Abstract: We study automatic Contract Clause Extraction (CCE) by modeling implicit relations in legal contracts. Existing CCE methods mostly treat contracts as plain text, creating a substantial barrier to understanding contracts of high complexity. In this work, we first comprehensively analyze the complexity issues of contracts and distill out three implicit relations commonly found in contracts, namely,… ▽ More We study automatic Contract Clause Extraction (CCE) by modeling implicit relations in legal contracts. Existing CCE methods mostly treat contracts as plain text, creating a substantial barrier to understanding contracts of high complexity. In this work, we first comprehensively analyze the complexity issues of contracts and distill out three implicit relations commonly found in contracts, namely, 1) Long-range Context Relation that captures the correlations of distant clauses; 2) Term-Definition Relation that captures the relation between important terms with their corresponding definitions; and 3) Similar Clause Relation that captures the similarities between clauses of the same type. Then we propose a novel framework ConReader to exploit the above three relations for better contract understanding and improving CCE. Experimental results show that ConReader makes the prediction more interpretable and achieves new state-of-the-art on two CCE tasks in both conventional and zero-shot settings. △ Less

Submitted 16 October, 2022; originally announced October 2022.

Comments: To appear at EMNLP 2022 main conference

Revamping Multilingual Agreement Bidirectionally via Switched Back-translation for Multilingual Neural Machine Translation

Authors: Hongyuan Lu, Haoyang Huang, Shuming Ma, Dongdong Zhang, Furu Wei, Wai Lam

Abstract: Despite the fact that multilingual agreement (MA) has shown its importance for multilingual neural machine translation (MNMT), current methodologies in the field have two shortages: (i) require parallel data between multiple language pairs, which is not always realistic and (ii) optimize the agreement in an ambiguous direction, which hampers the translation performance. We present \textbf{B}idirec… ▽ More Despite the fact that multilingual agreement (MA) has shown its importance for multilingual neural machine translation (MNMT), current methodologies in the field have two shortages: (i) require parallel data between multiple language pairs, which is not always realistic and (ii) optimize the agreement in an ambiguous direction, which hampers the translation performance. We present \textbf{B}idirectional \textbf{M}ultilingual \textbf{A}greement via \textbf{S}witched \textbf{B}ack-\textbf{t}ranslation (\textbf{BMA-SBT}), a novel and universal multilingual agreement framework for fine-tuning pre-trained MNMT models, which (i) exempts the need for aforementioned parallel data by using a novel method called switched BT that creates synthetic text written in another source language using the translation target and (ii) optimizes the agreement bidirectionally with the Kullback-Leibler Divergence loss. Experiments indicate that BMA-SBT clearly improves the strong baselines on the task of MNMT with three benchmarks: TED Talks, News, and Europarl. In-depth analyzes indicate that BMA-SBT brings additive improvements to the conventional BT method. △ Less

Submitted 15 May, 2023; v1 submitted 28 September, 2022; originally announced September 2022.

Regularization of the Hill four-body problem with oblate bodies

Authors: Edward Belbruno, Marian Gidea, Wai-Ting Lam

Abstract: We consider the Hill four-body problem where three oblate, massive bodies form a relative equilibrium triangular configuration, and the fourth, infinitesimal body orbits in a neighborhood of the smallest of the three massive bodies. We regularize collisions between the infinitesimal body and the smallest massive body, via McGehee coordinate transformation. We describe the corresponding collision m… ▽ More We consider the Hill four-body problem where three oblate, massive bodies form a relative equilibrium triangular configuration, and the fourth, infinitesimal body orbits in a neighborhood of the smallest of the three massive bodies. We regularize collisions between the infinitesimal body and the smallest massive body, via McGehee coordinate transformation. We describe the corresponding collision manifold and show that it undergoes a bifurcation when the oblateness coefficient of the small massive body passes through the zero value. △ Less

Submitted 4 February, 2023; v1 submitted 27 September, 2022; originally announced September 2022.

Informative Text Generation from Knowledge Triples

Authors: Zihao Fu, Yijiang River Dong, Lidong Bing, Wai Lam

Abstract: As the development of the encoder-decoder architecture, researchers are able to study the text generation tasks with broader types of data. Among them, KB-to-text aims at converting a set of knowledge triples into human readable sentences. In the original setting, the task assumes that the input triples and the text are exactly aligned in the perspective of the embodied knowledge/information. In t… ▽ More As the development of the encoder-decoder architecture, researchers are able to study the text generation tasks with broader types of data. Among them, KB-to-text aims at converting a set of knowledge triples into human readable sentences. In the original setting, the task assumes that the input triples and the text are exactly aligned in the perspective of the embodied knowledge/information. In this paper, we extend this setting and explore how to facilitate the trained model to generate more informative text, namely, containing more information about the triple entities but not conveyed by the input triples. To solve this problem, we propose a novel memory augmented generator that employs a memory network to memorize the useful knowledge learned during the training and utilizes such information together with the input triples to generate text in the operational or testing phase. We derive a dataset from WebNLG for our new setting and conduct extensive experiments to investigate the effectiveness of our model as well as uncover the intrinsic characteristics of the setting. △ Less

Submitted 26 September, 2022; originally announced September 2022.

Detecting Political Biases of Named Entities and Hashtags on Twitter

Authors: Zhiping Xiao, Jeffrey Zhu, Yining Wang, Pei Zhou, Wen Hong Lam, Mason A. Porter, Yizhou Sun

Abstract: Ideological divisions in the United States have become increasingly prominent in daily communication. Accordingly, there has been much research on political polarization, including many recent efforts that take a computational perspective. By detecting political biases in a corpus of text, one can attempt to describe and discern the polarity of that text. Intuitively, the named entities (i.e., the… ▽ More Ideological divisions in the United States have become increasingly prominent in daily communication. Accordingly, there has been much research on political polarization, including many recent efforts that take a computational perspective. By detecting political biases in a corpus of text, one can attempt to describe and discern the polarity of that text. Intuitively, the named entities (i.e., the nouns and the phrases that act as nouns) and hashtags in text often carry information about political views. For example, people who use the term "pro-choice" are likely to be liberal, whereas people who use the term "pro-life" are likely to be conservative. In this paper, we seek to reveal political polarities in social-media text data and to quantify these polarities by explicitly assigning a polarity score to entities and hashtags. Although this idea is straightforward, it is difficult to perform such inference in a trustworthy quantitative way. Key challenges include the small number of known labels, the continuous spectrum of political views, and the preservation of both a polarity score and a polarity-neutral semantic meaning in an embedding vector of words. To attempt to overcome these challenges, we propose the Polarity-aware Embedding Multi-task learning (PEM) model. This model consists of (1) a self-supervised context-preservation task, (2) an attention-based tweet-level polarity-inference task, and (3) an adversarial learning task that promotes independence between an embedding's polarity dimension and its semantic dimensions. Our experimental results demonstrate that our PEM model can successfully learn polarity-aware embeddings that perform well classification tasks. We examine a variety of applications and we thereby demonstrate the effectiveness of our PEM model. We also discuss important limitations of our work and encourage caution when applying the it to real-world scenarios. △ Less

Submitted 17 March, 2023; v1 submitted 16 September, 2022; originally announced September 2022.

Comments: Submitted to EPJ -- Data Science, under review

MSC Class: 68T09 (Primary) 68T07 (Secondary)

Data efficient reinforcement learning and adaptive optimal perimeter control of network traffic dynamics

Authors: C. Chen, Y. P. Huang, W. H. K. Lam, T. L. Pan, S. C. Hsu, A. Sumalee, R. X. Zhong

Abstract: Existing data-driven and feedback traffic control strategies do not consider the heterogeneity of real-time data measurements. Besides, traditional reinforcement learning (RL) methods for traffic control usually converge slowly for lacking data efficiency. Moreover, conventional optimal perimeter control schemes require exact knowledge of the system dynamics and thus would be fragile to endogenous… ▽ More Existing data-driven and feedback traffic control strategies do not consider the heterogeneity of real-time data measurements. Besides, traditional reinforcement learning (RL) methods for traffic control usually converge slowly for lacking data efficiency. Moreover, conventional optimal perimeter control schemes require exact knowledge of the system dynamics and thus would be fragile to endogenous uncertainties. To handle these challenges, this work proposes an integral reinforcement learning (IRL) based approach to learning the macroscopic traffic dynamics for adaptive optimal perimeter control. This work makes the following primary contributions to the transportation literature: (a) A continuous-time control is developed with discrete gain updates to adapt to the discrete-time sensor data. (b) To reduce the sampling complexity and use the available data more efficiently, the experience replay (ER) technique is introduced to the IRL algorithm. (c) The proposed method relaxes the requirement on model calibration in a "model-free" manner that enables robustness against modeling uncertainty and enhances the real-time performance via a data-driven RL algorithm. (d) The convergence of the IRL-based algorithms and the stability of the controlled traffic dynamics are proven via the Lyapunov theory. The optimal control law is parameterized and then approximated by neural networks (NN), which moderates the computational complexity. Both state and input constraints are considered while no model linearization is required. Numerical examples and simulation experiments are presented to verify the effectiveness and efficiency of the proposed method. △ Less

Submitted 13 September, 2022; originally announced September 2022.

PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentation

Authors: Hongyuan Lu, Wai Lam

Abstract: Curriculum Data Augmentation (CDA) improves neural models by presenting synthetic data with increasing difficulties from easy to hard. However, traditional CDA simply treats the ratio of word perturbation as the difficulty measure and goes through the curriculums only once. This paper presents \textbf{PCC}: \textbf{P}araphrasing with Bottom-k Sampling and \textbf{C}yclic Learning for \textbf{C}urr… ▽ More Curriculum Data Augmentation (CDA) improves neural models by presenting synthetic data with increasing difficulties from easy to hard. However, traditional CDA simply treats the ratio of word perturbation as the difficulty measure and goes through the curriculums only once. This paper presents \textbf{PCC}: \textbf{P}araphrasing with Bottom-k Sampling and \textbf{C}yclic Learning for \textbf{C}urriculum Data Augmentation, a novel CDA framework via paraphrasing, which exploits the textual paraphrase similarity as the curriculum difficulty measure. We propose a curriculum-aware paraphrase generation module composed of three units: a paraphrase candidate generator with bottom-k sampling, a filtering mechanism and a difficulty measure. We also propose a cyclic learning strategy that passes through the curriculums multiple times. The bottom-k sampling is proposed to generate super-hard instances for the later curriculums. Experimental results on few-shot text classification as well as dialogue generation indicate that PCC surpasses competitive baselines. Human evaluation and extensive case studies indicate that bottom-k sampling effectively generates super-hard instances, and PCC significantly improves the baseline dialogue agent. △ Less

Submitted 30 January, 2023; v1 submitted 17 August, 2022; originally announced August 2022.

Comments: Accepted to EACL 2023 (main)

Precise mass determination for the keystone sub-Neptune planet transiting the mid-type M dwarf G 9-40

Authors: R. Luque, G. Nowak, T. Hirano, D. Kossakowski, E. Pallé, M. C. Nixon, G. Morello, P. J. Amado, S. H. Albrecht, J. A. Caballero, C. Cifuentes, W. D. Cochran, H. J. Deeg, S. Dreizler, E. Esparza-Borges, A. Fukui, D. Gandolfi, E. Goffo, E. W. Guenther, A. P. Hatzes, T. Henning, P. Kabath, K. Kawauchi, J. Korth, T. Kotani , et al. (23 additional authors not shown)

Abstract: Context. Despite being a prominent subset of the exoplanet population discovered in the past three decades, the nature and provenance of sub-Neptune-sized planets are still one of the open questions in exoplanet science. Aims. For planets orbiting bright stars, precisely measuring the orbital and planet parameters of the system is the best approach to distinguish between competing theories regardi… ▽ More Context. Despite being a prominent subset of the exoplanet population discovered in the past three decades, the nature and provenance of sub-Neptune-sized planets are still one of the open questions in exoplanet science. Aims. For planets orbiting bright stars, precisely measuring the orbital and planet parameters of the system is the best approach to distinguish between competing theories regarding their formation and evolution. Methods. We obtained 69 new radial velocity observations of the mid-M dwarf G 9-40 with the CARMENES instrument to measure for the first time the mass of its transiting sub-Neptune planet, G 9-40 b, discovered in data from the K2 mission. Results. Combined with new observations from the TESS mission during Sectors 44, 45, and 46, we are able to measure the radius of the planet to an uncertainty of 3.4% (Rb = 1.900 +- 0.065 Re) and determine its mass with a precision of 16% (Mb = 4.00 +- 0.63 Me). The resulting bulk density of the planet is inconsistent with a terrestrial composition and suggests the presence of either a water-rich core or a significant hydrogen-rich envelope. Conclusions. G 9-40 b is referred to as a keystone planet due to its location in period-radius space within the radius valley. Several theories offer explanations for the origin and properties of this population and this planet is a valuable target for testing the dependence of those models on stellar host mass. By virtue of its brightness and small size of the host, it joins L 98-59 d as one of the two best warm (Teq ~ 400 K) sub-Neptunes for atmospheric characterization with JWST, which will probe cloud formation in sub-Neptune-sized planets and break the degeneracies of internal composition models. △ Less

Submitted 15 August, 2022; originally announced August 2022.

Comments: Accepted for publication in A&A

Journal ref: A&A 666, A154 (2022)