Showing 1–50 of 497 results for author: Roberts, A

Benchmarking the design of the cryogenics system for the underground argon in DarkSide-20k

Authors: DarkSide-20k Collaboration, :, F. Acerbi, P. Adhikari, P. Agnes, I. Ahmad, S. Albergo, I. F. M. Albuquerque, T. Alexander, A. K. Alton, P. Amaudruz, M. Angiolilli, E. Aprile, R. Ardito, M. Atzori Corona, D. J. Auty, M. Ave, I. C. Avetisov, O. Azzolini, H. O. Back, Z. Balmforth, A. Barrado Olmedo, P. Barrillon, G. Batignani, P. Bhowmick , et al. (294 additional authors not shown)

Abstract: DarkSide-20k (DS-20k) is a dark matter detection experiment under construction at the Laboratori Nazionali del Gran Sasso (LNGS) in Italy. It utilises ~100 t of low radioactivity argon from an underground source (UAr) in its inner detector, with half serving as target in a dual-phase time projection chamber (TPC). The UAr cryogenics system must maintain stable thermodynamic conditions throughout t… ▽ More DarkSide-20k (DS-20k) is a dark matter detection experiment under construction at the Laboratori Nazionali del Gran Sasso (LNGS) in Italy. It utilises ~100 t of low radioactivity argon from an underground source (UAr) in its inner detector, with half serving as target in a dual-phase time projection chamber (TPC). The UAr cryogenics system must maintain stable thermodynamic conditions throughout the experiment's lifetime of >10 years. Continuous removal of impurities and radon from the UAr is essential for maximising signal yield and mitigating background. We are developing an efficient and powerful cryogenics system with a gas purification loop with a target circulation rate of 1000 slpm. Central to its design is a condenser operated with liquid nitrogen which is paired with a gas heat exchanger cascade, delivering a combined cooling power of >8 kW. Here we present the design choices in view of the DS-20k requirements, in particular the condenser's working principle and the cooling control, and we show test results obtained with a dedicated benchmarking platform at CERN and LNGS. We find that the thermal efficiency of the recirculation loop, defined in terms of nitrogen consumption per argon flow rate, is 95 % and the pressure in the test cryostat can be maintained within $\pm$(0.1-0.2) mbar. We further detail a 5-day cool-down procedure of the test cryostat, maintaining a cooling rate typically within -2 K/h, as required for the DS-20k inner detector. Additionally, we assess the circuit's flow resistance, and the heat transfer capabilities of two heat exchanger geometries for argon phase change, used to provide gas for recirculation. We conclude by discussing how our findings influence the finalisation of the system design, including necessary modifications to meet requirements and ongoing testing activities. △ Less

Submitted 26 August, 2024; originally announced August 2024.

Comments: 45 pages, 24 figures

RealMedQA: A pilot biomedical question answering dataset containing realistic clinical questions

Authors: Gregory Kell, Angus Roberts, Serge Umansky, Yuti Khare, Najma Ahmed, Nikhil Patel, Chloe Simela, Jack Coumbe, Julian Rozario, Ryan-Rhys Griffiths, Iain J. Marshall

Abstract: Clinical question answering systems have the potential to provide clinicians with relevant and timely answers to their questions. Nonetheless, despite the advances that have been made, adoption of these systems in clinical settings has been slow. One issue is a lack of question-answering datasets which reflect the real-world needs of health professionals. In this work, we present RealMedQA, a data… ▽ More Clinical question answering systems have the potential to provide clinicians with relevant and timely answers to their questions. Nonetheless, despite the advances that have been made, adoption of these systems in clinical settings has been slow. One issue is a lack of question-answering datasets which reflect the real-world needs of health professionals. In this work, we present RealMedQA, a dataset of realistic clinical questions generated by humans and an LLM. We describe the process for generating and verifying the QA pairs and assess several QA models on BioASQ and RealMedQA to assess the relative difficulty of matching answers to questions. We show that the LLM is more cost-efficient for generating "ideal" QA pairs. Additionally, we achieve a lower lexical similarity between questions and answers than BioASQ which provides an additional challenge to the top two QA models, as per the results. We release our code and our dataset publicly to encourage further research. △ Less

Submitted 16 August, 2024; originally announced August 2024.

Comments: Accepted at AMIA Annual Symposium 2024

Describe Data to get Science-Data-Ready Tooling: Awkward as a Target for Kaitai Struct YAML

Authors: Manasvi Goyal, Andrea Zonca, Amy Roberts, Jim Pivarski, Ianna Osborne

Abstract: In some fields, scientific data formats differ across experiments due to specialized hardware and data acquisition systems. Researchers need to develop, document, and maintain experiment-specific analysis software to interact with these data formats. These software are often tightly coupled with a particular data format. This proliferation of custom data formats has been a prominent challenge for… ▽ More In some fields, scientific data formats differ across experiments due to specialized hardware and data acquisition systems. Researchers need to develop, document, and maintain experiment-specific analysis software to interact with these data formats. These software are often tightly coupled with a particular data format. This proliferation of custom data formats has been a prominent challenge for small to mid-scale experiments. The widespread adoption of ROOT has largely mitigated this problem for the Large Hadron Collider experiments. However, many smaller experiments continue to use custom data formats to meet specific research needs. Therefore, simplifying the process of accessing a unique data format for analysis holds immense value for scientific communities within HEP. We have added Awkward Arrays as a target language for Kaitai Struct for this purpose. Researchers can describe their custom data format in the Kaitai Struct YAML (KSY) language. The Kaitai Struct Compiler generates C++ code to fill the LayoutBuilder buffers using the KSY format. In a few steps, the Kaitai Struct Awkward Runtime API can convert the generated C++ code into a compiled Python module. Finally, the raw data can be passed to the module to produce Awkward Arrays. This paper introduces the Awkward Target for the Kaitai Struct Compiler and the Kaitai Struct Awkward Runtime API. It also demonstrates the conversion of a given KSY for a specific custom file format to Awkward Arrays. △ Less

Submitted 19 July, 2024; originally announced July 2024.

Comments: presented at ACAT2024

Efficient prediction of static and dynamical responses of functional graded beams using sparse multiscale patches

Authors: Thien Tran-Duc, J. E. Bunder, A. J. Roberts

Abstract: We develop a multiscale patch scheme for studying the system level characteristics of heterogeneous functional graded beams. The algorithm computes the detailed beam dynamics on the microscale, but only in small patches of the beam domain, and then applies symmetry-preserving interpolation to these patches to accurately predict the macroscale behaviour. To validate the algorithm, two examples of f… ▽ More We develop a multiscale patch scheme for studying the system level characteristics of heterogeneous functional graded beams. The algorithm computes the detailed beam dynamics on the microscale, but only in small patches of the beam domain, and then applies symmetry-preserving interpolation to these patches to accurately predict the macroscale behaviour. To validate the algorithm, two examples of functionally graded beams are investigated, namely cross-sectionally graded and axially graded. Gradient patterns are defined via volume fractions of aluminium and silicon carbine either over the beam's cross section or along its axial direction. In these examples the multiscale patch scheme only computes over a fraction of the beam's full-domain. Beam deflection and natural frequencies from the patch computations agree very well with existing experimental data and the full-domain computations. The algorithm is stable and robust, with errors consistently small and reliably reducible by increasing the number of patches. The reduction in the spatial domain of computation substantially improves the computational efficiency, with the computational time reducing by a factor of up to 17 when the patches cover 27% of the beam. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Light Dark Matter Constraints from SuperCDMS HVeV Detectors Operated Underground with an Anticoincidence Event Selection

Authors: SuperCDMS Collaboration, M. F. Albakry, I. Alkhatib, D. Alonso-González, D. W. P. Amaral, J. Anczarski, T. Aralis, T. Aramaki, I. J. Arnquist, I. Ataee Langroudy, E. Azadbakht, C. Bathurst, R. Bhattacharyya, A. J. Biffl, P. L. Brink, M. Buchanan, R. Bunker, B. Cabrera, R. Calkins, R. A. Cameron, C. Cartaro, D. G. Cerdeño, Y. -Y. Chang, M. Chaudhuri, J. -H. Chen , et al. (117 additional authors not shown)

Abstract: This article presents constraints on dark-matter-electron interactions obtained from the first underground data-taking campaign with multiple SuperCDMS HVeV detectors operated in the same housing. An exposure of 7.63 g-days is used to set upper limits on the dark-matter-electron scattering cross section for dark matter masses between 0.5 and 1000 MeV/$c^2$, as well as upper limits on dark photon k… ▽ More This article presents constraints on dark-matter-electron interactions obtained from the first underground data-taking campaign with multiple SuperCDMS HVeV detectors operated in the same housing. An exposure of 7.63 g-days is used to set upper limits on the dark-matter-electron scattering cross section for dark matter masses between 0.5 and 1000 MeV/$c^2$, as well as upper limits on dark photon kinetic mixing and axion-like particle axioelectric coupling for masses between 1.2 and 23.3 eV/$c^2$. Compared to an earlier HVeV search, sensitivity was improved as a result of an increased overburden of 225 meters of water equivalent, an anticoincidence event selection, and better pile-up rejection. In the case of dark-matter-electron scattering via a heavy mediator, an improvement by up to a factor of 25 in cross-section sensitivity was achieved. △ Less

Submitted 18 July, 2024; v1 submitted 10 July, 2024; originally announced July 2024.

Comments: 7 pages + title and references, 4 figures, and 1 table

DarkSide-20k sensitivity to light dark matter particles

Authors: DarkSide-20k Collaboration, :, F. Acerbi, P. Adhikari, P. Agnes, I. Ahmad, S. Albergo, I. F. M. Albuquerque, T. Alexander, A. K. Alton, P. Amaudruz, M. Angiolilli, E. Aprile, R. Ardito, M. Atzori Corona, D. J. Auty, M. Ave, I. C. Avetisov, O. Azzolini, H. O. Back, Z. Balmforth, A. Barrado Olmedo, P. Barrillon, G. Batignani, P. Bhowmick , et al. (289 additional authors not shown)

Abstract: The dual-phase liquid argon time projection chamber is presently one of the leading technologies to search for dark matter particles with masses below 10 GeV/c$^2$. This was demonstrated by the DarkSide-50 experiment with approximately 50 kg of low-radioactivity liquid argon as target material. The next generation experiment DarkSide-20k, currently under construction, will use 1,000 times more arg… ▽ More The dual-phase liquid argon time projection chamber is presently one of the leading technologies to search for dark matter particles with masses below 10 GeV/c$^2$. This was demonstrated by the DarkSide-50 experiment with approximately 50 kg of low-radioactivity liquid argon as target material. The next generation experiment DarkSide-20k, currently under construction, will use 1,000 times more argon and is expected to start operation in 2027. Based on the DarkSide-50 experience, here we assess the DarkSide-20k sensitivity to models predicting light dark matter particles, including Weakly Interacting Massive Particles (WIMPs) and sub-GeV/c$^2$ particles interacting with electrons in argon atoms. With one year of data, a sensitivity improvement to dark matter interaction cross-sections by at least one order of magnitude with respect to DarkSide-50 is expected for all these models. A sensitivity to WIMP--nucleon interaction cross-sections below $1\times10^{-42}$ cm$^2$ is achievable for WIMP masses above 800 MeV/c$^2$. With 10 years exposure, the neutrino fog can be reached for WIMP masses around 5 GeV/c$^2$. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: submitted to Nature Communications

Construct accurate multi-continuum micromorphic homogenisations in multi-D space-time with computer algebra

Authors: A. J. Roberts

Abstract: Homogenisation empowers the efficient macroscale system level prediction of physical problems with intricate microscale structures. Here we develop an innovative powerful, rigorous and flexible framework for asymptotic homogenisation of dynamics at the finite scale separation of real physics, with proven results underpinned by modern dynamical systems theory. The novel systematic approach removes… ▽ More Homogenisation empowers the efficient macroscale system level prediction of physical problems with intricate microscale structures. Here we develop an innovative powerful, rigorous and flexible framework for asymptotic homogenisation of dynamics at the finite scale separation of real physics, with proven results underpinned by modern dynamical systems theory. The novel systematic approach removes most of the usual assumptions, whether implicit or explicit, of other methodologies. By no longer assuming averages the methodology constructs so-called multi-continuum or micromorphic homogenisations systematically based upon the microscale physics. The developed framework and approach enables a user to straightforwardly choose and create such homogenisations with clear physical and theoretical support, and of highly controllable accuracy and fidelity. △ Less

Submitted 3 July, 2024; originally announced July 2024.

MSC Class: 35B27; 74H10; 37L10; 35B40

Using graph neural networks to reconstruct charged pion showers in the CMS High Granularity Calorimeter

Authors: M. Aamir, B. Acar, G. Adamov, T. Adams, C. Adloff, S. Afanasiev, C. Agrawal, C. Agrawal, A. Ahmad, H. A. Ahmed, S. Akbar, N. Akchurin, B. Akgul, B. Akgun, R. O. Akpinar, E. Aktas, A. AlKadhim, V. Alexakhin, J. Alimena, J. Alison, A. Alpana, W. Alshehri, P. Alvarez Dominguez, M. Alyari, C. Amendola , et al. (550 additional authors not shown)

Abstract: A novel method to reconstruct the energy of hadronic showers in the CMS High Granularity Calorimeter (HGCAL) is presented. The HGCAL is a sampling calorimeter with very fine transverse and longitudinal granularity. The active media are silicon sensors and scintillator tiles readout by SiPMs and the absorbers are a combination of lead and Cu/CuW in the electromagnetic section, and steel in the hadr… ▽ More A novel method to reconstruct the energy of hadronic showers in the CMS High Granularity Calorimeter (HGCAL) is presented. The HGCAL is a sampling calorimeter with very fine transverse and longitudinal granularity. The active media are silicon sensors and scintillator tiles readout by SiPMs and the absorbers are a combination of lead and Cu/CuW in the electromagnetic section, and steel in the hadronic section. The shower reconstruction method is based on graph neural networks and it makes use of a dynamic reduction network architecture. It is shown that the algorithm is able to capture and mitigate the main effects that normally hinder the reconstruction of hadronic showers using classical reconstruction methods, by compensating for fluctuations in the multiplicity, energy, and spatial distributions of the shower's constituents. The performance of the algorithm is evaluated using test beam data collected in 2018 prototype of the CMS HGCAL accompanied by a section of the CALICE AHCAL prototype. The capability of the method to mitigate the impact of energy leakage from the calorimeter is also demonstrated. △ Less

Submitted 30 June, 2024; v1 submitted 17 June, 2024; originally announced June 2024.

Comments: Prepared for submission to JINST

Metasurfaces for infrared multi-modal microscopy: phase contrast and bright field

Authors: Shaban B. Sulejman, Lukas Wesemann, Mikkaela McCormack, Jiajun Meng, James A. Hutchison, Niken Priscilla, Gawain McColl, Katrina Read, Wilson Sim, Andrey A. Sukhorukov, Kenneth B. Crozier, Ann Roberts

Abstract: Different imaging modalities are used to extract the diverse information carried in an optical field. Two prominent modalities include bright field and phase contrast microscopy that can visualize the amplitude and phase features of a sample, respectively. However, capturing both of these images on the same camera typically requires interchanging optical components. Metasurfaces are ultra-thin nan… ▽ More Different imaging modalities are used to extract the diverse information carried in an optical field. Two prominent modalities include bright field and phase contrast microscopy that can visualize the amplitude and phase features of a sample, respectively. However, capturing both of these images on the same camera typically requires interchanging optical components. Metasurfaces are ultra-thin nanostructures that can merge both of these operations into a single miniaturized device. Here, a silicon-based metasurface that supports a Mie resonance is demonstrated to perform near-infrared phase contrast and bright field multi-modal microscopy that can be tuned by changing the polarization of the illumination. We performed experiments using optical fields with phase variations synthesized by a spatial light modulator and introduced by propagation through semi-transparent samples, including C. elegans, unstained human prostate cancer cells and breast tissue. The results demonstrate the potential of metasurfaces for label-free point-of-care testing. △ Less

Submitted 9 June, 2024; v1 submitted 6 June, 2024; originally announced June 2024.

Comments: Main text 18 pages, 5 main figures, Supplementary information 19 pages, 19 supplementary figures, 2 supplementary videos

A new hybrid gadolinium nanoparticles-loaded polymeric material for neutron detection in rare event searches

Authors: DarkSide-20k Collaboration, :, F. Acerbi, P. Adhikari, P. Agnes, I. Ahmad, S. Albergo, I. F. Albuquerque, T. Alexander, A. K. Alton, P. Amaudruz, M. Angiolilli, E. Aprile, R. Ardito, M. Atzori Corona, D. J. Auty, M. Ave, I. C. Avetisov, O. Azzolini, H. O. Back, Z. Balmforth, A. Barrado Olmedo, P. Barrillon, G. Batignani, P. Bhowmick , et al. (290 additional authors not shown)

Abstract: Experiments aimed at direct searches for WIMP dark matter require highly effective reduction of backgrounds and control of any residual radioactive contamination. In particular, neutrons interacting with atomic nuclei represent an important class of backgrounds due to the expected similarity of a WIMP-nucleon interaction, so that such experiments often feature a dedicated neutron detector surround… ▽ More Experiments aimed at direct searches for WIMP dark matter require highly effective reduction of backgrounds and control of any residual radioactive contamination. In particular, neutrons interacting with atomic nuclei represent an important class of backgrounds due to the expected similarity of a WIMP-nucleon interaction, so that such experiments often feature a dedicated neutron detector surrounding the active target volume. In the context of the development of DarkSide-20k detector at INFN Gran Sasso National Laboratory (LNGS), several R&D projects were conceived and developed for the creation of a new hybrid material rich in both hydrogen and gadolinium nuclei to be employed as an essential element of the neutron detector. Thanks to its very high cross-section for neutron capture, gadolinium is one of the most widely used elements in neutron detectors, while the hydrogen-rich material is instrumental in efficiently moderating the neutrons. In this paper results from one of the R&Ds are presented. In this effort the new hybrid material was obtained as a poly(methyl methacrylate) (PMMA) matrix, loaded with gadolinium oxide in the form of nanoparticles. We describe its realization, including all phases of design, purification, construction, characterization, and determination of mechanical properties of the new material. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Training LLMs over Neurally Compressed Text

Authors: Brian Lester, Jaehoon Lee, Alex Alemi, Jeffrey Pennington, Adam Roberts, Jascha Sohl-Dickstein, Noah Constant

Abstract: In this paper, we explore the idea of training large language models (LLMs) over highly compressed text. While standard subword tokenizers compress text by a small factor, neural text compressors can achieve much higher rates of compression. If it were possible to train LLMs directly over neurally compressed text, this would confer advantages in training and serving efficiency, as well as easier h… ▽ More In this paper, we explore the idea of training large language models (LLMs) over highly compressed text. While standard subword tokenizers compress text by a small factor, neural text compressors can achieve much higher rates of compression. If it were possible to train LLMs directly over neurally compressed text, this would confer advantages in training and serving efficiency, as well as easier handling of long text spans. The main obstacle to this goal is that strong compression tends to produce opaque outputs that are not well-suited for learning. In particular, we find that text naïvely compressed via Arithmetic Coding is not readily learnable by LLMs. To overcome this, we propose Equal-Info Windows, a novel compression technique whereby text is segmented into blocks that each compress to the same bit length. Using this method, we demonstrate effective learning over neurally compressed text that improves with scale, and outperforms byte-level baselines by a wide margin on perplexity and inference speed benchmarks. While our method delivers worse perplexity than subword tokenizers for models trained with the same parameter count, it has the benefit of shorter sequence lengths. Shorter sequence lengths require fewer autoregressive generation steps, and reduce latency. Finally, we provide extensive analysis of the properties that contribute to learnability, and offer concrete suggestions for how to further improve the performance of high-compression tokenizers. △ Less

Submitted 13 August, 2024; v1 submitted 4 April, 2024; originally announced April 2024.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data

Authors: Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey, Rafael Rafailov, Henry Sleight, John Hughes, Tomasz Korbak, Rajashree Agrawal, Dhruv Pai, Andrey Gromov, Daniel A. Roberts, Diyi Yang, David L. Donoho, Sanmi Koyejo

Abstract: The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs? Recent investigations into model-data feedback loops proposed that such loops would lead to a phenomenon termed model collapse, under which performance progressively degrades with each model-data feedback iteration… ▽ More The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs? Recent investigations into model-data feedback loops proposed that such loops would lead to a phenomenon termed model collapse, under which performance progressively degrades with each model-data feedback iteration until fitted models become useless. However, those studies largely assumed that new data replace old data over time, where an arguably more realistic assumption is that data accumulate over time. In this paper, we ask: what effect does accumulating data have on model collapse? We empirically study this question by pretraining sequences of language models on text corpora. We confirm that replacing the original real data by each generation's synthetic data does indeed tend towards model collapse, then demonstrate that accumulating the successive generations of synthetic data alongside the original real data avoids model collapse; these results hold across a range of model sizes, architectures, and hyperparameters. We obtain similar results for deep generative models on other types of real data: diffusion models for molecule conformation generation and variational autoencoders for image generation. To understand why accumulating data can avoid model collapse, we use an analytically tractable framework introduced by prior work in which a sequence of linear models are fit to the previous models' outputs. Previous work used this framework to show that if data are replaced, the test error increases with the number of model-fitting iterations; we extend this argument to prove that if data instead accumulate, the test error has a finite upper bound independent of the number of iterations, meaning model collapse no longer occurs. △ Less

Submitted 29 April, 2024; v1 submitted 1 April, 2024; originally announced April 2024.

The Unreasonable Ineffectiveness of the Deeper Layers

Authors: Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, Daniel A. Roberts

Abstract: We empirically study a simple layer-pruning strategy for popular families of open-weight pretrained LLMs, finding minimal degradation of performance on different question-answering benchmarks until after a large fraction (up to half) of the layers are removed. To prune these models, we identify the optimal block of layers to prune by considering similarity across layers; then, to "heal" the damage… ▽ More We empirically study a simple layer-pruning strategy for popular families of open-weight pretrained LLMs, finding minimal degradation of performance on different question-answering benchmarks until after a large fraction (up to half) of the layers are removed. To prune these models, we identify the optimal block of layers to prune by considering similarity across layers; then, to "heal" the damage, we perform a small amount of finetuning. In particular, we use parameter-efficient finetuning (PEFT) methods, specifically quantization and Low Rank Adapters (QLoRA), such that each of our experiments can be performed on a single A100 GPU. From a practical perspective, these results suggest that layer pruning methods can complement other PEFT strategies to further reduce computational resources of finetuning on the one hand, and can improve the memory and latency of inference on the other hand. From a scientific perspective, the robustness of these LLMs to the deletion of layers implies either that current pretraining methods are not properly leveraging the parameters in the deeper layers of the network or that the shallow layers play a critical role in storing knowledge. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Comments: 12 + 10 pages, 5 + 4 figures

Report number: MIT-CTP/5694

A Linear, Exponential-Discontinuous Scheme for Discrete-Ordinates Calculations in Slab Geometry

Authors: Jeremy A. Roberts

Abstract: Presented here is a preliminary study of a strictly linear, discontinuous-Petrov-Galerkin scheme for the discrete-ordinates method in slab geometry. By ``linear'', we mean the discretization does not depend on the solution itself as is the case in classical ``fix-up'' schemes and other nonlinear schemes that have been explored to maintain positive solutions with improved accuracy. By discontinuous… ▽ More Presented here is a preliminary study of a strictly linear, discontinuous-Petrov-Galerkin scheme for the discrete-ordinates method in slab geometry. By ``linear'', we mean the discretization does not depend on the solution itself as is the case in classical ``fix-up'' schemes and other nonlinear schemes that have been explored to maintain positive solutions with improved accuracy. By discontinuous, we mean the angular flux $ψ$ and scalar flux $φ$ are piecewise continuous functions that may exhibit discontinuities at cell boundaries. Finally, by ``Petrov-Galerkin,'' we mean a finite-element scheme in which the ``trial'' and ``test'' functions differ. In particular, we find that a trial basis consisting of a constant and exponential function that exactly represents the step-characteristic solution with a constant and linear test basis produces a scheme (1) with slightly better local errors than the linear-discontinuous (LD) scheme (for thin cells), (2) accuracy that approaches the linear-characteristic (LC) scheme (when the LC solution is positive), and (3) is positive as long as the first two source Legendre moments satisfy $|s_1| < 3 s_0$. △ Less

Submitted 14 February, 2024; originally announced March 2024.

Comments: 4 pages, 2 figures, submitted to 2024 ANS Annual Meeting

Gemma: Open Models Based on Gemini Research and Technology

Authors: Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Pier Giuseppe Sessa, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Slone, Amélie Héliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari , et al. (83 additional authors not shown)

Abstract: This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language understanding, reasoning, and safety. We release two sizes of models (2 billion and 7 billion parameters), and provide both pretrained and fine-tuned checkpoints. Ge… ▽ More This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language understanding, reasoning, and safety. We release two sizes of models (2 billion and 7 billion parameters), and provide both pretrained and fine-tuned checkpoints. Gemma outperforms similarly sized open models on 11 out of 18 text-based tasks, and we present comprehensive evaluations of safety and responsibility aspects of the models, alongside a detailed description of model development. We believe the responsible release of LLMs is critical for improving the safety of frontier models, and for enabling the next wave of LLM innovations. △ Less

Submitted 16 April, 2024; v1 submitted 13 March, 2024; originally announced March 2024.

A 3-D Full-Wave Model to Study the Impact of Soybean Components and Structure on L-Band Backscatter

Authors: Kaiser Niknam, Jasmeet Judge, A. Kaleo Roberts, Alejandro Monsivais-Huertero, Robert Moore, Kamal Sarabandi, Jiayi Wu

Abstract: Microwave remote sensing offers a powerful tool for monitoring the growth of short, dense vegetation like soybean. As the plants mature, changes in their biomass and 3-D structure impact the electromagnetic (EM) backscatter signal. This backscatter information holds valuable insights into crop health and yield, prompting the need for a comprehensive understanding of how structural and biophysical… ▽ More Microwave remote sensing offers a powerful tool for monitoring the growth of short, dense vegetation like soybean. As the plants mature, changes in their biomass and 3-D structure impact the electromagnetic (EM) backscatter signal. This backscatter information holds valuable insights into crop health and yield, prompting the need for a comprehensive understanding of how structural and biophysical properties of soybeans as well as soil characteristics contribute to the overall backscatter signature. In this study, a full-wave model is developed for simulating L-band backscatter from soybean fields. Leveraging the ANSYS High-Frequency Structure Simulator (HFSS) framework, the model solves for the scattering of EM waves from realistic 3-D structural models of soybean, explicitly incorporating the interplant scattering effects. The model estimates of backscatter match well with the field observations from the SMAPVEX16-MicroWEX and SMAPVEX12, with average differences of 1-2 dB for co-pol and less than 4 dB for cross-pol. Furthermore, the model effectively replicates the temporal dynamics of crop backscatter throughout the growing season. The HFSS analysis revealed that the stems and pods are the primary contributors to HH-pol backscatter, while the branches contribute to VV-pol, and leaves impact the cross-pol signatures. In addition, a sensitivity study with 3-D bare soil surface resulted in an average variation of 8 dB in co- and cross-pol, even when the root mean square height and correlation length were held constant. △ Less

Submitted 3 February, 2024; originally announced February 2024.

Question answering systems for health professionals at the point of care -- a systematic review

Authors: Gregory Kell, Angus Roberts, Serge Umansky, Linglong Qian, Davide Ferrari, Frank Soboczenski, Byron Wallace, Nikhil Patel, Iain J Marshall

Abstract: Objective: Question answering (QA) systems have the potential to improve the quality of clinical care by providing health professionals with the latest and most relevant evidence. However, QA systems have not been widely adopted. This systematic review aims to characterize current medical QA systems, assess their suitability for healthcare, and identify areas of improvement. Materials and method… ▽ More Objective: Question answering (QA) systems have the potential to improve the quality of clinical care by providing health professionals with the latest and most relevant evidence. However, QA systems have not been widely adopted. This systematic review aims to characterize current medical QA systems, assess their suitability for healthcare, and identify areas of improvement. Materials and methods: We searched PubMed, IEEE Xplore, ACM Digital Library, ACL Anthology and forward and backward citations on 7th February 2023. We included peer-reviewed journal and conference papers describing the design and evaluation of biomedical QA systems. Two reviewers screened titles, abstracts, and full-text articles. We conducted a narrative synthesis and risk of bias assessment for each study. We assessed the utility of biomedical QA systems. Results: We included 79 studies and identified themes, including question realism, answer reliability, answer utility, clinical specialism, systems, usability, and evaluation methods. Clinicians' questions used to train and evaluate QA systems were restricted to certain sources, types and complexity levels. No system communicated confidence levels in the answers or sources. Many studies suffered from high risks of bias and applicability concerns. Only 8 studies completely satisfied any criterion for clinical utility, and only 7 reported user evaluations. Most systems were built with limited input from clinicians. Discussion: While machine learning methods have led to increased accuracy, most studies imperfectly reflected real-world healthcare information needs. Key research priorities include developing more realistic healthcare QA datasets and considering the reliability of answer sources, rather than merely focusing on accuracy. △ Less

Submitted 24 January, 2024; originally announced February 2024.

Comments: Accepted to the Journal of the American Medical Informatics Association (JAMIA)

REACT: Autonomous Intrusion Response System for Intelligent Vehicles

Authors: Mohammad Hamad, Andreas Finkenzeller, Michael Kühr, Andrew Roberts, Olaf Maennel, Vassilis Prevelakis, Sebastian Steinhorst

Abstract: Autonomous and connected vehicles are rapidly evolving, integrating numerous technologies and software. This progress, however, has made them appealing targets for cybersecurity attacks. As the risk of cyber threats escalates with this advancement, the focus is shifting from solely preventing these attacks to also mitigating their impact. Current solutions rely on vehicle security operation center… ▽ More Autonomous and connected vehicles are rapidly evolving, integrating numerous technologies and software. This progress, however, has made them appealing targets for cybersecurity attacks. As the risk of cyber threats escalates with this advancement, the focus is shifting from solely preventing these attacks to also mitigating their impact. Current solutions rely on vehicle security operation centers, where attack information is analyzed before deciding on a response strategy. However, this process can be time-consuming and faces scalability challenges, along with other issues stemming from vehicle connectivity. This paper proposes a dynamic intrusion response system integrated within the vehicle. This system enables the vehicle to respond to a variety of incidents almost instantly, thereby reducing the need for interaction with the vehicle security operation center. The system offers a comprehensive list of potential responses, a methodology for response evaluation, and various response selection methods. The proposed solution was implemented on an embedded platform. Two distinct cyberattack use cases served as the basis for evaluating the system. The evaluation highlights the system's adaptability, its ability to respond swiftly, its minimal memory footprint, and its capacity for dynamic system parameter adjustments. The proposed solution underscores the necessity and feasibility of incorporating dynamic response mechanisms in smart vehicles. This is a crucial factor in ensuring the safety and resilience of future smart mobility. △ Less

Submitted 16 January, 2024; v1 submitted 9 January, 2024; originally announced January 2024.

Comments: 20 pages

Scintillation of Ar/CF$_4$ mixtures: glass-THGEM characterization with 1% CF$_4$ at 1-1.5 bar

Authors: P. Amedo, R. Hafeji, A. Roberts, A. Lowe, S. Ravinthiran, S. Leardini, K. Majumdar, K. Mavrokoridis, D. González-Díaz

Abstract: Argon gas doped with 1% wavelength-shifter (CF$_4$) has been employed in an optical time projection chamber (OTPC) to image cosmic radiation. We present results obtained during the system commissioning, performed with two stacked glass-THGEMs and an EMCCD camera at 1 bar. Preliminary estimates indicate that the combined optical gain was of the order of 10$^6$ (ph/e), producing sharp and high-contr… ▽ More Argon gas doped with 1% wavelength-shifter (CF$_4$) has been employed in an optical time projection chamber (OTPC) to image cosmic radiation. We present results obtained during the system commissioning, performed with two stacked glass-THGEMs and an EMCCD camera at 1 bar. Preliminary estimates indicate that the combined optical gain was of the order of 10$^6$ (ph/e), producing sharp and high-contrast raw images without resorting to any filtering or post-processing. A first assessment of the impact of pressurization showed no change in the attainable gains when operating at 1.5 bar △ Less

Submitted 8 April, 2024; v1 submitted 12 December, 2023; originally announced December 2023.

Comments: Added reference with DOI, fixed error with axis label in plot

Model-independent extraction of form factors and $|V_{cb}|$ in $\overline{B} \rightarrow D \ell^- \overlineν_\ell$ with hadronic tagging at BaBar

Authors: BaBar Collaboration, J. P. Lees, V. Poireau, V. Tisserand, E. Grauges, A. Palano, G. Eigen, D. N. Brown, Yu. G. Kolomensky, M. Fritsch, H. Koch, R. Cheaib, C. Hearty, T. S. Mattison, J. A. McKenna, R. Y. So, V. E. Blinov, A. R. Buzykaev, V. P. Druzhinin, E. A. Kozyrev, E. A. Kravchenko, S. I. Serednyakov, Yu. I. Skovpen, E. P. Solodov, K. Yu. Todyshev , et al. (186 additional authors not shown)

Abstract: Using the entire BaBar $Υ(4S)$ data set, the first two-dimensional unbinned angular analysis of the semileptonic decay $\overline{B} \rightarrow D \ell^- \overlineν_\ell$ is performed, employing hadronic reconstruction of the tag-side $B$ meson from $Υ(4S)\to B\overline{B}$. Here, $\ell$ denotes the light charged leptons $e$ and $μ$. A novel data-driven signal-background separation procedure with… ▽ More Using the entire BaBar $Υ(4S)$ data set, the first two-dimensional unbinned angular analysis of the semileptonic decay $\overline{B} \rightarrow D \ell^- \overlineν_\ell$ is performed, employing hadronic reconstruction of the tag-side $B$ meson from $Υ(4S)\to B\overline{B}$. Here, $\ell$ denotes the light charged leptons $e$ and $μ$. A novel data-driven signal-background separation procedure with minimal dependence on simulation is developed. This procedure preserves all multi-dimensional correlations present in the data. The expected $\sin^2θ_\ell$ dependence of the differential decay rate in the Standard Model is demonstrated, where $θ_\ell$ is the lepton helicity angle. Including input from the latest lattice QCD calculations and previously available experimental data, the underlying form factors are extracted using both model-independent (BGL) and dependent (CLN) methods. Comparisons with lattice calculations show flavor SU(3) symmetry to be a good approximation in the $B_{(s)}\to D_{(s)}$ sector. Using the BGL results, the CKM matrix element $|V_{cb}|=(41.09\pm 1.16)\times 10^{-3}$ and the Standard Model prediction of the lepton-flavor universality violation variable $\mathcal{R}(D)=0.300\pm 0.004$, are extracted. The value of $|V_{cb}|$ from $\overline{B} \rightarrow D \ell^- \overlineν_\ell$ tends to be higher than that extracted using $\overline{B} \rightarrow D \ell^- \overlineν_\ell$. The Standard Model $\mathcal{R}(D)$ calculation is at a $1.97σ$ tension with the latest HFLAV experimental average. △ Less

Submitted 25 November, 2023; originally announced November 2023.

Reconfigurable Image Processing Metasurfaces with Phase-Change Materials

Authors: Michele Cotrufo, Shaban B. Sulejman, Lukas Wesemann, Md. Ataur Rahman, Madhu Bhaskaran, Ann Roberts, Andrea Alù

Abstract: Optical metasurfaces have been enabling reduced footprint and power consumption, as well as faster speeds, in the context of analog computing and image processing. While various image processing and optical computing functionalities have been recently demonstrated using metasurfaces, most of the considered devices are static and lack reconfigurability. Yet, the ability to dynamically reconfigure p… ▽ More Optical metasurfaces have been enabling reduced footprint and power consumption, as well as faster speeds, in the context of analog computing and image processing. While various image processing and optical computing functionalities have been recently demonstrated using metasurfaces, most of the considered devices are static and lack reconfigurability. Yet, the ability to dynamically reconfigure processing operations is key for metasurfaces to be able to compete with practical computing systems. Here, we demonstrate a passive edge-detection metasurface operating in the near-infrared regime whose image processing response can be drastically modified by temperature variations smaller than 10° C around a CMOS-compatible temperature of 65° C. Such reconfigurability is achieved by leveraging the insulator-to-metal phase transition of a thin buried layer of vanadium dioxide which, in turn, strongly alters the nonlocal response of the metasurface. Importantly, this reconfigurability is accompanied by performance metrics - such as high numerical aperture, high efficiency, isotropy, and polarization-independence - close to optimal, and it is combined with a simple geometry compatible with large-scale manufacturing. Our work paves the way to a new generation of ultra-compact, tunable, passive devices for all-optical computation, with potential applications in augmented reality, remote sensing and bio-medical imaging. △ Less

Submitted 21 November, 2023; originally announced November 2023.

Comments: Main text (18 pages, 5 figures), followed by high-resolution vector-graphic versions of the figures and by the Supplementary Information

Opportunities in Mental Health Support for Informal Dementia Caregivers Suffering from Verbal Agitation

Authors: Taewook Kim, Hyeok Kim, Angela Roberts, Maia Jacobs, Matthew Kay

Abstract: People with dementia (PwD) often present verbal agitation such as cursing, screaming, and persistently complaining. Verbal agitation can impose mental distress on informal caregivers (e.g., family, friends), which may cause severe mental illnesses, such as depression and anxiety disorders. To improve informal caregivers' mental health, we explore design opportunities by interviewing 11 informal ca… ▽ More People with dementia (PwD) often present verbal agitation such as cursing, screaming, and persistently complaining. Verbal agitation can impose mental distress on informal caregivers (e.g., family, friends), which may cause severe mental illnesses, such as depression and anxiety disorders. To improve informal caregivers' mental health, we explore design opportunities by interviewing 11 informal caregivers suffering from verbal agitation of PwD. In particular, we first characterize how the predictability of verbal agitation impacts informal caregivers' mental health and how caregivers' coping strategies vary before, during, and after verbal agitation. Based on our findings, we propose design opportunities to improve the mental health of informal caregivers suffering from verbal agitation: distracting PwD (in-situ support; before), prompting just-in-time maneuvers (information support; during), and comfort and education (social & information support; after). We discuss our reflections on cultural disparities between participants. Our work envisions a broader design space for supporting informal caregivers' well-being and describes when and how that support could be provided. △ Less

Submitted 17 November, 2023; originally announced November 2023.

Comments: 26 pages, 1 figure, 2 tables. Accepted to PACM HCI (CSCW 2024)

Report of the 1st Workshop on Generative AI and Law

Authors: A. Feder Cooper, Katherine Lee, James Grimmelmann, Daphne Ippolito, Christopher Callison-Burch, Christopher A. Choquette-Choo, Niloofar Mireshghallah, Miles Brundage, David Mimno, Madiha Zahrah Choksi, Jack M. Balkin, Nicholas Carlini, Christopher De Sa, Jonathan Frankle, Deep Ganguli, Bryant Gipson, Andres Guadamuz, Swee Leng Harris, Abigail Z. Jacobs, Elizabeth Joh, Gautam Kamath, Mark Lemley, Cass Matthews, Christine McLeavey, Corynne McSherry , et al. (10 additional authors not shown)

Abstract: This report presents the takeaways of the inaugural Workshop on Generative AI and Law (GenLaw), held in July 2023. A cross-disciplinary group of practitioners and scholars from computer science and law convened to discuss the technical, doctrinal, and policy challenges presented by law for Generative AI, and by Generative AI for law, with an emphasis on U.S. law in particular. We begin the report… ▽ More This report presents the takeaways of the inaugural Workshop on Generative AI and Law (GenLaw), held in July 2023. A cross-disciplinary group of practitioners and scholars from computer science and law convened to discuss the technical, doctrinal, and policy challenges presented by law for Generative AI, and by Generative AI for law, with an emphasis on U.S. law in particular. We begin the report with a high-level statement about why Generative AI is both immensely significant and immensely challenging for law. To meet these challenges, we conclude that there is an essential need for 1) a shared knowledge base that provides a common conceptual language for experts across disciplines; 2) clarification of the distinctive technical capabilities of generative-AI systems, as compared and contrasted to other computer and AI systems; 3) a logical taxonomy of the legal issues these systems raise; and, 4) a concrete research agenda to promote collaboration and knowledge-sharing on emerging issues at the intersection of Generative AI and law. In this report, we synthesize the key takeaways from the GenLaw workshop that begin to address these needs. All of the listed authors contributed to the workshop upon which this report is based, but they and their organizations do not necessarily endorse all of the specific claims in this report. △ Less

Submitted 2 December, 2023; v1 submitted 10 November, 2023; originally announced November 2023.

Integrable maps in 4D and modified Volterra lattices

Authors: A. N. W. Hone, J. A. G. Roberts, P. Vanhaecke, F. Zullo

Abstract: In recent work, we presented the construction of a family of difference equations associated with the Stieltjes continued fraction expansion of a certain function on a hyperelliptic curve of genus $g$. As well as proving that each such discrete system is an integrable map in the Liouville sense, we also showed it to be an algebraic completely integrable system. In the discrete setting, the latter… ▽ More In recent work, we presented the construction of a family of difference equations associated with the Stieltjes continued fraction expansion of a certain function on a hyperelliptic curve of genus $g$. As well as proving that each such discrete system is an integrable map in the Liouville sense, we also showed it to be an algebraic completely integrable system. In the discrete setting, the latter means that the generic level set of the invariants is an affine part of an abelian variety, in this case the Jacobian of the hyperelliptic curve, and each iteration of the map corresponds to a translation by a fixed vector on the Jacobian. In addition, we demonstrated that, by combining the discrete integrable dynamics with the flow of one of the commuting Hamiltonian vector fields, these maps provide genus $g$ algebro-geometric solutions of the infinite Volterra lattice, which justified naming them Volterra maps, denoted ${\cal V}_g$. The original motivation behind our work was the fact that, in the particular case $g=2$, we could recover an example of an integrable symplectic map in four dimensions found by Gubbiotti, Joshi, Tran and Viallet, who classified birational maps in 4D admitting two invariants (first integrals) with a particular degree structure, by considering recurrences of fourth order with a certain symmetry. Hence, in this particular case, the map ${\cal V}_2$ yields genus two solutions of the Volterra lattice. The purpose of this note is to point out how two of the other 4D integrable maps obtained in the classification of Gubbiotti et al. correspond to genus two solutions of two different forms of the modified Volterra lattice, being related via a Miura-type transformation to the $g=2$ Volterra map ${\cal V}_2$. We dedicate this work to a dear friend and colleague, Decio Levi. △ Less

Submitted 14 February, 2024; v1 submitted 30 October, 2023; originally announced October 2023.

Journal ref: Open Communications in Nonlinear Mathematical Physics, Special Issue in Memory of Decio Levi (February 15, 2024) ocnmp:12491

Feature Learning and Generalization in Deep Networks with Orthogonal Weights

Authors: Hannah Day, Yonatan Kahn, Daniel A. Roberts

Abstract: Fully-connected deep neural networks with weights initialized from independent Gaussian distributions can be tuned to criticality, which prevents the exponential growth or decay of signals propagating through the network. However, such networks still exhibit fluctuations that grow linearly with the depth of the network, which may impair the training of networks with width comparable to depth. We s… ▽ More Fully-connected deep neural networks with weights initialized from independent Gaussian distributions can be tuned to criticality, which prevents the exponential growth or decay of signals propagating through the network. However, such networks still exhibit fluctuations that grow linearly with the depth of the network, which may impair the training of networks with width comparable to depth. We show analytically that rectangular networks with tanh activations and weights initialized from the ensemble of orthogonal matrices have corresponding preactivation fluctuations which are independent of depth, to leading order in inverse width. Moreover, we demonstrate numerically that, at initialization, all correlators involving the neural tangent kernel (NTK) and its descendants at leading order in inverse width -- which govern the evolution of observables during training -- saturate at a depth of $\sim 20$, rather than growing without bound as in the case of Gaussian initializations. We speculate that this structure preserves finite-width feature learning while reducing overall noise, thus improving both generalization and training speed in deep networks with depth comparable to width. We provide some experimental justification by relating empirical measurements of the NTK to the superior performance of deep nonlinear orthogonal networks trained under full-batch gradient descent on the MNIST and CIFAR-10 classification tasks. △ Less

Submitted 12 June, 2024; v1 submitted 11 October, 2023; originally announced October 2023.

Comments: v2: numerical experiments updated with more data, plots updated to match, conclusions unchanged. 30+12 pages, 20 figures

Report number: MIT-CTP/5625

Rotation Curve Fitting Model

Authors: Sophia Natalia Cisneros, Richard Ott, Meagan Crowley, Amy Roberts, Marcus Paz, Zaneeyiah Brown, Landon Joyal, Roberto Real Rico, Elizabeth Gutierrez-Gutierrez, Phong Pham, Zac Holland, Amanda Livingston, Lily Castrellon, Summer Graham, Shanon J. Rubin, Aaron Ashleya, Dillon Battaglia, Daniel Lopez, Maya Salwa

Abstract: One key piece of evidence for dark matter is the flat rotation curve problem: the disagreement between measured galactic rotation curves and their luminous mass. A novel solution to this problem is presented here. A model of relativistic frame effects on Doppler shifts due to the slightly curved frames of an emitting galaxy and the Milky Way is derived. This model predicts observed Doppler shifted… ▽ More One key piece of evidence for dark matter is the flat rotation curve problem: the disagreement between measured galactic rotation curves and their luminous mass. A novel solution to this problem is presented here. A model of relativistic frame effects on Doppler shifts due to the slightly curved frames of an emitting galaxy and the Milky Way is derived. This model predicts observed Doppler shifted spectra (in excess of the luminous mass) based only on the observed luminous matter profile and one free model parameter. Fits to the 175 galaxies reported in the SPARC database of galactic rotation profiles and accurate photometry measurements are compared between this novel model and dark matter and MOND (RAR) models. We find on the SPARC sample of 175 galaxies; that MOND-RAR has an average reduced chisquare of $χ^2_r = 4.22$ for 175 galaxies fitted, the isothermal dark matter model has $χ^2_r = 1.90$ for 165 galaxies fitted, and the new model we present has $χ^2_r = 2.39$ for 172 galaxies fitted. Implications of this model are discussed. △ Less

Submitted 28 August, 2024; v1 submitted 6 October, 2023; originally announced October 2023.

Comments: 19 pages, 19 figures

A family of integrable maps associated with the Volterra lattice

Authors: A. N. W. Hone, J. A. G. Roberts, P. Vanhaecke

Abstract: Recently Gubbiotti, Joshi, Tran and Viallet classified birational maps in four dimensions admitting two invariants (first integrals) with a particular degree structure, by considering recurrences of fourth order with a certain symmetry. The last three of the maps so obtained were shown to be Liouville integrable, in the sense of admitting a non-degenerate Poisson bracket with two first integrals i… ▽ More Recently Gubbiotti, Joshi, Tran and Viallet classified birational maps in four dimensions admitting two invariants (first integrals) with a particular degree structure, by considering recurrences of fourth order with a certain symmetry. The last three of the maps so obtained were shown to be Liouville integrable, in the sense of admitting a non-degenerate Poisson bracket with two first integrals in involution. Here we show how the first of these three Liouville integrable maps corresponds to genus 2 solutions of the infinite Volterra lattice, being the $g=2$ case of a family of maps associated with the Stieltjes continued fraction expansion of a certain function on a hyperelliptic curve of genus $g\geqslant 1$. The continued fraction method provides explicit Hankel determinant formulae for tau functions of the solutions, together with an algebro-geometric description via a Lax representation for each member of the family, associating it with an algebraic completely integrable system. In particular, in the elliptic case ($g=1$), as a byproduct we obtain Hankel determinant expressions for the solutions of the Somos-5 recurrence, but different to those previously derived by Chang, Hu and Xin. By applying contraction to the Stieltjes fraction, we recover integrable maps associated with Jacobi continued fractions on hyperelliptic curves, that one of us considered previously, as well as the Miura-type transformation between the Volterra and Toda lattices. △ Less

Submitted 22 August, 2024; v1 submitted 5 September, 2023; originally announced September 2023.

Comments: Final version including Maple code in Appendix B

Journal ref: Nonlinearity 37 095028 (2024)

Sample Size in Natural Language Processing within Healthcare Research

Authors: Jaya Chaturvedi, Diana Shamsutdinova, Felix Zimmer, Sumithra Velupillai, Daniel Stahl, Robert Stewart, Angus Roberts

Abstract: Sample size calculation is an essential step in most data-based disciplines. Large enough samples ensure representativeness of the population and determine the precision of estimates. This is true for most quantitative studies, including those that employ machine learning methods, such as natural language processing, where free-text is used to generate predictions and classify instances of text. W… ▽ More Sample size calculation is an essential step in most data-based disciplines. Large enough samples ensure representativeness of the population and determine the precision of estimates. This is true for most quantitative studies, including those that employ machine learning methods, such as natural language processing, where free-text is used to generate predictions and classify instances of text. Within the healthcare domain, the lack of sufficient corpora of previously collected data can be a limiting factor when determining sample sizes for new studies. This paper tries to address the issue by making recommendations on sample sizes for text classification tasks in the healthcare domain. Models trained on the MIMIC-III database of critical care records from Beth Israel Deaconess Medical Center were used to classify documents as having or not having Unspecified Essential Hypertension, the most common diagnosis code in the database. Simulations were performed using various classifiers on different sample sizes and class proportions. This was repeated for a comparatively less common diagnosis code within the database of diabetes mellitus without mention of complication. Smaller sample sizes resulted in better results when using a K-nearest neighbours classifier, whereas larger sample sizes provided better results with support vector machines and BERT models. Overall, a sample size larger than 1000 was sufficient to provide decent performance metrics. The simulations conducted within this study provide guidelines that can be used as recommendations for selecting appropriate sample sizes and class proportions, and for predicting expected performance, when building classifiers for textual healthcare data. The methodology used here can be modified for sample size estimates calculations with other datasets. △ Less

Submitted 5 September, 2023; originally announced September 2023.

Comments: Submitted to Journal of Biomedical Informatics

The Rate of Convergence for Selberg's Central Limit Theorem under the Riemann Hypothesis

Authors: Asher Roberts

Abstract: We assume the Riemann hypothesis to improve upon the rate of convergence of $(\log\log\log T)^2/\sqrt{\log\log T}$ in Selberg's central limit theorem for $\log|ζ(1/2+it)|$ given by the author. We achieve a rate of convergence of $\sqrt{\log\log\log\log T}/\sqrt{\log\log T}$ in the Dudley distance. The proof is an adaptation of the techniques used by the author, based on the work of Radziwill and S… ▽ More We assume the Riemann hypothesis to improve upon the rate of convergence of $(\log\log\log T)^2/\sqrt{\log\log T}$ in Selberg's central limit theorem for $\log|ζ(1/2+it)|$ given by the author. We achieve a rate of convergence of $\sqrt{\log\log\log\log T}/\sqrt{\log\log T}$ in the Dudley distance. The proof is an adaptation of the techniques used by the author, based on the work of Radziwill and Soundararajan and Arguin et al., combined with a lemma of Selberg that provides for a mollifier close to the critical line $\operatorname{Re}(s)=1/2$ under the Riemann hypothesis. △ Less

Submitted 18 August, 2023; originally announced August 2023.

Comments: This paper is submitted to the the Proceedings of CANT 2023. 13 pages

MSC Class: 11M06; 60F05

Efficient computational homogenisation of 2D beams of heterogeneous elasticity using the patch scheme

Authors: Thien Tran-Duc, J. E. Bunder, A. J. Roberts

Abstract: Modern 'smart' materials have complex heterogeneous microscale structure, often with unknown macroscale closure but one we need to realise for large scale engineering and science. The multiscale Equation-Free Patch Scheme empowers us to non-intrusively, efficiently, and accurately predict the large scale, system level, solutions through computations on only small sparse patches of the given detail… ▽ More Modern 'smart' materials have complex heterogeneous microscale structure, often with unknown macroscale closure but one we need to realise for large scale engineering and science. The multiscale Equation-Free Patch Scheme empowers us to non-intrusively, efficiently, and accurately predict the large scale, system level, solutions through computations on only small sparse patches of the given detailed microscale system. Here the microscale system is that of a 2D beam of heterogeneous elasticity, with either fixed fixed, fixed-free, or periodic boundary conditions. We demonstrate that the described multiscale Patch Scheme simply, efficiently, and stably predicts the beam's macroscale, with a controllable accuracy, at finite scale separation. Dynamical systems theory supports the scheme. This article points the way for others to use this systematic non-intrusive approach, via a developing toolbox of functions, to model and compute accurately macroscale system-levels of general complex physical and engineering systems. △ Less

Submitted 17 August, 2023; originally announced August 2023.

Development of a Knowledge Graph Embeddings Model for Pain

Authors: Jaya Chaturvedi, Tao Wang, Sumithra Velupillai, Robert Stewart, Angus Roberts

Abstract: Pain is a complex concept that can interconnect with other concepts such as a disorder that might cause pain, a medication that might relieve pain, and so on. To fully understand the context of pain experienced by either an individual or across a population, we may need to examine all concepts related to pain and the relationships between them. This is especially useful when modeling pain that has… ▽ More Pain is a complex concept that can interconnect with other concepts such as a disorder that might cause pain, a medication that might relieve pain, and so on. To fully understand the context of pain experienced by either an individual or across a population, we may need to examine all concepts related to pain and the relationships between them. This is especially useful when modeling pain that has been recorded in electronic health records. Knowledge graphs represent concepts and their relations by an interlinked network, enabling semantic and context-based reasoning in a computationally tractable form. These graphs can, however, be too large for efficient computation. Knowledge graph embeddings help to resolve this by representing the graphs in a low-dimensional vector space. These embeddings can then be used in various downstream tasks such as classification and link prediction. The various relations associated with pain which are required to construct such a knowledge graph can be obtained from external medical knowledge bases such as SNOMED CT, a hierarchical systematic nomenclature of medical terms. A knowledge graph built in this way could be further enriched with real-world examples of pain and its relations extracted from electronic health records. This paper describes the construction of such knowledge graph embedding models of pain concepts, extracted from the unstructured text of mental health electronic health records, combined with external knowledge created from relations described in SNOMED CT, and their evaluation on a subject-object link prediction task. The performance of the models was compared with other baseline models. △ Less

Submitted 17 August, 2023; originally announced August 2023.

Comments: Accepted at AMIA 2023, New Orleans

Directionality of nuclear recoils in a liquid argon time projection chamber

Authors: The DarkSide-20k Collaboration, :, P. Agnes, I. Ahmad, S. Albergo, I. F. M. Albuquerque, T. Alexander, A. K. Alton, P. Amaudruz, M. Atzori Corona, M. Ave, I. Ch. Avetisov, O. Azzolini, H. O. Back, Z. Balmforth, A. Barrado-Olmedo, P. Barrillon, A. Basco, G. Batignani, V. Bocci, W. M. Bonivento, B. Bottino, M. G. Boulay, J. Busto, M. Cadeddu , et al. (243 additional authors not shown)

Abstract: The direct search for dark matter in the form of weakly interacting massive particles (WIMP) is performed by detecting nuclear recoils (NR) produced in a target material from the WIMP elastic scattering. A promising experimental strategy for direct dark matter search employs argon dual-phase time projection chambers (TPC). One of the advantages of the TPC is the capability to detect both the scint… ▽ More The direct search for dark matter in the form of weakly interacting massive particles (WIMP) is performed by detecting nuclear recoils (NR) produced in a target material from the WIMP elastic scattering. A promising experimental strategy for direct dark matter search employs argon dual-phase time projection chambers (TPC). One of the advantages of the TPC is the capability to detect both the scintillation and charge signals produced by NRs. Furthermore, the existence of a drift electric field in the TPC breaks the rotational symmetry: the angle between the drift field and the momentum of the recoiling nucleus can potentially affect the charge recombination probability in liquid argon and then the relative balance between the two signal channels. This fact could make the detector sensitive to the directionality of the WIMP-induced signal, enabling unmistakable annual and daily modulation signatures for future searches aiming for discovery. The Recoil Directionality (ReD) experiment was designed to probe for such directional sensitivity. The TPC of ReD was irradiated with neutrons at the INFN Laboratori Nazionali del Sud, and data were taken with 72 keV NRs of known recoil directions. The direction-dependent liquid argon charge recombination model by Cataudella et al. was adopted and a likelihood statistical analysis was performed, which gave no indications of significant dependence of the detector response to the recoil direction. The aspect ratio R of the initial ionization cloud is estimated to be 1.037 +/- 0.027 and the upper limit is R < 1.072 with 90% confidence level △ Less

Submitted 28 July, 2023; originally announced July 2023.

Comments: 20 pages, 10 figures, submitted to Eur. Phys. J. C

Journal ref: Eur. Phys. J. C 84:24 (2024)

A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity

Authors: Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, Daphne Ippolito

Abstract: Pretraining is the preliminary and fundamental step in developing capable language models (LM). Despite this, pretraining data design is critically under-documented and often guided by empirically unsupported intuitions. To address this, we pretrain 28 1.5B parameter decoder-only models, training on data curated (1) at different times, (2) with varying toxicity and quality filters, and (3) with di… ▽ More Pretraining is the preliminary and fundamental step in developing capable language models (LM). Despite this, pretraining data design is critically under-documented and often guided by empirically unsupported intuitions. To address this, we pretrain 28 1.5B parameter decoder-only models, training on data curated (1) at different times, (2) with varying toxicity and quality filters, and (3) with different domain compositions. First, we quantify the effect of pretraining data age. A temporal shift between evaluation data and pretraining data leads to performance degradation, which is not overcome by finetuning. Second, we explore the effect of quality and toxicity filters, showing a trade-off between performance on standard benchmarks and risk of toxic generations. Our findings indicate there does not exist a one-size-fits-all solution to filtering training data. We also find that the effects of different types of filtering are not predictable from text domain characteristics. Lastly, we empirically validate that the inclusion of heterogeneous data sources, like books and web, is broadly beneficial and warrants greater prioritization. These findings constitute the largest set of experiments to validate, quantify, and expose many undocumented intuitions about text pretraining, which we hope will help support more informed data-centric decisions in LM development. △ Less

Submitted 13 November, 2023; v1 submitted 22 May, 2023; originally announced May 2023.

A Hilbertian projection method for constrained level set-based topology optimisation

Authors: Zachary J. Wegert, Anthony P. Roberts, Vivien J. Challis

Abstract: We present an extension of the projection method proposed by Challis et al. (Int J Solids Struct 45(14$\unicode{x2013}$15):4130$\unicode{x2013}$4146, 2008) for constrained level set-based topology optimisation that harnesses the Hilbertian velocity extension-regularisation framework. Our Hilbertian projection method chooses a normal velocity for the level set function as a linear combination of (1… ▽ More We present an extension of the projection method proposed by Challis et al. (Int J Solids Struct 45(14$\unicode{x2013}$15):4130$\unicode{x2013}$4146, 2008) for constrained level set-based topology optimisation that harnesses the Hilbertian velocity extension-regularisation framework. Our Hilbertian projection method chooses a normal velocity for the level set function as a linear combination of (1) an orthogonal projection operator applied to the extended optimisation objective shape sensitivity and (2) a weighted sum of orthogonal basis functions for the extended constraint shape sensitivities. This combination aims for the best possible first-order improvement of the optimisation objective in addition to first-order improvement of the constraints. Our formulation utilising basis orthogonalisation naturally handles linearly dependent constraint shape sensitivities. Furthermore, use of the Hilbertian extension-regularisation framework ensures that the resulting normal velocity is extended away from the boundary and enriched with additional regularity. Our approach is generally applicable to any topology optimisation problem to be solved in the level set framework. We consider several benchmark constrained microstructure optimisation problems and demonstrate that our method is effective with little-to-no parameter tuning. We also find that our method performs well when compared to a Hilbertian sequential linear programming method. △ Less

Submitted 10 September, 2023; v1 submitted 5 May, 2023; originally announced May 2023.

Comments: 23 pages, 8 figures

Journal ref: Struct Multidisc Optim 66, 204 (2023)

Maximum Spherical Mean Value (mSMV) Filtering for Whole Brain Quantitative Susceptibility Mapping

Authors: Alexandra G. Roberts, Dominick J. Romano, Mert Şişman, Alexey V. Dimov, Pascal Spincemaille, Thanh D. Nguyen, Ilhami Kovanlikaya, Susan A. Gauthier, Yi Wang

Abstract: To develop a tissue field filtering algorithm, called maximum Spherical Mean Value (mSMV), for reducing shadow artifacts in quantitative susceptibility mapping (QSM) of the brain without requiring brain tissue erosion.Residual background field is a major source of shadow artifacts in QSM. The mSMV algorithm filters large field values near the border, where the maximum value of the harmonic backgro… ▽ More To develop a tissue field filtering algorithm, called maximum Spherical Mean Value (mSMV), for reducing shadow artifacts in quantitative susceptibility mapping (QSM) of the brain without requiring brain tissue erosion.Residual background field is a major source of shadow artifacts in QSM. The mSMV algorithm filters large field values near the border, where the maximum value of the harmonic background field is located. The effectiveness of mSMV for artifact removal was evaluated by comparing with existing QSM algorithms in numerical brain simulation as well as using in vivo human data acquired from 11 healthy volunteers and 93 patients. Numerical simulation showed that mSMV reduces shadow artifacts and improves QSM accuracy. Better shadow reduction, as demonstrated by lower QSM variation in the gray matter and higher QSM image quality score, was also observed in healthy subjects and in patients with hemorrhages, stroke and multiple sclerosis. The mSMV algorithm allows QSM maps that are substantially equivalent to those obtained using SMV-filtered dipole inversion without eroding the volume of interest. △ Less

Submitted 27 November, 2023; v1 submitted 22 April, 2023; originally announced April 2023.

Comments: 12 pages, 5 figures

UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining

Authors: Hyung Won Chung, Noah Constant, Xavier Garcia, Adam Roberts, Yi Tay, Sharan Narang, Orhan Firat

Abstract: Pretrained multilingual large language models have typically used heuristic temperature-based sampling to balance between different languages. However previous work has not systematically evaluated the efficacy of different pretraining language distributions across model scales. In this paper, we propose a new sampling method, UniMax, that delivers more uniform coverage of head languages while mit… ▽ More Pretrained multilingual large language models have typically used heuristic temperature-based sampling to balance between different languages. However previous work has not systematically evaluated the efficacy of different pretraining language distributions across model scales. In this paper, we propose a new sampling method, UniMax, that delivers more uniform coverage of head languages while mitigating overfitting on tail languages by explicitly capping the number of repeats over each language's corpus. We perform an extensive series of ablations testing a range of sampling strategies on a suite of multilingual benchmarks, while varying model scale. We find that UniMax outperforms standard temperature-based sampling, and the benefits persist as scale increases. As part of our contribution, we release: (i) an improved and refreshed mC4 multilingual corpus consisting of 29 trillion characters across 107 languages, and (ii) a suite of pretrained umT5 model checkpoints trained with UniMax sampling. △ Less

Submitted 18 April, 2023; originally announced April 2023.

Identifying Mentions of Pain in Mental Health Records Text: A Natural Language Processing Approach

Authors: Jaya Chaturvedi, Sumithra Velupillai, Robert Stewart, Angus Roberts

Abstract: Pain is a common reason for accessing healthcare resources and is a growing area of research, especially in its overlap with mental health. Mental health electronic health records are a good data source to study this overlap. However, much information on pain is held in the free text of these records, where mentions of pain present a unique natural language processing problem due to its ambiguous… ▽ More Pain is a common reason for accessing healthcare resources and is a growing area of research, especially in its overlap with mental health. Mental health electronic health records are a good data source to study this overlap. However, much information on pain is held in the free text of these records, where mentions of pain present a unique natural language processing problem due to its ambiguous nature. This project uses data from an anonymised mental health electronic health records database. The data are used to train a machine learning based classification algorithm to classify sentences as discussing patient pain or not. This will facilitate the extraction of relevant pain information from large databases, and the use of such outputs for further studies on pain and mental health. 1,985 documents were manually triple-annotated for creation of gold standard training data, which was used to train three commonly used classification algorithms. The best performing model achieved an F1-score of 0.98 (95% CI 0.98-0.99). △ Less

Submitted 5 April, 2023; v1 submitted 3 April, 2023; originally announced April 2023.

Comments: 5 pages, 2 tables, submitted to MEDINFO 2023 conference

Critical curves of rotations

Authors: John A G Roberts, Asaki Saito, Franco Vivaldi

Abstract: In rotations with a binary symbolic dynamics, a critical curve is the locus of parameters for which the boundaries of the partition that defines the symbolic dynamics are connected via a prescribed number of iterations and symbolic itinerary. We study the arithmetical and geometrical properties of these curves in parameter space. In rotations with a binary symbolic dynamics, a critical curve is the locus of parameters for which the boundaries of the partition that defines the symbolic dynamics are connected via a prescribed number of iterations and symbolic itinerary. We study the arithmetical and geometrical properties of these curves in parameter space. △ Less

Submitted 31 March, 2023; originally announced March 2023.

Comments: 21 pages, 4 figures

MSC Class: 37E10 (Primary) 37B10 (Secondary)

First measurement of the nuclear-recoil ionization yield in silicon at 100 eV

Authors: M. F. Albakry, I. Alkhatib, D. Alonso, D. W. P. Amaral, P. An, T. Aralis, T. Aramaki, I. J. Arnquist, I. Ataee Langroudy, E. Azadbakht, S. Banik, P. S. Barbeau, C. Bathurst, R. Bhattacharyya, P. L. Brink, R. Bunker, B. Cabrera, R. Calkins, R. A. Cameron, C. Cartaro, D. G. Cerdeño, Y. -Y. Chang, M. Chaudhuri, R. Chen, N. Chott , et al. (115 additional authors not shown)

Abstract: We measured the nuclear--recoil ionization yield in silicon with a cryogenic phonon-sensitive gram-scale detector. Neutrons from a mono-energetic beam scatter off of the silicon nuclei at angles corresponding to energy depositions from 4\,keV down to 100\,eV, the lowest energy probed so far. The results show no sign of an ionization production threshold above 100\,eV. These results call for furthe… ▽ More We measured the nuclear--recoil ionization yield in silicon with a cryogenic phonon-sensitive gram-scale detector. Neutrons from a mono-energetic beam scatter off of the silicon nuclei at angles corresponding to energy depositions from 4\,keV down to 100\,eV, the lowest energy probed so far. The results show no sign of an ionization production threshold above 100\,eV. These results call for further investigation of the ionization yield theory and a comprehensive determination of the detector response function at energies below the keV scale. △ Less

Submitted 3 March, 2023; originally announced March 2023.

Journal ref: Physical Review Letters 131.9 (2023): 091801

Assessing Parameterized Geometric Models of Woven Composites using Image-Based Simulations

Authors: Collin W. Foster, Lincoln N. Collins, Francesco Panerai, Scott A. Roberts

Abstract: Mesoscale simulations of woven composites using parameterized analytical geometries offer a way to connect constituent material properties and their geometric arrangement to effective composite properties and performance. However, the reality of as-manufactured materials often differs from the ideal, both in terms of tow geometry and manufacturing heterogeneity. As such, resultant composite proper… ▽ More Mesoscale simulations of woven composites using parameterized analytical geometries offer a way to connect constituent material properties and their geometric arrangement to effective composite properties and performance. However, the reality of as-manufactured materials often differs from the ideal, both in terms of tow geometry and manufacturing heterogeneity. As such, resultant composite properties may differ from analytical predictions and exhibit significant local variations within a material. We employ mesoscale finite element method simulations to compare idealized analytical and as-manufactured woven composite materials and study the sensitivity of their effective properties to the mesoscale geometry. Three-dimensional geometries are reconstructed from X-ray computed tomography, image segmentation is performed using deep learning methods, and local fiber orientation is obtained using the structure tensor calculated from image scans. Suitable approximations to composite properties, using analytical unit cell calculations and effective media theory, are assessed. Our findings show that an analytical geometry and sub-unit cell geometry provide reasonable predictions for the effective thermal properties of a multi-layer production composite. △ Less

Submitted 19 February, 2023; originally announced February 2023.

A Search for Low-mass Dark Matter via Bremsstrahlung Radiation and the Migdal Effect in SuperCDMS

Authors: M. F. Albakry, I. Alkhatib, D. Alonso, D. W. P. Amaral, T. Aralis, T. Aramaki, I. J. Arnquist, I. Ataee Langroudy, E. Azadbakht, S. Banik, C. Bathurst, R. Bhattacharyya, P. L. Brink, R. Bunker, B. Cabrera, R. Calkins, R. A. Cameron, C. Cartaro, D. G. Cerdeño, Y. -Y. Chang, M. Chaudhuri, R. Chen, N. Chott, J. Cooley, H. Coombes , et al. (108 additional authors not shown)

Abstract: We present a new analysis of previously published of SuperCDMS data using a profile likelihood framework to search for sub-GeV dark matter (DM) particles through two inelastic scattering channels: bremsstrahlung radiation and the Migdal effect. By considering these possible inelastic scattering channels, experimental sensitivity can be extended to DM masses that are undetectable through the DM-nuc… ▽ More We present a new analysis of previously published of SuperCDMS data using a profile likelihood framework to search for sub-GeV dark matter (DM) particles through two inelastic scattering channels: bremsstrahlung radiation and the Migdal effect. By considering these possible inelastic scattering channels, experimental sensitivity can be extended to DM masses that are undetectable through the DM-nucleon elastic scattering channel, given the energy threshold of current experiments. We exclude DM masses down to $220~\textrm{MeV}/c^2$ at $2.7 \times 10^{-30}~\textrm{cm}^2$ via the bremsstrahlung channel. The Migdal channel search provides overall considerably more stringent limits and excludes DM masses down to $30~\textrm{MeV}/c^2$ at $5.0 \times 10^{-30}~\textrm{cm}^2$. △ Less

Submitted 17 February, 2023; originally announced February 2023.

Comments: Submitted to PRD

Report number: 112013

Journal ref: Phys. Rev. D 107, 2023

Search for $B$ Mesogenesis at BABAR

Authors: BABAR Collaboration, J. P. Lees, V. Poireau, V. Tisserand, E. Grauges, A. Palano, G. Eigen, D. N. Brown, Yu. G. Kolomensky, M. Fritsch, H. Koch, R. Cheaib, C. Hearty, T. S. Mattison, J. A. McKenna, R. Y. So, V. E. Blinov, A. R. Buzykaev, V. P. Druzhinin, V. B. Golubev, E. A. Kozyrev, E. A. Kravchenko, A. P. Onuchin, S. I. Serednyakov, Yu. I. Skovpen , et al. (218 additional authors not shown)

Abstract: A new mechanism has been proposed to simultaneously explain the presence of dark matter and the matter-antimatter asymmetry in the universe. This scenario predicts exotic $B$ meson decays into a baryon and a dark sector anti-baryon ($ψ_D$) with branching fractions accessible at $B$ factories. We present a search for $B \rightarrow Λψ_D$ decays using data collected by the $BABAR$ experiment at SLAC… ▽ More A new mechanism has been proposed to simultaneously explain the presence of dark matter and the matter-antimatter asymmetry in the universe. This scenario predicts exotic $B$ meson decays into a baryon and a dark sector anti-baryon ($ψ_D$) with branching fractions accessible at $B$ factories. We present a search for $B \rightarrow Λψ_D$ decays using data collected by the $BABAR$ experiment at SLAC. This reaction is identified by fully reconstructing the accompanying $B$ meson and requiring the presence of a single $Λ$ baryon in the remaining particles. No significant signal is observed, and bounds on the $B \rightarrow Λψ_D$ branching fraction are derived in the range $0.13 - 5.2\times 10^{-5}$ for $1.0 < m_{ψ_D} < 4.2$ GeV/$c^{2}$. These results set strong constraints on the parameter space allowed by the theory. △ Less

Submitted 31 January, 2023; originally announced February 2023.

Journal ref: PHYS. REV. D 107, 092001 (2023)

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

Authors: Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, Adam Roberts

Abstract: We study the design decisions of publicly available instruction tuning methods, and break down the development of Flan 2022 (Chung et al., 2022). Through careful ablation studies on the Flan Collection of tasks and methods, we tease apart the effect of design decisions which enable Flan-T5 to outperform prior work by 3-17%+ across evaluation settings. We find task balancing and enrichment techniqu… ▽ More We study the design decisions of publicly available instruction tuning methods, and break down the development of Flan 2022 (Chung et al., 2022). Through careful ablation studies on the Flan Collection of tasks and methods, we tease apart the effect of design decisions which enable Flan-T5 to outperform prior work by 3-17%+ across evaluation settings. We find task balancing and enrichment techniques are overlooked but critical to effective instruction tuning, and in particular, training with mixed prompt settings (zero-shot, few-shot, and chain-of-thought) actually yields stronger (2%+) performance in all settings. In further experiments, we show Flan-T5 requires less finetuning to converge higher and faster than T5 on single downstream tasks, motivating instruction-tuned models as more computationally-efficient starting checkpoints for new tasks. Finally, to accelerate research on instruction tuning, we make the Flan 2022 collection of datasets, templates, and methods publicly available at https://github.com/google-research/FLAN/tree/main/flan/v2. △ Less

Submitted 14 February, 2023; v1 submitted 31 January, 2023; originally announced January 2023.

Accurate and efficient multiscale simulation of a heterogeneous elastic beam via computation on small sparse patches

Authors: A. J. Roberts, Thien Tran-Duc, J. E. Bunder, Yannis Kevrekidis

Abstract: Modern `smart' materials have complex microscale structure, often with unknown macroscale closure. The Equation-Free Patch Scheme empowers us to non-intrusively, efficiently, and accurately simulate over large scales through computations on only small well-separated patches of the microscale system. Here the microscale system is a solid beam of random heterogeneous elasticity. The continuing chall… ▽ More Modern `smart' materials have complex microscale structure, often with unknown macroscale closure. The Equation-Free Patch Scheme empowers us to non-intrusively, efficiently, and accurately simulate over large scales through computations on only small well-separated patches of the microscale system. Here the microscale system is a solid beam of random heterogeneous elasticity. The continuing challenge is to compute the given physics on just the microscale patches, and couple the patches across un-simulated macroscale space, in order to establish efficiency, accuracy, consistency, and stability on the macroscale. Dynamical systems theory supports the scheme. This research program is to develop a systematic non-intrusive approach, both computationally and analytically proven, to model and compute accurately macroscale system levels of general complex physical and engineering systems. △ Less

Submitted 20 January, 2023; originally announced January 2023.

SingSong: Generating musical accompaniments from singing

Authors: Chris Donahue, Antoine Caillon, Adam Roberts, Ethan Manilow, Philippe Esling, Andrea Agostinelli, Mauro Verzetti, Ian Simon, Olivier Pietquin, Neil Zeghidour, Jesse Engel

Abstract: We present SingSong, a system that generates instrumental music to accompany input vocals, potentially offering musicians and non-musicians alike an intuitive new way to create music featuring their own voice. To accomplish this, we build on recent developments in musical source separation and audio generation. Specifically, we apply a state-of-the-art source separation algorithm to a large corpus… ▽ More We present SingSong, a system that generates instrumental music to accompany input vocals, potentially offering musicians and non-musicians alike an intuitive new way to create music featuring their own voice. To accomplish this, we build on recent developments in musical source separation and audio generation. Specifically, we apply a state-of-the-art source separation algorithm to a large corpus of music audio to produce aligned pairs of vocals and instrumental sources. Then, we adapt AudioLM (Borsos et al., 2022) -- a state-of-the-art approach for unconditional audio generation -- to be suitable for conditional "audio-to-audio" generation tasks, and train it on the source-separated (vocal, instrumental) pairs. In a pairwise comparison with the same vocal inputs, listeners expressed a significant preference for instrumentals generated by SingSong compared to those from a strong retrieval baseline. Sound examples at https://g.co/magenta/singsong △ Less

Submitted 29 January, 2023; originally announced January 2023.

MusicLM: Generating Music From Text

Authors: Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, Christian Frank

Abstract: We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff". MusicLM casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task, and it generates music at 24 kHz that remains consistent over several minutes. Our experiments show that MusicLM outperforms previous s… ▽ More We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff". MusicLM casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task, and it generates music at 24 kHz that remains consistent over several minutes. Our experiments show that MusicLM outperforms previous systems both in audio quality and adherence to the text description. Moreover, we demonstrate that MusicLM can be conditioned on both text and a melody in that it can transform whistled and hummed melodies according to the style described in a text caption. To support future research, we publicly release MusicCaps, a dataset composed of 5.5k music-text pairs, with rich text descriptions provided by human experts. △ Less

Submitted 26 January, 2023; originally announced January 2023.

Comments: Supplementary material at https://google-research.github.io/seanet/musiclm/examples and https://kaggle.com/datasets/googleai/musiccaps

ARIADNE+: Large scale demonstration of fast optical readout for dual-phase LArTPCs at the CERN Neutrino Platform

Authors: Adam Lowe, Pablo Amedo, Diego González-Díaz, Alexander Deisting, Krishanu Majumdar, Konstantinos Mavrokoridis, Marzio Nessi, Barney Philippou, Francesco Pietropaolo, Sudikshan Ravinthiran, Filippo Resnati, Adam Roberts, Angela Saá Hernández, Christos Touramanis, Jared Vann

Abstract: Optical readout of large scale dual-phase liquid Argon TPCs is an attractive alternative to charge readout and has been successfully demonstrated on a 2x2m active region within the CERN protoDUNE cold box. ARIADNE+ uses four Timepix3 cameras imaging the S2 light produced by 16 novel, patent pending, glass THGEMs. ARIADNE+ takes advantage of the raw Timepix3 data coming natively 3D and zero suppres… ▽ More Optical readout of large scale dual-phase liquid Argon TPCs is an attractive alternative to charge readout and has been successfully demonstrated on a 2x2m active region within the CERN protoDUNE cold box. ARIADNE+ uses four Timepix3 cameras imaging the S2 light produced by 16 novel, patent pending, glass THGEMs. ARIADNE+ takes advantage of the raw Timepix3 data coming natively 3D and zero suppressed with a 1.6ns timing resolution. Three of the four THGEM quadrants implement readout in the visible light range through wavelength shifting, with the fourth featuring a VUV light intensifier, thus removing the need for wavelength shifting altogether. Cosmic ray reconstruction and energy calibration was performed. Presented is a summary of the detector setup and experimental run, preliminary analysis of the run data and future outlook for the ARIADNE program. △ Less

Submitted 25 August, 2023; v1 submitted 6 January, 2023; originally announced January 2023.

Comments: Proceedings for NuFACT2022

Character-Aware Models Improve Visual Text Rendering

Authors: Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mohammad Norouzi, Noah Constant

Abstract: Current image generation models struggle to reliably produce well-formed visual text. In this paper, we investigate a key contributing factor: popular text-to-image models lack character-level input features, making it much harder to predict a word's visual makeup as a series of glyphs. To quantify this effect, we conduct a series of experiments comparing character-aware vs. character-blind text e… ▽ More Current image generation models struggle to reliably produce well-formed visual text. In this paper, we investigate a key contributing factor: popular text-to-image models lack character-level input features, making it much harder to predict a word's visual makeup as a series of glyphs. To quantify this effect, we conduct a series of experiments comparing character-aware vs. character-blind text encoders. In the text-only domain, we find that character-aware models provide large gains on a novel spelling task (WikiSpell). Applying our learnings to the visual domain, we train a suite of image generation models, and show that character-aware variants outperform their character-blind counterparts across a range of novel text rendering tasks (our DrawText benchmark). Our models set a much higher state-of-the-art on visual spelling, with 30+ point accuracy gains over competitors on rare words, despite training on far fewer examples. △ Less

Submitted 3 May, 2023; v1 submitted 20 December, 2022; originally announced December 2022.

The Multivariate Rate of Convergence for Selberg's Central Limit Theorem

Authors: Asher Roberts

Abstract: In this paper we quantify the rate of convergence in Selberg's central limit theorem for $\log|ζ(1/2+it)|$ based on the method of proof given by Radziwill and Soundararajan. We achieve the same rate of convergence of $(\log\log\log T)^2/\sqrt{\log\log T}$ as Selberg in the Kolmogorov distance by using the Dudley distance instead. We also prove the theorem for the multivariate case given by Bourgad… ▽ More In this paper we quantify the rate of convergence in Selberg's central limit theorem for $\log|ζ(1/2+it)|$ based on the method of proof given by Radziwill and Soundararajan. We achieve the same rate of convergence of $(\log\log\log T)^2/\sqrt{\log\log T}$ as Selberg in the Kolmogorov distance by using the Dudley distance instead. We also prove the theorem for the multivariate case given by Bourgade with the same rate of convergence as in the single variable case. △ Less

Submitted 5 March, 2024; v1 submitted 2 December, 2022; originally announced December 2022.

Comments: Some corrections and additions, now 23 pages

MSC Class: 11M06; 60F05

Shotgun assembly of random graphs

Authors: Tom Johnston, Gal Kronenberg, Alexander Roberts, Alex Scott

Abstract: In the graph shotgun assembly problem, we are given the balls of radius $r$ around each vertex of a graph and asked to reconstruct the graph. We study the shotgun assembly of the Erdős-Rényi random graph $\mathcal G(n,p)$ from a wide range of values of $r$. We determine the threshold for reconstructibility for each $r\geq 3$, extending and improving substantially on results of Mossel and Ross for… ▽ More In the graph shotgun assembly problem, we are given the balls of radius $r$ around each vertex of a graph and asked to reconstruct the graph. We study the shotgun assembly of the Erdős-Rényi random graph $\mathcal G(n,p)$ from a wide range of values of $r$. We determine the threshold for reconstructibility for each $r\geq 3$, extending and improving substantially on results of Mossel and Ross for $r=3$. For $r=2$, we give upper and lower bounds that improve on results of Gaudio and Mossel by polynomial factors. We also give a sharpening of a result of Huang and Tikhomirov for $r=1$. △ Less

Submitted 3 June, 2023; v1 submitted 25 November, 2022; originally announced November 2022.

Comments: 36 pages, 3 figures