Showing 51–100 of 497 results for author: Roberts, A

VeLO: Training Versatile Learned Optimizers by Scaling Up

Authors: Luke Metz, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, Jascha Sohl-Dickstein

Abstract: While deep learning models have replaced hand-designed features across many domains, these models are still trained with hand-designed optimizers. In this work, we leverage the same scaling approach behind the success of deep learning to learn versatile optimizers. We train an optimizer for deep learning which is itself a small neural network that ingests gradients and outputs parameter updates. M… ▽ More While deep learning models have replaced hand-designed features across many domains, these models are still trained with hand-designed optimizers. In this work, we leverage the same scaling approach behind the success of deep learning to learn versatile optimizers. We train an optimizer for deep learning which is itself a small neural network that ingests gradients and outputs parameter updates. Meta-trained with approximately four thousand TPU-months of compute on a wide variety of optimization tasks, our optimizer not only exhibits compelling performance, but optimizes in interesting and unexpected ways. It requires no hyperparameter tuning, instead automatically adapting to the specifics of the problem being optimized. We open source our learned optimizer, meta-training code, the associated train and test data, and an extensive optimizer benchmark suite with baselines at velo-code.github.io. △ Less

Submitted 17 November, 2022; originally announced November 2022.

Large Language Models Struggle to Learn Long-Tail Knowledge

Authors: Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, Colin Raffel

Abstract: The Internet contains a wealth of knowledge -- from the birthdays of historical figures to tutorials on how to code -- all of which may be learned by language models. However, while certain pieces of information are ubiquitous on the web, others appear extremely rarely. In this paper, we study the relationship between the knowledge memorized by large language models and the information in pre-trai… ▽ More The Internet contains a wealth of knowledge -- from the birthdays of historical figures to tutorials on how to code -- all of which may be learned by language models. However, while certain pieces of information are ubiquitous on the web, others appear extremely rarely. In this paper, we study the relationship between the knowledge memorized by large language models and the information in pre-training datasets scraped from the web. In particular, we show that a language model's ability to answer a fact-based question relates to how many documents associated with that question were seen during pre-training. We identify these relevant documents by entity linking pre-training datasets and counting documents that contain the same entities as a given question-answer pair. Our results demonstrate strong correlational and causal relationships between accuracy and relevant document count for numerous question answering datasets (e.g., TriviaQA), pre-training corpora (e.g., ROOTS), and model sizes (e.g., 176B parameters). Moreover, while larger models are better at learning long-tail knowledge, we estimate that today's models must be scaled by many orders of magnitude to reach competitive QA performance on questions with little support in the pre-training data. Finally, we show that retrieval-augmentation can reduce the dependence on relevant pre-training information, presenting a promising approach for capturing the long-tail. △ Less

Submitted 27 July, 2023; v1 submitted 15 November, 2022; originally announced November 2022.

Comments: ICML 2023 Camera Ready Version

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Authors: BigScience Workshop, :, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major , et al. (369 additional authors not shown)

Abstract: Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access… ▽ More Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License. △ Less

Submitted 27 June, 2023; v1 submitted 9 November, 2022; originally announced November 2022.

Performance of the CMS High Granularity Calorimeter prototype to charged pion beams of 20$-$300 GeV/c

Authors: B. Acar, G. Adamov, C. Adloff, S. Afanasiev, N. Akchurin, B. Akgün, M. Alhusseini, J. Alison, J. P. Figueiredo de sa Sousa de Almeida, P. G. Dias de Almeida, A. Alpana, M. Alyari, I. Andreev, U. Aras, P. Aspell, I. O. Atakisi, O. Bach, A. Baden, G. Bakas, A. Bakshi, S. Banerjee, P. DeBarbaro, P. Bargassa, D. Barney, F. Beaudette , et al. (435 additional authors not shown)

Abstract: The upgrade of the CMS experiment for the high luminosity operation of the LHC comprises the replacement of the current endcap calorimeter by a high granularity sampling calorimeter (HGCAL). The electromagnetic section of the HGCAL is based on silicon sensors interspersed between lead and copper (or copper tungsten) absorbers. The hadronic section uses layers of stainless steel as an absorbing med… ▽ More The upgrade of the CMS experiment for the high luminosity operation of the LHC comprises the replacement of the current endcap calorimeter by a high granularity sampling calorimeter (HGCAL). The electromagnetic section of the HGCAL is based on silicon sensors interspersed between lead and copper (or copper tungsten) absorbers. The hadronic section uses layers of stainless steel as an absorbing medium and silicon sensors as an active medium in the regions of high radiation exposure, and scintillator tiles directly readout by silicon photomultipliers in the remaining regions. As part of the development of the detector and its readout electronic components, a section of a silicon-based HGCAL prototype detector along with a section of the CALICE AHCAL prototype was exposed to muons, electrons and charged pions in beam test experiments at the H2 beamline at the CERN SPS in October 2018. The AHCAL uses the same technology as foreseen for the HGCAL but with much finer longitudinal segmentation. The performance of the calorimeters in terms of energy response and resolution, longitudinal and transverse shower profiles is studied using negatively charged pions, and is compared to GEANT4 predictions. This is the first report summarizing results of hadronic showers measured by the HGCAL prototype using beam test data. △ Less

Submitted 27 May, 2023; v1 submitted 9 November, 2022; originally announced November 2022.

Comments: Accepted for publication by JINST

Crosslingual Generalization through Multitask Finetuning

Authors: Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, Colin Raffel

Abstract: Multitask prompted finetuning (MTF) has been shown to help large language models generalize to new tasks in a zero-shot setting, but so far explorations of MTF have focused on English data and models. We apply MTF to the pretrained multilingual BLOOM and mT5 model families to produce finetuned variants called BLOOMZ and mT0. We find finetuning large multilingual language models on English tasks wi… ▽ More Multitask prompted finetuning (MTF) has been shown to help large language models generalize to new tasks in a zero-shot setting, but so far explorations of MTF have focused on English data and models. We apply MTF to the pretrained multilingual BLOOM and mT5 model families to produce finetuned variants called BLOOMZ and mT0. We find finetuning large multilingual language models on English tasks with English prompts allows for task generalization to non-English languages that appear only in the pretraining corpus. Finetuning on multilingual tasks with English prompts further improves performance on English and non-English tasks leading to various state-of-the-art zero-shot results. We also investigate finetuning on multilingual tasks with prompts that have been machine-translated from English to match the language of each dataset. We find training on these machine-translated prompts leads to better performance on human-written prompts in the respective languages. Surprisingly, we find models are capable of zero-shot generalization to tasks in languages they have never intentionally seen. We conjecture that the models are learning higher-level capabilities that are both task- and language-agnostic. In addition, we introduce xP3, a composite of supervised datasets in 46 languages with English and machine-translated prompts. Our code, datasets and models are freely available at https://github.com/bigscience-workshop/xmtf. △ Less

Submitted 29 May, 2023; v1 submitted 3 November, 2022; originally announced November 2022.

Comments: 9 main pages (119 with appendix), 16 figures and 11 tables

A Solvable Model of Neural Scaling Laws

Authors: Alexander Maloney, Daniel A. Roberts, James Sully

Abstract: Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in either parameters or dataset size until bottlenecked by the other resource. To understand this better, we first identify the necessary properties allowing such sca… ▽ More Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in either parameters or dataset size until bottlenecked by the other resource. To understand this better, we first identify the necessary properties allowing such scaling laws to arise and then propose a statistical model -- a joint generative data model and random feature model -- that captures this neural scaling phenomenology. By solving this model in the dual limit of large training set size and large number of parameters, we gain insight into (i) the statistical structure of datasets and tasks that lead to scaling laws, (ii) the way nonlinear feature maps, such as those provided by neural networks, enable scaling laws when trained on these datasets, (iii) the optimality of the equiparameterization scaling of training sets and parameters, and (iv) whether such scaling laws can break down and how they behave when they do. Key findings are the manner in which the power laws that occur in the statistics of natural datasets are extended by nonlinear random feature maps and then translated into power-law scalings of the test loss and how the finite extent of the data's spectral power law causes the model's performance to plateau. △ Less

Submitted 30 October, 2022; originally announced October 2022.

Comments: 73 + 23 pages, 14 + 5 figures

Report number: MIT-CTP/5463

Mechanistic forecasts of species responses to climate change: the promise of biophysical ecology

Authors: Natalie J. Briscoe, Shane D. Morris, Paul D. Mathewson, Lauren B. Buckley, Marko Jusup, Ofir Levy, Ilya M. D. Maclean, Sylvain Pincebourde, Eric A. Riddell, Jessica A. Roberts, Rafael Schouten, Michael W. Sears, Michael R. Kearney

Abstract: A challenge in global change biology is to predict how species will respond to future environmental change and to manage these responses. To make such predictions and management actions robust to novel futures, we need to accurately characterize how organisms experience their environments and the biological mechanisms by which they respond. All organisms are thermodynamically connected to their en… ▽ More A challenge in global change biology is to predict how species will respond to future environmental change and to manage these responses. To make such predictions and management actions robust to novel futures, we need to accurately characterize how organisms experience their environments and the biological mechanisms by which they respond. All organisms are thermodynamically connected to their environments through the exchange of heat and water at fine spatial and temporal scales and this exchange can be captured with biophysical models. Although mechanistic models based on biophysical ecology have a long history of development and application, their use in global change biology remains limited despite their enormous promise and increasingly accessible software. We contend that greater understanding and training in the theory and methods of biophysical models is vital to expand their application. Our review shows how biophysical models can be implemented to understand and predict climate change impacts on species' behavior, phenology, survival, distribution, and abundance. We illustrate the types of outputs that can be generated, and the data inputs required for different implementations. Examples range from simple calculations of body temperature to more complex analyses of species' distribution limits based on projected energy and water balances, accounting for behavior and phenology. We outline challenges that currently limit the widespread application of biophysical models. We discuss progress and future developments that could allow these models to be applied to many species across large spatial extents and timeframes. We highlight how biophysical models are uniquely suited to solve global change biology problems that involve predicting and interpreting responses to environmental variability and extremes, multiple or shifting constraints, and novel abiotic or biotic environments. △ Less

Submitted 29 October, 2022; originally announced October 2022.

Two novel families of multiscale staggered patch schemes efficiently simulate large-scale, weakly damped, linear waves

Authors: J. Divahar, A. J. Roberts, Trent W. Mattner, J. E. Bunder, Ioannis G. Kevrekidis

Abstract: Many multiscale wave systems exhibit macroscale emergent behaviour, for example, the fluid dynamics of floods and tsunamis. Resolving a large range of spatial scales typically requires a prohibitively high computational cost. The small dissipation in wave systems poses a significant challenge to further developing multiscale modelling methods in multiple dimensions. This article develops and evalu… ▽ More Many multiscale wave systems exhibit macroscale emergent behaviour, for example, the fluid dynamics of floods and tsunamis. Resolving a large range of spatial scales typically requires a prohibitively high computational cost. The small dissipation in wave systems poses a significant challenge to further developing multiscale modelling methods in multiple dimensions. This article develops and evaluates two families of equation-free multiscale methods on novel 2D staggered patch schemes, and demonstrates the power and utility of these multiscale schemes for weakly damped linear waves. A detailed study of sensitivity to numerical roundoff errors establishes the robustness of developed staggered patch schemes. Comprehensive eigenvalue analysis over a wide range of parameters establishes the stability, accuracy, and consistency of the multiscale schemes. Analysis of the computational complexity shows that the measured compute times of the multiscale schemes may be 10^5 times smaller than the compute time for the corresponding full-domain computation. This work provides the essential foundation for efficient large-scale simulation of challenging nonlinear multiscale waves. △ Less

Submitted 27 October, 2022; originally announced October 2022.

Comments: 35 pages, 12 figures, and 6 tables

Scaling Instruction-Finetuned Language Models

Authors: Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang , et al. (10 additional authors not shown)

Abstract: Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects d… ▽ More Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models. △ Less

Submitted 6 December, 2022; v1 submitted 20 October, 2022; originally announced October 2022.

Comments: Public checkpoints: https://huggingface.co/docs/transformers/model_doc/flan-t5

The Future of High Energy Physics Software and Computing

Authors: V. Daniel Elvira, Steven Gottlieb, Oliver Gutsche, Benjamin Nachman, S. Bailey, W. Bhimji, P. Boyle, G. Cerati, M. Carrasco Kind, K. Cranmer, G. Davies, V. D. Elvira, R. Gardner, K. Heitmann, M. Hildreth, W. Hopkins, T. Humble, M. Lin, P. Onyisi, J. Qiang, K. Pedro, G. Perdue, A. Roberts, M. Savage, P. Shanahan , et al. (3 additional authors not shown)

Abstract: Software and Computing (S&C) are essential to all High Energy Physics (HEP) experiments and many theoretical studies. The size and complexity of S&C are now commensurate with that of experimental instruments, playing a critical role in experimental design, data acquisition/instrumental control, reconstruction, and analysis. Furthermore, S&C often plays a leading role in driving the precision of th… ▽ More Software and Computing (S&C) are essential to all High Energy Physics (HEP) experiments and many theoretical studies. The size and complexity of S&C are now commensurate with that of experimental instruments, playing a critical role in experimental design, data acquisition/instrumental control, reconstruction, and analysis. Furthermore, S&C often plays a leading role in driving the precision of theoretical calculations and simulations. Within this central role in HEP, S&C has been immensely successful over the last decade. This report looks forward to the next decade and beyond, in the context of the 2021 Particle Physics Community Planning Exercise ("Snowmass") organized by the Division of Particles and Fields (DPF) of the American Physical Society. △ Less

Submitted 8 November, 2022; v1 submitted 11 October, 2022; originally announced October 2022.

Comments: Computational Frontier Report Contribution to Snowmass 2021; 41 pages, 1 figure. v2: missing ref and added missing topical group conveners. v3: fixed typos

CompF5: End User Analysis Topical Group Report

Authors: Gavin S. Davies, Peter Onyisi, Amy Roberts

Abstract: This report summarizes the work of the Computational Frontier topical group on end user analysis for Snowmass 2021. End User Analysis refers to the extraction of physics results from reconstructed and simulated experimental data. High energy physics experiments produce systems that perform common reconstruction, calibration, and simulation tasks, resulting in shared data samples. These detailed da… ▽ More This report summarizes the work of the Computational Frontier topical group on end user analysis for Snowmass 2021. End User Analysis refers to the extraction of physics results from reconstructed and simulated experimental data. High energy physics experiments produce systems that perform common reconstruction, calibration, and simulation tasks, resulting in shared data samples. These detailed data samples are then reduced to create a range of analysis samples, often optimized for a particular physics topic by a trigger or data object selection. End users (analyzers) then analyze those samples to produce physics results. Community discussions converged on categorizing the end user analysis ensemble into Analysis Ecosystems, Analysis Models, Dataset Bookkeeping and Formats, Collaborative Software, and Training. We present findings and recommendations based on these key areas that impact end user analysis. △ Less

Submitted 29 September, 2022; originally announced September 2022.

Comments: Report of the Computational Frontier Topical Group on End User Analysis for Snowmass 2021

Thin film notch filters as platforms for biological image processing

Authors: Shaban B. Sulejman, Niken Priscilla, Lukas Wesemann, Wendy S. L. Lee, Jieqiong Lou, Elizabeth Hinde, Timothy J. Davis, Ann Roberts

Abstract: Many image processing operations involve the modification of the spatial frequency content of images. Here we demonstrate object-plane spatial frequency filtering utilizing the angular sensitivity of a commercial spectral bandstop filter. This approach to all-optical image processing is shown to generate real-time pseudo-3D images of transparent biological and other samples, such as human cervical… ▽ More Many image processing operations involve the modification of the spatial frequency content of images. Here we demonstrate object-plane spatial frequency filtering utilizing the angular sensitivity of a commercial spectral bandstop filter. This approach to all-optical image processing is shown to generate real-time pseudo-3D images of transparent biological and other samples, such as human cervical cancer cells. This work demonstrates the potential of non-local, non-interferometric approaches to image processing for uses in label-free biological cell imaging and dynamical monitoring. △ Less

Submitted 20 September, 2022; originally announced September 2022.

Comments: manuscript 14 pages, 5 figures, supplementary material 7 pages, 4 supplementary figures

Embed to rigorously and accurately homogenise quasi-periodic multi-scale heterogeneous PDEs, with computer algebra

Authors: A. J. Roberts

Abstract: For microscale heterogeneous PDEs, this article further develops novel theory and methodology for their macroscale mathematical/asymptotic homogenization. This article specifically encompasses the case of quasi-periodic heterogeneity with finite scale separation: no scale separation limit is required. Dynamical systems theory frames the homogenization as a slow manifold of the ensemble of all phas… ▽ More For microscale heterogeneous PDEs, this article further develops novel theory and methodology for their macroscale mathematical/asymptotic homogenization. This article specifically encompasses the case of quasi-periodic heterogeneity with finite scale separation: no scale separation limit is required. Dynamical systems theory frames the homogenization as a slow manifold of the ensemble of all phase-shifts of the heterogeneity. Depending upon any perceived scale separation within the quasi-periodic heterogeneity, the homogenization may be done in either one step, or two sequential steps: the results are equivalent. The theory not only assures us of the existence and emergence of the homogenization, it also provides a practical systematic method to construct the homogenization to any specified order. For a class of heterogeneities, we show that the macroscale homogenization is potentially valid down to lengths which are just twice that of the microscale heterogeneity! This methodology provides a new rigorous and flexible approach to homogenization that potentially also provides correct initial and boundary conditions, treatment of forcing and control, and analysis of uncertainty. △ Less

Submitted 6 September, 2022; originally announced September 2022.

Sensitivity projections for a dual-phase argon TPC optimized for light dark matter searches through the ionization channel

Authors: P. Agnes, I. Ahmad, S. Albergo, I. F. M. Albuquerque, T. Alexander, A. K. Alton, P. Amaudruz, M. Atzori Corona, D. J. Auty, M. Ave, I. Ch. Avetisov, R. I. Avetisov, O. Azzolini, H. O. Back, Z. Balmforth, V. Barbarian, A. Barrado Olmedo, P. Barrillon, A. Basco, G. Batignani, E. Berzin, A. Bondar, W. M. Bonivento, E. Borisova, B. Bottino , et al. (274 additional authors not shown)

Abstract: Dark matter lighter than 10 GeV/c$^2$ encompasses a promising range of candidates. A conceptual design for a new detector, DarkSide-LowMass, is presented, based on the DarkSide-50 detector and progress toward DarkSide-20k, optimized for a low-threshold electron-counting measurement. Sensitivity to light dark matter is explored for various potential energy thresholds and background rates. These stu… ▽ More Dark matter lighter than 10 GeV/c$^2$ encompasses a promising range of candidates. A conceptual design for a new detector, DarkSide-LowMass, is presented, based on the DarkSide-50 detector and progress toward DarkSide-20k, optimized for a low-threshold electron-counting measurement. Sensitivity to light dark matter is explored for various potential energy thresholds and background rates. These studies show that DarkSide-LowMass can achieve sensitivity to light dark matter down to the solar neutrino floor for GeV-scale masses and significant sensitivity down to 10 MeV/c$^2$ considering the Migdal effect or interactions with electrons. Requirements for optimizing the detector's sensitivity are explored, as are potential sensitivity gains from modeling and mitigating spurious electron backgrounds that may dominate the signal at the lowest energies. △ Less

Submitted 20 June, 2023; v1 submitted 2 September, 2022; originally announced September 2022.

Journal ref: Phys. Rev. D 107, 112006 (2023)

Staggered grids for multidimensional multiscale modelling

Authors: J. Divahar, A. J. Roberts, Trent W. Mattner, J. E. Bunder, Ioannis G. Kevrekidis

Abstract: Numerical schemes for wave-like systems with small dissipation are often inaccurate and unstable due to truncation errors and numerical roundoff errors. Hence, numerical simulations of wave-like systems lacking proper handling of these numerical issues often fail to represent the physical characteristics of wave phenomena. This challenge gets even more intricate for multiscale modelling, especiall… ▽ More Numerical schemes for wave-like systems with small dissipation are often inaccurate and unstable due to truncation errors and numerical roundoff errors. Hence, numerical simulations of wave-like systems lacking proper handling of these numerical issues often fail to represent the physical characteristics of wave phenomena. This challenge gets even more intricate for multiscale modelling, especially in multiple dimensions. When using the usual collocated grid, about two-thirds of the resolved wave modes are incorrect with significant dispersion. But, numerical schemes on staggered grids (with alternating variable arrangement) are significantly less dispersive and preserve much of the wave characteristics. Also, the group velocity of the energy propagation in the numerical waves on a staggered grid is in the correct direction, in contrast to the collocated grid. For high accuracy and to preserve much of the wave characteristics, this article extends the concept of staggered grids in full-domain modelling to multidimensional multiscale modelling. Specifically, this article develops 120 multiscale staggered grids and demonstrates their stability, accuracy, and wave-preserving characteristic for equation-free multiscale modelling of weakly damped linear waves. But most characteristics of the developed multiscale staggered grids must also hold in general for multiscale modelling of many complex spatio-temporal physical phenomena such as the general computational fluid dynamics. △ Less

Submitted 25 July, 2022; originally announced July 2022.

Comments: 42 pages, 13 figures, and 1 table

DDKtor: Automatic Diadochokinetic Speech Analysis

Authors: Yael Segal, Kasia Hitczenko, Matthew Goldrick, Adam Buchwald, Angela Roberts, Joseph Keshet

Abstract: Diadochokinetic speech tasks (DDK), in which participants repeatedly produce syllables, are commonly used as part of the assessment of speech motor impairments. These studies rely on manual analyses that are time-intensive, subjective, and provide only a coarse-grained picture of speech. This paper presents two deep neural network models that automatically segment consonants and vowels from unanno… ▽ More Diadochokinetic speech tasks (DDK), in which participants repeatedly produce syllables, are commonly used as part of the assessment of speech motor impairments. These studies rely on manual analyses that are time-intensive, subjective, and provide only a coarse-grained picture of speech. This paper presents two deep neural network models that automatically segment consonants and vowels from unannotated, untranscribed speech. Both models work on the raw waveform and use convolutional layers for feature extraction. The first model is based on an LSTM classifier followed by fully connected layers, while the second model adds more convolutional layers followed by fully connected layers. These segmentations predicted by the models are used to obtain measures of speech rate and sound duration. Results on a young healthy individuals dataset show that our LSTM model outperforms the current state-of-the-art systems and performs comparably to trained human annotators. Moreover, the LSTM model also presents comparable results to trained human annotators when evaluated on unseen older individuals with Parkinson's Disease dataset. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Comments: Accepted to Interspeech 2022

Intrinsic Fano factor of nuclear recoils for dark matter searches

Authors: M. Matheny, A. Roberts, A. Srinivasan, A. N. Villano

Abstract: Nuclear recoils in germanium and silicon are shown to have much larger variance in electron-hole production than their electron-recoil counterparts for recoil energies between 10 and 200\,keV. This effect--owing primarily to deviations in the amount of energy given to the crystal lattice in response to a nuclear recoil of a given energy--has been predicted by the Lindhard model. We parameterize th… ▽ More Nuclear recoils in germanium and silicon are shown to have much larger variance in electron-hole production than their electron-recoil counterparts for recoil energies between 10 and 200\,keV. This effect--owing primarily to deviations in the amount of energy given to the crystal lattice in response to a nuclear recoil of a given energy--has been predicted by the Lindhard model. We parameterize the variance in terms of an intrinsic nuclear recoil Fano factor which is 24.3$\pm$0.2 and 26$\pm$8 at around 25\,keV for silicon and germanium respectively. The variance has important effects on the expected signal shapes for experiments utilizing low-energy nuclear recoils such as direct dark matter searches and coherent neutrino-nucleus scattering measurements. △ Less

Submitted 12 December, 2022; v1 submitted 27 June, 2022; originally announced June 2022.

Comments: 9 pages, 6 figures

Journal ref: Phys. Rev. D 106, 123009 (2022)

Multi-instrument Music Synthesis with Spectrogram Diffusion

Authors: Curtis Hawthorne, Ian Simon, Adam Roberts, Neil Zeghidour, Josh Gardner, Ethan Manilow, Jesse Engel

Abstract: An ideal music synthesizer should be both interactive and expressive, generating high-fidelity audio in realtime for arbitrary combinations of instruments and notes. Recent neural synthesizers have exhibited a tradeoff between domain-specific models that offer detailed control of only specific instruments, or raw waveform models that can train on any music but with minimal control and slow generat… ▽ More An ideal music synthesizer should be both interactive and expressive, generating high-fidelity audio in realtime for arbitrary combinations of instruments and notes. Recent neural synthesizers have exhibited a tradeoff between domain-specific models that offer detailed control of only specific instruments, or raw waveform models that can train on any music but with minimal control and slow generation. In this work, we focus on a middle ground of neural synthesizers that can generate audio from MIDI sequences with arbitrary combinations of instruments in realtime. This enables training on a wide range of transcription datasets with a single model, which in turn offers note-level control of composition and instrumentation across a wide range of instruments. We use a simple two-stage process: MIDI to spectrograms with an encoder-decoder Transformer, then spectrograms to audio with a generative adversarial network (GAN) spectrogram inverter. We compare training the decoder as an autoregressive model and as a Denoising Diffusion Probabilistic Model (DDPM) and find that the DDPM approach is superior both qualitatively and as measured by audio reconstruction and Fréchet distance metrics. Given the interactivity and generality of this approach, we find this to be a promising first step towards interactive and expressive neural synthesis for arbitrary combinations of instruments and notes. △ Less

Submitted 12 December, 2022; v1 submitted 10 June, 2022; originally announced June 2022.

A Pseudo-Two-Dimensional (P2D) Model for FeS2 Conversion Cathode Batteries

Authors: Jeffrey S. Horner, Grace Whang, Igor V. Kolesnichenko, Timothy N. Lambert, Bruce S. Dunn, Scott A. Roberts

Abstract: Conversion cathode materials are gaining interest for secondary batteries due to their high theoretical energy and power density. However, practical application as a secondary battery material is currently limited by practical issues such as poor cyclability. To better understand these materials, we have developed a pseudo-two-dimensional model for conversion cathodes. We apply this model to FeS2… ▽ More Conversion cathode materials are gaining interest for secondary batteries due to their high theoretical energy and power density. However, practical application as a secondary battery material is currently limited by practical issues such as poor cyclability. To better understand these materials, we have developed a pseudo-two-dimensional model for conversion cathodes. We apply this model to FeS2 - a material that undergoes intercalation followed by conversion during discharge. The model is derived from the half-cell Doyle-Fuller-Newman model with additional loss terms added to reflect the converted shell resistance as the reaction progresses. We also account for polydisperse active material particles by incorporating a variable active surface area and effective particle radius. Using the model, we show that the leading loss mechanisms for FeS2 are associated with solid-state diffusion and electrical transport limitations through the converted shell material. The polydisperse simulations are also compared to a monodisperse system, and we show that polydispersity has very little effect on the intercalation behavior yet leads to capacity loss during the conversion reaction. We provide the code as an open-source Python Battery Mathematical Modelling (PyBaMM) model that can be used to identify performance limitations for other conversion cathode materials. △ Less

Submitted 16 July, 2022; v1 submitted 1 June, 2022; originally announced June 2022.

Journal ref: Journal of Power Sources Volume 544, 1 October 2022, 231893

Effective Field Theory Analysis of CDMSlite Run 2 Data

Authors: SuperCDMS Collaboration, M. F. Albakry, I. Alkhatib, D. W. P. Amaral, T. Aralis, T. Aramaki, I. J. Arnquist, I. Ataee Langroudy, E. Azadbakht, S. Banik, C. Bathurst, D. A. Bauer, L. V. S. Bezerra, R. Bhattacharyya, P. L. Brink, R. Bunker, B. Cabrera, R. Calkins, R. A. Cameron, C. Cartaro, D. G. Cerdeño, Y. -Y. Chang, M. Chaudhuri, R. Chen, N. Chott , et al. (105 additional authors not shown)

Abstract: CDMSlite Run 2 was a search for weakly interacting massive particles (WIMPs) with a cryogenic 600 g Ge detector operated in a high-voltage mode to optimize sensitivity to WIMPs of relatively low mass from 2 - 20 GeV/$c^2$. In this article, we present an effective field theory (EFT) analysis of the CDMSlite Run 2 data using an extended energy range and a comprehensive treatment of the expected back… ▽ More CDMSlite Run 2 was a search for weakly interacting massive particles (WIMPs) with a cryogenic 600 g Ge detector operated in a high-voltage mode to optimize sensitivity to WIMPs of relatively low mass from 2 - 20 GeV/$c^2$. In this article, we present an effective field theory (EFT) analysis of the CDMSlite Run 2 data using an extended energy range and a comprehensive treatment of the expected background. A binned likelihood Bayesian analysis was performed on the recoil energy data, taking into account the parameters of the EFT interactions and optimizing the data selection with respect to the dominant background components. Energy regions within 5$σ$ of known activation peaks were removed from the analysis. The Bayesian evidences resulting from the different operator hypotheses show that the CDMSlite Run 2 data are consistent with the background-only models and do not allow for a signal interpretation assuming any additional EFT interaction. Consequently, upper limits on the WIMP mass and coupling-coefficient amplitudes and phases are presented for each EFT operator. These limits improve previous CDMSlite Run 2 bounds for WIMP masses above 5 GeV/$c^2$. △ Less

Submitted 23 May, 2022; originally announced May 2022.

Comments: 16 pages, 8 figures

Investigating the sources of low-energy events in a SuperCDMS-HVeV detector

Authors: SuperCDMS Collaboration, M. F. Albakry, I. Alkhatib, D. W. P. Amaral, T. Aralis, T. Aramaki, I. J. Arnquist, I. Ataee Langroudy, E. Azadbakht, S. Banik, C. Bathurst, D. A. Bauer, R. Bhattacharyya, P. L. Brink, R. Bunker, B. Cabrera, R. Calkins, R. A. Cameron, C. Cartaro, D. G. Cerdeño, Y. -Y. Chang, M. Chaudhuri, R. Chen, N. Chott, J. Cooley , et al. (104 additional authors not shown)

Abstract: Recent experiments searching for sub-GeV/$c^2$ dark matter have observed event excesses close to their respective energy thresholds. Although specific to the individual technologies, the measured excess event rates have been consistently reported at or below event energies of a few-hundred eV, or with charges of a few electron-hole pairs. In the present work, we operated a 1-gram silicon SuperCDMS… ▽ More Recent experiments searching for sub-GeV/$c^2$ dark matter have observed event excesses close to their respective energy thresholds. Although specific to the individual technologies, the measured excess event rates have been consistently reported at or below event energies of a few-hundred eV, or with charges of a few electron-hole pairs. In the present work, we operated a 1-gram silicon SuperCDMS-HVeV detector at three voltages across the crystal (0 V, 60 V and 100 V). The 0 V data show an excess of events in the tens of eV region. Despite this event excess, we demonstrate the ability to set a competitive exclusion limit on the spin-independent dark matter--nucleon elastic scattering cross section for dark matter masses of $\mathcal{O}(100)$ MeV/$c^2$, enabled by operation of the detector at 0 V potential and achievement of a very low $\mathcal{O}(10)$ eV threshold for nuclear recoils. Comparing the data acquired at 0 V, 60 V and 100 V potentials across the crystal, we investigated possible sources of the unexpected events observed at low energy. The data indicate that the dominant contribution to the excess is consistent with a hypothesized luminescence from the printed circuit boards used in the detector holder. △ Less

Submitted 11 October, 2022; v1 submitted 17 April, 2022; originally announced April 2022.

Learning high-order spatial discretisations of PDEs with symmetry-preserving iterative algorithms

Authors: J. E. Bunder, A. J. Roberts

Abstract: Common techniques for the spatial discretisation of PDEs on a macroscale grid include finite difference, finite elements and finite volume methods. Such methods typically impose assumed microscale structures on the subgrid fields, so without further tailored analysis are not suitable for systems with subgrid-scale heterogeneity or nonlinearities. We provide a new algebraic route to systematically… ▽ More Common techniques for the spatial discretisation of PDEs on a macroscale grid include finite difference, finite elements and finite volume methods. Such methods typically impose assumed microscale structures on the subgrid fields, so without further tailored analysis are not suitable for systems with subgrid-scale heterogeneity or nonlinearities. We provide a new algebraic route to systematically approximate, in principle exactly, the macroscale closure of the spatially-discrete dynamics of a general class of heterogeneous non-autonomous reaction-advection-diffusion PDEs. This holistic discretisation approach, developed through rigorous theory and verified with computer algebra, systematically constructs discrete macroscale models through physics informed by the PDE out-of-equilibrium dynamics, thus relaxing many assumptions regarding the subgrid structure. The construction is analogous to recent gray-box machine learning techniques in that predictions are directed by iterative layers (as in neural networks), but informed by the subgrid physics (or 'data') as expressed in the PDEs. A major development of the holistic methodology, presented herein, is novel inter-element coupling between subgrid fields which preserve self-adjointness of the PDE after macroscale discretisation, thereby maintaining the spectral structure of the original system. This holistic methodology also encompasses homogenisation of microscale heterogeneous systems, as shown here with the canonical examples of heterogeneous 1D waves and diffusion. △ Less

Submitted 13 April, 2022; originally announced April 2022.

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?

Authors: Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel

Abstract: Large pretrained Transformer language models have been shown to exhibit zero-shot generalization, i.e. they can perform a wide variety of tasks that they were not explicitly trained on. However, the architectures and pretraining objectives used across state-of-the-art models differ significantly, and there has been limited systematic comparison of these factors. In this work, we present a large-sc… ▽ More Large pretrained Transformer language models have been shown to exhibit zero-shot generalization, i.e. they can perform a wide variety of tasks that they were not explicitly trained on. However, the architectures and pretraining objectives used across state-of-the-art models differ significantly, and there has been limited systematic comparison of these factors. In this work, we present a large-scale evaluation of modeling choices and their impact on zero-shot generalization. In particular, we focus on text-to-text models and experiment with three model architectures (causal/non-causal decoder-only and encoder-decoder), trained with two different pretraining objectives (autoregressive and masked language modeling), and evaluated with and without multitask prompted finetuning. We train models with over 5 billion parameters for more than 170 billion tokens, thereby increasing the likelihood that our conclusions will transfer to even larger scales. Our experiments show that causal decoder-only models trained on an autoregressive language modeling objective exhibit the strongest zero-shot generalization after purely unsupervised pretraining. However, models with non-causal visibility on their input trained with a masked language modeling objective followed by multitask finetuning perform the best among our experiments. We therefore consider the adaptation of pretrained models across architectures and objectives. We find that pretrained non-causal decoder models can be adapted into performant generative causal decoder models, using autoregressive language modeling as a downstream task. Furthermore, we find that pretrained causal decoder models can be efficiently adapted into non-causal decoder models, ultimately achieving competitive performance after multitask finetuning. Code and checkpoints are available at https://github.com/bigscience-workshop/architecture-objective. △ Less

Submitted 12 April, 2022; originally announced April 2022.

PaLM: Scaling Language Modeling with Pathways

Authors: Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin , et al. (42 additional authors not shown)

Abstract: Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Tran… ▽ More Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM. We trained PaLM on 6144 TPU v4 chips using Pathways, a new ML system which enables highly efficient training across multiple TPU Pods. We demonstrate continued benefits of scaling by achieving state-of-the-art few-shot learning results on hundreds of language understanding and generation benchmarks. On a number of these tasks, PaLM 540B achieves breakthrough performance, outperforming the finetuned state-of-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark. A significant number of BIG-bench tasks showed discontinuous improvements from model scale, meaning that performance steeply increased as we scaled to our largest model. PaLM also has strong capabilities in multilingual tasks and source code generation, which we demonstrate on a wide array of benchmarks. We additionally provide a comprehensive analysis on bias and toxicity, and study the extent of training data memorization with respect to model scale. Finally, we discuss the ethical considerations related to large language models and discuss potential mitigation strategies. △ Less

Submitted 5 October, 2022; v1 submitted 5 April, 2022; originally announced April 2022.

Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$

Authors: Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen , et al. (18 additional authors not shown)

Abstract: Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves. Scaling can be complicated due to various factors including the need to distribute computation on supercomputer clusters (e.g., TPUs), prevent bottlenecks when infeeding data, and ensure reproducible results. In this work, we presen… ▽ More Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves. Scaling can be complicated due to various factors including the need to distribute computation on supercomputer clusters (e.g., TPUs), prevent bottlenecks when infeeding data, and ensure reproducible results. In this work, we present two software libraries that ease these issues: $\texttt{t5x}$ simplifies the process of building and training large language models at scale while maintaining ease of use, and $\texttt{seqio}$ provides a task-based API for simple creation of fast and reproducible training data and evaluation pipelines. These open-source libraries have been used to train models with hundreds of billions of parameters on datasets with multiple terabytes of training data. Along with the libraries, we release configurations and instructions for T5-like encoder-decoder models as well as GPT-like decoder-only architectures. $\texttt{t5x}$ and $\texttt{seqio}$ are open source and available at https://github.com/google-research/t5x and https://github.com/google/seqio, respectively. △ Less

Submitted 31 March, 2022; originally announced March 2022.

A Strategy for Low-Mass Dark Matter Searches with Cryogenic Detectors in the SuperCDMS SNOLAB Facility

Authors: SuperCDMS Collaboration, M. F. Albakry, I. Alkhatib, D. W. P. Amaral, T. Aralis, T. Aramaki, I. J. Arnquist, I. Ataee Langroudy, E. Azadbakht, S. Banik, C. Bathurst, D. A. Bauer, R. Bhattacharyya, P. L. Brink, R. Bunker, B. Cabrera, R. Calkins, R. A. Cameron, C. Cartaro, D. G. Cerdeno, Y. -Y. Chang, M. Chaudhuri, R. Chen, N. Chott, J. Cooley , et al. (103 additional authors not shown)

Abstract: The SuperCDMS Collaboration is currently building SuperCDMS SNOLAB, a dark matter search focused on nucleon-coupled dark matter in the 1-5 GeV/c$^2$ mass range. Looking to the future, the Collaboration has developed a set of experience-based upgrade scenarios, as well as novel directions, to extend the search for dark matter using the SuperCDMS technology in the SNOLAB facility. The experienced-ba… ▽ More The SuperCDMS Collaboration is currently building SuperCDMS SNOLAB, a dark matter search focused on nucleon-coupled dark matter in the 1-5 GeV/c$^2$ mass range. Looking to the future, the Collaboration has developed a set of experience-based upgrade scenarios, as well as novel directions, to extend the search for dark matter using the SuperCDMS technology in the SNOLAB facility. The experienced-based scenarios are forecasted to probe many square decades of unexplored dark matter parameter space below 5 GeV/c$^2$, covering over 6 decades in mass: 1-100 eV/c$^2$ for dark photons and axion-like particles, 1-100 MeV/c$^2$ for dark-photon-coupled light dark matter, and 0.05-5 GeV/c$^2$ for nucleon-coupled dark matter. They will reach the neutrino fog in the 0.5-5 GeV/c$^2$ mass range and test a variety of benchmark models and sharp targets. The novel directions involve greater departures from current SuperCDMS technology but promise even greater reach in the long run, and their development must begin now for them to be available in a timely fashion. The experienced-based upgrade scenarios rely mainly on dramatic improvements in detector performance based on demonstrated scaling laws and reasonable extrapolations of current performance. Importantly, these improvements in detector performance obviate significant reductions in background levels beyond current expectations for the SuperCDMS SNOLAB experiment. Given that the dominant limiting backgrounds for SuperCDMS SNOLAB are cosmogenically created radioisotopes in the detectors, likely amenable only to isotopic purification and an underground detector life-cycle from before crystal growth to detector testing, the potential cost and time savings are enormous and the necessary improvements much easier to prototype. △ Less

Submitted 1 April, 2023; v1 submitted 16 March, 2022; originally announced March 2022.

Comments: contribution to Snowmass 2021; v2 updated (assorted corrections and improvements to forecasts) October 2022; v3 updated (corrected SuperCDMS SNOLAB sensitivity curves in upgrade forecast plots in body of text) April 2023

Dark-matter And Neutrino Computation Explored (DANCE) Community Input to Snowmass

Authors: Amy Roberts, Christopher Tunnell, Belina von Krosigk, Tyler Anderson, Jason Brodsky, Micah Buuck, Tina Cartaro, Melissa Cragin, Gavin S. Davies, Miriam Diamond, Alden Fan, Aaron Higuera, Valerio Ippolito, Chris Jillings, Scott Kravitz, Luke Krezko, Ivy Li, Maria Elena Monzani, Igor Ostrovskiy, Fernanda Psihas, Andrew Renshaw, Quentin Riffard, Joel Sander, Samuele Sangiorgio, Reto Trappitsch , et al. (1 additional authors not shown)

Abstract: This paper summarizes the needs of the dark matter and neutrino communities as it relates to computation. The scope includes data acquisition, triggers, data management and processing, data preservation, simulation, machine learning, data analysis, software engineering, career development, and equity and inclusion. Beyond identifying our community needs, we propose actions that can be taken to str… ▽ More This paper summarizes the needs of the dark matter and neutrino communities as it relates to computation. The scope includes data acquisition, triggers, data management and processing, data preservation, simulation, machine learning, data analysis, software engineering, career development, and equity and inclusion. Beyond identifying our community needs, we propose actions that can be taken to strengthen this community and to work together to overcome common challenges. △ Less

Submitted 15 March, 2022; originally announced March 2022.

Comments: Needs identified during DANCE Workshop series. Submitted to Snowmass. 33 pages and 1 picture

Snowmass2021 Cosmic Frontier: Modeling, statistics, simulations, and computing needs for direct dark matter detection

Authors: Yonatan Kahn, Maria Elena Monzani, Kimberly J. Palladino, Tyler Anderson, Deborah Bard, Daniel Baxter, Micah Buuck, Concetta Cartaro, Juan I. Collar, Miriam Diamond, Alden Fan, Simon Knapen, Scott Kravitz, Rafael F. Lang, Benjamin Nachman, Ibles Olcina Samblas, Igor Ostrovskiy, Aditya Parikh, Quentin Riffard, Amy Roberts, Kelly Stifter, Matthew Szydagis, Christopher Tunnell, Belina von Krosigk, Dennis Wright , et al. (12 additional authors not shown)

Abstract: This paper summarizes the modeling, statistics, simulation, and computing needs of direct dark matter detection experiments in the next decade. This paper summarizes the modeling, statistics, simulation, and computing needs of direct dark matter detection experiments in the next decade. △ Less

Submitted 27 December, 2022; v1 submitted 15 March, 2022; originally announced March 2022.

Comments: Contribution to Snowmass 2021

Correspondence on ACMG STATEMENT: ACMG SF v3.0 list for reporting of secondary findings in clinical exome and genome sequencing: a policy statement of the American College of Medical Genetics and Genomics (ACMG) by Miller et al

Authors: Kathryn A. McGurk, Sean L. Zheng, Albert Henry, Katherine Josephs, Matthew Edwards, Antonio de Marvao, Nicola Whiffin, Angharad Roberts, Thomas R. Lumbers, Declan P. O Regan, James S. Ware

Abstract: We were interested to read the recent update on recommendations for reporting of secondary findings in clinical sequencing1, and the accompanying updated list of genes in which secondary findings should be sought (ACMG SF v3.0)2. Though the authors discuss challenges around incomplete penetrance in considerable detail, we are concerned that the recommendations do not fully convey the degree of unc… ▽ More We were interested to read the recent update on recommendations for reporting of secondary findings in clinical sequencing1, and the accompanying updated list of genes in which secondary findings should be sought (ACMG SF v3.0)2. Though the authors discuss challenges around incomplete penetrance in considerable detail, we are concerned that the recommendations do not fully convey the degree of uncertainty regarding the penetrance of variants in genes associated with inherited cardiomyopathies, which make up almost a quarter of the list. Since penetrance is incomplete and age-related, individuals found to carry variants will often require surveillance, rather than a one-off definitive diagnostic assessment. There is a lack of evidence regarding benefits, harms, and healthcare costs associated with opportunistic screening. △ Less

Submitted 9 March, 2022; originally announced March 2022.

Journal ref: Genet Med. 2022 Mar;24(3):744-746

A Search for Low-mass Dark Matter via Bremsstrahlung Radiation and the Migdal Effect in SuperCDMS

Authors: SuperCDMS Collaboration, Musaab Al-Bakry, Imran Alkhatib, Dorian Praia do Amaral, Taylor Aralis, Tsuguo Aramaki, Isaac Arnquist, Iman Ataee Langroudy, Elham Azadbakht, Samir Banik, Corey Bathurst, Dan Bauer, Lucas Bezerra, Rik Bhattacharyya, Paul Brink, Ray Bunker, Blas Cabrera, Robert Calkins, Robert Cameron, Concetta Cartaro, David Cerdeno, Yen-Yung Chang, Mouli Chaudhuri, Ran Chen, Nicholas Chott , et al. (106 additional authors not shown)

Abstract: In this paper, we present a re-analysis of SuperCDMS data using a profile likelihood approach to search for sub-GeV dark matter particles (DM) through two inelastic scattering channels: bremsstrahlung radiation and the Migdal effect. By considering possible inelastic scattering channels, experimental sensitivity can be extended to DM masses that would otherwise be undetectable through the DM-nucle… ▽ More In this paper, we present a re-analysis of SuperCDMS data using a profile likelihood approach to search for sub-GeV dark matter particles (DM) through two inelastic scattering channels: bremsstrahlung radiation and the Migdal effect. By considering possible inelastic scattering channels, experimental sensitivity can be extended to DM masses that would otherwise be undetectable through the DM-nucleon elastic scattering channel, given the energy threshold of current experiments. We exclude DM masses down to $220~\textrm{MeV}/c^2$ at $2.7 \times 10^{-30}~\textrm{cm}^2$ via the bremsstrahlung channel. The Migdal channel search excludes DM masses down to $30~\textrm{MeV}/c^2$ at $5.0 \times 10^{-30}~\textrm{cm}^2$. △ Less

Submitted 19 May, 2022; v1 submitted 4 March, 2022; originally announced March 2022.

Comments: This paper is being withdrawn due to an error in data selection during the analysis. Although incorrect, the limits are roughly representative of the sensitivity. The new corrected version of the result will be uploaded once ready

Decomposing random permutations into order-isomorphic subpermutations

Authors: Carla Groenland, Tom Johnston, Dániel Korándi, Alexander Roberts, Alex Scott, Jane Tan

Abstract: Two permutations $s$ and $t$ are $k$-similar if they can be decomposed into subpermutations $s^1, \ldots, s^k$ and $t^1, \ldots, t^k$ such that $s^i$ is order-isomorphic to $t^i$ for all $i$. Recently, Dudek, Grytczuk and Ruciński posed the problem of determining the minimum $k$ for which two permutations chosen independently and uniformly at random are $k$-similar. We show that two such permutati… ▽ More Two permutations $s$ and $t$ are $k$-similar if they can be decomposed into subpermutations $s^1, \ldots, s^k$ and $t^1, \ldots, t^k$ such that $s^i$ is order-isomorphic to $t^i$ for all $i$. Recently, Dudek, Grytczuk and Ruciński posed the problem of determining the minimum $k$ for which two permutations chosen independently and uniformly at random are $k$-similar. We show that two such permutations are $O(n^{1/3}\log^{11/6}(n))$-similar with high probability, which is tight up to a polylogarithmic factor. Our result also generalises to simultaneous decompositions of multiple permutations. △ Less

Submitted 22 January, 2023; v1 submitted 22 February, 2022; originally announced February 2022.

Comments: 11 pages, 2 figures

Ionization yield measurement in a germanium CDMSlite detector using photo-neutron sources

Authors: SuperCDMS Collaboration, M. F. Albakry, I. Alkhatib, D. W. P. Amaral, T. Aralis, T. Aramaki, I. J. Arnquist, I. Ataee Langroudy, E. Azadbakht, S. Banik, C. Bathurst, D. A. Bauer, L. V. S. Bezerra, R. Bhattacharyya, M. A. Bowles, P. L. Brink, R. Bunker, B. Cabrera, R. Calkins, R. A. Cameron, C. Cartaro, D. G. Cerdeño, Y. -Y. Chang, M. Chaudhuri, R. Chen , et al. (104 additional authors not shown)

Abstract: Two photo-neutron sources, $^{88}$Y$^{9}$Be and $^{124}$Sb$^{9}$Be, have been used to investigate the ionization yield of nuclear recoils in the CDMSlite germanium detectors by the SuperCDMS collaboration. This work evaluates the yield for nuclear recoil energies between 1 keV and 7 keV at a temperature of $\sim$ 50 mK. We use a Geant4 simulation to model the neutron spectrum assuming a charge yie… ▽ More Two photo-neutron sources, $^{88}$Y$^{9}$Be and $^{124}$Sb$^{9}$Be, have been used to investigate the ionization yield of nuclear recoils in the CDMSlite germanium detectors by the SuperCDMS collaboration. This work evaluates the yield for nuclear recoil energies between 1 keV and 7 keV at a temperature of $\sim$ 50 mK. We use a Geant4 simulation to model the neutron spectrum assuming a charge yield model that is a generalization of the standard Lindhard model and consists of two energy dependent parameters. We perform a likelihood analysis using the simulated neutron spectrum, modeled background, and experimental data to obtain the best fit values of the yield model. The ionization yield between recoil energies of 1 keV and 7 keV is shown to be significantly lower than predicted by the standard Lindhard model for germanium. There is a general lack of agreement among different experiments using a variety of techniques studying the low-energy range of the nuclear recoil yield, which is most critical for interpretation of direct dark matter searches. This suggests complexity in the physical process that many direct detection experiments use to model their primary signal detection mechanism and highlights the need for further studies to clarify underlying systematic effects that have not been well understood up to this point. △ Less

Submitted 27 June, 2022; v1 submitted 14 February, 2022; originally announced February 2022.

Journal ref: Phys. Rev. D 105, 122002 (2022)

LaMDA: Language Models for Dialog Applications

Authors: Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao , et al. (35 additional authors not shown)

Abstract: We present LaMDA: Language Models for Dialog Applications. LaMDA is a family of Transformer-based neural language models specialized for dialog, which have up to 137B parameters and are pre-trained on 1.56T words of public dialog data and web text. While model scaling alone can improve quality, it shows less improvements on safety and factual grounding. We demonstrate that fine-tuning with annotat… ▽ More We present LaMDA: Language Models for Dialog Applications. LaMDA is a family of Transformer-based neural language models specialized for dialog, which have up to 137B parameters and are pre-trained on 1.56T words of public dialog data and web text. While model scaling alone can improve quality, it shows less improvements on safety and factual grounding. We demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding. The first challenge, safety, involves ensuring that the model's responses are consistent with a set of human values, such as preventing harmful suggestions and unfair bias. We quantify safety using a metric based on an illustrative set of human values, and we find that filtering candidate responses using a LaMDA classifier fine-tuned with a small amount of crowdworker-annotated data offers a promising approach to improving model safety. The second challenge, factual grounding, involves enabling the model to consult external knowledge sources, such as an information retrieval system, a language translator, and a calculator. We quantify factuality using a groundedness metric, and we find that our approach enables the model to generate responses grounded in known sources, rather than responses that merely sound plausible. Finally, we explore the use of LaMDA in the domains of education and content recommendations, and analyze their helpfulness and role consistency. △ Less

Submitted 10 February, 2022; v1 submitted 20 January, 2022; originally announced January 2022.

Ultrahigh quality infrared polaritonic resonators based on bottom-up-synthesized van der Waals nanoribbons

Authors: Shang-Jie Yu, Yue Jiang, John A. Roberts, Markus A. Huber, Helen Yao, Xinjian Shi, Hans A. Bechtel, Stephanie N. Gilbert Corder, Tony F. Heinz, Xiaolin Zheng, Jonathan A. Fan

Abstract: van der Waals nanomaterials supporting phonon polariton quasiparticles possess unprecedented light confinement capabilities, making them ideal systems for molecular sensing, thermal emission, and subwavelength imaging applications, but they require defect-free crystallinity and nanostructured form factors to fully showcase these capabilities. We introduce bottom-up-synthesized α-MoO3 structures as… ▽ More van der Waals nanomaterials supporting phonon polariton quasiparticles possess unprecedented light confinement capabilities, making them ideal systems for molecular sensing, thermal emission, and subwavelength imaging applications, but they require defect-free crystallinity and nanostructured form factors to fully showcase these capabilities. We introduce bottom-up-synthesized α-MoO3 structures as nanoscale phonon polaritonic systems that feature tailorable morphologies and crystal qualities consistent with bulk single crystals. α-MoO3 nanoribbons serve as low-loss hyperbolic Fabry-Pérot nanoresonators, and we experimentally map hyperbolic resonances over four Reststrahlen bands spanning the far- and mid-infrared spectral range, including resonance modes beyond the tenth order. The measured quality factors are the highest from phonon polaritonic van der Waals structures to date. We anticipate that bottom-up-synthesized polaritonic van der Waals nanostructures will serve as an enabling high-performance and low-loss platform for infrared optical and optoelectronic applications. △ Less

Submitted 10 January, 2022; originally announced January 2022.

Electrically Driven Hyperbolic Nanophotonic Resonators as High Speed, Spectrally Selective Thermal Radiators

Authors: John Andris Roberts, Po-Hsun Ho, Shang-Jie Yu, Jonathan A. Fan

Abstract: We introduce and experimentally demonstrate a new class of electrically driven thermal emitter based on globally aligned carbon nanotube metamaterials patterned as nanoscale ribbons. The metamaterial ribbons exhibit electronic and photonic properties with extreme anisotropy, which enable low loss, wavelength-compressed hyperbolic photonic modes along one axis and high electrical resistivity and ef… ▽ More We introduce and experimentally demonstrate a new class of electrically driven thermal emitter based on globally aligned carbon nanotube metamaterials patterned as nanoscale ribbons. The metamaterial ribbons exhibit electronic and photonic properties with extreme anisotropy, which enable low loss, wavelength-compressed hyperbolic photonic modes along one axis and high electrical resistivity and efficient Joule heating along the other axis. Devices batch-fabricated on a single chip emit linearly polarized thermal radiation with peak wavelengths dictated by their hyperbolic resonances, and their low thermal mass yields infrared radiation modulation rates as high as one megahertz. As a proof-of-concept demonstration, we show that two sets of thermal emitters on a single chip, each operating with different spectral peak positions and modulation rates, can be used to sense carbon dioxide with a single detector. We anticipate that the combination of batch fabrication, wide modulation bandwidth, and customized spectral tuning with hyperbolic chip-based thermal emitters will enable new modalities in multiplexed infrared sources for sensing, imaging, and metrology applications. △ Less

Submitted 7 January, 2022; originally announced January 2022.

Journal ref: Nano Letters 2022, 22, 14, 5832-5840

Multistage Optimization of a Petroleum Production System with Material Balance Model

Authors: Cyrille Vessaire, Jean-Philippe Chancelier, Michel de Lara, Pierre Carpentier, Alejandro Rodríguez-Martínez, Anna Roberts

Abstract: In this paper, we propose a mathematical formulation for the management of an oil production network as a multistage optimization problem. The reservoir is modeled as a controlled dynamical system by using material balance equations. We use a dynamic programming algorithm to solve the optimization problem. Two numerical applications illustrate our work: the first one consists in optimizing the pro… ▽ More In this paper, we propose a mathematical formulation for the management of an oil production network as a multistage optimization problem. The reservoir is modeled as a controlled dynamical system by using material balance equations. We use a dynamic programming algorithm to solve the optimization problem. Two numerical applications illustrate our work: the first one consists in optimizing the production of a gas reservoir, whereas the second one tackles an oil reservoir with water injection. △ Less

Submitted 20 September, 2022; v1 submitted 4 January, 2022; originally announced January 2022.

An unstructured CD-grid variational formulation for sea ice dynamics

Authors: Giacomo Capodaglio, Mark R. Petersen, Adrian K. Turner, Andrew F. Roberts

Abstract: For the numerical simulation of earth system models, Arakawa grids are largely employed. A quadrilateral mesh is assumed for their classical definition, and different types of grids are identified depending on the location of the discretized quantities. The B-grid has both velocity components at the center of a cell, the C-grid places the velocity components on the edges in a staggered fashion, an… ▽ More For the numerical simulation of earth system models, Arakawa grids are largely employed. A quadrilateral mesh is assumed for their classical definition, and different types of grids are identified depending on the location of the discretized quantities. The B-grid has both velocity components at the center of a cell, the C-grid places the velocity components on the edges in a staggered fashion, and the D-grid is a ninety-degree rotation of a C-grid. Historically, B-grid formulations of sea ice dynamics have been dominant because they have matched the grid type used by ocean models. In recent years, as ocean models have increasingly progressed to C-grids, sea ice models have followed suit on quadrilateral meshes, but few if any implementations of unstructured C-grid sea ice models have been developed. In this work, we present an unstructured CD-grid type formulation of the elastic-viscous-plastic rheology, where the velocity unknowns are located at the edges, rather than at the vertices, as in the B-grid. The notion of a CD-grid has been recently introduced and assumes that the velocity components are co-located at the edges. The mesh cells in our analysis have $n$ sides, with $n$ greater than or equal to four. Numerical results are included to investigate the features of the proposed method. Our framework of choice is the Model for Prediction Across Scales (MPAS) within E3SM, the climate model of the U.S. Department of Energy, although our approach is general and could be applied to other models as well. While MPAS-Seaice is currently defined on a B-grid, MPAS-Ocean runs on a C-grid, hence interpolation operators are heavily used when coupled simulations are performed. The discretization introduced here aims at transitioning the dynamics of MPAS-Seaice to a CD-grid, to ultimately facilitate improved coupling with MPAS-Ocean and reduce numerical errors associated with this communication. △ Less

Submitted 6 July, 2022; v1 submitted 30 December, 2021; originally announced December 2021.

Report number: LA-UR-21-32250

Response of a CMS HGCAL silicon-pad electromagnetic calorimeter prototype to 20-300 GeV positrons

Authors: B. Acar, G. Adamov, C. Adloff, S. Afanasiev, N. Akchurin, B. Akgün, F. Alam Khan, M. Alhusseini, J. Alison, A. Alpana, G. Altopp, M. Alyari, S. An, S. Anagul, I. Andreev, P. Aspell, I. O. Atakisi, O. Bach, A. Baden, G. Bakas, A. Bakshi, S. Bannerjee, P. Bargassa, D. Barney, F. Beaudette , et al. (364 additional authors not shown)

Abstract: The Compact Muon Solenoid Collaboration is designing a new high-granularity endcap calorimeter, HGCAL, to be installed later this decade. As part of this development work, a prototype system was built, with an electromagnetic section consisting of 14 double-sided structures, providing 28 sampling layers. Each sampling layer has an hexagonal module, where a multipad large-area silicon sensor is glu… ▽ More The Compact Muon Solenoid Collaboration is designing a new high-granularity endcap calorimeter, HGCAL, to be installed later this decade. As part of this development work, a prototype system was built, with an electromagnetic section consisting of 14 double-sided structures, providing 28 sampling layers. Each sampling layer has an hexagonal module, where a multipad large-area silicon sensor is glued between an electronics circuit board and a metal baseplate. The sensor pads of approximately 1 cm$^2$ are wire-bonded to the circuit board and are readout by custom integrated circuits. The prototype was extensively tested with beams at CERN's Super Proton Synchrotron in 2018. Based on the data collected with beams of positrons, with energies ranging from 20 to 300 GeV, measurements of the energy resolution and linearity, the position and angular resolutions, and the shower shapes are presented and compared to a detailed Geant4 simulation. △ Less

Submitted 31 March, 2022; v1 submitted 12 November, 2021; originally announced November 2021.

Real time phase imaging with an asymmetric transfer function metasurface

Authors: Lukas Wesemann, Jon Rickett, Timothy J. Davis, Ann Roberts

Abstract: The conversion of phase variations in an optical wavefield into intensity information is of fundamental importance for optical imaging technology including microscopy of biological cells. While conventional approaches to phase-imaging commonly rely on bulky optical components or computational post processing, meta-optical devices have recently demonstrated all-optical, ultracompact image processin… ▽ More The conversion of phase variations in an optical wavefield into intensity information is of fundamental importance for optical imaging technology including microscopy of biological cells. While conventional approaches to phase-imaging commonly rely on bulky optical components or computational post processing, meta-optical devices have recently demonstrated all-optical, ultracompact image processing methods. Here we describe a metasurface that exploits photonic spin-orbit coupling to create an asymmetric optical transfer function for real time phase-imaging. The effect of the asymmetry on transmission through the device is demonstrated experimentally with the generation of high contrast pseudo-3D intensity images of phase variations in an optical wavefield without the need for post-processing. This non-interferometric method has potential applications in biological live cell imaging and real-time wavefront sensing. △ Less

Submitted 9 November, 2021; originally announced November 2021.

Comments: manuscript 11 pages, supplement 7 pages

Report number: UOM-1676159

Journal ref: ACS Photonics 9 2022 1803-1807

Articulatory Coordination for Speech Motor Tracking in Huntington Disease

Authors: Matthew Perez, Amrit Romana, Angela Roberts, Noelle Carlozzi, Jennifer Ann Miner, Praveen Dayalu, Emily Mower Provost

Abstract: Huntington Disease (HD) is a progressive disorder which often manifests in motor impairment. Motor severity (captured via motor score) is a key component in assessing overall HD severity. However, motor score evaluation involves in-clinic visits with a trained medical professional, which are expensive and not always accessible. Speech analysis provides an attractive avenue for tracking HD severity… ▽ More Huntington Disease (HD) is a progressive disorder which often manifests in motor impairment. Motor severity (captured via motor score) is a key component in assessing overall HD severity. However, motor score evaluation involves in-clinic visits with a trained medical professional, which are expensive and not always accessible. Speech analysis provides an attractive avenue for tracking HD severity because speech is easy to collect remotely and provides insight into motor changes. HD speech is typically characterized as having irregular articulation. With this in mind, acoustic features that can capture vocal tract movement and articulatory coordination are particularly promising for characterizing motor symptom progression in HD. In this paper, we present an experiment that uses Vocal Tract Coordination (VTC) features extracted from read speech to estimate a motor score. When using an elastic-net regression model, we find that VTC features significantly outperform other acoustic features across varied-length audio segments, which highlights the effectiveness of these features for both short- and long-form reading tasks. Lastly, we analyze the F-value scores of VTC features to visualize which channels are most related to motor score. This work enables future research efforts to consider VTC features for acoustic analyses which target HD motor symptomatology tracking. △ Less

Submitted 28 September, 2021; originally announced September 2021.

A Novel Manufacturing Process for Glass THGEMs and First Characterisation in an Optical Gaseous Argon TPC

Authors: Adam Lowe, Krishanu Majumdar, Konstantinos Mavrokoridis, Barney Philippou, Adam Roberts, Christos Touramanis

Abstract: This paper details a novel, patent pending, abrasive machining manufacturing process for the formation of sub-millimetre holes in THGEMs, with the intended application in gaseous and dual-phase TPCs. Abrasive machining favours a non-ductile substrate such as glasses or ceramics. This innovative manufacturing process allows for unprecedented versatility in THGEM substrates, electrodes, and hole geo… ▽ More This paper details a novel, patent pending, abrasive machining manufacturing process for the formation of sub-millimetre holes in THGEMs, with the intended application in gaseous and dual-phase TPCs. Abrasive machining favours a non-ductile substrate such as glasses or ceramics. This innovative manufacturing process allows for unprecedented versatility in THGEM substrates, electrodes, and hole geometry and pattern. Consequently, THGEMs produced via abrasive machining can be tailored for specific properties, for example: high stiffness, low total thickness variation, radiopurity, moisture absorption/outgassing and/or carbonisation resistance. This paper specifically focuses on three glass substrate THGEMs (G-THGEMs) made from Schott Borofloat 33 and Fused Silica. Circular and hexagonal hole shapes are also investigated. The G-THGEM electrodes are made from Indium Tin Oxide (ITO), with a resistivity of 150 $Ω$/Sq. All G-THGEMs were characterised in an optical (EMCCD) readout GArTPC, and compared to a traditionally manufactured FR4 THGEM, with their charging and secondary scintillation (S2) light production behaviour analysed. △ Less

Submitted 10 November, 2021; v1 submitted 7 September, 2021; originally announced September 2021.

Binary self-dual codes of various lengths with new weight enumerators from a modified bordered construction and neighbours

Authors: Joe Gildea, Adrian Korban, Adam Michael Roberts, Alexander Tylyshchak

Abstract: In this work, we define a modification of a bordered construction for self-dual codes which utilises $λ$-circulant matrices. We provide the necessary conditions for the construction to produce self-dual codes over finite commutative Frobenius rings of characteristic 2. Using the modified construction together with the neighbour construction, we construct many binary self-dual codes of lengths 54,… ▽ More In this work, we define a modification of a bordered construction for self-dual codes which utilises $λ$-circulant matrices. We provide the necessary conditions for the construction to produce self-dual codes over finite commutative Frobenius rings of characteristic 2. Using the modified construction together with the neighbour construction, we construct many binary self-dual codes of lengths 54, 68, 82 and 94 with weight enumerators that have previously not been known to exist. △ Less

Submitted 2 September, 2021; originally announced September 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:2108.09184, arXiv:2106.12355, arXiv:2102.10354

MSC Class: 94B05; 15B10; 15B33

Adaptively detect and accurately resolve macro-scale shocks in an efficient Equation-Free multiscale simulation

Authors: John Maclean, J. E. Bunder, I. G. Kevrekidis, A. J. Roberts

Abstract: The Equation-Free approach to efficient multiscale numerical computation marries trusted micro-scale simulations to a framework for numerical macro-scale reduction -- the patch dynamics scheme. A recent novel patch scheme empowered the Equation-Free approach to simulate systems containing shocks on the macro-scale. However, the scheme did not predict the formation of shocks accurately, and it coul… ▽ More The Equation-Free approach to efficient multiscale numerical computation marries trusted micro-scale simulations to a framework for numerical macro-scale reduction -- the patch dynamics scheme. A recent novel patch scheme empowered the Equation-Free approach to simulate systems containing shocks on the macro-scale. However, the scheme did not predict the formation of shocks accurately, and it could not simulate moving shocks. This article resolves both issues, as a first step in one spatial dimension, by embedding the Equation-Free, shock-resolving patch scheme within a classic framework for adaptive moving meshes. Our canonical micro-scale problems exhibit heterogeneous nonlinear advection and heterogeneous diffusion. We demonstrate many remarkable benefits from the moving patch scheme, including efficient and accurate macro-scale prediction despite the unknown macro-scale closure. Equation-free methods are here extended to simulate moving, forming and merging shocks without a priori knowledge of the existence or closure of the shocks. Whereas adaptive moving mesh equations are typically stiff, typically requiring small time-steps on the macro-scale, the moving macro-scale mesh of patches is typically not stiff given the context of the micro-scale time-steps required for the sub-patch dynamics. △ Less

Submitted 25 August, 2021; originally announced August 2021.

Comments: 35 A5 pages, 12 figures, submitted to SISC

New binary self-dual codes of lengths 56, 62, 78, 92 and 94 from a bordered construction

Authors: Joe Gildea, Adrian Korban, Adam Michael Roberts, Alexander Tylyshchak

Abstract: In this paper, we present a new bordered construction for self-dual codes which employs $λ$-circulant matrices. We give the necessary conditions for our construction to produce self-dual codes over a finite commutative Frobenius ring of characteristic 2. Moreover, using our bordered construction together with the well-known building-up and neighbour methods, we construct many binary self-dual code… ▽ More In this paper, we present a new bordered construction for self-dual codes which employs $λ$-circulant matrices. We give the necessary conditions for our construction to produce self-dual codes over a finite commutative Frobenius ring of characteristic 2. Moreover, using our bordered construction together with the well-known building-up and neighbour methods, we construct many binary self-dual codes of lengths 56, 62, 78, 92 and 94 with parameters in their weight enumerators that were not known in the literature before. △ Less

Submitted 3 February, 2022; v1 submitted 20 August, 2021; originally announced August 2021.

Comments: corrected typos; other minor corrections. arXiv admin note: substantial text overlap with arXiv:2102.10354, arXiv:2106.12355, arXiv:2102.12326

MSC Class: 94B05; 15B10; 15B33

Group LCD and Group Reversible LCD Codes

Authors: Steven T. Dougherty, Joe Gildea, Adrian Korban, Adam M. Roberts

Abstract: In this paper, we give a new method for constructing LCD codes. We employ group rings and a well known map that sends group ring elements to a subring of the $n \times n$ matrices to obtain LCD codes. Our construction method guarantees that our LCD codes are also group codes, namely, the codes are ideals in a group ring. We show that with a certain condition on the group ring element $v,$ one can… ▽ More In this paper, we give a new method for constructing LCD codes. We employ group rings and a well known map that sends group ring elements to a subring of the $n \times n$ matrices to obtain LCD codes. Our construction method guarantees that our LCD codes are also group codes, namely, the codes are ideals in a group ring. We show that with a certain condition on the group ring element $v,$ one can construct non-trivial group LCD codes. Moreover, we also show that by adding more constraints on the group ring element $v,$ one can construct group LCD codes that are reversible. We present many examples of binary group LCD codes of which some are optimal and group reversible LCD codes with different parameters. △ Less

Submitted 11 August, 2021; originally announced August 2021.

Comments: 17 pages

MSC Class: 94B05

Electrochemical Modeling of GITT Measurements for Improved Solid-State Diffusion Coefficient Evaluation

Authors: Jeffrey S. Horner, Grace Whang, David S. Ashby, Igor V. Kolesnichenko, Timothy N. Lambert, Bruce S. Dunn, A. Alec Talin, Scott A. Roberts

Abstract: Galvanostatic Intermittent Titration Technique (GITT) is widely used to evaluate solid-state diffusion coefficients in electrochemical systems. However, the existing analysis methods for GITT data require numerous assumptions, and the derived diffusion coefficients typically are not independently validated. To investigate the validity of the assumptions and derived diffusion coefficients, we emplo… ▽ More Galvanostatic Intermittent Titration Technique (GITT) is widely used to evaluate solid-state diffusion coefficients in electrochemical systems. However, the existing analysis methods for GITT data require numerous assumptions, and the derived diffusion coefficients typically are not independently validated. To investigate the validity of the assumptions and derived diffusion coefficients, we employ a direct pulse fitting method for interpreting GITT data that involves numerically fitting an electrochemical pulse and subsequent relaxation to a one-dimensional, single-particle, electrochemical model coupled with non-ideal transport to directly evaluate diffusion coefficients that are independently verified through cycling predictions. Extracted from GITT measurements of the intercalation regime of FeS2 and used to predict the discharge behavior, our non-ideal diffusion coefficients prove to be two orders of magnitude more accurate than ideal diffusion coefficients extracted using conventional methods. We further extend our model to a polydisperse set of particles to show the validity of a single-particle approach when the modeled radius is proportional to the total volume-to-surface-area ratio of the system. △ Less

Submitted 22 October, 2021; v1 submitted 13 July, 2021; originally announced July 2021.

Journal ref: ACS Appl. Energy Mater. 2021

New binary self-dual codes of lengths 80, 84 and 96 from composite matrices

Authors: Joe Gildea, Adrian Korban, Adam Michael Roberts

Abstract: In this work, we apply the idea of composite matrices arising from group rings to derive a number of different techniques for constructing self-dual codes over finite commutative Frobenius rings. By applying these techniques over different alphabets, we construct best known singly-even binary self-dual codes of lengths 80, 84 and 96 as well as doubly-even binary self-dual codes of length 96 that w… ▽ More In this work, we apply the idea of composite matrices arising from group rings to derive a number of different techniques for constructing self-dual codes over finite commutative Frobenius rings. By applying these techniques over different alphabets, we construct best known singly-even binary self-dual codes of lengths 80, 84 and 96 as well as doubly-even binary self-dual codes of length 96 that were not known in the literature before. △ Less

Submitted 23 June, 2021; originally announced June 2021.

Comments: arXiv admin note: text overlap with arXiv:2102.10354

MSC Class: 94B05; 16S34; 15B10; 15B33

The Principles of Deep Learning Theory

Authors: Daniel A. Roberts, Sho Yaida, Boris Hanin

Abstract: This book develops an effective theory approach to understanding deep neural networks of practical relevance. Beginning from a first-principles component-level picture of networks, we explain how to determine an accurate description of the output of trained networks by solving layer-to-layer iteration equations and nonlinear learning dynamics. A main result is that the predictions of networks are… ▽ More This book develops an effective theory approach to understanding deep neural networks of practical relevance. Beginning from a first-principles component-level picture of networks, we explain how to determine an accurate description of the output of trained networks by solving layer-to-layer iteration equations and nonlinear learning dynamics. A main result is that the predictions of networks are described by nearly-Gaussian distributions, with the depth-to-width aspect ratio of the network controlling the deviations from the infinite-width Gaussian description. We explain how these effectively-deep networks learn nontrivial representations from training and more broadly analyze the mechanism of representation learning for nonlinear models. From a nearly-kernel-methods perspective, we find that the dependence of such models' predictions on the underlying learning algorithm can be expressed in a simple and universal way. To obtain these results, we develop the notion of representation group flow (RG flow) to characterize the propagation of signals through the network. By tuning networks to criticality, we give a practical solution to the exploding and vanishing gradient problem. We further explain how RG flow leads to near-universal behavior and lets us categorize networks built from different activation functions into universality classes. Altogether, we show that the depth-to-width ratio governs the effective model complexity of the ensemble of trained networks. By using information-theoretic techniques, we estimate the optimal aspect ratio at which we expect the network to be practically most useful and show how residual connections can be used to push this scale to arbitrary depths. With these tools, we can learn in detail about the inductive bias of architectures, hyperparameters, and optimizers. △ Less

Submitted 24 August, 2021; v1 submitted 18 June, 2021; originally announced June 2021.

Comments: 471 pages, to be published by Cambridge University Press; v2: hyperlinks fixed, index added

Report number: MIT-CTP/5306

Journal ref: Cambridge University Press (2022)

The Black Hole Mass of NGC 4151 from Stellar Dynamical Modeling

Authors: Caroline A. Roberts, Misty C. Bentz, Eugene Vasiliev, Monica Valluri, Christopher A. Onken

Abstract: The mass of a supermassive black hole ($M_\mathrm{BH}$) is a fundamental property that can be obtained through observational methods. Constraining $M_\mathrm{BH}$ through multiple methods for an individual galaxy is important for verifying the accuracy of different techniques, and for investigating the assumptions inherent in each method. NGC 4151 is one of those rare galaxies for which multiple m… ▽ More The mass of a supermassive black hole ($M_\mathrm{BH}$) is a fundamental property that can be obtained through observational methods. Constraining $M_\mathrm{BH}$ through multiple methods for an individual galaxy is important for verifying the accuracy of different techniques, and for investigating the assumptions inherent in each method. NGC 4151 is one of those rare galaxies for which multiple methods can be used: stellar and gas dynamical modeling because of its proximity ($D=15.8\pm0.4$ Mpc from Cepheids), and reverberation mapping because of its active accretion. In this work, we re-analyzed $H-$band integral field spectroscopy of the nucleus of NGC 4151 from Gemini NIFS, improving the analysis at several key steps. We then constructed a wide range of axisymmetric dynamical models with the new orbit-superposition code Forstand. One of our primary goals is to quantify the systematic uncertainties in $M_\mathrm{BH}$ arising from different combinations of the deprojected density profile, inclination, intrinsic flattening, and mass-to-light ratio. As a consequence of uncertainties on the stellar luminosity profile arising from the presence of the AGN, our constraints on \mbh are rather weak. Models with a steep central cusp are consistent with no black hole; however, in models with more moderate cusps, the black hole mass lies within the range of $0.25\times10^7\,M_\odot \lesssim M_\mathrm{BH} \lesssim 3\times10^7\,M_\odot$. This measurement is somewhat smaller than the earlier analysis presented by Onken et al., but agrees with previous $M_\mathrm{BH}$ values from gas dynamical modeling and reverberation mapping. Future dynamical modeling of reverberation data, as well as IFU observations with JWST, will aid in further constraining $M_\mathrm{BH}$ in NGC 4151. △ Less

Submitted 4 June, 2021; originally announced June 2021.

Comments: 16 pages, 11 figures, 3 tables, accepted for publication in ApJ

ByT5: Towards a token-free future with pre-trained byte-to-byte models

Authors: Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel

Abstract: Most widely-used pre-trained language models operate on sequences of tokens corresponding to word or subword units. By comparison, token-free models that operate directly on raw text (bytes or characters) have many benefits: they can process text in any language out of the box, they are more robust to noise, and they minimize technical debt by removing complex and error-prone text preprocessing pi… ▽ More Most widely-used pre-trained language models operate on sequences of tokens corresponding to word or subword units. By comparison, token-free models that operate directly on raw text (bytes or characters) have many benefits: they can process text in any language out of the box, they are more robust to noise, and they minimize technical debt by removing complex and error-prone text preprocessing pipelines. Since byte or character sequences are longer than token sequences, past work on token-free models has often introduced new model architectures designed to amortize the cost of operating directly on raw text. In this paper, we show that a standard Transformer architecture can be used with minimal modifications to process byte sequences. We characterize the trade-offs in terms of parameter count, training FLOPs, and inference speed, and show that byte-level models are competitive with their token-level counterparts. We also demonstrate that byte-level models are significantly more robust to noise and perform better on tasks that are sensitive to spelling and pronunciation. As part of our contribution, we release a new set of pre-trained byte-level Transformer models based on the T5 architecture, as well as all code and data used in our experiments. △ Less

Submitted 7 March, 2022; v1 submitted 28 May, 2021; originally announced May 2021.

Comments: To be published in TACL 2022