-
Automated, Reliable, and Efficient Continental-Scale Replication of 7.3 Petabytes of Climate Simulation Data: A Case Study
Authors:
Lukasz Lacinski,
Lee Liming,
Steven Turoscy,
Cameron Harr,
Kyle Chard,
Eli Dart,
Paul Durack,
Sasha Ames,
Forrest M. Hoffman,
Ian T. Foster
Abstract:
We report on our experiences replicating 7.3 petabytes (PB) of Earth System Grid Federation (ESGF) climate simulation data from Lawrence Livermore National Laboratory (LLNL) in California to Argonne National Laboratory (ANL) in Illinois and Oak Ridge National Laboratory (ORNL) in Tennessee. This movement of some 29 million files, twice, undertaken in order to establish new ESGF nodes at ANL and OR…
▽ More
We report on our experiences replicating 7.3 petabytes (PB) of Earth System Grid Federation (ESGF) climate simulation data from Lawrence Livermore National Laboratory (LLNL) in California to Argonne National Laboratory (ANL) in Illinois and Oak Ridge National Laboratory (ORNL) in Tennessee. This movement of some 29 million files, twice, undertaken in order to establish new ESGF nodes at ANL and ORNL, was performed largely automatically by a simple replication tool, a script that invoked Globus to transfer large bundles of files while tracking progress in a database. Under the covers, Globus organized transfers to make efficient use of the high-speed Energy Sciences network (ESnet) and the data transfer nodes deployed at participating sites, and also addressed security, integrity checking, and recovery from a variety of transient failures. This success demonstrates the considerable benefits that can accrue from the adoption of performant data replication infrastructure.
△ Less
Submitted 30 April, 2024;
originally announced April 2024.
-
Improving Visual Representation Learning through Perceptual Understanding
Authors:
Samyakh Tukra,
Frederick Hoffman,
Ken Chatfield
Abstract:
We present an extension to masked autoencoders (MAE) which improves on the representations learnt by the model by explicitly encouraging the learning of higher scene-level features. We do this by: (i) the introduction of a perceptual similarity term between generated and real images (ii) incorporating several techniques from the adversarial training literature including multi-scale training and ad…
▽ More
We present an extension to masked autoencoders (MAE) which improves on the representations learnt by the model by explicitly encouraging the learning of higher scene-level features. We do this by: (i) the introduction of a perceptual similarity term between generated and real images (ii) incorporating several techniques from the adversarial training literature including multi-scale training and adaptive discriminator augmentation. The combination of these results in not only better pixel reconstruction but also representations which appear to capture better higher-level details within images. More consequentially, we show how our method, Perceptual MAE, leads to better performance when used for downstream tasks outperforming previous methods. We achieve 78.1% top-1 accuracy linear probing on ImageNet-1K and up to 88.1% when fine-tuning, with similar results for other downstream tasks, all without use of additional pre-trained models or data.
△ Less
Submitted 28 March, 2023; v1 submitted 29 December, 2022;
originally announced December 2022.
-
Evaluation of CMIP models with IOMB: Rates of contemporary ocean carbon uptake linked with vertical temperature gradients and transport to the ocean interior
Authors:
Weiwei Fu,
J. Keith Moore,
Francois Primeau,
Nathan Collier,
Oluwaseun O. Ogunro,
Forrest M. Hoffman,
James T. Randerson
Abstract:
The International Ocean Model Benchmarking (IOMB) software package is a new community resource used here to evaluate surface and upper ocean variables from CMIP5 and CMIP6 Earth System Models (ESMs) Our analysis reveals general improvement in the multi-model mean of CMIP6 compared to CMIP5 for most of the variables we examined including surface nutrients, temperature, and salinity. We find that bo…
▽ More
The International Ocean Model Benchmarking (IOMB) software package is a new community resource used here to evaluate surface and upper ocean variables from CMIP5 and CMIP6 Earth System Models (ESMs) Our analysis reveals general improvement in the multi-model mean of CMIP6 compared to CMIP5 for most of the variables we examined including surface nutrients, temperature, and salinity. We find that both CMIP5 and CMIP6 ocean models underestimate anthropogenic carbon dioxide uptake after the 1970s. For the period of 1994 to 2007, the multi-model mean from CMIP6 yields a mean cumulative carbon uptake of 27.2 +-2.2 Pg C, which is about 15% lower than the 32.0+-5.7 Pg C estimate derived from two sets of observations. Negative biases in the change in anthropogenic carbon inventory exist in the northern North Atlantic and at mid-latitudes in the southern hemisphere (30-60°S). For the few models that provided simulations of chlorofluorocarbon (CFC), we demonstrate that regions with negative anthropogenic DIC biases coincide with regions that have a negative bias in CFC concentrations. This relationship suggests that underestimates of anthropogenic carbon storage in some models originates, in part, from weak transport between the surface and interior ocean. To examine the robustness of this attribution across the full suite of CMIP5 and CMIP6 models, we examined the vertical temperature gradient between 200 and 1000m as a metric for stratification and exchange between the surface and deeper waters. On a global scale across different models and different MIPs we find a linear relationship between the bias of vertical temperature gradients and the bias in anthropogenic carbon uptake, consistent with the hypothesis that model biases in the ocean carbon sink are related to biases in surface-to-interior transport.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
Searching for Cool Dust: II. Infrared Imaging of the OH/IR Supergiants, NML Cyg, VX Sgr, S Per and the Normal Red Supergiants RS Per and T Per
Authors:
Michael S. Gordon,
Roberta M. Humphreys,
Terry J. Jones,
Dinesh Shenoy,
Robert D. Gehrz,
L. Andrew Helton,
Massimo Marengo,
Philip M. Hinz,
William F. Hoffman
Abstract:
New MMT/MIRAC (9-11 μm), SOFIA/FORCAST (11-37 μm), and Herschel/PACS (70 and 160 μm) infrared (IR) imaging and photometry is presented for three famous OH/IR red supergiants (NML Cyg, VX Sgr, and S Per) and two normal red supergiants (RS Per and T Per). We model the observed spectral energy distributions (SEDs) using radiative transfer code DUSTY. Azimuthal average profiles from the SOFIA/FORCAST…
▽ More
New MMT/MIRAC (9-11 μm), SOFIA/FORCAST (11-37 μm), and Herschel/PACS (70 and 160 μm) infrared (IR) imaging and photometry is presented for three famous OH/IR red supergiants (NML Cyg, VX Sgr, and S Per) and two normal red supergiants (RS Per and T Per). We model the observed spectral energy distributions (SEDs) using radiative transfer code DUSTY. Azimuthal average profiles from the SOFIA/FORCAST imaging, in addition to dust mass distribution profiles from DUSTY, constrain the mass-loss histories of these supergiants. For all of our observed supergiants, the DUSTY models suggest that constant mass-loss rates do not produce enough dust to explain the observed infrared emission in the stars' SEDs. Combining our results with Shenoy et al. (2016) (Paper I) we find mixed results with some red supergiants showing evidence for variable and high mass-loss events while others have constant mass loss over the past few thousand years.
△ Less
Submitted 4 April, 2018; v1 submitted 31 July, 2017;
originally announced August 2017.
-
Light Absorption in Arctic Sea Ice: Chlorophyll and Black Carbon
Authors:
Oluwaseun Ogunro,
Scott Elliott,
Nicole Jeffery,
Forrest Hoffman,
Hailong Wang,
Oliver Wingenter
Abstract:
Arctic sea ice extent has declined continuously for the past decade, owing partially to light absorption by black carbon (BC) and other impurities deposited on snow and the underlying pack. We present simulations for the contemporary period showing that the optical depth contributed by Arctic ice algal chlorophyll may be comparable during Boreal Spring to the corresponding values attributable to B…
▽ More
Arctic sea ice extent has declined continuously for the past decade, owing partially to light absorption by black carbon (BC) and other impurities deposited on snow and the underlying pack. We present simulations for the contemporary period showing that the optical depth contributed by Arctic ice algal chlorophyll may be comparable during Boreal Spring to the corresponding values attributable to BC. The largest chlorophyll attenuation is obtained in the bottom layer, which supports pigment concentrations of about 300 to 1000 mg/m3 in the Bering Sea and Sea of Okhotsk. However, chlorophyll concentrations for the ice interior in regions north of 75° N and across the Canadian Archipelago are less than 0.1 micro g/m3. Freeboard and infiltration communities lead to intermediate levels of light removal. Since BC works its way downward from the atmospheric interface, there will be regions where it appropriates photosynthetic capability. Where ice thicknesses permit significant penetration through the pack column the ice algae may be crucial absorbers. We propose a continuous increase in relative chlorophyll activity and attenuation in the future, as biological activity becomes stronger in thin ice toward the center of the Arctic basin. A shift in relative importance of the two absorber types could occur as total BC mixing ratios are reduced because of environmental advocacy.
△ Less
Submitted 10 February, 2017;
originally announced February 2017.
-
Transit times and mean ages for nonautonomous and autonomous compartmental systems
Authors:
Martin Rasmussen,
Alan Hastings,
Matthew J. Smith,
Folashade B. Agusto,
Benito M. Chen-Charpentier,
Forrest M. Hoffman,
Jiang Jiang,
Katherine E. O. Todd-Brown,
Ying Wang,
Ying-Ping Wang,
Yiqi Luo
Abstract:
We develop a theory for transit times and mean ages for nonautonomous compartmental systems. Using the McKendrick-von Förster equation, we show that the mean ages of mass in a compartmental system satisfy a linear nonautonomous ordinary differential equation that is exponentially stable. We then define a nonautonomous version of transit time as the mean age of mass leaving the compartmental system…
▽ More
We develop a theory for transit times and mean ages for nonautonomous compartmental systems. Using the McKendrick-von Förster equation, we show that the mean ages of mass in a compartmental system satisfy a linear nonautonomous ordinary differential equation that is exponentially stable. We then define a nonautonomous version of transit time as the mean age of mass leaving the compartmental system at a particular time and show that our nonautonomous theory generalises the autonomous case. We apply these results to study a nine-dimensional nonautonomous compartmental system modeling the terrestrial carbon cycle, which is a modification of the Carnegie-Ames-Stanford approach (CASA) model, and we demonstrate that the nonautonomous versions of transit time and mean age differ significantly from the autonomous quantities when calculated for that model.
△ Less
Submitted 8 March, 2016;
originally announced March 2016.
-
Bayesian dose-response analysis for epidemiological studies with complex uncertainty in dose estimation
Authors:
Deukwoo Kwon,
F. Owen Hoffman,
Brian E. Moroz,
Steven L. Simon
Abstract:
Most conventional risk analysis methods rely on a single best estimate of exposure per person which does not allow for adjustment for exposure-related uncertainty. Here, we propose a Bayesian model averaging method to properly quantify the relationship between radiation dose and disease outcomes by accounting for shared and unshared uncertainty in estimated dose. Our Bayesian risk analysis method…
▽ More
Most conventional risk analysis methods rely on a single best estimate of exposure per person which does not allow for adjustment for exposure-related uncertainty. Here, we propose a Bayesian model averaging method to properly quantify the relationship between radiation dose and disease outcomes by accounting for shared and unshared uncertainty in estimated dose. Our Bayesian risk analysis method utilizes multiple realizations of sets (vectors) of doses generated by a two-dimensional Monte Carlo simulation method that properly separates shared and unshared errors in dose estimation. The exposure model used in this work is taken from a study of the risk of thyroid nodules among a cohort of 2,376 subjects following exposure to fallout resulting from nuclear testing in Kazakhstan. We assessed the performance of our method through an extensive series of simulation tests and comparisons against conventional regression risk analysis methods. We conclude that when estimated doses contain relatively small amounts of uncertainty, the Bayesian method using multiple realizations of possibly true dose vectors gave similar results to the conventional regression-based methods of dose-response analysis. However, when large and complex mixtures of shared and unshared uncertainties are present, the Bayesian method using multiple dose vectors had significantly lower relative bias than conventional regression-based risk analysis methods as well as a markedly increased capability to include the pre-established 'true' risk coefficient within the credible interval of the Bayesian-based risk estimate. An evaluation of the dose-response using our method is presented for an epidemiological study of thyroid disease following radiation exposure.
△ Less
Submitted 2 September, 2014;
originally announced September 2014.
-
A fresh look at 3D microwave ionization curves of hydrogen Rydberg atoms
Authors:
G. N. Rockwell,
V. F. Hoffman,
Th. Clausen,
R. Blümel
Abstract:
Analytical arguments and numerical simulations suggest that the shapes of 3D microwave ionization curves measured by Koch and collaborators (see P. M. Koch and K. A. H. van Leeuwen, Phys. Rep. {\bf 255}, 289 (1995)) depend only weakly on the angular momentum of the atoms in the initial microcanonical ensemble, but strongly on the principal quantum number and the magnetic quantum number. Based on…
▽ More
Analytical arguments and numerical simulations suggest that the shapes of 3D microwave ionization curves measured by Koch and collaborators (see P. M. Koch and K. A. H. van Leeuwen, Phys. Rep. {\bf 255}, 289 (1995)) depend only weakly on the angular momentum of the atoms in the initial microcanonical ensemble, but strongly on the principal quantum number and the magnetic quantum number. Based on this insight, coupled with the computational power of a high-end 60-node Beowulf PC cluster, we present the first 3D quantum calculations of microwave ionization curves in the experimentally relevant parameter regime.
△ Less
Submitted 26 July, 2001;
originally announced July 2001.
