-
Integration of Data Reduction and Near Real-Time Archiving into the Keck Observing Model
Authors:
Max Brodheim,
John O'Meara,
Jeffrey A. Mader,
G. Bruce Berriman,
Matthew Brown,
Lucas Furhman,
Tyler Tucker,
Christopher R. Gelino,
Meca S. Lynn,
Melanie A. Swain
Abstract:
The W. M. Keck Observatory is welcoming a new era where data reduction and archiving are tightly integrated into our observing model, under the auspices of the Observatory's Data Services Initiative (DSI) project. While previously the Keck Observatory Archive (KOA) archived minimally processed, raw science data the day after observing, Keck is transitioning to a model in which it archives both raw…
▽ More
The W. M. Keck Observatory is welcoming a new era where data reduction and archiving are tightly integrated into our observing model, under the auspices of the Observatory's Data Services Initiative (DSI) project. While previously the Keck Observatory Archive (KOA) archived minimally processed, raw science data the day after observing, Keck is transitioning to a model in which it archives both raw frames and reduced data in near real-time. These data will be made available to observers and collaborators immediately upon ingestion through a dedicated new interface that will support collaboration and sharing among teams, as well as stream data directly to personal computers without access to WMKO's internal networks. Both the raw and science-ready data products will be made publicly available upon the expiration of data protections.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
The Observers' Data Access Portal at the Keck Observatory Archive
Authors:
T. Oluyide,
M. S. Lynn,
T. Coda,
G. B. Berriman,
M. Brown,
L. Fuhrman,
C. Gelino,
J. Good,
J. Hayashi,
C. -H. Lee,
J. Mader,
M. A. Swain
Abstract:
For all active instruments, the Keck Observatory Archive (KOA) now ingests raw data from the Keck Telescopes within 1 minute of acquisition, quick-look reduced data within 5 minutes of creation, and science ready reduced data for four instruments as they are created by their automated pipelines. On August 1, 2023, KOA released the Observers Data Access Portal (ODAP), which enables observers at the…
▽ More
For all active instruments, the Keck Observatory Archive (KOA) now ingests raw data from the Keck Telescopes within 1 minute of acquisition, quick-look reduced data within 5 minutes of creation, and science ready reduced data for four instruments as they are created by their automated pipelines. On August 1, 2023, KOA released the Observers Data Access Portal (ODAP), which enables observers at the telescope and their collaborators anywhere in the world to securely monitor and download science, calibration, and quick-look data as they are ingested into the archive. The portal is built using Python Socket IO.WebSockets that ensure metadata appear in the portal as the data themselves are ingested. The portal itself is a dynamic web interface built with React. It enables users to view and customize metadata fields, filter metadata according to data type, and download data as they are ingested or in bulk through wget scripts. Observers have used the ODAP since its release and have provided feedback that will guide future releases.
△ Less
Submitted 6 February, 2024;
originally announced February 2024.
-
Processing All-Sky Images At Scale On The Amazon Cloud: A HiPS Example
Authors:
G. Bruce Berriman,
John C. Good
Abstract:
We report here on a project that has developed a practical approach to processing all-sky image collections on cloud platforms, using as an exemplar application the creation of three-color Hierarchical Progressive Survey (HiPS) maps of the 2MASS data set with the Montage Image Mosaic Engine on Amazon Web Services. We will emphasize issues that must be considered by scientists wishing to use cloud…
▽ More
We report here on a project that has developed a practical approach to processing all-sky image collections on cloud platforms, using as an exemplar application the creation of three-color Hierarchical Progressive Survey (HiPS) maps of the 2MASS data set with the Montage Image Mosaic Engine on Amazon Web Services. We will emphasize issues that must be considered by scientists wishing to use cloud platforms to perform such parallel processing, so providing a guide for scientists wishing to exploit cloud platforms for similar large-scale processing. A HiPS map is based on the HEALPix sky-tiling scheme. Progressive zooming of a HiPS map reveals an image sampled at ever smaller or larger spatial scales that are defined by the HEALPix standard. Briefly, the approach used by Montage involves creating a base mosaic at the lowest required HEALPix level, usually chosen to match as closely as possible the spatial sampling of the input images, then cutting out the HiPS cells in PNG format from this mosaic. The process is repeated at successive HEALPix levels to create a nested collection of FITS files, from which PNG files are created that are shown in HiPS viewers. Stretching FITS files to produce PNGs is based on an image histogram. For composite regions (up and including the whole sky), the histograms for each tile can be combined to create a composite histogram for the region. Using this single histogram for each of the individual FITS files means all the PNGs are on the same brightness scale and displaying them side by side in a HiPS viewer produces a continuous uniform map across the entire sky.
△ Less
Submitted 6 February, 2024;
originally announced February 2024.
-
Real-time Data Ingestion at the Keck Observatory Archive (KOA)
Authors:
G. Bruce Berriman,
M. Brodheim,
M. Brown,
L. Fuhrman,
C. R. Gelino,
M. Kong,
C. -H. Lee,
M. S. Lynn,
J. Mader,
T. Oluyide,
M. A. Swain,
T. Tucker,
A. Laity,
J. Riley
Abstract:
Since February of this year, KOA began to prepare, transfer, and ingest data as they were acquired in near-real time; in most cases data are available to observers through KOA within one minute of acquisition. Real-time ingestion will be complete for all active instruments by the end of Summer 2022. The observatory is supporting the development of modern Python data reduction pipelines, which when…
▽ More
Since February of this year, KOA began to prepare, transfer, and ingest data as they were acquired in near-real time; in most cases data are available to observers through KOA within one minute of acquisition. Real-time ingestion will be complete for all active instruments by the end of Summer 2022. The observatory is supporting the development of modern Python data reduction pipelines, which when delivered, will automatically create science-ready data sets at the end of each night for ingestion into the archive. This presentation will describe the infrastructure developed to support real-time data ingestion, itself part of a larger initiative at the Observatory to modernize end-to-end operations.
During telescope operations, the software at WMKO is executed automatically when a newly acquired file is recognized through monitoring a keyword-based observatory control system; this system is used at Keck to execute virtually all observatory functions. The monitor uses callbacks built into the control system to begin data preparation of files for transmission to the archive on an individual basis: scheduling scripts or file system related triggers are unnecessary. An HTTP-based system called from the Flask micro-framework enables file transfers between WMKO and NExScI and triggers data ingestion at NExScI. The ingestion system at NEXScI is a compact (4 KLOC), highly fault-tolerant, Python-based system. It uses a shared file system to transfer data from WMKO to NExScI. The ingestion code is instrument agnostic, with instrument parameters read from configuration files. It replaces an unwieldy (50 KLOC) C-based system that had been in use since 2004.
△ Less
Submitted 5 December, 2022;
originally announced December 2022.
-
Astronomical Image Processing at Scale With Pegasus and Montage
Authors:
G. Bruce Berriman,
John C. Good,
Ewa Deelman,
Ryan Tanaka,
Karan Vahi
Abstract:
Image processing at scale is a powerful tool for creating new data sets and integrating them with existing data sets and performing analysis and quality assurance investigations. Workflow managers offer advantages in this type of processing, which involves multiple data access and processing steps. Generally, they enable automation of the workflow by locating data and resources, recovery from fail…
▽ More
Image processing at scale is a powerful tool for creating new data sets and integrating them with existing data sets and performing analysis and quality assurance investigations. Workflow managers offer advantages in this type of processing, which involves multiple data access and processing steps. Generally, they enable automation of the workflow by locating data and resources, recovery from failures, and monitoring of performance. In this focus demo we demonstrate how the Pegasus Workflow Manager Python API manages image processing to create mosaics with the Montage Image Mosaic engine. Since 2001, Pegasus has been developed and maintained at USC/ISI. Montage was in fact one of the first applications used to design Pegasus and optimize its performance. Pegasus has since found application in many areas of science. LIGO exploited it in making discoveries of black holes. The Vera C. Rubin Observatory used it to compare the cost and performance of processing images on cloud platforms. While these are examples of projects at large scale, small team investigations on local clusters of machines can benefit from Pegasus as well.
△ Less
Submitted 22 November, 2021;
originally announced November 2021.
-
Best Practices for Data Publication in the Astronomical Literature
Authors:
Tracy X. Chen,
Marion Schmitz,
Joseph M. Mazzarella,
Xiuqin Wu,
Julian C. van Eyken,
Alberto Accomazzi,
Rachel L. Akeson,
Mark Allen,
Rachael Beaton,
G. Bruce Berriman,
Andrew W. Boyle,
Marianne Brouty,
Ben Chan,
Jessie L. Christiansen,
David R. Ciardi,
David Cook,
Raffaele D'Abrusco,
Rick Ebert,
Cren Frayer,
Benjamin J. Fulton,
Christopher Gelino,
George Helou,
Calen B. Henderson,
Justin Howell,
Joyce Kim
, et al. (20 additional authors not shown)
Abstract:
We present an overview of best practices for publishing data in astronomy and astrophysics journals. These recommendations are intended as a reference for authors to help prepare and publish data in a way that will better represent and support science results, enable better data sharing, improve reproducibility, and enhance the reusability of data. Observance of these guidelines will also help to…
▽ More
We present an overview of best practices for publishing data in astronomy and astrophysics journals. These recommendations are intended as a reference for authors to help prepare and publish data in a way that will better represent and support science results, enable better data sharing, improve reproducibility, and enhance the reusability of data. Observance of these guidelines will also help to streamline the extraction, preservation, integration and cross-linking of valuable data from astrophysics literature into major astronomical databases, and consequently facilitate new modes of science discovery that will better exploit the vast quantities of panchromatic and multi-dimensional data associated with the literature. We encourage authors, journal editors, referees, and publishers to implement the best practices reviewed here, as well as related recommendations from international astronomical organizations such as the International Astronomical Union (IAU) for publication of nomenclature, data, and metadata. A convenient Checklist of Recommendations for Publishing Data in the Literature is included for authors to consult before the submission of the final version of their journal articles and associated data files. We recommend that publishers of journals in astronomy and astrophysics incorporate a link to this document in their Instructions to Authors.
△ Less
Submitted 16 April, 2022; v1 submitted 2 June, 2021;
originally announced June 2021.
-
Cost Management on Commercial Cloud Platforms
Authors:
G. Bruce Berriman,
William O'Mullane,
Arik Mitschang,
Ivelina Momcheva
Abstract:
Commercial cloud platforms are a powerful technology for astronomical research. Despite the benefits of cloud computing -- such as on-demand scalability and reduction of systems management overhead -- confusion over how to manage costs remains, for many, one of the biggest barriers to entry. This confusion is exacerbated by the rapid growth in services offered by commercial providers, by the growt…
▽ More
Commercial cloud platforms are a powerful technology for astronomical research. Despite the benefits of cloud computing -- such as on-demand scalability and reduction of systems management overhead -- confusion over how to manage costs remains, for many, one of the biggest barriers to entry. This confusion is exacerbated by the rapid growth in services offered by commercial providers, by the growth in the number of these providers, and by storage, compute, and I/O metered at separate rates -- all of which can change without notice. As a rule, processing is very cheap, storage is more expensive, and downloading is very expensive. Thus, an application that produces large image data sets for download will be far more expensive than an application that performs extensive processing on a small data set. This Birds of a Feather (BoF) session aimed to quantify the above statement by presenting case studies of the costing of astronomy applications on commercial clouds that covered a range of processing scenarios; these presentations were the basis for discussion by the attendees.
△ Less
Submitted 10 December, 2020;
originally announced December 2020.
-
The International Virtual Observatory Alliance (IVOA) in 2020
Authors:
G. Bruce Berriman,
The IVOA Executive Committee,
The IVOA Technical Coordination Group,
The IVOA Community
Abstract:
The International Virtual Observatory Alliance (IVOA) develops the technical standards needed for seamless discovery of and access to astronomy data worldwide, according to the Findable, Accessible, Interoperable and Reusable (FAIR) principles, with the goal of realizing the Virtual Observatory (VO). There are 21 member organizations. The Netherlands VO applied for membership in 2020. Astronomical…
▽ More
The International Virtual Observatory Alliance (IVOA) develops the technical standards needed for seamless discovery of and access to astronomy data worldwide, according to the Findable, Accessible, Interoperable and Reusable (FAIR) principles, with the goal of realizing the Virtual Observatory (VO). There are 21 member organizations. The Netherlands VO applied for membership in 2020. Astronomical communities from other nations have shown interest in joining the IVOA. This paper describes the activities of the IVOA in 2020 and summarizes the May and November 2020 virtual "interoperability meetings." The May meeting was the first to be held online and the first to have over 200 registrants.
△ Less
Submitted 10 December, 2020;
originally announced December 2020.
-
Creating High Quality All-Sky Visualizations of Astronomy Image Data Sets: HiPS and Montage
Authors:
G. Bruce Berriman,
John C. Good,
Vandana Desai,
Steven L. Groom
Abstract:
We describe a case study to use the Montage image mosaic engine to create maps of the ALLWISE image data set in the Hierarchical Progressive Survey (HiPS) sky-tesselation scheme. Our approach demonstrates that Montage reveals the science content of infrared images in greater detail than has hitherto been possible in HiPS maps. The approach exploits two unique (to our knowledge) characteristics of…
▽ More
We describe a case study to use the Montage image mosaic engine to create maps of the ALLWISE image data set in the Hierarchical Progressive Survey (HiPS) sky-tesselation scheme. Our approach demonstrates that Montage reveals the science content of infrared images in greater detail than has hitherto been possible in HiPS maps. The approach exploits two unique (to our knowledge) characteristics of the Montage image mosaic engine: background modeling to rectify the time variable image backgrounds to common levels; and an adaptive image stretch to present images for visualization. The creation of the maps is supported by the development of four new tools that when fully tested will become part of the Montage distribution. The compute intensive part of the processing lies in the reprojection of the images, and we show how we optimized the processing for efficient creation of mosaics that are used in turn to create maps in the HiPS tiling scheme. We plan to apply our methodology to infrared image data sets such a those delivered by Spitzer, 2MASS, IRAS and Planck.
△ Less
Submitted 28 January, 2020;
originally announced January 2020.
-
The International Virtual Observatory Alliance in 2019
Authors:
Mark A. Allen,
Patrick Dowler,
Janet D. Evans,
Chenzhou Cui,
Tim Jenness,
Bruno Merin,
G. Bruce Berriman,
J. J. Kavelaars
Abstract:
The International Virtual Observatory Alliance (IVOA) held its bi-annual Interoperability Meetings in May 2019, and in October 2019 following the ADASS XXIX conference. We provide a brief report on the status of the IVOA and the activities of the Interoperability Meetings.
The International Virtual Observatory Alliance (IVOA) held its bi-annual Interoperability Meetings in May 2019, and in October 2019 following the ADASS XXIX conference. We provide a brief report on the status of the IVOA and the activities of the Interoperability Meetings.
△ Less
Submitted 3 December, 2019;
originally announced December 2019.
-
Joint Survey Processing of LSST, Euclid and WFIRST: Enabling a broad array of astrophysics and cosmology through pixel level combinations of datasets
Authors:
R. Chary,
G. Brammer,
P. Capak,
W. Dawson,
A. Faisst,
S. Fajardo-Acosta,
H. C. Ferguson,
C. J. Grillmair,
S. Hemmati,
A. Koekemoer,
B. Lee,
R. Lupton,
S. Malhotra,
P. Melchior,
I. Momcheva,
J. Newman,
J. Masiero,
R. Paladini,
A. Prakash,
J. Rhodes,
B. Rusholme,
M. Schneider,
N. Stickley,
A. Smith,
W. M. Wood-Vasey
, et al. (1 additional authors not shown)
Abstract:
Joint survey processing (JSP) is the pixel level combination of LSST, Euclid, and WFIRST datasets. By combining the high spatial resolution of the space-based datasets with deep, seeing-limited, ground-based images in the optical bands, systematics like source confusion and astrometric mismatch can be addressed to derive the highest precision optical/infrared photometric catalogs. This white paper…
▽ More
Joint survey processing (JSP) is the pixel level combination of LSST, Euclid, and WFIRST datasets. By combining the high spatial resolution of the space-based datasets with deep, seeing-limited, ground-based images in the optical bands, systematics like source confusion and astrometric mismatch can be addressed to derive the highest precision optical/infrared photometric catalogs. This white paper highlights the scientific motivation, computational and algorithmic needs to build joint pixel level processing capabilities, which the individual projects by themselves will not be able to support. Through this white paper, we request that the Astro2020 decadal committee recognize the JSP effort as a multi-agency project with the natural outcome being a collaborative effort among groups which are normally supported by a single agency. JSP will allow the U.S. (and international) astronomical community to manipulate the flagship data sets and undertake innovative science investigations ranging from solar system object characterization, exoplanet detections, nearby galaxy rotation rates and dark matter properties, to epoch of reionization studies. It will also result in the ultimate constraints on cosmological parameters and the nature of dark energy, with far smaller uncertainties and a better handle on systematics than by any one survey alone.
△ Less
Submitted 2 October, 2019;
originally announced October 2019.
-
Image Processing in Python With Montage
Authors:
John Good,
G. Bruce Berriman
Abstract:
The Montage image mosaic engine has found wide applicability in astronomy research, integration into processing environments, and is an examplar application for the development of advanced cyber-infrastructure. It is written in C to provide performance and portability. Linking C/C++ libraries to the Python kernel at run time as binary extensions allows them to run under Python at compiled speeds a…
▽ More
The Montage image mosaic engine has found wide applicability in astronomy research, integration into processing environments, and is an examplar application for the development of advanced cyber-infrastructure. It is written in C to provide performance and portability. Linking C/C++ libraries to the Python kernel at run time as binary extensions allows them to run under Python at compiled speeds and enables users to take advantage of all the functionality in Python. We have built Python binary extensions of the 59 ANSI-C modules that make up version 5 of the Montage toolkit. This has involved a turning the code into a C library, with driver code fully separated to reproduce the calling sequence of the command-line tools; and then adding Python and C linkage code with the Cython library, which acts as a bridge between general C libraries and the Python interface. We will demonstrate how to use these Python binary extensions to perform image processing, including reprojecting and resampling images, rectifying background emission to a common level, creation of image mosaics that preserve the calibration and astrometric fidelity of the input images, creating visualizations with an adaptive stretch algorithm, processing HEALPix images, and analyzing and managing image metadata.
△ Less
Submitted 26 August, 2019;
originally announced August 2019.
-
Increasing the Discovery Space in Astrophysics - A Collation of Six Submitted White Papers
Authors:
G. Fabbiano,
M. Elvis,
A. Accomazzi,
G. B. Berriman,
N. Brickhouse,
S. Bose,
D. Carrera,
I. Chilingarian,
F. Civano,
B. Czerny,
R. D'Abrusco,
B. Diemer,
J. Drake,
R. Emami Meibody,
J. R. Farah,
G. G. Fazio,
E. Feigelson,
F. Fornasini,
Jay Gallagher,
J. Grindlay,
L. Hernquist,
D. J. James,
M. Karovska,
V. Kashyap,
D. -W. Kim
, et al. (24 additional authors not shown)
Abstract:
We write in response to the call from the 2020 Decadal Survey to submit white papers illustrating the most pressing scientific questions in astrophysics for the coming decade. We propose exploration as the central question for the Decadal Committee's discussions.The history of astronomy shows that paradigm changing discoveries are not driven by well formulated scientific questions, based on the kn…
▽ More
We write in response to the call from the 2020 Decadal Survey to submit white papers illustrating the most pressing scientific questions in astrophysics for the coming decade. We propose exploration as the central question for the Decadal Committee's discussions.The history of astronomy shows that paradigm changing discoveries are not driven by well formulated scientific questions, based on the knowledge of the time. They were instead the result of the increase in discovery space fostered by new telescopes and instruments. An additional tool for increasing the discovery space is provided by the analysis and mining of the increasingly larger amount of archival data available to astronomers. Revolutionary observing facilities, and the state of the art astronomy archives needed to support these facilities, will open up the universe to new discovery. Here we focus on exploration for compact objects and multi messenger science. This white paper includes science examples of the power of the discovery approach, encompassing all the areas of astrophysics covered by the 2020 Decadal Survey.
△ Less
Submitted 18 March, 2019; v1 submitted 15 March, 2019;
originally announced March 2019.
-
Deep Learning for Multi-Messenger Astrophysics: A Gateway for Discovery in the Big Data Era
Authors:
Gabrielle Allen,
Igor Andreoni,
Etienne Bachelet,
G. Bruce Berriman,
Federica B. Bianco,
Rahul Biswas,
Matias Carrasco Kind,
Kyle Chard,
Minsik Cho,
Philip S. Cowperthwaite,
Zachariah B. Etienne,
Daniel George,
Tom Gibbs,
Matthew Graham,
William Gropp,
Anushri Gupta,
Roland Haas,
E. A. Huerta,
Elise Jennings,
Daniel S. Katz,
Asad Khan,
Volodymyr Kindratenko,
William T. C. Kramer,
Xin Liu,
Ashish Mahabal
, et al. (23 additional authors not shown)
Abstract:
This report provides an overview of recent work that harnesses the Big Data Revolution and Large Scale Computing to address grand computational challenges in Multi-Messenger Astrophysics, with a particular emphasis on real-time discovery campaigns. Acknowledging the transdisciplinary nature of Multi-Messenger Astrophysics, this document has been prepared by members of the physics, astronomy, compu…
▽ More
This report provides an overview of recent work that harnesses the Big Data Revolution and Large Scale Computing to address grand computational challenges in Multi-Messenger Astrophysics, with a particular emphasis on real-time discovery campaigns. Acknowledging the transdisciplinary nature of Multi-Messenger Astrophysics, this document has been prepared by members of the physics, astronomy, computer science, data science, software and cyberinfrastructure communities who attended the NSF-, DOE- and NVIDIA-funded "Deep Learning for Multi-Messenger Astrophysics: Real-time Discovery at Scale" workshop, hosted at the National Center for Supercomputing Applications, October 17-19, 2018. Highlights of this report include unanimous agreement that it is critical to accelerate the development and deployment of novel, signal-processing algorithms that use the synergy between artificial intelligence (AI) and high performance computing to maximize the potential for scientific discovery with Multi-Messenger Astrophysics. We discuss key aspects to realize this endeavor, namely (i) the design and exploitation of scalable and computationally efficient AI algorithms for Multi-Messenger Astrophysics; (ii) cyberinfrastructure requirements to numerically simulate astrophysical sources, and to process and interpret Multi-Messenger Astrophysics data; (iii) management of gravitational wave detections and triggers to enable electromagnetic and astro-particle follow-ups; (iv) a vision to harness future developments of machine and deep learning and cyberinfrastructure resources to cope with the scale of discovery in the Big Data Era; (v) and the need to build a community that brings domain experts together with data scientists on equal footing to maximize and accelerate discovery in the nascent field of Multi-Messenger Astrophysics.
△ Less
Submitted 1 February, 2019;
originally announced February 2019.
-
A Study of the Efficiency of Spatial Indexing Methods Applied to Large Astronomical Databases
Authors:
G. B. Berriman,
J. C. Good,
B. Shiao,
T. Donaldson
Abstract:
We report the results of a study to compare the performance of two common database indexing methods, HTM and HEALPix, on Solaris and Windows database servers installed with PostgreSQL, and a Windows Server installed with MS SQL Server. The indexing was applied to the 2MASS All-Sky Catalog and to the Hubble Source Catalog, which approximate the diversity of catalogs common in astronomy. On each ser…
▽ More
We report the results of a study to compare the performance of two common database indexing methods, HTM and HEALPix, on Solaris and Windows database servers installed with PostgreSQL, and a Windows Server installed with MS SQL Server. The indexing was applied to the 2MASS All-Sky Catalog and to the Hubble Source Catalog, which approximate the diversity of catalogs common in astronomy. On each server, the study compared indexing performance by submitting 1 million queries at each index level with random sky positions and random cone search radius, which was computed on a logarithmic scale between 1 arcsec and 1 degree, and measuring the time to complete the query and write the output. These simulated queries, intended to model realistic use patterns, were run in a uniform way on many combinations of indexing method and indexing depth. The query times in all simulations are strongly I/O-bound and are linear with number of records returned for large numbers of sources. There are, however, considerable differences between simulations, which reveal that hardware I/O throughput is a more important factor in managing the performance of a DBMS than the choice of indexing scheme. The choice of index itself is relatively unimportant: for comparable index levels, the performance is consistent within the scatter of the timings. At small index levels (large cells; e.g. level 4; cell size 3.7 deg), there is large scatter in the timings because of wide variations in the number of sources found in the cells. At larger index levels, performance improves and scatter decreases, but the improvement at level 8 (14 arcmin) and higher is masked to some extent in the timing scatter caused by the range of query sizes. At very high levels (20; 0.0004 arsec), the granularity of the cells becomes so high that a large number of extraneous empty cells begin to degrade performance.
△ Less
Submitted 22 June, 2018;
originally announced June 2018.
-
Sustaining the Montage Image Mosaic Engine Since 2002
Authors:
G. Bruce Berriman,
John C. Good
Abstract:
This paper describes how we have sustained the Montage image mosaic engine (http://montage.ipac.caltech.edu) first released in 2002, to support the ever-growing scale and complexity of modern data sets. The key to its longevity has been its design as a toolkit written in ANSI-C, with each tool performing one distinct task, for easy integration into scripts, pipelines and workflows. The same code b…
▽ More
This paper describes how we have sustained the Montage image mosaic engine (http://montage.ipac.caltech.edu) first released in 2002, to support the ever-growing scale and complexity of modern data sets. The key to its longevity has been its design as a toolkit written in ANSI-C, with each tool performing one distinct task, for easy integration into scripts, pipelines and workflows. The same code base now supports Windows, JavaScript and Python by taking advantage of recent advances in compilers. The design has led to applicability of Montage far beyond what was anticipated when Montage was first built, such as supporting observation planning for the JWST. Moreover, Montage is highly scalable and is in wide use within the IT community to develop advanced, fault-tolerant cyber-infrastructure, such as job schedulers for grids, workflow orchestration, and restructuring techniques for processing complex workflows and pipelines.
△ Less
Submitted 11 June, 2018;
originally announced June 2018.
-
The KELT Follow-Up Network and Transit False Positive Catalog: Pre-vetted False Positives for TESS
Authors:
Karen A. Collins,
Kevin I. Collins,
Joshua Pepper,
Jonathan Labadie-Bartz,
Keivan Stassun,
B. Scott Gaudi,
Daniel Bayliss,
Joao Bento,
Knicole D. Colón,
Dax Feliz,
David James,
Marshall C. Johnson,
Rudolf B. Kuhn,
Michael B. Lund,
Matthew T. Penny,
Joseph E. Rodriguez,
Robert J. Siverd,
Daniel J. Stevens,
Xinyu Yao,
George Zhou,
Mundra Akshay,
Giulio F. Aldi,
Cliff Ashcraft,
Supachai Awiphan,
Özgür Baştürk
, et al. (86 additional authors not shown)
Abstract:
The Kilodegree Extremely Little Telescope (KELT) project has been conducting a photometric survey for transiting planets orbiting bright stars for over ten years. The KELT images have a pixel scale of ~23"/pixel---very similar to that of NASA's Transiting Exoplanet Survey Satellite (TESS)---as well as a large point spread function, and the KELT reduction pipeline uses a weighted photometric apertu…
▽ More
The Kilodegree Extremely Little Telescope (KELT) project has been conducting a photometric survey for transiting planets orbiting bright stars for over ten years. The KELT images have a pixel scale of ~23"/pixel---very similar to that of NASA's Transiting Exoplanet Survey Satellite (TESS)---as well as a large point spread function, and the KELT reduction pipeline uses a weighted photometric aperture with radius 3'. At this angular scale, multiple stars are typically blended in the photometric apertures. In order to identify false positives and confirm transiting exoplanets, we have assembled a follow-up network (KELT-FUN) to conduct imaging with higher spatial resolution, cadence, and photometric precision than the KELT telescopes, as well as spectroscopic observations of the candidate host stars. The KELT-FUN team has followed-up over 1,600 planet candidates since 2011, resulting in more than 20 planet discoveries. Excluding ~450 false alarms of non-astrophysical origin (i.e., instrumental noise or systematics), we present an all-sky catalog of the 1,128 bright stars (6<V<10) that show transit-like features in the KELT light curves, but which were subsequently determined to be astrophysical false positives (FPs) after photometric and/or spectroscopic follow-up observations. The KELT-FUN team continues to pursue KELT and other planet candidates and will eventually follow up certain classes of TESS candidates. The KELT FP catalog will help minimize the duplication of follow-up observations by current and future transit surveys such as TESS.
△ Less
Submitted 19 September, 2018; v1 submitted 5 March, 2018;
originally announced March 2018.
-
The Astrophysics Source Code Library: What's new, what's coming
Authors:
Alice Allen,
G. Bruce Berriman,
Kimberly DuPrie,
Jessica Mink,
Robert Nemiroff,
P. Wesley Ryan,
Judy Schmidt,
Lior Shamir,
Keith Shortridge,
Mark Taylor,
Peter Teuben,
John Wallin,
Rein H. Warmels
Abstract:
The Astrophysics Source Code Library (ASCL, ascl.net), established in 1999, is a citable online registry of source codes used in research that are available for download; the ASCL's main purpose is to improve the transparency, reproducibility, and falsifiability of research. In 2017, improvements to the resource included real-time data backup for submissions and newly-published entries, improved c…
▽ More
The Astrophysics Source Code Library (ASCL, ascl.net), established in 1999, is a citable online registry of source codes used in research that are available for download; the ASCL's main purpose is to improve the transparency, reproducibility, and falsifiability of research. In 2017, improvements to the resource included real-time data backup for submissions and newly-published entries, improved cross-matching of research papers with software entries in ADS, and expansion of preferred citation information for the software in the ASCL.
△ Less
Submitted 8 December, 2017;
originally announced December 2017.
-
Software metadata: How much is enough?
Authors:
Alice Allen,
Peter Teuben,
G. Bruce Berriman,
Kimberly DuPrie,
Keith Shortridge,
Rein Warmels
Abstract:
Broad efforts are underway to capture metadata about research software and retain it across services; notable in this regard is the CodeMeta project. What metadata are important to have about (research) software? What metadata are useful for searching for codes? What would you like to learn about astronomy software? This BoF sought to gather information on metadata most desired by researchers and…
▽ More
Broad efforts are underway to capture metadata about research software and retain it across services; notable in this regard is the CodeMeta project. What metadata are important to have about (research) software? What metadata are useful for searching for codes? What would you like to learn about astronomy software? This BoF sought to gather information on metadata most desired by researchers and users of astro software and others interested in registering, indexing, capturing, and doing research on this software. Information from this BoF could conceivably result in changes to the Astrophysics Source Code Library (ASCL) or other resources for the benefit of the community or provide input into other projects concerned with software metadata.
△ Less
Submitted 6 December, 2017;
originally announced December 2017.
-
NASA's Long-Term Astrophysics Data Archives
Authors:
L. M. Rebull,
V. Desai,
H. Teplitz,
S. Groom,
R. Akeson,
G. B. Berriman,
G. Helou,
D. Imel,
J. M. Mazzarella,
A. Accomazzi,
T. McGlynn,
A. Smale,
R. White
Abstract:
NASA regards data handling and archiving as an integral part of space missions, and has a strong track record of serving astrophysics data to the public, beginning with the the IRAS satellite in 1983. Archives enable a major science return on the significant investment required to develop a space mission. In fact, the presence and accessibility of an archive can more than double the number of pape…
▽ More
NASA regards data handling and archiving as an integral part of space missions, and has a strong track record of serving astrophysics data to the public, beginning with the the IRAS satellite in 1983. Archives enable a major science return on the significant investment required to develop a space mission. In fact, the presence and accessibility of an archive can more than double the number of papers resulting from the data. In order for the community to be able to use the data, they have to be able to find the data (ease of access) and interpret the data (ease of use). Funding of archival research (e.g., the ADAP program) is also important not only for making scientific progress, but also for encouraging authors to deliver data products back to the archives to be used in future studies. NASA has also enabled a robust system that can be maintained over the long term, through technical innovation and careful attention to resource allocation. This article provides a brief overview of some of NASA's major astrophysics archive systems, including IRSA, MAST, HEASARC, KOA, NED, the Exoplanet Archive, and ADS.
△ Less
Submitted 27 September, 2017;
originally announced September 2017.
-
The Application of the Montage Image Mosaic Engine To The Visualization Of Astronomical Images
Authors:
G. Bruce Berriman,
J. C. Good
Abstract:
The Montage Image Mosaic Engine was designed as a scalable toolkit, written in C for performance and portability across *nix platforms, that assembles FITS images into mosaics. The code is freely available and has been widely used in the astronomy and IT communities for research, product generation and for developing next-generation cyber-infrastructure. Recently, it has begun to finding applicabi…
▽ More
The Montage Image Mosaic Engine was designed as a scalable toolkit, written in C for performance and portability across *nix platforms, that assembles FITS images into mosaics. The code is freely available and has been widely used in the astronomy and IT communities for research, product generation and for developing next-generation cyber-infrastructure. Recently, it has begun to finding applicability in the field of visualization. This has come about because the toolkit design allows easy integration into scalable systems that process data for subsequent visualization in a browser or client. And it includes a visualization tool suitable for automation and for integration into Python: mViewer creates, with a single command, complex multi-color images overlaid with coordinate displays, labels, and observation footprints, and includes an adaptive image histogram equalization method that preserves the structure of a stretched image over its dynamic range. The Montage toolkit contains functionality originally developed to support the creation and management of mosaics but which also offers value to visualization: a background rectification algorithm that reveals the faint structure in an image; and tools for creating cutout and down-sampled versions of large images. Version 5 of Montage offers support for visualizing data written in HEALPix sky-tessellation scheme, and functionality for processing and organizing images to comply with the TOAST sky-tessellation scheme required for consumption by the World Wide Telescope (WWT). Four online tutorials enable readers to reproduce and extend all the visualizations presented in this paper.
△ Less
Submitted 8 February, 2017;
originally announced February 2017.
-
Implementing Ideas for Improving Software Citation and Credit
Authors:
Peter Teuben,
Alice Allen,
G. Bruce Berriman,
Kimberly DuPrie,
Jessica Mink,
Thomas Robitaille,
Keith Shortridge,
Mark Taylor,
Rein Warmels
Abstract:
Improving software citation and credit continues to be a topic of interest across and within many disciplines, with numerous efforts underway. In this Birds of a Feather (BoF) session, we started with a list of actionable ideas from last year's BoF and other similar efforts and worked alone or in small groups to begin implementing them. Work was captured in a common Google document; the session or…
▽ More
Improving software citation and credit continues to be a topic of interest across and within many disciplines, with numerous efforts underway. In this Birds of a Feather (BoF) session, we started with a list of actionable ideas from last year's BoF and other similar efforts and worked alone or in small groups to begin implementing them. Work was captured in a common Google document; the session organizers will disseminate or otherwise put this information to use in or for the community in collaboration with those who contributed.
△ Less
Submitted 18 November, 2016;
originally announced November 2016.
-
Astrophysics Source Code Library: Here we grow again!
Authors:
Alice Allen,
G. Bruce Berriman,
Kimberly DuPrie,
Jessica Mink,
Robert Nemiroff,
Thomas Robitaille,
Judy Schmidt,
Lior Shamir,
Keith Shortridge,
Mark Taylor,
Peter Teuben,
John Wallin
Abstract:
The Astrophysics Source Code Library (ASCL) is a free online registry of research codes; it is indexed by ADS and Web of Science and has over 1300 code entries. Its entries are increasingly used to cite software; citations have been doubling each year since 2012 and every major astronomy journal accepts citations to the ASCL. Codes in the resource cover all aspects of astrophysics research and man…
▽ More
The Astrophysics Source Code Library (ASCL) is a free online registry of research codes; it is indexed by ADS and Web of Science and has over 1300 code entries. Its entries are increasingly used to cite software; citations have been doubling each year since 2012 and every major astronomy journal accepts citations to the ASCL. Codes in the resource cover all aspects of astrophysics research and many programming languages are represented. In the past year, the ASCL added dashboards for users and administrators, started minting Digital Objective Identifiers (DOIs) for software it houses, and added metadata fields requested by users. This presentation covers the ASCL's growth in the past year and the opportunities afforded it as one of the few domain libraries for science research codes.
△ Less
Submitted 18 November, 2016;
originally announced November 2016.
-
The Astropy Problem
Authors:
Demitri Muna,
Michael Alexander,
Alice Allen,
Richard Ashley,
Daniel Asmus,
Ruyman Azzollini,
Michele Bannister,
Rachael Beaton,
Andrew Benson,
G. Bruce Berriman,
Maciej Bilicki,
Peter Boyce,
Joanna Bridge,
Jan Cami,
Eryn Cangi,
Xian Chen,
Nicholas Christiny,
Christopher Clark,
Michelle Collins,
Johan Comparat,
Neil Cook,
Darren Croton,
Isak Delberth Davids,
Éric Depagne,
John Donor
, et al. (129 additional authors not shown)
Abstract:
The Astropy Project (http://astropy.org) is, in its own words, "a community effort to develop a single core package for Astronomy in Python and foster interoperability between Python astronomy packages." For five years this project has been managed, written, and operated as a grassroots, self-organized, almost entirely volunteer effort while the software is used by the majority of the astronomical…
▽ More
The Astropy Project (http://astropy.org) is, in its own words, "a community effort to develop a single core package for Astronomy in Python and foster interoperability between Python astronomy packages." For five years this project has been managed, written, and operated as a grassroots, self-organized, almost entirely volunteer effort while the software is used by the majority of the astronomical community. Despite this, the project has always been and remains to this day effectively unfunded. Further, contributors receive little or no formal recognition for creating and supporting what is now critical software. This paper explores the problem in detail, outlines possible solutions to correct this, and presents a few suggestions on how to address the sustainability of general purpose astronomical software.
△ Less
Submitted 10 October, 2016;
originally announced October 2016.
-
The Next Generation of the Montage Image Mosaic Toolkit
Authors:
G. Bruce Berriman,
J. C. Good,
B. Rusholme,
T. Robitaille
Abstract:
The scientific computing landscape has evolved dramatically in the past few years, with new schemes for organizing and storing data that reflect the growth in size and complexity of astronomical data sets. In response to this changing landscape, we are, over the next two years, deploying the next generation of the Montage toolkit ([ascl:1010.036]). The first release (October 2015) supports multi-d…
▽ More
The scientific computing landscape has evolved dramatically in the past few years, with new schemes for organizing and storing data that reflect the growth in size and complexity of astronomical data sets. In response to this changing landscape, we are, over the next two years, deploying the next generation of the Montage toolkit ([ascl:1010.036]). The first release (October 2015) supports multi-dimensional data sets ("data cubes"), and insertion of XMP/AVM tags that allows images to "drop-in" to the WWT. The same release offers a beta-version of web-based interactive visualization of images; this includes wrappers for visualization in Python. Subsequent releases will support HEALPix (now standard in cosmic background experiments); incorporation of Montage into package managers (which enable automated management of software builds), and support for a library that will enable Montage to be called directly from Python. This next generation toolkit will inherit the architectural benefits of the current engine - component based tools, ANSI-C portability across Unix platforms and scalability for distributed processing. With the expanded functionality under development, Montage can be viewed not simply as a mosaic engine, but as a scalable, portable toolkit for managing, organizing and processing images.
△ Less
Submitted 8 August, 2016;
originally announced August 2016.
-
A case study in adaptable and reusable infrastructure at the Keck Observatory Archive: VO interfaces, moving targets, and more
Authors:
G. Bruce Berriman,
Richard W. Cohen,
Andrew Colson,
Christopher R. Gelino,
John C. Good,
Mihseh Kong,
Anastasia C. Laity,
Jeffrey A. Mader,
Melanie A. Swain,
Hien D. Tran,
Shin-Ywan Wang
Abstract:
This paper describes how the Keck Observatory Archive (KOA) is extending open source software components to develop new services. In August 2015, KOA deployed a program interface to discover public data from all instruments equipped with an imaging mode. The interface complies with version 2 of the Simple Imaging Access Protocol (SIAP), under development by the International Virtual Observatory Al…
▽ More
This paper describes how the Keck Observatory Archive (KOA) is extending open source software components to develop new services. In August 2015, KOA deployed a program interface to discover public data from all instruments equipped with an imaging mode. The interface complies with version 2 of the Simple Imaging Access Protocol (SIAP), under development by the International Virtual Observatory Alliance (IVOA), which defines a standard mechanism for discovering images through spatial queries. The heart of the KOA service is an R-tree-based, database-indexing mechanism prototyped by the Virtual Astronomical Observatory (VAO) and further developed by the Montage Image Mosaic project, designed to provide fast access to large imaging data sets as a first step in creating wide-area image mosaics. The KOA service uses the results of the spatial R-tree search to create an SQLite data database for further relational filtering. The service uses a JSON configuration file to describe the association between instrument parameters and the service query parameters, and to make it applicable beyond the Keck instruments.
The R-tree program was itself extended to support temporal (in addition to spatial) indexing, in response to requests from the planetary science community for a search engine to discover observations of Solar System objects. With this 3D-indexing scheme, the service performs very fast time and spatial matches between the target ephemerides, obtained from the JPL SPICE service. Our experiments indicate these matches can be more than 100 times faster than when separating temporal and spatial searches. Images of the tracks of the moving targets, overlaid with the image footprints, are computed with a new command-line visualization tool, mViewer, released with the Montage distribution. The service is currently in test and will be released in Fall 2016.
△ Less
Submitted 8 August, 2016;
originally announced August 2016.
-
Improving Software Citation and Credit
Authors:
Alice Allen,
G. Bruce Berriman,
Kimberly DuPrie,
Jessica Mink,
Robert Nemiroff,
Thomas Robitaille,
Lior Shamir,
Keith Shortridge,
Mark Taylor,
Peter Teuben,
John Wallin
Abstract:
The past year has seen movement on several fronts for improving software citation, including the Center for Open Science's Transparency and Openness Promotion (TOP) Guidelines, the Software Publishing Special Interest Group that was started at January's AAS meeting in Seattle at the request of that organization's Working Group on Astronomical Software, a Sloan-sponsored meeting at GitHub in San Fr…
▽ More
The past year has seen movement on several fronts for improving software citation, including the Center for Open Science's Transparency and Openness Promotion (TOP) Guidelines, the Software Publishing Special Interest Group that was started at January's AAS meeting in Seattle at the request of that organization's Working Group on Astronomical Software, a Sloan-sponsored meeting at GitHub in San Francisco to begin work on a cohesive research software citation-enabling platform, the work of Force11 to "transform and improve" research communication, and WSSSPE's ongoing efforts that include software publication, citation, credit, and sustainability.
Brief reports on these efforts were shared at the BoF, after which participants discussed ideas for improving software citation, generating a list of recommendations to the community of software authors, journal publishers, ADS, and research authors. The discussion, recommendations, and feedback will help form recommendations for software citation to those publishers represented in the Software Publishing Special Interest Group and the broader community.
△ Less
Submitted 24 December, 2015;
originally announced December 2015.
-
The First Data Release of the KODIAQ Survey
Authors:
J. M. O'Meara,
N. Lehner,
J. C. Howk,
J. X. Prochaska,
A. J. Fox,
M. A. Swain,
C. R. Gelino,
G. B. Berriman,
H. Tran
Abstract:
We present and make publicly available the first data release (DR1) of the Keck Observatory Database of Ionized Absorption toward Quasars (KODIAQ) survey. The KODIAQ survey is aimed at studying galactic and circumgalactic gas in absorption at high-redshift, with a focus on highly-ionized gas traced by OVI, using the HIRES spectrograph on the Keck-I telescope. KODIAQ DR1 consists of a fully-reduced…
▽ More
We present and make publicly available the first data release (DR1) of the Keck Observatory Database of Ionized Absorption toward Quasars (KODIAQ) survey. The KODIAQ survey is aimed at studying galactic and circumgalactic gas in absorption at high-redshift, with a focus on highly-ionized gas traced by OVI, using the HIRES spectrograph on the Keck-I telescope. KODIAQ DR1 consists of a fully-reduced sample of 170 quasars at 0.29 < z_em < 5.29 observed with HIRES at high resolution (36,000 <= R <= 103,000) between 2004 and 2012. DR1 contains 247 spectra available in continuum normalized form, representing a sum total exposure time of ~1.6 megaseconds. These co-added spectra arise from a total of 567 individual exposures of quasars taken from the Keck Observatory Archive (KOA) in raw form and uniformly processed using a HIRES data reduction package made available through the XIDL distribution. DR1 is publicly available to the community, housed as a higher level science product at the KOA. We will provide future data releases that make further QSOs, including those with pre-2004 observations taken with the previous-generation HIRES detectors.
△ Less
Submitted 13 May, 2015;
originally announced May 2015.
-
The Virtual Astronomical Observatory: Re-engineering Access to Astronomical Data
Authors:
R. J. Hanisch,
G. B. Berriman,
T. J. W. Lazio,
S. Emery Bunn,
J. Evans,
T. A. McGlynn,
R. Plante
Abstract:
The U.S. Virtual Astronomical Observatory was a software infrastructure and development project designed both to begin the establishment of an operational Virtual Observatory (VO) and to provide the U.S. coordination with the international VO effort. The concept of the VO is to provide the means by which an astronomer is able to discover, access, and process data seamlessly, regardless of its phys…
▽ More
The U.S. Virtual Astronomical Observatory was a software infrastructure and development project designed both to begin the establishment of an operational Virtual Observatory (VO) and to provide the U.S. coordination with the international VO effort. The concept of the VO is to provide the means by which an astronomer is able to discover, access, and process data seamlessly, regardless of its physical location. This paper describes the origins of the VAO, including the predecessor efforts within the U.S. National Virtual Observatory, and summarizes its main accomplishments. These accomplishments include the development of both scripting toolkits that allow scientists to incorporate VO data directly into their reduction and analysis environments and high-level science applications for data discovery, integration, analysis, and catalog cross-comparison. Working with the international community, and based on the experience from the software development, the VAO was a major contributor to international standards within the International Virtual Observatory Alliance. The VAO also demonstrated how an operational virtual observatory could be deployed, providing a robust operational environment in which VO services worldwide were routinely checked for aliveness and compliance with international standards. Finally, the VAO engaged in community outreach, developing a comprehensive web site with on-line tutorials, announcements, links to both U.S. and internationally developed tools and services, and exhibits and hands-on training .... All digital products of the VAO Project, including software, documentation, and tutorials, are stored in a repository for community access. The enduring legacy of the VAO is an increasing expectation that new telescopes and facilities incorporate VO capabilities during the design of their data management systems.
△ Less
Submitted 8 April, 2015;
originally announced April 2015.
-
Significant problems in FITS limit its use in modern astronomical research
Authors:
Brian Thomas,
Tim Jenness,
Frossie Economou,
Perry Greenfield,
Paul Hirst,
David S. Berry,
Erik Bray,
Norman Gray,
Demitri Muna James Turner,
Miguel de Val-Borro,
Juande Santander Vela,
David Shupe,
John Good,
G. Bruce Berriman
Abstract:
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard is, however, showing its age. Developed in the late 1970s the FITS authors made a number of implementation choices for the format that, while common at the time, are now seen to limit its utility wit…
▽ More
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard is, however, showing its age. Developed in the late 1970s the FITS authors made a number of implementation choices for the format that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not appreciate the challenges which we would be facing today in astronomical computing. Difficulties we now face include, but are not limited to, having to address the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets and the need to capture significantly more complex metadata and data relationships.
There are members of the community today who find some (or all) of these limitations unworkable, and have decided to move ahead with storing data in other formats. This reaction should be taken as a wakeup call to the FITS community to make changes in the FITS standard, or to see its usage fall. In this paper we detail some selected important problems which exist within the FITS standard today. It is not our intention to prescribe specific remedies to these issues; rather, we hope to call attention of the FITS and greater astronomical computing communities to these issues in the hopes that it will spur action to address them.
△ Less
Submitted 20 February, 2015;
originally announced February 2015.
-
Learning from FITS: Limitations in use in modern astronomical research
Authors:
Brian Thomas,
Tim Jenness,
Frossie Economou,
Perry Greenfield,
Paul Hirst,
David S. Berry,
Erik Bray,
Norman Gray,
Demitri Muna,
James Turner,
Miguel de Val-Borro,
Juande Santander-Vela,
David Shupe,
John Good,
G. Bruce Berriman,
Slava Kitaeff,
Jonathan Fay,
Omar Laurino,
Anastasia Alexov,
Walter Landry,
Joe Masters,
Adam Brazier,
Reinhold Schaaf,
Kevin Edwards,
Russell O. Redman
, et al. (13 additional authors not shown)
Abstract:
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard, however, is showing its age. Developed in the late 1970s, the FITS authors made a number of implementation choices that, while common at the time, are now seen to limit its utility with modern data.…
▽ More
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard, however, is showing its age. Developed in the late 1970s, the FITS authors made a number of implementation choices that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not anticipate the challenges which we are facing today in astronomical computing. Difficulties we now face include, but are not limited to, addressing the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets, and capturing significantly more complex metadata and data relationships.
There are members of the community today who find some or all of these limitations unworkable, and have decided to move ahead with storing data in other formats. If this fragmentation continues, we risk abandoning the advantages of broad interoperability, and ready archivability, that the FITS format provides for astronomy. In this paper we detail some selected important problems which exist within the FITS standard today. These problems may provide insight into deeper underlying issues which reside in the format and we provide a discussion of some lessons learned. It is not our intention here to prescribe specific remedies to these issues; rather, it is to call attention of the FITS and greater astronomical computing communities to these problems in the hope that it will spur action to address them.
△ Less
Submitted 10 February, 2015; v1 submitted 3 February, 2015;
originally announced February 2015.
-
Astrophysics Source Code Library Enhancements
Authors:
Robert J. Hanisch,
Alice Allen,
G. Bruce Berriman,
Kimberly DuPrie,
Jessica Mink,
Robert J. Nemiroff,
Judy Schmidt,
Lior Shamir,
Keith Shortridge,
Mark Taylor,
Peter J. Teuben,
John Wallin
Abstract:
The Astrophysics Source Code Library (ASCL; ascl.net) is a free online registry of codes used in astronomy research; it currently contains over 900 codes and is indexed by ADS. The ASCL has recently moved a new infrastructure into production. The new site provides a true database for the code entries and integrates the WordPress news and information pages and the discussion forum into one site. Pr…
▽ More
The Astrophysics Source Code Library (ASCL; ascl.net) is a free online registry of codes used in astronomy research; it currently contains over 900 codes and is indexed by ADS. The ASCL has recently moved a new infrastructure into production. The new site provides a true database for the code entries and integrates the WordPress news and information pages and the discussion forum into one site. Previous capabilities are retained and permalinks to ascl.net continue to work. This improvement offers more functionality and flexibility than the previous site, is easier to maintain, and offers new possibilities for collaboration. This presentation covers these recent changes to the ASCL.
△ Less
Submitted 7 November, 2014;
originally announced November 2014.
-
Metadata and Data Management for the Keck Observatory Archive
Authors:
H. D. Tran,
J. Holt,
R. W. Goodrich,
J. A. Mader,
M. Swain,
A. C. Laity,
M. Kong,
C. R. Gelino,
G. B. Berriman
Abstract:
A collaboration between the W. M. Keck Observatory (WMKO) in Hawaii and the NASA Exoplanet Science Institute (NExScI) in California, the Keck Observatory Archive (KOA) was commissioned in 2004 to archive observing data from WMKO, which operates two classically scheduled 10 m ground-based telescopes. The observing data from Keck is not suitable for direct ingestion into the archive since the metada…
▽ More
A collaboration between the W. M. Keck Observatory (WMKO) in Hawaii and the NASA Exoplanet Science Institute (NExScI) in California, the Keck Observatory Archive (KOA) was commissioned in 2004 to archive observing data from WMKO, which operates two classically scheduled 10 m ground-based telescopes. The observing data from Keck is not suitable for direct ingestion into the archive since the metadata contained in the original FITS headers lack the information necessary for proper archiving. Coupled with different standards among instrument builders and the heterogeneous nature of the data inherent in classical observing, in which observers have complete control of the instruments and their observations, the data pose a number of technical challenges for KOA. We describe the methodologies and tools that we have developed to successfully address these difficulties, adding content to the FITS headers and "retrofitting" the metadata in order to support archiving Keck data, especially those obtained before the archive was designed. With the expertise gained from having successfully archived observations taken with all eight currently active instruments at WMKO, we have developed lessons learned from handling this complex array of heterogeneous metadata that help ensure a smooth ingestion of data not only for current but also future instruments, as well as a better experience for the archive user.
△ Less
Submitted 21 August, 2014;
originally announced August 2014.
-
The Design and Operation of The Keck Observatory Archive
Authors:
G. Bruce Berriman,
Christopher R. Gelino,
Robert W. Goodrich,
Jennifer Holt,
Mihseh Kong,
Anastasia C. Laity,
Jeffrey A. Mader,
Melanie Swain,
Hien D. Tran
Abstract:
The Infrared Processing and Analysis Center (IPAC) and the W. M. Keck Observatory (WMKO) operate an archive for the Keck Observatory. At the end of 2013, KOA completed the ingestion of data from all eight active observatory instruments. KOA will continue to ingest all newly obtained observations, at an anticipated volume of 4 TB per year. The data are transmitted electronically from WMKO to IPAC f…
▽ More
The Infrared Processing and Analysis Center (IPAC) and the W. M. Keck Observatory (WMKO) operate an archive for the Keck Observatory. At the end of 2013, KOA completed the ingestion of data from all eight active observatory instruments. KOA will continue to ingest all newly obtained observations, at an anticipated volume of 4 TB per year. The data are transmitted electronically from WMKO to IPAC for storage and curation. Access to data is governed by a data use policy, and approximately two-thirds of the data in the archive are public.
△ Less
Submitted 4 August, 2014;
originally announced August 2014.
-
Creating A Galactic Plane Atlas With Amazon Web Services
Authors:
G. Bruce Berriman,
Ewa Deelman,
John Good,
Gideon Juve,
Jamie Kinney,
Ann Merrihew,
Mats Rynge
Abstract:
This paper describes by example how astronomers can use cloud-computing resources offered by Amazon Web Services (AWS) to create new datasets at scale. We have created from existing surveys an atlas of the Galactic Plane at 16 wavelengths from 1 μm to 24 μm with pixels co-registered at spatial sampling of 1 arcsec. We explain how open source tools support management and operation of a virtual clus…
▽ More
This paper describes by example how astronomers can use cloud-computing resources offered by Amazon Web Services (AWS) to create new datasets at scale. We have created from existing surveys an atlas of the Galactic Plane at 16 wavelengths from 1 μm to 24 μm with pixels co-registered at spatial sampling of 1 arcsec. We explain how open source tools support management and operation of a virtual cluster on AWS platforms to process data at scale, and describe the technical issues that users will need to consider, such as optimization of resources, resource costs, and management of virtual machine instances.
△ Less
Submitted 23 December, 2013;
originally announced December 2013.
-
The NASA Exoplanet Archive: Data and Tools for Exoplanet Research
Authors:
R. L. Akeson,
X. Chen,
D. Ciardi,
M. Crane,
J. Good,
M. Harbut,
E. Jackson,
S. R. Kane,
A. C. Laity,
S. Leifer,
M. Lynn,
D. L. McElroy,
M. Papin,
P. Plavchan,
S. V. Ramirez,
R. Rey,
K. von Braun,
M. Wittman,
M. Abajian,
B. Ali,
C. Beichman,
A. Beekley,
G. B. Berriman,
S. Berukoff,
G. Bryden
, et al. (10 additional authors not shown)
Abstract:
We describe the contents and functionality of the NASA Exoplanet Archive, a database and tool set funded by NASA to support astronomers in the exoplanet community. The current content of the database includes interactive tables containing properties of all published exoplanets, Kepler planet candidates, threshold-crossing events, data validation reports and target stellar parameters, light curves…
▽ More
We describe the contents and functionality of the NASA Exoplanet Archive, a database and tool set funded by NASA to support astronomers in the exoplanet community. The current content of the database includes interactive tables containing properties of all published exoplanets, Kepler planet candidates, threshold-crossing events, data validation reports and target stellar parameters, light curves from the Kepler and CoRoT missions and from several ground-based surveys, and spectra and radial velocity measurements from the literature. Tools provided to work with these data include a transit ephemeris predictor, both for single planets and for observing locations, light curve viewing and normalization utilities, and a periodogram and phased light curve service. The archive can be accessed at http://exoplanetarchive.ipac.caltech.edu.
△ Less
Submitted 10 July, 2013;
originally announced July 2013.
-
A Tale Of 160 Scientists, Three Applications, A Workshop and A Cloud
Authors:
G. Bruce Berriman,
Carolyn Brinkworth,
Dawn Gelino,
Dennis K. Wittman,
Ewa Deelman,
Gideon Juve,
Mats Rynge,
Jamie Kinney
Abstract:
The NASA Exoplanet Science Institute (NExScI) hosts the annual Sagan Workshops, thematic meetings aimed at introducing researchers to the latest tools and methodologies in exoplanet research. The theme of the Summer 2012 workshop, held from July 23 to July 27 at Caltech, was to explore the use of exoplanet light curves to study planetary system architectures and atmospheres. A major part of the wo…
▽ More
The NASA Exoplanet Science Institute (NExScI) hosts the annual Sagan Workshops, thematic meetings aimed at introducing researchers to the latest tools and methodologies in exoplanet research. The theme of the Summer 2012 workshop, held from July 23 to July 27 at Caltech, was to explore the use of exoplanet light curves to study planetary system architectures and atmospheres. A major part of the workshop was to use hands-on sessions to instruct attendees in the use of three open source tools for the analysis of light curves, especially from the Kepler mission. Each hands-on session involved the 160 attendees using their laptops to follow step-by-step tutorials given by experts. We describe how we used the Amazon Elastic Cloud 2 to run these applications.
△ Less
Submitted 16 November, 2012;
originally announced November 2012.
-
Running a distributed virtual observatory: US Virtual Astronomical Observatory operations
Authors:
Thomas A. McGlynn,
Robert J. Hanisch,
G. Bruce Berriman,
Aniruddha R. Thakar
Abstract:
Operation of the US Virtual Astronomical Observatory shares some issues with modern physical observatories, e.g., intimidating data volumes and rapid technological change, and must also address unique concerns like the lack of direct control of the underlying and scattered data resources, and the distributed nature of the observatory itself. In this paper we discuss how the VAO has addressed these…
▽ More
Operation of the US Virtual Astronomical Observatory shares some issues with modern physical observatories, e.g., intimidating data volumes and rapid technological change, and must also address unique concerns like the lack of direct control of the underlying and scattered data resources, and the distributed nature of the observatory itself. In this paper we discuss how the VAO has addressed these challenges to provide the astronomical community with a coherent set of science-enabling tools and services. The distributed nature of our virtual observatory-with data and personnel spanning geographic, institutional and regime boundaries-is simultaneously a major operational headache and the primary science motivation for the VAO. Most astronomy today uses data from many resources. Facilitation of matching heterogeneous datasets is a fundamental reason for the virtual observatory. Key aspects of our approach include continuous monitoring and validation of VAO and VO services and the datasets provided by the community, monitoring of user requests to optimize access, caching for large datasets, and providing distributed storage services that allow user to collect results near large data repositories. Some elements are now fully implemented, while others are planned for subsequent years. The distributed nature of the VAO requires careful attention to what can be a straightforward operation at a conventional observatory, e.g., the organization of the web site or the collection and combined analysis of logs. Many of these strategies use and extend protocols developed by the international virtual observatory community.
△ Less
Submitted 20 June, 2012;
originally announced June 2012.
-
The organization and management of the Virtual Astronomical Observatory
Authors:
G. Bruce Berriman,
Robert J. Hanisch,
T. Joseph W. Lazio,
Alexander Szalay,
Giussepina Fabbiano
Abstract:
The U.S. Virtual Astronomical Observatory (VAO; http://www.us-vao.org/) has been in operation since May 2010. Its goal is to enable new science through efficient integration of distributed multi-wavelength data. This paper describes the management and organization of the VAO, and emphasizes the techniques used to ensure efficiency in a distributed organization. Management methods include using an…
▽ More
The U.S. Virtual Astronomical Observatory (VAO; http://www.us-vao.org/) has been in operation since May 2010. Its goal is to enable new science through efficient integration of distributed multi-wavelength data. This paper describes the management and organization of the VAO, and emphasizes the techniques used to ensure efficiency in a distributed organization. Management methods include using an annual program plan as the basis for establishing contracts with member organizations, regular communication, and monitoring of processes.
△ Less
Submitted 18 June, 2012;
originally announced June 2012.
-
The role in the Virtual Astronomical Observatory in the era of massive data sets
Authors:
G. Bruce Berriman,
Robert J. Hanisch,
T. Joseph W. Lazio
Abstract:
The Virtual Observatory (VO) is realizing global electronic integration of astronomy data. One of the long-term goals of the U.S. VO project, the Virtual Astronomical Observatory (VAO), is development of services and protocols that respond to the growing size and complexity of astronomy data sets. This paper describes how VAO staff are active in such development efforts, especially in innovative s…
▽ More
The Virtual Observatory (VO) is realizing global electronic integration of astronomy data. One of the long-term goals of the U.S. VO project, the Virtual Astronomical Observatory (VAO), is development of services and protocols that respond to the growing size and complexity of astronomy data sets. This paper describes how VAO staff are active in such development efforts, especially in innovative strategies and techniques that recognize the limited operating budgets likely available to astronomers even as demand increases. The project has a program of professional outreach whereby new services and protocols are evaluated.
△ Less
Submitted 18 June, 2012;
originally announced June 2012.
-
How Will Astronomy Archives Survive The Data Tsunami?
Authors:
G. Bruce Berriman,
Steven L. Groom
Abstract:
The field of astronomy is starting to generate more data than can be managed, served and processed by current techniques. This paper has outlined practices for developing next-generation tools and techniques for surviving this data tsunami, including rigorous evaluation of new technologies, partnerships between astronomers and computer scientists, and training of scientists in high-end software en…
▽ More
The field of astronomy is starting to generate more data than can be managed, served and processed by current techniques. This paper has outlined practices for developing next-generation tools and techniques for surviving this data tsunami, including rigorous evaluation of new technologies, partnerships between astronomers and computer scientists, and training of scientists in high-end software engineering engineering skills.
△ Less
Submitted 31 October, 2011;
originally announced November 2011.
-
The NStED Periodogram Service and Interface for Public CoRoT Data
Authors:
K. von Braun,
M. Abajian,
A. Beekley,
G. B. Berriman,
G. Bryden,
B. Chan,
D. R. Ciardi,
J. Good,
M. Harbut,
S. R. Kane,
A. Laity,
C. Lau,
M. Lynn,
D. McElroy,
P. Plavchan,
M. Regelson,
R. Rey,
S. V. Ramirez,
J. Stauffer,
A. Zhang
Abstract:
As part of the NASA-CNES agreement, the NASA Star and Exoplanet Database (NStED) serves as the official US portal for the public CoRoT data products. NStED is a general purpose archive with the aim of providing support for NASA's planet finding and characterization goals. Consequently, the NASA Exoplanet Science Institute (NExScI) developed, and NStED adapted, a periodogram service for CoRoT data…
▽ More
As part of the NASA-CNES agreement, the NASA Star and Exoplanet Database (NStED) serves as the official US portal for the public CoRoT data products. NStED is a general purpose archive with the aim of providing support for NASA's planet finding and characterization goals. Consequently, the NASA Exoplanet Science Institute (NExScI) developed, and NStED adapted, a periodogram service for CoRoT data to determine periods of variability phenomena and create phased photometric light curves. Through the NStED periodogram interface, the user may choose three different period detection algorithms to use on any photometric time series product, or even upload and analyze their own data. Additionally, the NStED periodogram is remotely accessed by the CoRoT archive as part of its interface. NStED is available at {\bf http://nsted.ipac.caltech.edu}.
△ Less
Submitted 11 July, 2011;
originally announced July 2011.
-
A Cross-Match of 2MASS and SDSS. II. Peculiar L Dwarfs, Unresolved Binaries, and the Space Density of T Dwarf Secondaries
Authors:
Kerstin Geißler,
Stanimir Metchev,
J. Davy Kirkpatrick,
G. Bruce Berriman,
Dagny Looper
Abstract:
We present the completion of a program to cross-correlate the SDSS Data Release 1 and 2MASS Point Source Catalog in search for extremely red L and T dwarfs. The program was initiated by Metchev and collaborators, who presented the findings on all newly identified T dwarfs in SDSS DR1, and estimated the space density of isolated T0--T8 dwarfs in the solar neighbourhood. In the current work we prese…
▽ More
We present the completion of a program to cross-correlate the SDSS Data Release 1 and 2MASS Point Source Catalog in search for extremely red L and T dwarfs. The program was initiated by Metchev and collaborators, who presented the findings on all newly identified T dwarfs in SDSS DR1, and estimated the space density of isolated T0--T8 dwarfs in the solar neighbourhood. In the current work we present most of the L dwarf discoveries. Our red-sensitive (z-J > 2.75 mag) cross-match proves to be efficient in detecting peculiarly red L dwarfs, adding two new ones, including one of the reddest known L dwarfs. Our search also nets a new peculiarly blue L7 dwarf and, surprisingly, two M8 dwarfs. We further broaden our analysis to detect unresolved binary L or T dwarfs through spectral template fitting to all L and T dwarfs presented here and in the earlier work by Metchev and collaborators. We identify nine probable binaries, six of which are new and eight harbour likely T dwarf secondaries. We combine this result with current knowledge of the mass ratio distribution and frequency of substellar companions to estimate an overall space density of 0.005--0.05 pc^{-3} for individual T0--T8 dwarfs.
△ Less
Submitted 6 March, 2011;
originally announced March 2011.
-
Collaborative Astronomical Image Mosaics
Authors:
Daniel S. Katz,
G. Bruce Berriman,
Robert G. Mann
Abstract:
This chapter describes how astronomical imaging survey data have become a vital part of modern astronomy, how these data are archived and then served to the astronomical community through on-line data access portals. The Virtual Observatory, now under development, aims to make all these data accessible through a uniform set of interfaces. This chapter also describes the scientific need for one com…
▽ More
This chapter describes how astronomical imaging survey data have become a vital part of modern astronomy, how these data are archived and then served to the astronomical community through on-line data access portals. The Virtual Observatory, now under development, aims to make all these data accessible through a uniform set of interfaces. This chapter also describes the scientific need for one common image processing task, that of composing individual images into large scale mosaics and introduces Montage as a tool for this task. Montage, as distributed, can be used in four ways: as a single thread/process on a single CPU, in parallel using MPI to distribute similar tasks across a parallel computer, in parallel using grid tools (Pegasus/DAGMan) to distributed tasks across a grid, or in parallel using a script-driven approach (Swift). An on-request web based Montage service is available for users who do not need to build a local version. We also introduce some work on a new scripted version of Montage, which offers ease of customization for users. Then, we discuss various ideas where Web 2.0 technologies can help the Montage community.
△ Less
Submitted 23 November, 2010;
originally announced November 2010.
-
The Application of Cloud Computing to Astronomy: A Study of Cost and Performance
Authors:
G. Bruce Berriman,
Ewa Deelman,
Gideon Juve,
Moira Regelson,
Peter Plavchan
Abstract:
Cloud computing is a powerful new technology that is widely used in the business world. Recently, we have been investigating the benefits it offers to scientific computing. We have used three workflow applications to compare the performance of processing data on the Amazon EC2 cloud with the performance on the Abe high-performance cluster at the National Center for Supercomputing Applications (NCS…
▽ More
Cloud computing is a powerful new technology that is widely used in the business world. Recently, we have been investigating the benefits it offers to scientific computing. We have used three workflow applications to compare the performance of processing data on the Amazon EC2 cloud with the performance on the Abe high-performance cluster at the National Center for Supercomputing Applications (NCSA). We show that the Amazon EC2 cloud offers better performance and value for processor- and memory-limited applications than for I/O-bound applications. We provide an example of how the cloud is well suited to the generation of a science product: an atlas of periodograms for the 210,000 light curves released by the NASA Kepler Mission. This atlas will support the identification of periodic signals, including those due to transiting exoplanets, in the Kepler data sets.
△ Less
Submitted 22 October, 2010;
originally announced October 2010.
-
The Application of Cloud Computing to the Creation of Image Mosaics and Management of Their Provenance
Authors:
G. Bruce Berriman,
Ewa Deelman,
Paul Groth,
Gideon Juve
Abstract:
We have used the Montage image mosaic engine to investigate the cost and performance of processing images on the Amazon EC2 cloud, and to inform the requirements that higher-level products impose on provenance management technologies. We will present a detailed comparison of the performance of Montage on the cloud and on the Abe high performance cluster at the National Center for Supercomputing Ap…
▽ More
We have used the Montage image mosaic engine to investigate the cost and performance of processing images on the Amazon EC2 cloud, and to inform the requirements that higher-level products impose on provenance management technologies. We will present a detailed comparison of the performance of Montage on the cloud and on the Abe high performance cluster at the National Center for Supercomputing Applications (NCSA). Because Montage generates many intermediate products, we have used it to understand the science requirements that higher-level products impose on provenance management technologies. We describe experiments with provenance management technologies such as the "Provenance Aware Service Oriented Architecture" (PASOA).
△ Less
Submitted 24 June, 2010;
originally announced June 2010.
-
Montage: a grid portal and software toolkit for science-grade astronomical image mosaicking
Authors:
Joseph C. Jacob,
Daniel S. Katz,
G. Bruce Berriman,
John Good,
Anastasia C. Laity,
Ewa Deelman,
Carl Kesselman,
Gurmeet Singh,
Mei-Hui Su,
Thomas A. Prince,
Roy Williams
Abstract:
Montage is a portable software toolkit for constructing custom, science-grade mosaics by composing multiple astronomical images. The mosaics constructed by Montage preserve the astrometry (position) and photometry (intensity) of the sources in the input images. The mosaic to be constructed is specified by the user in terms of a set of parameters, including dataset and wavelength to be used, locati…
▽ More
Montage is a portable software toolkit for constructing custom, science-grade mosaics by composing multiple astronomical images. The mosaics constructed by Montage preserve the astrometry (position) and photometry (intensity) of the sources in the input images. The mosaic to be constructed is specified by the user in terms of a set of parameters, including dataset and wavelength to be used, location and size on the sky, coordinate system and projection, and spatial sampling rate. Many astronomical datasets are massive, and are stored in distributed archives that are, in most cases, remote with respect to the available computational resources. Montage can be run on both single- and multi-processor computers, including clusters and grids. Standard grid tools are used to run Montage in the case where the data or computers used to construct a mosaic are located remotely on the Internet. This paper describes the architecture, algorithms, and usage of Montage as both a software toolkit and as a grid portal. Timing results are provided to show how Montage performance scales with number of processors on a cluster computer. In addition, we compare the performance of two methods of running Montage in parallel on a grid.
△ Less
Submitted 24 May, 2010;
originally announced May 2010.
-
The Role of Provenance Management in Accelerating the Rate of Astronomical Research
Authors:
G. Bruce Berriman,
Ewa Deelman
Abstract:
The availability of vast quantities of data through electronic archives has transformed astronomical research. It has also enabled the creation of new products, models and simulations, often from distributed input data and models, that are themselves made electronically available. These products will only provide maximal long-term value to astronomers when accompanied by records of their provenanc…
▽ More
The availability of vast quantities of data through electronic archives has transformed astronomical research. It has also enabled the creation of new products, models and simulations, often from distributed input data and models, that are themselves made electronically available. These products will only provide maximal long-term value to astronomers when accompanied by records of their provenance; that is, records of the data and processes used in the creation of such products. We use the creation of image mosaics with the Montage grid-enabled mosaic engine to emphasize the necessity of provenance management and to understand the science requirements that higher-level products impose on provenance management technologies. We describe experiments with one technology, the "Provenance Aware Service Oriented Architecture" (PASOA), that stores provenance information at each step in the computation of a mosaic. The results inform the technical specifications of provenance management systems, including the need for extensible systems built on common standards. Finally, we describe examples of provenance management technology emerging from the fields of geophysics and oceanography that have applicability to astronomy applications.
△ Less
Submitted 19 May, 2010;
originally announced May 2010.
-
The LAEX and NASA portals for CoRoT public data
Authors:
E. Solano,
K. von Braun,
A. Velasco,
D. R. Ciardi,
R. Gutierrez,
D. L. McElroy,
M. Lopez,
M. Abajian,
M. Garcia,
B. Ali,
L. M. Sarro,
G. B. Berriman,
G. Bryden,
B. Chan,
J. Good,
S. R. Kane,
A. C. Laity,
C. Lau,
A. N. Payne,
P. Plavchan,
S. Ramirez,
M. Schmitz,
J. R. Stauffer,
P. L. Wyatt,
A. Zhang
Abstract:
* Aims. We describe here the main functionalities of the LAEX (Laboratorio de Astrofisica Estelar y Exoplanetas/Laboratory for Stellar Astrophysics and Exoplanets) and NASA portals for CoRoT Public Data. The CoRoT archive at LAEX was opened to the community in January 2009 and is managed in the framework of the Spanish Virtual Observatory. NStED (NASA Star and Exoplanet Database) serves as the C…
▽ More
* Aims. We describe here the main functionalities of the LAEX (Laboratorio de Astrofisica Estelar y Exoplanetas/Laboratory for Stellar Astrophysics and Exoplanets) and NASA portals for CoRoT Public Data. The CoRoT archive at LAEX was opened to the community in January 2009 and is managed in the framework of the Spanish Virtual Observatory. NStED (NASA Star and Exoplanet Database) serves as the CoRoT portal for the US astronomical community. NStED is a general purpose stellar and exoplanet archive with the aim of providing support for NASA planet finding and characterisation goals, and the planning and support of NASA and other space missions. CoRoT data at LAEX and NStED can be accessed at http://sdc.laeff.inta.es/corotfa/ and http://nsted.ipac.caltech.edu,respectively.
* Methods. Based on considerable experience with astronomical archives, the aforementioned archives are designed with the aim of delivering science-quality data in a simple and efficient way.
* Results. LAEX and NStED not only provide access to CoRoT Public Data but furthermore serve a variety of observed and calculated astrophysical data. In particular, NStED provides scientifically validated information on stellar and planetary data related to the search for and characterization of extrasolar planets, and LAEX makes any information from Virtual Observatory services available to the astronomical community.
△ Less
Submitted 20 July, 2009;
originally announced July 2009.
-
The NASA/IPAC/NExScI Star and Exoplanet Database
Authors:
G. B. Berriman,
B. Ali,
R. Baker,
K. von Braun,
N-M. Chiu,
D. R. Ciardi,
J. Good,
S. R. Kane,
M. Kong,
A. C. Laity,
D. L. McElroy,
S. Monkewitz,
A. N. Payne,
S. Ramirez,
M. Schmitz,
J. S. Stauffer,
P. L. Wyatt
Abstract:
The NASA/IPAC/NExScI Star and Exoplanet Database (NStED) is a general purpose stellar archive which supports NASA planet-finding and planet-characterization goals, stellar astrophysics, and the planning of NASA and other space missions. There are two principal components of NStED: a database of 140,000 nearby stars and exoplanet-hosting stars, and an archive dedicated to high precision photometr…
▽ More
The NASA/IPAC/NExScI Star and Exoplanet Database (NStED) is a general purpose stellar archive which supports NASA planet-finding and planet-characterization goals, stellar astrophysics, and the planning of NASA and other space missions. There are two principal components of NStED: a database of 140,000 nearby stars and exoplanet-hosting stars, and an archive dedicated to high precision photometric surveys for transiting exoplanets (NStED-ETSS). We present summaries of these components. The NStED stellar database currently serves published parameters for 140,000 stars. These parameters include coordinates, multiplicity, proper motion, parallax, spectral type, multiband photometry, radial velocity, metallicity, chromospheric and coronal activity index, rotation velocity/period, infrared excess. NStED-ETSS currently serves data from the TrES survey of the Kepler field as well as dedicated photometric surveys of four stellar clusters. NStED-ETSS aims to serve both the surveys and the broader astronomical community by archiving these data and making them available in a homogeneous format.
△ Less
Submitted 13 March, 2009;
originally announced March 2009.