Cosmology Data Repository
The Cosmology Data Repository curates publicly available cosmology datasets and co-locates them at NERSC alongside the computing resources necessary to analyze them. This enables work such as multimodal AI model training, which requires simultanous access to petabyte-scale datasets and supercomputer-level resources.
Flagship Datasets
DESI Imaging Legacy Surveys
The DESI Imaging Legacy Surveys jointly fit imaging data from multiple ground- and space-based telescopes to generate a catalog of 2.8 billion objects measured in 4 optical bands (g,r,i,z) and 4 infrared bands spanning over 20k square degrees.
DESI Spectroscopy
The Dark Energy Spectroscopic Instrument (DESI) Data Release 1 includes spectra and redshifts of of 18 million galaxies, quasars, and stars.
Multimodal Universe
The Multimodal Universe (MMU) gathered 100 TB of data from 20 major astronomical surveys into a common structure to simplify training AI foundation models.
AION-1
The AstronomIcal Omnimodal Network (AION-1) is an AI foundation model built upon the Multimodal Universe dataset.
Additional Datasets
Other datasets in the Cosmology Data Repository include 2mass, CFHT-Luau, Gaia, galex, WISE, SCUSS, SDSS, tycho2, UCAC, WISE, and ZTF lightcurves.
Data Access
The primary purpose of the Cosmology Data Repository is to provide
large scale direct access to the files at NERSC located at
/dvs_ro/cfs/cdirs/cosmo/data/ .
For browsing the directory structure and downloading individual
files, these are also available at
https://portal.nersc.gov/project/cosmo/data/.
For bulk downloads, users are encouraged to get the data from the original sources when possible, though they are also available for download via the NERSC DTN Globus endpoint. A valid NERSC login account is needed to access this endpoint. In addition, a fully public NERSC Cosmology Data Repository endpoint is available for use without a NERSC account.
In the future, we will coordinate with the American Science Cloud (AmSC) to provide these datasets within the AmSC infrastructure as part of the Genesis Mission.
Partners
