DYAMOND Winter#

Simulation Period:

20 January 2020 - 1 March 2020

with 10 day spin-up peripode and 30 days analysis periode

Field Experiment:

align with the EUREC4a field study.

Experiment:

Global atmosphere-only and coupled atmosphere-ocean models with a storm resolving grid spacing of 5 km or less. Detailed information can be found in the DYAMOND Winter protocol (pdf).

Initital data is provided at the ESiWACE webpage.

Completion of the simulation had been done untill summer 2022.

Participating models and description#

The DYAMOND coordination team at DKRZ was so kind to standardize most of the data sets to simplify their analysis and ease comparisons. As we are still in the process of standardizing FILE NAMES CAN CHANGE AT ANY TIME WITHOUT PRIOR WARNING. Please have a look at the overview of the processed variables for variable names and data availability.

Grid information

For several models, the grid information is not included in the output. Instead it can be found in a grid.nc file. You can find this grid file in the frequency directory fx/ of the directory structure of the data, e.g. /fastdata/ka1081/DYAMOND/data/winter_data/DYAMOND_WINTER/NOAA/SHiELD-3km/DW-ATM/atmos/fx/gn/grid.nc for the SHiELD data.

To associate data with the grid information, cdo will need a -setgrid,GRIDFILENAME, e.g.

cdo -sellonlatbox,0,20,40,60 -setgrid,GRIDFILENAME INFILE OUTFILE

Getting the data#

Since the DYAMOND data sets form hundreds of terabytes of data, only parts are available on disk. To retrive DYAMOND Winter data files from tape, DKRZ provides the Python tool outtake. It is setup to download the files (with read only permissions) to /fastdata/ka1081/DYAMOND/data/winter_data, thereby making it accessible to all DYAMOND users. Examples how to use it in a Python notebook or with the command-line script find_files can be found at Loading data from the catalog and Getting file names from intake with a command-line python script. Unused data files will be deleted on a regular basis.

Working with the data#

You are most invited to use our systems for analyzing the data (instead of copying them around). Our post-processing project 1153 has some compute resources that can be used for analysis scripts. Therefore, we recommend to store project data under /work/bb1153/<user-id>/ and use the temporary storage in your scratch folder /scratch/*/<user-id> for processing of large data sets. For more information about the Levantes file systhem its quotas and backups, see The file systems of Levante (docs.dkrz.de).

For model and run descriptions see Participating models and description above.

Using DKRZs JupyterHub and intake-esm#

DKRZ provides a JupytherHub server to analyse the big data sets stored at DKRZ by using Python, R or Julia scripts running directly on Levante. For further information visit our page DKRZs JupyterHub.

All post-processed DYAMOND Winter data sets are included in the intake-esm catalog of the DYAMOND and nextGEMS project: /work/ka1081/Catalogs/dyamond-nextgems.json. Intake-esm is a Python library that allows you to easily access data from a variety of simulations in a consistant manner. Take a look our various Pythons scripts!

Direct file access#

All stadardized data sets can be found in The DYAMOND Data Library at /fastdata/ka1081/DYAMOND/data/winter_data/ stored at the DKRZ’s Levante supercomputer. The data sets are read-only, so you will have to define the respective subfolder as the source directory for your post-processing scripts.

Browse the data by using our intake-esm catalog#

All post-processed DYAMOND Winter data sets are included in the intake-esm catalog of the DYAMOND and nextGEMS project: /work/ka1081/Catalogs/dyamond-nextgems.json.

Browse the data by using GEMS#

For an overview of the processed files, please have a look at the GEMS webfrontend (use your mistral ID or the “guest” button to log in).

To browse the files in the shell, you can use the freva command line interface:

module load clint gems

# use tab-completion
freva databrowser project=dyamond_winter experiment= TAB TAB

# all surface air temperature files for ifs-4km with 1hr time step
freva databrowser project=dyamond_winter product=dw-cpl model=ifs-4km variable=tas time_frequency=1hr

# call cdo on multiple files
cdo -sinfov $(freva databrowser project=dyamond_winter product=dw-atm model=arpege-nh-2km variable=rst)

See also freva databrowser --help and the comand given at the GEMS webfrontend.

To brows the files in a Python notebook, see the example on the freva documentation webpage.