Getting DYAMOND Summer data

Since the DYAMOND data sets contain hundreds of terabytes of data, most of the data is archived in DKRZ’s HSM tape archive. For a smooth and convenient provision of data sets, a data library, called ”Dyamond data library”, serves as a central point for data access. It is situated at the DKRZ’s Mistral supercomputer and can be found at /work/bk1040/DYAMOND/.

1) Get/check your user account at DKRZ:

To access the DYAMOND data library you will need a user account for the project “1153”, named “DYAMOND Data Intercomparison” at DKRZ. To do so, please visit https://luv.dkrz.de. For more information about how to start at DKRZ see: https://www.dkrz.de/up/systems/mistral/hpc-concepts.

After you get a confirming e-mail you are able to login to mistral ( e.g. ssh <user-account>@mistral.dkrz.de). The DYAMOND project can be found under /work/bb1153/.

2) Use the DYAMOND data library:

The archived data sets can be searched within the library and and requests for required data tarballs can be made. These files will then be downloaded for you to the folder data/summer_data/<model> by the DYAMOND coordination team and will be accessible to all DYAMOND users. There, also already requested files can be found. A detailed description how to use the DYAMOND data library is given in the README file /work/bk1040/DYAMOND/README.

The data sets will be read-only, so you will have to define the respective subfolder as the source directory for your post-processing scripts in your home/ or scratch/ folder. Since the quota of your home directory is limited to 24 GiB, we recommend to store project data under /work/bb1153/<user-account>/ and use the temporary storage in your scratch folder /scratch/*/<user-account>/ for processing of large data sets.

In order not to blow up the amount of memory used in this project, unused data sets will be deleted from the project folder on a regular basis.

For any questions, please contact the ESiWACE coordination team.

Important

If you work with the data stored at DKRZ, we request that you acknowledge DKRZ and ESiWACE in any publication resulting from this work.