nextGEMS simulations#

nextGEMS is building prototypes for a new generation of earth system models to advance science, guide policy, and inform applications to support the sustainable management of our planet. Today’s Earth System Models use grids designed to capture the horizontal, but not the vertical, motion fields of atmospheric circulations. This shortcut, while computationally expedient, requires important processes to be neglected, or represented empirically. By using a fifty-fold finer horizontal grid (3 km compared to the 150 km) storm resolving models are able to explicitly represent the circulations that make the storms in the atmosphere, the eddies in the ocean, and cracks in the ice, a leap in realism that is only now becoming possible thanks to advances in super-computing.

nextGEMS drives the development of two European storm-resolving Earth-system Models, IFS and ICON. The models undergo three development cycles with successive improvement and optimization of the model configuration and workflow. Below we provide information on the model development cycles and their respective simulations at various horizontal resolutions and changing model configuration. We intend to provide information as soon as they become available. However, this is by no means a complete model description but shall rather provide a quick overview of available experiments. Also, please keep in mind that the models are under constant change during the development phase. We try to keep track for you here. The following links point to relevant background knowledge on the models:

Icosahedral Nonhydrostatic Weather and Climate Model (ICON)
The ECMWF Integrated Forecasting System (IFS)

IFS-H is a hydrostatic spectral transform model with a semi-Lagrangian semi-implicit solution procedure, discretised on a cubic octahedral grid (Wedi 2014, Malardel et al. 2016). Here, the single precision version of IFS (cycle 45r2) is used.

Contact
ICON simulations: René Redler (MPI-M) and Daniel Klocke (MPI-M)
IFS simulations: Thomas Rackow (ECMWF) and Xabier Pedruzo (ECMWF)
Project office: Theresa Mieslinger (MPI-M)

Model development overview#

nextGEMS project timeline.

  • DYAMOND Winter (multi-model intercomparison for 40 days)

  • Cycle 1 (simulations performed late 2021)

  • Cycle 2 (simulations performed mid 2022)

  • Cycle 3 (simulations performed mid 2023)

Cycle 1#

The Cycle 1 simulations are based on the DYAMOND Winter simulations, but run for one year at about 5 km horizontal resolution and for 2 to 3 months at 2.5 km resolution. Within Cycle 1, first bugs were detected and fixed such that some further runs are available with improved configuration.

Please find here an overview of the processed variables of Cycle 1

IFS#

Here’s a first overview of the IFS Cycle 1 simulations:

ICON#

The original Cycle 1 simulation from ICON developed out of the latest DYAMOND-Winter run (dpp0029) and is called dpp0052. This simulation was analyzed during the nextGEMS Cycle 1 Hackathon in October 2021.

Continuous development and especially the transition from the old (Mistral) to the new (Levante) supercomputer lead to significant workflow improvements (new compiler, new scripting). The dpp0066 (5 km) and dpp0067 (2.5 km) runs are the latest Cycle 1 simulations run on Levante. They include improvements on the model physics such as a bug fix for the surface wind stress over the ocean. A publication describing these simulations will be submitted end on June 2022. Those runs also served as a basis for a student hackathon at UCM (Madrid) in April 2022.

Cycle 2#

The nextGEMS Cycle 2 simulations build up on the Cycle 1 simulations for both models, IFS/FESOM and ICON. They include improvements in the configuration (new schemes and features) and a change in the output. The simulations will be analyzed during the Cycle 2 hackathon in Vienna end of June 2022.

ICON#

What’s new in Cycle 2?

  • radiation scheme changed from psRad to RRTMGp

  • river discharge coupling

  • new ocean spin-up with nudging to observed SST

  • new ocean vertical coordinate (z*) with thin surface levels

  • bug fix of dry static energy over land

Up to date we provide a 2-year simulation at 5 km resolution called ngc2009 and a 10-year simulation at 10 km resolution called ngc2012.

Monthly quicklook images for both runs are available on the DKRZ SWIFT browser.

As of Nov 2022, we have two 30-year simulations at 10km resolution called ngc2013 and rthk001. They both use TTE scheme for atmospheric turbulence (instead of Smagorinsky). The two differ only by thickness of the surface ocean layer.

In addition, we provide two simulations with and without aerosol perturbations over 10 days at 5 km resolution called ngc2009_irad32 and ngc2009_irad33.

Jupyter notebooks for an initial analysis are collected on the GitLab of DKRZ.

IFS#

What’s new in Cycle 2?

  • Based on IFS cycle 47r3.3

  • Signficantly reduced global water and energy imbalances

  • Several upgrades in the surface-atmosphere interaction:

    • Multi-layer snow scheme

    • Improved high-resolution land, sea, lake, and glacier masks

    • Revised surface orographic drag

  • In FESOM:

    • Refactored code allowing hybrid MPI/OpenMP support; significantly faster in the single-executable setup with IFS

    • Newly developed NG5-grid with about 7.5 million surface nodes; down to ~5km resolution

    • New NG5 stand-alone ocean spin-up forced by ERA5 (without nudging to observed SST compared to ICON)

    • Allows for linear kinematic features (sea ice cracks)

    • Eddy-resolving in large areas of the globe; Tropical Instability Waves also resolved

  • WAM wave model:

    • For Cycle 2, we provide for the first time output of the wave model

  • High-frequency model output (‘DDH’):

    • At 30 selected points over the globe

The IFS model in combination with the FESOM and NEMO ocean models was run in 3 different main experiments for Cycle 2:

8 months-long and highest-resolved simulation with atmosphere at 2.5 km (IFS) coupled to ~5 km ocean (FESOM); Deep convection parameterization OFF:

Main Cycle 2 simulation over 1 full year with atmosphere at 4 km (IFS) coupled to ~5 km ocean (FESOM); Deep convection parameterization OFF:

Baseline simulation with atmosphere at 9 km (IFS) coupled to 0.25 degree ocean (NEMO); Deep convection parameterization ON:

To get an idea what the data can provide, animations of selected variables from the IFS-FESOM2 simulations (with date string) are available at the AWI Nextcloud (last access: 27 June 2022).

Cycle 3#

The nextGEMS Cycle 3 simulations build up on the Cycle 1 and 2 simulations for both models, IFS/FESOM and ICON. They include improvements in the configuration (new schemes and features) and some changes in the output. The simulations will be analyzed during the Cycle 3 hackathon in Madrid in May/June 2023.

All Cycle 3 simulations can be accessed through this intake catalog:

import intake
cat = intake.open_catalog("https://data.nextgems-h2020.eu/catalog.yaml")

ICON#

What’s new in Cycle 3?
  • Output:

    • The output is now written on the HEALPix grid. For details and tipps on processing see here: Hierarchical HEALPix output

    • There is a hierarchy of frequencies with half-hourly (PT30M), 3-hourly (PT3H), and daily (P1D) output fields.

      • half-hourly output contains 2D fields of atmosphere and land

      • 3-hourly ouptut contains additional 3D atmospheric and land fields

      • daily output contains additional oceanic fields.

  • Ocean:
    • New ocean vertical mixing parameter settings

      • Reduction of c_k parameter from 0.2 to 0.1 (because this improves the equatorial Atlantic zonal SST gradient)

      • Additional Langmuir turbulence parameterisation in TKE scheme switched on (because this very slightly deepens the too shallow mixed layer)

    • activated coupling of sea level pressure (in Cycle 2, this was erroneously not done for the ocean, only for sea ice)

    • new spinup with the above changes, otherwise same spinup procedure as in Cycle 2 (e.g. with SST nudging)

    • ocean time step increased

  • Atmosphere:

    • fix in sensible heat flux over ocean and sea ice

    • energy fix in microphysics

    • horizontal tracer advection set to miura (2nd order, linear reconstruction) with subcycling for all tracers

    • mixed precision (single precision in parts of the dynamical core)

  • Land:

    • use land heat capacity/conductivity maps

    • extpar soil texture in hydrology

Simulations:

  • ngc3028 (5 years, coupled, 5km horizontal resolution)

  • HAMOCC (bonus simulation for the Fisheries topic at the Cycle 3 Hackathon; ocean-only, 10km horizontal resolution, includes ocean biogeochemistry)

IFS#

What’s new in Cycle 3?
  • Based on IFS cycle 48r1, a general overview of 48r1 modifications with respect to the previous IFS Cycle 47r3 can be found here: https://confluence.ecmwf.int/display/FCST/Implementation+of+IFS+Cycle+48r1. New features are for example:

    • a radiatively interactive prognostic ozone using new HLO scheme

    • a new precipitation category: freezing drizzle

    • a revised computation of Semi-Lagrangian advection departure points

    • a new model top sponge layer formulation and semi-Lagrangian vertical filter

  • On top of a significantly reduced global water and energy imbalance that was already available in cycle 2, the TOA radiation imbalance has been tuned by:

    • a decrease of a threshold that limits the minimum size of ice effective radius, increasing high clouds in the tropics and therefore reducing outgoing longwave radiation

    • a reduction of cloud edge erosion, increasing low-level cloud amount and therefore reducing TOA net shortwave radiation

    • a reduction of the cloud inhomogeneity, which increases low-level cloud amount as it reduces the rate of accretion. This change is in line with nextGEMS’s km-scale resolutions as cloud inhomogeneity is expected to be smaller at high resolutions

    • a change restricting the detrainment of mid-level convection to the liquid phase in most situations

    • in agreement with nextGEMS Cycle 2 but unlike in the operational IFS, linear rather than cubic interpolation is used for the departure point interpolation of the Semi-Lagrangian advection scheme for all moist species except water vapour

    • a modification of the microphysics for rain and snow evaporation.

  • At the highest resolution (4.4 km), instead of switching the deep convection scheme off completely, its activity is strongly reduced (the cloud base mass flux is reduced by a factor of 5 compared to the default value). This results in a realistic PDF of intense precipitation.

  • Surface-atmosphere interaction upgrades:
  • In the FESOM ocean model:
    • Science changes:

      • Coupling of ocean surface currents

      • More consistent ocean heat flux setting (cooler Southern Ocean)

      • Snow takes heat from the ocean in order to melt when falling into it (cooler Southern Ocean)

      • More consistent heat flux into the ice (shortwave fix)

    • Technical changes:

      • multi-field coupling, reducing a bottleneck

      • Added extra ocean variables (MLD, SST, sea ice conc, ocean currents, 300m-averaged T, S, and depth of 20C isotherm, ..) that are available on IFS grid with its GRIB code.

      • Added 3-hourly upper 300m of 3D model levels to better match the Storms & Ocean data request

    • For a complete overview see FESOM/fesom2

  • High-frequency model output (‘DDH’):

    • We do not provide the DDH output in Cycle 3 anymore.

  • the OUTPUT: all IFS output has been processed on-the-fly by multIO and written to a local FDB. This means that the output either needs to be retrieved with FDB tools or with gribscan. The gribscan approach is reccommended. We are able to provide multiple forms of processed output:
    • On the original grid, we provide hourly output of all 2D variables and 6-hourly output of 3D fields on pressure levels (700, 750, 800, 850, 875, 900, 925, 950, 975, 1000 hPa). Some selected 3D variables are provided hourly on certain pressure levels:

      • vertical velocity (both in Pa/s and m/s) at 500 hPa and 850 hPa

      • horizontal velocity u and v on 850 hPa

      • temperature at 700 hPa

      • geopotential at 300, 500 and 850 hPa

    • The same output is also conservatively interpolated to two coarser grids, a 0.25°x0.25° grid and a 1°x1° grid. On these coarser grids, the 3D fields are available on the full range of pressure levels (1, 5, 10, 20, 30, 50, 70, 100, 150, 200, 250, 300, 400, 500, 600, 700, 750, 800, 850, 875, 900, 925, 950, 975, 1000 hPa).

    • For all 2D and 3D variables, monthly means are provided, both on the original grid and on the coarser grids, in both cases providing the full range of pressure levels.

The IFS model in combination with the FESOM and NEMO ocean models was run in 5 different configurations for Cycle 3:

  • IFS_4.4-FESOM_5-cycle3: The main cycle 3 simulation. 5 years and highest-resolved simulation with atmosphere at ~4.4 km (IFS) coupled to ~5 km ocean (FESOM); Deep convection parameterization ON with 1/5 cloud base mass flux. Variable lists: FESOM, IFS.

  • IFS_9-FESOM_5-cycle3: 1 year with atmosphere at ~9 km (IFS) coupled to ~5 km ocean (FESOM); Deep convection parameterization ON. Variable lists: FESOM, IFS.

  • IFS_9-NEMO_25-cycle3: 5 years with atmosphere at ~9 km (IFS) coupled to 0.25° ocean (NEMO); Deep convection parameterization ON

  • IFS_28-FESOM_25-cycle3: 5 years with atmosphere at ~28 km (IFS) coupled to 0.25° ocean (FESOM); Deep convection parameterization ON. Variable lists: FESOM, IFS.

  • IFS_28-NEMO_25-cycle3: 5 years with atmosphere at ~28 km (IFS) coupled to 0.25° ocean (NEMO); Deep convection parameterization ON

We also provide FESOM stand alone age Tracer Experiment. This simulation was conducted using the setup and mesh derived from sensitivity experiments associated with vertical mixing schemes in the tropics. The model mesh employed a resolution of 13 km in the tropical Atlantic and approximately 50 km in the remaining ocean regions. Starts from PHC3 climatology in 2015, and initial age data from CESM2 run on ERDA. Forced by hourly ERA5 data. Output for 2020 available with 3hour time step in two different grid (only interpolated): Interpolated to regular grid with 0.1degree spacing for region lon(-60, 15), lat(-9.95, 29.95), covering the uppermost 2000 meters of the ocean. Interpolated to regular grid with 0.5 degree encompassing the entire ocean. Variable list: FESOM.