{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Selecting and Reindexing of Area of Interest" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### _The elegant way of dealing with large output_" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "_Before working through this script, it is helpful to have had a look into [Triangular Meshes and Basic Plotting](tripcolor.ipynb) to get a basic understanding of plotting on a triangular basis._\n", "\n", "Three-dimensional global ICON output for example requires many times more memory than two-dimensional output. The handling of such large amounts of data can very quickly lead to the exhaustion of the given memory. If you are sitting right next to a supercomputer, you are tempted to just request more RAM and go for it. However, there are elegant solutions besides the powerful one and since a lot of RAM also means a lot of electricity and a lot of coolant, the motivation for an elegant way is also to save limited resources.\n", "\n", "In many cases, we rarely look at the complete global output, but rather at a specific, selected area. It is therefore advisable to cut out only this area. For this purpose it is very helpful to reduce the grid information from the global grid file to the area of interest but in such a way that the indexing makes sense starting at 0 and counting up continuously. The advantage is that we generate a new local grid-file, which looks like the global grid-file but is much smaller in terms of storage capacity and therefore easier and faster to handle.\n", "\n", "So let's do something for the climate 🌍 and first of all load the necessary libraries and the global grid-file:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import xarray as xr\n", "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Importing the Grid-File" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<xarray.Dataset>\n", "Dimensions: (cell: 20971520, nv: 3, vertex: 10485762,\n", " ne: 6, edge: 31457280, no: 4, nc: 2,\n", " max_stored_decompositions: 4, two_grf: 2,\n", " cell_grf: 14, max_chdom: 1, edge_grf: 24,\n", " vert_grf: 13)\n", "Coordinates:\n", " clon (cell) float64 ...\n", " clat (cell) float64 ...\n", " vlon (vertex) float64 ...\n", " vlat (vertex) float64 ...\n", " elon (edge) float64 ...\n", " elat (edge) float64 ...\n", "Dimensions without coordinates: cell, nv, vertex, ne, edge, no, nc,\n", " max_stored_decompositions, two_grf, cell_grf,\n", " max_chdom, edge_grf, vert_grf\n", "Data variables: (12/91)\n", " clon_vertices (cell, nv) float64 ...\n", " clat_vertices (cell, nv) float64 ...\n", " vlon_vertices (vertex, ne) float64 ...\n", " vlat_vertices (vertex, ne) float64 ...\n", " elon_vertices (edge, no) float64 ...\n", " elat_vertices (edge, no) float64 ...\n", " ... ...\n", " edge_dual_normal_cartesian_x (edge) float64 ...\n", " edge_dual_normal_cartesian_y (edge) float64 ...\n", " edge_dual_normal_cartesian_z (edge) float64 ...\n", " cell_circumcenter_cartesian_x (cell) float64 ...\n", " cell_circumcenter_cartesian_y (cell) float64 ...\n", " cell_circumcenter_cartesian_z (cell) float64 ...\n", "Attributes: (12/43)\n", " title: ICON grid description\n", " institution: Max Planck Institute for Meteorology/Deutscher ...\n", " source: git@git.mpimet.mpg.de:GridGenerator.git\n", " revision: d00fcac1f61fa16c686bfe51d1d8eddd09296cb5\n", " date: 20180529 at 222250\n", " user_name: Rene Redler (m300083)\n", " ... ...\n", " topography: modified SRTM30\n", " subcentre: 1\n", " number_of_grid_used: 15\n", " history: Thu Aug 16 11:05:44 2018: ncatted -O -a ICON_gr...\n", " ICON_grid_file_uri: http://icon-downloads.mpimet.mpg.de/grids/publi...\n", " NCO: netCDF Operators version 4.7.5 (Homepage = http...
<xarray.Dataset>\n", "Dimensions: (cell: 70, vertex: 50, edge: 119)\n", "Coordinates:\n", " * cell (cell) int64 4282376 4282377 4282378 ... 4283128 4283129 4283130\n", " * vertex (vertex) int32 2144929 2144932 2144933 ... 2145238 2145239 2145286\n", " * edge (edge) int32 6427302 6427311 6427312 ... 6428418 6428419 6428420\n", "Data variables:\n", " *empty*
<xarray.Dataset>\n", "Dimensions: (cell: 70, nv: 3, vertex: 50, ne: 6,\n", " edge: 119, no: 4, nc: 2,\n", " max_stored_decompositions: 4, two_grf: 2,\n", " cell_grf: 14, max_chdom: 1, edge_grf: 24,\n", " vert_grf: 13)\n", "Coordinates:\n", " clon (cell) float64 0.1804 0.1805 ... 0.174\n", " clat (cell) float64 0.9453 0.946 ... 0.9488\n", " vlon (vertex) float64 0.1815 0.1807 ... 0.1717\n", " vlat (vertex) float64 0.9456 0.9466 ... 0.9482\n", " elon (edge) float64 0.1811 0.1814 ... 0.1724\n", " elat (edge) float64 0.9461 0.9471 ... 0.9487\n", " * cell (cell) int64 4282376 4282377 ... 4283130\n", " * vertex (vertex) int32 2144929 2144932 ... 2145286\n", " * edge (edge) int32 6427302 6427311 ... 6428420\n", "Dimensions without coordinates: nv, ne, no, nc, max_stored_decompositions,\n", " two_grf, cell_grf, max_chdom, edge_grf, vert_grf\n", "Data variables: (12/91)\n", " clon_vertices (cell, nv) float64 0.1794 0.1802 ... 0.173\n", " clat_vertices (cell, nv) float64 0.9457 0.9446 ... 0.9492\n", " vlon_vertices (vertex, ne) float64 9.969e+36 ... 9.969e+36\n", " vlat_vertices (vertex, ne) float64 9.969e+36 ... 9.969e+36\n", " elon_vertices (edge, no) float64 0.1807 0.1805 ... 0.1728\n", " elat_vertices (edge, no) float64 0.9466 0.946 ... 0.9485\n", " ... ...\n", " edge_dual_normal_cartesian_x (edge) float64 -0.6706 -0.7373 ... -0.7362\n", " edge_dual_normal_cartesian_y (edge) float64 -0.5071 0.4974 ... 0.4973\n", " edge_dual_normal_cartesian_z (edge) float64 0.5414 0.4572 ... 0.4589\n", " cell_circumcenter_cartesian_x (cell) float64 0.576 0.5755 ... 0.5739\n", " cell_circumcenter_cartesian_y (cell) float64 0.105 0.105 ... 0.1003 0.1009\n", " cell_circumcenter_cartesian_z (cell) float64 0.8107 0.8111 ... 0.8127\n", "Attributes: (12/43)\n", " title: ICON grid description\n", " institution: Max Planck Institute for Meteorology/Deutscher ...\n", " source: git@git.mpimet.mpg.de:GridGenerator.git\n", " revision: d00fcac1f61fa16c686bfe51d1d8eddd09296cb5\n", " date: 20180529 at 222250\n", " user_name: Rene Redler (m300083)\n", " ... ...\n", " topography: modified SRTM30\n", " subcentre: 1\n", " number_of_grid_used: 15\n", " history: Thu Aug 16 11:05:44 2018: ncatted -O -a ICON_gr...\n", " ICON_grid_file_uri: http://icon-downloads.mpimet.mpg.de/grids/publi...\n", " NCO: netCDF Operators version 4.7.5 (Homepage = http...