Searching catalog metadata with search_yaml

Intake catalogs can contain metadata information about their entries. To search for this, you can use search_yaml.

To load search_yaml, use

module use /work/k20200/k202134/hsm-tools/outtake/module
module load hsm-tools/unstable

search_yaml allows you to search for metadata in the catalogs. A simple search would be:

search_yaml FESOM -s variable-long_names=temp
IFS_4.4-FESOM_5-cycle3.2D_1h_1deg {'variable-long_names': ['sea surface temperature']}
IFS_4.4-FESOM_5-cycle3.2D_1h_native {'variable-long_names': ['sea surface temperature']}
IFS_4.4-FESOM_5-cycle3.3D_3h_native {'variable-long_names': ['temperature']}
[...]

Or

search_yaml ICON -s variables=ta
ngc4008 {'variables': ['FrshFlux_TotalIce', 'atmos_fluxes_FrshFlux_Precipitation', 'atmos_fluxes_HeatFlux_Total', 'heat_content_total', 'ta', 'tas', 'tauu', 'tauv']}
ngc4007 {'variables': ['FrshFlux_TotalIce', 'atmos_fluxes_FrshFlux_Precipitation', 'atmos_fluxes_HeatFlux_Total', 'heat_content_total', 'ta', 'tas', 'tauu', 'tauv']}
[...]

Use a bogus key-value pair to get a list of all metadata attributes that are attached to some dataset:

search_yaml ICON -s abc=def
ERROR:root:Found no metadata key matching abc
ERROR:root:Possible keys:
ERROR:root:
    CDI
    CDI_grid_type
    CDO
    Conventions
    DOKU_License
    DOKU_Name
    DOKU_authors
    [...]

Use -s KEY=.* to get all matches for this key.

search_yaml ICON -s project=.*
ngc4008 {'project': ['nextGEMS']}
ngc4007 {'project': ['nextGEMS']}
ngc4006 {'project': ['nextGEMS']}
ngc4005 {'project': ['nextGEMS']}
ngc3028 {'project': ['nextGEMS']}
ngc3028_bc_land {'project': ['nextGEMS']}
ngc3026_WILL_BE_DELETED {'project': ['nextGEMS']}
HAMOCC.HEALPix {'project': ['nextGEMS']}
HAMOCC.OneDegree {'project': ['nextGEMS']}
erc1011.atmos.gr025.2d_daily_max {'project': ['EERIE']}
erc1011.atmos.gr025.2d_daily_mean {'project': ['EERIE']}
[...]