.. _dkrzs_jupyterhub:

DKRZs JupyterHub
=========================

You like to analyse the big data sets stored at DKRZ by using  Python, R or Julia scripts? 

Then, DKRZs JupytherHub is the perfect tool to do so! 

It allows to execute the notebook directly on the DKRZ HPC system Levante and therefore has the following advantages: 

* **no need to copy any data**, e.g. to your home device
* Levantes **nodes which might be faster** than your ones
* you can make use of **parallel computation** of your scripts 
* **project compute resources** can be used
* nextGEMS and DYAMOND provide intake catalogues of their experiments, so you can **search and load data files directly into your Python script** 
* notebooks can easily be **shared with others**



.. note::
 JupyterHub is available at https://jupyterhub.dkrz.de.
 You need a DKRZ user account (user-id), which has to be part of a compute project with resources to Levante. 


**DKRZs Documentations pages**

* `DKRZ JupyterHub documentation (docs.dkrz.de) <https://docs.dkrz.de/doc/software%26services/jupyterhub/index.html>`__
* `DKRZ JupyterHub blog posts (docs.dkrz.de) <https://docs.dkrz.de/blog/category/jupyterhub.html>`__




Our related tutorials, tips and tricks
-------------------------------------------------------


First step tutorials
~~~~~~~~~~~~~~~~~~~~~~


From our ICON beginners guide (Python) :ref:`for-beginners`: 

* detailed explenaition how to setup a :ref:`beginner_dpp0029_jupyter_notebook`
* :doc:`../For-Beginners/Tutorial01`

Searching for data files and directly load it into your Python script: 

* :doc:`../Intake/catalog-basics`
* or use :doc:`../Intake/find_files` to get detailed file infos or use it within CDO post-processing
 


Query the memory footprint of your Python jobs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To query the memory footprint of your (processing) jobs, you can use these commands:

.. code:: bash

  # put these function definitions in your .bashrc
  function maxmem_running() {  sstat --format=AveCPU,MaxRSS,AveRSS,JobID -j $1 --allsteps; }
  function maxmem_terminated() {  sacct --format=JobID,JobName,MaxRSS,Elapsed -j $1 ; }

  # call on a job-id
  maxmem_running MY_RUNNING_JOB_ID
  maxmem_terminated MY_TERMINATED_JOB_ID