Hints for processing data

Note

For passing on jupyterhub notebooks as links, two things seem necessary. a) The recipient needs to first start a jupyterhub session. b) The link needs to be modified to have the recipients user id between /user/ and /preset/:

https://jupyterhub.dkrz.de/user/USER_ID/preset/tree/SOME_PATH

To query the memory footprint of your (processing) jobs, you can use these commands:

# put these function definitions in your .bashrc
function maxmem_running() {  sstat --format=AveCPU,MaxRSS,AveRSS,JobID -j $1 --allsteps; }
function maxmem_terminated() {  sacct --format=JobID,JobName,MaxRSS,Elapsed -j $1 ; }

# call on a job-id
maxmem_running MY_RUNNING_JOB_ID
maxmem_terminated MY_TERMINATED_JOB_ID