.. _getting-started: Contributing to easy.gems ========================== Great to have you here 👋 and thank you very much for your interest in contributing to easy.gems 👍. We welcome contributions from anyone of **any** kind, e.g. notes about errors, requests for more articles or better descriptions, questions about specific examples, direct updates to articles or new articles, better illustrations and more. This document should give you a little guide, how we try to organise these kinds of contributions. Our process of updating easy.gems is closely tied to workflows common to gitlab, so they might be familiar, but probably there's something new to learn as well. In particular, we are using `gitlab.dkrz.de `_ for collaboration. If you don't already have an DKRZ user account, you'll have to `create one `_ to contribute. We collect things which are not yet written out in the form of **issues**. These could be ideas, plans, requests, bug reports etc... all the things for which you don't have an immediate solution at hand, want to discuss with someone or seek for others to do it 😁. Thus, if you would like to have some things improved, please have a look at `our existing issues `_, where you might already find similar requests and can add some emphasis or additional input. If you don't find anything related, please `open a new one `_ and describe what you like to see. Each issue opens something like a small chat forum for exchange about the requested topic. It's also a place to collect required information or further plans if needed. Changes to the easy.gems website happen via **merge requests**. A merge request is a bit similar to an issue, as it also opens up a small chat forum. However, it is always tied to a certain set of changes to the source code of easy.gems. As easy.gems technically is a static website, the source code consists of all the things which make up the site, that is the content, the layout and styles and the instructions how to generate the site. Changes to any of these will happen in a ``git branch`` (:ref:`more on that later `) and those changes will be reviewed and merged back into the ``main`` version of easy.gems using a merge request. Please have a look at `our existing merge requests `_. If you see some open requests on topics you know a bit or two about, please help us out by :ref:`reviewing` it. If you want to add your own changes, there are two options: very small changes (e.g. spelling errors) can be made quickly by using the *Edit this page* button on each page. This will direct you to a little text editor where you can type your changes and which will lead you to the creation of a merge request. For anything larger, you'll have to get a local copy of the `git` repository as described in :ref:`Adding Content`. .. _adding content: Adding Content -------------- New content is added using the `git `_ version control system. There are many good resources describing git in general, this document will only cover the basics. If you search for more information, maybe the `git book `_ or `gitready `_ are good starting points. The starting point will be the source repository for this documentation at https://gitlab.dkrz.de/easy/gems/. If you haven't done already, please go there and "request access". Once you have access, you can perform small edits on the files directly there (just as if you use the *Edit this page* button on an article page). For bigger changes, we recommend you clone the repository via SSH:: git clone git@gitlab.dkrz.de:easy/gems.git cd gems .. admonition:: How can I work with git, SSH-keys and gitlab.dkrz.de? :class: tip, dropdown Once you have logged in on gitlab (same login/password as for levante), you need to set up SSH key auth (follow the advice at https://gitlab.dkrz.de/-/profile/keys for generating ssh keys on your machine). You can register your key on gitlab (same link as above). After that you can run the download command from your local home over terminal. At this point, you have a fresh local copy of the source files of the easy.gems book on your computer. You can edit this copy as you wish and send these edits back to the main repository afterwards. To do so, you may want to create a new git branch at this point already (``git checkout -b``, see below), but you can also do this later. For some more specific hints about how to edit pages, please refer to the sections below. To preview your edits, you need sphinx and some other dependencies:: python3 -m pip install -r requirements.txt .. admonition:: easy.gems environment on Levante :class: tip, dropdown On levante, you can load a python virtual environment:: module load python3 source /work/kv0653/k202134/venvs/easy.gems/bin/activate To leave it again, call:: deactivate module rm python3 (You don't need the python module to start the python from the venv, but it comes with pandoc and possibly other tools required for building the pages). Further below, you can find some specific hints on adding content in our two mainly used formats :ref:`reStructuredText ` and :ref:`Jupyter Notebooks `. After doing your edits, you are ready to go and can build the book by running:: make html You'll find the index page at `build/html/index.html`, please open in in your favourite web browser. You can repeatedly edit the source files, run ``make html`` and refresh the web-page until you are happy with your change. Afterwards, it's time to send your changes back to the main repository. For automatic cleanup, we use `pre-commit `_. Please activate it for your local repository once before doing your first commit using:: pre-commit install If you haven't done so yet, please create a git branch (you'll have to choose a name):: git checkout -b MY_CREATIVE_BRANCH_NAME Afterwards, you'd be adding your changed files to the so called "staging area", which marks them as to be committed:: git add [MY_CHANGED_FILES] The next step is to actually commit your changes to your local clone of the repository. For each commit, you should come up with a descriptive message, explaining what you did. If it's a short one, you can add the message on the command line using the ``-m`` option. If you want to explain more, just skip the message and ``git`` will open a text editor for you:: git commit -m 'my fancy changes' .. warning:: For python code and jupyter notebooks we use the `black code formatter `_ to re-format the files. This leads to rejected commits, see below for more info on the workflow. Now, it's time to do the actual upload of your branch (you'll have to specify the same name as above):: git push -u origin MY_CREATIVE_BRANCH_NAME Finally, please create a `merge request `_ for your branch, which will do some checks and build your new version in a "review environment" (after a few minutes, you'll find a link to "View app" on the merge request page). This will give others the option to have a quick look at what you did, and if everything is fine, your changes will be merged into the current `main` branch and become publicly visible at https://easy.gems.dkrz.de/. .. _adding_notebooks: Adding jupyterlab/ipython notebooks ----------------------------------- You can directly add ipython notebooks to the gitlab repository. They will be parsed by the `nbsphinx `__ plugin. The raw notebook can be downloaded via the *Page source* link at the bottom of the page. We are using `jupytext paired notebooks `_ within Easy Gems, because we want a plain-text notebook format to facilitate reviews. In addition we use the `black code formatter `_ to ensure consistently formatted code. These scripts are automatically run by pre-commit if pre-commit and the environment are configured as detailed above. Usually, your first attempt at committing will be refused, and new changes will be added to your jupyterhub notebook and the matching paired :file:`.md` plain text version. You will get a message like .. code:: bash (easy.gems)$ git add source/Processing/healpix/healpix_cartopy-Copy1.ipynb (easy.gems)$ git commit -m 'add a duplicate healpix cartopy notebook' [WARNING] Unstaged files detected. [INFO] Stashing unstaged files to /home/XY/.cache/pre-commit/patch123567890. jupytext.................................................................Failed - hook id: jupytext - exit code: 2 - files were modified by this hook reformatted - All done! 1 file reformatted. [jupytext] Reading source/Processing/healpix/healpix_cartopy-Copy1.ipynb in format ipynb [jupytext] Executing black - [jupytext] Updating source/Processing/healpix/healpix_cartopy-Copy1.ipynb [jupytext] Error: the git index is outdated. Please add the paired notebook with: git add source/Processing/healpix/healpix_cartopy-Copy1.ipynb [jupytext] Updating source/Processing/healpix/healpix_cartopy-Copy1.md [jupytext] Error: the git index is outdated. Please add the paired notebook with: git add source/Processing/healpix/healpix_cartopy-Copy1.md black................................................(no files to check)Skipped [INFO] Restored changes from /home/XY/.cache/pre-commit/patch123567890 From this, take both git add commands, run them, and try to commit again. This time it should work. .. code:: bash git add source/Processing/healpix/healpix_cartopy-Copy1.ipynb git add source/Processing/healpix/healpix_cartopy-Copy1.md git commit -m 'add a duplicate healpix cartopy notebook' Links between notebooks ~~~~~~~~~~~~~~~~~~~~~~~ If you want to link between notebooks within easy.gems, please add **relative** links in the Markdown parts of a notebook like: .. code:: markdown [name of the link](../relative/path_to/notebook.ipynb) This way, links will also work if notebooks are run interactively in jupyterhub. They will be converted appropriately for the rendered easy.gems website and they will also do the right thing for review builds. .. _adding_rst: Adding text articles -------------------- easy.gems uses `reStructuredText `_ (reST) as markup language for text articles. Please refer to the linked document for a general introduction into reST. Below are some further tricks for writing special content blocks: Adding Math ~~~~~~~~~~~ .. code:: rst :math:`\sum_{i=1}^n i^2` .. math:: \sum_{i=1}^n i^2 yields :math:`\sum_{i=1}^n i^2` .. math:: \sum_{i=1}^n i^2 Adding code snippets ~~~~~~~~~~~~~~~~~~~~ Use the following syntax: .. code:: rst .. code:: python print ("hello world") To produce .. code:: python print ("hello world") Note that there needs to be an empty line before the :code:`.. code-block::` directive, and the code needs to be indented. Similarly many other languages are supported. For inline code, you can use .. code:: rst :code:`sample code` to produce :code:`sample code`. .. _reviewing: Reviewing --------- To review a `merge request `_, one would usually go to the individual merge request page (you can check if there's currently an open one, otherwise please have a look at existing *Merged* requests). On the *Overview* page, you'll find the initial description of the merge request as well as the discussion. On the *Changes* page, you'll find a diff-view of the files which actually have been changed. It's a good idea to look through those changes first, such that you see which parts of the site have been touched and which haven't. Generally, only those parts which are described on the *Overview* page should be touched by the changes. For every commit/merge request, gitlab tries to build a preview version of the easy.gems site with the via the *CI/CD pipelines*. If this was successful for the merge request at hand, you can find a *View app* button in the pipeline panel at the top of the overview page, that takes you to the preview version for this merge request, so you can see if everything looks as expected. When reviewing a merge request, please check if the actual *Changes* correspond to the description in the *Overview* and check if the content of the changes looks good to you. You may consider suggestions for material that makes sense to add as part of the merge-request, but only if they are truly germane to the request at hand. If issues which extend beyond the scope of the merge request arise, please open a new issue and reference back to the originating merge request. This will help us to finalize merge requests quickly and not to loose time in overly long discussions. Useful Links ------------ Sphinx `tutorial `__ reST & Sphinx `cheat sheet `__