NCAR CISL NSF Logo

ESDS 2024 - Xarray and Dask Tutorial#

Jupyter Build Made withJupyter Commits

Welcome to ESDS 2024 Xarray and Dask Tutorials!

Authored by: Negin Sobhani, Brian Vanderwende, and Ben Kirk

The materials and notebooks in this tutorial is published as a Jupyter book here. Jupyter Book Badge)

Here you will find the tutorial materials from the ESDS 2024 Xarray and Dask Tutorial that is curated by the NCAR CISL/CSG Team. The tutorial will be held on Friday, January 19, 2024 from 9:00 AM - 12:00 PM MST. We will have office hourse to answer any questions on Friday, January 19, 2024 from 1:00 PM - 2:00 PM MST.

The tutorial content here is written as Jupyter Notebooks. You can either browse rendered versions of these notebooks on this website, or execute the code examples interactively following [our guidelines](## ⌨️ Getting set up).

This tutorial is open to non-UCAR staff. If you don’t have access to the HPC systems, you may not be able to follow along with the second part of the tutorial. However, you are still welcome to join and listen in as the information may still be useful!

For a more detailed version of this tutorial, please visit our 2023 Dask Tutorial

Video Recoding: Will be available after the event

Prerequisites#

Before beginning any of the tutorials, it is highly recommended that you have a basic understanding of Python programming and Python libraries such as NumPy, pandas, and Xarray.

⌨️ Getting set up#

This tutorial is open to non-UCAR staff. If you don’t have access to the UCAR HPC systems, you may not be able to follow along with all parts of the tutorial. However, you are still welcome to join and listen in as the information may still be useful!

NCAR JupyterHub#

This is the preferred way to interact with this tutorial. Users with access to Casper can run the notebooks interactively, and will be able to save their work and pull in new updates. To connect to NCAR JupyterHub, please open this link in a web browser: https://jupyterhub.hpc.ucar.edu/

Once you logged in, start new a server using Casper Login and hit “Launch Server”.

Next, Clone the repository to your local directory:

git clone https://github.com/NCAR/dask-tutorial

Finally, open the notebooks and interact with them. Make sure to choose the NPL 2024a kernel.

Using Binder#

Users without access to the NCAR/UCAR Casper cluster can still use binder to run through the first notebook . badge

Local installation instructions#

Users without access to the NCAR/UCAR Casper cluster can run through the first notebook on their machine.

To run the notebooks locally:

First clone this repository to your local machine via:

git clone https://github.com/NCAR/Xarray-Dask-ESDS-2024

If you do not already have the conda package manager installed, please follow the instructions here.

Now, create a conda environment:

Navigate to the Xarray-Dask-ESDS-2024/ directory and create a new conda environment with the required packages via:

cd Xarray-Dask-ESDS-2024
conda env update --file environment.yml

This will create a new conda environment named “xarray-dask-esds-2024”.

Next, activate the environment:

conda activate xarray-dask-esds-2024

Finally, launch JupyterLab with:

jupyter lab

Contributing#

We welcome contributions from the community! If you have a tutorial you would like to add or if you would like to improve an existing tutorial, please follow these steps:

Fork the repository.

Clone the repository to your local machine:

git clone https://github.com/your-username/Xarray-Dask-ESDS-2024.git

Create a new branch for your changes:

git checkout -b my-new-tutorial

Make your changes and commit them:

git add .
git commit -m "Add my new tutorial"

Push your changes to your fork:

git push origin my-new-tutorial

Submit a pull request to the original repository.

Support#

If you have any questions or need help with the tutorials, please open a GitHub issue in the repository.

Additional Resources#

The materials in this tutorial are based on the following resources:

  • NCAR DASK HPC Tutorial : This includes comprehensive tutorial on Dask and scaling up Xarray resources with Dask on HPC systems.

  • Dask Cookbook

  • Xarray Tutorial : This includes comprehensive tutorial on different Xarray related topics.

👍 Acknowledgments#

  • NCAR CISL/CSG Team

  • ESDS Initiative

License#

The tutorials in this repository are released under the MIT License.