Posted in 2022
Recap of a (Re)Introduction to Earth System Data Science (ESDS) Across NCAR/UCAR/UCP
- 19 December 2022
ESDS hosted a (Re)Introduction to Earth System Data Science (ESDS) Across NCAR/UCAR/UCP on November 10-11, 2022. This blog post serves to consolidate and share resources from that event.
The final agenda from the event can be found here.
Regridding using xESMF and an existing weights file
- 06 December 2022
A fairly common request is to use an existing ESMF weights file to regrid a Xarray Dataset (1, 2). Applying weights in general should be easy: read weights then apply them using dot or tensordot on the input dataset.
In the Xarray/Dask/Pangeo ecosystem, xESMF provides an interface to ESMF for convenient regridding, includiing parallelization with Dask. Here we demonstrate how to use an existing ESMF weights file with xESMF specifically for CAM-SE.
Calculating Temporal Averages with GeoCAT-comp vs Xarray
- 29 November 2022
With temporally large datasets, computing seasonal and annual averages are a great ways to summarize the data and make it easier to manage and understand. You may want to take hourly, daily, or monthly data and compute seasonal or annual averages.
When using data that has a daily or finer resolution (e.g. hourly), calculating an annual average is simple. Every day and hour has the same length, so an unweighted average will work.
Preparation for a (Re)Introduction to Earth System Data Science (ESDS) Across NCAR/UCAR/UCP
- 02 November 2022
ESDS is hosting a (Re)Introduction to Earth System Data Science (ESDS) Across NCAR/UCAR/UCP on November 10th and 11th (Thursday-Friday). This event will be hybrid, in-person in NCAR’s Mesa Lab Main Seminar Room and virtual over Zoom or Google Meet. More information is in this previous post.
This tutorial is designed to be run from Binder. You can do that by navigating to the GeoCat Tutorial Repository and selecting “Launch Binder” in the README.md.
A (Re)Introduction to Earth System Data Science (ESDS) Across NCAR/UCAR/UCP
- 18 October 2022
ESDS is hosting a (Re)Introduction to Earth System Data Science (ESDS) Across NCAR/UCAR/UCP on November 10th and 11th (Thursday-Friday). This event will be hybrid, in-person in NCAR’s Mesa Lab Main Seminar Room and virtual over Zoom and/or Google Meet.
Please register to the event through the Registration Form by Friday, November 4, 2022!
New Office Hours Appointment System
- 10 August 2022
The ESDS office hours are moving to an appointment scheduling page.
The previous office hours system was two one-hour blocks at the same time every week via a Zoom call where software engineers sit in and await scientists to show up and receive help. This was not ideal because some weeks no one showed up (which eventually leads to attrition of software engineers), and other weeks many people showed up and could not all receive the individualized help they needed. Also we had many software engineers and scientists with scheduling conflicts during these two static time blocks. You may have noticed these are no longer on the ESDS calendar.
What I’ve learned about debugging
- 12 May 2022
It can be easy to get discouraged when working on a coding problem. When you think you’ve solved a problem in your code, it turns out that you misunderstood or didn’t fully grasp the root of the problem. Then, you’re facing a new coding challenge where the computer continually returns a different result than expected. This can turn into a game of cat and mouse as you chase down the bugs that are in the way of your new shiny feature being implemented.
When this happens to me, I feel disheartened and imposter syndrome creeps up - and then I can shut down, wishing that coding was easier. For me, the silver lining here is that a lot of my work is geared towards educating geoscientists to be better coders. Despite always being STEM-oriented in my passions and work, coding can be hard. This is the experience for a lot of people, and certainly is the experience for a lot of the scientists I have worked with. Reflecting on my own struggles in my own software development practices, can help me teach coding from empathetic and helpful perspectives.
The Python Tutorial Series Returns this Summer!
- 11 May 2022
The Python Tutorial Series returns on every other Thurday at 1 PM Mountain starting May 26th with the 5-part “Your First” Python Tutorial series. The first lesson, “Opening a .txt File”, covers how to create your first Python script to open some simple data. Some tutorials will be 1-off lessons on specific topics, while others will span multiple sessions.
We will begin by following through the Xdev tutorial website, designed to introduce Python to scientists who already have a good deal of programming experience, albeit with a different language. Our main audience is NCAR scientists, but anyone who has some experience working in the terminal and understanding of basic coding structures who wants geoscience-applicable examples will find this tutorial series helpful and reqarding.
Debugging dask workflows: Detrending
- 31 March 2022
Detrending - subtracting a trend, commonly a linear fit, from the data - along the time dimension is a common workflow in the climate sciences.
Here’s an example
Batch Processing Jupyter Notebooks with Papermill
- 17 March 2022
Jupyter notebooks are really good for doing the heavy lifting of data analysis by:
Allowing you to showcase your work in a single place. You can see the complete “paper trail” of what was done. This includes the code, results, visuals and the narrative accompanying your analysis.
“Thinking with Xarray” Tutorial
- 04 March 2022
The NCAR/UCAR virtual Python Tutorial Seminar Series continues with an intermediate-level lesson on analysis with Xarray titled “Thinking with Xarray” on March 9th at 1 PM Mountain Standard Time led by Deepak Cherian.
Xarray is an open source project and Python package that makes working with multi-dimensional arrays simple and effient by introducing labels in the form of dimensions, coordinates and attributes on top of raw NumPy-like arrays. See the Xarray Documentation
Sparse arrays and the CESM land model component
- 24 February 2022
An underappreciated feature of Xarray + Dask is the ability to plug in different array types. Usually we work with Xarray wrapping a Dask array which in turn uses NumPy arrays for each block; or just Xarray wrapping NumPy arrays directly. NumPy arrays are dense in-memory arrays. Other array types exist:
sparse for sparse arrays
MetPy Tutorial
- 07 February 2022
The NCAR/UCAR virtual Python Tutorial Seminar Series continues with a lesson on MetPy on February 9th at 1 PM Mountain Standard Time led by Drew Camron.
MetPy is a collection of tools in Python for reading, visualizing, and performing calculations with weather data. See the MetPy Documentation