CESM analysis tools#
We have provided some information about tools the CESM users and developers use for analysis of model simulations below. This list is not comprehensive and is intended to provide you with information to start your searches.
Analysis Software#
Many data analysis and visualization software packages are freely available for use on CISL-managed resources. These packages include some developed and supported by NCAR and CISL. Some of these resources are open source while others require licences.
Some of these packages include:
Numerous python packages
Interactive Data Language (IDL)
MATLAB
NCAR Command Language (NCL)
Python#
Python is an open source, general-purpose programming language.
Python is known for:
having a wide range of applications and packages available. There is a huge user base and rougly ~1 gazillion online tutorials.
active development in packages related to the geosciences.
Python is becoming the dominant language for CESM developers and users, so most of the active development of tools for the CESM project at large are done in this language. We provide more detailed information below about some of the tools available for python users on NCAR computing assets.
Jupyter Hub#
The JupyterHub deployment that CISL manages allows “push-button” access to NCAR’s supercomputing resource cluster of nodes used for data analysis and visualization, machine learning, and deep learning.
JupyterHub gives users the ability to create, save, and share Jupyter Notebooks through the JupyterLab interface and to run interactive, web-based analysis, visualization and compute jobs on derecho and casper.
Information about getting started with JupyterHub on NCAR computing resources, environments, and documentation is avaiable at the website below.
Earth System Data Science initiative (ESDS)#
ESDS is an NCAR initiative that seeks to foster a collaborative, open, inclusive community for Earth Science data analysis. ESDS promotes deeper collaboration centered on analytics, improving our capacity to deliver impactful, actionable, reproducible science and serve the university community by transforming how geoscientists synthesize and extract information from large, diverse data sets.
More information, including FAQs and a blog with examples can be found at the website below.
Project Pythia#
If you are new to Python and its application to the geosciences, then starting with Project Pythia is a good first step. Project Pythia is the education working group for Pangeo and is an educational resource for the entire geoscience community. Together these initiatives are helping geoscientists make sense of huge volumes of numerical scientific data using tools that facilitate open, reproducible science, and building an inclusive community of practice around these goals.
GeoCAT#
The Geoscience Community Analysis Toolkit (GeoCAT) is a software engineering effort at NCAR. GeoCAT aims to create scalable data analysis and visualization tools for Earth System Science data to serve the geosciences community in the scientific Python ecosystem. GeoCAT tools are built upon the cornerstone technologies in the Pangeo stack such as Xarray, Dask, and Jupyter Notebooks. In addition, some of the functionalities in the GeoCAT stack are inspired/reimplemented from NCL.
MetPy#
MetPy is a collection of tools in Python for reading, visualizing, and performing calculations with weather data. The website below has information about getting started as well as examples and a reference guide.
NCAR Command Language (NCL)#
NCL is an open source tool developed at NCAR that is free to download and use. It can be run at the command line in interactive mode or as a batch mode. While once a widely used language for CESM developers and users, NCL is now in a maintenence stage and is no longer in development. Much of the active development is now being done with python tools.
NCL is known for:
easy input/output use with netCDF, Grib, Grib2, shapefiles, ascii, and binary files.
good graphics that are very flexible.
functions tailored to the geosciences community.
a central website with 1000+ examples. There are also mini language and processing manuals.
Panopoly#
Panopoly is a graphic user interface (GUI) application that allows the user to quickly view data in a number of file formats. Panopoly is similar to ncview, but it’s more powerful. Panopoly works with files in netCDF, HDF, or GRIB format (among others). It also allows the user to perform simple calculations, apply masks, and quickly create spatial or line plots.
The Panopoly website provies more documentation, including How-To’s and demonstration videos.
Image Magick#
ImageMagick is a free suite of software that that can be used to display, manipulate, or compare images. It works with a wide range of file types (ps, pdf, png, gif, jpg, etc.). It can also be used to create movies. You can also alter an image at the command line. There are many options available when converting images, and more information can be found at the website below.