Introduction

Welcome to the GDEX Examples documentation! This repository provides example notebooks and scripts that demonstrate how to access data from NCAR’s GDEX for cool geoscience applications and visualizations.

How is the repository organized?¶

While intake-ESM was originally designed for Earth System Model output, we extend its capabilities to support a broader range of Earth science data including:

• Observations (satellite, in-situ measurements)

• Reanalysis datasets (ERA5, JRA-3Q, etc.)

• Model output (CESM, CMIP, etc.)

• Other Earth science datasets

Key Features¶

🛠️ Custom Catalog Generation¶

Our primary tool generator/create_catalog.py creates intake-ESM catalogs for any dataset directory with flexible configuration options for different data formats and structures.

🌐 Multiple Access Methods¶

Generated catalogs support three access patterns :

• POSIX: Direct filesystem access for NCAR HPC users

• HTTPS: Web-based access for remote users

• OSDF: Distributed access through Open Science Data Federation

📊 Broad Dataset Support¶

Compatible with diverse data formats including NetCDF, Zarr, and Kerchunk reference files, following vocabulary conventions used by major data providers (DKRZ, Copernicus, NASA, NOAA).

Quick Start¶

Generate a basic catalog:

python generator/create_catalog.py /path/to/data \
    --out /output/directory \
    --catalog_name my_catalog \
    --description "My dataset catalog"

For comprehensive usage examples:

• NCAR HPC users: gdex-examples

• OSDF users: osdf_examples

Repository Structure¶

• generator/ - Core catalog generation tools

• notebooks/ - Example Jupyter notebooks demonstrating usage

• examples/ - Python script examples for generating dataset catalog

• test/ - Test scripts and validation tools

Content¶

This documentation provides: