HiRes-CESM Interactive Dashboard Example#

In this example, our goal is to create an interactive dashboard, using images already output on a remote server.

We are specifically interested in looking at plots created using the HighRes-CESM-Analysis repository.

Images created by this package can be visualized through an interactive dashboard using Panelify

import pandas as pd
import panel as pn
import panelify

pn.extension()

Accessing the Data (Plots)#

The plots from this example are stored on the CGD webext machine, specifically from this specific case, which we use as the path.

paths = (
    "https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA_HR.TL319_t13.004/",
    "https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA_HR.TL319_t13.003/",
)

Read in the CSV File Containing Metadata#

We use pandas to remotely read in the data, dropping the Unnamed column, and merging the absolute path to the directory and the relative filepaths

df_list = []
for path in paths:
    df = pd.read_csv(f"{path}png_catalog.csv").drop(columns="Unnamed: 0")

    # Convert the relative filepaths to absolute filepaths
    df["absolute_filepath"] = path + df.filepath.astype(str)

    df_list.append(df)

# Merge the dataframes
df = pd.concat(df_list)

df
plot_type varname casename apply_log10 time_period isel_dict filepath date absolute_filepath
0 histogram CaCO3_FLUX_100m g.e22.G1850ECO_JRA_HR.TL319_t13.004 False 0001-01-01_0001-12-31 {} histogram/CaCO3_FLUX_100m.0001-01-01_0001-12-3... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
1 histogram CaCO3_FLUX_100m g.e22.G1850ECO_JRA_HR.TL319_t13.004 True 0001-01-01_0001-12-31 {} histogram/CaCO3_FLUX_100m.0001-01-01_0001-12-3... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
2 histogram CaCO3_FLUX_100m g.e22.G1850ECO_JRA_HR.TL319_t13.004 False 0002-01-01_0002-12-31 {} histogram/CaCO3_FLUX_100m.0002-01-01_0002-12-3... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
3 histogram CaCO3_FLUX_100m g.e22.G1850ECO_JRA_HR.TL319_t13.004 True 0002-01-01_0002-12-31 {} histogram/CaCO3_FLUX_100m.0002-01-01_0002-12-3... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
4 histogram CaCO3_FLUX_100m g.e22.G1850ECO_JRA_HR.TL319_t13.004 False 0003-01-01_0003-12-31 {} histogram/CaCO3_FLUX_100m.0003-01-01_0003-12-3... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
... ... ... ... ... ... ... ... ... ...
719 trend_map SiO3 g.e22.G1850ECO_JRA_HR.TL319_t13.003 NaN 0002-01-01_0004-12-31 {'z_t': 0} trend_map/SiO3.0002-01-01_0004-12-31.z_t--3510... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
720 trend_map SiO3 g.e22.G1850ECO_JRA_HR.TL319_t13.003 NaN 0003-01-01_0004-12-31 {'basins': 0} trend_map/SiO3.0003-01-01_0004-12-31.basins--A... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
721 trend_map SiO3 g.e22.G1850ECO_JRA_HR.TL319_t13.003 NaN 0003-01-01_0004-12-31 {'basins': 0} trend_map/SiO3.0003-01-01_0004-12-31.basins--G... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
722 trend_map SiO3 g.e22.G1850ECO_JRA_HR.TL319_t13.003 NaN 0003-01-01_0004-12-31 {'basins': 0} trend_map/SiO3.0003-01-01_0004-12-31.basins--I... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
723 trend_map SiO3 g.e22.G1850ECO_JRA_HR.TL319_t13.003 NaN 0003-01-01_0004-12-31 {'basins': 0} trend_map/SiO3.0003-01-01_0004-12-31.basins--P... NaN https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...

1969 rows × 9 columns

Dealing with Relative vs. Absolute Paths#

In the previous cell, we edited the filepaths… that is because the image filepaths are relative paths, but we would ideally like absolute paths. We make use of the dataframe path to assign the new absolute filepaths. An example is given below.

print("Original filepath: ", df.filepath.values[0])

# Convert the relative filepaths to absolute filepaths
df["filepath"] = path + df.filepath.astype(str)

print("New filepath ", df.filepath.values[0])

Original filepath:  histogram/CaCO3_FLUX_100m.0001-01-01_0001-12-31.png
New filepath  https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA_HR.TL319_t13.003/histogram/CaCO3_FLUX_100m.0001-01-01_0001-12-31.png

Build the Dashboard#

First, we check which types of plots are included in the analysis.

df.plot_type.unique()

array(['histogram', 'summary_map', 'time_series', 'trend_hist',
       'trend_map'], dtype=object)

Although there are five different plot types, we focus on the first three.

# Since we are using https, we leave this dictionary empty
storage_options = {}

# Create the timestep dashboard
summary_map = panelify.create_dashboard(
    keys=["casename", "varname", "date", "apply_log10"],
    df=df.loc[df.plot_type == "summary_map"],
    path_column="absolute_filepath",
    storage_options=storage_options,
    column_widget_types={"date": "discrete_slider"},
)

# Create the timeseries dashboard
time_series = panelify.create_dashboard(
    keys=[
        "casename",
        "varname",
    ],
    df=df.loc[df.plot_type == "time_series"],
    path_column="absolute_filepath",
    storage_options=storage_options,
)

# Create the histogram dashboard
histogram = panelify.create_dashboard(
    keys=["casename", "varname", "time_period", "apply_log10"],
    df=df.loc[df.plot_type == "histogram"],
    path_column="absolute_filepath",
    storage_options=storage_options,
    column_widget_types={"time_period": "discrete_slider"},
)

# Bring the various dashboard components into a single canvas
canvas = panelify.Canvas(
    {
        "Summary Map": summary_map.view,
        "Time Series": time_series.view,
        "Histogram": histogram.view,
    }
)

Run the Dashboard Inline#

Now that we have built the dashboard, we run it within the notebook.

hv_logo = '<a href="https://holoviz.org"><img src="https://holoviz.org/assets/holoviz-logo-stacked.svg" width=80></a>'
thumbnail = '<a href="https://github.com/marbl-ecosys/"><img src="https://raw.githubusercontent.com/NCAR/cesm-lens-aws/master/thumbnail.png" width=80 height=80></a>'
menu_background = "#659dbd"
menu_text = "<div><p>0.1 degree POP / CICE run with ocean BGC</p><br><br><p><a href='https://github.com/marbl-ecosys/HiRes-CESM-analysis'>HiRes-CESM analysis repo</a></p><p><a href='https://github.com/andersy005/HiRes-CESM-dashboard'>HiRes-CESM dashboard repo</a></p></div><br><br>"
menu = pn.Column(
    menu_text,
    pn.Row(
        pn.Spacer(width=10),
        pn.Pane(thumbnail),
        pn.Spacer(width=10),
        pn.Pane(hv_logo),
    ),
    background=menu_background,
)

dashboard = pn.Row(menu, canvas.show())
dashboard.servable("HiRes-CESM Diagnostics Dashboard")

Since these plots are not rendered directly in this notebook online, here is the ouput deployed using Dokku/Heroku

https://img.shields.io/badge/%E2%86%91_Deploy_to-Heroku-7056bf.svg

Jupyter Notebooks Tutorial FAQ   Matplotlib Tutorial