HiRes-CESM Interactive Dashboard Example#

In this example, our goal is to create an interactive dashboard, using images already output on a remote server.

We are specifically interested in looking at plots created using the HighRes-CESM-Analysis repository.

Images created by this package can be visualized through an interactive dashboard using Panelify

import pandas as pd
import panel as pn
import panelify

pn.extension()

Accessing the Data (Plots)#

The plots from this example are stored on the CGD webext machine, specifically from this specific case, which we use as the path.

paths = (
    "https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA_HR.TL319_t13.004/",
    "https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA_HR.TL319_t13.003/",
)

Read in the CSV File Containing Metadata#

We use pandas to remotely read in the data, dropping the Unnamed column, and merging the absolute path to the directory and the relative filepaths

df_list = []
for path in paths:
    df = pd.read_csv(f"{path}png_catalog.csv").drop(columns="Unnamed: 0")

    # Convert the relative filepaths to absolute filepaths
    df["absolute_filepath"] = path + df.filepath.astype(str)

    df_list.append(df)

# Merge the dataframes
df = pd.concat(df_list)

df

	plot_type	varname	casename	apply_log10	time_period	isel_dict	filepath	date	absolute_filepath
0	histogram	CaCO3_FLUX_100m	g.e22.G1850ECO_JRA_HR.TL319_t13.004	False	0001-01-01_0001-12-31	{}	histogram/CaCO3_FLUX_100m.0001-01-01_0001-12-3...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
1	histogram	CaCO3_FLUX_100m	g.e22.G1850ECO_JRA_HR.TL319_t13.004	True	0001-01-01_0001-12-31	{}	histogram/CaCO3_FLUX_100m.0001-01-01_0001-12-3...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
2	histogram	CaCO3_FLUX_100m	g.e22.G1850ECO_JRA_HR.TL319_t13.004	False	0002-01-01_0002-12-31	{}	histogram/CaCO3_FLUX_100m.0002-01-01_0002-12-3...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
3	histogram	CaCO3_FLUX_100m	g.e22.G1850ECO_JRA_HR.TL319_t13.004	True	0002-01-01_0002-12-31	{}	histogram/CaCO3_FLUX_100m.0002-01-01_0002-12-3...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
4	histogram	CaCO3_FLUX_100m	g.e22.G1850ECO_JRA_HR.TL319_t13.004	False	0003-01-01_0003-12-31	{}	histogram/CaCO3_FLUX_100m.0003-01-01_0003-12-3...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
...	...	...	...	...	...	...	...	...	...
719	trend_map	SiO3	g.e22.G1850ECO_JRA_HR.TL319_t13.003	NaN	0002-01-01_0004-12-31	{'z_t': 0}	trend_map/SiO3.0002-01-01_0004-12-31.z_t--3510...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
720	trend_map	SiO3	g.e22.G1850ECO_JRA_HR.TL319_t13.003	NaN	0003-01-01_0004-12-31	{'basins': 0}	trend_map/SiO3.0003-01-01_0004-12-31.basins--A...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
721	trend_map	SiO3	g.e22.G1850ECO_JRA_HR.TL319_t13.003	NaN	0003-01-01_0004-12-31	{'basins': 0}	trend_map/SiO3.0003-01-01_0004-12-31.basins--G...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
722	trend_map	SiO3	g.e22.G1850ECO_JRA_HR.TL319_t13.003	NaN	0003-01-01_0004-12-31	{'basins': 0}	trend_map/SiO3.0003-01-01_0004-12-31.basins--I...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...
723	trend_map	SiO3	g.e22.G1850ECO_JRA_HR.TL319_t13.003	NaN	0003-01-01_0004-12-31	{'basins': 0}	trend_map/SiO3.0003-01-01_0004-12-31.basins--P...	NaN	https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA...

1969 rows × 9 columns

Dealing with Relative vs. Absolute Paths#

In the previous cell, we edited the filepaths… that is because the image filepaths are relative paths, but we would ideally like absolute paths. We make use of the dataframe path to assign the new absolute filepaths. An example is given below.

print("Original filepath: ", df.filepath.values[0])

# Convert the relative filepaths to absolute filepaths
df["filepath"] = path + df.filepath.astype(str)

print("New filepath ", df.filepath.values[0])

Original filepath:  histogram/CaCO3_FLUX_100m.0001-01-01_0001-12-31.png
New filepath  https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA_HR.TL319_t13.003/histogram/CaCO3_FLUX_100m.0001-01-01_0001-12-31.png

Build the Dashboard#

First, we check which types of plots are included in the analysis.

df.plot_type.unique()

array(['histogram', 'summary_map', 'time_series', 'trend_hist',
       'trend_map'], dtype=object)

Although there are five different plot types, we focus on the first three.

# Since we are using https, we leave this dictionary empty
storage_options = {}

# Create the timestep dashboard
summary_map = panelify.create_dashboard(
    keys=["casename", "varname", "date", "apply_log10"],
    df=df.loc[df.plot_type == "summary_map"],
    path_column="absolute_filepath",
    storage_options=storage_options,
    column_widget_types={"date": "discrete_slider"},
)

# Create the timeseries dashboard
time_series = panelify.create_dashboard(
    keys=[
        "casename",
        "varname",
    ],
    df=df.loc[df.plot_type == "time_series"],
    path_column="absolute_filepath",
    storage_options=storage_options,
)

# Create the histogram dashboard
histogram = panelify.create_dashboard(
    keys=["casename", "varname", "time_period", "apply_log10"],
    df=df.loc[df.plot_type == "histogram"],
    path_column="absolute_filepath",
    storage_options=storage_options,
    column_widget_types={"time_period": "discrete_slider"},
)

# Bring the various dashboard components into a single canvas
canvas = panelify.Canvas(
    {
        "Summary Map": summary_map.view,
        "Time Series": time_series.view,
        "Histogram": histogram.view,
    }
)

Run the Dashboard Inline#

Now that we have built the dashboard, we run it within the notebook.

hv_logo = '<a href="https://holoviz.org"><img src="https://holoviz.org/assets/holoviz-logo-stacked.svg" width=80></a>'
thumbnail = '<a href="https://github.com/marbl-ecosys/"><img src="https://raw.githubusercontent.com/NCAR/cesm-lens-aws/master/thumbnail.png" width=80 height=80></a>'
menu_background = "#659dbd"
menu_text = "<div><p>0.1 degree POP / CICE run with ocean BGC</p><br><br><p><a href='https://github.com/marbl-ecosys/HiRes-CESM-analysis'>HiRes-CESM analysis repo</a></p><p><a href='https://github.com/andersy005/HiRes-CESM-dashboard'>HiRes-CESM dashboard repo</a></p></div><br><br>"
menu = pn.Column(
    menu_text,
    pn.Row(
        pn.Spacer(width=10),
        pn.Pane(thumbnail),
        pn.Spacer(width=10),
        pn.Pane(hv_logo),
    ),
    background=menu_background,
)

dashboard = pn.Row(menu, canvas.show())
dashboard.servable("HiRes-CESM Diagnostics Dashboard")

Since these plots are not rendered directly in this notebook online, here is the ouput deployed using Dokku/Heroku

Jupyter Notebooks Tutorial FAQ Matplotlib Tutorial

19 March 2021

Recent Posts

Archives