Database

METviewer is a database-and-display system built around the ASCII statistics that the MET tools produce. Instead of re-parsing thousands of .stat files every time you want a plot, you load that output once into a relational database, then query it interactively or in batch. This page explains the three pieces that make that work: the schema that stores the data, the loader that fills it, and the batch engine that turns any plot into a fully scriptable XML specification.

Why a database sits under METviewer

MET’s statistics tools (Point-Stat, Grid-Stat, Ensemble-Stat, MODE, MTD, and the tropical-cyclone tools) write their results as columnar ASCII text. Each row is a line of a particular line type — for example a CNT continuous-statistics line or a CTC contingency-table-counts line — and each line carries a header that records the context: which model, which forecast variable and level, which lead time, which verification region, and the valid and initialization times.

That format is perfect for a tool to emit but awkward to explore. To draw a single time series you might need to scan many files, filter to one model and one region, and pull one column out of one line type. METviewer solves this by loading the ASCII output into a relational database once. After loading, every plot is just a query: pick the fixed values you care about, pick the statistic, and let the database do the filtering and aggregation.

In one sentence

METviewer is “a database and display system designed to work with the ASCII output statistics produced by the MET software.” The database is the durable store; the GUI and the batch engine are two front ends that query it.

The relational schema, at a high level

The schema mirrors the structure of MET output: a small family of header tables that record the context of each block of output, paired with the line data that holds the actual statistics. A header row answers “what is this and where did it come from?” while the line-data rows hold the numbers, linked back to their header.

Header tables

The loading documentation refers to header tables by name. Conceptually each corresponds to one output family:

Table	Holds the header for	Loaded by
stat_header	STAT output — the line types written by Point-Stat, Grid-Stat, Ensemble-Stat, and related tools	<load_stat>
mode_header	MODE object-based output	<load_mode>
mtd_header	MODE Time Domain (MTD) output	<load_mtd>
instance_info	Bookkeeping about each load run — a record is created per load, and it can carry your load notes / description	every load

The loader exposes <stat_header_db_check>, <mode_header_db_check>, and <mtd_header_db_check> options that control whether it checks the corresponding header table for an existing match before inserting — a performance knob, since header rows are heavily shared.

What a header keys on

A header row captures the verification context. Across the MET line types these are the dimensions you typically slice a plot by:

Model & run

The model name, plus the initialization and valid times, and the forecast lead.

Field

The forecast variable and its vertical level, with the matching observation field.

Region

The verification mask / region the statistics were computed over.

Because the header carries all of this, a plot specification can hold most of these fixed (one model, one region, one variable) and let one of them vary along an axis — for example, lead time on the x-axis of a series plot.

On exact column names

This guide names the header tables and the slicing dimensions the documentation describes, but it does not invent exact column names for the line-data tables. The loading chapter names the header tables and the load flags; the precise per-column layout of each line-data table is part of the schema definition shipped with the tooling rather than spelled out on that chapter.

Loading: one XML load specification drives everything

Loading is performed by a load-script entry point (the documentation invokes it as mv_load.sh) that is driven entirely by a single XML document whose root element is <load_spec>. The load spec is the complete, declarative description of what to load, from where, and into which database — so a load is itself reproducible.

The load spec gathers several concerns under that one root:

Where to connect — A <connection> block supplies the database host, database name, and credentials.

What time window — A <date_list> bounds the dates being loaded.

What kinds of data — Boolean switches — <load_stat>, <load_mode>, <load_mtd>, <load_mpr> (matched pairs), and <load_orank> (observation rank) — turn each output family on or off.

Which files — Folder templates and a <load_files> / <folder_tmpl> + <load_val> mechanism point the loader at the files to ingest. For STAT, the loader will pick up files with the .stat suffix.

How to behave — Tuning and safety options: <insert_size> (batch insert size), <force_dup_file> (re-load files already loaded), <verbose>, the *_db_check header options, index-management options, and a <group> / <description> that annotate the load in instance_info.

Input comes from MET’s ASCII products: the STAT files (.stat) written by the statistics tools, MODE and MTD object output, and the tropical-cyclone statistics (.tcst) tracked through the same load machinery. Each is routed by its corresponding load_* flag into the matching header + line-data tables.

Ecosystem link

In the modern METplus ecosystem the database loading is performed by METdbLoad, the loader that lives in METdataio. It reads the same XML load specification and writes into the same schema described here — so think of “the loader” on this page and METdataio’s METdbLoad as the same job, described from two sides.

Database engines

METviewer stores its data in a SQL relational database. The documented engine is MySQL (the loader’s troubleshooting examples reference the MySQL JDBC driver and MySQL constraint errors). In practice the MariaDB drop-in is used interchangeably with MySQL across the METplus ecosystem; both speak the same SQL dialect and JDBC interface that the loader expects.

Practical note

Because the connection is just host + database + credentials in the <connection> block, the same load spec and plot spec work against any compatible MySQL/MariaDB server — local, shared, or containerized.

Source caveat: the loading chapter explicitly references MySQL. MariaDB compatibility is stated here as ecosystem practice rather than quoted from that page.

The load-and-query path

The whole system is a pipeline with the database in the middle. ASCII output is loaded once on the left; on the right, either the GUI or the batch engine turns a selection into a query and a plot.

Figure 1. MET ASCII output is loaded once via an XML load spec; the loader (METdbLoad, in METdataio) writes header and line-data tables into MySQL/MariaDB. METviewer queries those tables, and a plot XML (<plot_spec>) captures the selection so the query can be replayed in batch.

The batch engine: a plot is an XML script

Anything you can build in the METviewer GUI can also be expressed as a plot specification — an XML document whose root is <plot_spec>. Because the XML captures every selection (the data range, the fixed field values, the statistic, the series, and the rendering details), a plot becomes fully reproducible and scriptable outside the GUI. You can version-control it, diff it, and regenerate the exact same figure later.

The batch engine runs a plot spec from the command line via mv_batch:

mv_batch [-list] [-printSql] plot_spec_file [job_name]

plot_spec_file — The XML plot specification document to run.

job_name (optional) — Run only the named plot job from the file (a spec may define several).

-list — List the available plot jobs in the file without executing them.

-printSql — Print the SQL statements the engine generates — invaluable for understanding exactly which query a plot turns into.

Why this matters

The GUI is great for exploration, but it is not reproducible on its own. The plot spec is the durable artifact: the same <plot_spec> + the same database = the same plot, every time, on any machine, with no clicking.

Inside a plot specification

The plot spec mirrors the database in XML. A handful of sections decide which rows are selected, how they map to axes, and how the rendered figure is labeled.

In the GUI (see the plot gallery)	In the plot XML
Fixed values	<plot_fix>
Series variable (lines/colors)	<series1> / <series2>
Independent variable (X axis)	<indep>
Y-axis statistic	<dep1> / <dep2> (each a <fcst_var> + <stat>)

The same choices the GUI offers, and the XML element each one becomes — so a plot built by clicking maps directly onto the batch spec.

Where to connect & where to write

A <connection> block points at the database; <rscript> selects the Rscript used to render, and <folders> (with <r_tmpl>, <r_work>, <plots>) sets the template, working, and output directories.

The <rscript>, <r_tmpl>, and <r_work> elements reflect METviewer’s historical R rendering path; newer releases deprecate R in favor of Python and METplotpy (see the overview), so treat these R-oriented elements as the legacy rendering route.

Which rows to select

<plot_fix> — The fixed database field values that stay constant for this plot — for example, pin to one model, one forecast variable and level, one verification region, one lead. Each distinct combination of fixed values produces its own query.

<plot_cond> — An additional SQL condition clause that further constrains which records are pulled.

<date_list> · <date_range> · <date_range_list> — The temporal window for the selection.

How rows map to the figure

<indep> — The independent (x-axis) variable — commonly a database dimension such as lead time.

<series1> · <series2> — The grouping fields that split the data into separate curves/bars.

<dep1> · <dep2> — The dependent (y-axis) quantities. Each carries a <fcst_var> and a <stat> — i.e. the forecast field and the statistic (the column of a line type) being plotted.

<template> — Which R plotting template renders the figure — for example a series plot, bar plot, box plot, or ROC template.

How the figure is labeled

A <tmpl> block names the intermediate artifacts and labels — the data file, the plot file, the R file, the title, axis labels and caption — and a <val_map> provides key/value substitutions used in templated text and paths.

Plots inherit

A <plot> can declare name, inherits, and run attributes, so one base plot can seed many variants — change the region or the statistic and re-run, without repeating the whole spec.

Workflow: load once, then plot

1. Point at your MET output. Gather the .stat (and .tcst, MODE, MTD) files you want to analyze, and note the directory layout — you will template the path in the load spec.
2. Write the load spec. In <load_spec>, fill in <connection>, the <date_list>, the load_* flags for the output families you have, and the file/folder templates. Add a <description> so the run is annotated in instance_info.
3. Load the database. Run the loader (METdbLoad, via the load script) against the spec. It inserts header rows into stat_header / mode_header / mtd_header and the matching line data, de-duplicating headers via the *_db_check options.
4. Build a plot — in the GUI or by hand. Choose the fixed values, the independent axis, the series, and the statistic. The GUI writes these into a <plot_spec>; you can also author the XML directly.
5. Run it in batch. Execute mv_batch plot_spec_file. Add -printSql to inspect the generated query, or -list to enumerate jobs.
6. Reproduce & share. Commit the plot spec. Anyone with the same database can regenerate the identical figure by re-running the same XML — no clicking required.

Ecosystem link

The loading half of this workflow is owned by METdataio and its METdbLoad loader. METviewer focuses on the querying and plotting half once the data is in the database.

Element reference

A quick map of the XML elements named on this page and what they do.

Element	Document	Role
<load_spec>	load	Root of the load specification.
<connection>	load & plot	Database host, name, and credentials.
<date_list>	load & plot	Date window to load or query.
<load_stat>	load	Load STAT output (.stat) into stat_header + line data.
<load_mode> / <load_mtd>	load	Load MODE / MTD object output.
<load_mpr> / <load_orank>	load	Load matched-pair / observation-rank data.
<stat_header_db_check>	load	Check the header table for an existing match before insert (perf knob).
<plot_spec>	plot	Root of the batch plot specification.
<plot_fix>	plot	Fixed field values that constrain the query.
<plot_cond>	plot	Extra SQL condition clause.
<indep>	plot	Independent (x-axis) variable.
<series1> / <series2>	plot	Grouping fields that split the data into curves.
<dep1> / <dep2>	plot	Y-axis quantities — each with <fcst_var> + <stat>.
<template>	plot	R plotting template (series / bar / box / ROC).
<tmpl> / <val_map>	plot	Output file names, labels, and template substitutions.

Element and table names above are drawn from the METviewer database-loading and batch chapters. Where a fact is ecosystem practice rather than a direct quote (MariaDB; METdbLoad as the loader implementation), it is flagged inline.

---

A derived, human-readable re-presentation — not official documentation. Sources: METviewer User’s Guide — Database Loading · Batch Engine · METviewer User’s Guide