Creating a .vdf file

The first step in converting your data to a VDC is to generate a .vdf metafile that describes your data. Several methods exist for creating a .vdf file. This document describes the creation of a .vdf file using command line tools included in the VAPOR package. The choice of which command line tool to use depends on the source of your data, as show in the table below. Additionally, the VDCWizard GUI may be used to generate a .vdf file. File formats supported by vdcwizard are also shown in the table below.

Creating a .vdf file from a WRF-ARW data set

As a convenience to WRF-ARW users a command line utility called wrfvdfcreate may be used. The WRF output files to be converted must be in netCDF format and have the following dimensions defined: west_east, south_north, bottom_top, and their staggered versions, and Time.  The PH, PHB, and Times variables must also be present. Unlike vdfcreate, the wrfvdfcreate command obtains most if its metadata information directly from the WRF-ARW data set, stored as an ordered sequence of netCDF files. Very few command line options need to be specified. Critical .vdf elements, such as the time, are determined automatically from the netCDF input data.

Example 1: converting all variables

Assuming the WRF-ARW data set consists of three netCDF output files are named wrfout_2005-08-29_00, wrfout_2005-08-29_01, and wrfout_2005-08-29_02, the following command invocation would generate a .vdf file suitable for containing all variables at all time steps in the three files:

wrfvdfcreate wrfout2005082900 wrfout2005082901 wrfout2005082902 mywrfdata.vdf

WRF-ARW data set requirements

To operate on a WRF-ARW with VAPOR requires that the data set is stored in the netCDF file format. Furthermore, the following dimensions and variables must be present:

• Required dimension names: WEST_EAST, SOUTH_NORTH, BOTTOM_TOP, as well as the staggered versions of these dimensions
• Required variable names: PH, PHB, Times

Creating a .vdf file from a MOM, POP, ROMS, CAM, or GRIB data set

For a discussion on the conversion of MOM, POP, or ROMS ocean model data consult the momvdfcreate (for both MOM and POP), or romsvdfcreate (for ROMS) manual pages as appropriate.  Converters for GRIB and CAM can be found on the gribvdfcreate and camvdfcreate manual pages.

Creating a .vdf file from other netCDF data sets

If your data are stored in netCDF files, and were not generated from one of the numerical models listed above (e.g. ROMS, POP, MOM, and WRF) you may be able to use the ncdfvdfcreate command. You should only use this command if one of the model-specific commands described above are not appropriate for your data. The ncdfvdfcreate command assumes only that your data are sampled on a 2D or 3D structured grid, and are stored in one or more netCDF files.

Creating a .vdf file with vdfcreate

If a model-specific command line tool does not exist for your data the vdfcreate command line utility may be used. The user may specify a limited number of metadata elements including: the volume dimension, number of time steps, number of data approximations, and the variable names. Not all of the XML elements supported by VAPOR's .vdf file can be specified with vdfcreate. For example, vdfcreate does not support setting comments on variables. However, for most applications the capabilities of vdfcreate are sufficient. If more advanced features of the .vdf metafile are needed one of VAPOR's programatic interfaces may be used to build a specialized .vdf creation utility.

Example 1: scalar 3D data

Assuming you have a gridded data set with the following attributes:

• the computation grid has a sampling resolution of  512^3;
• the 3D solution field variables are temperature and pressure, named t and p;
• and there are 90 time steps saved.

The command invocation:

vdfcreate -dimension 512x512x512 -numts 100 -level 3 -vars3d t:p mydata.vdf

would produce a .vdf file named mydata.vdf. This file would describe a type I VDC containing up to 100 time steps, starting from 0 and running through 99; two 3D field variables, t and p; each volume would have a spatial resolution of 512^3; and three wavelet transforms would be applied. Thus the data will be accessible at the following resolutions: 512^3, 256^3, 128^3, and 64^3.

Other things to note about this example:

1. Although the original data contained 90 time steps we specified a .vdf file with up to 100 time steps. Having room for more time steps in a .vdf file than necessary does not present a problem, or impose any significant storage overhead (a few hundred bytes). The number of time steps in a .vdf file must be at least as great as the number of actual time steps we intend to populate the VDC with.
2. We did not specify a time coordinate for each of the time steps. Hence, vdfcreate will use a default time sampling of 1.0. This is generally OK for scalar data, but for flow data we will want to provide correct time sampling information.
3. The X, Y, and Z dimensions specified as an argument to the -dimension option, although irrelevant in this example, are given in the order from fastest varying dimension to slowest (X followed by Y, then Z). I.e. NXxNXxNZ.

Example 2: scalar and vector 3D data

Although vaporgui's scalar visualization tools will function properly without correct time information, many flow analysis tools will not. In this example we now have, in addition to our scalar temperature and pressure data above, the three components of a velocity field named, vx, vy, and vz. Moreover, the time sampling between time steps in the user's coordinate system is uniformly sampled at 100.0 units.

vdfcreate -dimension 512x512x512 -numts 100 -level 3 -vars3d t:p:vx:vy:vz -start 0.0 -deltat 100.0 foo.vdf

In this example the time associated with the first time step in the file is set to 0.0, the second time step will have a time of 100.0, the third 200.0, and so on.

Example 3: 2D data

The previous examples used only 3D field variables. The VDC supports 2D field variables as well, provided they are sampled on one of the three axis-aligned planes of the 3D grid. For example, if we had a 2D variable sampled on the XY plane that contained the total moisture in a column of air along the Z axis we could specify its inclussion in a .vdf file with the command:

vdfcreate -dimension 512x512x512 -numts 100 -level 3 -vars3d t:p -vars2dxy totalmoisture mydata.vdf

Example 4: Type II VDC

The previous examples produced .vdf metafiles for a type I VDC. To create a type II VDC a type II .vdf metafile is needed. To create a type II VDC the -vdc2 option is used. For example:

vdfcreate -vdc2 -dimension 512x512x512 -vars3d t:p myvdc.vdf

The resulting VDC will use the default compression ratios. To specify alternate compression ratios the -cratio command line option may be used.

Learning more

Complete documentation on the vdfcreate and wrfvdfcreate command line utilities is available in the reference manual.

Invoking either command with the -help option will generate a listing of all the available command line options and provide a terse description of their meaning.

Information on various programatic APIs for generating a .vdf metafile may be found here [need a link]