Converting raw binary data into VDC#
The two tools for converting raw binary data into VDC are vdccreate and raw2vdc. Advanced options can be seen here, or by issuing the commands without any arguments.
Create .vdc metadata file#
In the directory where Vapor 3 is installed, there is a command line utility called vdccreate. Issue this command in a terminal (Unix) or command prompt (Windows) with necessary arguments, followed by the name of the .vdc file to be written. You must at least include the variable names in your dataset as well as your grid size. Advanced options can be seen below.
vdccreate -dimension 512x512x512 -vars3d U:V:W myVDCFile.vdc
Advanced options for vdccreate:
.. _vdccreate:
Usage: vdccreate [options] master.nc
OPTION NUM_ARGS DEFAULT
------ -------- -------
-bs 1 64:64:64
Internal storage blocking factor expressed in grid points (NX:NY:NZ)
-cratios 1 500:100:10:1
Colon delimited list compression ratios. The default is 500:100:10:1.
The maximum compression ratio is wavelet and block size dependent.
-dimension 1 512x512x512
Data volume dimensions expressed in grid points (NXxNYxNZ)
-extents 1 ""
Colon delimited 6-element vector specifying domain extents in user
coordinates (X0:Y0:Z0:X1:Y1:Z1)
-force 0 false
Create a new VDC master file even if a VDC data directory already
exists. Results may be undefined if settings between the new master
file and old data directory do not match.
-help 0 false
Print this message and exit
-ncvars2dxy 1 ""
Colon delimited list of 2D XY-plane variable names (not compressed) to
be included in the VDC
-ncvars2dxz 1 ""
Colon delimited list of 3D XZ-plane variable names (not compressed) to
be included in the VDC
-ncvars2dyz 1 ""
Colon delimited list of 3D YZ-plane variable names (not compressed) to
be included in the VDC
-ncvars3d 1 ""
Colon delimited list of 3D variable names (not compressed) to be
included in the VDC
-nthreads 1 0
Specify number of execution threads 0 => use number of cores
-numts 1 1
Number of timesteps in the data set
-vars2dxy 1 ""
Colon delimited list of 2D XY-plane variable names (compressed) to be
included in the VDC
-vars2dxz 1 ""
Colon delimited list of 3D XZ-plane variable names (compressed) to be
included in the VDC
-vars2dyz 1 ""
Colon delimited list of 3D YZ-plane variable names (compressed) to be
included in the VDC
-vars3d 1 ""
Colon delimited list of 3D variable names (compressed) to be included
in the VDC
-wname 1 bior4.4
Wavelet family used for compression Valid values are bior1.1, bior1.3,
bior1.5, bior2.2, bior2.4 ,bior2.6, bior2.8, bior3.1, bior3.3, bior3.5,
bior3.7, bior3.9, bior4.4
-xtype 1 float
External data type representation. Valid values are uint8 int8 int16
int32 int64 float double
Generate VDC Data#
Once we have a .vdc file, the metadata has been recorded and we can transform the data into the VDC format. From Vapor 3’s installation directory, issue the command raw2vdc, followed by the arguments for your target variable and target timestep. The last argument will be the .vdc file that was made in Step 1. raw2vdc must be run on each data file that contains values for a single variable, at a single timestep.
raw2vdc -ts 1 -varname U myVDCFile.vdc binaryFile.bin
raw2vdc -ts 1 -varname V myVDCFile.vdc binaryFile.bin
raw2vdc -ts 1 -varname W myVDCFile.vdc binaryFile.bin
Advanced options for raw2vdc:
Usage: raw2vdc [options] vdcFile rawDataFile
OPTION NUM_ARGS DEFAULT
------ -------- -------
-debug 0 false
Enable diagnostic
-help 0 false
Print this message and exit
-lod 1 -1
Compression levels saved. 0 => coarsest, 1 => next refinement, etc. -1
=> all levels defined by the netcdf file
-nthreads 1 0
Specify number of execution threads 0 => use number of cores
-swapbytes 0 false
Swap bytes in data as they are read from disk
-ts 1 0
Specify time step offset
-type 1 float32
Primitive type of input data
-varname 1 var1
Name of variable
Copyright 2007 The National Center for Atmospheric Research
Version: 3.1.0 (2019-08-05T21:04:24Z) www.vapor.ucar.edu