Converting raw binary data into VDC

The two tools for converting raw binary data into VDC are vdccreate and raw2vdc. Advanced options can be seen here, or by issuing the commands without any arguments.

Create .vdc metadata file

In the directory where Vapor 3 is installed, there is a command line utility called vdccreate. Issue this command in a terminal (Unix) or command prompt (Windows) with necessary arguments, followed by the name of the .vdc file to be written. You must at least include the variable names in your dataset as well as your grid size. Advanced options can be seen below.

vdccreate -dimension 512x512x512 -vars3d U:V:W myVDCFile.vdc

Advanced options for vdccreate:

.. _vdccreate:

Usage: vdccreate [options] master.nc
    OPTION        NUM_ARGS    DEFAULT         
    ------        --------    -------         
    -bs           1           64:64:64        
        Internal storage blocking factor expressed in grid points (NX:NY:NZ) 
    -cratios      1           500:100:10:1    
        Colon delimited list compression ratios. The default is 500:100:10:1. 
        The maximum compression ratio is wavelet and block size dependent. 
    -dimension    1           512x512x512     
        Data volume dimensions expressed in grid points (NXxNYxNZ) 
    -extents      1           ""              
        Colon delimited 6-element vector specifying domain extents in user 
        coordinates (X0:Y0:Z0:X1:Y1:Z1) 
    -force        0           false           
        Create a new VDC master file even if a VDC data directory already 
        exists. Results may be undefined if settings between the new master 
        file and old data directory do not match. 
    -help         0           false           
        Print this message and exit 
    -ncvars2dxy   1           ""              
        Colon delimited list of 2D XY-plane variable names (not compressed) to 
        be included in the VDC 
    -ncvars2dxz   1           ""              
        Colon delimited list of 3D XZ-plane variable names (not compressed) to 
        be included in the VDC 
    -ncvars2dyz   1           ""              
        Colon delimited list of 3D YZ-plane variable names (not compressed) to 
        be included in the VDC 
    -ncvars3d     1           ""              
        Colon delimited list of 3D variable names (not compressed) to be 
        included in the VDC 
    -nthreads     1           0               
        Specify number of execution threads 0 => use number of cores 
    -numts        1           1               
        Number of timesteps in the data set 
    -vars2dxy     1           ""              
        Colon delimited list of 2D XY-plane variable names (compressed) to be 
        included in the VDC 
    -vars2dxz     1           ""              
        Colon delimited list of 3D XZ-plane variable names (compressed) to be 
        included in the VDC 
    -vars2dyz     1           ""              
        Colon delimited list of 3D YZ-plane variable names (compressed) to be 
        included in the VDC 
    -vars3d       1           ""              
        Colon delimited list of 3D variable names (compressed) to be included 
        in the VDC 
    -wname        1           bior4.4         
        Wavelet family used for compression Valid values are bior1.1, bior1.3, 
        bior1.5, bior2.2, bior2.4 ,bior2.6, bior2.8, bior3.1, bior3.3, bior3.5, 
        bior3.7, bior3.9, bior4.4 
    -xtype        1           float           
        External data type representation. Valid values are uint8 int8 int16 
        int32 int64 float double 

Generate VDC Data

Once we have a .vdc file, the metadata has been recorded and we can transform the data into the VDC format. From Vapor 3’s installation directory, issue the command raw2vdc, followed by the arguments for your target variable and target timestep. The last argument will be the .vdc file that was made in Step 1. raw2vdc must be run on each data file that contains values for a single variable, at a single timestep.

raw2vdc -ts 1 -varname U myVDCFile.vdc binaryFile.bin
raw2vdc -ts 1 -varname V myVDCFile.vdc binaryFile.bin
raw2vdc -ts 1 -varname W myVDCFile.vdc binaryFile.bin

Advanced options for raw2vdc:

Usage: raw2vdc [options] vdcFile rawDataFile
    OPTION       NUM_ARGS    DEFAULT    
    ------       --------    -------    
    -debug       0           false      
        Enable diagnostic 
    -help        0           false      
        Print this message and exit 
    -lod         1           -1         
        Compression levels saved. 0 => coarsest, 1 => next refinement, etc. -1 
        => all levels defined by the netcdf file 
    -nthreads    1           0          
        Specify number of execution threads 0 => use number of cores 
    -swapbytes   0           false      
        Swap bytes in data as they are read from disk 
    -ts          1           0          
        Specify time step offset 
    -type        1           float32    
        Primitive type of input data 
    -varname     1           var1       
        Name of variable 

Copyright 2007 The National Center for Atmospheric Research

Version: 3.1.0 (2019-08-05T21:04:24Z) www.vapor.ucar.edu