|
PIO
2.5.4
|
To build both the Unit and Performance tests for PIO, follow the general instructions for building PIO in either the Installation page or the [Machine Walk-Through](CMake Install Walk-through) page. During the Build step after (or instead of) the make command, type make tests.
The Parallel IO library comes with more than 20 built-in unit tests to verify that the library is installed and working correctly. These tests utilize the CMake and CTest automation framework. Because the Parallel IO library is built for parallel applications, the unit tests should be run in a parallel environment. The simplest way to do this is to submit a PBS job to run the ctest command.
For a library built into the example directory /scratch/user/PIO_build/, an example PBS script would be:
#!/bin/bash #PBS -q normal #PBS -l nodes=1:ppn=4 #PBS -N piotests #PBS -e piotests.e$PBS_JOBID #PBS -o piotests.o$PBS_JOBID cd /scratch/user/PIO_build ctest
The output from the unit tests will be reported in the piotests.o$JOBID file. This should look something like:
Another option would be to launch an interactive session, change into the build directory, and run the ctest command.
On Yellowstone, the unit tests can run using the execca or execgy commands as:
> setenv DAV_CORES 4 > execca ctest
To run the performance tests, you will need to add two files to the tests/performance** subdirectory of the PIO build directory. First, you will need a decomp file. You can download one from our google code page here: https://svn-ccsm-piodecomps.cgd.ucar.edu/trunk/ .
You can use any of these files, and save them to your home or base work directory. Secondly, you will need to add a namelist file, named "pioperf.nl". Save this file in the directory with your pioperf executable (this is found in the tests/performance subdirectory of the PIO build directory).
The contents of the namelist file should look like:
&pioperf decompfile = "/u/home/user/piodecomp30tasks01dims06.dat" pio_typenames = 'pnetcdf' niotasks = 30 rearrangers = 1 nvars = 2 /
Here, the second line ("decompfile") points to the path for your decomp file (wherever you saved it). For the rest of the lines, each item added to the list adds another test to be run. For instance, to test all of the types of supported IO, your pio_typenames would look like:
pio_typenames = 'pnetcdf','netcdf','netcdf4p','netcdf4c'
HDF5 is netcdf4p, and Parallel-Netcdf is pnetcdf.
To test with different numbers of IO tasks, you could do:
niotasks = 30,15,5
(These tasks are the subset of the run tasks that are designated IO tasks)
To test with both of the rearranger algorithms:
rearrangers = 1,2
(Each rearranger is a different algorithm for converting from data in memory to data in a file on disk. The first one, BOX, is the older method from PIO1, the second, SUBSET, is a newer method that seems to be more efficient in large numbers of tasks)
To test with different numbers of variables:
nvars = 8,5,3,2
(The more variables you use, the higher data throughput goes, usually)
To run, submit a job with 'pioperf' as the executable, and at least as many tasks as you have specified in the decomposition file. On yellowstone, a submit script could look like:
#!/bin/tcsh #BSUB -P P00000000 # project code #BSUB -W 00:10 # wall-clock time (hrs:mins) #BSUB -n 30 # number of tasks in job #BSUB -R "span[ptile=16]" # run 16 MPI tasks per node #BSUB -J pio_perftest # job name #BSUB -o pio_perftest.%J.out # output file name in which %J is replaced by the job ID #BSUB -e pio_perftest.%J.err # error file name in which %J is replaced by the job ID #BSUB -q small # queue #run the executable mpirun.lsf /glade/p/work/katec/pio_work/pio_build/tests/performance/pioperf
The result(s) will look like a line in the output file such as:
You can decode this as:
Last updated: 05-17-2016
1.8.17