7. Utilities
Introduction
The output file of mcarats code is in a simple binary format, which is easily treated with GrADS. Several utility tools are provided to process the data files. It is not difficult to make a new tool to process the data files. If you made a useful tool, you can contribute to the project by providing it.
bin_couple to do a simple calculation with a couple of binary data files
bin_convert to convert a binary data with a simple equation
bin_dump to convert binary data to a text format
bin_exposure to diagnose recommended exposure data for image generation
bin_gray to generate gray scale images from a binary data file
bin_sample to (re)sample data from a binary data file
bin_stat to calculate spatial statistics of binary-format data
row_stat to calculate spatial statistics of text-format data
When operating GrADS data files, control files are assumed to exist at the same directory as the data files.
bin_couple
This is used to do a simple calculation with a couple of binary data files.
usage: bin_couple method thresh inCtlFile1 inCtlFile2 outfile
method:
0 (sum) : in1 + in2
1 (difference) : in1 - in2
2 (product) : in1 * in2
3 (ratio) : in1 / in2
4 (relative diff. 1) : (in1 - in2) / in2
5 (relative diff. 2) : 2*(in1 - in2) / (in1 + in2)
6 (square diff.) : in1*in1 - in2*in2
7 (root square diff.): sqrt(in1*in1 - in2*in2)
8 (diff. square) : (in1 - in2)*(in1 - in2)
9 (root diff.) : sqrt(in1) - sqrt(in2)
thresh: threshold of the denominator when method=3-5.
If the denominator is less than the threshold, then the output value is reset
at a very small value (-1.0E+30).
inCtlFile1 : GrADS control file for input data file 1
inCtlFile2 : GrADS control file for input data file 2
outfile : Output data file
The output file is in the same format as the input. Numbers of data elements and dimension sizes should be the same between the two input files.
bin_convert
This is used to convert binary data with a simple equation.
usage: bin_convert nx ny method A B inCtlFile outfile
nx : # of x-dimension elements for output.
ny : # of y-dimension elements for output.
method=0 : v = u (does nothing)
1 : v = A + B * u (linear)
2 : v = A * u^B (power-law)
3 : v = exp(A + B*u) (exponential)
4 : v = ln(A + B*u) (logarithmic)
5 : v = A + (1 + B*G)*u (random noises are superimposed)
where G is a normalized Gaussian noise (average: 0; variance:1).
inCtlFile : GrADS control file for an input file
If nx*ny=0, then these are reset as defined in the input file. Otherwise the averaging is operated.
The output file is in the same format (binary) as the input, but if nx or ny is smaller than the input, then the amount of output data is reduced.
bin_exposure
This computes recommended exposure data, which can be used as Rmax values for bin_gray utility. For multiple data files (e.g., each RGB channel), average exposure can be calculated. If data for multiple time steps are available, time series of exposure values can be convolved by a filter with a user-specified time width and time lag. The time lag is useful to realistically simulate that real optical instrument and human eyes adjust their aperture size with a time delay.
This tool would be useful as a utility for image sequence generation, for example.
usage: bin_exposure timeLag timeWid fmax ctlFile1 (ctlFile2 ctlFile3 ...)
timeLag : time lag (integer)
timeWid : temporal filter width (integer, 1 means no filtering)
fmax : factor for automatic determination of Rmax (usually around 1.2)
ctlFile : GrADS control files for binary data
See also bin_gray for how obtained data can be used.
bin_gray
This generates gray scale images in a simple 1-byte-per-pixel format.
usage: bin_gray factor Rmax power controlFile outfileHead (iz ivar it) (< exposureFile)
controlFile: GrADS control file for the input data file
outfileHead: filename head for output files
Scaling formula: Y = min(Rmax, X * fac)
Conversion formula: Z = int(Y / Rmax**pwr * 255) = [0, 255]
If Rmax == 0, then Rmax is read in from standard input (exposureFile).
If iz, ivar, and it are given, only the specified data will be processed.
Appropriate exposure data can be pre-determined and recorded in the exposureFile. If Rmax=0, then the pre-determined exposure data (Rmax values) will be read in from the stdin and used. A tool, bin_exposure, was made for this purpose.
Multiple output files will be generated if multiple data sets are stored in the input file. One output file should contains one image data.
Example of usage: at examples/ directory,
$ ../bin/bin_gray 1.182 1.1 0.75 out_ci0045.ctl img_ci0045
400 300 1.1000000 img_ci0045_t1_var1_z1.gray
400 300 1.1000000 img_ci0045_t1_var2_z1.gray
Generated files will be listed with numbers of pixels and Rmax values. If ImageMagick is installed and "convert" command is available, then you can try
$ convert -depth 8 -size 640x480+0 img_ci0045_t1_var1_z1.gray img_ci0045_t1_var1_z1.png
This will generate a PNG format image file. Then you can view the image, by
$ display -rotate 180 img_ci0045_t1_var1_z1.gif
bin_dump
This converts binary data to a text format.
usage : bin_dump inCtlFile > outfile
inCtlFile : GrADS control file for an input file
outfile : output file (text format)
Results are printed in the standard output. Example of usage:
$ bin/bin_dump examples/out_rs0067.ctl | less
# it, ivar, n : 1 1 1
0.43441936 0.39797837 0.37406304 0.43220648 0.42556950 ...
...
bin_sample
This is used to resample data from a binary data file. This is useful to extract partial data from the file.
usage: bin_sample ixi ixs nx iyi iys ny inCtlFile outfile
ixi : initial index for X
ixs : step of X-index
nx : # of x-pixels for output
iyi : initial index for Y
iys : step of Y-index
ny : # of y-pixels for output
An example is here: at examples/ directory,
% ../bin/bin_sample 2 2 30 2 2 30 out_rs0067.ctl sample_rs0067
In this case, original data for 60x60 pixels are available, and output result from the above will be for 30x30 pixels.
bin_stat
This is used to calculate basic statistics of binary-format datasets.
usage : bin_stat inCtlFile (mmask thresh1 thresh2) > outfile
inCtlFile : input GrADS control file
outfile : output file (text format)
mmask=0 : no data masking
1 : dat > thresh1
2 : dat < thresh1
3 : thresh1 < dat < thresh2
4 : dat < thresh1 or dat > thresh2
Output data file reports fraction of data possibly masked by the threshold(s), minimum, maximum, average, standard deviation, root-mean-square (rms), and skewness of read data set. One set of data corresponds to X*Y two dimensional array, which is defined in the control file. The output are printed to stdout.
Usage example:
$ bin/bin_stat examples/test_rs0067.ctl
# iz, ivar, it, frac, min, max, average, stdev, rms, skewness
1 1 1 1.0000 3.3940E-01 6.8978E-01 4.8468E-01 7.1038E-02 4.8986E-01 4.2904E-01
1 2 1 1.0000 3.6220E-01 2.2394E+00 1.0513E+00 4.2648E-01 1.1345E+00 8.8130E-01
...
row_stat
This is used to calculate basic statistics of a text data sequence.
usage: row_stat N func ithresh (thresh1 thresh2) < infile > outfile
N : # of sequential data for calculating statistics, in a line
func : 0, raw data; 1, ln(dat); 2, exp(dat);
The conversion is performed before the statistics computation.
ithresh
=0 : no threshold,
=1 : dat >= thresh1,
=2 : dat <= thresh1,
=3 : thresh1 <= dat <= thresh2,
=4 : dat <= thresh1 or dat >= thresh2
Output data file reports fraction of data filtered by the threshold(s), minimum, maximum, average, standard deviation, root-mean-square (rms), and skewness of read data set. One set of data is read in from a line in the input.
Usage example:
$ bin/bin_dump examples/out_rs0067.ctl > tmp
$ bin/row_stat 3600 0 0 < tmp
# frac, min, max, average, stdev, rms, skewness
1 1.0000 3.0509E-01 7.0611E-01 4.8130E-01 7.7988E-02 4.8757E-01 2.8613E-01
2 1.0000 3.7049E-01 2.2713E+00 1.0711E+00 4.4416E-01 1.1595E+00 9.5030E-01
...
The output are printed to stdout. Statistics for data sequences, each one of which contains 3600 data, are reported in this example.