Analysis and programming on HDF5

Abstract:

HDF5 is a hierarchical data format developed by the NCSA to store science data. HDF5 files can contain a variety of datasets like ‘n’ dimensional matrices, images, tables, scalar values and groups to organize them as per their category. Further, each object inside the file can contain attributes giving brief descriptions about them.

In this project, an appreciation of the capabilities of this data format, its suitability for remote sensing applications was studied. A software program to open, read, write and manipulate HDF5 objects was developed. To solve the remote sensing requirements, tools to

· Read and reorganize existing multiple 2D datasets into a single compact 3D dataset

· Export the 3D dataset into an ASCII file by which other Image Processing software that do not support HDF5 files can read the data.

· Subset the 3D dataset to desired dimensions

· Form a 3D cube visualization in which individual bands are stacked one behind with a mild offset in X and Y directions for perspective viewing.

were developed. The software was developed in C using the libraries and functions released by the “HDF group” for HDF5 file handling. The operations were performed on hyperspectral datasets of IMS-1 HySI sensor released by NRSC in HDF5 format.

At the end, a set of futuristic ideas with HDF5 files like holding pixel attribute information, multiple dimensional datasets, object meta data for logging the process run on each object, and all the above in a single file obviating the dependency on auxiliary files is discussed.

Introduction

In today’s scenario, with increasing number of satellite sensors and image-processing packages, the data formats too are increasing. Different data formats have different features suiting specific application requirements. Since not all packages support all formats and due to limited interoperability between data formats a need is felt for a versatile data format to house science data. HDF5 (Hierarchical Data Format 5) is one such developed by NCSA (National Centre for Supercomputing Applications). This report is organized into three sections, HDF5 organization is explained in the first section, the analysis of HySI datasets and about the software developed is explained in the second section. Finally, the observations, discussions and ideas for future are discussed in the concluding section.

HDF5 organization

A HDF5 file (extension *.H5) is a composite file which can contain a variety of objects like images, tables, datasets (matrices), links, attributes and groups. These objects are arranged hierarchically in a tree structure with the root group at the apex and other groups at the leaf nodes. All other objects are bundled inside groups and those objects without groups are kept directly under the root group.

Datasets in HDF5

Datasets are the base contents of HDF5 files. These are ‘n’ dimensional matrices. Every object type is essentially a dataset but with some variation. The value of attribute “CLASS” of objects differentiates between them.

There are two types of datasets, simple – with uniform datatype and compound – with multiple datatypes. Image data and other science data are held in simple datasets while tabular information is stored in compound datasets. Dataspaces contain information about the rank, dimensionality and datatype of datasets.

The ‘n’ dimensional matrices are highly suitable for storing satellite images systematically. Multi and hyper spectral images can be stored in Band Sequential (BSQ) format in 3D matrices. The coordinates [X] and [Y] can give the pixel location and [Z] can give the band number.

Images in HDF5

Images are datasets with attribute CLASS = IMAGE. There are two types of images in HDF5. One is a 8 bit, 2D raster images called indexed images. Here each pixel as correspondingly linked to a palette for colour information. Only one band is present. Another type is a 24-bit true colour image with 3 colour planes in red, blue and green. While images can be used to store thumbnail pictures of satellite images, these are not useful for storing multiple bands of satellite imageries.

Colour information for images is stored in image palettes. Palettes again are datasets but with attribute CLASS = PALETTE. 8-bit image palettes are simple where a specific RGB combination is given a colour code and an index value. These combinations are reserved to only those appearing in the image. A 24-bit palette contains all combinations of RGB in 8-bit quantization (0-255).

Attributes in HDF5

Attributes are datasets with attribute CLASS = ATTRIBUTE. These contain information about the object they refer to. Certain objects such as images, palettes, attributes have default attributes like CLASS etc, which help in distinguishing them. Custom defined attributes give meta data about the object.

As HDF5 supports hassle free storage of different object types, it can bundle all of a project’s data. For instance, original satellite images can be stored in a group as 3D datasets. The processed data can be in another similar group and the release images can be in another. Tables can store specific information about the sensors and other image parameters. Attributes can give object specific information furthering the details. Images can store print ready maps.

HySI HDF5 data analysis and programming.

Program Architecture

IMS HySI (Hyperspectral Imager) datasets are released in HDF5 format by NRSC (National Remote Sensing Centre). Since HDF5 is yet to be supported by proprietary image processing packages, a software program was developed to handle these files. The program was built upon individual .C files containing object specific functions. They were linked using a custom header file and compiled with a common main file. The user interface was built in the main file, which calls the appropriate functions based on the user’s options. The architecture is displayed in the figure 1. A review of the functions in the header file would give their parameters and return types.

All files and functions are supplied with commented headers giving the essential details. All functions start with the prefix ‘fn’ and all identifiers start with their type as prefix. For example, H5 identifiers start with ‘h’ and integers start with ‘i’. Similarly arrays and pointers are prefixed with ‘a’ and ‘p’ respectively.

Figure 1: Program Architecture

All HDF5 functions are sourced in “hdf5.dll” and called through the library file “hdf5dll.lib” and the header “hdf5.h”. With these source files, custom files’ objects and the executable “HySIOperations.exe” the entire software program is independent and can be run anywhere. As the user runs the program, a menu is displayed as shown in Figure 2.

Figure 2: User Menu

As options are chosen, specific functions are called and at the end, the user is routed back to the interface. The following sections explain each of the functionalities.

Creating a complete HDF5 file

This option calls the file, group and dataset create functions in a row. As soon as the file is created, it is opened and a group is created. Then this group is opened and a dataset is created. Now the file is ready for use. These functions are only wrappers around the original H5 files H5Fcreate(), H5Gcreate() and HFDcreate() which create the objects. The wrapper functions handle any errors and exceptions should they occur and the respective error message is displayed on the screen. The names of the objects are “h5ops.h5” for file, “matrices” for group and “3Dset” for the dataset.

The dataset “3Dset” is of dimensions 241 x 256 x 64 to match the X and Y dimensions of the HySI data supplied. 64 is the number of bands. At the end, all the objects are closed to prevent run time errors.

Reading the contents of HySI file

This option opens the NRSC supplied .H5 file, reads the contents in the root group and lists them in a table on the user interface. The functions called are fnFileH5Open(), fnObjectList() and finally closes all the file opened by calling fnFileH5Close(). The function fnObjectList() internally calls H5Gget_num_objs() a H5 file to collect the number of objects. This information is used to list the objects by calling the function H5Gget_objname_by_idx.

Option 3 opens the group “science_data” inside the H5 file and performs the same operations. It calls the same set of functions which now list the contents of the group as displayed in Figure 3. It can be observed that the 64 bands are organized in individual 2D datasets that are 64 in number. The last two datasets contain ancillary information.

Figure 3: Contents of science_data listed

Creating a 3D dataset

Figure 3 points out the scope for better data management by organizing the 64 individual 2D datasets into a single 3D dataset. This can be performed since all bands are of same dimension which is 241 lines by 256 pixels. Thus option 4 in the program uses the earlier created 3D dataset “3Dset” of dimension 241 x 256 x 64 in the file “h5ops.h5” created in option 1 to store the collected data.

This option opens the source and destination files and datasets using the functions fnFileH5Open() and fnDsetOpen(). Next it calls the function fnHySIDataRead() which recursively opens each of the 64 datasets, reads the data and stores it in a 3D array of dimension 241 x 256 x 64. Each time a dataset is read, the 3^rd dimension increments by 1; thus each band is stored sequentially in the 3^rd dimension. Next it calls the function fnHySIDataWrite() which prints this matrix exactly into the 3D dataset opened in “h5ops.h5”. Figures 4 shows the process opening, reading and closing the bands and writing into the destination dataset. Figure 5 shows the contents of 3D dataset when opened in the HDFview software.

Figure 4 Writing into 3D dataset

Figure 5 3D Dataset viewed in HDFView

Exporting the HySI file as ASCII

The HDF5 files were hitherto unreadable in standard image processing packages. The best solution would be to export all the 64 bands as ASCII text files which can be read as “generic binary” in image processing softwares and later can be stored in their native formats.

Option 5 performs this by calling the same function fnHySIDataRead(). The 3D matrix now is exported as an ASCII file using standard file writing functions of C. The pixel values are written in BSQ (band sequential) interleaving meaning that at the end of the first band, the second band is written in the same text file. This file can be imported into any software by specifying the dimensions of the image as 241 x 256 x 64 and the interleaving as BSQ. Figure 6 and 7display FCC formed from HySI datasets imported into an image processing software.

Figure 6 FCC formed with HySI ASCII export

Figure 7 FCC formed with HySI ASCII export

3D cube formation

A popular concept in hyperspectral image analysis is to form a 3D cube using the multiple bands. A cube is formed by stacking the bands sequentially one behind the other. Each time a band is added, it is moved slightly by one pixel in both X and Y directions. Thus the second band would have moved 1 pixel in X and Y and the 3^rd would have moved 2 pixels with respect to the first band in the same direction. Finally, after stacking several bands, the image would appear like a cube in 3D. The cube’s top face would contain only the first row (scan) of all bands (as the offset in Y direction is 1). Similarly, the right face would contain only the last column (pixel) of all bands (as the offset in X direction is 1). As the offset’s value increases, the number of rows and columns would increase proportionally and so would the perspective of the image formed. But, any value greater than 1 would produce an unnatural cube.

These cubes are a visualization means. In one go, the scientist can view the pattern in which the brightness value changes across the bands. As only the edge pixels are visible, one can ascertain the variation in brightness only along the edges of the image. To view the cube on desired regions of the image, a corresponding subset of the region should be made and a cube should be built over it.

Building a cube is computationally complex. Although by idea the cube is in 3D, it has to be finally viewed or printed onto a 2D surface. Thus, the 3D cube should be in 2D to view it. Hence, a 2D dataset is created to house the 3D cube to be formed.

Computing the dimensions of the cube is important. The front face would contain one complete band of dimension 241 x 256 pixels. In addition, it would have 64 pixels in top and to the right as the rest of the bands are added with an offset value 1 in X and Y directions. Thus the final dimension would be [241 + 64], [256 + 64] = [305] x [320]. A 2D dataset of same dimension was created in the file “h5ops.h5”. Figure 8. Shows the cube’s dimension when placed in a 2D dataset and the coordinates of the vertices in rows and column values.

The next task is filling the cube with pixel data. The top left and bottom right of the image should blend with the background and hence should be filled with 0.

One of the approaches would be to fill the cube row by row. But that would be computationally complex to fill the right side of the cube.

Figure 9 3D cube formed using subset of HySI data

Now the cube is formed. This array is then written to the 2D dataset in the file “h5ops.h5”. Also an ASCII file of the cube is exported. It was observed that since, the bands 11 to 63 in all HySI files have values missing in the bounding 5 pixels in all 4 sides, the top and right faces of the cube did not come out well. Hence a subset of the imagery in all bands omitting the surrounding 5 pixels was created. Now the new dimensions of the HySI file was [231][246][64]. Corresponding changes were made to the dimensions of the 2D dataset which reduced by 10 in both X and Y directions. Values of the loops were also changed and the cube was built again.

The cube of the subset came out as expected. Figure 9 displays the cube seen with HDFview. The end of land mass can be seen with a sudden drop in brightness value on the top face. In the right face, a bright cloud in the bottom right leaves a trail.

Observations and Ideas for future works.

3D datasets

The idea behind 3D datasets is that it would reduce the size of the entire file. Storing the values of [241]x[256]x[64] integers would require the same space whether stored in individual 2D datasets or in a single 3D dataset. But in the former case, additional memory is required to store information about 63 extra datasets which is removed in the case of a single 3D dataset. Further, referencing a pixel value is simple in the case of a 3D dataset without having to open multiple bands as in the case of 2D datasets.

Observation

Unlike expected, the “h5ops.h5” file with a 3D dataset occupied twice the size of the original “hysi.h5” file with multiple 2D datasets. This was unexpected and warrants a deeper understanding at how file objects are organized in HDF5.

Idea

Similar to 3D datasets, we could have 4D and 5D (etc.) datasets. We could have the original satellite imagery in the first 3 dimensions and the processed output in the 4^th dimension. For instance, the original HySI information can be stored in [1][2][3] dimensions and if a high pass filter is run on all or a selected few bands, that can be stored as [1][2][4]. Any process that does not change the X and Y dimension of the image can be stored in additional dimensions. The advantage is that, one can select any band combination, one from original and two from processed and form an FCC and visualize.

Visualizing the 4^th dimension is simple. If the X and Y (1^st and 2^nd ) dimensions represent the cover page of a book and if the Z (3^rd) represents the pages in it, the 4^th dimension would represent another set of pages of the same size of the cover page. When required, the original pack of pages can be removed and the 4^th dimension pages can be fed in. In other words, the dimensions from 3 onwards can be swapped between each other for visualization.

3D cube

Observation

The perspective is wrong. In general, the near face should be larger than the far end of the cube. But in our cube, the far end appears to be larger than the near end. Further the lines in the Z dimension should converge as they tend toward the far end. But in our cube, they rather appear to diverge. Yet, it is worth while to note that, the lines do not diverge as they are separated by a standard 64 pixels. Their appearance of divergence is an optical illusion. And for scientific reasons, we should do away with perspective as skipping a few rows and columns would be detrimental.

Ideas

It would be of great impact, if we could draw a box on the front face and haul it out as a cuboid from the original cube. Further, if we could rotate the cuboid, in other words, be able to see the other faces of the cuboid, it would be instrumental in visualization. This can be achieved by computing the dimensions of the 2D dataset and the coordinates of the vertices on the fly instead of hard coding them into the loops.

While visualizing the cube, if the scientist needs to know which band does the current pixel belong to what would we do? One way would be to use the pixel’s coordinate and back calculate the band number. Another would be to declare the pixel datatype as float (instead of the earlier integer) and to feed the band number as the decimal value. Thus the 35^th band’s pixels would have values like 567.35, 677.35. The analyst should interpret the decimal value as the band number.

File content listing

Observation

When we listed the contents of the group “science_data” in a HySI.H5 file, the number of elements was 64 bands + 2 tables = 66. Similar was the result when opened with the viewer HDFview. But when opened using HDFexplorer, it listed several other scalar datatypes containing useful information.

This is because, the software program we developed and the HDFview software use HDF5.1.6 library version while the HDFexplorer uses a recent release HDF5.1.8. One of the limitation of 1.6 is that it cannot list the scalar datatypes created using 1.8 library versions.

Other Ideas

The HySI files are released with a table containing latitude and longitude of every 8^th pixel. Plotting these pixel coordinates in a graph came out as in Figure 10. The tilt is due to the inclination of the orbit of the satellite. Since HySI is a non rotating mirror scanner, the pixel to pixel topology should be preserved in the image. Hence we can self-georeference the image by projecting these coordinates on to a datum.

Figure 10 Pixel coordinates of a HySI image

Another is to use the flexible attribute storage of HDF files to log the details of various process run on the image. In future, the same HDF file would house the source and result datasets. The attribute of the result would contain the series of process run to achieve it. This would be instrumental in constructing the image lineage. This image chain approach to remote sensing is essential since even a single convolution filter would distort the original pixel values thereby reducing the purity of a pixel. Next when an analyst uses this processed dataset, he /she would be aware of the limitations of the dataset by looking at its lineage.

Conclusion

From the study, it is clear that HDF5 can be regarded as the file format for future remote sensing requirements. Certain limitations like inability to store vector information directly should be rectified in the future versions. The software program created is only an attempt to program on HDF files. The futuristic ideas mentioned in the last section would be implemented serially in the successive versions with a better user interface.

Next HySI HDF5 tool page to download and run the HySI Operations program.