Considerations in Mapping DECTRIS Eiger data

A discussion paper by Herbert J. Bernstein, (yayahjb at gmail dot com) 9 December 2014

The purpose of this page is to present issues relating to mapping from currently available DECTRIS Eiger detector data files to NeXus and to explore what changes, if any, may be desirable in the format of the files or in the NeXus application definition for such files. This is a preliminary document. Comments and corrections greatly appreciated.

The DECTRIS Eiger detector provides data in HDF5 format and is intended to be NeXus compliant. As of September 2014, a sample Eiger 1M HDF5 dataset, x1_insu_0p00125d_0p00125s_0p01beam, of 72,000 images became available from DECTRIS. The dataset consists of a master hdf5 file and 72 data hdf5 subfiles of 1000 images each. See

x1_insu_0p00125d_0p00125s_0p01beam_master

for a listing of the master file and

x1_insu_0p00125d_0p00125s_0p01beam_data_000001

for an example of one of the 72 1000-image data files. The current NIAC-recommended NeXus presentation of such data is given in the NXmx application definition. The current DECTRIS specification of their intended fle format is available on their web page at https://www.dectris.com/nexus.html#main_head_navigation that is accessible after sign-in to the DECTRIS web site.

The data set is readable with HDF5 code. The issues discussed below have to do with conformance with NeXus conventions and the software base that is developing for NeXus/HDF5 support.

Issue 1. Use of the @NX_class attribute

NeXus uses the @NX_class attribute to associate a NeXus class with an HDF5 group. While there has been some discussion of extending this concept to HDF5 datasets to clarify their use as NeXus fields, this is not, at present, used in NeXus. The sample Eiger data files uses @NX_class for only one of the HDF5 groups, "entry" to identify it as an NXentry. The sample Eiger data also uses @NX_class as an attribute for each of the HDF5 datasets that contains a block of images, identifying each such dataset as an NXdata class. In NeXus, the @NX_class="NXdata" attribute should only be applied to an HDF5 group, not to an HDF5 dataset. Normally, any NeXus file having data should have either the data or links to the data in an NXdata group.

Suggested Resolution: We suggest that

1. Every HDF5 group in the file that is intended to correspond to a NeXus class have an @NX_class attribute, with a value equal to the name of that class, added; and
2. Every HDF5 group in the file that is intended to be excluded from validation as a NeXus class be given the attribute @NX_class="NXcollection"; and
3. That the @NX_class="NXdata" attribute be removed from each of the datasets to which it has been applied; and
4. At least one NXdata group should be included in the file, but the use of just one such NXdata group with multiple blocks of data contained within would be an extension of the NeXus specification as would the use of multiple NXdata groups each with one block of data. We recommend that the NeXus definition be extended in both ways, but the the former extension be used in this particular case, because the data block axis definitions other then the frame number are uniform among the blocks, allowing for the use of a single set of canSAS array @axes attributes for the group. For other detectors, such as the Pilatus 12M-DLS, we recommend that the other extension be adopted to permit full use of the array @axes attributes that might conflict among different NXdata groups. In the case of the Eiger, it should be sufficient to introduce a data:NXdata group in the master file with 72 links named data_000... similar to the current links, but one level down, under /entry:NXentry/data:NXdata. If existing processing programs need the links in their current position, they can be left in place but deprecated for new work. In the subfiles, while it would be desirable to remove the non-compliant @NX_class="NXdata" attrobutes from the datasets, if they are needed for existing processing programs, they can be left in place but deprecated. If the subfiles are, themselves to be NeXus files, it would be helpful to introduce NXdata groups in them as well. It would also help in understanding the files, if the names used in the subfiles were consistent with the data_000... names in the master file.

Issue 2. Specification of Geometry

NeXus provides two major approaches to the specification of the physical layout of components of a system: the NXgeometry class in terms of an orientation and absolute translation. See

http://download.nexusformat.org/doc/old/html/Design-CoordinateSystem.html

for a description of the coordinate system and an explanation of its use in the NXgeometry class. The NXgeometry class is more fully described in

http://download.nexusformat.org/doc/html/classes/base_classes/NXgeometry.html#nxgeometry

The DECTRIS discussion of the Eiger geometry in the DectrisGeometryDocumentation.pdf document on the https://www.dectris.com/nexus.html page proposes to use, instead, the newer approach taken from CBF axis handling which explicitly should the nesting and relationship of axes defining the layout of components, and which would allow complete specification, module-by-module of the detector layout. The sample data uses NXgeometry for the overall detector, and does not provide the module-by-module information.

Suggested Resolution: We suggest that

1. The use of the older NXgeometry be replace by the newer CBF-style axis handling as described in DectrisGeometryDocumentation.pdf
2. That the module-by-module information be provided when available

Issue 3. Pixel Mask Specification

There seems to be some disagreement among choices of pixel mask specifications. The NXmx application definition says

• bit 0: gap (pixel with no sensor)
• bit 1: dead
• bit 2: under responding
• bit 3: over responding
• bit 4: noisy
• bit 5: -undefined-
• bit 6: pixel is part of a cluster of problematic pixels (bit set in addition to others)
• bit 7: -undefined-
• bit 8: user defined mask (e.g. around beamstop)
• bits 9-30: -undefined-
• bit 31: virtual pixel (corner pixel with interpolated value)

The DECTRIS web site definition agrees on bits 0 through 4 and leaves bits 5 - 31 as undefined.

Suggested Resolution: Adopt the NXmx specification in the web site documentation.

Issue 4. The @unit versus the @units attribute

The sample data uses an "@unit" attribute. The normally used NeXus attribute is "@units" using SI units strings for values.

Suggested Resolution: Adopt the NeXus "@units" attribute

Issue 5. The @data_type attribute

The sample data introduces an @data_type without a clear specification of the meaning ans intended use of this attribute, also introducing values such as "float"and "string". Usually the types of values are specified in class definitions and application definitions and as metadata in HDF5 files. It is not clear what is intended here and why this additional and undocumented type specification is needed.

Suggested Resolution: Either remove this attribute or explain what it is an how it should be used.

Issue 6. Undocumented fields

There are several undocumented field names in the example data, e.g. rxbias_0, rxbias_1, ... It will not be possible to validate files with undocumented tags that are not shielded within an NXcollection group or otherwise clearly documented as not to be validated.

Suggested Resolution: Document tags that exposed or make it clear which tags are not to be validated.

More specific issues will be explored, but it would be helpful if most of the issues above were addressed first, so that more automated tools can be used.