Quality of Semantic Standards

Quality of semantic standards was the focus of PhD Research, in which an instrument to measure the quality of a semantic standard has been developed. This research was sponsored by TNO, University of Twente, Novay and Netherlands Open in Connection. It started in 2009, and finished in 2012. The abstract from the book:

Little scientific literature addresses the issue of quality of semantic standards, albeit a problem with high economic and social impact. Our problem survey, including 34 semantic Standard Setting Organizations (SSOs), gives evidence that quality of standards can be improved, but for improvement a quality measurement instrument is needed. 81% of the survey respondents is interested in using such instrument. It can be expected that improved quality of semantic standards will lead to improved interoperability and improved economic welfare and social life.

Our main research question is: What are the characteristics of an instrument to measure quality of semantic standards that will aid standard developers in improving their standards?

Based on design science methodology this research developed such an instrument, called iQMSS (instrument for Quality Model of Semantic Standards). The iQMSS involves three types of artifacts: artifacts related to the Quality Model of Semantic Standards (QMSS), the Semantic Standard Model (SSM), and the instrumentalization. On the highest abstraction level SMO (Software Measurement Ontology) from the software engineering domain was selected and used for the creation of both QMSS and SSM with consistent concepts and definitions.

The extensive research process included coverage of both scientific studies and practical experiences, and led to the highly customizable final version of the iQMSS, in line with the gathered requirements.

The main artifact QMSS consists of in total 100 quality aspects structured within three hierarchical trees; product quality (intrinsic), process quality (the organization of the standard), and quality in practice (application of the standard).

The SSM is applied upfront to gather knowledge about the standard, create overview and understanding about what we are able to measure in practice for a specific standard. The main concepts within the model are the standards’ context, its content, its development and maintenance processes, and its application. On the lowest level it contains 33 aspects of a semantic standard.

The glue between the artifacts is the usage model, which describes how the iQMSS should be applied including four roles. The end result after each application is the analysis report which provides the basis for a standards’ improvement project.

But does the iQMSS aid standard developers in improving their standards? Our validation cases, in the temporary staffing and education domains, suggest that the iQMSS identifies improvement suggestions. Based on the validation workshops we held in both the United States and the Netherlands, we believe that the iQMSS will be used in practice as aid for standards developers.