Computer systems are used in many critical applications where a failure can have serious consequences (loss of lives or property). Developing systematic ways to relate the software quality attributes of a system to the system’s architecture provides a sound basis for making objective decisions about design trade-offs and enables engineers to make reasonably accurate predictions about a system’s attributes that are free from bias and hidden assumptions. The ultimate goal is the ability to quantitatively evaluate and trade off multiple software quality attributes to arrive at a better overall system. The purpose of this report is to take a small step in the direction of developing a unifying approach for reasoning about multiple software quality attributes. In this report, we define software quality, introduce a generic taxonomy of attributes, discuss the connections between the attributes, and discuss future work leading to an attribute-based methodology for evaluating software architectures.

Computer systems are used in many critical applications where a failure can have serious con- sequences (loss of lives or property). Critical applications have the following characteristics:

• The applications have long life cycles (decades rather than years) and require evolutionary upgrades.
• The applications require continuous or nearly non-stop operation.
• The applications require interaction with hardware devices.
• The applications assign paramount importance to quality attributes such as timeliness, reliability, safety, interoperability, etc.

Developing systematic ways to relate the software quality attributes of a system to the system’s architecture provides a sound basis for making objective decisions about design trade- offs and enables engineers to make reasonably accurate predictions about a system’s at- tributes that are free from bias and hidden assumptions. The ultimate goal is the ability to quantitatively evaluate and trade off multiple software quality attributes to arrive at a better overall system.

Developers of critical systems are responsible for identifying the requirements of the application, developing software that implements the requirements, and for allocating appropriate resources (processors and communication networks). It is not enough to merely satisfy functional requirements. Critical systems in general must satisfy security, safety, dependability, performance, and other, similar requirements as well. Software quality is the degree to which software possesses a desired combination of attributes (e.g., reliability, interoperability) [IEEE 1061].

There are different schools/opinions/traditions concerning the properties of critical systems and the best methods to develop them:

• Performance — from the tradition of hard real-time systems and capacity planning
• Dependability — from the tradition of ultra-reliable, fault-tolerant systems
• Security — from the traditions of the government, banking and academic communities
• Safety — from the tradition of hazard analysis and system safety engineering

Systems often fail to meet user needs (i.e., lack quality) when designers narrowly focus on meeting some requirements without considering the impact on other requirements or by taking them into account too late in the development process.

Designers need to analyze trade-offs between multiple conflicting attributes to satisfy user requirements. The ultimate goal is the ability to quantitatively evaluate and trade off multiple quality attributes to arrive at a better overall system. We should not look for a single, universal metric, but rather for quantification of individual attributes and for trade-off between these different metrics, starting with a description of the software architecture.

• Performance:

The performance quality is concerned with response times and similar measures for various events.

• Security:

• Non-repudiation
  
• Confidentiality
  
• Integrity
  
• Assurance or authenticity
  
• Availability (no denial of service)
  
• Auditing
  

• Testability:

The testability attribute is concerned with detecting failure modes. Typically, 40% of the cost of a large project is spent on testing. This means architectural support for testing that reduces test cost is time well spent. We need to control the internal state of and inputs to each unit, then observe the corresponding output of that unit.

• Usability:

• How easy it is to learn the features of the system
  
• How efficiently the user can use the system
  
• How well the system handles user errors
  
• How well the system adapts to user needs
  
• To what degree the system gives the user confidence in the correctness of its actions.
  

• Business Quality Attributes:

• Time to Market: architectural reuse affects development time.
• Cost and Benefit: in-house architectural expertise is cheaper than outside expertise.
• Projected Lifetime of the System: long-lived systems require architectures that are modifiable and scalable.
• Targeted Market: architecture affects what platforms will be compatible and incompatible with the system.
• Roll-out Schedule: if functionality is planned to increase over time, the architecture needs to be customisable and flexible.
• Integration with Legacy Systems: the architecture of the legacy system being integrated will influence the overall system’s architecture.

• Architectural Quality Attributes:

• Conceptual Integrity is the underlying vision or theme unifying the components and their interactions. The architecture should do similar things in similar ways.
• Correctness and Completeness is concerned with checking the architecture for errors and omissions.
• Buildability is concerned with the organization’s capabilities to actually construct the architecture in question.