An Empirical Analysis of Risk Components and Performance on Software Projects

Abstract:

Risk management and performance enhancement have always been the focus of software project management studies. The present paper shows the findings from an empirical study based on 115 software projects on analyzing the probability of occurrence and impact of the six dimensions comprising 27 software risks on project performance. The MANOVA analysis revealed that the probability of occurrence and composite impact have significant differences on six risk dimensions. Moreover, it indicated that no association between the probability of occurrence and composite impact among the six risk dimensions exists and hence, it is a crucial consideration for project managers when deciding the suitable risk management strategy.

A pattern analysis of risks across high, medium, and low-performance software projects also showed that:

1. The ‘‘requirement’’ risk dimension is the primary area among the six risk dimensions regardless of whether the project performance belongs to high, medium, or low;
2. For medium-performance software projects, project managers, aside from giving importance to ‘‘requirement risk’’, must also continually monitor and control the ‘‘planning and control’’ and the ‘‘project complexity’’ risks so that the project performance can be improved; and,
3. Improper management of the ‘‘team’’, ‘‘requirement’’, and ‘‘planning and control’’ risks are the primary factors contributing to a low-performance project.

Introduction:

Software development is a highly complex and unpredictable activity. The Standish Group CHAOS Report in 2004 indicated that 53% of software projects were unable to deliver on schedule, within budget, and with the required functions, while 18% of the software projects were cancelled (Standish Group International, 2004). This stresses the fact that software projects pose various risks and daunting tasks for many organizations (Charette, 2005; Chris and Christine, 2003; Hoffman, 1999; McConnell, 1996). Now, as organizations invest substantial resources and effort on software development, controlling the risks associated with software projects becomes crucial (Kumar, 2002). Hence, understanding the nature of the various software risks and their relationship with project performance has become increasingly important since the risk management strategy and plan depends on it.

Developing an efficient and suitable risk management strategy depends on understanding two basic components— probability of occurrence and impact on project performance (Boehm, 1991). The probability of occurrence of each software risk is different and its degree of impact on project cost, schedule and quality is also different. Hence, these two software risk components must be taken into consideration to develop a good software risk management strategy and plan. If not, the real benefits of the risk management activity may be lower than the resources invested (Boehm, 1991; Longstaff et al., 2000; Kumar, 2002; DeMarco and Lister, 2003).

Although several articles have already identified various software risks, we currently lack an understanding of the relative probability of occurrence and the various impacts of different software risks across projects in general. If project managers do not realize the natures of the different kinds of software risks and their relationship to project performance, they cannot develop an effective and appropriate strategy to mitigate or control them.

Many studies have proven that proper management of software risks affects the success of software development projects (Jiang and Klein, 2000; Wallace and Keil, 2004). However, previous studies on software risk management failed to analyze the gap or pattern between software risks and project performance. For example, we can identify the software risks that negatively affect the project performance and thus, should be well-controlled in order to improve the project performance. Such insights regarding the patterns between software risks and project performance can assist project managers in developing a software risk management strategy that can mitigate the risks associated with software development projects.

The present paper explores the relationships between software risks and their impact on project performance. The term ‘‘project performance’’ is defined as ‘‘the degree to which the software project achieves success in the perspective of process and product’’ (Nidumolu, 1996). In the present study, we began to address this issue by analyzing the probability of occurrence and impact on project performance of 27 software risks. These risks are classified into six dimensions namely, user, requirement, project complexity, planning and control, team, and organizational environment. A dataset from 115 historical software projects were analyzed using the MANOVA and Two-Step Cluster Analysis methods. These analyses addressed the following questions: (1) What are the different patterns for both the probability of occurrence and the impact in six software risk dimensions? (2) What relationships exist among the software risks in the three project performance clusters (high, medium, and low)? (3) What are the top 10 software risks and what are their associated probability of occurrence and impact?

Software Risks:

• User:
  • Users resistant to change
  • Conflict between users
  • Users with negative attitudes toward the project
  • Users not committed to the project
  • Lack of cooperation from users
• Requirement:
  • Continually changing system requirements
  • System requirements not adequately identified
  • Unclear system requirements
  • Incorrect system requirements
• Project Complexity:
  • Project involved the use of new technology
  • High level of technical complexity
  • Immature technology
  • Project involves use of technology that has not been used in prior projects
• Planning & Control:
  • Lack of an effective project management methodology
  • Project progress not monitored closely enough
  • Inadequate estimation of required resources
  • Poor project planning
  • Project milestones not clearly defined
  • Inexperienced project manager
  • Ineffective communication
• Team:
  • Inexperienced team members
  • Inadequately trained development team members
  • Team members lack specialized skills required by the project
• Organizational Environment:
  • Change in organizational management during the project
  • Corporate politics with negative effect on project
  • Unstable organizational environment
  • Organization undergoing restructuring during the project

The Top 10 Software Risks:

1. Continually changing system requirements
2. System requirements not adequately identified
3. Unclear system requirements
4. Lack of an effective project management methodology
5. Incorrect system requirements
6. Poor project planning
7. Inadequate estimation of required resources
8. Project involved the use of new technology
9. Project progress not monitored closely enough
10. Corporate politics with negative effect on project

Conclusion:

Achieving effective software risk management requires project managers to understand the nature of software risks. Thus, information about the probability of occurrence and impact of software risks on project performance can help the project managers to develop a better risk management strategy.

This empirical study considered risk information on 115 software projects. The results indicate that a positive correlation does not exist between the probability of occurrence and impact among the six risk dimensions. The relationship between software risks and project performance was also examined in the high, medium, and low-performance projects. The results show that the ‘‘requirement’’ risk dimension is the principal factor affecting the project performance. Aside from this, one of the ways to improve project performance is by properly planning the development activities and reducing the project complexity. Likewise, if the project manager is unable to effectively manage the requirements of the whole project life cycle, and does not well-plan nor monitor the software risk management plan, the software projects is likely to perform poorly.

The performance of the software projects could also benefit from exploring the relationship between risk components and some important attributes of software projects such as the type of software systems and project duration. These are the possible research topics for the future.