Research‎ > ‎

W2RP - When-to-release Planner Decision Support Tool

Jason Ho - Shawn Shahnewaz - Guenther Ruhe

 Table of Content

1. Introduction - Research paper
Decisions about when-to-release a product or a release are inherently complex: The potential competitive advantage through faster delivery needs to be balanced against the degree of readiness of the product (overall quality) and the added value through new features. Pro-active analysis of various release scenarios based on large available and predictive data will enable product managers to make the most informed decision of when to release a product, with consistent and predictable outcomes.
In this short paper, a prototype tool called W2RP (When-to-release Planner) is created to demonstrate our methodologies of analytic-driven decision support. In existing release planning tools such as IBM Focal Point, Ontime (for Scrum-based development) or ReleasePlanner, when-to-release decisions are often made manually based on experience and "gut feeling" of the product managers. W2RP allows pro-active analysis of various release scenarios. It offers a set of most promising plans, together with the projected impact of releasing earlier (or later) in terms of reduced (added) functionality and/or quality. The product managers can then make the final release decision, and continue to monitor the release situation as the data changes over the course of product implementation.

When-to-release Planner Tool Paper: W2RP.pdf

As a proof-of-concept, we guide the users through the steps of usage by running a real-world project. The calculation of the values by the tool is included in this illustrative examples.
Calculation illustrative example: Illustrative Example of Calculation.docx

2. Tool Download

The tool can be downloaded using the following link: w2rp.zip
For assistant during installation, please refer to Section 3: Users' walkthrough.

Installation Screen


3. Users's walkthrough

The steps to utilize the tool are demonstrated in the below video. Please use the sample project data below for your reference in creating sample data for your own experiments.

Downloadable File: W2RP_walkthrough
Youtube Video: 

W2RP: walkthrough

Step-by-step walkthrough: W2RP_walkthrough.docx

Project: Social Product Manager (as used in research paper)
Input Sample FIle: Input-Sample_new.xlsx
Output Sample FIle for One chosen solution: Output-OneSolution-Sample.xlsx 
Output Sample FIle for All trade-off solutionsOutput-TradeOff-Sample.xlsx
Calculation illustrative example: Illustrative Example of Calculation.docx
Sample files for Manual calculation: SPM_Manual solutions.zip


4. Modeling - When-to-release Planner Process

In this section, more details from the paper modeling will be explained. The main reference frame should be the paper in section 1. Please read the paper prior to understanding this section better.

Definitions: 

A.  Software Release Planning
A product feature is a set of logically related requirements that provide a capability to the user and enable the satisfaction of business objectives [13]. Release planning as addressed in this paper is the problem of assigning the optimized set of features into a release and providing projection of the end product’s business values and quality assurance. 

B. Release Time

For this problem, a when-to-release date (RD) is a fixed date for release to be completed . This date is often predetermined from strategic planning and market research.

The tool W2RP varies the predetermined release date by a small duration of time DT. DT can be anywhere between ±1,2,3 days to ± 1,2,3 weeks. While these duration are still within acceptable range from the strategic release date, they may significantly alter the value or quality of the release (as defined and demonstrated in later sections).

C. Value of Features

A feature value is evaluated based on the extent of functionality to enable users and satisfy business objectives and operation [12]. Additionally, values can sometimes be evaluated by business values and estimated revenues the features may potentially generate.

The final aggregated total release value is a weighted average value of all features based on all criteria by all stakeholders.

The application of this definition and principle is demonstrated in Example of Value calculation.docx

D.  Quality of a Release

Quality of a software system is hard to evaluate. In this paper, we approximate quality through the result of the effort invested in testing (running all test cases), and the effort of developers to fix any defects found from testing [2]. For each feature f(n), based on its estimated size and complexity, a specified number of test cases will be designed and conducted. 

A target quality level for the released product is defined. For instance, 95% target quality level means that, for the released product, at most 5% of operation transactions are allowed to fail during operation. We calculate the probability of achieving this target quality based on the total number of test cases and an average defect detection rate applied. This principle is applied to both all the individual feature units and to the whole integrated system.

The final success probability (Total release quality) of F0 is defined as the combined probability of having no any individual feature in the release will fail, AND the system will not have an error at integration. This means, by varying the number of  test cases from the features, we can predictably lowering the probability of failure, and hence maximize release quality.

The application of this definition and principle is demonstrated in Example of Quality Calculation.docx


E.  Problem Statement

Based on the initial (baseline) when-to-release plan with fixed release date RD0 with features set F0, we study a series of release scenarios {(RDi, Fi)}i=1,2,…. Each of these scenarios is a variation of the original one, and is determined from (RD0, F0) by re-balancing the effort allocated to quality (testing) versus functionality implementable within the release duration RDi.

The when-to-release problem [4] is to determine as set of trade-off release plans generated from a series scenarios defined by the user. For a given set of scenarios, we define (RD*, F*), as a trade-off solution, if and only if there does not exist another release plan (RD’, F’) such that

(TRQ(F’), TRV(F’)) (TRQ(F*), TRV(F*)) and

(TRQ(F’), TRV(F’)) (TRQ(F*), TRV(F*)) 

Similarly, the application of this principle of selection for trade-off solutions is demonstrated in Example of Trade off solution genration process.docx


W2RP Process: 

Our approach, as similar to the approach in [4], automate the re-balancing of effort and selection of trade-off solutions, with the above definitions of quality, and release time. 

W2RP process starts with a baseline release plan (RD0, F0) that is imported by user via an external planning system or by using Greedy algorithm on the set of all features. Process 'Define scenarios' allows specification of relative changes to be made against the baseline plan. In 'Run scenario', from the trade-off triangle between release time, quality and value, one parameter becomes fixed, and trade-off solutions are generated from varying the two remaining ones. Three principal use cases are possible. Users can configure the feasible range of changes for all the three parameters, with DT is the maximum range of days duration can vary, DV(Fi) is the change in value due to functionality change, and DQ(Fi) is the change in quality due to the change in test cases. These factors will control how much of the timeline, business value and quality level are acceptable to be varied. 

From all the scenarios explored, a pool of release plans is generated. 'Analyze results' eliminates all plans that are dominated (as defined by Definition 7) by another one. All (remaining) qualified trade-off solutions will be maintained in a pool of candidate release planning alternatives. In 'Select and re-iterate', the product manager(s) can either go back to define another scenario or terminate the scenario playing process and select the most attractive plan, representing the best balance between the competing criteria. 

W2RP Workflow


Figure 2. W2RP Tool Workflow process

In each of the scenarios, one of the parameters from of {release time, release quality, release value} is fixed to a variation of the respective baseline values. The new value, e. g., shorter release duration, is taken from the specified interval of exploration. Subsequently, the user can now study the implications of varying among the remaining criteria either by re-allocating effort or by modifying the set of features to be offered.

Back to top


5. References

[1] B. Boehm, V. R. Basili, "Defect Reduction Top 10 List”, in Computer, vol. 34, no. 1, 2001, pp. 135-137.
[2] J. Campanella, "Principles of Quality Costs: Principles, Implementation and Use", ASQ Quality Press, 1999.
[3] B.H. Far, “Software Reliability Models”, in Dependability & Reliability of Software Systems (Lecture Notes), Chapter 6, 2012.
[4] J. Ho, G. Ruhe, "Releasing Sooner or Later: An Optimization Approach and Its Case Study Evaluation", RELENG, 2013.
[5] R. Lai, G. Mohit, P. K. Kapur, "A Study of When to Release a Software Product from the Perspective of Software Reliability Models" in Journal of Software, vol. 6, 2011, pp. 651-661.
[6] J. McElroy, G. Ruhe, “When-to-release decisions for features with time-dependent value functions”, in Requirements Engineering Journal, vol. 15, 2010, pp. 337-358.
[7] C. Morris A., J. Eliasberg, TH. Ho, "New product development: The performance and time-to-market tradeoff." in Management Science vol. 42.2, 1996, pp. 173-186.
[8] A. Ngo-The, G. Ruhe, "Optimized Resource Allocation for Software Release Planning", IEEE TSE, vol. 35, 2009, pp. 109-123.
[9] H. Ohtera, S. Yamada, "Optimum Software-Release Time Considering an Error-Detection Phenomenon During Operation," in IEEE Trans. on Reliability, vol. 39, 1990, pp. 596-599.
[10] L. Richard, M. Garg. "A Detailed Study of NHPP Software Reliability Models", in Journal of Software 7, vol. 6, 2012.
[11] G. Ruhe, “Product Release Planning: Methods, Tools and Applications”,  CRC Press, 2010.
[12] K. E. Wiegers, "Software requirements," Microsoft Press, 2009.
[13] G. Zorn-Pauli, B. Paech, T. Beck, H. Karey, G. Ruhe, “Analyzing An Industrial Strategic Release Planning Process – A Case Study at Roche Diagnostics,” Proc. REFSQ, 2013.

ą
TrongTan Ho,
Jul 23, 2013, 12:55 PM
ĉ
TrongTan Ho,
Jul 23, 2013, 12:55 PM
ĉ
TrongTan Ho,
Jul 23, 2013, 12:55 PM
ĉ
TrongTan Ho,
Jul 23, 2013, 12:55 PM
ą
TrongTan Ho,
Jul 23, 2013, 12:55 PM
ą
TrongTan Ho,
Jul 23, 2013, 12:55 PM
Ċ
TrongTan Ho,
Jul 23, 2013, 12:55 PM
Ċ
TrongTan Ho,
Jul 23, 2013, 12:55 PM
ĉ
TrongTan Ho,
Jul 23, 2013, 12:55 PM
Ĉ
TrongTan Ho,
Jul 23, 2013, 12:55 PM
ċ
SPM_Manual solutions.zip
(51k)
TrongTan Ho,
Jul 23, 2013, 12:55 PM
Ĉ
TrongTan Ho,
Jul 23, 2013, 12:55 PM
Ĉ
TrongTan Ho,
Jul 23, 2013, 12:55 PM
Comments