Actionable Metrics

Each change in the Transformation Backlog must have acceptance criteria for gauging its successful application. The Rollout Team will verify whether or not these criteria are being met, and must do so at least once every Transformation Heartbeat.

Regardless of whether or not the expected Acceptance Criteria have been satisfied, however, we must understand the usefulness of implementing it. Are the expected benefits to the organization being achieved? The Transformation Rollout Team must therefore look for actionable metrics which can inform them if the Acceptance Criteria were indeed appropriate, or if other work must be done if the desired benefits are to be achieved.

Suppose for example that a change involves the Limited WIP pattern. This pattern details the following consequences that can be expected from its implementation:

value is delivered earlier
there will be less stock-on-hand that will be subject to depreciation
throughput and forecasting will improve
for projects, the risk of failing to meet a Sprint Goal will be reduced as less work is likely to remain in progress by the end of the iteration

Any or all of these consequences might be used to define “actionable metrics” for gauging the successful application of a corresponding change in which WIP is limited. For example, if the Acceptance Criteria stipulated that WIP must be limited to 3 items, and this was indeed verified, the change may then be retired. However, it does not necessarily follow that any of the above benefits were in fact realized. Measurements will need to be taken in order to find out. We need empirical evidence as to whether or not value was delivered earlier, or depreciation was reduced, or throughput rose, or progress towards a goal was better controlled. These metrics will inform us of how successful the change was and to take further action accordingly.

Actionable metrics should be evaluated for every change that is currently in progress. This must occur at least once every Transformation Heartbeat. These metrics can then be used to revise the plan for applying the change, and/or to make revisions to the Transformation Backlog.

Once its Acceptance Criteria are met, the change ought to be retired, irrespective of whether or not the hypothesized benefits are attained. The next change item will then be drawn from the backlog by the Transformation Rollout Team and actioned.

Note that if the expected benefits have not been realized for a specific change after retirement, then a new item is typically needed. It may appeal to different patterns or implement them in a different way with a different action plan, and the Acceptance Criteria may need be revised. The change will then be added to the Transformation Backlog and prioritized accordingly. This feedback loop, in which value and quality are subject to continuous improvement, is referred to as “Validated Learning”.

Synonyms: Transformational Metrics