Choosing the right metrics
In diabetes services, the temptation is to use national level metrics such as the National Diabetes Audit (NDA) or indeed QOF metrics which are already captured at GP level. The Improvement and Assessment Framework for clinical commissioning groups (CCGs) also contains metrics for diabetes around submission to the NDA, care processes and structured education.
However some key questions emerge. Are these metrics enough when it comes to showing change that might have taken place as a result of integrating diabetes services? Is it possible to attribute any improvement to the service transformation? How long might this take? Would we be measuring any negative consequences of the service changes? Of key concern to the CCG, is it possible to predict how much the service will cost and what savings might accrue?
When we consulted with our local GPs, we found they were frustrated with the data that came back from national metrics such as the NDA. We heard that although the reports were helpful for benchmarking services across the country, there was an implementation gap between knowing how well they were doing and what they could do to improve. This is where small scale, rapid improvement at an individual patient or professional level can help bridge the gap.
Testing small-scale change
Quality improvement (QI) methodology enables clinicians to see the effect of small-scale changes, and potentially how these could be scaled-up to deliver the larger, population-wide changes that are typically measured in national audits. This ability to think small also allows any unintended consequences that might arise from implementing a new service to be identified. By gaining insight into what works, its effect on a small population and any action that may need to be taken to avoid unwanted side-effects of service transformation, CCGs are in a better position to predict the value of the improvement to the whole health economy in the future.
At an RCP seminar, Mastering the modern tools and methods of improvement - what do physicians need to know about quality improvement science? Alice Joy (RCP Patient and Carer Network member) presented a patients’ perspective on the risk of making big changes leading to little or no improvement in quality of care. Alice made a strong case for using QI methodology. As she noted, it helps us stop moving in circles and move forward. Understanding the problem is core to the QI approach. To quote Alice:
“You have to know what is wrong before you put it right. If you don’t know what is wrong, you can’t make it right. You can use all the measures you like in all the world and they may tell you what the numbers are, but they may not tell you why it is wrong.” Alice Joy, RCP Patient and Carer Network member
Establishing aims for integration
In Oxfordshire, we knew there were issues with duplication, delays and variation in the management of diabetes care. With the help of an improvement analyst, we started by establishing the aims (expected benefits) of the transformation, and considering how we could translate these aims into short-term processes that we could more easily monitor and work on improving. Our overall aim in integrating diabetes service is to improve the experience of providing and receiving care. Within the overall aim, we were aiming to:
· avoid duplication of activities and interventions,
· eliminate delays in communication and interventions
· minimise the variation in management of care.
We then translated these into more detailed aim statements. At this stage, there was a temptation to attach metrics and measures to the aim statements and set up an electronic system of collecting them. However, we were advised to pause, re-think our approach, and avoid jumping to conclusions before examining the processes.
Patients and processes
This process enabled us to gain a better understanding of what and how to measure. Instead of focusing on quality indicators alone, we will focus on short-term processes too. While performing the change, we will be guided by two statements: what are the benefits for patients, and what matters in the process.
We identified a local improvement analyst with the skills to undertake field work and support the clinical team and commissioners in understanding where things go wrong, agreeing solutions and measuring change. Crucially, the analyst works alongside clinicians to observe, record, analyse and report on change, enabling the team to make informed decisions as to whether the change is effective. Lastly, we implemented a process to inform everyone about their tasks and the timeframe, starting with data sharing agreements that needed to be in place before the launch of the new service.
Through this process we came to understand that, during transformation, engaging with people to identify frustrations and understand why expected outcomes are not being achieved is an important part of the process. Effective engagement is supported by measuring for improvement and reporting quickly and responsively on a range of outcomes, which should include process and balancing measures. Process measures show that change is happening, and balancing measures identify and monitor any adverse consequences.
For example, if the aim is to reduce hyperglycaemia in people with diabetes by providing them with advice on a healthy diet and exercise, the following measures may be considered.
· Outcome measure: hyperglycaemia (raised HbA1c) in people who were given lifestyle advice
· Process measure: rate of compliance with the protocol regarding thecontent and delivery of information
· Balancing measure: number of hypoglycaemic episodes.
Visibility of the data and the impact of small-scale change helps teams engage with the transformation process in a way that is tangible and easily understood. This engagement will prove vital as larger-scale change gets underway.