Evaluations
A Whole Systems Approach is a different way of working and therefore requires a different approach to evaluation. Two different evaluations of our implementation of a WSA in Dundee took place between 2021-2023:
A national process evaluation of Early Adopter Sites, commissioned by Public Health Scotand.
A local developmental evaluation lead by a team of researcher from Public Health Intervention Responsive Study team (PHIRST).
1. Process Evaluation
Dundee (alongside 6 other areas) contributed to the field work of this evaluation between May 2021 - April 2022 which was undertaken by Social Marketing Gateway.
A final process evaluation report was published in December 2022, with 9 recommendations being considered by national partners in order to inform the Scottish Government about future roll out of the Whole Systems Approach across other areas in Scotland.
Local case studies were also published (including Dundee) with key recommendations that can be considered at a local level . A summary of what came through from the process evaluation that is meaningful locally:
1. The importance of having visible, very senior backing at the outset.
2. Establish a broad base of senior involvement that will be actively involved in the working group.
3. Guidance on how to engage the community, and how to adapt the methodology to a non-academic audience, would be helpful.
4. sufficient time should be devoted to workshop activities and causal mapping activities.
5. A blended approach to the process could be considered (COVID related, all online).
6. A more concentrated process, with workshop activities being closer to one another, could make it easier for stakeholders to stay fully engaged (links with 2).
7. mechanisms for more stakeholder voices in the causal mapping and action development.
We will use the above findings, in conjunction with 2. Developmental evaluation findings (due to be finalised by May / June 2023) to inform how we take forward a WSA across Tayside.
2. Developmental Evaluation
We began working alongside a team of academic researchers from a range of universities that make up the Public Health Intervention Responsive Study Team (PHIRST) in May 2021.
The aim of the evaluation was to 'Learn from the experience in Dundee & apply that learning to the roll out in other local authorities' through 3 research questions:
1. Does the approach taken in Dundee support key stakeholders to recognise what they can do in relation to actions at different levels within the system? (stakeholder interviews and survey)
2. How do/ can co-produced activities (e.g. with children, vulnerable communities) shape this approach?
3.What adaptations to the approach taken in Dundee would be required over time to connect this approach to other 'systems’ (e.g. Mental Health) and implement this approach in other Local Authorities in Tayside?
The questions were addressed through a range of research activity over 4 key work packages,including: 1. stakeholder interviews 2. Peer researcher led interviews 3. an online survey 4. action learning sets
Below is a timeline of key activities that were undertaken as part of the developmental evaluation .
Draft Recommendations from 2. Developmental Evaluation
Source funding for the further development and implementation of the approach to be able to demonstrate change.
Create more awareness of structures and strategies among stakeholders to support child healthy weight in Dundee.
Increase stakeholders’ confidence in feeling able to influence individual people and improve places.
More awareness raising and developing training to help members see how they fit in the approach.
Develop clear and positive communication with communities about activities on offer that are framed in an accessible and encouraging ways to involve more community members and parents.
Facilitate ongoing targeted communication and diverse involvement opportunities for different stakeholder groups.
Map and link WSA to CHW activities across Tayside to align priorities and identify additional resources and new funding opportunities.
* The final evaluation report is in the final write-up stage, due to be published by Summer 2023 *