ACP- Service Evaluation Project (SEP)

Length: 3,000- 4,000 words (excluding references and appendices only)

Introduction to ACP-SEP

Clinical psychologists are in a unique position in the NHS in that they are both clinically qualified and trained in research skills. They therefore have the opportunity (and are increasingly expected) to play a lead role in the development and evaluation of services. Research and evaluation skills will also play an important role in consultancy and working in teams. We think that the best way to equip trainees with the knowledge and confidence that they will need to fulfil this role is to give you the experience of doing it. The rationale for the ACP-SEP, then, is that it represents an integration of clinical skills, research skills and clinical practice skills such as leadership and communication. It is a chance for trainees to use research expertise in an applied setting and to see how this can be of practical benefit in addressing meaningful questions about clinical practice or service delivery within the constraints of a real service. This kind of work is essential if we want services to meet the needs of the population, and to be as effective as possible. 

The aim of the ACP-SEP is to report on a small-scale project that evaluates some aspect of a service or programme. This will usually be an NHS clinical or health psychology service, but it may inclde the voluntary sector, training courses and other bodies such as HCPC or the BPS. The project must have the potential to be useful to the commissioning organisation and to be potentially publishable in a peer-reviewed journal or reviewed periodical. It should relate to some aspect of a current service or planned service development and should always be driven by the commissioners of the project. It should fully consider in the context of the commissioning service/provider and will be limited by the time and resources that are available for it. The ACP-SEP is therefore undertaken with an acceptance of the limitations that will be placed on the design by the demands of the situation and the acknowledgement that this is real life clinical practice (and not a tightly controlled laboratory paradigm). The aims of the assignment are to encourage trainees to see research skills as having a core role in routine evaluation of clinical practice or service provision. Service evaluation is vital because it is typically about decision-making: it aims to answer important questions about resource allocation and service delivery. The project also places an emphasis on the consumption of research findings and the written report produced is directed at a specified target audience (usually whoever commissioned the project and/or others who can use the information contained within it to make decisions).

The range of potential evaluation topics is very broad, and a comprehensive list would be impossible. There is a range of evaluation frameworks that offer guidance on developing suitable evaluation questions; a good starting point is to think about the different elements of a particular service that might be amenable to evaluation:

1. Service structure (e.g. appointment systems, referral pathways, case notes)

2. Process (e.g. treatment procedures, communication)

3. Outcome (e.g. clinical outcomes, satisfaction, quality of life, cost efficiency)

4. Organisational processes or structures (e.g. team working, recruitment)

In some instances, commissioning services/charities may request specific evaluations. However, a key part of the task is assisting the commissioners in identifying realistic evaluation aims.

Identifying a Service Evaluation Project

The service evaluation project will constitute part of a trainee's ongoing clinical work. It can relate specifically to work carried out within the placement, or it may centre upon another service or charity although this needs to be negotiated with placement supervisors and the programme (see below).

Trainees should discuss the opportunities for conducting a service evaluation early in the relevant placement to give sufficient time to plan and implement an appropriate project. Some projects will also be advertised to trainees directly. Commissioners are asked to complete a commissioning form, to be sent to the Research Support Office, that is checked and approved by a member of the academic team. Trainees would normally be expected to assist in the completion of this form. If a trainee is planning to conduct a service evaluation placement that is not based on their current placement, it is essential that they have discussed and agreed this with their placement supervisor and clinical tutor, as they will need to negotiate time away from placement to do this work. 

Time Allowed for Service Evaluation Projects

We estimate that trainees can usually expect to allocate up to one half day each week over a limited period during one placement to complete their service evaluation project. Any time taken out of a clinical placement must be by prior arrangement with the placement supervisor. It should be noted that this arrangement is purely for work on planning and collecting data for the service evaluation project: it is NOT an entitlement to additional study time and the usual study time should be used to write up the work.

Programme Requirements for the ACP-SEP

The service evaluation project must involve the collection and/or analysis of data and the written report must include a set of recommendations for changes or improvements that have been clearly derived from the data. In some cases, trainees have analysed and reported on pre-existing data sets for their service evaluation projects. In these situation, the trainee should design the analysis protocol and must demonstrate sufficient intellectual input to the report to justify submission of the work to the Programme for assessment. On occasion, follow-up data collection is necessary and appropriate. If a large project has been identified, a trainee may consider working collaboratively with another trainee. Trainees must remember that this is an examined piece of work, and so they must each submit a clearly distinct and separate piece of written work. The best way to do this is usually for each trainee to take responsibility for a different aspect of the project from the outset and to focus on this in their submitted ACP-SEP. The extend of the collaboration and limits of the trainee's own contribution to the project must be clearly specified in a short 'statement of contribution' section that follows the title page.

Your ACP-SEP must satisfy the following basis criteria:

1. Topic: must be service-orientated (i.e. it must have implications for the effectiveness and efficiency of service delivery). The service must be relevant to clinical psychology practice in some way and the project must be either a service evaluation or an audit (i.e. not research)

2. Methodology: no methodology is specifically endorsed, but the project will be assessed on how well the chosen methodology attempts to answer the question(s) posed and how well the trainee communicates the implications for service delivery and/or development. The methodology must complement that chosen for the research thesis, so that if the trainee is undertaking a quantitative research thesis, they must normally employ qualitative methods for their service evaluation (and vice versa). It is the trainee's responsibility to ensure that this breadth is maintained, and where a piece of work does not fulfil the criterion, a new piece of work may be required. Trainees must also ensure that their data and analytic methods are appropriate to the question they are posing. As this is a small-scale project, it is likely that more descriptive methods of data collection and analysis will be most suitable (e.g. descriptive forms of qualitative analysis such as thematic analysis and content analysis are likely to be more appropriate than other methods such as grounded theory).

3. Length: The format of ACP-SEP is described below. It should contain an executive summary of no more than two pages (one page is preferable). A standalone report should be prepared to a specified journal format, including an abstract, and with a word count within the range of 3,000- 4,000 words (excluding references and appendices). A set of brief appendices containing copies of materials used and of other dissemination outputs (see below).

4. Trainees should take the major responsibility for the design and the methods employed for the project, but they should liaise closely with the commissioner with regards to the implementation. Additional advice may be obtained from your academic/personal tutor. The trainee is also responsible for ensuring that the conduct of the evaluation is ethically sound, and to ensure that any relevant ethical and/or clinical governance permissions are granted prior to implementation of the project. If you have any doubt please contact the Chair of the Department of Psychology Ethics Sub-Committee and/or the person responsible for clinical governance within the commissioning NHS site.

Financial Support for Service Evaluation Projects

Costs incurred for service evaluation projects must be borne by the service and cannot be claimed against University research funds. You are encouraged to discuss with your supervisor/the commissioner how the service or Trust might benefit from your work, and whether sources of financial support might be available to you for this work.

Dissemination of the Service Evaluation Project

On completion of the project, you must ensure that the commissioner is presented with a copy of all sections of the evaluation and that they provide written approval for the results to be disseminated. The ACP-SEP can be disseminated to the service prior to submission/marking. Written approval from the commissioners to disseminate and/or dissemination to the service/commissioners cannot be considered as ensuring that the submitted piece of work will achieve a pass grade. Trainees should then take the appropriate steps to ensure that the report is fed into the relevant organisational structures or disseminated as suggested by the commissioners. Copies of posters or presentations conveying feedback from the evaluation should where possible be contained within an appendix. Where feedback has not yet taken place the plans for this should be contained within an appendix (this is likely to include a draft presentation). The programme requires trainees and commissioners to consider submitting completed versions of ACP-SEP for publication in professional periodicals or peer review journals and consequently the main report should be prepared in the format of a specified publication.

Format of the ACP-SEP

Handbook guidance however supersedes any specific journal presentational requirements.

Title Page This must contain a statement of contribution (i.e. specify your role), and indicate which journal the report has been prepared for. In doing this please indicate the allowable article word count for the chosen journal as specified in the guidance for authors and include a web link to that guidance. Also include the word count for the piece of work.

Section 1 A prominent executive summary (a one to two-page outline of the work conducted and must identify the actual recommendations for action) written for a specific and defined audience (e.g. a sector or service manager, a purchasing group, a voluntary organisation.

Section 2 A report for a specified periodical/journal (e.g. British Journal of Clinical Psychology, Clinical Psychology Forum, Behavioural and Cognitive Psychotherapy etc) but using APA referencing style and ensuring adherence to the handbook guidelines. The report would usually contain an abstract; background or introduction to the evaluation (citing any standards, evidence or policies acting as drivers for the work); a description of the method (including details of participants, approvals gained, resources involved, and a description of any service user involvement (copies of surveys, questionnaires, or interview schedules should be contained with the Appendix); a summary of the findings (this may include the use of graphs or other pictorial forms of representation where this aids understanding of the findings. Use of percentages should be avoided when reporting findings from samples of less than 100); a clear statement of the practical recommendations arising from the evaluation (as well as containing recommendations for the subject area evaluated this section may also include recommendations for further evaluations); a list of references (in APA format regardless of journal format).

Section 3 A brief set of appendices (containing copies of surveys etc, forms of feedback such as posters, summary leaflets, or powerpoint presentations that have been used or are in draft form, plus a signed statement from the lead commissioner stating that they are happy for the findings to be disseminated.

Please note that only one ACP should be submitted for any one deadline.