Examples and Inspiration

More than one way to evaluate

Below are some examples of evaluation tools from across study hubs, and from the Study Hubs in Place team. 

If you have a tool you would like to share with others, please email ship@studyhubsaustralia.edu.au

Recording and transcription

Recording interviews, focus groups and evaluation activities can improve the accuracy of evaluation data and reduce the reliance on facilitator notes. 

For example, interviews may be a short 1-3 Q&A with participants after an activity, recorded using video or audio functions on a phone.

Guidelines and ethics

The checklist below provides a quick guide on taking an ethical approach to recording and transcription. Recording is optional. Choose methods that are appropriate for your participants, activity and privacy requirements.

End-of-session reflection

At the end of the activity, participants could be asked:

• How confident do you feel now? ⭐⭐⭐⭐⭐

• Thinking back to before this activity, how confident do you think you felt then? ⭐⭐☆☆☆

• What helped increase your confidence?

This approach may be particularly useful when you are short on time, or for activities focused on aspirations, confidence, belonging, awareness of pathways, or study readiness.

Participatory approaches to evaluation

Many widening participation outcomes are difficult to capture through surveys alone.

Creative and participatory methods can provide rich insights into student experiences, learning and aspirations.

These methods can be particularly useful when working with younger participants, community groups, or participants who may be less comfortable completing written surveys.

Participatory activities generate valuable discussion and insights, but they still require planning and documentation. Facilitators should record participant responses, discussion themes and key observations so findings can be analysed and reported.

Good activities may become weak evidence if not planned and recorded well.

Draw–Write–Tell

Participants draw an experience or idea, write a few words about it and explain what it means to them. Variations may include building or creating objects using materials such as playdough or craft supplies.

Draw–Write–Tell is particularly useful with younger participants. It encourages reflection and generates both visual and verbal data.

It is useful for exploring aspirations, learning, belnging, and program experiences. It is suitable for younger participants, encourages reflection, and generates visual and verbal data. 

Example prompt

Draw what your future “after school” might look like.

Follow-up questions:

• Tell me about your drawing.

• What part feels most important?

• Has anything changed since participating in this program?

Further information

BetterEvaluation, Evaluating with and by, rather than for or of, children, https://www.betterevaluation.org/blog/week-6-evaluation-rather-for-or-children 

Reflection Walls

Participants respond to prompts on sticky notes, cards, whiteboards or online boards, providing quick feedback and insights into their experiences. Individuals can add to the “wall” throughout an activity, or at the end of an activity. 

It is useful for learning, immediate reflections and quick feedback while generating qualitative evidence about participant experiences.

Example prompts

• Today I learned...

• I was surprised by...

• Now I think...

• One thing I will do next is...

Further information

Guidance on how to run a reflection wall as a group activity: https://drive.google.com/file/d/1ccTuGx4Ra4DxOlIVCY-a2XrvJqLkfWTH/view 

Photovoice

Participants select or create images that represent a particular experience, outcome or idea and explain why they chose them. Look for common themes, differing perspectives and unexpected outcomes in participants' responses.

It is useful for exploring personal meaning, community perspectives, aspirations and belonging.

Example prompt

Choose an image that represents belonging, learning or aspirations and tell us why you selected it.

Example

In the example pictured here, team members from hubs across the SHIP project were asked to choose images relating to different themes and explain why they selected those images. The exercise was done using an interactive online whiteboard.

Further information

Guidelines on Photovoice, this method typically uses cameras (rather than selecting images): https://www.evalacademy.com/articles/a-picture-is-worth-a-thousand-words-photovoice  

An in-depth read from the creators of the method (1994, 1997): Photovoice: Concept, Methodology, and Use for Participatory Needs Assessment, https://deepblue.lib.umich.edu/items/095e892d-d6b7-4688-8327-fc5d14b7f31b 

The Ten Seed Technique

Participants are given ten counters (such as seeds, tokens or stickers) and distribute them across outcomes or priorities, or a scale. The discussion during allocation is as valuable as the final result.

Clearly describe the activity and its purpose before the exercise starts and create clear guidelines for expectations on listening to others and respecting differing views. 

TST is useful for exploring priorities, decision-making, community perspectives and the relative importance of different outcomes.

Example prompt

Ask participants to distribute ten seeds across outcomes such as awareness, confidence, aspirations, ask what is the most important of the three when they think about their future study, then discuss why they made those choices.

Further information

The original guide on TST developed by Ravi Jayakaran: https://www.echocommunity.org/en/resources/96a8b0e9-dfa2-4d58-875d-9bfe687ed60b  

A short overview and example in an Australian context: https://communityfirstdevelopment.org.au/stories/stories/the-ten-seed-technique 

Most Significant Change

Participants identify and describe the most important change they experienced as a result of a program.

It is useful for understanding personal impact, unexpected outcomes and not only whether change occurred, but which changes matter most to participants.

Example prompt

What has been the most significant change for you through this program?

Follow up by asking why the change was important, what contributed to it and what difference it has made.

Further reading

A guide created by the developers of the method (2005), https://www.mande.co.uk/wp-content/uploads/2005/MSCGuide.pdf