CASE STUDY
CASE STUDY
Applying the 4 methods to a sample campaign
A Communications Campaign in Uganda
Context: The rise of Covid-19 has also seen the rise of violence against children and women in the house. To create a focus in which we can apply this context to in-depth interviews, focus group discussions, surveys and A/B testing, we focus on the following questions for our communications campaign:
How can we make it salient that VAW is on the rise?
How can we safely connect potential victims to resources?
Our Objective
To shift public perceptions and behaviors about violence against women given Covid dynamics.
We are interested in a communications campaign that 1) raises awareness; and 2) safely links potential victims to resources.
This can take the form of:
Radio shows/edutainment
Community events
Resource manuals
A capital city-wide messaging campaign
Our Audience
Men, women and youth in the Kampala metropolitan area
Success Outcomes:
Increased awareness of the rise of VAW during Covid;
Increased use of community resources (helplines, women’s resource centers);
Shifting attitudes to the violation of committing VAW
Our Prioritized Draft Messaging
“When I see violence against women, I report it. What about you? Call the toll-free hotline: 777”
"Reporting violence against women saves lives. Call the toll-free number: 777 to report a case of violence against women."
“7 women die of violence in the household every week in Kampala. Call the toll-free number: 777 to report a case of violence against women.”
Other draft messagings:
Help protect your community. Report violence in the household to this toll-free community resource: 777
When I see violence against women, I report it. What about you? Call the toll-free hotline: 777
Kampala: It is your responsibility to end violence against women. What will you do? Call 777 to report.
Identified the goals of our campaign;
Identified the audience(s) we need to engaged with in order for change to happen;
Identified what the success outcomes of our campaign are;
Created some initial draft messages appropriate for our difference audiences.
Applied case studies
In-Depth Interviews
Focus Group Discussion
Surveys
A/B Testing
1
In-Depth Interviews
“When I see violence against women, I report it. What about you? Call the toll-free hotline: 777”
Information campaign
Campaign Message;
“When I see violence against women, I report it. What about you? Call the toll-free hotline: 777”
Problem: CSO X is concerned with whether the information in this campaign will (i) increase awareness and usage of the reporting helpline, and (ii) attitudes of people about reporting gender based violence.
Solution: We test the campaign message in a in-depth interview before scaling it up.
How to test the campaign using IDI's
Define research questions
GBV Resource Center Workers
Do you think that people in this community would be willing to report GBV using a hotline instead of other channels like police/community leaders?
What potential challenges do you foresee with the use of a hotline?
Community leaders:
How many GBV cases against women are typically reported to you in a month?
Do you think that your community would be willing to report GBV cases against women using a helpline?
What do you think of the campaign message? Do you think that it can nudge people to report GBV cases?
Identify target respondents
“When I see violence against women, I report it. What about you? Call the toll-free hotline: 777”
What type of people do you think are ideal to participate in the IDI session to test the campaign message? Some groups that can be considered include:
Community leaders
GBV Resource center workers
Local police
Develop the interview guide
Considerations
Avoid questions with only “Yes/No” Type of responses. Include follow-up questions such as “If yes, how/why?”
Ensure most of the questions are open-ended.
Start with easy questions to put the respondent at ease. Ask the respondent self-reported questions first such as name, age, gender, level of education, role etc.
The interview guide should not be too long as it will lead to respondent fatigue.
IDI’s in a low-budget resource can typically investigate 1-2 themes.
Train the facilitators and test the guide
Considerations: The facilitator must be well trained to have some of the following skills
Interpersonal skills to build rapport with the study participants.
Non-judgemental and can manage his/her own biases.
Can manage difficult respondents.
Is conversant with the interview guide.
Testing the guide
You can conduct internal tests of the interview guide with staff to check for:
Flow of the questions and skip patterns
Length of the survey tool
Sensitive/Intrusive questions
Recruit respondents
Considerations:
Using the sampling strategy identified in Step 2, you will recruit study participants.
The key informants must have some knowledge or experience with GBV against women in the community.
20-30 key informants are sufficient to obtain insights about GBV in Kampala.
Conduct IDIs in a safe space
Considerations:
The IDIs must be conducted in a safe space free from distractions and human interference.
Safe spaces are also important to protect the identity of the survivors of violence.
In a medium-resource budget, you may hire rooms to conduct the interviews.
In low-resource budget the interviews can be done outside e.g under trees but ensure that you are seated far from people who may listen to your conversations.
Analyze the data using thematic and statistical analysis
Considerations:
Review the research questions and read through the transcripts/scripts for qualitative type of feedback.
Quantitative questions can be analysed statistically using tools such as Excel, Stata, SPSS among others.
The outputs from statistical analysis can be percentages, numbers and graphs
Make summaries and draw conclusions
Create categories of emerging themes and arrange responses according to the categories
Formulate the research narrative and write the report
2
Focus Group Discussion
“When I see violence against women, I report it. What about you? Call the toll-free hotline: 777”
Information campaign
Campaign Message;
“When I see violence against women, I report it. What about you? Call the toll-free hotline: 777”
Problem: CSO X is concerned with whether the information in this campaign will (i) increase awareness and usage of the reporting helpline, and (ii) attitudes of people about reporting gender-based violence.
Solution: We test the campaign message in a focus group discussion before scaling it up.
How to test the campaign using IDI's
Define research questions
Research questions
To what end do people witness violence against women in the community?
Who do you report the violence against women to? Which methods do you use?
Have you ever reported violence against women using the toll-free number 777? If yes, what motivated you to report.
If no, why did you choose not to report?
How many times have you done this?
When you reported, were your concerns attended to? Please describe the nature of the support that you received.
Identify target respondents
“When I see violence against women, I report it. What about you? Call the toll-free hotline: 777”
What type of people do you think are ideal to participate in the FGD session to test the campaign message? Some of the groups that can be considered include:
Men and women aged 18+
Community leaders
Religious leaders
Young children (9-17 years)
Craft open-ended Questions
Considerations
Review the research questions you have developed and ensure that they are open -ended enough.
Open-ended questions are ideal to avoid ‘Yes/No’ type of responses.
If the questions are mostly closed-ended, revise them to begin with “How, Why, What, Where and When”.
Considerations: The facilitator must be well trained to have some of the following skills
Interpersonal skills to build rapport with the study participants.
Non-judgemental and can manage his/her own biases.
Can manage group dynamics and difficult respondents.
Is conversant with the interview guide.
Testing the guide
You can conduct internal tests of the interview guide with staff to check for:
Flow of the questions and skip patterns
Length of the survey tool
Sensitive/Intrusive questions
Training the facilitation team
Considerations
Ensure that the facilitator is well trained to be able to:
Manage difficult respondents and control group dynamics.
Has good knowledge of the FGD guide and can ask probing questions where need be.
Can overcome their individual biases and use gender sensitive language such as GBV survivor instead of GBV victims.
Sufficient interpersonal skills to build rapport with all the study participants.
Recruit respondents
Considerations:
Nationally representative sample: This is a sample that represents the country’s population in key demographics.
Snowball sample: This option relies on your close network or at least one or more identified people who fit your inclusion criteria.
Random sample: With this option, every member of the population you are targeting has a chance of being selected for the test.
Targeted sample: This approach is preferred when you have a clear definition of your sample.
Implement and record FGDs in a safe space
Considerations:
Ensure you get an isolated space to conduct the FGDs.
Private spaces encourage study participants to talk more freely as they are sure that no one is listening into the conversation.
Also avoid areas where you are likely have have a lot of movement and interruptions.
In low-resource budgets, areas such as under trees can be ideal for conducting FGDs.
Unused office spaces can also be used in low/medium resource budgets.
Ensure you obtain consent from all the participants before you start recording the session.
Analyze the data using thematic and statistical analysis
Considerations:
Review the research questions and read through the transcripts/scripts.
Make summaries and draw conclusions
Create categories of emerging themes and arrange responses according to the categories
Formulate the research narrative and write the report
3
Surveys
Objective
CSO X is concerned that Violence Against Women (VAW) is increasing in urban areas.
CSO X wants to reduce VAW by creating awareness of the resources available to report violent incidents. Such resources include helplines of police stations and government institutions. The campaign will be deployed in Kampala.
The awareness is to be done through an information/message campaign.
The campaign message will be relayed through posters in schools, churches, mosques, public offices, and billboards placed strategically in the targeted areas.
Information campaign
Campaign Message;
"137 women are killed by a member of their family every day. ACT NOW. Call 999 - police helpline or 001220 - National Domestic Violence Hotline to report a case of violence against a woman."
Problem: CSO X is concerned with whether the information in this campaign will (i) increase awareness and usage of the reporting helplines, and (ii) change attitudes of people towards violence against women.
Solution: We test the information campaign message through surveys before roll out.
How to test the campaign using IDI's
Define research questions
Research questions
Does the message change people’s attitudes towards violence against women?
Does the message increase people’s awareness of the resources to use for reporting VAW?
Are people using the given hotlines for reporting VAW?
Identify target respondents
Inclusion criteria:
People living in Kampala
Adults of 18 years and above
People who can read and understand English
Exclusion criteria:
Anyone below 18 years of age
Anyone not living in Kampala
Anyone who cannot read and understand English
Calculate sample size
Considerations
The size of the sample isn’t calculated statistically, and is dependent on your budget and timeline.
If you are planning to pay incentives to your participants, consider how much of your budget can support this.
Craft survey questions
Considerations
Use your research questions to draft your survey questions.
Survey questions can either be close-ended or open-ended.
Here’s (survey questions tab) what we developed as survey questions.
Facilitator’s training
Considerations:
Use internal staff
Consider interviewers with experience in running surveys.
Approximately 2 support staff, with one running the surveys, and one monitoring activities.
Respondents recruitment
Considerations:
Snowball sample is best suitable for this test. This option relies on your close network or at least one or more identified people who fit your inclusion criteria.
If you are interested to get representative views of people from Kampala as a whole, then ensure you recruit from the different divisions of Kampala.
Data collection
Considerations:
Have a data collection plan in mind.
Consider the following in your plan:
The different tasks/activities to be done
Who is responsible for those tasks
The timeline for each activity
Status of each activity
See an example of a data collection plan here (Data Collection plan tab).
Analysis plan
Considerations:
Think of what it is you want to know from the data collected, as well as from the research questions.
Think of what statistical approaches can be used to achieve your objectives.
Think of the statistical tool you can use to achieve your objectives.
See here (Analysis plan tab) for an example of an analysis plan.
Analysis
Considerations:
Understand what your data looks like, i.e. what do the rows and columns mean.
Clean your data to make it ready for analysis. This includes removing those who didn’t complete the survey, remove duplicate data, among other checks.
Execute your analysis plan.
See what the execution will look like here (Analysis tab).
4
A/B Testing
Objective
CSO X is concerned that Violence Against Women (VAW) is increasing in urban areas.
CSO X wants to reduce VAW by creating awareness of the resources available to report violence incidents. Such resources include helplines of police stations and government institutions. The campaign will be deployed in Kampala.
The awareness is to be done through an information/message campaign.
The campaign message will be relayed through posters in schools, churches, mosques, public offices, and billboards placed strategically in the targeted areas.
Information campaign
Campaign Message;
"Gain framed message: Reporting violence against women saves lives. Call the toll- free number: 777 to report a case of violence against women."
"Loss framed message: 7 women die of violence in the household every week in Kampala. Call the toll-free number: 777 to report a case of violence against women."
Problem: CSO X is unsure of the 2 messages above, which message will have the best results in terms of (i) increasing awareness and usage of the reporting helplines, and (ii) changing attitudes of people towards violence against women.
Solution: We conduct an AB test to see which message performs best before roll out.
How to test the campaign using IDI's
Define research questions
Research questions
Which message leads to a change in people’s attitudes towards violence against women?
Which message leads to an increase in people’s awareness of the resources to use for reporting VAW?
Identify target respondents
Inclusion criteria:
People living in Kampala
Adults of 18 years and above
People who can read and understand English
Exclusion criteria:
Anyone below 18 years of age
Anyone not living in Kampala
Anyone who cannot read and understand English
Calculate sample size
Considerations
For a low resource AB test, the size of the sample doesn’t have to be calculated statistically. It is dependent on your budget and timeline.
If you are planning to pay incentives to your participants, consider how much of your budget can support this.
Design messages
Considerations
Gain framed message: This is a message framed on a positive outcome, e.g. “Reporting violence against women saves lives. Call the toll free number: 777 to report a case of violence against women”
Loss framed message: This is a message focusing on a loss, e.g. “7 women die of violence in the household every week in Kampala. Call the toll free number: 777 to report a case of violence against women”
Conduct pilot runs
Considerations:
Pilot runs are meant to test the following aspects:
Is the process of running the test clear to the research team?
Are instructions clear to your participants?
Is the test collecting meaningful data?
Run the pilot with a few random participants, as long as they meet your inclusion criteria.
The pilot is to be run in the same way you’ll deploy your test.
Respondents recruitment
Considerations:
Use a random sampling approach: With this option, every member of the population you are targeting has a chance of being selected for the test, as long as they meet your inclusion criteria
For AB tests, it is important you randomly choose who to participate in your test.
In this case, you’ll visit the 5 divisions of Kampala and randomly choose participants from each division. For instance, if your sample size is 100, then you’ll need to randomly choose 25 people from each of the 5 divisions of Kampala.
Randomization
Considerations:
Randomization is not always a necessity for AB tests. In case you avoid randomization, the approach will be to show the 2 messages to all of your participants and getting their perceptions.
If you decide to randomize, you’ll basically be determining which participant gets to see which message. For this, every participant will see only one message.
For instance, if you have 100 participants, you can use the randomization function in Excel to assign whether a participant gets message A or B. See here (Randomization tab) how we’ve randomized the participants into either message A or B.
Data collection
Considerations:
Have a data collection plan in mind.
Consider the following in your plan:
The different tasks/activities to be done
Who is responsible for those tasks
The timeline for each activity
Status of each activity
See an example of a data collection plan here (Data Collection plan tab).
The general process of collecting the data will be to show the message to the participant, then ask them some follow up questions around your testing objectives. See here (Questions tab) an example of questions to ask. Responses from these questions are what will inform the research team on which message worked best.
Analyse results using statistical software
Analysis plan
Considerations:
Think of what it is you want to know from the data collected, as well as from the research questions.
Think of what statistical approaches can be used to achieve your objectives.
Think of the statistical tool you can use to achieve your objectives.
See here (Analysis plan tab) for an example of an analysis plan.
Analysis
Considerations:
Understand what your data looks like, i.e. what do the rows and columns mean.
Clean your data to make it ready for analysis. This includes removing those who didn’t complete the survey, remove duplicate data, among other checks.
Execute your analysis plan.
See what the execution will look like here (Analysis tab).