Equity Guidance

This coding guide has been designed to help us identify and extract information about how impact evaluations and systematic reviews address equity considerations.

The coding includes answering three questions, summarised in the table below and described in more detail in the text. The first two questions have fixed options for answers, and coders may select more than one answer as applicable. The final question is an open answer, designed to provide more detailed descriptions to corroborate the answers to questions 1 and 2. For further background on what we consulted for this guide, please see Morgan et al. (2016), Welch et al. (2017) and Masset and Snilstveit (2016).

Equity

Equity is the absence of avoidable and unfair conditions between or among people that hinder or prevent them from attaining their full potential.  It is inherently a moral judgment of fairness.  Since who gets to judge is almost always determined by a dominant power paradigm that considers one group of society unequal to another.

Sex and Gender (Welch et al. (2017; page 2))

• Sex is commonly used to refer to genetic, biological, and physiological processes. 

• Gender refers to the roles, relationships, behaviors, relative power, and other traits that societies ascribe to women, men, and people of diverse gender identities.

Sex and gender interact with each other and other characteristics to influence outcomes. For example, research indicates there are significant physiological differences in cardiac function between males and females as well as gender differences in how men and women who have heart disease are diagnosed and treated. Failure to take these differences into account, not just between men and women, but also across other characteristics such as sexual identity, age, income, education, ethnicity, religion, caste, location, can have serious, even lifethreatening, consequences for individual patients.

Gender analysis

Gender analysis is a socio-economic analytical framework for identifying and assessing inequality that come from: 1) different gender norms, roles and relations; 2) unequal power relations between and among women and men or girls and boys, and 3) the interaction of contextual factors with gender such as age, sexual orientation, ethnicity, education, employment status, caste, income. Such an analysis is systematically applied to all stages of the research process, starting with the formulation of the initial research question, followed by the development of methodology, conduct of the analysis, and interpretation of results and reflecting on their implications.

How does this study consider equity? (EQUITY FOCUS)

Please select one or more answers as applicable.

Does not address gender or equity - The impact evaluation does not explicitly address equity. If the analysis determines only average effects, the results are not likely to take equity into account.

Sub-group analysis by sex - Subgroup analyses may be done as a means of investigating heterogeneous results. They involve splitting all the participant data into subsets (subgroups), often so as to make comparisons between them. Choose this code if effects of the intervention were estimated for males and females separately. Find a table reporting the findings of the study. If the term gender, sex, female etc. is used as the label for sex-disaggregation of findings - then you know the study reports ‘Sub-group analysis by sex’.  CAUTION! Make sure you do not confuse the findings table with the table reporting the demographic composition of study participants. Reporting gender differences in baseline characteristics between intervention/control group does not count as ‘sub-group analysis by sex’. Also this needs to be disaggregated data rather than an interaction term in a regression or controlling for sex/gender as a covariate.

Sub-group analysis (other than sex) - Subgroup analyses may be done as a means of investigating heterogeneous results. They involve splitting all the participant data into subsets (subgroups), often so as to make comparisons between them. Choose this code if the impact evaluation presents outcomes disaggregated by an equity dimension other than sex (e.g. income, education, age, ethnicity, disability. Find a table reporting the findings of the study. If the term income, education, age, ethnicity etc. is used as the label for disaggregation of findings - then you know the study reports ‘Sub-group analysis (other than sex)’.  CAUTION! Make sure you do not confuse the findings table with the table reporting the demographic composition of study participants. Reporting differences in baseline characteristics between intervention/control group does not count as a sub-group analysis. Also this needs to be disaggregated data rather than an interaction term in a regression or controlling for ‘equity dimension’ (caste, poverty status) as a covariate.

Heterogeneity analysis (other than sub-group) – Does the impact evaluation go beyond calculating average treatment effects using a sub-group analysis? This can be typically done through:

- interacting the treatment with different characteristics 

- a quantile regression, which examines the effects across the range of the outcome variable as opposed to changes in the mean impact as is the case for Ordinary Least Squares regression (you can check it by searching for the term: ‘quantile reg’)

- estimating marginal effects of the treatment, e.g. at different ages (15, 20, 25 etc.) 

Equity sensitive analytical framework - Does the impact evaluation discuss the role of any drivers of equity considerations around the intervention and context in their analytical framework and/or theory of change? For example, an impact evaluation that presents a gender framework that considers theoretically how gendered social relations and institutions that determine and reinforce gendered relations relate to the intervention and outcomes being considered. Look at the methods section.  Ideally there will be a gender analysis framework mentioned that has a reference.  If not, see if there is any mention of gender analysis or any other theoretical framework, which is sensitive to equity considerations (e.g. social analysis, empowerment theory, sociological theories of Intimate partner violence (IPV). In either case, code yes for equity sensitive theoretical frameworks and/ or theory of change being explicitly mentioned in methods. Caution! Even if the intervention was designed to be equity-sensitive, only if an equity-sensitive theoretical framework is used in the analysis would we consider this code to apply.

Equity sensitive methodology - Does the study include any study components to assess the how and why (including mixed and qualitative methods) of differential impacts based on social and structural inequality? For example, in-depth interviews, focus groups or life histories with women only or a certain caste. This information will normally be contained in the methods section.

Equity sensitive research process - Is it informed by gender or equity considerations (who are the respondents, who collects/analyses data, when, where, who is present?) Do the authors of the impact evaluation consider the equity implications of data collection, including how sampling was undertaken, who was present during interviews, who was the person collecting data? For example, did the researchers consider the different work burdens of men and women and ensure that they chose times that were convenient for both to undertake data collection? Did they consider that if both males and females are present this may change the quality and accuracy of the data collected, as each may be reluctant to share information about their lives and work? Did they consider the sex, age, race, ethnicity, gender norms or occupation of the person collecting data and how this may affect the data collected? Have they eliminated risks to safety of women and girls in FCAS contexts? Do they provide confidential reporting of sexual harassment or gender-based threats of violence? Have data collectors received adequate training and supervision to help them become aware of their gender biases and to try to minimize these biases within the research process.

Measures effects on an inequality outcome - Does the impact evaluation assess the impact of the intervention on a measure of inequality? For example a study on the impact of cash transfers on income inequality or if the dependent variable is who (man or woman) makes the decisions in the household. This information will normally be included in the objectives, research questions and/ or methods section.

Ethics informed by equity - Does the impact evaluation consider the ethics of conducting research with vulnerable populations beyond ethics approval from internal review board? This information will normally be included in the data collection or methods section.

Open answer – provide a description of how the study considers equity, and for which population or group (EQUITY DESCRIPTION)

This is to corroborate and elaborate on the previous answers. For example, describe the sub-group analysis undertaken, how the intervention targets a researcher-defined disadvantaged group or how the authors used an equity-sensitive framework to inform their study. Please also note if and how the study considers intersecting inequalities, and if there is any consideration of how gender norms may influence outcomes. The answer can be verbatim copy-paste from the paper, or you can summarise it in the case of lengthy passages. Please also note the page number(s) where this information can be found.