Rubin, Paolini, and Crisp’s (2010) Perceived Awareness of the Research Hypothesis (PARH) scale is a quick and convenient quantitative method for measuring the potential influence of demand characteristics in psychology research situations. It can help to refute the idea that research results are due to demand characteristics in the research situation.
Demand characteristics are "the totality of cues which convey an experimental hypothesis to the subject" (Orne, 1962, p. 779). If participants become aware of the research hypotheses, then they may respond in a way that they believe will confirm the hypotheses in order to be "good" participants and not "ruin" the research (Orne, 1962; for a review, see Strohmetz, 2008). These unnatural responses compromise the ecological validity of the research. To address the extent to which this may be a problem in their research, researchers may ask participants to complete Rubin et al.’s (2010) PARH scale near the end of their research session.
The PARH Scale
Your Thoughts About the Research
Please indicate how much you agree or disagree with each of the following statements:
1. I knew what the researchers were investigating in this research.
Indirect postexperimental feedback questions require research participants to describe the research hypotheses in response to open-ended prompts such as "What did you think this research was all about?" The problem with this indirect approach is that participants may be unclear about exactly what is being asked and, consequently, they may provide vague and uninformative answers (Page, 1973). In contrast, more direct questions about the research hypothesis have the potential to reveal the hypothesis to participants and result in false positive responses: Participants may indicate that they were aware of the hypothesis when, in fact, that awareness was only present after they read the feedback questions (e.g., Berkowitz, 1971). The PARH scale strikes a balance between these two concerns: It asks clear and direct questions about participants' awareness of the research hypotheses, but it does so without specifying the hypothesis.
The PARH scale is also innovative in using closed-ended items and a quantitative response format. This approach has four advantages:
Scores from the PARH scale can be used in a number of ways to test the possibilty that research results are the product of demand characteristics:
1. Identify positive outliers on the PARH scale: Positive outliers on the PARH scale (i.e., +3.00 or +2.50 standard deviations from the PARH mean) represent participants who are relatively confident that they are aware of the research hypotheses. Researchers can perform their statistical analyses with and without these outliers in order to demonstrate that their general pattern of research results is unaffected by these participants’ responses.
Perform a one sample
t test on the mean PARH value: If participants’ mean PARH
score is significantly below the scale midpoint (i.e., < 4.00) or not
significantly different from this midpoint, then researchers can claim
that, in general, their participants reported feeling unclear about the research
hypotheses. It is important to note here that the PARH cannot be used to establish whether or not participants are actually aware of the research hypotheses. It can only be used to assess the extent to which participants perceive that they are aware of the research hypotheses.
3. Perform correlation or regression analyses that include the PARH scale and the dependent variable(s): Nonsignificant relationships between PARH scores and the dependent variable(s) indicate that participants’ reported awareness of the research hypotheses is unrelated to their responses on the dependent variable(s).
Include the PARH scale as a covariate in analyses: If research
effects persist while controlling for PARH scores, then they are likely
to occur over and above the influence of any demand characteristics.
My colleagues and I have used the PARH scale to investigate the influence of demand characteristics in our own research (Owuamalam, Rubin, & Issmer, 2016; Meleady, Hopthrow, & Crisp, 2013;Rubin, 2011; Rubin, Paolini, & Crisp, 2011; Rubin, Paolini, & Crisp, 2013; Rubin & Paolini, 2014). Several independent researchers have also used the PARH scale in articles published in journals such as the Journal of Experimental Social Psychology, Personality and Individual Differences, and the British Journal of Health Psychology (Fuchs, Schreier, van Osselae, 2015; Milner, Wagner, & Crouch, 2016; Prior & Sargent-Cox, 2014; Poon & Chen, 2014; Stautz, Frings, Albery, Moss, & Marteau, 2016; Su, 2014; Wang, Chen, Poon, Teng, & Jin, 2016). Finally, several researchers have recommended the PARH scale (Allen & Smith, 2012; Ata, Thompson, & Small, 2013; Morrison, Madden, Odum, Friedel, & Twohig, 2014).
The PARH scale is free to use and reproduce for research purposes. No permission is required. However, the scale may not be reproduced for commercial purposes.
Please click here to download a hard copy of the scale for use in paper-and-pencil surveys.
Referencing the PARH Scale
The PARH scale was first used in a journal article that investigated bias against migrants. To refer to the PARH scale in published work, please use the following reference:
A self-archived version of this journal article is available here.
This webpage may be referenced as follows:
Rubin, M. (2016). A measure of the influence of demand characteristics. Retrieved from http://bit.ly/parhscale
Orne, M. (1962). On the social psychology of the psychology experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17, 776 783.
Page, M. M. (1973). On detecting demand awareness by postexperimental questionnaire. The Journal of Social Psychology, 91, 305-323.
Strohmetz, D. B. (2008). Research artifacts and the social psychology of psychological experiments. Social and Personality Psychology Compass, 2, 861-877.