Embedded Files
EXPERIMENTAL DESIGN OVERVIEW
EXPERIMENTAL DESIGN OVERVIEW
NOTES - EXPERIMENTAL DESIGN PRINCIPLES
NOTES - EXPERIMENTAL DESIGN PRINCIPLES
Posing an investigative question about a given experimental situation
Posing an investigative question about a given experimental situation
Sufficient time needs to be allocated for students to research the context and acquire appropriate and relevant contextual knowledge. For all grades, students need to identify a purpose and develop an investigative question which is informed by this contextual knowledge.
The investigative question needs to be about the effect of the treatment that is used in the experiment.
An appropriate investigative question would be: ‘Does being blindfolded have an effect on how long you can balance on one leg? I am going to investigate this for my year 13 Statistics class at XXX College’.
Planning the experiment using experimental design principles
Planning the experiment using experimental design principles
The plan needs to identify:
the experimental units (participants),
the treatment and response variables,
how the experimental units are randomly allocated to the treatment and control groups. (It is not appropriate for any experimental unit to be a member of both the treatment and control groups.)
Glossary notes from NZMaths: Experiment, Randomisation, Experimental design principles, Causal relationship claim, Treatment, Response Variable, Placebo
VIDEOS EXPLAINING EXPERIMENTAL DESIGN PRINCIPLES
VIDEOS EXPLAINING EXPERIMENTAL DESIGN PRINCIPLES
Other considerations - ETHICS & DATA issues
Other considerations - ETHICS & DATA issues
Ethical Considerations
Ethics refers to the correct rules of conduct necessary when carrying out research. We have a moral responsibility to protect research participants from harm.
Consent Form - template
Every participant must sign a consent form before participating in your experiment. Your consent form should cover the following:- The intent of the research
- What the participant will be asked to do and how long it will take
- Any risks associated with the experiment and what is done to minimise these
- How the data will be recorded and used
- Promise of the right to withdraw
- Note: You don’t need to tell them the entire purpose as long as it is ethical, so that they are “blind” to what treatment or control the two groups will receive.
- Eg. For an experiment testing the effect of distraction on reaction time, you would not say that participants will either be distracted or not, otherwise those who know that they were distracted might try harder or less hard and therefore change the results. You would just say “we will be testing your reaction time by completing an online test”, and they wouldn’t realise that other participants have different conditions.
- However in an experiment testing the effect of caffeine on reaction time, it wouldn’t be ethical to not tell them they were going to be given caffeine. So you would say “You will be given a drink, that may or may not contain caffeine and then we will test your reaction time by …”. That way they are still ‘blind’ to which group they are in (control or treatment), but have given informed consent for the possible treatment.
Recording your data
Make sure you record your data in a spreadsheet that can be used easily for data analysis.
Eg. If 2 independent groups, Put “Group” (i.e. treatment or control group) in column A, and
“Response” in Column BIf paired data, put “Group” in Column A (i.e. treatment or control group), “before” in column B and “after” in Column C, then find “Difference” in column D.
Save your data as a csv file so that you can make appropriate graphs in iNZight lite.
'Cleaning' your data
If something goes wrong in your experiment (eg a participant does not follow the guidelines) and you think this will affect your results unduly, you can justify excluding these results from your analysis.
If you do so, make sure you explain and justify this decision clearly.
You can't just take out any results that didn't follow your expectation/hypothesis/research.
ANALYSIS of Experimental RESULTS
& using RE-RANDOMISATION to make an INFERENCE
ANALYSIS of Experimental RESULTS
& using RE-RANDOMISATION to make an INFERENCE
Analysing Experimental Results
2 Independent Groups Experiments
create a box and whisker comparing the two groups (control and treatment group or 2 treatment groups)
comment on centre (mean, median), spread (IQR and range), and shape (skewness, peaks), as well as any unusual experimental results that you may or may not be able to explain.
Creating Box and Whisker Graph - 2 Independent Groups.webm
Paired Experiments
Create an arrow diagram using Google Sheets
Make sure that your control group are coloured differently or on a different graph to your treatment group so that you can see the differences in their results.
Comment on:
Proportion increasing/decreasing
Change in spread of the arrows
If the changes are different for the control group.
Paired experiment arrow diagram.webm
Also Create a box and whisker of the differences between the "before and after" for both the experimental and control group and then compare them (centre, spread, shape and unusual features).
Box Plot of Differences Paired experiment.webmRe-Randomisation Test to make a formal inference.
In essence this is a process where we randomly reallocate all the data to Group 1 and Group 2 and see how much difference there can be just due to chance alone. If the difference we saw in the result of our experiment is unlikely to occur by chance alone (eg less than 10% chance) then we can be confident that our intervention/treatment actually may have made a difference in the response variable.
From NZQA clarifications "The formal statistical inference needs to be a causal inference based on the strength of the evidence from the re-randomisation of the experimental data. An appropriate causal inference would be:
‘As the tail proportion is less than 10% this tells me that the observed difference in the median length of time the students in my year 13 Statistics class at XXX College balance on one leg with and without a blindfold is unlikely when chance is acting alone. Because the students were randomly assigned to the groups we may claim that wearing a blindfold was effective in lowering the balancing time for these students. The students were not randomly selected so we need to consider carefully as to which wider groups the conclusions may apply."
If the tail proportion is: <10% unlikely due to chance alone <5% very unlikely <1% highly unlikely
What does a small tail proportion mean? The smaller the tail proportion, the less likely it is that the “luck of the draw” or chance alone, would produce a difference this big. A small tail proportion tells us that our experimental results are out in the tails of the randomisation distribution, so that random reallocation almost always gives differences that are smaller than what we got in our experiment.
What does a larger tail proportion mean? It tells us that random re-allocation often produces a difference larger than we got in our experiment. Our experimental results therefore give no evidence a true treatment difference exists.
Example:
In this experiment the group that saw 2mm dots had a mean estimate that was 8.46 units higher than the group that saw 6mm dots.
The re-randomisation test showed that a difference this large or higher, happened only 6 times out of 1000. Because the tail proportion is less than 1% this tells me that the observed difference in the mean estimate with 2mm dots and 6mm dots is highly unlikely to be due to chance alone. Therefore I can claim that observing smaller 2mm dots was effective in improving the accuracy of estimate for these students.
To understand this a bit more, you may want to look at this video about randomisation variation which explains how much variation could happen within data from experiments just by chance alone.
IMPORTANT - definitely watch this one!
This video explains how to use the "randomisation test" part of iNZight to see if the results from your experiment could have been due to chance alone.
EXEMPLARS and ASSESSMENT INFO
EXEMPLARS and ASSESSMENT INFO
Note: these exemplars are 'excerpts' - meaning that some of the parts of the student response may be missing - so you can't necessarily assume this means that if you do everything on these, you'll get the same grade. They are good for a general guide though.
NZQA Exemplars on "Estimation" experiments (some at each grade level)
NZQA Exemplars on "Tricky Question" experiments (some at each grade level)
Assessment Information
If you take Psychology, you may do this internal in conjunction with the psychology standard and will only need to produce one product as long as it meets both standards. The main differences is that the Statistics standard requires random allocation of participants to 2 groups (treatment and control) and the analysis requires you to do a re-randomisation test.
Experiment
in conjunction with your group plan an experiment on an aspect of a psychological phenomenon - eg. perception, cognition, attention, memory or a social. Before going further than the question, check your question idea with your teacher. Your plan must be checked and ethical approval given before carrying our your experiment.
Assessment Tasks:
plan and conduct an experiment (in groups, but written up individually)
complete a planning document in milestones on the google classroom to show the process and analysis - you will be able to get feedback on each milestone.
Final Product: Create a poster with digital or live presentation or a newspaper article summarising your experiment and findings. Both of these will have some form of appendix or supplementary information with your research, full analysis and bibliography.
Success in this topic will gain 4 credits. Generic Mark Schedule here
Communicating your findings
In the world of science, including psychology, research findings are often communicated through scientific research posters that are displayed and sometimes also presented orally at academic conventions.
Findings are also communicated to the public via newspaper articles - this is a great opportunity for you to show your creative writing skills! (and could also possibly count for your English portfolio as long as it is above 600 words)
For this reason, we would like your final product to be one of these 3 formats:
research poster with extra info available either on QR codes, in an appendix or via live or videoed presentation
slide show presentation with extra info given via live or videoed presentation/pod casts
newspaper article summarising your experiment and results, with appendix or QR code to extra detail.
Some examples of how canva can be used to make a research poster: https://www.canva.com/posters/templates/research/
An example of a research poster with QR codes to get more information.
There is a current move away from scientific posters with heaps of info, to clearer more succinct ones with extra information available if desired through QR codes.
Interesting article with ideas "The evolution of research scientific posters" and Better Posters.
QR code generator: https://www.qr-code-generator.com/
Note: Even the APA (American Psychological Association) have begun using this format at their conventions: https://convention.apa.org/blog/rethinking-the-science-poster . This link has a downloadable exemplar if you want to use it.
An example of a newspaper article or press release that summarises an experiment:
https://www.nzherald.co.nz/nz/text-speak-may-strain-your-brain-report/B7TKLQPOSUAPHRZLBXBTCCKOEE/
Report abuse