This criterion assesses the extent to which the student’s report provides evidence that weaknesses and limitations in the investigative methodology have been assessed, and improvements have been suggested.
2 strands :
The report explains the relative impact of specific methodological weaknesses or limitations.
Realistic improvements to the investigation, that are relevant to the identified weaknesses or limitations, are explained.
methodological and procedural weaknesses and limitations
evaluation of the relative impact of weaknesses and limitations
evidence supporting the identified weaknesses and limitations
a clear understanding of the topic in the suggested context and of the methodology used.
Methodological refers to the overall approach to the investigation of the research question as well as procedural steps.
Weaknesses could relate to issues regarding the control of variables, the precision of measurement or the variation in the data.
There is no expectation that a student will address all aspects relating to methodological weakness and limitations. Nevertheless, when evaluating the results of an investigation, students should explain the relative impact of those that are significant. They can do this in a qualitative way, identifying minor and major weaknesses by explaining how the issue would affect the results.
Discussion of methodological weaknesses needs to consider both the issues in the methodology and their effect on the quality of the data. Weaknesses do not include errors due to careless manipulation skills or hypothetical events for which there is no evidence.
Discussion of limitations acknowledges that experiments will only go so far in answering the research question. Even if conditions are perfect, an experiment will still have its shortcomings. For example, a simulation may have few methodological weaknesses, but it will have some limitations.
The degree of impact of these weaknesses and limitations on the outcome of the investigation needs to be judged qualitatively.
The reliability of the results needs to be judged in the light of the uncertainties that have been established. The direction of any systematic error should be stated and related to methodological weaknesses and limitations.
Instruments that are faulty or that have not been calibrated correctly cause systematic errors. These errors, which affect accuracy, can also be caused by human error.
Random uncertainties are unpredictable in size and direction. The precision (measurement uncertainty) of instruments varies due to random errors. Judging the degree of impact of each measuring instrument on the results is an important task in science.
In investigations using databases, the student should not refer to the validity of the sources because this should have been done in research design. However, there are issues in the curation of databases, and a reflection in this regard adds value to the conclusion. Problems resulting from experimental and theoretical values present the same challenges.
Issues such as the cause of random uncertainties and systematic errors can be addressed. Limitations relate to the range and frequency of the collected data, procedural issues in data collection, and the precision and accuracy of the data. The student must explicitly consider whether control variables have been adequately dealt with.
The limitations should be consistent with the analysis and interpretation of uncertainties presented in data analysis. They should be supported by evidence instead of speculation. For example, a comment such as “the temperature of the surroundings was not controlled or monitored and may have changed during the extended testing period” has limited value. Limitations such as limited data or procedural weaknesses (e.g. an uncalibrated ammeter) are generic limitations.
Students are often familiar with certain methodological limitations (e.g. heat losses in calorimetry). These are valid limitations but will only add value when the student has tried to minimize their impact during design. For example, if the student worked with an open container without insulation, referring to heat losses in calorimetry is weak evidence of the understanding of this methodological limitation.
An appreciation of the limitations of an investigation can be shown by discussion of the range of data, including a justification of the chosen range of the independent variable.
Graphs may reveal systematic shifts. The student should discuss the size and direction of such a shift, relating it to weaknesses or limitations in the methodology.
realistic and relevant improvements
a clear understanding of the topic in the suggested context and the methodology used.
Suggested improvements should be realistic and relevant to the investigation. The improvements must be related to the weaknesses or limitations that have been identified, and should be feasible in a school environment or field course. They need to be based on the identified weaknesses that are relevant to the research question and methodology.
The student should avoid generalities such as “take more measurements” or “use a more precise measuring method”. Only if these generic issues are connected to specific issues can they be seen as improvements to weaknesses. The student should also avoid generic comments such as “eliminate friction” or “perform the experiment in a vacuum”.
During the design phase, changing to a more precise instrument may not be an option if the choice of instruments is limited. A student might realize that inserting an ammeter affected the measurement of current in a circuit, or that using a cold thermometer to measure the temperature of hot water affected the results. Taking steps to minimize such effects would be an improvement.