Validity, reliability, accuracy and precision

• • Validity: variables controlled so that any measured effect is likely to be due to the independent variable.

  • Reliability: consistent values in repeats and independent replicates.

  • Accuracy: data, or means of data sets, are close to the true value.

  • Precision: measured values are close to each other.

(a) Pilot study

• Integral to the development of an investigation, a pilot study is used to help plan procedures, assess validity and check techniques

• This allows evaluation and modification of experimental design

• The use of a pilot study can ensure an appropriate range of values for the independent variable

• In addition, it allows the investigator to establish the number of repeat measurements required to give a representative value for each independent datum point

(b) Experimental design

(i) Independent and dependent variables

• An independent variable is the variable that is changed in a scientific experiment.

• A dependent variable is the variable being measured in a scientific experiment.

• Independent and dependent variables can be continuous or discrete

• Experiments involve the manipulation of the independent variable by the investigator

• The experimental treatment group is compared to a control group

• The use and limitations of simple (one independent variable) and multifactorial (more than one independent variable) experimental designs.

  • The control of laboratory conditions allows simple experiments to be conducted more easily than in the field. 

  • However, a drawback of a simple experiment is that its findings may not be applicable to a wider setting.

  • A multifactorial experiment involves a combination of more than one independent variable or combination of treatments.

• Investigators may use groups that already exist, so there is no truly independent variable

• Observational studies are good at detecting correlation, but since they do not directly test a hypothesis, they are less useful for determining causation.

  • In observational studies the independent variable is not directly controlled by the investigator, for ethical or logistical reasons.

(ii) Confounding variables

• Due to the complexities of biological systems, other variables besides the independent variable may affect the dependent variable.

• These confounding variables must be held constant if possible, or at least monitored so that their effect on the results can be accounted for in the analysis.

• In cases where confounding variables cannot easily be controlled, a randomised block design could be used

  • Randomised blocks of treatment and control groups can be distributed in such a way that the influence of any confounding variable is likely to be the same across the treatment and control groups.

(iii) Controls

• Control results are used for comparison with the results of treatment groups

• Negative and positive controls may be used

  • The negative control provides results in the absence of a treatment. 

  • A positive control is a treatment that is included to check that the system can detect a positive result when it occurs.

• Use of placebos and the placebo effect.

  • Placebos can be included as a treatment without the presence of the independent variable being investigated.

  • Placebo effect is a measurable change in the dependent variable as a result of a patient’s expectations, rather than changes in the independent variable.

(iv) In vivo and in vitro studies

• In vitro refers to the technique of performing a given procedure in a controlled environment outside of a living organism.

  • Examples of in vitro experiments: cells growing in culture medium, proteins in solution, purified organelles.

• In vivo refers to experimentation using a whole, living organism

• Advantages and disadvantages of in vivo and in vitro studies

(c) Sampling

• Where it is impractical to measure every individual, a representative sample of the population is selected

• The extent of the natural variation within a population determines the appropriate sample size

• More variable populations require a larger sample size

• A representative sample should share the same mean and the same degree of variation about the mean as the population as a whole

• Random, systematic and stratified sampling

  • In random sampling, members of the population have an equal chance of being selected. 

  • In systematic sampling, members of a population are selected at regular intervals. 

  • In stratified sampling, the population is divided into categories that are then sampled proportionally.

(d) Reliability

• Variation in experimental results may be due to the reliability of measurement methods and/or inherent variation in the specimens

• The precision and accuracy of repeated measurements

  • The reliability of measuring instruments or procedures can be determined by repeated measurements or readings of an individual datum point. 

  • The variation observed indicates the precision of the measurement instrument or procedure but not necessarily its accuracy.

• The natural variation in the biological material being used can be determined by measuring a sample of individuals from the population

• The mean of these repeated measurements will give an indication of the true value being measured

• The range of values is a measure of the extent of variation in the results

• If there is a narrow range then the variation is low

• Independent replication should be carried out to produce independent data sets

  • Overall results can only be considered reliable if they can be achieved consistently.

• These independent data sets should be compared to determine the reliability of the results.

(e) Presentation of data

• Discrete and continuous variables give rise to qualitative, quantitative, or ranked data

• Qualitative data is subjective and descriptive.

• Quantitative data can be measured objectively, usually with a numerical value.

• Ranked data refers to the data transformation in which numerical values are replaced by their rank when the data are sorted from lowest to highest.

• The type of variable being investigated has consequences for any graphical display or statistical tests that may be used

• Identification and calculation of mean, median and mode

• Use of box plots to show variation within and between data sets

  • Median, lower quartile, upper quartile and inter-quartile range.

• Interpret error bars on graphical data

• Correlation exists if there is a relationship between two variables

  • Correlation is an association and does not imply causation. 

  • Causation exists if the changes in the values of the independent variable are known to cause changes to the value of the dependent variable.

• Positive and negative correlations

  • A positive correlation exists when an increase in one variable is accompanied by an increase in the other variable.

  • A negative correlation exists when an increase in one variable is accompanied by a decrease in the other variable.

• Strong and weak correlations

  • Strength of correlation is proportional to spread of values from line of best fit.

  • Correlation values are not required.