4.5 Notes/Examples

Observational Studies and Experiments

When researchers collect data about subjects without imposing any type of treatment, they are doing an observational study. Many conclusions have been based on observational studies. The discovery that smoking causes lung cancer was initially theorized based on observational studies. Many consumers of cigarettes and tobacco companies questioned the validity of such studies, suggesting that it could have been some other variables that caused the cancers, not the cigarettes. Retrospective studies, based on past history of lung cancer patients showed that a high proportion of them were smokers. This did not convince those who either enjoyed smoking, or were making money off of tobacco. There could be some lurking variables to blame, extra variables that were not taken into account, but were actually the cause. Prospective studies, following people in the future, were undertaken in an effort to see whether or not there was a link between cigarette smoking and lung cancer. The statistics were still called into question because statisticians know that the only way to truly show causation is through a controlled experiment.

An experiment imposes some 'treatment' on the subjects. A controlled experiment involves having more than one group, where the only variable that is different between the groups is the treatment being tested. And, subjects will need to be assigned at random (left to impersonal chance) to the various treatment groups to control for lurking variables. With regard to cigarettes and lung cancer, researchers would need to find a group of non-smokers and randomly divide them into two groups. The randomization will divide up lurking variables that the researchers cannot control for. Also, there needs to be a fairly large number of subjects in each group so that the results do not appear to be some kind of a fluke. The researchers would then need to force one group to smoke cigarettes, while making sure that those in the control group did not smoke. This would go on for several years and both groups would need to be checked for lung cancer regularly. Clearly, there is no ethical way to do such an experiment. We cannot force people to do something that we suspect may cause cancer! Scientists were able to experiment on rats to see whether or not cigarettes cause cancer, and it did. Eventually, the compilation of all of these studies convinced everyone that smoking does cause cancer.

The Three Elements of Good Experimental Design are:

• 1. Randomization--Subjects must be randomly assigned to treatment groups in an effort to divide up any lurking variables

  2. Control--There should be a control group-- a group that does not receive the treatment. Having more than one group, where the only difference is the treatment being tested, allows for comparisons to be made.

  3. Replication--There should be a large enough number of subjects so that the results seem believable. Also, the experiment should be able to be replicated on a different group of subjects.

[Figure2]

Experimental Design

In an experiment, the people, animals, or objects, that are being experimented on are called the subjects. The treatment that is being tested is the explanatory variable. The result, outcome, or change that happens (or doesn't happen) is the response variable. Keep in mind that sometimes it is necessary to give a pre-test prior to imposing the treatment. For example, if we are testing a medication that claims to lower cholesterol levels, we will certainly need to know the cholesterol levels of all of our subjects prior to giving them the treatment. At the end of the experiment we will again test them and then we can compare any change in cholesterol level.

The control group may be given no treatment at all. Or, you may want to use the control group as a way to compare a new treatment to an old treatment. For example, if someone has developed a new medication that they believe will cure headaches, they will want to compare it to aspirin, acetaminophen, and ibuprofen. Such researchers will likely form four randomly assigned groups (Groups A, B, C, and D), assigning the subjects in each respective group to take a specific one of the treatments whenever they have a headache and to record whether or not it worked and how quickly. After some length of time, the researchers will collect the data from the four groups and compare the results. With the only difference being which treatment was taken, researchers can make conclusions determining which treatment (if any) worked better than the others.

There are some other potential problems here though. For instance, would you want the subjects to know which medication they are receiving? It is very possible that they may have some preconceived notions regarding the effectiveness of one or more of these medicines. Such unconscious bias can influence how they perceive the treatment to work. What researchers often do to avoid any bias that the subjects will bring with them is to not tell them what treatment they are receiving. Such an experiment is said to be blind. It is also possible that the researcher may have preconceived notions, or hopeful expectations, regarding the effectiveness of one or all of the treatments. To avoid this, a third party can package the various treatments in similar looking containers, each marked only with a code, before the researcher distributes them to the subjects. In this case neither the subjects nor the researcher distributing the treatments know who is getting what. This is a double blind experiment, and is used often in clinical trials to limit bias.

Another issue is that often a patient's symptoms may improve just at the 'idea' of getting a medication. This is called the placebo effect. Imagine a child who is crying dramatically over a scraped knee, but stops immediately once mom puts a bandaid on. The bandaid is the placebo. It is also common for a participant, who believes that she or he is receiving a potentially promising medication, to have symptoms improve simply because of her or his expectation that they will. To account for this placebo effect, researchers will often give the control group a fake treatment called a placebo. A placebo is sometimes called a "sugar pill"-- it looks like the real treatment, but has no active ingredients. Placebos make blind and double-blind experiments possible. An experiment could involve a placebo shot, or even a placebo surgery (aka sham surgery).

We will demonstrate how to outline an experiment through the following examples. See the sample outline above as a reference.

Example 1

Suppose that a group of scientists have developed a medication that they believe will cure mean-ness. They are calling it Kind At Last (KAL). There are 520 mean people who are willing to participate in this study (300 males and 220 females). This pill needs to be taken twice daily and it may take a few weeks to be fully absorbed into a person's system. Identify the following, and outline a completely randomized experiment.

a) Subjects

b) Explanatory Variable

c) Response Variable

d) Will it be blind? Double-blind? placebo controlled? is a pre-test necessary?

e) Outline a completely randomized experiment

[Figure4]

The previous example is a completely randomized experiment because all of the subjects started in one group. All subjects were then randomly assigned to treatment groups, with any combination of genders being possible. What if it was theorized that this medication actually has different effects on males than on females? With a completely randomized design it is very possible that we would not end up with an equal number of males and females in each treatment group. If that were to happen, we would not be able to tell whether the treatment affected different genders in the same way or not

Randomized Block Designs

In such a case, it is a good idea to involve blocking in your experimental design. When it is suspected that different subgroups may respond differently to the treatment, the statisticians separate them at the beginning into intentional subgroups called blocks. The subjects in an experiment may be blocked by age, gender, race, previous medical history, etc. Be sure that you do not say that you will randomly assign to the blocks. You cannot randomly choose who is male or female, and you cannot randomly choose who is which race, etc. Each block is then randomly divided among the various treatment groups. This assures a more equal distribution of the subjects among the treatments. It also directly addresses the effects of this suspected lurking variable. Experimental designs in which blocking is used are called randomized block designs.

Example 2 -

Outline a randomized block design to test the KAL pills that blocks by gender. (Continued from example 1)

*Once you have done the comparisons within blocks, you will also want to compare across blocks to see if there are differences. For example, perhaps this KAL medicine works really well on males, but doesn't do a thing for females. Or, maybe one gender experiences negative side effects from the medication.

BACK