Research Methods -- CRJU 3601 – Knowledge-Base Quiz 9

(The Null Hypothesis)

The Process of Operationalization

Procedure for Operationalizing a Study

1. Define the population and select the sample: Visualize a survey with two questions on it: one measures the independent variable (the "cause") and one measures the dependent variable (the "effect"). Identify the subjects or participants in the sample (to whom the survey questions are being asked).

2. Determine the dependent variable: Y^ (What is being measured? i.e. the "effect")

3. Determine the independent variable: µ (What is the "treatment?" i.e. the "cause")

4. Determine the type of data for each variable and the data-type combination: What is the type of data for each variable (metric or nominal)? Determine the data-type combination, i.e. metric-metric, nominal-metric, or nominal-nominal.

5. Determine the study type & procedure: If neither variable is nominal, it is a “relationship” study. If either or both are nominal, it is a “differences” study. Then, identify the appropriate procedure based on the data-type combination and the study design:

Pearson r

Bivariate Regression

One-sample t-test

Independent-sample t-test

Paired sample t-test

Analysis of Variance (ANOVA)

One-way Chi-square X²

Two-way Chi-square X²

6. Construct the model: to show the linear association between the two variables.

Y^ = u + µ + e

7. Formulate the Theory: Crystallize the theory into a 6 to 10 word statement that implies a cause-effect interaction. Examine the study and isolate what is being theorized.

8. Formulate the null hypothesis: It will always be stated as the “chance” hypothesis.

9. Test the null, record the results: Use the appropriate procedure for the data-type combination.

10. Write the analysis: State if the theory is supported by the data; if the null is rejected, calculate and report the Magnitude of Effect (MOE).

A Closer Look at the Theory

We can simplify the notion of a theory as a reaction to an action. “If I do this (action “A”), then this (action “B”) will take place. Therefore, a theory can be turned into sort of an “if/then” statement.

We know that in the Criminal Justice sciences, a theory proposes a possible explanation for a condition in society. (i.e. a so-called “cause” and effect.) Actually, a theory is really only a “guess” about why things are the way they are. So, theory becomes an idea that researchers wish to either validate or refute. They do this by formulating the null hypothesis, and then “testing” the null hypothesis. At the conclusion of the test, the “results” tell us if the theory is supported or not supported. The Criminal Justice scientist then provides an “analysis” of the results, to convey to the world what she or he discovered about the theory (that it is true or false).

Let’s use an example: We know that Education has a mitigating effect on criminal behavior. We also know that marital status has an effect on criminal behavior. Let’s study the effect of marital status on education. A researcher theorizes that a person’s educational level will vary according to their marital status. In other words, she wanted to determine what effect getting married has on a person’s education. She’s not sure how to construct the theory, there are two arguments.

1. On one hand, it could be argued that married persons are motivated to attain higher levels of education in to help pay for the high cost of raising a family. Married persons need higher education to be able to get a “career” and settle down, so to speak. 2. On the other hand it could be argued that furthering one’s education is a luxury not available to a married man or woman. This is because the demands of raising a family make paying for college classes impossible.

Which argument is chosen is ultimately up to the researcher, but at this point she is thinking about this idea and how to test it using basic social-science methodology. Ultimately she will use the Null hypothesis to test her theory, which ever one it is.

A Closer Look at the Theory

1. A theory is examined to discover the merits of the arguments supporting or opposing the theory. But which argument is supportable? The only way to confidently know is to “test” the theory.

2. The researcher then formulates and tests a “null hypothesis” for the purpose of testing the theory.

3. This then produces “results” based on the outcome of the test.

4. The Criminal Justice scientist then provides an “analysis” of the results which states whether or not the theory is supported by the data.

5. The bottom line is this: After performing the steps of operationalization the researcher demonstrates if the theory is supported or not supported based on the test of the null hypothesis.

A Closer Look at the Null Hypothesis

What is the “null hypothesis,” and how do we test it? The null will be explained in great detail later, but essentially the null hypothesis states that the so-called “cause and effect” between the two variables under investigation is not really a “cause” at all, but an illusion. The null says that the independent variable, which is the suspected “cause,” is actually just masquerading as a cause, and that other factors explain changes in the dependent variable. The social force you believe is “causing” the outcome might have just been a “one-shot-deal,” and based on the uninformed observations we obtained at first glance, is simply not supportable in the “scientific” community. The outcome you observed could be attributable only to chance. “Things are not as they seem,” you might say. That is why the theory must be “tested” (although it is tested indirectly by “testing” the null hypothesis).

How do we test the null? We have basically two choices: We can validate ideas using either words or numbers. Let’s think about this for a minute. Unfortunately, words can be manipulated to create confusion about ideas. The truth or falseness of an idea can

be clouded by expressing things in so-called “relative terms.” Therefore, social scientists choose to make use of numbers to validate ideas. This is because numbers, unlike words, can be used to clarify ideas, not to cloud or add ambiguity to them.

So the answer to “how do we test the null?” is: “we compare one number to another.” That sounds simplistic but it is really all we do. Specifically, we compare a number we compute (a “statistic”) to a number in a table in the back of this book (it’s called a “critical value”). That’s it. Test accomplished. If the “statistic” exceeds the “critical value,” we reject the null.

The process of operationalization culminates with the final four steps: State the theory. Test the null hypothesis, This produces our “results.” Then, the researcher analyzes the results to produce “the analysis”. Theory, null, results, and analysis.

A Closer Look at the Null Hypothesis

1. The difference null hypothesis states there is no difference present. The relationship null states there is no relationship present. The theory usually states that there is either a difference or a relationship among the variables.

2. Social scientists have found it is more convincing to use numbers to support a theory than words. Why? Because the merits of an argument made with words can be judged in relative terms (the eye of the beholder so to speak). However, you can always definitively state whether one number is greater than another.

3. How do we come up with numbers for a theory that is stated in words? We compute a single number (a statistic, by using one of our eight procedures) and then compare that single number to a “critical value” located in the appendix. We simply compare one number to another to see which is greater of the two values.

4. The bottom line is: The theory is not tested directly, but instead, a null hypothesis is formulated, then the null is tested by comparing one number to another. This provides a “numerically absolute” quality to our research.

Changing Words to Numbers, The Process of Operationalization

The process from the start of a theory to the analysis involves the magic of turning an idea into a number. That magic is performed using the process of “operationalization.”

Let’s return to the example: The theory states that a person’s educational level will vary according to their marital status. A simple glance at the descriptive statistics informs us that (it appears) “Single persons will have higher education than married persons.” How does she turn her “idea” into a testable theory that she can either validate or refute? She must operationalize. Here’s how to do it.

She interviewed a sample of people randomly selected from the population, each was asked two questions: (1) how many years of schooling have you completed and (2) what is your marital status? The variable “education” was measured by number of years school completed, and the variable “marital status” was determined by whether or not the person is, or had ever been, married.

We now engage in the process of operationalization, by following this procedure

1. How is the population defined and to what subjects in the sample are the questionnaires being distributed? Population = Adults. Sample = a number of randomly selected adults, who are the “subjects” or “participants” in the sample -- What are the two questions being asked? (1) What is your marital status? (2) How many of years school have you completed?

2. What is being measured as the dependent variable? Education

3. What is the “treatment”, or the independent variable? Marital Status: Single or Married

4. What is the type of data for each variable and what is the data-type combination??

The dependent variable is metric; the independent variable is nominal with two levels, making this a metric/nominal data-type combination.

5. What type of study is it and what procedure is appropriate based on the study design? Differences study -- Independent samples t­-test because the means of two independent groups are being compared.

6. Construct the model: Y^ = u + µ + e

Y^ [Dependent variable] = educational level of a given person

u [mean] = average educational level in group studied

µ [Independent variable(s)] = effect of treatment (marital status)

e [error] = might include IQ, SES, job type, family circumstances, or age.

7. What is the theory? “Single persons will have higher education than married persons.”

8. What is the null hypothesis? There is not a statistically significant difference in mean educational level between married respondents and single respondents.

9. (We cannot perform this step yet) Test the Null and Report the results: (We will either Reject or Retain the null hypothesis, and report the “test statistic” value, degrees of freedom, and the probability.)

10. (We cannot perform this step yet) Provide the Analysis: (We will report the findings of the study. State if the data supports the theory, and the effect size, which is the “Magnitude of Effect” (MOE) if any.)

A quick review of the descriptive statistics provides the researcher with a confirmation of what the theory is actually proposing. In the table below, it can be seen in the table below that that educational level is higher for single persons than it is for married persons.

Mean Std Dev Cases

MARITAL STATUS: Married 11.47 3.0762 827

MARITAL STATUS: Single 13.33 2.8576 282

POPULATION 12.00 3.1841 1109

Is the difference “caused” by the treatment: marriage? The theory must be “tested” using the null hypothesis. The null essentially states that despite what the descriptive statistics seem to show (that there is a difference between mean educational level between married and single persons), there is no real or tangible difference present.

The null further implies that the numbers we got from these people (that were randomly selected to be in our sample) show difference between these two groups (“married and single” persons), but the difference actually occurred by chance. In other words, the single people we drew at random from the population just happened to have more years of education, by pure chance or “luck of the draw” than the married persons we drew.

The “typical” level of education in the population (regardless of treatment, that is, married or unmarried) is 12 years. In fact, in the absence of other information about the independent variable, the arithmetic mean is our best predictor of what any given score in the population is (assuming there are no outlying scores that would skew the distribution). At first glance, the treatment (getting married) seems to have an effect on education. Notice that married persons have only 11.47 years and single persons have 13.33 years.

The task of the researcher will be to test the theory that the educational level (dependent variable) is “dependent” upon, or is associated with and related to, the levels of treatment, in this case marital status (independent variable). By formulating and testing the null hypothesis, it is possible to ascribe importance to the independent variable as being associated with education, or consider it as only marginally associated, if at all.

Criminal Justice researchers and social scientists test theories. They collect data and are then asked to determine either (a) if there is a significant difference between the means of groups of subjects who have received a different “treatment” or (b) if there is a relationship between two variables, one of which is “causing” the other one to change. She or he must determine if the difference or relationship under study is due to chance. If it is discovered that (based on data obtained in the sample) the likelihood of this difference or relationship occurring by chance is less than 5 in 100 (in the population), the calculated statistic is significant. That involves the 10 steps of operationalization.

Changing Words to Numbers, The Process of Operationalization

1. In order for a criminal justice theory to have credibility, it needs to apply to, or tell us something about, society in general. Therefore it needs to pertain to a large group (a “population”). The problem with that is, most of the time it is not possible, or financially feasible, to study and get information about everybody in the population. So, we settle for information collected from a small part (or random sample) of the population to draw our conclusions from (a sample).

2. Getting the information we desire from the data collected in the sample is no easy task. (Where to start? What does it all mean? etc.) Therefore, a stepwise guide on how to proceed is necessary. That guide is the procedure for “operationalization.”

3. The bottom line is this: if we get a sample that is very similar to (or representative of) the population, we can treat the sample as a “mini-population,” and apply the conclusions we make from the small group to the large group. But first, to get that information, we must accomplish the steps of “operationalization.”

1. A theory suggests an “if / then” relationship between two forces (if I do “X,” than “Y” will happen). In the Criminal Justice sciences, a theory suggests an interaction between two traits or characteristics. Example: in neighborhoods that have more establishments selling liquor, beer, and wine, you will find more prostitution. So, a theory is basically a “guess” about the interaction between two attributes or “variables.”

2. Words can be used to express relative importance from one idea to the next, but with Criminal Justice theories, we find that numbers give extra clarity to the notion of testing something “scientifically.”

3. Ultimately we want to “verify” a theory. We want to change if from a “guess” to an authoritative and meaningful statement. Therefore, it needs to be transformed from an idea to a number. For example, a researcher looked at the number of years of education among persons in the population. She noticed that single persons had more years of education than married persons. She theorized (guessed) that marital status affects educational level. The theory is an idea, but in order to provide support for it, she needs to turn it into number.

4. When she goes to determine if the theory is true or false, she will use the null hypothesis. Why? Because the null implies that her results from the sample occurred “by chance” and are really not an accurate representation of the forces at work in the population. If she can demonstrate that the null is most likely false, then the results are not attributable to “chance,” but have a measure of “scientific” validity.

5. The bottom line is, scientists transform words into numbers. They do it to test ideas in order to provide conclusions that can be verified by tangible, concrete (“scientific”) evidence. In the Criminal Justice sciences, such tangible or concrete evidence is hard to produce when only words are available to back up a theory. Therefore, numbers (statistics) provide the legitimacy necessary for a community of investigators to draw conclusions that are otherwise unavailable from mere statements of conjecture.