For Instructors

Using OSL in the Classroom

OSL labs are intended to be completed during class or as out-of-class assignments. For each lab, you will find a published article, an activity, and a data set. It is not necessary for students to read the article before completing the activity. Each activity contains a study description that summarizes the main concepts and hypotheses of the paper so that students may perform the data analyses without reading ahead of time (which would be very time consuming, especially if the labs are used during class).

In my own class, I typically demonstrate various data preparation techniques and all of the analyses using a different data set than the one the students will be using. These demonstrations may be performed during the class prior to the lab day or as part of a pre-lecture video. I personally use the pre-lecture video approach, so that students can watch and re-watch the videos to better learn the steps of the analyses. Then, on the day of the lab, I hand out the activity and provide students (who work in groups of 2-3) with a link to the data set. For the rest of the class period, I walk around and provide general advice. However, I try to avoid click-by-click instruction, which is provided in the pre-lecture videos.

I also often modify the activities based on my instructional goals. For example, toward the end of the semester, I like to have one lab that is very difficult. Students are provided with very little information about the data set, other than the study hypotheses. They must figure out what the design of the study must have been, and then decide upon the appropriate analyses. My experience has been that students are surprised (and encouraged) when they discover that they are able to run the same analyses that are reported in the results sections of a published paper.

Grading

One of the great things about using open data for lab activities is that students can “check their work” by comparing their results to those reported in published research articles. For those of us who teach statistics, however, this creates a problem. How do we grade assignments where students have all of the “correct” answers? By extension, how do we stop students from simply copying the results that are reported in the published papers? Below, I discuss four possible solutions to these issues:

1. Withhold the citation

This is the approach I use in my own classes, especially early on in the semester. Students complete the activities with no idea that they are analyzing real data from published papers. On the day they submit their assignments, I provide them with the citation and we discuss the papers, along with the potential issues that they encountered.

2. Results + Additional Analyses

Of course, students are savvy and may start searching for the published papers using the information provided in the study descriptions. To ensure that students actually conduct the analyses, I often ask students to provide their syntax or results files, in addition to their APA-style results sections. Also, when possible, I assign students to reproduce the analyses reported in the results section, PLUS conduct an additional analysis of their own choosing (that is not reported in the published paper). I can then assess the appropriateness and sophistication of that analysis, as well as the accuracy of the written description they provide.

3. Results + Explanation of Analyses

Another option is to ask students to reproduce the results reported in the published article and provide a written explanation of WHY the authors conducted the analyses that they did. Then students can turn in their syntax/output, as well as their explanation of the analyses. I find that students have the most difficulty understanding when to use particular tests and why they are appropriate. Requiring students to make the reasons for conducting a particular test explicit (in the context of a research question) may help students develop a deeper understanding of statistics and help you identify students’ misunderstandings.

4. "Reverse Engineer" the Results Paragraph

Finally, you might consider providing students with the Results section and the Data Set from the paper, but withhold the 'Analyses' portion of the Activity handout. Thus, the assignment is for student to decipher the Results paragraph, and determine which analyses the authors performed and why. Students would then perform the analyses themselves, making sure they matched the published results. For example, a given paper's results paragraph may indicate that the authors performed a t-test, but not indicate that it was an independent samples t-test. Based on the nature of the design, students will need to decide which is the appropriate test to perform.

Have another use for these activities? Send me an email describing how you use them to: kmcintyr@trinity.edu. -- Kevin