This week we were assigned to read a research paper related to our projects, and I chose to read "Will Teachers Receive Higher Student Evaluations by Giving Higher Grades and Less Course Work?" by John A. Centra.

This paper summarized the findings of a large-scale study over 55,000 courses at all levels in various subjects into the correlation between the grades received and student evaluations of their quality. This paper differs from the many others in the same niche topic by a few factors:

1. Scale: depending on the number of courses purged for each subject and evaluation area, between 46,182 and 55,549 courses were used for the study's analysis.
2. Student grade determination: other papers in this have tested the relationship between received grades and faculty evaluation through assigning students randomized grades before assigning an evaluation. Centra criticizes this method by pointing out that if a student were performing at an A level and expected an A grade, the sudden assignment of a B- or C grade would surely skew their perception of the course overall. In fact, student expectations of their grades at the end of the course are consistent with their final grades and therefore basing an analysis off of expected grades rather than final scores is appropriate.
3. Subject differences: Centra analyzes all of the disciplines represented in the study both separately and together to determine that if a relationship exists between students' grades and evaluations, the subject area in which it exists. Other studies generally examine an aggregated course body, which ignores the grading, teaching style, and student expectation differences represented in different disciplines.
4. Student evaluation format: Centra used courses that utilized the Student Instructional Report II (SIR II) as their method of evaluating a course and faculty. This evaluation survey differs from others in its emphasis on student learning. The questions ask about overall quality and other indicators in relation to the student's perceived learning in the class rather than a generic overall quality rating. In addition, the survey questions about workload and difficulty ask the student to compare the course to others, asking "I was challenged by this course" with options such as "much more than most" or "about the same" as other courses.
5. Workload definition: Centra's analysis of student workload does not include student estimates of time spent outside of class due to the formatting of the SRI II.

Due to the size and organization of this analysis, the results are much more consistent than in other papers on the topic of student grades and evaluations. Although the general belief in universities and colleges is that professors who give higher grades receive higher ratings (in other studies enumerated by a ~0.20 correlation between grade and evaluation), this was not supported by the analysis on any of the subject areas in Centra's study. In fact, expected grade correlated with overall evaluation by only 0.11. Even this level of correlation does not indicate that professors pander to students in order to receive more favorable ratings--Centra notes that students who receive a higher grade often enjoy the class more and rate the professor more highly. Similarly, a professor who teaches in an accessible way will often impart more information onto students, leading to both higher grades and high teaching evaluations. Overall, difficulty and pace had a greater influence on evaluation than expected grades in all subjects.

The most interesting and most highlighted result of Centra's work is that course evaluations and course difficulty have a quadratic relationship for most subjects. Courses that were evaluated as being too difficult were rated poorly, but so were courses that were rated as too easy or too elementary. This indicates that students respond to courses from their own levels of preparation for the material previous to entering the course. Professors who miscalculate the level of student preparation are the ones who receive the lowest evaluation, not those who make the course easy.

This research strongly relates to Anlan and my research topic of examining course evaluations and student performance/retention. One of our concerns for the research was that we would be predicting course performance off of skewed data--meaning that we'd be predicting grades based off of evaluations that too were impacted by grades--but this study implies that evaluations don't function on a quid pro quo basis. However, there are a couple of key differences between Centra's work and our own. Firstly, George Mason University does not use an evaluation format similar to the SIR II. The GMU evaluation survey does not indicate questions that prompt students to consider their own learning in the course except for the " The assignments (projects, papers, presentations, etc.) helped me learn the material" question, which is a very large departure from the style of Centra's evaluation. In addition, the question answers are not given on a relative basis, meaning that the scoring is absolute within the course instead of in relation to other courses. Further, GMU offers no questions that estimate student workload. In addition, Centra notes that for engineering and technology courses, workload/difficulty of the course were more strongly negatively correlated with overall course evaluation than for other subjects (where the correlation was occasionally a positive one).

Overall, this paper gave me much more of an insight into the details of student perceptions of courses and how they might affect the course evaluation.

Along reading the research paper and looking up unknown terms and procedures, this week brought a lot of independent learning about various statistical concepts and DM techniques. Here are some of the things I learned about:

• F-Test: sklearn's feature selection for f_regression calculates the linear relationship between features and some continuous variable of interest. This method returns both the F-values of features and the p-values of the F scores. For regression problems, F-values convey the strength of one model using a feature as an explanatory term over another using simply an intercept. This is tested with a residual sum of squares to enumerate the advantage of one model over the other.
• Beta weight: the amount in standard deviations that the variable of interest changes when an independent variable changes.