Assessment

Easy Does It

Many hear the word "assessment" and imagine that it must involve a burdensome amount of extra work only to learn that we fall short and should feel bad about ourselves. But the reality is that assessment need not be like that, and that best practice in assessment should empower faculty to make life better for students and themselves. Fortunately, our accreditors subscribe to the latter version. According to Middle States Characteristics of Excellence in Higher Education: Eligibility Requirements and Standards of Accreditation, assessment should follow these principles.

Easy Does It

o Useful (leads to improvement and development of programs and resource allocations, flexible and adaptable to needs)

o Cost-effective (simple and focused)

o Reasonably accurate and truthful (multiple measures, use of quantitative and qualitative measures)

o Planned (aligned to mission and goals)

o Organized, systematic and sustained (ongoing)

Clear and Simple (Barbara Walvoord)

In Assessment Clear and Simple, Walvoord lays out three steps for assessment: Goals, Information, and Actions.

Goals

What do we expect our students to be able to do as a consequence of our work with them? The goal statements should be meaningful to you and to your students. The goals or outcomes to be assessed do not need to comprehensive, and may be restated and narrowed to what would be useful to be examined at this time.

Information

How can we know our students can do what we expect? This can be a matter of measurement or evidence or observation or documentation or . . . . Remember that the standard is "reasonably accurate and truthful," and that qualitative measures or multiple measures can serve you better than relying only on objective tests.

What helps students to learn, and what gets in their way? A simple survey can provide context for interpreting the results from direct measures of achievement. Walvoord suggests three simple questions: 1. How well did you achieve the goal/s? 2. What helped with your learning? 3. What would help with your learning? The answers can help identify specific improvements in instruction.

Actions

Of course the last step is to make improvements and take other actions to address problems and to support and maintain areas of strength. Improvement is the true end of assessment, and if no actions are taken then assessment is wasted.

Notes on Scientific Rigor and Assessment

Sometimes, but not always, the measurements for assessments are amenable to high standards of scientific rigor; and in those cases the data collection and analysis should be rigorous. But in other cases that kind of rigor is not possible or just not feasible. In those cases the data collection and analysis should still be done as carefully as possible and within best practice for the kind of data and analysis involved. Remember: "Reasonably accurate and truthful."

Assessment is local, science is global. In science the inferences from data are used to support hypotheses designed to verify theory about very large, even global, populations. In assessment, analysis of data is used to find areas in need of improvement in our classes for our students. Theory and inference need not go any further than the immediate situations in our courses. Assessment is categorically not science but sometimes uses scientific procedures and methods.

Sample size and random selection. The most important feature of a sample for an assessment is whether it is representative of the population in factors that are important for that assessment. Any sample should be judged on that first. The old twenty percent rule from SUNY Administration can be laid aside in many cases; there was never a statistical basis for that to begin with. At the capstone level in most programs the populations are small enough to include everyone. If the measure is qualitative and in-depth (applying a rubric to an extensive portfolio) a surprisingly small sample of 8-10 cases could be enough. Larger sample sizes become important only when statistical analysis is planned, in which case an adequate sample size depends partly on the variation in the population on the factor being measured. So there is no easy answer--not even using a huge sample.