ASTE 2015

Download from App/Play Store

Formative Assessment Techniques

Continuous Formative Assessment

Employing the Scan & Post Technique for Continuous Formative Assessment of Student Problem-Solving Skills in Science Classes

Norman Herr, Mike Rivas, Tae Chang

Department of Secondary Education

California State University, Northridge

Abstract: 

The prevalence of student-owned, camera-equipped mobile devices that are capable of accessing free collaborative cloud-based file-hosting services, offers instructors the opportunity to monitor student understanding during science instruction without the use of dedicated hardware or fee-based applications.  Using the scan & post technique, science instructors can now collect photographs, scans, and movies of student diagrams, multi-step solutions, observations and experimental results in real-time to make formative assessments of student skills. The instructor creates a shared folder using a file synchronization service (e.g. DropBox®, Box®, OneDrive®, Google Drive®) for each class, subfolders for each day, and sub-subfolders for each question, problem, or observation.  Students draw their diagrams and perform solutions on paper, scan them with appropriate mobile apps (e.g. iOS camera apps, CamScanner®, TurboScan®) and upload them to the shared folders in the cloud.  Students can also upload photographs and/or videos of homework, lab-setups, observations, and experimental results.  Within a few moments, the instructor sees everyone’s work in a single matrix and can thereby ascertain the level of student understanding so that instruction can be adjusted to meet real-time needs.  Instructors obtain a permanent cloud-based digital record of all student work, allowing them to track student progress over time.  

Formative Assessment 

Most traditional formative assessment techniques provide insufficient data to assess student skills.  For example, verbal responses or hand signals can not adequately illustrate competence with multi-step problems.  Posing oral questions of the class may provide inadequate data because the oral response of a few students does not necessarily reflect the class as a whole. Other techniques, such as choral responses and hand signals, may offer inaccurate data because those students who are uncertain how to respond to questions or prompts may imitate their peers to avoid embarrassment. Although some traditional formative assessment techniques provide sufficient, adequate, and accurate data, they do so in a belated manner.  For example, portfolio checks, exit cards, and quizzes may provide sufficient, adequate and accurate information, but only after such information is no longer useful for adjusting instruction. For example, a portfolio entry may show a student’s detailed solution to a problem, but because such information is collected after class is dismissed, it does not help reform instruction to meet immediate needs. One of the best techniques for formative assessment of student understanding in science is the student-controlled white-board technique advocated by modeling theorists (Hestenes, 1987; Liang, Fulmer, Majerich, Clevenstine, Howanski, 2012) in which students draw models of the physical world on individual white boards and then share them with the instructor and class.  Although the white-board technique allows instructors to scan the complex models and solutions of the entire class, students must hold their boards while the teacher scans the room, and this ephemeral data is lost as soon as the boards are erased. Although traditional formative assessment techniques provide useful information, there is a significant need to continuously collect data of student understanding that is sufficient, adequate, accurate, timely, and permanent. 

Classroom Response Systems

A variety of Classroom Response Systems (CRS) have been developed to overcome the deficiencies associated with traditional formative assessment techniques. Examples include dedicated audience response systems such as Turning Technologies®, iClicker®, and Audience Response Systems®, as well as mobile apps such as Socrative®, Poll Everywhere®, TopHat®, ClickerSchool®, Text the Mob®, Shakespeak®, Naiku Quick Question®, and Edmodo®.)  Such systems track individual responses, display results from polls, confirm understanding of key points, and gather data for reporting and analysis. Studies have shown improved student participation, attendance, and engagement with the use of CRS’s (Beatty & Gerace, 2009; Bennett & Cunningham, 2009; Buchanan, 2001; Chevalier, 2013; Gok, 2011; Peat & Franklin, 2002). Such systems not only provide information regarding student understanding, but also increase students’ accountability for their own learning (Han & Finkelstein, 2013; Akpanudo, Sutherlin, & Sutherlin, 2013; Kaleta, 2007).  Although CRS’s have been shown to be a valuable formative assessment tool, current systems do not provide adequate means for collecting spontaneous free-form data such as student diagrams, multi-step solutions, observations and experimental results. Most systems require instructors to create multiple choice and short answer questions prior to instruction and are incapable of soliciting free-form responses necessary to assess higher levels of understanding (Price, 2012).  As a result, lessons may tend to be scripted and rigid as teachers adjust their instruction to include pre-written questions, which has been identified as a major deterrent to successful formative assessment (Beatty, Feldman & Lee, 2011). 

Scan & Post Technique for Formative Assessment

To address the limitations and deficiencies of existing formative assessment techniques, the authors propose the scan & post technique that allows instructors to spontaneously collect photographs, scans, and movies of student diagrams, multi-step solutions, observations and experimental results in real-time to make formative assessments of student skills using data that is sufficient, adequate, accurate, timely, and permanent. The scan & post model employs student-owned, camera-equipped mobile devices (smart phones, media players, tablet computers) and free cloud-based file synchronization services (e.g. DropBox®, Box®, OneDrive®, Google Drive®) to collect such data (Herr & Tippens, 2013). The instructor creates a shared folder “in the cloud” for each class, subfolders for each day, and sub-subfolders for each question, problem, or observation.  Students draw their diagrams and perform solutions on paper, and then photograph or scan them with appropriate mobile apps (e.g. iOS camera apps, CamScanner®, TurboScan®) and upload them to the shared folders in the cloud (Figure 1).  Students can also upload photographs and/or videos of homework, lab-setups, observations, and experimental results.  Within a few moments the instructor sees everyone’s work in a single matrix and can thereby ascertain the level of student understanding so that instruction can be adjusted to meet real-time needs. If desired, the instructor can grant viewing privileges to all students so they can learn from the work of their peers.  Using the scan & post technique, the instructor obtains a permanent cloud-based digital record of all student work, and this allows them to track student progress during instruction as well as over multiple days. 

Focus and Value for ASTE Members

The scan & post method of formative assessment helps promote a paradigm shift in STEM education towards collaboration and accountability as encouraged by NGSS. Students can no longer hide behind their raised hands.  Instead, they must produce diagrams and solutions to illustrate their understanding.  These diagrams provide a permanent digital record that can be used by the instructor to gauge growth in student understanding accompanying instruction. Such diagrams provide benchmarks during instruction so that teachers can determine student skills and understanding before summative assessments are given.  

The scan & post technique allows instructors to spontaneously collect photographs, scans, and movies of student diagrams, multi-step solutions, observations and experimental results in real-time to make formative assessments of student skills using data that is sufficient, adequate, accurate, timely, and permanent.   The scan & post  technique promotes student metacognition and makes possible a degree of continuous formative assessment never before possible.  Teachers are provided real-time snapshots of student understanding that can be used to reform their instruction to meet real, rather than perceived, student needs. This workshop will be of particular interest to methods instructors who train future science teachers. 

Workshop Activities 

During this workshop, participants will learn how to  employ the scan & post technique to perform formative assessments of science learning. Participants will gain experience developing resources that they can use to help their own students master these skills. The effectiveness of this workshop will be evaluated by the data that is scanned & posted by participants and by their ability to provide examples of how it can be used to help teachers engage learners in science & engineering practices specified in Dimension-1 of NGSS.

Ongoing contact following worship – Participants will be encouraged to implement the scan & post technique of formative  assessment and pose questions and provide feedback through our website. 

Expertise of workshop presenters – The presenters have authored numerous works pertaining to the use of computer supported collaborative science of which scan & post is a specific strategy. (Herr et. al, 20141,b,c,d, e; 2013; 2012a,b; 2011a,b; 2010a,b).  The scan & post model presented in this workshop has been developed and implemented in pre-service and in-service sessions at our university. The scan & post model, along with companion cloud-based formative assessment techniques, is presently being used in the science teacher preparation program at CSUN, one of the largest producers of science teachers in the nation, and is the subject of research studies by university faculty and graduate students.

Budget - There will be no charge for attending the workshop.  

Audience -  20-25 participants

Material and technological needs - This workshop requires wireless Internet connectivity. Workshop attendees are requested to bring a laptop computer if possible. All work will be conducted “in the cloud” and will be available for participants to access after the conference.

References

Akpanudo, U. , Sutherlin, A., & Sutherlin, G. (2013). The Effect of Clickers in University Science Courses. Journal Of Science Education & Technology, 22(5), 651-666. doi:10.1007/s10956-012-9420-x

Beatty, I, Feldman, A., & Lee, H. (2012). Factors that Affect Science and Mathematics Teachers' Initial Implementation of Technology-Enhanced Formative Assessment Using a Classroom Response System. Journal Of Science Education & Technology, 21(5), 523-539. doi:10.1007/s10956-011-9344-x

Beatty, I. D., & Gerace, W. J. (2009). Technology-enhanced formative assessment: A research-based pedagogy for teaching science with classroom response technology. Journal of Science Education and Technology, 18(2), 146-162.

Bennett, K.R, Cunningham, A.C., (2009). Teaching formative assessment strategies to preservice teachers: Exploring the use of handheld computing to facilitate the action research process. Journal of Computing in Teacher Education, 25, 99-105. 

Buchanan, T. (2001). The efficacy of a World Wide Web mediated formative assessment. Journal of Computer Assisted Learning, 16(3), 193-200.

Chevalier, J. (2011). Teachers’ perception of handheld response systems as a tool for formative assessment in high school classrooms. ProQuest LLC., Ann Arbor, MI 

Gok, T. (2011). An evaluation of student response systems from the viewpoint of instructors and students. Turkish Online Journal of Educational Technology, 10(4), 67–83.

Han, J., & Finkelstein, A. (2013). Understanding the effects of professors' pedagogical development with Clicker Assessment and Feedback technologies and the impact on students' engagement and learning in higher education. Computers & Education, 6564(76). doi:10.1016/j.compedu.2013.02.002

Herr, N.; Rivas, M..; Chang, T.; Tippens, M.; Vandergon, V.; d'Alessio, M.; Nguyen-Graff, D. (2014 expected). Continuous formative assessment (CFA) during blended and online instruction using cloud-based collaborative documents. In Koç, S; Wachira, P.; Liu, X. Assessment in Online and Blended Learning Environments. In press.

Herr, N.,Rivas, M. (2014). Using Cloud-Based Collaborative Resources to Conduct Continuous Formative Assessment. Proceedings of the 12th Annual Hawaii International Conference on Education, January 5-8, Honolulu, Hawaii.

Herr, N., Rivas, M. (2014). Engaging Students in the Science and Engineering Practices of the Next Generation Science Standards (NGSS) with Computer Supported Collaborative Science (CSCS). Proceedings of the 12th Annual Hawaii International Conference on Education, January 5-8, Honolulu, Hawaii.

Foley, B., Reveles, J., Herr, N., Tippens, M., d'Alessio, M., Lundquist, L., Castillo, K., Vandergon, V. (2014). Computer Supported Collaborative Science (CSCS): An Instructional Model for Teaching the NGSS. Proceedings of the 2014 International Meeting of the Association for Science Teacher Education.

Herr, N., Rivas, M. (2014). Computer Supported Collaborative Science (CSCS): Engaging Students in the Science and Engineering Practices of the Next Generation Science Standards (NGSS) with Computer Supported Collaborative Science (CSCS). Proceedings of the 2014 International Meeting of the Association for Science Teacher Education.

Herr, N., & Tippens, M. (2013). Using Scanning Apps on Smart Phones to Perform Continuous Formative Assessments of Student Problem-Solving Skills During Instruction in Mathematics and Science Classes In T. Bastiaens & G. Marks (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2013 (pp. 1138-1143). Chesapeake, VA: AACE.

Herr, N., Rivas, M., Foley, B., d'Alessio, M. & Vandergon, V. (2012). Using Cloud-Based Collaborative Documents to Perform Continuous Formative Assessment During Instruction. In T. Bastiaens & G. Marks (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2012 (pp. 612-615). Chesapeake, VA: AACE.

Herr, N., Rivas, M., Foley, B., Vandergon, V., d'Alessio, M., Simila, G., Nguyen-Graff, D. & Postma, H. (2012). Employing Collaborative Online Documents for Continuous Formative Assessments. In P. Resta (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference 2012 (pp. 3899-3903). Chesapeake, VA: AACE.

Herr, Norman, Mike Rivas, Brian Foley, Virginia Vandergon,and Gerry Simila (2011) Using Collaborative Web-based documents to Instantly Collect and Analyze Whole Class Data. Proceedings of the 9th Annual Hawaii International Conference on Education, January 3-7, Honolulu, Hawaii.

Herr, Norman, Mike Rivas, Brian Foley, Virginia Vandergon, Gerry Simil, Matthew d'Alessio, and Henk Potsma (2011) Computer Supported Collaborative Education - Strategies for Using Collaborative Web-Based Technologies to Engage All Learners. Proceedings of the 9th Annual Hawaii International Conference on Education, January 3-7, Honolulu, Hawaii.

Herr, N. & Rivas, M. (2010). Teaching the Nature of Scientific Research by Collecting and Analyzing Whole-Class Data Using Collaborative Web-Based Documents. In J. Sanchez & K. Zhang (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2010 (pp. 1029-1034). Chesapeake, VA: AACE.

Herr, Norman and Mike Rivas (2010) The use of collaborative web-based documents and websites to build scientific research communities in science classrooms. Proceedings of the 8th Annual Hawaii International Conference on Education, January 7-10, Honolulu, Hawaii.

Herr, Norman (2010) Instructional Resources for the Teaching of Secondary School Science  - Application of Research Findings to the Teaching of Inquiry-Based Science. . Proceedings of the 8th Annual Hawaii International Conference on Education, January 7-10, Honolulu, Hawaii (abstract)

Hestenes D. (1987) Toward a modeling theory of physics instruction. American Journal of Physics 55(5):440-454