In order to investigate the results of crowdsourcing resources, a research study was conducted using the resource tool during the 2017/18 school year. Data was collected from two AP Environmental classes which had a total of 43 students enrolled. Five hundred resources were selected at random and processed for the study.
Finding #1: Students select text-based resources more often than videos but find the videos more useful. (Figure #1)
Results: Students selected resources that were text-based (readings) 2:1 over videos resource (325 of the resources were text-based and 167 were videos, with the remaining being resources such as podcasts and on-line simulations). However overall, students found the videos more useful than the text-based resources, giving videos an average rating of 4.3 and text-based resources and an average rating of 3.9. This difference proved significant; an unpaired t-test with a null hypothesis that the difference of the means was equal to zero returned a p-value = 3.15x10-9.
Discussion: In student-surveys conducted in my class, students consistently prefer and find teacher-led lessons more useful than chapter readings; videos are similar to lessons in that they can be more personal and engaging. Videos are also better at highlighting the main ideas since the presenter can pause or use inflections to draw attention to these key points. For this reason students find videos easier to digest. Although students may rate videos as more useful, it is still important to assign readings, since being able to read and pull information from text-based resources is an important skill that students need to practice and develop.
Finding #2: Of all the video resources, students found TED Talks the most useful. (Figure #2)
Results: The video resources that students selected were evenly split among the 3 categories of videos (Educational, Screencasts and Ted Talks). However students rated TED talks the highest (an ANOVA one-way analysis of variance confirmed that there was a significant difference in the mean ratings among the three categories of videos, p<0.01).
Discussion: Students found an equal number of videos for each of the three categories: Educational, Screencasts, and Ted Talks. However, they found TED talks significantly more useful,. TED talks are given by professionals who have a high degree of expertise in their fields, and because they specialize in the topics that they are talking about they are equipped to provide relevant, informative, and captivating presentations. The presenters of screencasts and other educational videos are generalists who know a little bit about a lot of topics. They may be better at generating videos for a whole course but TED presenters are better at presenting their specific lesson. In addition, TED Talks are usually between 10 and 15 minutes in length, the optimum time to allow for a solid presentation without losing the interest of the audience.
The most valuable aspect TED talks is that they often inspire students to continue discussing and researching the topics. This was evident by the fact that TED talks were some of the most shared resources in the class. Inspiring students to continue thinking about and discussing the topics is extremely important and leads to a greater understanding and long-term storage of the material.
Finding #3: Students select shorter videos more often but rate longer videos as more useful. (Figure #3)
Results: Although students preferred shorter videos of 5-10 minutes over longer ones that were greater than 10 minutes (a chi-squared test confirmed a significant difference between the number of selections for each video length, p-value = 2.69x10-7), students found longer videos more useful (an ANOVA one-way analysis of variance reported a strong significant difference between the mean ratings for each video length, p=3.42x10-5).
Finding #4: Students select shorter readings more often but rate longer readings as more useful. (Figure #4)
Results: The same trend that was found for the video resources was also true for the reading resources; students were more likely to choose shorter readings of 1-3 pages in length over longer readings greater than 3 pages in length (a chi-squared test confirmed a significant difference between the number of selections for each reading length, p-value = <2.20x10-16). However, students found the longer readings between 4-9 pages in length more useful (an ANOVA one-way analysis of variance returned a significant difference in the mean ratings for each reading length, p=6.03x10-3).
Discussion for Findings #3 and #4: One of the most important findings of this study was that although students chose shorter resources more often (5-10 minute videos and 1-3 page text-base resources) they found longer resource more useful. Shorter resources are initially more appealing because they are easier to get through, they also tend to come up first in Google searches because people are viewing and sharing these resources more often. A big part of digital literacy education is training students that length quite often is an indicator of usefulness. Students should take this into consideration when selecting a resource; a 1-page website or 5 minute video cannot provide thorough coverage of the topic.
Finding #5: Students chose educational text-based resources most often and may have found opinionated sites such as those posted by activist groups and non-reputable news sites as least useful.
Results: Of the text-based resources, students chose resources that were intended for education most often, 46% of the resources chosen by students were educational ones. Activism resources were the second most chosen resource, 34% of the resources were this type (a chi-squared test confirmed a significant preference by students for some sites over other, p-value = <2.2x10-3). Resources that had slight biases either because they were from non-profits that were advocating for certain causes or because they were from profit-driven non-reputable news sites that were littered with ads and clearly concerned with hooking their audience, received the worst ratings; however, an ANOVA one-way analysis of variance did not detect a significant difference between any of the average ratings, p=0.12.
Discussion: For the text-based resources, students were clearly drawn to educational sites. This is no surprise, with education as the goal, these sites are designed to provide general, unbiased, easy to digest coverage of the topic. Although the trend was not significant, it did appear that students found sites that were advocating for a cause and non-reputable news sites least useful. Activism sites tend to be written in a politicized, opinionated manner that can repel students. The non-reputable news sites are full of ads that can annoy students. These sites also tended to be written in an inflammatory, opinionated manner that can frustrate and bother students.
This study differentiated resources that students found most useful from resources that students were most frequently drawn to. Hopefully educators can use this information to guide their students away from enticing but inadequate sites and to sites that they will find more engaging, useful and effective at covering learning objectives.
One major assumption of this study was that students were capable of accurately rating the usefulness of resources; however, the validity of this assumption needs to be tested. Future studies will assess students after they use different types of resources to see how their assessment scores correlate with how useful they rate each resource. The aim is to identify the type of resources that students learn best from along with the ideal length for both videos and text-based resources.
Stay Tuned!