Feedback

Direct interaction with the students offers valuable information to evaluate the effectiveness of any teaching and learning models, but the standard student survey provides the main tool for this purpose, possibly complemented by other indicators such as attendance records. Data accumulated over several years can be used to confirm trends or reinforce perceptions, when correlated to any differences in the corresponding teaching and learning scenarios. The data collected during the last seven years (since 2013/14) will be subdivided into three periods:

2013/14, when I first took up a full time position in Norway — adopting for the first time a blended learning model based on a dedicated Google Site
2014/15 to 2017/18, during my term as vice-rector at the University of Porto (during this 4-year leave of absence I lectured only one course in Norway) — first week on-campus, remaining weeks online using the same Google Site organization (later replaced by Canvas)
2018/19 and 2019/20, when I resumed my full-time position at USN — blended learning model based on Canvas

The full data set collected through the student surveys since 2013/14 is made available as appendix #5, and their most relevant components will be presented in the following sections.

2013/14: Blended learning using a Google Site

The first implementation of the model presented in this document took place in 2013/14 using a dedicated Google Site. A detailed presentation of this framework can be found in a 2014 paper that is included as appendix #6, entitled "Flipped Classrooms: From Concept to Reality Using Google Apps". Two courses were lectured using this model, one to bachelor students (DFDS3101 Digital Systems), and the other one to master students (SESH6210 Hardware/Software Co-Development of Embedded Systems). The following excerpts were taken from the bachelor student forms (response rate was 91%, corresponding to 10 responses):

Fig. 19: DFDS3101 2013/14 — Excerpts of the student survey results.

The master students were likewise positive as illustrated below (response rate was 100%, corresponding to 10 responses):

Fig. 20: SESH6210 2013/14 — Excerpts of the student survey results.

2014/15 - 2017/18: One week on-campus, remaining weeks online using Google Site (first 3 years) or Canvas (4th year)

During my 4-year leave of absence period I only lectured the DFDS3101 Digital Systems course. Since the entire content of this course had been prepared to support online delivery, the blended learning model was replaced by a first week on-campus followed by weekly online sessions with each group of students. It is interesting to compare the survey results for the two delivery models, and further information with that respect will be presented ahead. Most interesting, however, was the fact that attendance was higher online than on-campus. The attendance data presented below corresponds to the weekly meetings with each group:

Fig. 21: DFDS3101 2014/15 — Student attendance during the 14 online weeks reached 95% (8 out of the 12 students never missed a single session, and only one student missed two sessions).

The same Google Site organization ensured 100% availability of the course content and full support for collaborative online discussions (through an embedded Google Group). In spite of the different delivery model, the positive student perception was similar to the blended learning model used in the previous year, as the following excerpts illustrate (response rate was 83%, corresponding to 10 responses):

Fig. 22: DFDS3101 2014/15 — Excerpts of the student survey results.

The same organization and delivery model was adopted in 2015/16 and the attendance rate was even higher, having reached 96%:

Fig. 23: DFDS3101 2015/16 — Student attendance during the 13 online weeks reached 96% (only one student missed more than one session).

The positive perception of the class continued as illustrated by the following excerpts taken from the student survey forms (response rate was 100%, corresponding to 10 responses):

Fig. 24: DFDS3101 2015/16 — Excerpts of the student survey results (the last comment shown above was the one quoted earlier in the “Planning” subsection).

It is particularly interesting to notice that the perception of teacher availability is entirely independent of his/her physical presence on-campus — which shouldn’t come as a surprise, since visiting the teacher's office to ask a question takes time and doesn't work every time.

The same Google Site organization and content was again adopted (for the last time) in 2016/17. Not surprisingly anymore, the attendance rate to weekly online activities was close to 100%.

Fig. 25: DFDS3101 2016/17 — Student attendance during the 13 online weeks reached 97% (no student missed more than one session).

Instead of using printed forms, the same student survey questions were presented to the students using Google Forms, collecting the comments grouped below (response rate was 100%, corresponding to 10 responses):

Fig. 26: DFDS3101 2016/17 — Comments collected in the "what is particularly good" section of the student survey.

Fig. 27: DFDS3101 2016/17 — Comments collected in the "possible improvements" section of the student survey.

Since the number of student survey responses collected from 2013/14 to 2016/17 was always exactly 10 (although the number of students varied from 10 to 12), it is interesting to present the most significant quantitative results side-by-side (response rates were 91%, 83%, 100%, 100%: total number of students was 11, 12, 10, 10):

Fig. 28: DFDS3101 2013/14 to 2016/17 — Teaching results.

Fig. 29: DFDS3101 2013/14 to 2016/17 — Teaching methods.

Fig. 30: DFDS3101 2013/14 to 2016/17 — Overall quality.

The complete set of graphs, covering the 4-year period where a Google Site was used to support the blended (2013/14) and online (2014/15 to 2016/17) delivery models, is available in appendix #5.

The delivery model comprising one week on-campus plus all remaining weeks online was last adopted in 2017/18, but in this case Canvas (which had meanwhile become available) was used instead of Google Sites. Contrary to what had happened in previous years, the attendance record dropped significantly — not because of Canvas, but rather because the course moved from the Fall semester to the Spring semester, when the students were struggling with the workload imposed by their 20 ECTS final bachelor project.

Fig. 31: DFDS3200 2017/18 (the course was the same, but the code changed from DFDS3101 to DFDS3200): Student attendance during the 9 online weeks fell to 73% due to overlapping with the 20 ECTS final bachelor project.

Response rate to the student survey fell as well, reaching only 56% in the mid-term survey, and 44% in the end-of-term survey (in a total of 18 students), where the problem indicated above was explicitly mentioned:

Fig. 32: DFDS3200 2017/18 — Comments collected in the "what is particularly good" section of the student survey.

Fig. 33: DFDS3200 2017/18 — Comments collected in the "possible improvements" section of the student survey.

The first comment shown above comes as an outlier in relation to the average perception of teacher availability, as collected over the preceding years using the same delivery model. It is particularly interesting because it reminds us that students have different (quantitative and qualitative) expectations regarding teacher availability. Actually the availability of the teacher in online delivery models goes well beyond email, since every group attended weekly synchronous video conference meetings — exactly as in the previous years. Although no data was collected to support quantitative proof of this claim, the comments collected from the students show that speed of response and support provided during online / blended delivery models can be well above the average that corresponds to traditional on-campus delivery. Nevertheless this comment shows that student expectations include the traditional on-campus face-to-face meetings, and it is therefore perceived as an advantage of the blended learning delivery that was resumed in 2018/19.

2018/19 - 2019/20: Blended learning using Canvas

The blended learning delivery model used in 2013/14 was readopted in 2018/19, but now using Canvas. Contrary to what was usual, the size of the class in the DFDS3200 Digital Systems course was much smaller in 2018/19, and the number of responses collected wasn't sufficient to support analysis and discussion (although they can be accessed in appendix #5). Further to this course, the same blended learning model was used to deliver another bachelor course (DFDV3100: Computer System Architectures and VHDL Programming), and one master course (ES-SHC4300: Software/Hardware Co-development of Embedded Systems, successor to the 2013/14 SESH6210 course with the same name).

The end-of-term survey results of the DFDV3100 course does not separate the activity of the two teachers that lectured different parts of the course, and that occasionally creates a difficulty to clarify possible follow-up actions (response rate was 67%, corresponding to 6 responses).

Fig. 34: DFDV3100 2018/19 — Comments collected in the "what is particularly good" section of the student survey.

Fig. 35: DFDV3100 2018/19 — Comments collected in the "possible improvements" section of the student survey.

It is particularly interesting to notice that students still appreciate traditional lectures ("men jeg savner likevel litt undervisning noen ganger"), and that the quality of the videos is important to promote acceptance of flipped classroom models based on recorded lectures ("videoene til Jose kan vaere monotone og lite motiverende å så på, detter har han sagt han jobber med").

The same blended delivery model was adopted in the 2018/19 ES-SHC4300 master course. The overall evaluation and the comments collected indicate a positive perception, but the response rate was only 36% (8 out of 22 students):

FIg. 36: ES-SHC4300 2018/19 — Overall evaluation.

Fig. 37: ES-SHC4300 2018/19 — Comments collected in the "what is particularly good" section of the student survey.

Fig. 38: ES-SHC4300 2018/19 — Comments collected in the "possible improvements" section of the student survey.

Student survey responses are already available for the Fall semester of 2019/20. The data available for the DFDV3100 Computer System Architectures and VHDL Programming now includes the responses to a mid-term survey, which clearly show acceptance and approval by the students (response rate was 65%, corresponding to 13 responses):

Fig. 39: DFDV3100 2019/20 — The instructor: Mid-term student survey results.

Fig. 40: DFDV3100 2019/20 — The course: Mid-term student survey.

Fig. 41: DFDV3100 2019/20 — What is best: Mid-term student survey.

Fig. 42: DFDV3100 2019/20 — Possible improvements: Mid-term student survey.

Fig. 43: DFDV3100 2019/20 — General comments: Mid-term student survey.

The end-of-term survey collected a smaller number of responses, which nevertheless confirm the positive perception of the class (response rate was 40%, corresponding to 8 responses):

Fig. 44: DFDV3100 2019/20 — Overall evaluation: End-of-term student survey (the full set of charts is available in appendix 5).

Fig. 45: DFDV3100 2019/20 — What is best: End-of-term student survey (comments addressing the other teacher were blurred out) .

Fig. 46: DFDV3100 2019/20 — Possible improvements: End-of-term student survey (comments addressing the other teacher were blurred out) .

The same blended learning delivery model was adopted in another master course (ES-OOP4200), but again the survey data forwarded to the two teachers does not provide individual feedback information, making it unusable in the context of this document.