Assessment spring 2021 courses

The theme spring 2021 has to have been disengagement. By March 12 there were already signs that students were disengaging in pure online academic courses. This term ESS 101w Walking for Fitness, MS 150 Statistics, and SC/SS 115 Ethnobotany were purely online courses. SC 130 Physical Science was blended with online videos replacing lectures/demonstrations while the laboratories were held on campus with reduced class sizes. 

By March 12 submission rates in SCSS115 and MS150 were abysmal

Notes on course structures

ESS 101w used the Strava app to track students walking a mile a day for at least five days each week. This worked remarkably well. All of the students had or obtained mobile technology, downloaded Strava, and tracked their walks. Students did not have to upload any proof of walking, this was directly available from Strava. The students only had to launch and start Strava. Walks were recorded in Schoology. This course did not not see significant disengagement.

MS 150 Statistics used Instructure Canvas as the learning management system and Google Sheets for assignment completion and submission. Google sheets permitted completion of assignments using either a laptop or, by using the Google Sheets app, a mobile device. Tests were in Canvas. The course was conducted purely online. 

SC/SS 115 Ethnobotany used PowerSchool Schoology as the learning management system. Students submitted assignments either via iNaturalist (plant observations) or YouTube (presentations). Tests were in Schoology.

SC 130 Physical Science used Instructure Canvas, Desmos, and Google Docs. Students submitted laboratory reports in Canvas. Data analysis of laboratories was done in Desmos. All work could be completed on a mobile device equipped with the necessary apps. 

Disengagement

March 21 a more detailed look at the time trend in disengagement in MS 150

A look at the time trend for submissions in MS 150 done on 21 March provided a more nuanced look at the collapse in the submissions over time. The term had started off on with submission rates close to those seen the prior fall term. Fall term 2020 MS 150 had been purely online. In the fall of 2019 MS 150 was taught residentially. The collapse in assignment submissions seen spring 2021 was unprecedented. 

Fall 2020 the students were using Schoology and spring 2021 the students were using Canvas. The students, however, self-reported that Canvas was no more difficult to use than Schoology, with some students reporting that submissions were easier in Canvas, especially on the more capable Canvas mobile app. Students were clearly able to submit, as evidenced by high submission rates in January. Too, SC/SS 115 used Schoology and saw the same collapse in submissions as the term progressed. The collapse in assignment submissions was seen for courses using Schoology and Canvas. The common factor was that both were purely online courses. 

In SC 130 Physical science, however, while there was some fall off in engagement late in the term, the students rallied in the last couple of weeks and submitted the missing work.

Late term rally in participation was stronger in the blended ("hybrid") course than the pure online course

A late term rally was seen in both MS 150 and SC 130, but the SC 130 rally started earlier, climbed more strongly, and ended higher for the blended online/residential course. 

Other faculty have been anecdotally reporting disengagement by students. The causes are not known. Being the spring term, these students survive online courses fall 2020 and had signed up for continuing in online courses. For the most part, the students knew the online terrain ahead and had the tools to succeed. The students started strong in January. And then, something happened. What happened is not known but phrases such as Zoom fatigue, pandemic wall, and social isolation burnout are bandied about in the popular press. 

Grade distributions

Grade distribution spring 2021

The impact of disengagement can be seen in a deeply bimodal grade distribution where the mode was an F. Diving into the details shows that the source of the A's was not the source of the F's.

Grade distribution by course

While ESS and SC 130 drove a significant number of grades of A, the high number of grades of F were being driven by the online courses of MS 150 and SC/SS 115. 

Learning outcomes achievement across multiple terms

A look at achievement of course learning outcomes across three terms for three of the courses provides context and background for the impact of the disengagement.

MS 150 Statistics has three course learning outcomes that focus on basic statistics, paired variables, and open data exploration. Fall 2019 was residential, fall 2020 was online, and spring 2021 was also online. The spring 2021 MS 150 course was essentially identical to the fall 2020 course. The negative impact of disengagement on course learning outcome mastery was nothing short of calamitous. 

The blended SC 130 Physical Science course delivered four course level learning outcomes centered on performing experiments, demonstrating knowledge of concepts, apply mathematical models to experimental systems, and writing skills as evidenced in laboratory reports. While some loss was seen in mastery of course learning outcome number three, performance on the other outcomes was on part with prior performances. That one day of contact per week made a difference in the teacher-student learning connection. Note that the uptick seen in course learning outcome two may be attributable to the shift to online, open book, unsupervised quizzes and tests. Of note was that where, in the past, students were quick to leave when lab was over, this term the students lingered in the laboratory, as if they did not want the period to be over. A chance to sit, chat, and connect with other people outside their immediate family. 

SC/SS 115 Ethnobotany delivers two course learning outcomes. The first focuses on botanic knowledge of plants, the second on cultural knowledge as to the uses of plants. Here again the losses in learning mastery were seen for the spring 2021 term. 

Closing notes

While summer session courses will continue to remain online or blended, a return to residential instruction is planned for fall 2021. The recommendation is to return statistics, ethnobotany, and physical science courses to residential instruction fall 2021. A single section of MS 150 Statistics is recommended to remain online in the fall to provide service to students on other islands.