Fall 2020 was the first full term of multiple courses online and I sought to learn where there was room for me to improve in my courses. The students were surveyed anonymously by the college using an online survey instrument. I was granted access to a subset of the data for my own courses.
Across my four courses, ESS 101w Walking for Fitness, MS 150 Statistics, SC/SS 115 Ethnobotany, and SC 130 Physical Science, 57 students responded to the survey. In the following analysis the combined results for all four courses are being reported. At the course level sample sizes are small (Walking 8, Statistics 26, Ethnobotany 16, Physci 7) and provide additional insight only in a few specific metrics.
The statements were responded on a five point scale:
1 Strong disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly agree
The statements were worded such that disagreement was a response in the negative and an agreement was a response in the positive.
Across all statements there was an average of 4.65 indicating a preponderance of strongly agree responses. In general students agreed and strongly agreed with the statements in the survey.
Although students usually chose to agree or strongly agree, one student put strongly disagree for every single statement. That they chose to strongly disagree with every single statement suggests the possibility that they were not reading the prompts and were simply clicking on one for every statement. They did make a couple of comments and those are valuable as they provide context for strongly disagreeing with every statement.
If the college continues to meet online next semester, would you continue to take online classes with the college? If not, why?
"NO!!! the reason why I don't intent to take online class its because these online class are giving me to much to do things on my own and its not helpful with out the instructors."
Other comments
"My experience is actually a quite a challenge but I do have a lot of problems during the summer until now. this online class its not helping me or students at comfsm to fully success what they have gain through this semesters. Its wasting our time and our bell grand if we are not fully understand what we going to achieve in our class. So that is my own experience during the summer until this spring."
Of note is that the anonymous student was listed as being in a hybrid course that met once a week on campus and was thus not a fully online course. The comments may be a more general response to all of the online courses that the student had.
The statements were grouped into three areas: evaluation of the instructor, evaluation of the course, and evaluation of the course materials.
Instructor metrics
Overall, this instructor was effective.
The instructor welcomed and encouraged questions and comments.
The instructor presented the course content clearly.
The instructor emphasized the major points and concept.
The instructor was always well prepared.
The instructor made sure that the students were aware of the Student Learning Outcomes (SLOs) for the course.
The instructor gave clear directions and explained activities or assignments that emphasized the course SLOs.
The instructor initiated regular contact with the student through discussions, review sessions, feedback on assignments, and/or emails.
The instructor presented data and information fairly and objectively, distinguishing between personal beliefs and professionally accepted views.
The instructor demonstrated thorough knowledge of the subject.
I received feedback on assignments/quizzes/exams in time to prepare for the next assignment/quiz/exam.
Instructor metrics
The red horizontal line shows the average across all metrics in all three areas. For the instructor evaluation statements the average on each item was above the overall average of 4.65. The differences in the averages between the items was not significant.
Note that the horizontal axis is set at a score of four, agree, for this and subsequent charts, slightly exaggerating the small differences seen in the averages. When looking for areas of improvement by examining metrics with a lower average one might be making relatively meaningless distinctions between levels of agreement with the statement.
That the averages are very close to each other suggests the possibility that some students were "voting a straight ticket" in instructor evaluation section: clicking either agree or strongly agree across all statements.
Course
Overall, this course was a valuable learning experience.
The course syllabus was clear and complete.
The student learning outcomes were clear.
The SLOs helped me focus in this course.
This course delivered online was equal or better value than if I had taken it face-to-face
Assignments, quizzes, and exams allowed me to demonstrate my knowledge and skills.
The testing and evaluation procedures were fair.
There was enough time to finish assignments.
Expectations were clearly stated.
Course metrics
On these metrics there are some areas of relative weakness. "This course delivered online was equal or better value than if I had taken it face-to-face" was the lowest rating for all of the metrics and the difference was statistically significant.
The students did not find that the student learning outcomes helped them focus in the course. There may also be room for improvement in the fairness of tests. The slightly low rating for the fairness tests was not something I expected. Tests had no time limits and were effectively open note, open book - they were unmonitored online tests. Tests were accepted late, no test or assignment was ever rejected for being too late.
Course materials
Course materials were relevant and useful.
The textbook for this course was appropriate to the course content
I was easily able to access the textbook for this course.
The assigned readings were relevant and useful.
The on-line resources were relevant and useful.
The course online grade book Schoology was satisfactory.
Course materials
Course materials had the lowest averages for an area - all of the means were below the overall average. Because all of these metrics were the overall average, breakouts by course were looked at.
ESS 101 was a walking for fitness class supported by daily videos that addressed a weekly theme. MS 150 and SC/SS 115 were statistically equivalent to the overall average. Content delivery in SC 130 was supported primarily by instructor generated video content. Although SC 130 is lower than the other three courses, one is still making distinctions between relative levels of agree and strongly agree with the underlying statements.
ESS 101 had no textbook, all content was delivered via videos. Of eight students in the ESS course who responded to the survey three agreed, one strongly agreed, and four answered "not applicable." Not applicable was the appropriate answer. This again lends support to the possibility that some students may have been "voting a straight ticket" without always reading and thinking about the statement.
In both SC/SS 115 and SC 130 the shift to online and hybrid course modes led to the development of videos that provided the content in those courses in lieu of the textbooks. Only in MS 150 were students referred to the textbook for further information on a topic. The decision to rely heavily on videos to deliver content was a result of surveys that indicated students preferred content delivery via videos.
Bearing in mind that ESS had no textbook the responses should have all been "not applicable." Yet only four students chose NA with two agreeing that the non-existent textbook was easily accessed and two strongly agreeing that the textbook was easily accessed. As noted above, only in MS 150 were students who requested more information referred to the textbook. In SC/SS 115 content was delivered via videos. There were links to the text, but the material in the text was reproduced in the videos. Thus the strong agreement here is curious. I guess the text was easy to access, albeit somewhat redundant.
With content being delivered via videos and through interactions on assignments, messages, interpreting these results is difficult. Given the presentation of content in videos, the texts in these courses act more as references than assigned readings. In ESS there were assigned videos, but not assigned readings. Perhaps the students were responding to the assigned videos.
Bearing in mind that one is attempting to distinguish between statistically small differences in agreement and strong agreement, there is still the suggestion that online resources for MS 150 and SC 130 can be improved.
SC 130 was the only hybrid course. Although the average was still above a 4.0 level of general agreement, the lower average suggests that more of the students found Schoology less than satisfactory in support of the hybrid course. ESS students only interacted with Schoology for a weekly quiz, assignments were submitted as proof of walking to a private social learning Facebook group. Students in other courses who interacted more regularly with Schoology rated the package lower. SC 130 will be switching to Instructure Canvas in the coming term as part of a pilot.
Course materials is the area in which there is the most opportunity for improvement. Access to course materials fall 2020 was split in some instances between an interactive calendar and syllabus on the college website that predated online instruction and the learning management system. Anecdotal reports from students suggested that the students primarily utilized the learning management system to access course materials. Plans for spring 2020 include improving course materials access from the learning management system and working to render the material on the college website redundant. A new learning management system, Canvas, being piloted in some courses provides better support for calendar and syllabi.
Would you take online courses next semester?
When asked if they would take online classes again next semester based on their experience this term, seven of the 57 students in my courses said no, another five said they would but only because they felt that they had no choice. A couple students noted that there were graduating and would not be attending in the spring term as a result. The remaining 43 said yes, they would take online classes in the coming term. That represents 78% of the continuing students in my courses. This suggests that for three-quarters of the students the term was a positive enough experience as to want to continue in online courses, an encouraging result for a first full sixteen week term of online courses. That said, the willingness to continue with online courses is not the same thing as saying that they felt the online courses were of equal or better value to residential courses, as noted in the depressed rating on that metric above. There is perceived value in face-to-face learning experiences. This is a competitive edge the college has over other institutions that could also conceivably extend online courses into the FSM.
My thanks to the Office of Institutional Effectiveness for providing the data for this analysis!
Comments
Post a Comment