Student evaluations

Student evaluations of instructor, course, and course materials provide guidance for the institutions on areas of relative strengths and areas where there may be room for improvement. Bearing in mind that old data is not usefully actionable data, the intent of this report is to convey broad themes emerging in the evaluations to decision makers as rapidly as possible. 

This report presumes familiarity with the student evaluations form in use at the institution. The responses were converted to numeric values:

Strongly disagree: 1
Disagree: 2
Neutral: 3
Agree: 4
Strongly agree: 5



The number of student responses by term is seen above. 



The differences are small between the terms. Due to the large sample sizes the differences are technically statistically significant, but the effect sizes are small to very small. At best the data only hints at the possibility that student ratings may be slightly lower this fall. 

As noted in the past, students tend to respond with agree or strongly agree. Teasing information out of means that are similar requires the realization that an excess of agrees relative to other ratings in the section represent a downgrade for that item. The following report looks to highlight what are small but potentially real differences. 

Instructor evaluations

Click to enlarge

As in prior terms, the first item rating the instructor's overall effectiveness was lower than the other metrics in this section of the student evaluations. The reason for this remains unknown. 



In terms of t-scores, the first item significantly underperformed relative to the other items. Areas of relative strength included awareness of the student learning outcomes, clear directions, and instructor's knowledge of the content area. Area of relative weakness included maintaining regular contact and instructional objectivity. These two area were responses to the following prompts:
  • The instructor initiated regular contact with the student through discussions, review sessions, feedback on assignments, and/or emails.
  • The instructor presented data and information fairly and objectively, distinguishing between personal beliefs and professionally accepted views.
These two areas have been areas of relative weakness in prior terms. 


Noting that hybrid refers to science courses with an online lecture and residential laboratory, the above chart suggests that the differences in delivery mode with respect to regular contact are small. One might expect that students will rate regular contact higher for courses with a residential component. That said, instructors in online courses should continue to seek ways to improve regular contact and engagement. One recommendation would be to encourage online faculty to utilize the Canvas Teacher app on their mobile devices. 

Timely feedback had been an area of relative weakness spring 2023, that area improved in summer 2023 and is now very close to the overall average.

Course evaluations




On metrics evaluating the course, there was relative weakness in three areas. These areas were whether the learning outcomes helped the students focus in the course, the fairness of testing and evaluation procedures, and sufficient time being provided to complete assignments. Areas of relative strength were the value of the learning experience, the clarity of the syllabus, and the assignments being appropriate.


How students interpret the underperforming prompt, "The student learning outcomes helped me focus in this course." is not clear to this author. From the student's point of view their focus is on getting assignments done, successfully answering questions on a tests, and completing other work as assigned in a course. How the student learning outcomes would help a student focus on these specific tasks seems unclear. This metric has consistently underperformed which may be an indication that the prompt is asking something that does not necessarily make sense from a student's point of view as to what they need to focus upon.

  • The testing and evaluation procedures were fair.
  • There was enough time to finish assignments.

The other two prompts (seen above) that strongly underperformed the course section mean are also areas open to interpretation. Bear in mind that the averages are between "Strongly agree" and "Agree" Only the mean value for these two metrics relative to the section mean is low. Students are not saying tests are unfair and there isn't enough time to finish assignments. They are simply rating these two metrics lower than the other metrics. The difficulty is that students tend to answer all prompts with a 4 or 5. Distinguishing meaning is challenging when the range is that small.

Course materials evaluations



In the course materials section the lowest mean rating was to textbook accessibility along with whether the textbook is appropriate to the course content.


Textbook accessibility and appropriateness has been an area of weakness for the past four terms, and the mean rating for textbook accessibility shows a downward trend. Ease of access to textbooks remains a challenge for the students and the institution.



In terms of materials, the area of greatest relative strength was the Canvas learning platform. Canvas has been an area of strength in past terms as well. Students react positively to Canvas. I have to suspect that if one differentiated between well set up courses and poorly set up courses, there would be an even stronger positive reaction to the well set up courses. 

Other metrics 


Student evaluation responses have come in from all five campuses that participated in the student evaluations.



Based on the submitted evaluations, the college continues to have just over a third of the student seats in online courses. 



As seen in prior evaluations, students still prefer in class lectures. As in the past, textbooks came in second as the preferred learning material, with online presentations edging out videos. The area marked "Online" is truncated: those are students who prefer online synchronous videoconferencing as a learning mode, such as Zoom or Google Meet. The take-away is that (after lecture as a mode) students prefer textbooks, hence textbook accessibility is important. Students also favor presentations they can view or review on their own schedule.



Looking at material preference by course type, the students in online courses still prefer "in class lectures" even though they are not in a residential course. Among online students, online presentations outperform online video and the textbook as a preferred learning material. This pattern has been seen in prior terms, leading this author to supplement the course textbook and videos with presentations containing the same material. Courses benefit from providing more than one mode of course material. 




Students in online courses strongly prefer asynchronous online courses to synchronous online courses. One of the strengths of online education is providing learning opportunities to students on a schedule that the student sets, and this happens best in an asynchronous online course. 

Comments

Post a Comment

Popular posts from this blog

Moodle bulk grading of manually marked quiz and test questions

Image
Manual marking of an essay type question on a quiz or test in Moodle can be done student by student but that process involves a lot of clicking and scrolling as one has to open each student's submission. There is a bulk marking screen available. In the test or quiz click on Results. At the upper left side is a stealth drop down list menu under the word "Grades" that includes the option "Manual grading." Choose manual grading. On the far right click on the blue "grade all" hyperlink. That will open up a all of the submissions for question one on a single page. One can now mark all of the question one submissions manually for all students in the course. 
Read more

Experimenting with the PlantNet and iNaturalist apps

Image
The application of image recognition technologies as the back end to botany apps is providing increasingly relevant search results. Over the recent winter break I began looking at whether the apps now available might be useful to the ethnobotany class here on Pohnpei. Some apps had a more regional focus in the past and were better at north american or european plants. Pohnpei has a both unique indigenous plants and many introduced plants from around the world -  a challenging environment for any image recognition system. A student working with a plant identification app for the first time My criterion was that the app had to be appropriate for use in my classroom. The app could not contain advertising and the app had to be free, my students typically do not have the ability to pay for an app. The app would also have to be fairly small as my students often use phones with limited memory and processing power. I settled on PlantNet and was impressed with the ability of the a...
Read more

From Canvas to Moodle

Image
Migration  There is no option to migrate from Instructure Canvas to Moodle by exporting courses from Canvas and importing those courses into Moodle. Canvas exports courses as Instructional Management Systems Common Cartridge version 1.1 files. Moodle only imports IMS Common Cartridge 1.0 files. The versions are incompatible and as far as this author is aware there are no converters between the versions.  IMSCC 1.0 is a 2008 standard and IMSCC 1.1 dates back to 2011 or earlier. In other words, both versions are out of date. The current version is IMSCC 1.3 and IMSCC 1.4 is under development. Neither Instructure nor Moodle appear to be working on code to handle any newer IMS CC version. The inability to easily move courses between learning management systems benefits vendor lock in.  As a result, migration is a heavy lift manual process of rebuilding element-by-element from a blank sheet. Or one can pay a third party significant sums of money to handle the migration. T...
Read more