Reflections on a summer session
The following is a look at the student and faculty evaluations from the summer online session.
The chart uses standardized mean values (also known as z-scores) to provide some insight into course strengths and weaknesses. A score of zero would mean that the question performed at the overall mean value. The x-axis is in standard deviations. Values beyond ±2 are considered unusual.
The areas of relative strength were that courses were perceived to be a valuable learning experience and that course syllabi were clear and complete. None of these were unusually high.
The area of strongest weakness was in perceived value of the online course versus the residential version of the course. This was unusually weak against the other averages.
There were over 400 comments made by students on their courses. By and large the comments were positive and supportive. Many students reported that their courses were useful, helpful, good, and beneficial. There really was an overwhelmingly positive reaction. The following were some of the few comments on areas where improvement might be sought.
Students...
Student evaluation of the course materials was most positive for the relevancy and usefulness of online materials. The weakest area was in textbook accessibility. As one student noted in a separate comment, online courses should have online textbooks.
For course materials there were over 400 comments. The comments were generally positive and supportive. Students found the materials to be useful, helpful, even perfect. Many of the less positive comments centered on connectivity, Internet bandwidth limitations, challenges in accessing online materials, and textbook availability at term start.
Students…
Faculty received the strongest agreement on their welcoming of questions and comments, this score was unusually high against the other averages. Faculty also emphasized the student learning outcomes for their courses. Although there were areas of weaker agreement, the difference from the average was not large. The area with the lowest average was that of instructor initiation of regular contact.
Student comments included words such as awesome, best, always positive, helpful, nice, good, and even great. Student responses were overwhelmingly positive.
Students...
In the closing question of the survey students were asked if they had other comments on their experience. Again, responses were very positive. Students did have some suggestions, some of which echoed earlier recommendations. Time is larger than the other words in the above cloud because time, as in "not enough time" was a recurring theme in the comments.
Students…
In reading through the hundreds of comments submitted, one theme that recurred was the importance of deliberate, positive feedback. In an online environment there is a loss of so many of the nonverbal cues that a learner uses to gauge their progress. With nonverbal cues unavailable students can feel a sense of not knowing how they are doing. In a sense an instructor may have to overcompensate for the loss of these cues and use more overt direct positive feedback. The student comments indicated appreciation for positive comments, positive written feedback. To not just provide a grade, but to include a positive comment, to encourage students. To use language to encourage students.
In general, the students are willing to continue online in the fall. Those who responded negatively had the following broad reasons for not wanting to return.
Prefer physical interaction: prefers residential instruction, prefer social interactions on campus, getting one-on-one assistance from the instructor, able to ask more questions, prefer face-to-face instruction, cannot get enough information and help online, online instruction is not as active and is more tedious.
Load is too heavy: had trouble keeping up with one or two classes in summer and cannot see being able to keep up during a full load fall term
Less effective: learned less, could not concentrate at home, distracted at home, had other duties in the house that interfered with learning, covered less material, subject matter requires hands-on experiences that were not available online
Too hard: Online learning is too hard, much harder than residential instruction
Technical issues: lack a laptop, limited bandwidth, power problems, technical problems.
College is a time of forging friendships and lifelong professional and social bonds. The challenge for student life is how to build a sense of community and belonging in an online space. How do services such as recreation, student organizations, and student activities move forward on an online campus? How do students get to meet each other, greet each other, and socialize?
What worked well for faculty were factors such as communication, the support provided by counseling and information technology services. Zoom worked well for faculty, as did Schoology. Faculty reported engagement and flexibility benefits. Some reported that the Internet worked well. And a few noted that videos worked well. Student factors refers to student motivation, dedication, student technical savvy, and improved student diversity in classes. A number of instructors were impressed by the ability of students to rise to the challenge.
Instructors noted a lack of access to technology. This included faculty access to laptop computers, access to the Internet, access to hardware and software that they did not previously need. Access to technology also included comments concerning students not having access to needed technology such as a laptop.
Communication, which featured as a leading area where things went well, was also cited as an area where things did not go well. One of the areas of shared concern was the longer response time for a back and forth conversation with a student when exchanging messages - the loss of the immediacy of the classroom.
While some faculty listed engagement as something that went well, other faculty felt that student engagement was lower than for residential instruction.
Instructional support gathered comments in which faculty expressed feeling a lack of support from instructional or student support services.
Internet connectivity was the leading area in which the summer did not go well. This included bandwidth issues, student Internet access issues.
Late assignments were another problematic area. Internet connectivity, more so for students than faculty, did not go well.
Power issues, for both faculty and students, was another area that did not go well.
Schoology Conferences received a number of negative comments whereas Zoom received a number of positive comments in the earlier section. The negative comments centered on difficulty of use and slow networks, but nothing more specific. Schoology was also listed as something that did not go well for some faculty.
Student factors included comments in areas such as a lack of motivation among some students, plagiarism, poor reading comprehension, cheating, inability to access the library from off campus, and the inability to follow directions/instructions.
Textbooks, specifically textbook availability, was another area of concern among faculty.
Faculty found online teaching to be more time consuming.
Training as an item here is treated as distinct from instructional support. Where instructional support referred to support from college systems, training was the request for more training both in the technologies of distance education and the pedagogies of distance education. Distance education requires not just an ability to use technologies, but an ability to re-envision a course and a curriculum for the online environment. This author has some knowledge in the technologies that might be used for online education, but is no expert in best practices and course design for an online environment. Best practices will vary by subject area and the nature of the course.
Faculty are keen on receiving more training, that is the second largest area of concern. Faculty also want to see more instructional support where this refers to staff who can digitize materials for faculty, upgrading hardware and software that faculty use, and more information flow between faculty during the term.
Finding ways to improve Internet connectivity and bandwidth is also a lead concern.
Other is a broad area of suggestions including having online office hours, using Facebook for student communication, accommodating students as visual learners, finding alternatives to Zoom (this may be related to the time limit for the free version), providing instructors' manuals for teachers, finding ways to make online education more fun, plagiarism checking software, counselling sessions for students, basic skills classes for students, developing a media studies class in which students would learn to read, write, and speak, turning the generator on in the morning and leaving it on, using software that works better than Schoology, using eBooks.
Two areas that came in at 8% of the responses each were to provide technology to instructors and to provide technology to students. For instructors this list includes laptops, cameras, tripods, microphones, ADSL, and Zoom licenses. For providing tech to students the focus was on ensuring students have laptop computers and access to the Internet.
Some suggestions focused on reducing faculty load and/or improving faculty compensation.
Six percent of the comments suggested returning to residential instruction.
The leading area for suggestions by faculty was for student training and ongoing student tech support during the term. In the lead were recommendations for Schoology training for students. Some faculty noted that training students in the use of Schoology fell to them during the summer, a role for which they were unprepared and did not feel was their responsibility. Faculty felt that students should receive training before starting their online class.
Resolving the textbook availability issue, especially at term start, once again appeared in the comments.
The author thanks the Interim President and the Department of Instructional Affairs for allowing him the opportunity to work with the data. All opinions, errors, and interpretations are strictly those of the author and the author alone.
Student evaluations and responses
Course evaluation
Course evaluation
The chart uses standardized mean values (also known as z-scores) to provide some insight into course strengths and weaknesses. A score of zero would mean that the question performed at the overall mean value. The x-axis is in standard deviations. Values beyond ±2 are considered unusual.
The areas of relative strength were that courses were perceived to be a valuable learning experience and that course syllabi were clear and complete. None of these were unusually high.
The area of strongest weakness was in perceived value of the online course versus the residential version of the course. This was unusually weak against the other averages.
Word cloud for course comments
There were over 400 comments made by students on their courses. By and large the comments were positive and supportive. Many students reported that their courses were useful, helpful, good, and beneficial. There really was an overwhelmingly positive reaction. The following were some of the few comments on areas where improvement might be sought.
Students...
- Requested more time to complete assignments including more lead time between an assignment being posted and being due.
- Asked that faculty be sensitive to individual student needs
- Reported Internet connectivity and bandwidth limitations
- Suggested that the textbook either be online, or uploaded if not available online.
- Noted that the course should be well organized, recommending weekly or unit folders to organize materials
Course materials
Course material evaluation
Student evaluation of the course materials was most positive for the relevancy and usefulness of online materials. The weakest area was in textbook accessibility. As one student noted in a separate comment, online courses should have online textbooks.
Word cloud for course materials comments
For course materials there were over 400 comments. The comments were generally positive and supportive. Students found the materials to be useful, helpful, even perfect. Many of the less positive comments centered on connectivity, Internet bandwidth limitations, challenges in accessing online materials, and textbook availability at term start.
Students…
- Requested that the bookstores ensure they have the textbook
- Asked that readings be put online
- Noted that online texts sometimes failed to load
Instructor evaluations by students
Instructor evaluation
Faculty received the strongest agreement on their welcoming of questions and comments, this score was unusually high against the other averages. Faculty also emphasized the student learning outcomes for their courses. Although there were areas of weaker agreement, the difference from the average was not large. The area with the lowest average was that of instructor initiation of regular contact.
Word cloud for instructor evaluation comments by students
Student comments included words such as awesome, best, always positive, helpful, nice, good, and even great. Student responses were overwhelmingly positive.
Students...
- Asked for regularly and timely feedback on assignments including notes on quizzes and tests.
- Requested that faculty initiate contact on missed assignments
- Recommended avoiding changing due dates to earlier dates than those originally set when an assignment was posted
- Suggested permitting longer time spans between posting an assignment and having the assignment due, for example, an assignment posted at midnight should not be due latter that same day.
- Asked that whenever their submission was not satisfactory or received a poor grade, they be given specific feedback on why.
Other comments by students
Word cloud for other comments
In the closing question of the survey students were asked if they had other comments on their experience. Again, responses were very positive. Students did have some suggestions, some of which echoed earlier recommendations. Time is larger than the other words in the above cloud because time, as in "not enough time" was a recurring theme in the comments.
Students…
- Asked for more time to complete assignments due to connectivity issues
- Noted that the tone of a response can convey negative reinforcement
- Found that obtaining rides to campus, laptop computers, and textbooks were all difficult.
- Experienced limited bandwidth and connectivity problems
- Reported that there were times a test would not fully load, and then when they tried to access the test after the failure to load, the test was locked with the message that no more attempts were permitted. There is a setting in Schoology that permits a test to be resumable if interrupted, if that is not set, then a student cannot resume an incompletely loaded test or after a power outage.
- Asked for video explanations, noting that they are visual learners.
- Requested less videos due to bandwidth challenges.
- Asked that more computers be made available on the campuses.
In reading through the hundreds of comments submitted, one theme that recurred was the importance of deliberate, positive feedback. In an online environment there is a loss of so many of the nonverbal cues that a learner uses to gauge their progress. With nonverbal cues unavailable students can feel a sense of not knowing how they are doing. In a sense an instructor may have to overcompensate for the loss of these cues and use more overt direct positive feedback. The student comments indicated appreciation for positive comments, positive written feedback. To not just provide a grade, but to include a positive comment, to encourage students. To use language to encourage students.
Will you continue online in the fall?
In general, the students are willing to continue online in the fall. Those who responded negatively had the following broad reasons for not wanting to return.
Prefer physical interaction: prefers residential instruction, prefer social interactions on campus, getting one-on-one assistance from the instructor, able to ask more questions, prefer face-to-face instruction, cannot get enough information and help online, online instruction is not as active and is more tedious.
Load is too heavy: had trouble keeping up with one or two classes in summer and cannot see being able to keep up during a full load fall term
Less effective: learned less, could not concentrate at home, distracted at home, had other duties in the house that interfered with learning, covered less material, subject matter requires hands-on experiences that were not available online
Too hard: Online learning is too hard, much harder than residential instruction
Technical issues: lack a laptop, limited bandwidth, power problems, technical problems.
College is a time of forging friendships and lifelong professional and social bonds. The challenge for student life is how to build a sense of community and belonging in an online space. How do services such as recreation, student organizations, and student activities move forward on an online campus? How do students get to meet each other, greet each other, and socialize?
Faculty self-reflection 2020
What worked well
What worked well for faculty were factors such as communication, the support provided by counseling and information technology services. Zoom worked well for faculty, as did Schoology. Faculty reported engagement and flexibility benefits. Some reported that the Internet worked well. And a few noted that videos worked well. Student factors refers to student motivation, dedication, student technical savvy, and improved student diversity in classes. A number of instructors were impressed by the ability of students to rise to the challenge.
What did not work well
Communication, which featured as a leading area where things went well, was also cited as an area where things did not go well. One of the areas of shared concern was the longer response time for a back and forth conversation with a student when exchanging messages - the loss of the immediacy of the classroom.
While some faculty listed engagement as something that went well, other faculty felt that student engagement was lower than for residential instruction.
Instructional support gathered comments in which faculty expressed feeling a lack of support from instructional or student support services.
Internet connectivity was the leading area in which the summer did not go well. This included bandwidth issues, student Internet access issues.
Late assignments were another problematic area. Internet connectivity, more so for students than faculty, did not go well.
Power issues, for both faculty and students, was another area that did not go well.
Schoology Conferences received a number of negative comments whereas Zoom received a number of positive comments in the earlier section. The negative comments centered on difficulty of use and slow networks, but nothing more specific. Schoology was also listed as something that did not go well for some faculty.
Student factors included comments in areas such as a lack of motivation among some students, plagiarism, poor reading comprehension, cheating, inability to access the library from off campus, and the inability to follow directions/instructions.
Textbooks, specifically textbook availability, was another area of concern among faculty.
Faculty found online teaching to be more time consuming.
Training as an item here is treated as distinct from instructional support. Where instructional support referred to support from college systems, training was the request for more training both in the technologies of distance education and the pedagogies of distance education. Distance education requires not just an ability to use technologies, but an ability to re-envision a course and a curriculum for the online environment. This author has some knowledge in the technologies that might be used for online education, but is no expert in best practices and course design for an online environment. Best practices will vary by subject area and the nature of the course.
Suggestions for improvement from a faculty perspective
Faculty are keen on receiving more training, that is the second largest area of concern. Faculty also want to see more instructional support where this refers to staff who can digitize materials for faculty, upgrading hardware and software that faculty use, and more information flow between faculty during the term.
Finding ways to improve Internet connectivity and bandwidth is also a lead concern.
Other is a broad area of suggestions including having online office hours, using Facebook for student communication, accommodating students as visual learners, finding alternatives to Zoom (this may be related to the time limit for the free version), providing instructors' manuals for teachers, finding ways to make online education more fun, plagiarism checking software, counselling sessions for students, basic skills classes for students, developing a media studies class in which students would learn to read, write, and speak, turning the generator on in the morning and leaving it on, using software that works better than Schoology, using eBooks.
Two areas that came in at 8% of the responses each were to provide technology to instructors and to provide technology to students. For instructors this list includes laptops, cameras, tripods, microphones, ADSL, and Zoom licenses. For providing tech to students the focus was on ensuring students have laptop computers and access to the Internet.
Some suggestions focused on reducing faculty load and/or improving faculty compensation.
Six percent of the comments suggested returning to residential instruction.
The leading area for suggestions by faculty was for student training and ongoing student tech support during the term. In the lead were recommendations for Schoology training for students. Some faculty noted that training students in the use of Schoology fell to them during the summer, a role for which they were unprepared and did not feel was their responsibility. Faculty felt that students should receive training before starting their online class.
Resolving the textbook availability issue, especially at term start, once again appeared in the comments.
Conclusion
Overall students came away from the summer experience with positive reactions. In reading hundreds of comments the dominant message was of responsive faculty who engendered learning and students who will be returning for online classes in the fall term. Students praised the efforts that their instructors made during the summer session. Although there were issues of technology, connectivity, textbooks, and power during the summer; instructors, students, and staff worked together systemwide to find solutions. This summer there was a sense among faculty not of belonging to any one campus but of being a part of a single systemwide campus. And while the college mascot may be well known for not having evolved significantly in the last 195 million years, this summer the COM-FSM Sharks evolved, adapted, and even thrived in a new learning environment.The author thanks the Interim President and the Department of Instructional Affairs for allowing him the opportunity to work with the data. All opinions, errors, and interpretations are strictly those of the author and the author alone.
Thank You for sharing.
ReplyDelete