A New Twist on End-of-Semester Evaluations

Just about a month ago the administration shared a Faculty Focus article titled A New Twist on End-of-Semester Evaluations. The article suggests shifting from summative scalar evaluations such as are used at the college at present (Was the instructor on time most of the time, some of the time, rarely, never...) to exploring the course experience for the students. The evaluation produces a qualitative result rather than a quantitative result. Qualitative data does not reduce down to an average or median score, and the answers to instructional quality questions are embedded in the details of the individual answers. This takes more reading time than "Instructor Sigrah has an average of 4.5 on the year end evaluation." Yet in the details are a richer and more nuanced view of the course, the material, the instructor, and the students.

Exploring whether longer lines of stationary marbles affect the number ejected

The eleven prompts are used as headers below, the students answers were transcribed as written. The course being evaluated was SC 130 Physical Science with laboratory taught by me. Many of the prompts included a "... because..." structure which is reflected in the format below. 

It most helped my learning of the content when...

• it comes to a test ... because... it is too struggle because of lack of ability
• things were experiments and I participated in it myself ... because... it helped me see what was taught and picture it out and it is easier to understand when I actually see things happened they way they really are.
• was the lab reports ... because... the labs are fun, interesting, and I enjoyed them.
• we do hands on learning ... because... I learn better visually than just listening and comprehending
• we watched the movies... because... it reminds me who came up with the laws

It would have helped my learning of the content if... we do more outdoor labs.

The activity that contributed the most to my learning was...

• our lab classes. I learned many things in our lab class which are very interesting. I want to learn more about there.
• the assignment having to with electricity. This is ... because... it was interesting. I won't get electrocuted any time soon.
• the color part... because... I love colors
• the lab reports I did from the beginning of this semester till the last report I did. Doing the activities/experiments hands on was easier than reporting them in words. But from the reports, it helped me on the details. So the reports were the most contributor to my learning and understanding for this course.

The activity that contributed the most to my learning was...

• for me was done the findings of velocity and acceleration of a Ripstick 
• the floral litmus lab ... because... this lab can be used in any kind of environment. This lab is great for a classroom that I am planning teach. 
• the lab we have experiment with the flowers ... because... it helps me recognize which chemical is acid or base. 
• the lectures and hands on experiments most helped my learning in this course. The lectures were not as boring as in other classes and the experiments we did outdoors and indoors did improved my learning.
• the one on electricity. We did something where we figured out how much power household appliances made and how much it cost to keep them running all day. That was helpful to know ... because... now I know what to turn off in order to lower my electricity bills.
• the rolling marble on the RipStik. It helped me learn a lot and expand my learning. I learn that we can roll a marble on a RipStik.
• throwing the rings ... because... I learned a lot and also fun
• when we form intro groups and uses the GPS to locate where you are in the campus that was the funniest and most exciting day of the semester. 

The approach I took to my own learning that contributed the most for me was... because... [no one chose to answer this prompt]

The biggest obstacle for me in my learning the material was...

• I now know how to construct my own lab report and how to make create tables along with its graph to support my data. I know how to find slopes and intercepts and some stuff that might help me in future abroad.
• mostly finding the slope when it was graphed  ... because... it was more confusing when I see dots scattered on the graph, especially when a graph isn't a linear graph.
• the colors, like for example the 132 laboratory 13, ... because... I know that all of us that took this course only know some of the colors so it is really hard trying to figure out the names of the colors. It is irritating knowing what you will do but a simple thing can destroy all your work, like finding out what color of the two colors you combine. 
• the periodic table of elements. I do study this in high school but I didn't learned it well. It kind of hard to me and I was kind of happy ... because...  I will not become a teacher some day.
• the science itself. I have a hard time understanding how it works due to the terms that are fancy but hard to understand. It always show on the analysis part of the lab reports, ... because... I have to struggle trying to relate my understanding stating in a level or way you can understand it scientifically. Or better say, I have to learn the "English" word.
• the videos... because... I usually fall asleep wherever I watched movie. I also could not hear clearly what they are saying in the videos.
• when we do the elements. I knew some of the names and but I knew all the atomic numbers. But what I don't know about the elements are that I don't remember how to counts the number of protons and neutrons. I still remember the electrons, but the rest I forgot.

A resource I know about that you might consider using is... because... [no one chose to answer this prompt]

I was most willing to take risks with learning new material when...

• doing the labs ... because... I am a curious person
• math was involved. When we had math equations to use or look for is when I was more into the learning. That's ... because... it then became a game or a puzzle I had to figure out. How the math was related to the science and why. That's what I liked.
• we have the last lab which is yesterday ... because... I am very scared of things falling on my face, then I take the risk of doing the juggle with 3 balls.

During the first day, I remember thinking...

• about failing this class ... because... when I looked around I did not see any familiar faces. Another reason I was thinking that is because this was my first time with this instructor and I have no idea how he would be. All those negative thoughts came to mind. Despite all the negativity, I have my number one goal is to do better and then actually pass the course. I'm thankful that I've learned our instructor is a great one that always encourage us to work together when we suppose to do. Finally now, I could have sensed that I'm not failing and I'm good with the classmates as well as the instructor. 
• man, am I going to pass this class? This is ... because... the class looked and felt hard.
• of the first experiment on density of soap ... because... it was the first time I did the lab on my own hands and struggled in this course
• that I I was going to fail this and now I think I will fail this class
• that I would not pass. This class ... because... I do not want anything to do with science. As time went by the content of the matter made me want to learn more and study about science. 
• that this class will be very easy, however I was wrong. The things that make this class hard and challenging for me were the lab reports. I had a hard time some of the lab reports, but I wouldn't complain ... because... I learned a lot from this class and the reports.
• that your class is hard but, no, it is size.
• this is going to be a fun class ... because... I had a fun teacher and enjoyed the way he taught. He made science fun, though he  did tell us we would be turning in a lab report every week. That was the only "not fun" part. But its college and its nothing compared to what universities expect our of you in a week.
• using the soaps and putting it in the water. If the soap is less than one, then it float and if its greater than one it will sink.

What I think I will remember five years from now is...

• doing the acid and base ... because... it was fun
• how to translate and interpret my ideas through mathematical graphs ... because... we did a lot of the lab reports.
• learning something new almost everywhere on national campus. Not just in class that we learn but also outside of class. It was tiring but worth it. Also taking a quiz outside the gym was odd but year in Dana's class everywhere students can learn. Which is also good for some students that always fall asleep in class. 
• Ohm's law, which is 112 laboratory 12 ... because... we try to bring electricity with battery and a wire. It is really useful to us because nowadays we always have power outage, imagine five years from now, it will be worse than this. Therefore, this is what I will remember.
• that I will remember the numbers of the planets and many other things. Like the dropping of the balls or the GPS activity which is like hide and seek.
• that pure water can barely pass electricity  ... because... salt is the element that conducts the pass of electricity
• the acid and base lab ... because... I enjoyed it. It was interesting how each flower changed color when mixed with those common household chemicals.
• the calculation of experimental error, slope, and the lab report
• the flower petal fluid. Ive just learned in the class that flowers do change colors if we apply acid and base to it,
• the lab reports on measuring the distance, calculating the problems for density, velocity, and acceleration... because... most of my time I will be calculating problems, walking from one distance to another.
• the last day of lab when we use balls for catching ... because... [student name] don't know how to catch the two balls.
• the material that I will remember and will be very useful for my future class are the floral litmus lab and measuring the distance of the planets from a made-up sun

What is something covered in this course material that you can do now that you could not do or did not fully understand at the beginning of the term?

• Before I took this class I literally had  no idea on how to write a lab report let alone what to write in a laboratory report. Now I fully understand what to include in a lab report and the format.
• I never really knew how to use a GPS recorder or understand formulas and such.
• I've learned many and various formulas of converting or measure distance, speed, velocity, and the others. I can now refer back to what we have learned and able to solve them mathematically. I'm also grateful to have learned that there's always some kinds of errors with almost every of those experimental we did either mathematically or scientifically. Even those there's errors, we at least get what was in relation to the ideal notion of the our findings. Overall, I could suggest that every single lab we have done, have taught me so much even though I always submit them late. I surely am now able to make a good lab report thanks to this course. Last, but not least, I have gained life survival awareness of the power, energy, and all the stuff we use for our needs. 
• Using the GPS and find where I am and where will I go according to what I have in the GPS. It is helpful ... because... I really do not know how to use that thing, but now, because of this course, I can use and read it at the same time understanding what is the GPS.
• When we talked about the conductors and insulators. Now if I want to know if something is a good conductor of heat, I know what to conduct to answer my confusion and to deal with insulators it is just the same process to find out.

From the course experience statements a richer picture of the course emerges. What was learned and, to use a word now banned in education, what was understood. As the instructor I gain useful insights into where students struggled and where they found joy. An outsider can detect that the course centers on laboratories with laboratory reports, and the core role mathematical models play in the course. 

Word cloud of the course experience statements

I chose to pilot the prompts only in a single course with thirty students. The prompts were handed out as an optional take-home exercise. Eighteen students returned answers to the prompts. The results are most useful to the individual instructor. There are clearly questions of whether this would scale efficiently. I opted not to try this in my eighty to ninety student statistics course. Even at 18 students there was a lot of data to transcribe and consider. If a college tried this system-wide in all sections there would be issues of potential data overload. Summative reports using numeric values starts to appear very attractive to an already overburdened administration. 

Perhaps one cost and time effective way to deploy qualitative course experience tools is to let individual faculty deploy the prompts as I did, and the faculty can then read the responses and use those to inform the development of their course and their instruction. 

The statements generated by my students share some common themes, perhaps after the first few "manual" runs a set of more generic statements could be derived. These statements could then be used in a more traditional Likert survey where students could rate their level of agreement or disagreement with these generic statements. An on line survey tool could then deliver the survey to the students. Each course might have to develop its own generic statements, and those should be based on actual statements made by students. Each term an open ended prompt could be included to fish for new statements to add to the collection. 

My thanks to Faculty Focus and to the administration for engendering this exploration.