Assessing learning in physical science fall 2020
SC 130 Physical Science fall 2020 was the first term for the course to be taught as a hybrid online and residential course. Content that would normally have been delivered on a Monday, Wednesday, and Friday "lecture discussion" schedule was online. The three hour laboratories on Thursdays were done residentially. In keeping with national, state, and college guidelines, laboratory sections were limited to ten students each.
The course consisted of two sections of ten students each at course start. Five students were withdrawn from the course.¹ At the end of the term eight students were in section one and seven students were in section two.
SC 130 Physical Science proposes to serve two institutional learning outcomes (ILO) through four general education program learning outcomes (GE PLO) addressed by four course level student learning outcomes (CLO). This report assesses learning under the course level learning outcomes which in turn support program and institutional learning outcomes.
The course has a focus on "doing" science, on science as a process, a way of understanding the natural physical world and the mathematics that underlies many physical systems. The course does not focus on memorized facts. The course is centered on science as being that which can be measured, observed, evidenced. The course is intended as a counter to memorized science. Once one shifts to memorized facts as the basis of a science, then any set of memorized facts can be seen by the learner as equally valid. Somewhere down at the bottom of that slope are those who are convinced the earth is flat, climate change is not happening, evolution does not explain the diversity of life, and no one actually walked on the moon. By doing simple experiments that seek to measure physical properties and quantities, by gathering and analyzing data, the intent is that students come to see science as a way of thinking about and analyzing the world around them. The design intent of the course is also to demonstrate The Unreasonable Effectiveness of Mathematics in the Natural Sciences.
The course directly supports institutional learning outcome eight and three general education outcomes.
ILO 8 Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
| GE PLO | SC 130 CLO |
|---|---|
| 3.5 Perform experiments that use scientific methods as part of the inquiry process. | 1. Explore physical science systems through experimentally based laboratories using scientific methodologies |
| 3.4 Define and explain scientific concepts, principles, and theories of a field of science. | 2. Define and explain concepts, theories, and laws in physical science. |
| 3.2 Present and interpret numeric information in graphic forms. | 3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science. |
The course also supports institutional learning outcome two.
ILO 2 Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
| GE PLO | SC 130 CLO |
|---|---|
| 1.1 Write a clear, well-organized paper using documentation and quantitative tools when appropriate. | 4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software. |
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
This term laboratory fourteen was assessed. In laboratory fourteen the students were asked to explore the relationship, if any, between the length of a pendulum and the period of a pendulum. The students were also asked to explore whether the weight affected the period.
In previous laboratories the students were guided by templates with headers for the introduction, equipment list, procedure, data table, data graph, numeric analysis, and discussion of results. For laboratory fourteen the students had only a blank document.
Laboratory fourteen was assessed to determine whether students explored physical science systems through experimentally based laboratories using scientific methodologies. This was measured by assessing whether students properly recorded data in a table, generated labelled xy scatter graphs, analyzed and reported the slope of the linear regression to their data, and meaningfully discussed the results.
Of the 15 students in the course, only nine submitted laboratory fourteen (60%). This represented a drop year-on-year.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
The concepts, theories, and laws of physical science were primarily delivered by videos which replaced the residential lecture-discussion sections. Twenty-nine physical science learning support videos were produced. Material delivered via video was included in quiz questions in a weekly quiz delivered online in the Schoology learning management system.
Fourteen of the fifteen students successfully demonstrated mastery of course learning outcome two by obtaining a score of 70% or more on CLO2 material at least five times. This performance derived from online quizzes on the material presented in the content videos. Fall 2020 was the first term that online, unproctored quizzes were used to measure CLO2 thus measures across terms are not meaningful.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
In the past course learning outcome three was evaluated using a proctored pre-assessment at term start and a proctored post-assessment at term end. This term an unproctored online mathematics pre-assessment showed that fundamental skills in calculating a slope, intercept, or identifying the correct line for a set of data were lacking. While 89% of the students could correctly identify a coordinate on an xy scatter graph, the average for the remaining questions was 50%.
This term, however, there was no post-assessment to look at the impact of the course on mathematical skills. In addition, an end of term analysis showed that the Schoology learning management system failed to track student learning outcome mastery measured by learning outcome linked criteria on rubrics. The underlying cause of this failure is not know. The rubrics with the criteria had been copied forward from a prior term and perhaps in this process the rubric criteria became delinked from the student learning outcomes. The failure had not occurred before, but the institution is presently exploring leaving the Schoology platform. This makes working on resolving this issue a low priority item. This term CLO3 cannot be reported upon in a meaningful manner other than to note that four of the nine students who submitted laboratory fourteen used appropriate mathematical techniques and concepts to obtain a quantitative solution to the pendulum system. This suggests a low rate of success on this learning outcome.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990's assessment data at the college suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheets and word processing software. The use of spreadsheets would be replaced by the use of Desmos to handle graphing and analysis of data.
For the purposes of course learning outcome four, laboratory thirteen was analyzed as 13 of the 15 students submitted this laboratory. 13 of the 15 students (87%) produced a laboratory report that demonstrated basic communication skills. Two students at term end were still struggling to produce written communication that demonstrated mastery of this learning outcome.
Overall, students who successfully completed the course are able to explore physical science systems through experimentally based laboratories using scientific methodologies; define and explain concepts, theories, and laws in physical science; generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science; and demonstrate basic written communication skills.
Of concern is that the course lost 25% of the students on the way to the end of the term. This is the highest loss rate in the history of the course. Of the 15 students still enrolled at term end, one student would fail to pass and a second passed with D. The loss and failure rate suggest that this is a challenging course even in a hybrid format. This casts doubt on whether the course could ever move purely online and see a reasonable success rate for students.
¹ One student never attended the course and submitted no work. A meeting with the student determined that student intended to take the class spring 2021. A second student attended the laboratory section once, submitted no work, and was withdrawn for non-participation. A third student attended three laboratory sections early in the term, submitted two homework from the first two weeks of class, and then no longer attended the course nor submitted further work. A fourth student withdrew to accompany the remains of his grandfather on a ship returning the body to the grandfather's home island. The fifth student submitted no work and attended the laboratory section only irregularly. In a discussion in the tenth week the student said that their not having done any of the assignments was their own fault. They had the ability and the access to technology to have completed the assignments, and had not worked on any of the assignments. Based on the challenge of making up ten weeks worth of work, the student concurred with the withdrawal.
Comments
Post a Comment