Assessing Learning in Physical Science
SC 130 Physical Science proposes to serve two institutional learning outcomes (ILO) through four general education program learning outcomes (GE PLO) addressed by four course level student learning outcomes (CLO). This report assesses learning under the course level learning outcomes which in turn support program and institutional learning outcomes.
Note that this course has a focus on "doing" science, on science as a process, a way of understanding the natural physical world and the mathematics that underlies many physical systems. The course does not focus on memorized facts. The course is centered on science as being that which can be measured, observed, evidenced. The course is intended as a counter to memorized science. Once one shifts to memorized facts as the basis of a science, then any set of memorized facts can be seen by the learner as equally valid. Somewhere down at the bottom of that slope are those who are convinced the earth is flat, climate change is not happening, evolution does not explain the diversity of life, and no one actually walked on the moon. By doing simple experiments that seek to measure physical properties and quantities, by gathering and analyzing data, the intent is that students come to see science as a way of thinking about and analyzing the world around them.
The course directly supports institutional learning outcome eight and three general education outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
The course also supports institutional learning outcome two.
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
This term laboratory fourteen was assessed. In laboratory fourteen the students were told to write a report on the relationship between speed and distance for a flying disk.
In previous laboratories the students were guided by templates with headers for the introduction, equipment list, procedure, data table, data graph, numeric analysis, and discussion of results. For laboratory fourteen the students had only a blank document. This term I did no introduction and did not assert that zero launch velocity should result in zero distance. The data gathered did not support a linear regression with a zero y-intercept.
Laboratory fourteen was assessed to determine whether students explored physical science systems through experimentally based laboratories using scientific methodologies. This was measured by assessing whether students properly recorded data in a table, generated labelled xy scatter graphs, analyzed and reported the slope of the linear regression to their data, and meaningfully discussed the results.
Of 19 students in the course, 13 submitted laboratory nine (68%).
This outcome is assessed by the students exceeding 70% on five or more assessments of this learning outcome. This term there were only nine opportunities for students to demonstrate mastery at least five times. This was drop in assessments of this learning outcome from 13 opportunities spring 2019.
Performance on course learning outcome two was negatively impacted by the reduced number of assessments that occurred.
Course learning outcome two achievement
Only six of the 19 students exceeded 70% on five or more assessments of course learning outcome two. Improving achievement on this course learning outcome will mean increasing the opportunities to measure this outcome during the spring 2020 term.
The focus in physical science is on science as a process, as a system of experimentally generated and verified knowledge, not as a collection of memorized facts. The course is not content free, but the heavier emphasis is on data gathering, fitting mathematical models, and writing up results in reports. This serves the twin intents of the course to put mathematical models and writing skills at center of the course. The course works with simple mathematical models that describe how a system will behave. Later in the term the students encounter general relativity and quantum mechanics from a descriptive point of view. The course cannot tackle the mathematics of these subjects which make predictions about the nature of reality. The students can see that math works to describe the simpler systems in the laboratory, comprehend the unreasonable effectiveness of mathematics in describing nature, and perhaps wrestle with the ideas coming from mathematics more complex than can be tackled in the course.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, an identical pre-assessment and post-assessment is included in the course. Neither the pre-assessment nor the post-assessment are announced in advance. The pre-assessment was done on the first day of class. The post-assessment was given on the penultimate day of class. The intent is to gauge what the students can do without having specifically studied.
Students demonstrated a statistically significant improvement from the pre-assessment to the post-assessment. The effect size was small. Students showed a small improvement in their ability to generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990's assessment data at the college suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
Course level student learning outcome four was directly evaluated up to eight times on rubrics used to mark odd numbered laboratory reports.
Of the 19 students in the course, 14 students demonstrated mastery five or more times on the eight times that this outcome was measured.
Given that the course is not a major requirement, one of the intents is to reach out to students who may not enjoy core science courses and show them science in a new light.
At the end of the term the students were asked to respond to the following questions:
1. Before I took this class my attitude towards science was:
☐ Positive: I liked science
☐ Neutral
☐ Negative: I did not like science
2. During the semester:
☐ I enjoyed physical science class
☐ Neutral
☐ I did not enjoy physical science class
3. After taking this class my attitude towards science is:
☐ Positive: I like science
☐ Neutral
☐ Negative: I do not like science
Prior to the term 67% of the students self-reported attitudes towards science that were neutral or negative.
During the course 100% of the students surveyed responded that they enjoyed the course. At term end 80% of the students had a positive attitude towards science.
As an instructor who has a passion for science, I would hope to encourage my students to also enjoy that passion. New scientists are not minted from students who dislike science and come away from the course with a negative attitude towards science. That the students enjoyed the course does not mean the course was easy, not with lab reports to be done each and every week. A demanding course can also be a positive experience, perhaps even enjoyable, perhaps fun. This term was unusual in that all of the students said they enjoyed the course during the term.
Overall, students who successfully completed the course are able to explore physical science systems through experimentally based laboratories using scientific methodologies; define and explain concepts, theories, and laws in physical science; generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science; and demonstrate basic written communication skills.
Note that this course has a focus on "doing" science, on science as a process, a way of understanding the natural physical world and the mathematics that underlies many physical systems. The course does not focus on memorized facts. The course is centered on science as being that which can be measured, observed, evidenced. The course is intended as a counter to memorized science. Once one shifts to memorized facts as the basis of a science, then any set of memorized facts can be seen by the learner as equally valid. Somewhere down at the bottom of that slope are those who are convinced the earth is flat, climate change is not happening, evolution does not explain the diversity of life, and no one actually walked on the moon. By doing simple experiments that seek to measure physical properties and quantities, by gathering and analyzing data, the intent is that students come to see science as a way of thinking about and analyzing the world around them.
The course directly supports institutional learning outcome eight and three general education outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
| GE PLO | SC 130 CLO |
|---|---|
| 3.5 Perform experiments that use scientific methods as part of the inquiry process. | 1. Explore physical science systems through experimentally based laboratories using scientific methodologies |
| 3.4 Define and explain scientific concepts, principles, and theories of a field of science. | 2. Define and explain concepts, theories, and laws in physical science. |
| 3.2 Present and interpret numeric information in graphic forms. | 3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science. |
Susan and Mayboleen measure the acceleration of gravity
The course also supports institutional learning outcome two.
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
| GE PLO | SC 130 CLO |
|---|---|
| 1.1 Write a clear, well-organized paper using documentation and quantitative tools when appropriate. | 4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software. |
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologiesThis term laboratory fourteen was assessed. In laboratory fourteen the students were told to write a report on the relationship between speed and distance for a flying disk.
Susan, Staisy, and Kimmy on the throwing line in light rain at 11:00
In previous laboratories the students were guided by templates with headers for the introduction, equipment list, procedure, data table, data graph, numeric analysis, and discussion of results. For laboratory fourteen the students had only a blank document. This term I did no introduction and did not assert that zero launch velocity should result in zero distance. The data gathered did not support a linear regression with a zero y-intercept.
Laboratory fourteen was assessed to determine whether students explored physical science systems through experimentally based laboratories using scientific methodologies. This was measured by assessing whether students properly recorded data in a table, generated labelled xy scatter graphs, analyzed and reported the slope of the linear regression to their data, and meaningfully discussed the results.
Of 19 students in the course, 13 submitted laboratory nine (68%).
Analysis of laboratory fourteen
Of the 13 students who submitted a report for laboratory fourteen, all 13 produced laboratory reports with data recorded in a properly formatted table. Thirteen students also generated a correct xy scattergraph. Ten students used Desmos to generate a mathematical model that they felt best fitted their data. Eight students were able to discuss their choice of a mathematical model and explain what the model was telling them about the system.
Performance on CLO 1 for laboratory reports across multiple terms
Relative performance on the individual criterion has remained generally stable on this rubric over the past six terms. About one third of the students did not turn in this laboratory report. During the term there are 12 laboratory reports due in, the percent of students submitting falls during the course of the term. This loss in the submission rate has remain resistant to improvement. Of the 13 students who did submit the report, all 13 students included a data table and an appropriate xy scattergraph. Ten students reported a mathematical model for the system and eight students were able to discuss their resulting model in a reasonably articulate manner.
Laboratory submission rates spring 2019 versus long term average rate since fall 2014
As noted above, the percent of students submitting reports falls during the term, with the above chart reflecting this in terms of the laboratory number. Admission rates start off above 80% and generally fall to around 70%. This term was no exception and differences from the longer term average submission rate for a given laboratory were not significant.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.This outcome is assessed by the students exceeding 70% on five or more assessments of this learning outcome. This term there were only nine opportunities for students to demonstrate mastery at least five times. This was drop in assessments of this learning outcome from 13 opportunities spring 2019.
Performance on course learning outcome two was negatively impacted by the reduced number of assessments that occurred.
Only six of the 19 students exceeded 70% on five or more assessments of course learning outcome two. Improving achievement on this course learning outcome will mean increasing the opportunities to measure this outcome during the spring 2020 term.
The focus in physical science is on science as a process, as a system of experimentally generated and verified knowledge, not as a collection of memorized facts. The course is not content free, but the heavier emphasis is on data gathering, fitting mathematical models, and writing up results in reports. This serves the twin intents of the course to put mathematical models and writing skills at center of the course. The course works with simple mathematical models that describe how a system will behave. Later in the term the students encounter general relativity and quantum mechanics from a descriptive point of view. The course cannot tackle the mathematics of these subjects which make predictions about the nature of reality. The students can see that math works to describe the simpler systems in the laboratory, comprehend the unreasonable effectiveness of mathematics in describing nature, and perhaps wrestle with the ideas coming from mathematics more complex than can be tackled in the course.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, an identical pre-assessment and post-assessment is included in the course. Neither the pre-assessment nor the post-assessment are announced in advance. The pre-assessment was done on the first day of class. The post-assessment was given on the penultimate day of class. The intent is to gauge what the students can do without having specifically studied.
Pre-assessment and post-assessment average correct
Students demonstrated a statistically significant improvement from the pre-assessment to the post-assessment. The effect size was small. Students showed a small improvement in their ability to generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.Course level learning outcome four focuses on communication, specifically writing. In the late 1990's assessment data at the college suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
Laboratory report in Google Docs
Course level student learning outcome four was directly evaluated up to eight times on rubrics used to mark odd numbered laboratory reports.
Five or more demonstrations of student learning outcome four above 70%
Affective Domain Assessment
SC 130 Physical Science functions as a science with laboratory requirement for the general education core at the college. Only one student, an electronics technology program student, was studying in a field based in physical science.Given that the course is not a major requirement, one of the intents is to reach out to students who may not enjoy core science courses and show them science in a new light.
At the end of the term the students were asked to respond to the following questions:
1. Before I took this class my attitude towards science was:
☐ Positive: I liked science
☐ Neutral
☐ Negative: I did not like science
2. During the semester:
☐ I enjoyed physical science class
☐ Neutral
☐ I did not enjoy physical science class
3. After taking this class my attitude towards science is:
☐ Positive: I like science
☐ Neutral
☐ Negative: I do not like science
Student attitudes towards science, 23 students completed the survey
As an instructor who has a passion for science, I would hope to encourage my students to also enjoy that passion. New scientists are not minted from students who dislike science and come away from the course with a negative attitude towards science. That the students enjoyed the course does not mean the course was easy, not with lab reports to be done each and every week. A demanding course can also be a positive experience, perhaps even enjoyable, perhaps fun. This term was unusual in that all of the students said they enjoyed the course during the term.
Overall, students who successfully completed the course are able to explore physical science systems through experimentally based laboratories using scientific methodologies; define and explain concepts, theories, and laws in physical science; generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science; and demonstrate basic written communication skills.
Comments
Post a Comment