Assessing Learning in Physical Science
SC 130 Physical Science proposes to serve two institutional learning outcomes (ILO) through four general education program learning outcomes (GE PLO) addressed by four course level student learning outcomes (CLO). This report assesses learning under the course level learning outcomes which in turn support program and institutional learning outcomes.
Note that this course has a focus on "doing" science, on science as a process, a way of understanding the natural physical world and the mathematics that underlies many physical systems. The course does not focus on memorized facts. The course is centered on science as being that which can be measured, observed, evidenced. The course is intended as a counter to memorized science. Once one shifts to memorized facts as the basis of a science, then any set of memorized facts can be seen by the learner as equally valid. Somewhere down at the bottom of that slope are those who are convinced the earth is flat, climate change is not happening, evolution does not explain the diversity of life, and no one actually walked on the moon. By doing simple experiments that seek to measure physical properties and quantities, by gathering and analyzing data, the intent is that students come to see science as a way of thinking about and analyzing the world around them.
The course directly supports institutional learning outcome eight and three general education outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
Laboratory nine was assessed to determine whether students explored physical science systems through experimentally based laboratories using scientific methodologies. This was measured by assessing whether students properly recorded data in a table, generated labelled xy scatter graphs, analyzed and reported the slope of the linear regression to their data, and meaningfully discussed the results.
Of 25 students in the course, 15 submitted laboratory nine (60%). Ten failed to submit a report (40%).
This outcome is assessed by the students exceeding 70% on five or more assessments of this learning outcome. This term there were thirteen opportunities for students to demonstrate mastery at least five times.
Performance on course learning outcome two was mixed this term.
Course learning outcome two achievement
Twelve of the twenty-five students exceeded 70% on five or more assessments of course learning outcome two.
The focus in physical science is on science as a process, as a system of experimentally generated and verified knowledge, not as a collection of memorized facts. The course is not content free, but the heavier emphasis is on data gathering, fitting mathematical models, and writing up results in reports. Once science becomes a collection of memorized and regurgitated factoids, then all collections of memorized and regurgitated factoids are equally valid. At that point one is left choosing among factoids to "believe" in and the result are those who "do not believe in science" whether that science is climate change, evolution, or any other area of science. Thus the course focuses on knowledge generated by the students and is guided in part by the concepts of non-overlapping magisteria and a constructivist epistemology.
Course learning outcome two is measured primarily by tests and quizzes. Performance on these items has been weaker over the terms, a likely result of the focus on the science as a process, on the laboratories and reports.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, an identical pre-assessment and post-assessment is included in the course. Neither the pre-assessment nor the post-assessment are announced in advance. The pre-assessment was done on the first day of class. The post-assessment was given on the penultimate day of class. The intent is to gauge what the students can do without having specifically studied.
Students demonstrated a statistically significant improvement from the pre-assessment to the post-assessment. The effect size was medium. Students showed a moderate improvement in their ability to generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990's assessment data at the college suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
This term the adoption of Schoology Institutional provided access to the Google Drive Assignments app. An earlier article covered the use of Google Drive Assignments in the course.
Course level student learning outcome four was directly evaluated up to seven times on rubrics used to mark odd numbered laboratory reports.
Of the 25 students in the course, 10 students demonstrated mastery five or more times where mastery was defined as exceeding 70%. Although only ten students demonstrated five or more times, 22 of the 25 students exceeded 70% at least once. This student learning outcome is being directly impacted by the low turn-in rate for the laboratory reports. With only seven laboratory reports marked for grammar, vocabulary, organization, and cohesion, there is a greater probability that a student will not achieve the five demonstrations of mastery above 70%. The number of times an outcome must be demonstrated cannot be set differently for different learning outcomes. While five makes sense for an outcome measured a dozen or more times, for an outcome measured only seven times, there is a greater risk that five demonstrations will not occur.
Given that the course is not a major requirement, one of the intents is to reach out to students who may not enjoy core science courses and show them science in a new light.
At the end of the term the students were asked to respond to the following questions:
1. Before I took this class my attitude towards science was:
☐ Positive: I liked science
☐ Neutral
☐ Negative: I did not like science
2. During the semester:
☐ I enjoyed physical science class
☐ Neutral
☐ I did not enjoy physical science class
3. During the semester:
☐ I was glad I signed up for physical science class
☐ Neutral
☐ I was not glad I signed up for physical science class
4. After taking this class my attitude towards science is:
☐ Positive: I like science
☐ Neutral
☐ Negative: I do not like science
Prior to the term 71% of the students self-reported attitudes towards science that were neutral or negative.
During the course 88% of the students surveyed responded that they enjoyed the course, 82% responded that they were glad they signed up for physical science. At term end 82% of the students had a positive attitude towards science.
As an instructor who has a passion for science, I would hope to encourage my students to also enjoy that passion. New scientists are not minted from students who dislike science and come away from the course with a negative attitude towards science. That the students enjoyed the course does not mean the course was easy, not with lab reports to be done each and every week. A demanding course can also be a positive experience, perhaps even enjoyable, perhaps fun.
All 15 respondents unanimously responded that there was nothing they disliked about Desmos.
A favorite laboratory was one that was fun and enjoyable, presented new information that the student did not previously know, was useful to the student, or was the first time for them to engage in a particular activity.
Students disliked a least favorite laboratory when the laboratory was confusing, difficult to understand, or perceived to be hard. Students also, curiously enough, disliked laboratories for which they were absent. This appears to be more of an expression of disliking that they missed the laboratory than a dislike of the content of the laboratory experience.
Overall, students who successfully completed the course are able to explore physical science systems through experimentally based laboratories using scientific methodologies; define and explain concepts, theories, and laws in physical science; generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science; and demonstrate basic written communication skills.
Note that this course has a focus on "doing" science, on science as a process, a way of understanding the natural physical world and the mathematics that underlies many physical systems. The course does not focus on memorized facts. The course is centered on science as being that which can be measured, observed, evidenced. The course is intended as a counter to memorized science. Once one shifts to memorized facts as the basis of a science, then any set of memorized facts can be seen by the learner as equally valid. Somewhere down at the bottom of that slope are those who are convinced the earth is flat, climate change is not happening, evolution does not explain the diversity of life, and no one actually walked on the moon. By doing simple experiments that seek to measure physical properties and quantities, by gathering and analyzing data, the intent is that students come to see science as a way of thinking about and analyzing the world around them.
The course directly supports institutional learning outcome eight and three general education outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
| GE PLO | SC 130 CLO |
|---|---|
| 3.5 Perform experiments that use scientific methods as part of the inquiry process. | 1. Explore physical science systems through experimentally based laboratories using scientific methodologies |
| 3.4 Define and explain scientific concepts, principles, and theories of a field of science. | 2. Define and explain concepts, theories, and laws in physical science. |
| 3.2 Present and interpret numeric information in graphic forms. | 3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science. |
Junida and Ashlyn measure optical distances for a plane mirror
The course also supports institutional learning outcome two.
| GE PLO | SC 130 CLO |
|---|---|
| 1.1 Write a clear, well-organized paper using documentation and quantitative tools when appropriate. | 4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software. |
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
This term the laboratory report usually analyzed for assessment of course learning outcome one, laboratory fourteen, did not occur due to the structure of the calendar spring 2019. For the purposes of evaluating this course learning outcome, laboratory nine was evaluated. In laboratory nine the time delay between seeing two boards clap and hearing the sound was timed using stopwatches. On campuses distances up to 500 meters are available, putting the delay duration well within human timing capabilities.
Emerika measures distances over which the sound clap seeing to hearing delay will be measured
Laboratory nine was assessed to determine whether students explored physical science systems through experimentally based laboratories using scientific methodologies. This was measured by assessing whether students properly recorded data in a table, generated labelled xy scatter graphs, analyzed and reported the slope of the linear regression to their data, and meaningfully discussed the results.
Of 25 students in the course, 15 submitted laboratory nine (60%). Ten failed to submit a report (40%).
Analysis of laboratory nine spring 2019
Of the fifteen students who submitted a report for laboratory nine, fourteen produced laboratory reports with data recorded in a properly formatted table. Thirteen students also generated a correct xy scattergraph. Eleven students ran a linear regression analysis using Desmos and of those eleven, seven generated a reasonably complete discussion of the meaning of the slope and the results of the laboratory.
Performance on CLO 1 analyzed laboratories across multiple terms
The 60% submission rate for laboratory nine represented a drop both term-on-term and against the long term submission rate for the analyzed laboratories (73%).
This lower than average submission rate for laboratory nine was a pattern seen in the laboratory submission rate for all but one laboratory spring 2019.
Laboratory submission rates spring 2019 versus long term average rate since fall 2014
With the exception of laboratory two, laboratory report submission rates for each laboratory spring 2019 were lower than historic average completion rates. This pattern of lower than long term average completion rates was also seen fall 2018. The cause of the reduction in submission rates this academic year is not know.
Each term also sees some slippage in the completion rate from lab-to-lab during the term. Some of this reflects students who opt to no longer attend class but have not officially withdrawn from the course. Some of this is students who are still actively attending but not completing and submitting their laboratory assignments. This term the submission rate remained near 60%. The lower rates for laboratory eleven and twelve may be a result of those two laboratories being "sandwiched" between the Rahn en Tiahk/Founding Day holidays and the Easter break holidays.
This pattern of low laboratory report completion and turn-in negatively impacted student performance and the course average, with the course average dropping below the long term mean. Note that the spring terms (75% average) consistently underperform the fall terms (81% average).
CLO 2
2. Define and explain concepts, theories, and laws in physical science.This outcome is assessed by the students exceeding 70% on five or more assessments of this learning outcome. This term there were thirteen opportunities for students to demonstrate mastery at least five times.
Mastery settings in Schoology
Performance on course learning outcome two was mixed this term.
Twelve of the twenty-five students exceeded 70% on five or more assessments of course learning outcome two.
The focus in physical science is on science as a process, as a system of experimentally generated and verified knowledge, not as a collection of memorized facts. The course is not content free, but the heavier emphasis is on data gathering, fitting mathematical models, and writing up results in reports. Once science becomes a collection of memorized and regurgitated factoids, then all collections of memorized and regurgitated factoids are equally valid. At that point one is left choosing among factoids to "believe" in and the result are those who "do not believe in science" whether that science is climate change, evolution, or any other area of science. Thus the course focuses on knowledge generated by the students and is guided in part by the concepts of non-overlapping magisteria and a constructivist epistemology.
Course learning outcome two is measured primarily by tests and quizzes. Performance on these items has been weaker over the terms, a likely result of the focus on the science as a process, on the laboratories and reports.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, an identical pre-assessment and post-assessment is included in the course. Neither the pre-assessment nor the post-assessment are announced in advance. The pre-assessment was done on the first day of class. The post-assessment was given on the penultimate day of class. The intent is to gauge what the students can do without having specifically studied.
Pre-assessment and post-assessment average correct
Students demonstrated a statistically significant improvement from the pre-assessment to the post-assessment. The effect size was medium. Students showed a moderate improvement in their ability to generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.Course level learning outcome four focuses on communication, specifically writing. In the late 1990's assessment data at the college suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
Laboratory report in Google Docs
This term the adoption of Schoology Institutional provided access to the Google Drive Assignments app. An earlier article covered the use of Google Drive Assignments in the course.
Course level student learning outcome four was directly evaluated up to seven times on rubrics used to mark odd numbered laboratory reports.
Five or more demonstrations of student learning outcome four above 70%
Affective Domain Assessment
SC 130 Physical Science functions as a science with laboratory requirement for the general education core at the college. The only major required to take the course is an associate of applied science in telecommunications technology two year program. This term none of the students were in the course as a requirement of their major.Given that the course is not a major requirement, one of the intents is to reach out to students who may not enjoy core science courses and show them science in a new light.
At the end of the term the students were asked to respond to the following questions:
1. Before I took this class my attitude towards science was:
☐ Positive: I liked science
☐ Neutral
☐ Negative: I did not like science
2. During the semester:
☐ I enjoyed physical science class
☐ Neutral
☐ I did not enjoy physical science class
3. During the semester:
☐ I was glad I signed up for physical science class
☐ Neutral
☐ I was not glad I signed up for physical science class
4. After taking this class my attitude towards science is:
☐ Positive: I like science
☐ Neutral
☐ Negative: I do not like science
Student attitudes towards science, 23 students completed the survey
As an instructor who has a passion for science, I would hope to encourage my students to also enjoy that passion. New scientists are not minted from students who dislike science and come away from the course with a negative attitude towards science. That the students enjoyed the course does not mean the course was easy, not with lab reports to be done each and every week. A demanding course can also be a positive experience, perhaps even enjoyable, perhaps fun.
Affective Domain Assessment of Applications Used During Course
The physical science course utilized three core applications during the term. Desmos calculator was used to handle data tables, graphs, and analysis. Google Docs was used for laboratory reports, And Schoology was used by students for receiving and submitting assignments. For each application the students were asked open ended questions as to what they liked and disliked about the application. Answers were then grouped into categories determined by the answers themselves.
The students also reacted very positively to the use Google Docs.
The course uses Schoology as the Learning Management System, this software is also very well received by the students.
Overall the students favored the use of the applications and found the applications to be easy to use and helpful. The students especially liked the east of use of Desmos, the automatic saving nature of Google Docs, and the ability to stay up to date on grades and assignments in Schoology.
Favored and Least favored laboratories
Some insight into the impact of the laboratories on learning can be gleaned from the reaction students had to the laboratories. At term end, the students were asked to provide feedback on a favorite laboratory and a least favorite laboratory. The feedback fell naturally into a few categories.
Overall, students who successfully completed the course are able to explore physical science systems through experimentally based laboratories using scientific methodologies; define and explain concepts, theories, and laws in physical science; generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science; and demonstrate basic written communication skills.
Comments
Post a Comment