Assessing Learning in Physical Science
SC 130 Physical Science proposes to serve two institutional learning outcomes (ILO) through four general education program learning outcomes (GE PLO) addressed by four course level student learning outcomes (CLO). Not listed are proposed specific student learning outcomes that in turn serve the course level learning outcomes. This report assesses learning under the proposed course level learning outcomes which in turn supports program and institutional learning outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term provided a vehicle for assessing this course level outcome. The students were given a system that was unfamiliar to them and asked to determine the underlying mathematical models for the objects in the system. The specific system was the launch velocity versus the flight distance for a variety of flying objects. The marking rubric for this final laboratory report of the term was not revealed to the students. The students should have been familiar with the structure of a laboratory report by this point in the term.
The laboratory reports were assessed to determine whether students properly recorded data in labelled tables, generated xy scatter graphs, made a decision on linearity and if deemed to be a linear relationship added linear trend lines to the graph, reported the slopes in their analysis, and discussed the results. Twenty-five of twenty-six students submitted this laboratory.
For the 25 laboratory reports submitted, students recorded their data in properly labelled tables and generated labelled xy scatter graphs. This term a number of students obtained random data distributions. Others had more linear data. Twenty-one students made a reasonable determination of the appropriate mathematical model for their data. Eleven students went on to discuss their results in a reasonably meaningful manner.
2. Define and explain concepts, theories, and laws in physical science.
The final examination asked students to plot data, interpret graphs, make calculations, and define concepts (facts) in physical science. This term the course generated 2091 points, with 1157 of those points in laboratory reports.
The focus in physical science is on science as a process, as a system of experimentally generated and verified knowledge, not as a collection of memorized facts. The course is not content free, but the heavier emphasis is on data gathering, fitting mathematical models, and writing up results in reports. This reflects my own biases. Once science becomes a collection of memorized and regurgitated factoids, then all collections of memorized and regurgitated factoids are equally valid. At that point one is left choosing among factoids to "believe" in and the result are those who "do not believe in science" whether that science is climate change, evolution, or any other area of science. Thus the course focuses on knowledge generated by the students and is guided in part by the concepts of non-overlapping magisteria and a constructivist epistemology.
Given the above, the final examination was not weighted and would not be weighted. At only 50 points, the final examination was only 2% of a student's overall mark. By the time of the final examination the work throughout the term, especially the laboratory work, has assessed the students. The students were aware of the low impact of the final examination and understood that the test would not have a large impact. High stakes testing is rarely a reliable measure of longer term knowledge retained. Performance levels may be lowered by these factors, but the hope is that the results more closely represent retained knowledge than memorized, regurgitated, and then forgotten knowledge.
Average success rates based on an item analysis of the final examination were aggregated by three skill areas: analysis and interpretations of graphs, citing of facts, and calculations. Aggregate average success rates based on an item analysis of the final examination were generally low but in line with pre-existing values.
The overall final examination average for the 26 students based on the item analysis was 64%, statistically identical to performance on the final examination summer 2016. The past two terms have seen a considerable improvement from the spring 2016 success rate of 51% and a return to the 66% success rate seen on the fall 2015 final examination.
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pre-assessment and post-assessment was included in the course. The post-assessment was embedded in the final examination.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, and twelve have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. By the end of the course students have repeatedly worked with linear relations. One relationship, one equation, at a time, not "problems one to thirty, even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
The first eleven questions on the final examination were identical to the eleven questions on a pre-assessment. The following bar chart depicts the percentage of students answering correctly on the pre-assessment on the right end of the bar, the percentage of students answering correctly on the post-assessment on the left end of the bar. Thus the chart depicts the improvement from the pre-assessment to the post-assessment.
On all questions performance improved. Note that the ability to plot data points on a graph remained statistically the same with an increase from 95% to 96% due to differences in the number of students who took the pre-assessment versus the post-assessment. Students added and dropped the course after the day one pre-assessment.
The post-assessment does not answer whether there will be long term student retention of the ability to present and interpret numeric information in graphic forms. The course has had, at least in the short run, a positive impact on the students' ability to work with numeric information in graphical forms.
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990's assessment data at the college suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
This term the laboratory report marking rubrics added student learning outcomes from the proposed* course outline:
CLO 4.0 Communication skills: Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
SLO 4.1 Produce laboratory reports: Produce laboratory reports with integrated tables, charts, and graphs
SLO 4.2 Use technology to produce laboratory reports: Use spreadsheets and word processing software
SLO 4.3 Effective communication: Communicate in writing using proper syntax and correctly used physical science vocabulary.
These were marked on a four point scale:
4 Exceeds expectations
3 Meets expectations
2 Does not meet expectations: needs improvement
1 Severe does not meet expectations: no evidence of outcome being met
Laboratory reports for laboratory fourteen were marked against the above metrics with the following results:
One student did not submit laboratory fourteen. That student also did not succeed in the course having not submitted any laboratory reports after early September. Of the remaining 25 students, all 25 either met or exceeded the student learning outcome.
By the end of the term these 25 students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was demonstrated by eleven students as measured by laboratory fourteen.
Although one student did not turn in laboratory reports after September, laboratory completion rates were high with student submissions of laboratories averaging 92%.
Spring term laboratory submission rates are lower in the spring term than in the fall and summer terms.
* The current course outline actually dates to 2007. Up to 2007 an out-of-date 88 outcome list of science facts constituted the SC 130 Physical Science outline. In 2007 that outline was essentially compressed into four broad subject areas as a first step in revising the outline and running the course with a writing focus and a new textbook. As the course was being taught in 2007 to 2010, work continued on revamping the outline. In 2011 the college undertook reformatting all outlines at the college, but instructors were instructed to not change content at that time. That meant that the outline approved in 2012 was still the outline designed in 2007. Each year since 2012 a revised outline has been prepared and submitted. The proposed outline has yet to be considered by the appropriate bodies. The course covers the material specified on both outlines, there has not been a loss of coverage of the 2007 specified materials. This assessment report, however, is organized around the proposed outline.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
| GE PLO | SC 130 CLO |
|---|---|
| 3.5 Perform experiments that use scientific methods as part of the inquiry process. | 1. Explore physical science systems through experimentally based laboratories using scientific methodologies |
| 3.4 Define and explain scientific concepts, principles, and theories of a field of science. | 2. Define and explain concepts, theories, and laws in physical science. |
| 3.2 Present and interpret numeric information in graphic forms. | 3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science. |
| GE PLO | SC 130 CLO |
|---|---|
| 1.1 Write a clear, well-organized paper using documentation and quantitative tools when appropriate. | 4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software. |
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term provided a vehicle for assessing this course level outcome. The students were given a system that was unfamiliar to them and asked to determine the underlying mathematical models for the objects in the system. The specific system was the launch velocity versus the flight distance for a variety of flying objects. The marking rubric for this final laboratory report of the term was not revealed to the students. The students should have been familiar with the structure of a laboratory report by this point in the term.
The laboratory reports were assessed to determine whether students properly recorded data in labelled tables, generated xy scatter graphs, made a decision on linearity and if deemed to be a linear relationship added linear trend lines to the graph, reported the slopes in their analysis, and discussed the results. Twenty-five of twenty-six students submitted this laboratory.
Analysis of laboratory fourteen
For the 25 laboratory reports submitted, students recorded their data in properly labelled tables and generated labelled xy scatter graphs. This term a number of students obtained random data distributions. Others had more linear data. Twenty-one students made a reasonable determination of the appropriate mathematical model for their data. Eleven students went on to discuss their results in a reasonably meaningful manner.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
The final examination asked students to plot data, interpret graphs, make calculations, and define concepts (facts) in physical science. This term the course generated 2091 points, with 1157 of those points in laboratory reports.
The focus in physical science is on science as a process, as a system of experimentally generated and verified knowledge, not as a collection of memorized facts. The course is not content free, but the heavier emphasis is on data gathering, fitting mathematical models, and writing up results in reports. This reflects my own biases. Once science becomes a collection of memorized and regurgitated factoids, then all collections of memorized and regurgitated factoids are equally valid. At that point one is left choosing among factoids to "believe" in and the result are those who "do not believe in science" whether that science is climate change, evolution, or any other area of science. Thus the course focuses on knowledge generated by the students and is guided in part by the concepts of non-overlapping magisteria and a constructivist epistemology.
Given the above, the final examination was not weighted and would not be weighted. At only 50 points, the final examination was only 2% of a student's overall mark. By the time of the final examination the work throughout the term, especially the laboratory work, has assessed the students. The students were aware of the low impact of the final examination and understood that the test would not have a large impact. High stakes testing is rarely a reliable measure of longer term knowledge retained. Performance levels may be lowered by these factors, but the hope is that the results more closely represent retained knowledge than memorized, regurgitated, and then forgotten knowledge.
Average success rates based on an item analysis of the final examination were aggregated by three skill areas: analysis and interpretations of graphs, citing of facts, and calculations. Aggregate average success rates based on an item analysis of the final examination were generally low but in line with pre-existing values.
Aggregate average success rates by skill area on the final
The overall final examination average for the 26 students based on the item analysis was 64%, statistically identical to performance on the final examination summer 2016. The past two terms have seen a considerable improvement from the spring 2016 success rate of 51% and a return to the 66% success rate seen on the fall 2015 final examination.
Multi-term final examination averages based on item analysis aggregate average
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pre-assessment and post-assessment was included in the course. The post-assessment was embedded in the final examination.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, and twelve have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. By the end of the course students have repeatedly worked with linear relations. One relationship, one equation, at a time, not "problems one to thirty, even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
The first eleven questions on the final examination were identical to the eleven questions on a pre-assessment. The following bar chart depicts the percentage of students answering correctly on the pre-assessment on the right end of the bar, the percentage of students answering correctly on the post-assessment on the left end of the bar. Thus the chart depicts the improvement from the pre-assessment to the post-assessment.
On all questions performance improved. Note that the ability to plot data points on a graph remained statistically the same with an increase from 95% to 96% due to differences in the number of students who took the pre-assessment versus the post-assessment. Students added and dropped the course after the day one pre-assessment.
The post-assessment does not answer whether there will be long term student retention of the ability to present and interpret numeric information in graphic forms. The course has had, at least in the short run, a positive impact on the students' ability to work with numeric information in graphical forms.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990's assessment data at the college suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
This term the laboratory report marking rubrics added student learning outcomes from the proposed* course outline:
CLO 4.0 Communication skills: Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
SLO 4.1 Produce laboratory reports: Produce laboratory reports with integrated tables, charts, and graphs
SLO 4.2 Use technology to produce laboratory reports: Use spreadsheets and word processing software
SLO 4.3 Effective communication: Communicate in writing using proper syntax and correctly used physical science vocabulary.
These were marked on a four point scale:
4 Exceeds expectations
3 Meets expectations
2 Does not meet expectations: needs improvement
1 Severe does not meet expectations: no evidence of outcome being met
Laboratory reports for laboratory fourteen were marked against the above metrics with the following results:
By the end of the term these 25 students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was demonstrated by eleven students as measured by laboratory fourteen.
Laboratory completion rates for each of the fourteen laboratory have, in some terms, shown a fall off in the percentage of students submitting laboratory reports as the term progresses. This was especially true in the spring terms of 2015 and 2016. This term there was no trend in the submission rates from laboratory one to laboratory fourteen.
Note that the y-axis does not start from zero: vertical differences are exaggerated
Spring term laboratory submission rates are lower in the spring term than in the fall and summer terms.
There was no difference in performance by gender on either the final examination or the overall course average. The course had 14 female students and 12 male students.
Affective domain feedback
An earlier article reported on student attitudes towards science, student responses to the textbook, and consideration of potentially majoring in a field of science. The fifteen laboratories were also surveyed for student reactions.
Laboratory one: Density of soap
Laboratory two: Velocity of a rolling ball
Laboratory three: Acceleration of gravity by dropping a ball
Laboratory four: Investigation of momentum of colliding marbles in, marbles out
Laboratory five: Force of friction
Laboratory six: Heat conduction in materials
Laboratory seven: Using a GPS to determine meters per minute of latitude or longitude
Laboratory eight: Coloring clouds
Laboratory nine: Sound:, clapping wood blocks to determine the speed of sound
Laboratory ten: Spectra, RGB colors, hue saturation luminosity, X11 colors, HTML
Laboratory eleven: Reflection in a mirror and apparent depth of pennies underwater
Laboratory twelve: Batteries and bulbs, conductors, Ohm's law
Laboratory thirteen: Chemistry, acid and base detection using flowers
Laboratory fourteen: Mathematical models of flying disks
Laboratory fifteen: Site Swap Notation
Students were asked "What was your favorite laboratory?" and "What was your most disliked (least favorite) laboratory?" Follow-up questions asked the students why they liked or disliked that laboratory.
Laboratory thirteen was a favored laboratory with comments including "nice to see the changes of chemicals," "I like chemistry," "fun to understand how to use flowers," "I get to see the way the color of the flower changed," and "while doing this experiment I feel like I am a scientist."
Laboratory seven led the disfavored laboratory list for reasons including, "I don't really like using a GPS," "way too far that cause us to walk and walk," "because little to know, the GPS doesn't give out the correct formation [sic]," "it is my first time to used, so I am still confusing about used the GPS to find location by walking," "I hate walking, I was so tired when we walked and try to the direction," and "Because I do not know how to use the GPS." Note that the distance being referred to is a 621 meter walk on campus done on Wednesday. The three students who picked the same laboratory as their favorite noted their reasons as being, "We were able to use the GPS in finding the latitude and longitude," "I learned so many things in this lab by using the GPS," "I learned how to locate things."
The overall pattern of liked and disliked laboratories for fall 2016 is reflective of the liked and disliked laboratories since 2009. The one difference this term is that laboratory eight is often an even split liked versus disliked, this term laboratory eight was liked. Eight has been shifted to include a set of weather and climate videos in the first part of the laboratory session.
Comments
Post a Comment