Assessing learning in physical science
SC 130 Physical Science proposes to serve two institutional learning outcomes (ILO) through four general education program learning outcomes (GE PLO) addressed by four course level student learning outcomes (CLO). Not listed are proposed specific student learning outcomes that in turn serve the course level learning outcomes. This report assesses learning under the proposed course level learning outcomes which in turn supports program and institutional learning outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term provided a vehicle for assessing this course level outcome. The students were given a system to explore and two questions to answer. The students were provided with flying disks and rings along with a sports radar gun to measure velocity and GPS units to measure distance. The first question the students tackled was determining the nature of the mathematical model that governed the velocity versus distance behavior. The second question asked whether a flying disk thrown horizontally outperforms a non-flying object thrown horizontally. The students were provided with minimal guidance beyond the questions and technical support in the use of the equipment,
The laboratory reports were assessed to determine whether students properly recorded data in a labeled table, generated an xy scatter graph, made a determination as to the nature of the relationship, ran a best fit regression for that relationship, opted to report the slope if appropriate, and then discussed their analysis and results in a meaningful manner. Many of the students gathered data that can be fit by a linear trend line.
Student performance was assessed using a simple binary rubric based on the metrics noted above. Eighteen students completed this term end laboratory.
All eighteen students have mastered the core mechanics of the laboratory report style used in SC 130 Physical Science. The students gathered data and reported results in a properly formatted and labeled table. The students used spreadsheet software to generate xy scatter graphs also properly formatted and labeled. All of the students added a trend line to the graph, but only eight specifically discussed their choice of mathematical relationship. Although one student opted for a quadratic regression, the remaining students chose a linear regression. Of these, nine students reported their slope in their analysis. Only seven students engaged in a meaningful discussion of their results. Some students tend to lapse into discussions of what they did, of their procedure. A few write only that the laboratory was fun.
Of concern is the number of laboratory fourteens submitted. The laboratory submission rate tends to fall off as each term progresses.
The course began with 30 students registered. One student never showed up for the class, a second withdrew at midterm, and a third stopped attending class at midterm. Thus there were 27 active students by term end. The fall off in laboratory submissions as the term progresses was seen last fall and occurred again this spring. In class the odd laboratories, which are longer form laboratory reports, are emphasized as being more important. Hence laboratory eleven and thirteen this term saw upticks in submission above ten and eleven. Note that laboratory eight has no formal report. Laboratory fourteen has no late turn-in, which brings up the submission count for earlier laboratories. Laboratory seven, although an odd laboratory, is due during midterms week. Correlation is not cause, but the midterm may be a reason for the low submission rate of laboratory seven.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
Forty-two items on the 83 item final examination asked students to define and explain concepts, theories, and laws in physical science. The average for twenty-seven students on these 42 items was 61%. Although low, fall 2014 the average was 54% on 32 items.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pretest and post-test was included in the course. The post-test was embedded in the final examination.
Student performance on the pretest can only be characterized as abysmal.
Although all but one student arrived in SC 130 Physical Science having completed MS 100 College Algebra or a higher course, only seven of twenty-five students could calculate the slope and intercept for a line with an intercept of zero - a direct relationship. Of note is that 24 of the 25 could correctly plot data points on an xy scatter graph. Beyond that, skills were limited at best. Less than half could correctly identify the slope and intercept when given an equation in the form y=b+mx instead of the traditional y=mx+b format. Mathematics taught outside of a content framework does not yield understanding nor longer term retention of knowledge.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, and twelve have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. An exercise in statics yields a rational function relationship. By the end of the course students have repeatedly worked with linear relations.One relationship at a time, not "problems one to thirty even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
By term end, the 25 students who were present on the day of the pretest had all improved significantly as measured by the post-test.
In the chart above, the left end of the line marks the number of students answering an item correctly on the pretest, the right end the number answering correctly on the post-test. Note that items such as the slope and y-intercept assessment improved on the first two items even though the problems were made more difficult by a non-zero y-intercept on the post-test.
With eleven items on the pre-test and post-test, scores for students could range from zero to eleven. The median score on the pre-test was three, on the post-test items was eight. The small sample size precludes significance in this difference of medians. The rise in the mean score from 4.2 to 7.9 was significant. The 25 score distributions as box plots provide some insight into the lift in scores from the pretest to the post-test.
The post-test does not answer whether there will be long term student retention of the ability to present and interpret numeric information in graphic forms.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990s assessment data suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
By the end of the term all students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was not accomplished by all students as measured by laboratory fourteen. Producing sophisticated scientific arguments was a bridge too far for a number of students. The analysis of laboratory fourteen suggested that seven of eighteen students could engage in a meaningful discussion of their data.
Inherent in supporting institutional learning outcome two, which course learning outcome four serves, is proper mechanics. Physcial Science laboratory report marking rubrics at the national campus include evaluation of four broad metrics: syntax (grammar), vocabulary and spelling, organization, cohesion and coherence. Each of these four metrics is measured on a five point scale yielding a total possible of twenty points. In general, students enter the course with writing skills. Errors of tense and agreement tend to mirror areas in students' first language that do not have similar tense or agreement structures. All students in the class are working in English as a second language.
In general there is no significant change in mechanics measurable from laboratory one to laboratory fourteen. The sample size is small and the change in individual scores is also small. Measured across all four metrics, the median score on the first laboratory was 16.5 points, the median at term end was 20 (n = 18). This represents an uptick from scores of four to five on each metric.
The mean score at term start was 16.9 out of 20, at term end the mean score was 18.4. The course may be more beneficial to the weakest writers.
Considering the four course level outcomes, there are a couple areas where improvement can be sought. The lowest hanging fruit would be to improve laboratory submission rates after midterm. A more challenging area for improving performance would be in the students ability to engage in more sophisticated discussions of their experimental results. The students do not have a rich and varied background in science, and the course is serving principally non-science majors.
The course also has the intent to communicate affective domain messages to the students. One is the idea that doing science is fun, the other is that the students can do science. Science is not a subject in which the students have all experienced success. As one student said the first day, "I am not good in math and science, I am going to fail." That student would later prove to be one of the more insightful students, at one point coming up with a theory as to how an optical system would behave. The student had become a scientist. This term these impacts were not measured, these are subtle and difficult to quantify.
Over the years, however, students will at times comment on an image posted by a student from the class. The comments are always positive, of the nature that the course was interesting and fun. Learning happens only where there is motivation to learn, and when an activity is enjoyable, a learner will engage more fully with that activity. Physics, mathematics, and fun are three words that do not often co-exist in a single sentence, a single class, or in the mind of a student. Yet they should. The mathematical nature of the world is fascinating and fun.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
| GE PLO | SC 130 CLO |
|---|---|
| 3.5 Perform experiments that use scientific methods as part of the inquiry process. | 1. Explore physical science systems through experimentally based laboratories using scientific methodologies |
| 3.4 Define and explain scientific concepts, principles, and theories of a field of science. | 2. Define and explain concepts, theories, and laws in physical science. |
| 3.2 Present and interpret numeric information in graphic forms. | 3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science. |
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
| GE PLO | SC 130 CLO |
|---|---|
| 1.1 Write a clear, well-organized paper using documentation and quantitative tools when appropriate. | 4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software. |
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term provided a vehicle for assessing this course level outcome. The students were given a system to explore and two questions to answer. The students were provided with flying disks and rings along with a sports radar gun to measure velocity and GPS units to measure distance. The first question the students tackled was determining the nature of the mathematical model that governed the velocity versus distance behavior. The second question asked whether a flying disk thrown horizontally outperforms a non-flying object thrown horizontally. The students were provided with minimal guidance beyond the questions and technical support in the use of the equipment,
Students throwing disks
The laboratory reports were assessed to determine whether students properly recorded data in a labeled table, generated an xy scatter graph, made a determination as to the nature of the relationship, ran a best fit regression for that relationship, opted to report the slope if appropriate, and then discussed their analysis and results in a meaningful manner. Many of the students gathered data that can be fit by a linear trend line.
Student performance was assessed using a simple binary rubric based on the metrics noted above. Eighteen students completed this term end laboratory.
All eighteen students have mastered the core mechanics of the laboratory report style used in SC 130 Physical Science. The students gathered data and reported results in a properly formatted and labeled table. The students used spreadsheet software to generate xy scatter graphs also properly formatted and labeled. All of the students added a trend line to the graph, but only eight specifically discussed their choice of mathematical relationship. Although one student opted for a quadratic regression, the remaining students chose a linear regression. Of these, nine students reported their slope in their analysis. Only seven students engaged in a meaningful discussion of their results. Some students tend to lapse into discussions of what they did, of their procedure. A few write only that the laboratory was fun.
Of concern is the number of laboratory fourteens submitted. The laboratory submission rate tends to fall off as each term progresses.
The course began with 30 students registered. One student never showed up for the class, a second withdrew at midterm, and a third stopped attending class at midterm. Thus there were 27 active students by term end. The fall off in laboratory submissions as the term progresses was seen last fall and occurred again this spring. In class the odd laboratories, which are longer form laboratory reports, are emphasized as being more important. Hence laboratory eleven and thirteen this term saw upticks in submission above ten and eleven. Note that laboratory eight has no formal report. Laboratory fourteen has no late turn-in, which brings up the submission count for earlier laboratories. Laboratory seven, although an odd laboratory, is due during midterms week. Correlation is not cause, but the midterm may be a reason for the low submission rate of laboratory seven.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
Forty-two items on the 83 item final examination asked students to define and explain concepts, theories, and laws in physical science. The average for twenty-seven students on these 42 items was 61%. Although low, fall 2014 the average was 54% on 32 items.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pretest and post-test was included in the course. The post-test was embedded in the final examination.
Student performance on the pretest can only be characterized as abysmal.
Although all but one student arrived in SC 130 Physical Science having completed MS 100 College Algebra or a higher course, only seven of twenty-five students could calculate the slope and intercept for a line with an intercept of zero - a direct relationship. Of note is that 24 of the 25 could correctly plot data points on an xy scatter graph. Beyond that, skills were limited at best. Less than half could correctly identify the slope and intercept when given an equation in the form y=b+mx instead of the traditional y=mx+b format. Mathematics taught outside of a content framework does not yield understanding nor longer term retention of knowledge.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, and twelve have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. An exercise in statics yields a rational function relationship. By the end of the course students have repeatedly worked with linear relations.One relationship at a time, not "problems one to thirty even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
By term end, the 25 students who were present on the day of the pretest had all improved significantly as measured by the post-test.
In the chart above, the left end of the line marks the number of students answering an item correctly on the pretest, the right end the number answering correctly on the post-test. Note that items such as the slope and y-intercept assessment improved on the first two items even though the problems were made more difficult by a non-zero y-intercept on the post-test.
With eleven items on the pre-test and post-test, scores for students could range from zero to eleven. The median score on the pre-test was three, on the post-test items was eight. The small sample size precludes significance in this difference of medians. The rise in the mean score from 4.2 to 7.9 was significant. The 25 score distributions as box plots provide some insight into the lift in scores from the pretest to the post-test.
The post-test does not answer whether there will be long term student retention of the ability to present and interpret numeric information in graphic forms.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990s assessment data suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
By the end of the term all students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was not accomplished by all students as measured by laboratory fourteen. Producing sophisticated scientific arguments was a bridge too far for a number of students. The analysis of laboratory fourteen suggested that seven of eighteen students could engage in a meaningful discussion of their data.
Inherent in supporting institutional learning outcome two, which course learning outcome four serves, is proper mechanics. Physcial Science laboratory report marking rubrics at the national campus include evaluation of four broad metrics: syntax (grammar), vocabulary and spelling, organization, cohesion and coherence. Each of these four metrics is measured on a five point scale yielding a total possible of twenty points. In general, students enter the course with writing skills. Errors of tense and agreement tend to mirror areas in students' first language that do not have similar tense or agreement structures. All students in the class are working in English as a second language.
In general there is no significant change in mechanics measurable from laboratory one to laboratory fourteen. The sample size is small and the change in individual scores is also small. Measured across all four metrics, the median score on the first laboratory was 16.5 points, the median at term end was 20 (n = 18). This represents an uptick from scores of four to five on each metric.
The mean score at term start was 16.9 out of 20, at term end the mean score was 18.4. The course may be more beneficial to the weakest writers.
Considering the four course level outcomes, there are a couple areas where improvement can be sought. The lowest hanging fruit would be to improve laboratory submission rates after midterm. A more challenging area for improving performance would be in the students ability to engage in more sophisticated discussions of their experimental results. The students do not have a rich and varied background in science, and the course is serving principally non-science majors.
The course also has the intent to communicate affective domain messages to the students. One is the idea that doing science is fun, the other is that the students can do science. Science is not a subject in which the students have all experienced success. As one student said the first day, "I am not good in math and science, I am going to fail." That student would later prove to be one of the more insightful students, at one point coming up with a theory as to how an optical system would behave. The student had become a scientist. This term these impacts were not measured, these are subtle and difficult to quantify.
Over the years, however, students will at times comment on an image posted by a student from the class. The comments are always positive, of the nature that the course was interesting and fun. Learning happens only where there is motivation to learn, and when an activity is enjoyable, a learner will engage more fully with that activity. Physics, mathematics, and fun are three words that do not often co-exist in a single sentence, a single class, or in the mind of a student. Yet they should. The mathematical nature of the world is fascinating and fun.
Comments
Post a Comment