Assessing learning in physical science
SC 130 Physical Science proposes to serve two institutional learning outcomes (ILO) through four general education program learning outcomes (GE PLO) addressed by four course level student learning outcomes (CLO). Not listed are proposed specific student learning outcomes that in turn serve the course level learning outcomes. This report assesses learning under the proposed course level learning outcomes which in turn supports program and institutional learning outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term provided a vehicle for assessing this course level outcome. The students were given a system to explore and two questions to answer. The students were provided with string and weight pendulums and asked to determine the relationship between the length and the period. The class started in the classroom working with lengths from 25 cm to 150 cm.
Using the gathered data and spreadsheets, the students fit linear regressions to the data. Over the lengths involved the non-linearity of the data is not apparent. The students then used their linear regressions to predict the period for a 700 cm pendulum.
After posting predictions on the board, the class walked down to the gym where a 700 cm pendulum can be swung. The predictions were up around 10 seconds, but the actual period was around 5 seconds (4.85 seconds and 5.38 seconds were the recorded values). The data did not fit the model. The students were then instructed to write up the laboratory taking into account the errant prediction.
The laboratory reports were assessed to determine whether students properly recorded data in a labeled table, generated an xy scatter graph, added a regression equation to their data, made a determination as to the non-linear nature of the relationship, and then discussed their analysis and results in a reasonably meaningful manner.
Student performance was assessed using a simple binary rubric based on the metrics noted above. Ten of the fourteen students completed this term end laboratory.
The ten students demonstrated mastery the core mechanics of the laboratory report style used in SC 130 Physical Science. The students gathered data and reported results in a properly formatted and labeled table. The students used spreadsheet software to generate xy scatter graphs also properly formatted and labeled. All of the students added a trend line to the graph, but only seven specifically discussed their choice of mathematical relationship. All ten students chose a linear regression. Only a single student noted that the data was "not really linear." That one student went on to note that there existed the possibility of the "math model used being wrong."
During the fall and spring terms, laboratory submission rates tend to fall as the term progresses. This summer saw no fall off in laboratory submission rates. One student simply failed to complete laboratories in the later half of the term, but the remaining thirteen consistently turned in laboratories after midterm with the only exception being the last laboratory. Being at the end of the term, the final laboratory (laboratory fourteen) had a narrower window for submission.
Summer term is short and intense. Only two students were taking classes other than physical science, thus this was the only course for the majority of the students. These factors may have contributed to the turn-in rate remaining high during the summer session.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
Twenty-nine items on the 66 item final examination asked students to define and explain concepts, theories, and laws in physical science. The average for fourteen students on these 29 items was 80%. This is significantly higher than the 61% success rate seen in the spring and the 54% success rate fall 2014. The strong performance is also likely a unique effect of the brevity and intensity of the summer term.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pretest and post-test was included in the course. The post-test was embedded in the final examination.
Student performance on the pretest can only be characterized as abysmal.
Nine of the fourteen students completed a 100 or higher level college mathematics course prior to enrolling in the SC 130. The mathematics background for five of the students was not determined. Given that background the general inability of the students to calculate linear slopes and intercepts is disappointing. The only skill the students consistently appear to bring into physical science is the ability to plot points on an xy scattergraph.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, and twelve have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. By the end of the course students have repeatedly worked with linear relations.One relationship at a time, not "problems one to thirty even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
By term end, the students who were present on the day of the pretest had all improved significantly as measured by the post-test. Note that ten students sat the preassessment, fourteen completed the post-assessment, hence the use of percentage success rate in the following chart.
In the chart above, the left end of the line marks the percent of students answering an item correctly on the pretest, the right end the percent answering correctly on the post-test. Note that items such as the slope and y-intercept assessment improved on the first two items even though the problems were made more difficult by a non-zero y-intercept on the post-test.
With eleven items on the pre-test and post-test, scores for students could range from zero to eleven. The median score on the pre-test was one, on the post-test items was eight. The rise in the mean score was from 1.43 to 7.5. The score distributions as box plots provide some insight into the lift in scores from the pretest to the post-test. Note that ten students sat the pre-assessment and fourteen sat the post-assessment.
The post-test does not answer whether there will be long term student retention of the ability to present and interpret numeric information in graphic forms. The course has had, at least in the short run, a strong positive impact on the students' ability to work with numeric information in graphical forms.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990s assessment data suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
By the end of the term students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was not accomplished by all students as measured by laboratory fourteen. In fact only one student questioned the whether the data was linear (the data was not linear).
Producing sophisticated scientific arguments was a bridge too far for a number of students. The analysis of laboratory fourteen suggested that only three of fourteen students (21%) could engage in a meaningful discussion of their data. Spring 2015 seven of eighteen students (39%) showed an ability to reasonably and meaningfully discuss their results. The compressed summer term may leave less time for reflection, contemplation, and insight.
Inherent in supporting institutional learning outcome two, which course learning outcome four serves, is proper mechanics. Physcial Science laboratory report marking rubrics at the national campus include evaluation of four broad metrics: syntax (grammar), vocabulary and spelling, organization, cohesion and coherence. Each of these four metrics is measured on a five point scale yielding a total possible of twenty points. In general, students enter the course with writing skills. Errors of tense and agreement tend to mirror areas in students' first language that do not have similar tense or agreement structures. All students in the class are working in English as a second language.
In general there is no significant change in mechanics measurable from laboratory one to laboratory fourteen. The sample size is small and the change in individual scores is also small. Measured across all four metrics, the median score went unchanged. On the first laboratory the median was the maximum possible of 20 points, the median at term end was 20 (n = 10). The students in the class generally had basic grammar skills when they arrived in the class. Over the duration of the summer only two students had frequent grammar and syntax issues.
The mean score at term start was 17.57 out of 20, at term end the mean score was 19.5. The course may be more beneficial to the weakest writers.
Considering the four course level outcomes, there are areas where improvement can be sought. A challenging area for improving performance would be in the students ability to engage in more sophisticated discussions of their experimental results. The students do not have a rich and varied background in science, and the course is serving principally non-science majors.
The course also has the intent to communicate affective domain messages to the students. One is the idea that doing science is fun, the other is that the students can do science. Science is not a subject in which the students have all experienced success. This term these impacts were not measured, these are subtle and difficult to quantify.
Over the years, however, students will at times comment on an image posted by a student from the class. The comments are always positive, of the nature that the course was interesting and fun. Learning happens only where there is motivation to learn, and when an activity is enjoyable, a learner will engage more fully with that activity.
Physics, mathematics, and fun are three words that do not often co-exist in a single sentence, a single class, or in the mind of a student. Yet they should. The mathematical nature of the world is fascinating and fun. My hope is that those three words can now co-exist in the minds of the physical science students.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
| GE PLO | SC 130 CLO |
|---|---|
| 3.5 Perform experiments that use scientific methods as part of the inquiry process. | 1. Explore physical science systems through experimentally based laboratories using scientific methodologies |
| 3.4 Define and explain scientific concepts, principles, and theories of a field of science. | 2. Define and explain concepts, theories, and laws in physical science. |
| 3.2 Present and interpret numeric information in graphic forms. | 3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science. |
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
| GE PLO | SC 130 CLO |
|---|---|
| 1.1 Write a clear, well-organized paper using documentation and quantitative tools when appropriate. | 4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software. |
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term provided a vehicle for assessing this course level outcome. The students were given a system to explore and two questions to answer. The students were provided with string and weight pendulums and asked to determine the relationship between the length and the period. The class started in the classroom working with lengths from 25 cm to 150 cm.
Using the gathered data and spreadsheets, the students fit linear regressions to the data. Over the lengths involved the non-linearity of the data is not apparent. The students then used their linear regressions to predict the period for a 700 cm pendulum.
After posting predictions on the board, the class walked down to the gym where a 700 cm pendulum can be swung. The predictions were up around 10 seconds, but the actual period was around 5 seconds (4.85 seconds and 5.38 seconds were the recorded values). The data did not fit the model. The students were then instructed to write up the laboratory taking into account the errant prediction.
The laboratory reports were assessed to determine whether students properly recorded data in a labeled table, generated an xy scatter graph, added a regression equation to their data, made a determination as to the non-linear nature of the relationship, and then discussed their analysis and results in a reasonably meaningful manner.
Student performance was assessed using a simple binary rubric based on the metrics noted above. Ten of the fourteen students completed this term end laboratory.
The ten students demonstrated mastery the core mechanics of the laboratory report style used in SC 130 Physical Science. The students gathered data and reported results in a properly formatted and labeled table. The students used spreadsheet software to generate xy scatter graphs also properly formatted and labeled. All of the students added a trend line to the graph, but only seven specifically discussed their choice of mathematical relationship. All ten students chose a linear regression. Only a single student noted that the data was "not really linear." That one student went on to note that there existed the possibility of the "math model used being wrong."
During the fall and spring terms, laboratory submission rates tend to fall as the term progresses. This summer saw no fall off in laboratory submission rates. One student simply failed to complete laboratories in the later half of the term, but the remaining thirteen consistently turned in laboratories after midterm with the only exception being the last laboratory. Being at the end of the term, the final laboratory (laboratory fourteen) had a narrower window for submission.
Summer term is short and intense. Only two students were taking classes other than physical science, thus this was the only course for the majority of the students. These factors may have contributed to the turn-in rate remaining high during the summer session.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
Twenty-nine items on the 66 item final examination asked students to define and explain concepts, theories, and laws in physical science. The average for fourteen students on these 29 items was 80%. This is significantly higher than the 61% success rate seen in the spring and the 54% success rate fall 2014. The strong performance is also likely a unique effect of the brevity and intensity of the summer term.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pretest and post-test was included in the course. The post-test was embedded in the final examination.
Student performance on the pretest can only be characterized as abysmal.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, and twelve have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. By the end of the course students have repeatedly worked with linear relations.One relationship at a time, not "problems one to thirty even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
By term end, the students who were present on the day of the pretest had all improved significantly as measured by the post-test. Note that ten students sat the preassessment, fourteen completed the post-assessment, hence the use of percentage success rate in the following chart.
In the chart above, the left end of the line marks the percent of students answering an item correctly on the pretest, the right end the percent answering correctly on the post-test. Note that items such as the slope and y-intercept assessment improved on the first two items even though the problems were made more difficult by a non-zero y-intercept on the post-test.
With eleven items on the pre-test and post-test, scores for students could range from zero to eleven. The median score on the pre-test was one, on the post-test items was eight. The rise in the mean score was from 1.43 to 7.5. The score distributions as box plots provide some insight into the lift in scores from the pretest to the post-test. Note that ten students sat the pre-assessment and fourteen sat the post-assessment.
The post-test does not answer whether there will be long term student retention of the ability to present and interpret numeric information in graphic forms. The course has had, at least in the short run, a strong positive impact on the students' ability to work with numeric information in graphical forms.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990s assessment data suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
By the end of the term students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was not accomplished by all students as measured by laboratory fourteen. In fact only one student questioned the whether the data was linear (the data was not linear).
Producing sophisticated scientific arguments was a bridge too far for a number of students. The analysis of laboratory fourteen suggested that only three of fourteen students (21%) could engage in a meaningful discussion of their data. Spring 2015 seven of eighteen students (39%) showed an ability to reasonably and meaningfully discuss their results. The compressed summer term may leave less time for reflection, contemplation, and insight.
Inherent in supporting institutional learning outcome two, which course learning outcome four serves, is proper mechanics. Physcial Science laboratory report marking rubrics at the national campus include evaluation of four broad metrics: syntax (grammar), vocabulary and spelling, organization, cohesion and coherence. Each of these four metrics is measured on a five point scale yielding a total possible of twenty points. In general, students enter the course with writing skills. Errors of tense and agreement tend to mirror areas in students' first language that do not have similar tense or agreement structures. All students in the class are working in English as a second language.
In general there is no significant change in mechanics measurable from laboratory one to laboratory fourteen. The sample size is small and the change in individual scores is also small. Measured across all four metrics, the median score went unchanged. On the first laboratory the median was the maximum possible of 20 points, the median at term end was 20 (n = 10). The students in the class generally had basic grammar skills when they arrived in the class. Over the duration of the summer only two students had frequent grammar and syntax issues.
The mean score at term start was 17.57 out of 20, at term end the mean score was 19.5. The course may be more beneficial to the weakest writers.
Considering the four course level outcomes, there are areas where improvement can be sought. A challenging area for improving performance would be in the students ability to engage in more sophisticated discussions of their experimental results. The students do not have a rich and varied background in science, and the course is serving principally non-science majors.
The course also has the intent to communicate affective domain messages to the students. One is the idea that doing science is fun, the other is that the students can do science. Science is not a subject in which the students have all experienced success. This term these impacts were not measured, these are subtle and difficult to quantify.
Over the years, however, students will at times comment on an image posted by a student from the class. The comments are always positive, of the nature that the course was interesting and fun. Learning happens only where there is motivation to learn, and when an activity is enjoyable, a learner will engage more fully with that activity.
Physics, mathematics, and fun are three words that do not often co-exist in a single sentence, a single class, or in the mind of a student. Yet they should. The mathematical nature of the world is fascinating and fun. My hope is that those three words can now co-exist in the minds of the physical science students.
Comments
Post a Comment