Assessing learning in physical science
SC 130 Physical Science proposes to serve two institutional learning outcomes (ILO) through four general education program learning outcomes (GE PLO) addressed by four course level student learning outcomes (CLO). Not listed are proposed specific student learning outcomes that in turn serve the course level learning outcomes. This report assesses learning under the proposed course level learning outcomes which in turn supports program and institutional learning outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term provided a vehicle for assessing this course level outcome. The students were given a system to explore and questions to answer. The students were provided with flying disks and asked to determine whether a mathematical relationship exists between the launch velocity and the flight distance. If a relationship is found to exist, is the relationship linear? If the relationship is linear, then what is the equation for distance given a launch velocity? The disks were thrown only horizontally. Launch speeds were obtained with a small sports radar gun.
The laboratory reports were assessed to determine whether students properly recorded data in a labeled table, generated an xy scatter graph, made a determination as to whether a mathematical relationship existed, added an appropriate trend line, reported the slope in their analysis, and then discussed their analysis and results in a reasonably meaningful manner.
Twenty-seven of thirty students completed this term end laboratory.
Twenty-seven students recorded their data in a properly labeled table and went on to generate a labeled xy scattergraph. While only 13 students made explicit mention of the nature of the mathematical relationship, the data was generally linear and 23 students added a linear trend line without specifically stating that the data was linear.
An explanation in the class prior to the lab noted that a non-flying object launched horizontally should travel a distance d = velocity × time where the time is a constant equal to the fall time of the object from the launch height. A launch just above a meter off the ground should produce a linear relationship for distance traveled with a slope around 0.5 seconds. That may sound strange, but bear in mind the x-axis is launch velocity in meters per second, the y-axis is distance in meters.
This information provided some of the background for their discussion of their results, but only six students had a reasonably meaningful discussion of their results.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
Twenty-eight items on the 35 item final examination asked students to define and explain concepts, theories, and laws in physical science. The average for thrity students on these 28 items was 66.4%. While summer term 2014 saw a success rate of 80% for eleven summer students, summer term is unique. For many summer students physical science is their only class, and the class meets daily. The fall success rate of 66.4% on this material is in line with the 61% success rate seen in spring 2015 and the 54% success rate fall 2014. The increase should note be seen as significant as the 54% and 61% success rates were based on an item analysis at the subquestion level. Fall 2015 there were 87 subquestions, many of the 35 questions had multiple parts. To facilitate analysis, item analysis was done at the question level, not subquestion level. Questions where the subquestions were answered predominantly correctly were deemed correct. If only a single subsquestion were missed, or if the subquestions were not deemed substantive to demonstrating understanding of the underlying material, then these questions were still counted as being substantively correct. This would inflate the success rate and explains the uptick in performance.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pretest and post-test was included in the course. The post-test was embedded in the final examination. The pretest data fall 2015 was lost in a corruption of a synchronization with a cloud storage service. Performance on the pretest had been very weak.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, twelve, and fourteen have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. By the end of the course students have repeatedly worked with linear relations. One relationship at a time, not "problems one to thirty even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
The first thirteen questions on the final examination were identical to the questions on the pretest. Performance on these thirteen questions was strong. Thirty students sat the final examination.
For the majority of the questions, more than 70% of the students scored above 70%. The following table replicates the data graphed above.
Note that performance on the first slope calculation from a graph question saw only a 60% success rate, the second calculation of a slope from graphed data had an 80% success rate. This is explained by the first graph having a non-zero y-intercept while the second graph had a zero y-intercept. Thus the second instance was easier.
On the 13 questions, a 70% success rate would have been a score of 9.1 correctly answered problems. 75% of the students answered nine or more problems correctly, and half of the thirty student class answered 11 or more correctly on the post-test. 25% answered all 13 correctly. Student performance on the post-test questions was strong.
The post-test does not answer whether there will be long term student retention of the ability to present and interpret numeric information in graphic forms. The course has had, at least in the short run, a strong positive impact on the students' ability to work with numeric information in graphical forms.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990s assessment data suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
By the end of the term students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was by only six students as measured by laboratory fourteen.
Inherent in supporting institutional learning outcome two, which course learning outcome four serves, is proper mechanics. Physcial Science laboratory report marking rubrics at the national campus include evaluation of four broad metrics: syntax (grammar), vocabulary and spelling, organization, cohesion and coherence. Each of these four metrics is measured on a five point scale yielding a total possible of twenty points. In general, students enter the course with writing skills. Errors of tense and agreement tend to mirror areas in students' first language that do not have similar tense or agreement structures. All students in the class are working in English as a second language.
In general there is no significant change in mechanics measurable from laboratory one to laboratory fourteen. The complications are that the sample size is small and scores on the first laboratory report are generally high for most students. There is little room for improvement for many students. Only a few students have serious problems with mechanics, and sample size prevents drawing significant conclusions. Some students report orally that the class has helped with their writing skills, but this is anecdotal only.
Overall scores on laboratory fourteen indicate that strong student performances. There laboratory was scored against a 61 point rubric with 25 points in document format, grammar, vocabulary, organization, and cohesion. Writing metrics were fully 41% of the points. A reasonable writing score would have placed a paper at 51 points (83.6%). As seen in the chart below, on the order of 75% of the students performed above this level on the final laboratory report.
Learning has occurred for all four outcomes. Areas of potential improvement have been identified. The item analysis of the final examination also provides detailed insights of use primarily to the instructor. Specific topics and student mastery of those topics is included in the item analysis. The following table image shows topic, subtopic, skill, and the number of students (n = 30) who answered that question correctly. This data can be analyzed by topic, subtopic, or skill to provide other insights to strengths and weaknesses which can then be used to adjust curriculum for future terms.
A qualitative analysis was run of liked and disliked laboratories. The following table tabulates the comments on the most liked laboatories and the most disliked laboratories during the term. The laboratory number is in the first column.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
| GE PLO | SC 130 CLO |
|---|---|
| 3.5 Perform experiments that use scientific methods as part of the inquiry process. | 1. Explore physical science systems through experimentally based laboratories using scientific methodologies |
| 3.4 Define and explain scientific concepts, principles, and theories of a field of science. | 2. Define and explain concepts, theories, and laws in physical science. |
| 3.2 Present and interpret numeric information in graphic forms. | 3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science. |
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
| GE PLO | SC 130 CLO |
|---|---|
| 1.1 Write a clear, well-organized paper using documentation and quantitative tools when appropriate. | 4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software. |
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term provided a vehicle for assessing this course level outcome. The students were given a system to explore and questions to answer. The students were provided with flying disks and asked to determine whether a mathematical relationship exists between the launch velocity and the flight distance. If a relationship is found to exist, is the relationship linear? If the relationship is linear, then what is the equation for distance given a launch velocity? The disks were thrown only horizontally. Launch speeds were obtained with a small sports radar gun.
Marsha measures the flight distance for a flying disk
The laboratory reports were assessed to determine whether students properly recorded data in a labeled table, generated an xy scatter graph, made a determination as to whether a mathematical relationship existed, added an appropriate trend line, reported the slope in their analysis, and then discussed their analysis and results in a reasonably meaningful manner.
Twenty-seven of thirty students completed this term end laboratory.
An explanation in the class prior to the lab noted that a non-flying object launched horizontally should travel a distance d = velocity × time where the time is a constant equal to the fall time of the object from the launch height. A launch just above a meter off the ground should produce a linear relationship for distance traveled with a slope around 0.5 seconds. That may sound strange, but bear in mind the x-axis is launch velocity in meters per second, the y-axis is distance in meters.
This information provided some of the background for their discussion of their results, but only six students had a reasonably meaningful discussion of their results.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
Twenty-eight items on the 35 item final examination asked students to define and explain concepts, theories, and laws in physical science. The average for thrity students on these 28 items was 66.4%. While summer term 2014 saw a success rate of 80% for eleven summer students, summer term is unique. For many summer students physical science is their only class, and the class meets daily. The fall success rate of 66.4% on this material is in line with the 61% success rate seen in spring 2015 and the 54% success rate fall 2014. The increase should note be seen as significant as the 54% and 61% success rates were based on an item analysis at the subquestion level. Fall 2015 there were 87 subquestions, many of the 35 questions had multiple parts. To facilitate analysis, item analysis was done at the question level, not subquestion level. Questions where the subquestions were answered predominantly correctly were deemed correct. If only a single subsquestion were missed, or if the subquestions were not deemed substantive to demonstrating understanding of the underlying material, then these questions were still counted as being substantively correct. This would inflate the success rate and explains the uptick in performance.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory fourteen assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pretest and post-test was included in the course. The post-test was embedded in the final examination. The pretest data fall 2015 was lost in a corruption of a synchronization with a cloud storage service. Performance on the pretest had been very weak.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, twelve, and fourteen have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. By the end of the course students have repeatedly worked with linear relations. One relationship at a time, not "problems one to thirty even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
The first thirteen questions on the final examination were identical to the questions on the pretest. Performance on these thirteen questions was strong. Thirty students sat the final examination.
For the majority of the questions, more than 70% of the students scored above 70%. The following table replicates the data graphed above.
| Question | Post |
| Calculate slope from line on graph | 0.60 |
| Determine y-intercept from line on graph | 0.87 |
| Write out the equation of line from graph | 0.73 |
| x-axis units identification | 1.00 |
| y-axis units identification | 0.80 |
| slope units identification | 0.83 |
| Plot xy data given a table labeled x, y | 1.00 |
| Calculate slope from data | 0.80 |
| Determine y-intercept from plotted line on graph | 0.83 |
| Calculate y given x | 0.83 |
| Calculate x given y | 0.60 |
| Determine slope from slope-intercept form | 0.83 |
| Determine y-intercept from slope-intercept form | 0.93 |
Note that performance on the first slope calculation from a graph question saw only a 60% success rate, the second calculation of a slope from graphed data had an 80% success rate. This is explained by the first graph having a non-zero y-intercept while the second graph had a zero y-intercept. Thus the second instance was easier.
On the 13 questions, a 70% success rate would have been a score of 9.1 correctly answered problems. 75% of the students answered nine or more problems correctly, and half of the thirty student class answered 11 or more correctly on the post-test. 25% answered all 13 correctly. Student performance on the post-test questions was strong.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990s assessment data suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
By the end of the term students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was by only six students as measured by laboratory fourteen.
Inherent in supporting institutional learning outcome two, which course learning outcome four serves, is proper mechanics. Physcial Science laboratory report marking rubrics at the national campus include evaluation of four broad metrics: syntax (grammar), vocabulary and spelling, organization, cohesion and coherence. Each of these four metrics is measured on a five point scale yielding a total possible of twenty points. In general, students enter the course with writing skills. Errors of tense and agreement tend to mirror areas in students' first language that do not have similar tense or agreement structures. All students in the class are working in English as a second language.
In general there is no significant change in mechanics measurable from laboratory one to laboratory fourteen. The complications are that the sample size is small and scores on the first laboratory report are generally high for most students. There is little room for improvement for many students. Only a few students have serious problems with mechanics, and sample size prevents drawing significant conclusions. Some students report orally that the class has helped with their writing skills, but this is anecdotal only.
Overall scores on laboratory fourteen indicate that strong student performances. There laboratory was scored against a 61 point rubric with 25 points in document format, grammar, vocabulary, organization, and cohesion. Writing metrics were fully 41% of the points. A reasonable writing score would have placed a paper at 51 points (83.6%). As seen in the chart below, on the order of 75% of the students performed above this level on the final laboratory report.
Learning has occurred for all four outcomes. Areas of potential improvement have been identified. The item analysis of the final examination also provides detailed insights of use primarily to the instructor. Specific topics and student mastery of those topics is included in the item analysis. The following table image shows topic, subtopic, skill, and the number of students (n = 30) who answered that question correctly. This data can be analyzed by topic, subtopic, or skill to provide other insights to strengths and weaknesses which can then be used to adjust curriculum for future terms.
A qualitative analysis was run of liked and disliked laboratories. The following table tabulates the comments on the most liked laboatories and the most disliked laboratories during the term. The laboratory number is in the first column.
| Lab | Liked | Disliked |
| 1 | First lab I did, I learned to calculate density and that not all soap sink. | My partner and I end up with the wrong density [I did not know labs late beyond one week would not be accepted] I was absent and it really effected my grade, but surely if I didn't miss I am sure I would have liked it too Learn why things float Mainly because it was my first laboratory research and I was clueless (I did not like being clueless) |
| 2 | I like labs 2-9, in each I did learn something new especially the parts where I learn to use the computer and solve problems of each lab. | Poor grade Hard, complicated, too many numbers I missed a lot of information and didn't turn in the lab |
| 3 | ||
| 4 | I found it interesting when I found out that the number of marbles roll in will be the same number of marbles that will come out | It is really confusing and I cannot find the speed of the marbles speed when it collides with the others. |
| 5 | I wasn't focusing on my subjects I don't like solving and I'm a little bit lost I did not really understand the lab and it somewhat complicates me I don't know what I am doing in this lab, I don't know how to measure or use the materials |
|
| 6 | I did not understand most of the lab | |
| 7 | I learned how to use GPS Very useful to me learning to use GPS I learn how to use a GPS and can now determine which way is latitude and which is longitude I used the GPS to determine where did the instructor hid and it is really fun. |
Did not understand I got tired of walking that day It involved a lot of walking and the report took forever First time to use a GPS so I found it confusing and hard to use it to determine the meters per minute of latitude or longitude |
| 8 | Easy I love to draw! |
|
| 9 | Interesting how far the distance is still you can hear the sound Very interesting to me I just realize that sounds from farther distances can reach us in seconds and I like the way we move farther and farther to detect the speed of sound I got to clap the wood blocks! |
Too much walking |
| 10 | I get to learn the names of different colors and how to mix them colors and make a new color | I could not remember the colors and which color makes what when mixed I didn't learned some of those things, I learned how to change colors but I don't know when it is turned to hue saturation luminosity, X11 colors, spectra and so forth. |
| 11 | ||
| 12 | I did the experiment the right way I found the lab helpful, teaching me knowledge I need to stay safe from electricity |
I am not becoming a electronic or PUC |
| 13 | Simple and great for elementary school teachers, very interesting I understood this more than the others Very fun boiling the flower and look at the color as it changes, also interesting to learn which substance is acid and bases, I learn that cleaning products are basic substances, fun lab. I like chemistry, it interest me when we add other materials and it detected whether its acid or base |
I have a lack of ability in chemistry I only know some colors, so when I mix the acids and bases sometimes I do not know the color |
| 14 | I have fun flying and using the GPS I like flying the disk and it wasn't so hard doing this lab |
|
| 15 | Nothing to turn in I got to learn different ways to juggle Fun and I found out that ****** don't know how to catch the ball or dapadap [juggle] I love this laboratory because I enjoy joggling the balls plus ****** don't know how to joggle the ball I get to learn how to juggle |
I do not get the main point of this lab, maybe if we do a report I will be able to consume some ideas. |
Comments
Post a Comment