Assessing learning in physical science
SC 130 Physical Science proposes to serve two institutional learning outcomes (ILO) through four general education program learning outcomes (GE PLO) addressed by four course level student learning outcomes (CLO). Not listed are proposed specific student learning outcomes that in turn serve the course level learning outcomes. This report assesses learning under the proposed course level learning outcomes which in turn supports program and institutional learning outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term usually provides a vehicle for assessing this course level outcome. In the past the students have been given a system to explore and questions to answer. This term the structure of the holidays and their interaction with the Thursday laboratory schedule deleted one laboratory from the syllabus. Laboratory fourteen was removed and laboratory nine was used to assess the first course learning outcome.
In laboratory nine the students timed the arrival delay between seeing two wooden boards clap and hearing the sound of the clap. The experiment was done at different distances such as to generate a linear relationship between the time and the distance. The slope of that relationship is the speed of sound.
The laboratory reports were assessed to determine whether students properly recorded data in a labelled table, generated an xy scatter graph, added a linear trend line to the graph, reported the slope in their analysis as the speed of sound, and then discussed their analysis and results in a reasonably meaningful manner.
By ninth week thirty students of the original thirty-two were still actively attending class. Twenty-one students turned in a laboratory report for laboratory nine.
For all twenty-one laboratory reports submitted, students recorded their data in a properly labelled table and went on to generate a labelled xy scatter graph. Twenty students correctly added a linear trend line to the graph. Only twelve went on to write up an analysis that explicitly noted that the slope of the line on the chart represented the speed of sound. The students are able to handle the software mechanics of producing tables and graphs, but understanding what that graph then means physically is far more difficult for the students.
Only five students then continued on with a meaningful discussion of the results including an error analysis against the published speed of sound at the air temperature measured on the day of the laboratory.
The submission of only twenty-one reports for thirty active students might seem low, but this rate is on par with rates seen a year ago during spring 2015.
The 70% submission rate for laboratory nine exceeded the submission rate, year-on-year, for that laboratory.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
The 34 item final examination asked students to define and explain concepts, theories, and laws in physical science. The final also had students make calculations, use formulas, and interpret graphical data. Average success rates based on an item analysis of the 34 item final examination were aggregated by topic and by skill. Aggregate average success rates based on an item analysis of the final examination were generally low.
The overall average for thirty students on these 34 items was 51%, a drop from the 66% seen on the fall 2015 final examination. Aggregate average performance on the final examination was lower than in prior terms.
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory nine assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pre-assessment and post-assessment was included in the course. The post-assessment was embedded in the final examination.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, and twelve have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. By the end of the course students have repeatedly worked with linear relations. One relationship at a time, not "problems one to thirty even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
The first thirteen questions on the final examination were identical to the questions on the pre-assessment. The following bar chart depicts the percentage of students answering correctly on the pre-assessment on the right end of the bar, the percentage of students answering correctly on the post-assessment on the left end of the bar. Thus the chart shows the improvement from the pre-assessment to the post-assessment. Of note is the overall weak performance on all items on the pre-assessment except for the physical plotting of (x,y) scatter plot points.
The average score for the students increased from 5.06 out of 13 to 8.63 out of 13. This increase was significant with a large effect size (0.94). Although improvement was seen for all questions, overall post-assessment performance (66%) is below a target value of 70%.
The post-assessment does not answer whether there will be long term student retention of the ability to present and interpret numeric information in graphic forms. The course has had, at least in the short run, a positive impact on the students' ability to work with numeric information in graphical forms.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990s assessment data suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
By the end of the term students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was demonstrated by only five students as measured by laboratory nine.
Inherent in supporting institutional learning outcome two, which course learning outcome four serves, is proper mechanics. Physical Science laboratory report marking rubrics at the national campus include evaluation of four broad metrics: syntax (grammar), vocabulary and spelling, organization, cohesion and coherence. Each of these four metrics is measured on a five point scale yielding a total possible of twenty points. In general, students enter the course with writing skills. Errors of tense and agreement tend to mirror areas in students' first language that do not have similar tense or agreement structures. All students in the class are working in English as a second language.
Two metrics were examined this term, syntax and cohesion. Syntax showed a significant improvement from laboratory one to laboratory thirteen with a medium effect size (0.64). Cohesion did not show a significant improvement from laboratory one to laboratory thirteen.
The course has an impact on syntax, possibly improving control of grammar. The student's ability to write cohesive science text that flows and connects logically from one idea to the next may not be positively impacted.
Learning has occurred for all course level outcomes, the program learning outcomes served by those course level outcomes, and the institutional learning outcomes served in turn by the program learning outcomes.
ILO 8. Quantitative Reasoning: ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations; comprehends and can create sophisticated arguments supported by quantitative evidence and can clearly communicate those arguments in a variety of formats.
| GE PLO | SC 130 CLO |
|---|---|
| 3.5 Perform experiments that use scientific methods as part of the inquiry process. | 1. Explore physical science systems through experimentally based laboratories using scientific methodologies |
| 3.4 Define and explain scientific concepts, principles, and theories of a field of science. | 2. Define and explain concepts, theories, and laws in physical science. |
| 3.2 Present and interpret numeric information in graphic forms. | 3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science. |
ILO 2. Effective written communication: development and expression of ideas in writing through work in many genres and styles, utilizing different writing technologies, and mixing texts, data, and images through iterative experiences across the curriculum.
| GE PLO | SC 130 CLO |
|---|---|
| 1.1 Write a clear, well-organized paper using documentation and quantitative tools when appropriate. | 4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software. |
CLO 1
Explore physical science systems through experimentally based laboratories using scientific methodologies
Laboratory fourteen in the penultimate week of the term usually provides a vehicle for assessing this course level outcome. In the past the students have been given a system to explore and questions to answer. This term the structure of the holidays and their interaction with the Thursday laboratory schedule deleted one laboratory from the syllabus. Laboratory fourteen was removed and laboratory nine was used to assess the first course learning outcome.
The clapper is back at that distant rise. Appears impossible from the photo, but the class could see the boards clapping together.
In laboratory nine the students timed the arrival delay between seeing two wooden boards clap and hearing the sound of the clap. The experiment was done at different distances such as to generate a linear relationship between the time and the distance. The slope of that relationship is the speed of sound.
Telephoto view from 500 meters out, the class would also obtain data at 550 meters.
The laboratory reports were assessed to determine whether students properly recorded data in a labelled table, generated an xy scatter graph, added a linear trend line to the graph, reported the slope in their analysis as the speed of sound, and then discussed their analysis and results in a reasonably meaningful manner.
By ninth week thirty students of the original thirty-two were still actively attending class. Twenty-one students turned in a laboratory report for laboratory nine.
Analysis of laboratory nine
For all twenty-one laboratory reports submitted, students recorded their data in a properly labelled table and went on to generate a labelled xy scatter graph. Twenty students correctly added a linear trend line to the graph. Only twelve went on to write up an analysis that explicitly noted that the slope of the line on the chart represented the speed of sound. The students are able to handle the software mechanics of producing tables and graphs, but understanding what that graph then means physically is far more difficult for the students.
Only five students then continued on with a meaningful discussion of the results including an error analysis against the published speed of sound at the air temperature measured on the day of the laboratory.
The submission of only twenty-one reports for thirty active students might seem low, but this rate is on par with rates seen a year ago during spring 2015.
Laboratory submission rates for each odd (major) lab report
The 70% submission rate for laboratory nine exceeded the submission rate, year-on-year, for that laboratory.
CLO 2
2. Define and explain concepts, theories, and laws in physical science.
The 34 item final examination asked students to define and explain concepts, theories, and laws in physical science. The final also had students make calculations, use formulas, and interpret graphical data. Average success rates based on an item analysis of the 34 item final examination were aggregated by topic and by skill. Aggregate average success rates based on an item analysis of the final examination were generally low.
Aggregate average success rates by topic and skill
The overall average for thirty students on these 34 items was 51%, a drop from the 66% seen on the fall 2015 final examination. Aggregate average performance on the final examination was lower than in prior terms.
Multi-term final examination averages based on item analysis aggregate average
CLO 3
3. Generate mathematical models for physical science systems and use appropriate mathematical techniques and concepts to obtain quantitative solutions to problems in physical science.
While the laboratory nine assessment above provides some data on this course learning outcome, this outcome supports the general education program learning outcome "3.2 Present and interpret numeric information in graphic forms." With this focus in mind, a pre-assessment and post-assessment was included in the course. The post-assessment was embedded in the final examination.
SC 130 Physical Science includes a focus on the mathematical models that underlie physical science systems. Laboratories one, two, three, five, seven, nine, eleven, and twelve have linear relationships. A number of assignments in the course also have linear relationships. The students also encounter a quadratic relationship in laboratory three. A plot of height versus velocity generates a power relationship, specifically a square root relationship. By the end of the course students have repeatedly worked with linear relations. One relationship at a time, not "problems one to thirty even problems only." Every equation is built from data that the students have gathered. From the concrete to the abstract, repeated throughout the term, providing cognitive hooks on which to "hang" their mathematical learning.
The first thirteen questions on the final examination were identical to the questions on the pre-assessment. The following bar chart depicts the percentage of students answering correctly on the pre-assessment on the right end of the bar, the percentage of students answering correctly on the post-assessment on the left end of the bar. Thus the chart shows the improvement from the pre-assessment to the post-assessment. Of note is the overall weak performance on all items on the pre-assessment except for the physical plotting of (x,y) scatter plot points.
The average score for the students increased from 5.06 out of 13 to 8.63 out of 13. This increase was significant with a large effect size (0.94). Although improvement was seen for all questions, overall post-assessment performance (66%) is below a target value of 70%.
CLO 4
4. Demonstrate basic communication skills by working in groups on laboratory experiments and by writing up the result of experiments, including thoughtful discussion and interpretation of data, in a formal format using spreadsheet and word processing software.
Course level learning outcome four focuses on communication, specifically writing. In the late 1990s assessment data suggested some students were graduating with limited writing communication skills. As noted by the languages and literature division at that time, two college level writing courses in the general education core cannot by themselves produce collegiate level writers. Writing must occur across the curriculum, across disciplines. In 2007 SC 130 Physical Science at the national campus was redesigned to put an emphasis on writing. A "fill-in the blank" cook book style laboratory manual was replaced by laboratories which led to laboratory reports constructed using spreadsheet and word processing software.
By the end of the term students could produce a laboratory report with tables and charts integrated from a spreadsheet package. The students could produce reports that included the use of quantitative tools.
As reported above, student ability to include thoughtful discussion and interpretation of data supported by their quantitative evidence was demonstrated by only five students as measured by laboratory nine.
Inherent in supporting institutional learning outcome two, which course learning outcome four serves, is proper mechanics. Physical Science laboratory report marking rubrics at the national campus include evaluation of four broad metrics: syntax (grammar), vocabulary and spelling, organization, cohesion and coherence. Each of these four metrics is measured on a five point scale yielding a total possible of twenty points. In general, students enter the course with writing skills. Errors of tense and agreement tend to mirror areas in students' first language that do not have similar tense or agreement structures. All students in the class are working in English as a second language.
Two metrics were examined this term, syntax and cohesion. Syntax showed a significant improvement from laboratory one to laboratory thirteen with a medium effect size (0.64). Cohesion did not show a significant improvement from laboratory one to laboratory thirteen.
The course has an impact on syntax, possibly improving control of grammar. The student's ability to write cohesive science text that flows and connects logically from one idea to the next may not be positively impacted.
Learning has occurred for all course level outcomes, the program learning outcomes served by those course level outcomes, and the institutional learning outcomes served in turn by the program learning outcomes.
Comments
Post a Comment