Assessment of test two in SC 130 Physical Science
Due to staff development day, test two in SC 130 Physical Science will function as the comprehensive midterm examination. An item analysis of the questions asked was regrouped by whether the question asked for a response that was purely a memory recall item, a mathematical calculation (which may require recall of a formula), an inference from data presented, an explanation of a system presented in class, or an explanation of a system not presented in class but explainable by the science presented in class. Test questions were all short answer, no multiple choice, no matching, nor true false questions were involved.
For 32 students (with two absent on test day), the following percentage of the 32 students answered the type specified correctly:
memory recall item: 73%
mathematical calculation: 42%
inference from data presented: 32%
explanation of a system presented in class: 47%
explanation of a system not presented in class: 16%
As had been found in the past, students perform best on recall of specific facts that can be memorized. Success rates fall for calculations. In Fall 2007, a 36% success rate was reported on calculations. Success rates on single memorized facts range as high as 84% and 90% for the final examinations in spring and fall 2008.
Calculations often require using a formula from memory and then making a calculation with the formula. The course focuses a on few core concepts and formulas. Forty-two percent of the students experienced success on these more complex problems.
The inference problems included reading a graph and explaining what was happening physically as indicated by the data on the graph. In fall 2008 39% of the students answered the same type of problem correctly. On test one this term, fifty percent of the students answered correctly the same questions as posed on the midterm. Only 30% of the students answered the same two questions correctly on the midterm. Overall, for all questions requiring an inference, only 32% of the students answered correctly. That the same two questions, questions which were covered in class after test one, saw a 20% drop in performance might be taken as an indicator of knowledge loss and/or a lack of studying. This also indicates that students have difficulty reading graphs and interpreting the physical meaning of a data trend.
Students did slightly better on explaining a system for which the explanation was given in class. When faced with a new, novel system that is explained by the same physical phenomenon as earlier systems, student success collapses to 16%. This is not unexpected. Although I do have such studies in hand, research I saw in the 1980s indicated that when faced with a novel system even science majors will often fall back on concepts held prior to entering science classes in school. Students tend to be unable to use newly learned science on novel systems. Science methods such as the use of discrepant events by authors such as Tik K. Liem attempt to tackle this phenomenon by confronting students with new systems that behave counter-intuitively and yet in complete accordance with the science learned by the student.
Although I employ some discrepant event science, that does not mean that this is a "magic bullet." In fact, that five students successfully explained a system that they had never seen before may be taken positively. When presented with the anomalous system, most people including many in the sciences would likely have difficulty explaining why the system behaves in the way that it does. In this particular case a thorough familiarity with fluid dynamics would have been necessary to reach the correct conclusion. Bear in mind also that SC 130 Physical Science is the "science for non-hard science majors." The marine science and health careers majors do not take physical science.
Although the course does include some memorized content, the intent is to center the course on exploring phenomenon and developing scientifically based explanations. Based on in-class conversations, a few students are gradually coming to understand that a laboratory cannot prove a scientific theory, only fail to disconfirm the theory.
For 32 students (with two absent on test day), the following percentage of the 32 students answered the type specified correctly:
memory recall item: 73%
mathematical calculation: 42%
inference from data presented: 32%
explanation of a system presented in class: 47%
explanation of a system not presented in class: 16%
As had been found in the past, students perform best on recall of specific facts that can be memorized. Success rates fall for calculations. In Fall 2007, a 36% success rate was reported on calculations. Success rates on single memorized facts range as high as 84% and 90% for the final examinations in spring and fall 2008.
Calculations often require using a formula from memory and then making a calculation with the formula. The course focuses a on few core concepts and formulas. Forty-two percent of the students experienced success on these more complex problems.
The inference problems included reading a graph and explaining what was happening physically as indicated by the data on the graph. In fall 2008 39% of the students answered the same type of problem correctly. On test one this term, fifty percent of the students answered correctly the same questions as posed on the midterm. Only 30% of the students answered the same two questions correctly on the midterm. Overall, for all questions requiring an inference, only 32% of the students answered correctly. That the same two questions, questions which were covered in class after test one, saw a 20% drop in performance might be taken as an indicator of knowledge loss and/or a lack of studying. This also indicates that students have difficulty reading graphs and interpreting the physical meaning of a data trend.
Students did slightly better on explaining a system for which the explanation was given in class. When faced with a new, novel system that is explained by the same physical phenomenon as earlier systems, student success collapses to 16%. This is not unexpected. Although I do have such studies in hand, research I saw in the 1980s indicated that when faced with a novel system even science majors will often fall back on concepts held prior to entering science classes in school. Students tend to be unable to use newly learned science on novel systems. Science methods such as the use of discrepant events by authors such as Tik K. Liem attempt to tackle this phenomenon by confronting students with new systems that behave counter-intuitively and yet in complete accordance with the science learned by the student.
Although I employ some discrepant event science, that does not mean that this is a "magic bullet." In fact, that five students successfully explained a system that they had never seen before may be taken positively. When presented with the anomalous system, most people including many in the sciences would likely have difficulty explaining why the system behaves in the way that it does. In this particular case a thorough familiarity with fluid dynamics would have been necessary to reach the correct conclusion. Bear in mind also that SC 130 Physical Science is the "science for non-hard science majors." The marine science and health careers majors do not take physical science.
Although the course does include some memorized content, the intent is to center the course on exploring phenomenon and developing scientifically based explanations. Based on in-class conversations, a few students are gradually coming to understand that a laboratory cannot prove a scientific theory, only fail to disconfirm the theory.
Comments
Post a Comment