General Education Laboratory Science Common Assignment Assessment and Institutional Memory
Institutional memory is always a challenge at any institution. Despite the best laid plans, acres of file cabinets, and buildings stuffed with servers and RAID arrays, institutional memory still often comes down to some old codger who can tell the story. This old codger, however, has difficulty remembering the specific details of days, dates, and times. My blog is a proxy for my faulty memory, a place I can write stuff down and then find it again using Google and a search restricted to site:danaleeling.blogspot.com. Thus my blog is really only notes to future self, a part of my personal memory, an extension of my mind.
Over the break I have been reading The Innovators and was reminded of the early tension between those who sought to create computers that would be functional equivalents of humans: intelligent agents, robots perhaps, which/who would function independently of people and those who sought a human/computer symbiosis where the computer extended the capabilities of humans. My blog is an extension of my memory systems, a computer as a symbiont.
This blog is such a note to myself, an update on the general education laboratory science assessment over the past five years. I would note that this history does not reflect my own ongoing assessment in physical science since 2007. Course level assessment in physical science in recorded up through 2009 via links on an assessment page and after 2009 in a thread within this same blog.
General education laboratory science program level assessment via a common assignment was begun with work towards adopting a rubric by which to assess a common assignment. This work was reported on in an article in early 2012. A retrospective overview was completed in August 2012 along with recommendations on next steps. Inter-year assessor issues were a cause for concern at that time.
Fall 2012 some laboratory reports were submitted. At that time the general education assessment was being overseen by a full time faculty member as an additional duty. The faculty member was effectively operating without a budget and could not stipend faculty to do the reading and rating work that had to be done. The faculty member also did not have any sort of organizational authority over faculty across the system, complicating efforts to get the common assignment submitted from all sites. Situations such as this helped drive the redevelopment of an existing unfilled assessment coordinator's position and the elevation of that position in the academic hierarchy of the college.
During the fall of 2013 instructors involved in the general education laboratory science assessment discussed using a different rubric. An instructor at the Chuuk campus shared a rubric in use on their campus, I also shared a 50 point rubric I use in my class.
An instructor at Pohnpei site expressed a preference for the 50 point rubric I had shared. Follow-up emails from the other instructors indicated a willingness to use the rubric I had shared, although the instructor in Chuuk expressed concern that the 50 point rubric was complex and difficult to use.
While the original idea was to gather laboratory reports and have an independent team read the lab reports and score them against the 50 point rubric, this did not occur.
Although the instructor in Chuuk had been the only instructor to express reservations about the 50 point rubric, the instructor in Chuuk would be the only instructor to report results against the 50 point rubric in a January 2014 email. The analysis was different than the one's I had been doing with the rubric, so I re-analyzed my laboratory reports using the criterion selected by the instructor in Chuuk. This led to the following data:
Chuuk campus:
Common assessment assignment (laboratory report) scores for SC117 Tropical Pacific Islands Ecology
Number of students: 26
7 students scored 80% or above on this assessment
5 students scored above 70%.
The other 14 either scored below 70% or did not turn in this lab.
National campus:
SC 130 Physcial science laboratory analyzed: 072 linear relationship between minutes of longitude and meters
Original number of students registered in the course at term start: 30
Number of students registered in course at time lab was due: 27
Greater than or equal to 80%: 13
Greater than or equal to 70%: 4
Below 70%: 4
Missing (not turned in): 6
Withdrawals from the course prior to laboratory being due: 3
At that time I asked the other instructors to perform the same analysis but never received a response. I felt that I had responsibility for obtaining results but no actual authority. I was also concerned that the data above was too aggregated to provide specific information for improving courses. Breaking out scores by the ten categories on the 50 point rubric might yield more actionable information.
As far as I know, laboratory reports were not gathered fall 2014 as a common assessment. Course level assessment and the impact on program learning outcomes continues to be reported.
One request by faculty from prior terms was to find a way to physically bring together all of the general education science laboratory instructors at some point, system-wide, to discuss what the data meant and which way general education science program level assessment should go in the next set of cycles. Although expensive, physical face-to-face meetings still cannot be replaced by electronic options. At some point all of the general education science laboratory instructors need to sit around a common table and hash out what we know, what we want to know, and how we will accomplish that mission going forward.
Post-script 21 May 2015: The departure of the assessment coordinator in January left a gap in the assessment structures of the college. I realize that there are those who would say there was no gap, but I would demur. On 10 March 2015 I sent out a plea to general education science faculty to submit laboratories to me and I would work on writing up an assessment. One faculty member reported that it was too late in the term for this request. 10 March was merely a week after midterm examinations, I did not feel it was too late. Two other faculty promised to get laboratory reports to me, but did not. As I noted above, this blog is really notes to myself, notes to keep track of what was done and what was not, and when.
Over the break I have been reading The Innovators and was reminded of the early tension between those who sought to create computers that would be functional equivalents of humans: intelligent agents, robots perhaps, which/who would function independently of people and those who sought a human/computer symbiosis where the computer extended the capabilities of humans. My blog is an extension of my memory systems, a computer as a symbiont.
This blog is such a note to myself, an update on the general education laboratory science assessment over the past five years. I would note that this history does not reflect my own ongoing assessment in physical science since 2007. Course level assessment in physical science in recorded up through 2009 via links on an assessment page and after 2009 in a thread within this same blog.
General education laboratory science program level assessment via a common assignment was begun with work towards adopting a rubric by which to assess a common assignment. This work was reported on in an article in early 2012. A retrospective overview was completed in August 2012 along with recommendations on next steps. Inter-year assessor issues were a cause for concern at that time.
Fall 2012 some laboratory reports were submitted. At that time the general education assessment was being overseen by a full time faculty member as an additional duty. The faculty member was effectively operating without a budget and could not stipend faculty to do the reading and rating work that had to be done. The faculty member also did not have any sort of organizational authority over faculty across the system, complicating efforts to get the common assignment submitted from all sites. Situations such as this helped drive the redevelopment of an existing unfilled assessment coordinator's position and the elevation of that position in the academic hierarchy of the college.
During the fall of 2013 instructors involved in the general education laboratory science assessment discussed using a different rubric. An instructor at the Chuuk campus shared a rubric in use on their campus, I also shared a 50 point rubric I use in my class.
An instructor at Pohnpei site expressed a preference for the 50 point rubric I had shared. Follow-up emails from the other instructors indicated a willingness to use the rubric I had shared, although the instructor in Chuuk expressed concern that the 50 point rubric was complex and difficult to use.
While the original idea was to gather laboratory reports and have an independent team read the lab reports and score them against the 50 point rubric, this did not occur.
Although the instructor in Chuuk had been the only instructor to express reservations about the 50 point rubric, the instructor in Chuuk would be the only instructor to report results against the 50 point rubric in a January 2014 email. The analysis was different than the one's I had been doing with the rubric, so I re-analyzed my laboratory reports using the criterion selected by the instructor in Chuuk. This led to the following data:
Chuuk campus:
Common assessment assignment (laboratory report) scores for SC117 Tropical Pacific Islands Ecology
Number of students: 26
7 students scored 80% or above on this assessment
5 students scored above 70%.
The other 14 either scored below 70% or did not turn in this lab.
National campus:
SC 130 Physcial science laboratory analyzed: 072 linear relationship between minutes of longitude and meters
Original number of students registered in the course at term start: 30
Number of students registered in course at time lab was due: 27
Greater than or equal to 80%: 13
Greater than or equal to 70%: 4
Below 70%: 4
Missing (not turned in): 6
Withdrawals from the course prior to laboratory being due: 3
At that time I asked the other instructors to perform the same analysis but never received a response. I felt that I had responsibility for obtaining results but no actual authority. I was also concerned that the data above was too aggregated to provide specific information for improving courses. Breaking out scores by the ten categories on the 50 point rubric might yield more actionable information.
As far as I know, laboratory reports were not gathered fall 2014 as a common assessment. Course level assessment and the impact on program learning outcomes continues to be reported.
One request by faculty from prior terms was to find a way to physically bring together all of the general education science laboratory instructors at some point, system-wide, to discuss what the data meant and which way general education science program level assessment should go in the next set of cycles. Although expensive, physical face-to-face meetings still cannot be replaced by electronic options. At some point all of the general education science laboratory instructors need to sit around a common table and hash out what we know, what we want to know, and how we will accomplish that mission going forward.
Post-script 21 May 2015: The departure of the assessment coordinator in January left a gap in the assessment structures of the college. I realize that there are those who would say there was no gap, but I would demur. On 10 March 2015 I sent out a plea to general education science faculty to submit laboratories to me and I would work on writing up an assessment. One faculty member reported that it was too late in the term for this request. 10 March was merely a week after midterm examinations, I did not feel it was too late. Two other faculty promised to get laboratory reports to me, but did not. As I noted above, this blog is really notes to myself, notes to keep track of what was done and what was not, and when.
Comments
Post a Comment