Assessing Learning in Introductory Statistics
To say this was an unusual term would be an understatement. The term would end on the Monday of week ten, the sixteenth of March 2020. In prior terms I had wrapped up section 10.1, which occurred this term just three days earlier on Friday 13 March, with the note that the term could now end as the students could now perform confidence interval hypothesis testing. I would note that the next set of material would not introduce any new statistical capabilities per se, just another way to do hypothesis testing. 10.2 introduces the t-statistic and 10.3 the p-value, and provided one always does two tailed hypothesis testing, the results should be those found with a confidence interval. There would be the missing material of sample on sample testing for a difference of the means.
This term the term did end with 10.1 due to the global pandemic SARS-CoV-2 virus and the associated Covid-19 illnesses. The redesign of the course to compress new material into the first three-quarters of the term and then engage in open data exploration analysis, and presentations in the final four plus weeks paid an unexpected dividend this term - the course came within a week and half of covering all material despite ending at the start of the tenth of sixteen weeks.
MS 150 Statistics is an introductory statistics course with a focus on statistical operations and methods. The course is guided by the 2007 Guidelines for Assessment and Instruction in Statistics Education (GAISE), the spring 2016 draft GAISE update, and the ongoing effort at the college to incorporate authentic assessment in courses. A history of the evolution open data open data exploration exercises and associated presentations as authentic assessment in the course was covered in a May 2017 report.
Three course level student learning outcomes currently guide MS 150 Introduction to Statistics:
Success rates were down from prior terms, but comparisons are essentially without statistical meaning as the term was not completed. A full term includes more data analysis presentations that reinforce the students' statistical skill set and practice final examinations. The low performance on the mode is spurious. The 15% success rate reflects the number of students who answered "No mode" Other correct answers were not tallied by the Schoology item analysis function which only looks at the first possible right answer of multiple right answers. Other correct answers included "There is no mode", "None", "There is no one single mode value," and so forth. All of these are also correct but are not reflected above.
1.0 Perform basic statistical calculations for a single variable up to and including graphical analysis, confidence intervals, hypothesis testing against an expected value, and testing two samples for a difference of means.
2.0 Perform basic statistical calculations for paired correlated variables.
3.0 Engage in data exploration and analysis using appropriate statistical techniques including numeric calculations, graphical approaches, and tests.
Student learning outcome 1.0 was evaluated 13 times this term, 2.0 was evaluated five times, and 3.0 was evaluated four times. Fall 2019 these same outcomes were evaluated 25, 7, and 11 times respectively.
In prior terms students had to demonstrate performance of the outcome at least five times in order to be deemed to have met the outcome. Due to the shortened term, this requirement was reduced to four demonstrations. Because the sample size varies from term to term, reporting the raw number of students who have demonstrated an outcome successfully four or more times would not be comparable across the terms. The four-or-more success rate was converted to a percentage of the number of students for term to term comparison purposes.
The reduction to four demonstrations had a very strong positive impact on mastery of basic statistics. There was no impact seen on student learning outcome two. Student learning outcome three was only evaluated four times, thus a student had to demonstrate mastery four out of four times including an analysis done the first week of classes.
The shortened term had a positive impact on the course average, perhaps indirect evidence that the series of open data analysis explorations in the final month of the term are more academically challenging than the homework assignments and in class tests. The term average set a record high since this statistic was first tracked twelve years ago.
Although the term ended early, the students covered the core material in an introductory basic statistics course for non-majors. The student also, by and large, demonstrated mastery of the material covered.
This term the term did end with 10.1 due to the global pandemic SARS-CoV-2 virus and the associated Covid-19 illnesses. The redesign of the course to compress new material into the first three-quarters of the term and then engage in open data exploration analysis, and presentations in the final four plus weeks paid an unexpected dividend this term - the course came within a week and half of covering all material despite ending at the start of the tenth of sixteen weeks.
MS 150 Statistics is an introductory statistics course with a focus on statistical operations and methods. The course is guided by the 2007 Guidelines for Assessment and Instruction in Statistics Education (GAISE), the spring 2016 draft GAISE update, and the ongoing effort at the college to incorporate authentic assessment in courses. A history of the evolution open data open data exploration exercises and associated presentations as authentic assessment in the course was covered in a May 2017 report.
Taneleen and Bethany presenting an analysis of dependent variable data
- Perform basic statistical calculations for a single variable up to and including graphical analysis, confidence intervals, hypothesis testing against an expected value, and testing two samples for a difference of means.
- Perform basic statistical calculations for paired correlated variables.
- Engage in data exploration and analysis using appropriate statistical techniques including numeric calculations, graphical approaches, and tests.
Midterm examination details and performance
Assessment of learning this term was based primarily on analysis of the midterm examination. Fifty-four students took the midterm examination in two sections of MS 150 Statistics, 25 in an 8:00 section and 29 in a 9:00 section of the course. The midterm presented data and asked for the following calculations to be made:
- Part I: What is the sample size n?
- Calculate the mode.
- Calculate the median.
- Calculate the mean.
- Calculate the sample standard deviation sx.
- Calculate the minimum:
- Calculate the maximum.
- Calculate the first quartile Q1.
- Calculate the third quartile Q3.
- Identify the correct histogram for the data.
- Identify the shape of the histogram.
- Part II: What is the sample size for this paired data?
- Calculate the slope.
- Calculate the y-intercept.
- Calculate the correlation coefficient r.
- Determine the strength of the correlation.
- Make a prediction based on the relationship.
- Identify limits to predictivity.
- Probability.
Databars for the data
Performance by section on the midterm and in the overall course average
The overall course average above includes test one, the midterm, homework assignments, and four data analysis presentations.
Performance differences by gender in the course and on the midterm exam
Significant gender differentials were not seen. The performance difference by gender seen on the midterm examination did not rise to statistical significance.
Student learning outcome performance over multiple terms
The introduction of Schoology Institutional in January 2018 has made possible tracking of performance based on student learning outcomes. Prior to January 2018 Schoology Basic permitted the entering of student learning outcomes, but the Basic version does not provide access to the Mastery screen. Once the college adopted the institutional version, however, Mastery data from as far back as the instructor measured against student learning outcomes becomes available. Data across seven terms is reported for the following three learning outcomes:1.0 Perform basic statistical calculations for a single variable up to and including graphical analysis, confidence intervals, hypothesis testing against an expected value, and testing two samples for a difference of means.
2.0 Perform basic statistical calculations for paired correlated variables.
3.0 Engage in data exploration and analysis using appropriate statistical techniques including numeric calculations, graphical approaches, and tests.
Student learning outcome 1.0 was evaluated 13 times this term, 2.0 was evaluated five times, and 3.0 was evaluated four times. Fall 2019 these same outcomes were evaluated 25, 7, and 11 times respectively.
Mastery standards spring 2020
In prior terms students had to demonstrate performance of the outcome at least five times in order to be deemed to have met the outcome. Due to the shortened term, this requirement was reduced to four demonstrations. Because the sample size varies from term to term, reporting the raw number of students who have demonstrated an outcome successfully four or more times would not be comparable across the terms. The four-or-more success rate was converted to a percentage of the number of students for term to term comparison purposes.
Performance on student learning outcomes over multiple terms.
Long term course average and standard deviation
The shortened term had a positive impact on the course average, perhaps indirect evidence that the series of open data analysis explorations in the final month of the term are more academically challenging than the homework assignments and in class tests. The term average set a record high since this statistic was first tracked twelve years ago.
Although the term ended early, the students covered the core material in an introductory basic statistics course for non-majors. The student also, by and large, demonstrated mastery of the material covered.
Comments
Post a Comment