Moodle impact of include outcomes in aggregation
In Moodle outcomes are connected to monotonically increasing scales and are not included by default in gradebook calculations.
Outcomes cannot be added to rubrics and are separately marked from rubrics.
The gradebook totals with the display set to the default Real setting and outcomes not included, the scores display with the default setting are raw scores.
The Course grade settings can be used to display the percentage.
From the Grader report individual categories (provided categories have been set up) can be edited to include outcomes.
To include outcomes, select the Include outcomes in aggregation checkbox. For this exploration only the laboratory reports have had the outcomes included in the aggregation.
The help bubble on this is a "Warning Will Robinson, Danger!" caution about potentially unintended consequences.
The gradebook totals after enabling inclusion of outcomes.
In these side-by-side comparisons with outcomes disincluded in columns one and three, outcomes included in columns two and four, the net impact has been negative. The total points have risen because the outcomes have points attached to them, the overall percentage has fallen for all but the student with 100%.
Student number one has fallen from a D to an F. Student number two from a B to a C. Student number three remains at 100%. Student number four has dropped from an A to a B. Student number five did not experience a grade impact.
With outcomes included in the aggregation, there are percentages displayed. Here the probable source of the negative impact can been seen. With the exception of Optimal, the other ratings will reduce any grade above 66.67%.
The above display is confusing. The scale in use is a No-evidence, suboptimal, sufficient, optimal scale.
Whatever the standard scale might be, that option is not enabled for use, even when setting up a new scale. There is no ability to set the score that a rating has. Moodle uses a monotonically increasing scale from one by default. There is a setting restricted to Moodle admins that permits the starting of scales from zero, but that too will generate a monotonically increasing set of integer values. In the above scale No evidence is one point, suboptimal is two points, sufficient is three points, and optimal is four points.
That this scale is being used can be seen in the Outcomes report. If a 0-1-2-3 scale was in use, then averages above three would not be possible. The averages above three are rated sufficient, which implies a 1-2-3-4 scale is being used.
The 1-2-3-4 scale would imply that Optimal is 4/4 for 100%, Sufficient is 3/4 for 75%, and Suboptimal is 2/4 for 50%. Ideally one would prefer a 0-3-4-5 scale so that the percentages match to grades better with suboptimal being a 3/5 60% and sufficient being an 4/5 80% but this is not possible in Moodle. Moodle apparently does not intend outcomes to be used towards grades, which is a defensible philosophy albeit inconvenient in practice.
The deeper puzzle is that the Outcomes report and the Moodle documentation support the underlying existence of a 1-2-3-4 scale, but the values displayed in the grader report are clearly using a denominator of three. Optimal is 3/3, Sufficient is 2/3, and Suboptimal is 1/3. This cannot be what is being used in the Outcomes report to calculate the averages. Thus the Grader report and Outcomes report are in mathematical disagreement. Which supports the thesis that outcomes were really not intended to be used in grade aggregation, there would be, as warned, unexpected consequences.
There is one other inconsistency in the earlier image: sufficient is intended to mean sufficient mastery of the outcome, optimal is optimal mastery and is the highest possible rating. Moodle displays all rating in red with the "x" meaning that this is a failing mark. There does not seem to be a way to set a "passing" mark for scales so that these display in green. When setting up a new scale there is no option to specify what a passing mark will be.
Students will have to be cautioned to ignore the color of these outcomes ratings.
Moodle Reports includes a Competency breakdown but no Outcomes breakdown
The handling of learning outçomes is one of those Moodle puzzles. Outcomes have been a important component of the United States higher education system since at least Barr and Tagg's 1995 From Teaching to Learning — A New Paradigm For Undergraduate Education paper. This author first encountered student learning outcomes in a curriculum development course in the fall of 1988.
The outcomes report lists neither individual student performance nor overall course level performance. The report is assignment by assignment, quiz by quiz. This report does not include a download option.
Moodle was first released in 2002, yet the handling of outcomes appears to problematic in 2025. Perhaps this is a reflection of the origins of the package in Australia and the emphasis on competencies rather than outcomes. In places, Moodle documentation suggests that competencies are preferable to the use of outcomes. Which might explain why there is a Competency breakdown report but no direct tools for generating individual student performance against course learning outcomes. The complication for schools accredited under regional accreditors such as the Accrediting Commission for Community and Junior Colleges of the Western Association of Schools and Colleges is that reporting on student learning outcomes is integral to accreditation.
Eligibility requirements for accreditation
11. Student Learning and Student Achievement
The institution defines standards for student achievement and assesses its performance against those standards. For each program, the institution publishes the program's expected student learning outcomes and any program-specific achievement outcomes. Through regular and systematic assessment, it demonstrates that students who complete programs, no matter where or how they are offered, achieve the identified outcomes and that the standards for student achievement are met.
12. General Education
The institution defines and incorporates into all of its degree programs a substantial component of general education designed to ensure breadth of knowledge and promote intellectual inquiry. The general education component includes an introduction to some of the major areas of knowledge. General education courses are selected to ensure students achieve comprehensive learning outcomes in the degree program. Degree credit for the general education component must be consistent with levels of quality and rigor appropriate to higher education.
Accreditation standards
2.2. The institution, relying on faculty and other appropriate stakeholders, designs and delivers academic programs that reflect relevant discipline and industry standards and support equitable attainment of learning outcomes and achievement of educational goals.
Emphasis by the author. As result of the standards, course outlines specific learning outcomes, not competency frameworks. And while instructors can add learning outcomes, only Moodle admins can set up competency frameworks and competencies.
The gradebook can be exported as a spreadsheet, and outcomes are exported as rating names. This allows a function to be used in the exported spreadsheet to assign the desired rating values.
=IF(ISNUMBER(H2),"",IFS(H2="No evidence",0,H2="Suboptimal",3,H2="Sufficient",4,H2="Optimal",5,H2="-",""))
The Moodle gradebook outputs null values as a physical hyphen. The number check is to resolved a #N/A error code that appears when the function is filled across columns with numeric scores. At present the spreadsheet is 213 columns wide and has 6236 non-empty cells.
Note that columns K, L, M were manually inserted and K2 contains the function seen above
Each assignment will generate outcomes data thus a set of AVERAGE functions across split columns will have to be used to produce the overall average per student for a given outcome.
=AVERAGE(DI2,DM2,DQ2,DU2,DY2,EC2,EG2,EP2,EV2,EZ2,FG2)
Once that is done, then those averages can be used to produce Nuventive Improve input as per an earlier article.
The split column average functions are in the yellow cells, SC130.2 has not been evaluated yet
This process is significantly more laborious and prone to column level errors than having a mastery gradebook with export of the individual students averages already calculated, and the process does require dancing with spreadsheets.
The one caveat would be that this would become an unmanageable amount of workload if specific learning outcomes were included. Nuventive Tracdat data entry should be course level learning outcomes, not specific learning outcomes. Specific learning outcomes should aggregate to course learning outcomes, not directly to program or institutional learning outcomes. Course learning outcomes should aggregate only to program level outcomes. Program level outcomes should then aggregate to institutional learning outcomes.
Restoring the disinclusion of outcomes in the aggregation restores the gradebook to default state. Given the mathematical inconsistency and the negative impact of outcomes, the default disinclusion of outcomes in aggregation has to be a recommended setting.
There could be ways to game the monotonic scale issue by using a No evidence, null01,null02,null03,Suboptimal, Sufficient, Optimal scale so that the gradebook would display percentage out of five and the Outcomes report would display averages out of six (maybe), but that could have its own unintended consequences. Post-export processing of the gradebook appears to be a preferable approach.
What an outcomes learning mastery gradebook would look like with both course and individual averages reported on five point scale with mastery set at four points
Comments
Post a Comment