Moodle and Canvas learning outcomes assessment usage and reporting

In Instructure Canvas outcome rating scales can be set to custom values. This allows the use of rating scales that can be used to mark assignments. In addition, Canvas permits outcomes to be added to rubrics, allowing assignments to be graded by the outcome rating.

Learning outcomes scales

In Canvas a custom 5-4-3-0 was deployed which produced 100%-60%-60%-0% when used in a rubric to mark an assignment. 

Setting up a scale in Moodle

In Moodle the rating scale is automatically set to be monotonically increasing from one (or zero). This produces a 25%-50%-75%-100% scale if the scale is left at the default start of one. Moodle does not directly include outcomes in rubrics, although one could conceivably add pseudo-outcomes as criterion. 

Canvas Speed grader side panel on the left, Moodle Grader side panel on the right

In Canvas outcomes can reside directly in the rubric and, at the instructor's option, contribute to the grade for the assignment. In Moodle learning outcomes cannot contribute to or be used to mark the assignment. Outcomes are located outside of the rubric and cannot be directly added to the rubric. 

Nuventive Solutions/Improve/Tracdat

In terms of reporting, the college reports learning into Nuventive Improve via statements such as, "23 (16 females; 7 males) out of 24 students (96%) achieved near mastery or better of this CSLO."

Canvas Learning Mastery Gradebook

Canvas provides support for data entry into Nuventive by reporting individual student performance × individual course learning outcome in the Learning Mastery Gradebook. The Learning Mastery Gradebook can also be exported as a CSV file retaining individual performance on each learning outcome.

Moodle outcomes report

The Moodle Outcomes report displays averages aggregated at the course level, not at the individual student level. 

Moodle Grader report

The Moodle Grader report can include individual student performance on an individual assignment. In the above course these assessments are currently done on every laboratory report and the aggregate performance determines student mastery. 

The Grader report, however, can be exported. 

Grader report export as displayed in LibreOffice.org Calc

Note that the Gender column, necessary for results entry in Nuventive, was added post hoc after export, as were columns L, M, N, and O. Columns L, N, and M convert the outcome rating to a numeric value. To maintain consistency with the scale in prior use, the following function is in L2:

=IFS(I2="Optimal",5,I2="Sufficient",4,I2="Suboptimal",3,I2="No evidence",0,I2="-","")

This process of adding columns and functions will have to be repeated for each assignment that includes outcomes evaluations. Then a set of overall average functions can be constructed to obtain an individual student average for each course learning outcome - averages that Canvas calculated internally and reported directly in the Learning Mastery Gradebook. 

Given the high workload for repeating this process across all assignments and outcomes in all courses taught by an instructor, selection of a representative sample subset of assignments for outcome assessment might be prudent. 

The results in the spreadsheet can then be used to construct the inputs for Nuventive as outlined in an earlier blog.

The one caveat is that using the 5-4-3-0 scale leads to a course learning outcome average that is different from the one reported in the Moodle Outcomes report which is based on the 4-3-2-1 scale. There is an argument to be made for shifting to the Moodle monotonic scale and using those values to determine learning outcome mastery. 

Post-script: Evaluating outcomes attached to quizzes

To evaluate outcomes that are attached to Quizzes, add the desired outcome or outcomes to the quiz when setting up the quiz. To evaluate each student go to the Grader report. Be sure to turn editing on, the drop down menus only appear in edit mode. A lengthy search of online information failed to mention this key fact.

With Edit mode enabled, drop down lists appear next to the score on the quiz allowing outcome ratings to be selected. When done, Save changes at the bottom of the Grader report.