Dr. Josh Grochow, along with Dr. Ryan Layer and Dr. Lijun Chen, has spent the last couple of years researching new designs for computer science courses that might reduce the temptation for students to simply look up answers to homework or exam problems online. He found wisdom in Cheating Lessons: Learning from Academic Dishonesty, which proposes that courses with very high stakes of performance often rely on single assessment mechanisms which ultimately set up the conditions that breed cheating. Mastery-based learning*, with several opportunities to demonstrate mastery of a concept, can create effective learning environments that foster intrinsic motivation, promote mastery, and instill the sense of self-efficacy that students need for deep learning.
Grochow discovered Sharona Krinsky’s work on Mastery-based Grading, and he and Dr. Lijun Chen decided to implement those ideas in the Spring 2020 semester. CSCI 3104, Algorithms, is a course with approximately 300 junior and senior CSCI majors, which Grochow was teaching with Dr. Chen, nine TAs, two graders, and six undergraduate course assistants (CAs). They identified 24 content standards which correspond to the learning goals for the course. At the end of the semester, the students’ grades would reflect the number of standards that they were able to master, eliminating the need for an ordinary, curved graded system. “At first we were worried about not having a curve—was everyone going to fail? What we would do if they were?" said Grochow. "But after the first midterm they were headed for a B average, and we were really happy that the majority of students ended up mastering the material and getting As.”
Each standard had at least 4 assessment opportunities: a homework assignment, a recitation quiz, a question on a midterm, and a question on the final. Students needed to show mastery twice in order to be complete with the standard. If they completed a standard, they did not need to answer any more questions about the standard on future assessments. The grading on these assessments was rigid; even if the student received 80% of the points for a problem, they could still receive a grade of “not mastered,” if something in their solutions demonstrated missing a key idea. The professors and TAs had to be very transparent with the students about the grading process and help them get used to this different style of grading. They initially encountered some frustration from students, but by the end most students were very happy with mastery grading. "The mastery grading was super nice since it meant that I didn't have to worry about getting poor grades on homeworks or quizzes," a student in the course reported. "If I ever did poorly, but learned it later, it was super nice that my grade wouldn't be impacted. It was also really nice that we only had to do some portions of the final, which makes sense that we only need to be tested on the things we haven't learned yet."
Given that the mastery-based course design allows for students to be on their own path throughout the course, the transition to remote learning was relatively smooth. "When we needed to move deadlines, or when students missed an assignment or two because of the pandemic, we could always just give them another opportunity to demonstrate mastery if they needed it." said Grochow. The professors were aware that the lack of proctoring for online assessments would be an issue. The students were explicitly told that that they could look up anything for homework, but were advised that if they found the solution, to not post it on Chegg. They were required to write the answer in their own words. The team also created complex questions, so that a student had to illustrate the intermediate steps even if they discovered the final answer online. Grochow emphasized to the students that, in preparing for future jobs in computer science, they need to really learn the material so that they can apply it creatively in their work, ultimately allowing them to learn or develop new algorithms. He felt that this kind of coaching would motivate them to understand the material, rather than simply look up answers.
Another key element of the course design was the opportunities to “re-quiz”. Students could get another quiz opportunity on a standard by completing a reflection assignment indicating that they really understood their mistakes on the first quiz opportunity. If students were having difficulties due to the pandemic, Grochow allowed them to re-quiz without the reflection assignment. Over two-thirds of the students in the course mastered 20+ of the 24 standards, earning them an A- or A. Grochow acknowledges that mastery-based grading isn’t perfect, but it has many advantages over ordinary points-based grading, both for instructors and students. Students demonstrate mastery of the material once they’ve mastered it, rather than only once or twice the first time they see the material. “The mastery-based grading introduced this year is such an improvement over what I've heard about [previously], and I feel like I will come away from this course with not only a greater knowledge about algorithms, but a better ability to think critically about the structure and logic of programming questions,” said a student in the course.
*At a recent conference on Mastery-based Grading, the community engaged in discussions about a potential change to this terminology as the term “mastery” is a racially charged word although it is not intended as such in this context.
References:
Cheating Lessons: Learning from Academic Dishonesty by James M. Lang (Harvard University Press, 2013)
Silvia Heubach & Sharona Krinsky (2019): Implementing Mastery-Based Grading at Scale in Introductory Statistics, PRIMUS, DOI: 10.1080/10511970.2019.1700576
Further reading & resources:
Resources from the 2020 Mastery Grading Conference: