In the Press - About my work

THINKING OUTSIDE THE ASSESSMENT BOX

by Michael S. Overholt

on April 20, 2020

Re-Blogged from https://blogs.cofc.edu/michael-s-overholt/2020/04/20/thinking-outside-the-assessment-box/

Our kids love a show called Wild Kratts in which two brothers Chris and Martin Kratt take kids on all kinds of animal adventures. The show starts out in regular film footage before transitioning to animation. If you’re familiar with Wild Kratts, you’ll know they transition with one little phrase: “What if?” It’s tiny but essential to enabling their imaginations to think outside the human box and get to know about an animal as the animal itself.

Maristela Petrovic-Dzerdz’ article “Gamifying Online Tests to Promote Retrieval-Based Learning” behaves like the Wild Kratts: “What if we quit fretting about students cheating with their textbooks in online tests and allowed online tests to be open book? Could we still achieve retrieval-based learning?” It’s an effort in ingenuity that works like a karate chop on the tired fear we have of students looking at course material when taking online tests. She concludes that “if we shift the focus from assessment to learning, then online, open-book tests have promising applications” (36).

Here’s the skinny on Petrovic-Dzerdz’ course. She designed a course with four sections: Level 1 tests (10%), Level 2 tests (10%), Midterm Exam (35%), and Final Exam (45%). Level 1 and 2 tests were online; Midterm and Final exams were in-class. The Level 1 tests students could take up to five times. When they scored at least 80% on the Level 1, the LMS unlocked the Level 2 test which was more difficult and could only be taken once. The more testing they did, the more exposure students got to test content that could appear on the Midterm and Final Exams. The online tests were open-book; Midterm and Final exams were not. In the end, the study found that “online, open-book tests designed using gamification principles are an effective strategy for using education technology to motivate students to repeatedly engage in retrieval-based learning activities and improve long-term knowledge retention, regardless of the course delivery mode” (24).

I perceive four important takeaways Petrovic-Dzerdz’ study has to offer. The first is that she identified not only learning objectives but also behavior objectives she wanted to elicit from the students. It’s an important step that determined course design. She wanted the students to test themselves multiple times (the objective behavior), and this behavioral objective led her to design the Level 1 and 2 tests in a way that enticed students to take them as often as possible. The objective behaviors determined test parameters.

Another is that the Level 1 and 2 tests were safe places for students to make mistakes. By and large, the pressure that high-stakes testing creates leads students to reach for less ethical means of scoring high on tests. A safe space for testing was created when she designed the Level 1 and 2 tests that could be taken multiple times for the purpose of repeatedly exposing them to course content with low percentage weights attached. Level 1 and 2 tests were, by definition, low-stakes tests.

A third is the chance to “level up.” The terminology comes from gaming and means beating one level to move up to another, more difficult level. The experience is usually accompanied with a chemical reaction in the body whereby dopamine is released into the blood stream and creates the excitement and focus gamers feel. Leveling up in Petrovic-Dzerdz’ course occurred when students scored at least 80% on Level 1 and got to move on to Level 2, which they could only take once. But the real incentive for taking Level 2 tests was that it counted for 10% of their final grade. For every Level 2 test students didn’t level up on, they lost on 2% of their final grade.

Lastly, Petrovic-Dzerdz’ Level 1 and 2 system used the LMS as an actual learning management system. Instead of acting as a repository where an instructor housed grades, papers, discussions and information, students could use the LMS to manage learning. They tested themselves multiple times, received feedback, tried again. Basically, they used the LMS for the tool it is, much like they might a video game they were trying to beat.

Nevertheless, what I appreciate the most about Petrovic-Dzerdz’ article is its process. It turns an oft-mentioned fear of instructors on its head. “Let ‘em have their books. I’ll still advance their learning.” Granted, this doesn’t solve the current dilemma we have when it comes to having high-stakes testing online, but if we apply her thought process, we might come up with new practices. We only need to begin with “What if?”

Stephen Downes: Pedagogical Principles of Design for Active, Reflective and Self-Regulated Learning: “Maristela Petrovic-Dzerdz & Anne Trepanier: “Online Hunting, Gathering and Sharing: Return to Experiential Learning in a Digital Age
Oct 16, 2017 By Stephen Downes

We were historically hunters and gatherers, and more than anything, sharers, This is how we have traditionally learned, in contrast to the organized learning of industrialized societies. In the latter, people work on their own - at their own desk, doing their own work, etc. And this mode of learning has largely propagated into the digital world: learn at your own pace, etc. But this may be an even more isolating process.
By contrast: learn from experience. We will share today a couple of assignments we created for a course ‘Introduction to Quebec Society’ where we emulate the traditional forms of learning(...)”

Teaching Online with Anne Trépanier (May 2019) "I believe that learning is about mobilizing the whole individual, brains and heart... I had to reflect about my learning experience before diving into the online teaching world. It was of course a challenge to be adapting a classroom university course for the online format, redesigning all aspects of the course. This big task, done in 2013 for Introduction to Québec society and in 2014 for Introduction à la culture et à la société québécoises, was carefully supported, encouraged and inspired by MaristelaPetrovic-Dzerdz, and prepared me well to create one «from scratch» in 2018, Critical Nationalism.
Maristela and I first discussed the assumptions of dryness of the platform, then challenged the idea that online teaching was going to be easy, and elaborated on the new kind of authorship the creation of an online course demands. (...) Maristela designed activities that matched the objectives I had for the classroom experience after I agreed to explicitly decide on specific objectives and outcomes for every weekly portion of the online course. The traditional lecture is non-transferable but the teaching/reaching techniques are."
TeachOnline.ca Pockets of Innovation (July 2014): Increasing Student Enrolment, Retention, and Participation through Online Delivery of an Introduction to Québec Society Course at Carleton University "The development of the online course was a team effort, with Maristela Petrovic-Dzerdz, an Instructional Designer in the Educational Development Centre, working with Dr. Trépanier on every step of combining the content, pedagogy, design, aesthetics, organization, creativity, and accommodations for disabilities into an effective learning package. "
Carleton University: Behind the Scenes (2013): How much does a class change when it’s moved online? More than you may think. "Adapting an in-class course to become exclusively online is a bit like turning a screenplay into a movie, says one of the main people involved in the process. In both cases, there’s a lot of give and take in terms of what goes in and what stays out of the course, according to Maristela Petrovic-Dzerdz. An instructional designer for Carleton’s Educational Development Centre, Petrovic-Dzerdz helps professors convert their courses to an online-focused equivalent."
June 2013 Site of the Month: "Maristela from the EDC was extremely helpful in providing me information on the kinds of technology I could use, and for helping me realize the need to enrich my slides with more visuals." Laura Newton Miller, Carleton University

Publications/Blog Posts/In the Press

Confirm copying