Multiple-choice, test-based games have been the most popular educational simulation for much of history.
How Do Online Learning Games Determine If Players Achieve The Learning Objectives
We can measure, but how do online learning games determine if players achieve the learning objectives? In a new paper, researchers examine how these games distinguish between positive and negative impacts on learning.
The primary goal of online learning games (OLGs) is to help students learn by helping them gain skills and knowledge that will improve their learning. But how these games know whether students are actually learning is open to debate.
In a new study led by a Harvard University psychology researcher, a team of researchers compared the performance of students in two distinct online learning games: a visual recognition OLG called Early Learning Maps (ELM) and a simulated human/computer interaction OLG called Scratch, both of which provided their participants with immediate feedback on learning outcomes. The team found that ELM consistently improved performance on a host of learning metrics, while Scratch yielded few improvements and even losses.
“Early Learning Maps is often presented as a simple program that delivers important vocabulary training by helping children learn to read through decoding vocabulary words they see on a ‘map’ on the screen,” says David P. E. Williams, the Abel Knapp Thompson Professor of Psychology in Psychology, Behaviour, and Health and the director of the school’s Social Cognition and Human Cognition Lab. “But our data showed that for kids at various learning stages, Early Learning Maps has little lasting effect on language fluency.”
“Our research underscores the importance of evaluating ongoing learning effectiveness,” says Joelle A. Chai, a psychology graduate student at Harvard and first author on the paper. “Given the rapid rise of OLS in K-12 classrooms, we believe that our findings have important implications for school districts looking to assess how OLS promotes learning outcomes at different ages and stages of development.”
Both ELM and Scratch presented their participants with different game scenarios. In Scratch, the user interacts with a mysterious coder who crafts words using color blocks that are essentially shapes (colors do not take on meaning the way letters do). In ELM, the user has to match letters or words to outline objects on a digital map.
Both of these OLS scenarios, which the study authors call scenerios, were designed to prompt participants to solve problems by matching shapes and words. ELM contained the most negative learning outcomes compared to Scratch, which contained the most positive learning outcomes. This result was driven not only by greater negative outcomes in ELM, but also by larger statistical errors for ELM that allowed for smaller improvements. For example, ELM succeeded at only 22 percent of the time, whereas Scratch succeeded at only 16 percent of the time.
Additional research will hopefully provide new insights to help teachers and students understand the best way to select one of these OLSs. “We have tested a wide variety of OLSs, and as we study this newer trend, we will be looking at how different OLSs develop over time, how the types of learning assessments they use can inform learning and teaching, and how they can be evaluated for effectiveness,” Williams says.
Chai adds, “What’s most exciting about OLSs in schools and classrooms is their potential to be used as a way to identify students who need additional instruction and to match each student with teachers who can work with them best.”
Some applications of the ELM and Scratch findings may help meet the need for early education programs in impoverished communities to build upon the resources in their classroom to provide both classroom and online instruction for students who don’t have access to alternative content, says Karson Zhang, a postdoctoral fellow at the Harvard T.H. Chan School of Public Health and the author of the paper.
“The findings from this study suggest that as a viable solution to close the gap of access between students from low-income families and those from higher-income families, schools that use OLSs can offer both online and classroom instruction,” Zhang says. “The findings may also have implications for creating new OLSs that are made less reliant on immediate feedback.”
Chai, J.R. & Zhang, K. (2018). Placebo effect at optimal level for persistence in earlier learning stages: Evidence from experimental work with Early Learning Maps (ELM) and Scratch (CL) online learning games. Psychological Science, 9(3), 165-170. DOI: 10.1177/0097343912268514