Videogame Assessment Tools Hit Their Mark
Date Posted: Wed Sep 10 2008
As the NASA-funded component of the NASA-sponsored Classroom of the Future's serious games research draws to a close, pilot study and preliminary analyses have shown that the embedded assessment tools the Classroom of the Future™ produced for NASA eEducation do, indeed, measure players' learning and perceived experience.
The Classroom of the Future created the
Selene videogame in 2007 as part of an investigation into how students can best learn NASA science through videogames.
Analyses used data from the phase one collection, May 16, 2007, through Aug. 1, 2007. These data represent the gameplay of 96 participants located online throughout the country. The research study used a randomized-groups design with repeated measures. This allowed researchers to manipulate player experience to study changes in players' learning and perceived experience. The research environment randomly assigns participants to groups that play Selene. Half of the participants watch round one gameplay. Half play round one. Both groups play Selene round two. Additionally, half of the players watch instructional videos after playing Selene round one. Half watch after completing Selene round two. Researchers used the embedded assessment tools to measure how the sequence and type of activity (playing or watching) affected player perceptions and player learning.
Selene features a timed report embedded assessment tool that measures changes in learning every 10 seconds. Using the timed report, researchers identified when significant learning occurred for Selene players. These results are preliminary. Analyses using the Classroom of the Future tool for perceived experience, the flowometer, are more complete.
The flowometer measures alignment between players' perception of current activity and optimal experience. Optimal experience is known as "flow." Game design theory and practice aim for games that engage players in flow experience. One goal of the Classroom of the Future is to define design principles for flow-inducing instructional games.
Flowometer data identified the segment of the Selene game that inspires the most flow. The flowometer consistently discriminated player activity (playing or watching). Participants consistently reported higher challenge while playing and higher skill while watching. Although players and watchers reported significantly different round one experiences, the groups showed the same round two learning trends. More importantly, across all conditions players demonstrated the same large jump in learning at the same round two Selene game subsegment.
The learning jump concerns the underlying gameplay mechanic. Selene was designed to simulate the fundamental physics and chemistry systems that science has developed to explain the universe. Timed report data show that Selene players progress toward the learning goal more than 75 percent of the time.