First Advisor

Williams, Mia Kim

Document Type

Dissertation

Date Created

12-1-2015

Department

College of Education and Behavioral Sciences, Educational Technology Programs, Ed Tech Student Work

Abstract

This study examined perceived effectiveness, usability, and motivational characteristics of using animated role playing situational simulations for Air War College distance learning (AWC/DL). The AWC/DL curriculum provides education to senior military officers who are geographically dispersed around the world with varying degrees of internet connectivity, creating specific limitations as to what methods of instruction are viable. An additional challenge is the very high student-to-teacher ratio (620:1). Traditionally, the AWC/DL program relied on text-based readings and tests to teach and evaluate students. Simulations might provide a realistic and valuable augmentation to the curriculum. The key potential advantages of adding simulations are increased motivation and better transfer of learning. However, the key disadvantages are the large expenditure of both time and money to develop simulations. The AWC/DL incorporated their original cultural simulation (OS) into the curriculum in January 2008. A second simulation entitled Visual Expeditionary Skills Training (VEST) was added in 2011 as an alternative to the original simulation. Most research on games and simulations analyze younger groups of students, whereas this research focused on AWC/DL students who are typically in their mid-30s to mid-40s. The geographical diversity, age range of the students, potential benefits from simulations, and high costs for creating simulations all justify research in this area. This study surveyed students who completed either OS or VEST. Students rated perceived effectiveness, usability, and motivation using Likert-scale questions. Motivation questions utilized Keller's (2010) 36-item Instructional Materials Motivation survey. Additionally, completion codes, reflecting choices students made in completing the OS, were analyzed. A total of 1,192 surveys and 2,671 simulation completion codes were analyzed using factor analysis, MANOVAs, stepwise discriminant analysis, and chi squared association analysis. Females generally reported lower levels of video game experience than did their male counterparts. The study found statistical significance between usability and gender as well as between usability and video game experience. Males and experienced video game users seemed to find the simulation more usable. However, the estimated effect size was small (2%). The analysis found no evidence of an interaction between gender and video game experience. The study did find significant associations between gender and the choices made during the simulation. Additionally, 1,871 comments from open-ended questions were analyzed and although there were issues with both simulations, students tended to view simulations as good learning tools. The OS required users to load the software on their computer, resulting in numerous technical issues. Furthermore, the structure of the OS led some students to be "caught in an endless loop," resulting in frustration that was specifically cited in 10.1% of surveys. The OS comments provided insight on the importance of how instructions are conveyed, how simulation progress is displayed, and the importance of making controls intuitive or automated. The VEST simulation was internet dependent and received low ratings from students in bandwidth restricted locations. However, those with robust internet connections generally found the simulation engaging and valuable. Both simulations clarified the challenges of using computer simulation in academically isolated and technologically diverse environments.

Keywords

Effectiveness, Military, Motivation, Role-playing, Simulation, Usability

Extent

337 pages

Local Identifiers

Armstrong_unco_0161D_10446.pdf

Comments

Fall 2015 Graduate Dean's Citation for Outstanding Thesis, Dissertation, and Capstone

Rights Statement

Copyright is held by the author.

Download

Share

COinS