I/ITSEC 2006 - HUMAN PERFORMANCE

Performance- Based Advancement Using SCORM 2004 2006 Paper No. 2697 Elizabeth Biddle, Ph.D., CMSP & Bruce Perrin, Ph.D. The Boeing Company Orlando, FL; St. Louis, MO William Y. Pike & Dean Marvin Joint Advanced Distributed Learning Co-Laboratory Orlando, FL The Advanced Distributed Learning (ADL) vision encompasses the use of scenario-based simulations to provide a rich environment for training complex tasks. At the same time, it introduces a complex assessment environment, which creates challenges in the accurate and efficient diagnosis of student needs as frequently student behaviors can be interpreted in several ways. Diagnosing student learning needs, consequently, becomes problematic. Unfortunately, there are currently no best practice guidelines for extracting and making use of performance data from a simulation-based training environment. However, methods that address these challenges are required for the successful integration of simulation-based training into the ADL Initiative. The research described in this paper is investigating the development of a scenario-based performance assessment method that leverages the Shareable Content Object Reference Model (SCORM), while using information on trends to isolate individual learning needs. Specifically, SCORM 2004 specifications enable a single Shareable Content Object (SCO) to be linked to (i.e., to set a value of or the status of) multiple learning objectives. Although the potential impact of this capability for assessment has not been widely recognized to date, it provides a means to interpret relatively complex responses in scenario-based training in terms of all of the learning objectives that may be implicated by a given action. The methods developed under this project will support changing the measures that reflect a student’s mastery of the underlying learning objectives as a result of study, practice, and forgetting. ... This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Use of Pilot Modeling Techniques to Define Deployable Simulator Metrics 2006 Paper No. 2601 Gary George Ph.D. P.E. GeorgeCo Enterprises Binghamton, N. Y. Peter Crane Ph.D. Air Force Research Laboratory Mesa, AZ. Frank Cardullo SUNY Binghamton Binghamton, N. Y. Roy Latham CGSD Corporation Mountain View, Ca When designing a vehicle simulator there are many possible variations in configuration. There are, for example, a great variety of motion and force cueing devices, and an even great variety of visual systems, architectures, each configurable with varying fields of view and resolution. Simulation designers always want the greatest value within the budget for the system, and in the case of deployable simulators, designers want the best system within the constraint of reasonable portability as well. This portability allows joint training on demand in theater rather than a schoolhouse or training base. A fundamental task is that of determining if one simulator configuration provides better pilot performance than another for a given application. To determine if a simulation training method is effective, one or more metrics must be identified to characterize the fidelity of the simulation. … This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Fidelity Trade-Offs for Deployable Training and Rehearsal 2006 Paper No. 2840 Brian T. Schreiber Lumir Research Institute Tempe, AZ Winston Bennett, Jr. Air Force Research Laboratory Mesa, AZ Sara Elizabeth Gehr The Boeing Company Mesa, AZ Distributed Mission Operations (DMO) training frequently relies on simulation to accomplish training objectives. Fidelity, in its broader definition and interpretation, encompasses both physical attributes (e.g., ergonomics, switches, symbols, etc.) and functional attributes (e.g., dynamics, models, exercising appropriate cognitive skills, etc.). On a continuum of possible “fidelity levels” what degree of physical and functional fidelity, however, constitutes “high fidelity” or “low fidelity?” What standard is employed and measured against to assign such labels? And, most importantly, what training trade-offs exist when sacrificing higher fidelity for lower cost? That is, in efforts to lower costs (and therefore fidelity), what training experiences are most sacrificed and how is that documented? In this paper we outline a method for evaluating simulation fidelity based upon a comprehensive list of warfighter-defined experiences critical to performing his/her job—that is, a proposed approach upon which simulation systems could be judged and compared. This proposed warfighter-centric approach leverages two credible processes/products already in existence, the Mission Essential Competencies (MECs) and Dash One Emergency Procedures (EPs). During the MEC process, operational warfighters determine the critical list … This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Effect of Tactile Feedback on Unmanned Aerial Vehicle Landings 2006 Paper No. 2849 Donald T. Aretz, Terence S. Andre and Christopher A. Brenaman United States Air Force Academy Colorado Springs, Colorado Brian P. Self California Polytechnic State University San Luis Obispo, California The U.S. military has developed and employed unmanned aerial vehicles (UAVs) in various operations since the late 1990s. Feedback from the UAV operating station is vital for the pilot to control the UAV. The current predator UAV control setup has a monitor that shows the pilot a map and another monitor that displays a nose camera view combined with a heads-up display (HUD). Since operators control the UAV from a ground control station that is remote from the actual aircraft, the pilot is not provided with the normal auditory, vestibular, and proprioceptive feedback they would receive in the actual aircraft. Because of the remote operation of the UAV, they are also not immune to mishaps. In fact, the landing task alone contributes up to 22% of all UAV mishaps. The purpose of this study is to assess the potential for a vibrotactile vest to provide tactile feedback to UAV pilots while landing. This study used two groups of participants. The first group used the UAV simulator as it is currently employed with no tactile vest. The second group used the UAV simulator while also using the tactile vest. We examined the learning performance of each group by measuring the number of trials it takes to obtain a passing landing score for the initial condition each participant is assigned. After obtaining a passing score, the participant switched to the other condition (vest or no vest) and completed an additional three trials. We then used the root mean square (RMS) error from an optimal flight path to analyze the impact of initial training with and without the tactile vest. This study included 30 participants with 15 participants assigned to each group. Participants were drawn from United States Air Force Academy cadets in an upper-division leadership class. … This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Effects of Interface Design and Input Control Method on Unmanned Aerial System Operator Performance 2006 Paper No. 2882 John Neumann University of Central Florida Orlando, FL Paula J. Durlach U.S. Army Research Institute SSRU Orlando, FL Although unmanned aerial systems (UAS) carry no pilot on board, they still require humans to plan and execute missions, and to interpret the sensor information they provide. Depending on the level of system autonomy and the nature of the assigned mission, humans may be required to control mission execution remotely and conduct on-line image interpretation. This will be especially likely for small unmanned aviation systems used for local, real-time reconnaissance in support of military operations in urban terrain. The need to quickly and reliably train soldiers to control small UAS operations demands that the human-system interface be intuitive and easy to master. In this study, participants completed a series of tests of spatial ability and were then trained (in simulation) to teleoperate a micro-unmanned aerial vehicle (MAV) equipped with forward and downward fixed cameras. Three aspects of the human-system interface were manipulated to assess the effects on control mastery and target detection. One factor was the input device. Participants used either a mouse or a specially programmed game controller/joystick (similar to that used with the Sony™ Playstation 2 video game console). A second factor was the nature of the flight control displays as either continuous or discrete (analog v. digital). The third factor involved the presentation of sensor imagery. The display could either provide streaming video from one camera at a time (in which case the user would have to manually switch between the 2 available camera views), or present the imagery from both cameras simultaneously, in separate windows. Dependent variables included: 1) time … This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Mobile Performance Support/e-Learning for British Army Recovery Mechanics 2006 Paper No. 2688 Steve Jarvis VEGA Group plc Welwyn Garden City, United Kingdom Dr David Swift Directorate of Individual Training (Army) Wiltshire, United Kingdom This paper describes an innovative study to develop and trial a dual mode, mobile Electronic Performance Support System (job aid)/e-learning solution for emergency procedures relating to military vehicle movements and recovery actions in the context of Peace Support Operations. On the critical path for achievement of this aim was the need to address empirically based concerns over the usability of COTS mobile e-Learning devices for the military personnel in question. The paper also highlights the scope for such a device to effectively deliver both technical (equipment related) and non-technical (e.g. cultural) knowledge to the soldier at the point of need. The study responds to research, which indicates the need for non-technical knowledge requirements to be addressed for deployed troops. It also embraces the aims of the UK Defence e-Learning Strategy and Vision to provide e-Learning to anyone, anytime and anywhere on demand. By such means the study seeks to enhance both operational effectiveness and cost effectiveness of Peace Support Operations. This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Information Distribution to Improve Team Performance in Military Helicopter Operations: An Experimental Study 2006 Paper No. 2859 Brian K. Sperling, Ph.D. Department of Systems Engineering United States Military Academy, West Point, New York Amy R. Pritchett, Ph.D School of Industrial and Systems Engineering Georgia Institute of Technology, Atlanta, Georgia Changes in task requirements and resulting system capabilities have led to the addition of crewmembers, information displays, and monitoring and coordination requirements in many domains. The overarching objective of this experimental study was to test whether providing task relevant information to individual team members in a time critical environment, while limiting their access to non task-relevant information, would change team interactions by developing complementary team mental models and consequently, improve performance. The results of this experiment provide supporting evidence for this hypothesis and give insight into a new understanding of how the distribution of information among team members effects the development of shared expectations and information requirements, team and individual performance, and communications that have not been empirically documented elsewhere. This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Human Performance Modeling and Simulation for Launch Team Applications 2006 Paper No. 3014 Cary J. Peaden, Stephen J. Payne, Richard M. Hoblitzell Jr. NASA Kennedy Space Center, FL Nils D. LaVine, Timothy M. Bagnall Alion Science and Technology Boulder, CO This paper describes ongoing research into modeling and simulation of humans for launch team analysis, training, and evaluation. It is focused on current and future launch team operations at Kennedy Space Center (KSC). The paper begins with a brief description of existing KSC launch team environments and procedures. It then describes the goals of new Simulation and Analysis of Launch Teams (SALT) research. The majority of this paper describes products from the SALT team’s initial proof-of-concept effort. Products include a task analysis and discrete event model and simulation of launch team performance during the final phase of a Shuttle countdown and a proof-of-concept training demonstration of launch team communications. In the proof-of-concept training demonstration the trainee plays any one role and the computer plays the roles of all other teammates. After the discussion of initial products, this paper describes possible next steps for the research team and provides conclusions. This research is sponsored by the National Aeronautics and Space Administration (NASA) Office of Safety and Mission Assurance (OSMA) and the NASA Exploration Program. It could provide significant value to any spaceflight program that utilizes communicating teams in recurring operations. This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
An Information Processing Taxonomy for FCS Task Analysis 2006 Paper No. 2618 David Olsen Dynamics Research Corporation Judith Converso, Ph.D and Carla Hutchinson, Ph.D Computer Sciences Corporation Chris Graves Human Resources Research Organization Information centric systems, such as the Future Combat Systems (FCS), require leaders and battle staff members to perform activities that span an information processing continuum. These activities are primarily cognitive in nature. This continuum begins with the Soldier accessing critical information from the battle command network. The information is analyzed and managed to enable the leaders and/or battle staff members to gain insight into a multitude of events shaping the operational environment. With the necessary insight, a course of action is planned in a collaborative fashion. As the course of action is implemented the leaders or battle staff members seek acknowledgement of the status of the action and distribute pertinent information resulting from the action. The use of a taxonomy to define an information processing continuum is a new and evolving approach that is being used by three teams conducting task analysis for leaders and battle staff members for FCS equipped Brigade Combat Teams (BCT). The taxonomy was developed by examining the activities that are part of the operational architecture for the weapon system. The taxonomy is being used in the development of task performance steps in preparation for the design of training products for FCS. By using the operational architecture as the foundation for the taxonomy two exceedingly important outcomes are being realized. First, the language of the FCS engineer is being merged with the language of the FCS BCT Soldier. Second, gaps in what the Soldier expects… This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Representing Context in Simulator-based Human Performance Measurement 2006 Paper No. 2856 Webb Stacy, Ph.D., Maj. Matt Puglisi, USMCR, and Craig Haimson, Ph.D. Aptima, Inc. Woburn, MA & Washington, DC Danielle C. Merket NAVAIR Training Systems Division Orlando, FL During a deliberate attack on an insurgent-held city, a Marine infantry company receives fire from a small building next to a mosque. What should the artillery Forward Observer (FO) do? The answer depends on context. If the fire coming from the building causes casualties, the FO should conduct an Immediate Suppression mission. If the insurgents’ fires do not have any effects on the Marines below and they can take cover, the FO needs to formulate a course of action with the company commander. How would we measure FO performance in simulator-based training for this scenario? It’s not enough simply to take obvious measurements like target location error or target/ammunition combination. We must have an understanding of the FO’s context, and measure and assess the FO’s performance accordingly. The performance measurement infrastructure in the training environment must support these activities. In this talk, we discuss a formal representation of context for human performance measurement in immersive training environments and how that representation fits into an innovative language for expressing those measurements, Human Performance Measurement Language (HPML). We show how context plays a role both as triggers for measurements and as key information for assessments, and demonstrate a method for convenient elicitation of context information from expert instructor/operators. We provide illustrative examples of training… This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Culture Representation in Human Reliability Analysis 2006 Paper No. 2897 David Gertman, Steven Novack, and Julie Marble Idaho National Laboratory (INL) Idaho Falls, Idaho Understanding human-system response is critical to being able to plan and predict mission success in the modern battlespace. Commonly, human reliability analysis has been used to predict failures of human performance in complex, critical systems. However, most human reliability methods fail to take culture into account. This paper takes an easily understood state of the art human reliability analysis method and extends that method to account for the influence of culture, including acceptance of new technology, upon performance. The cultural parameters used to modify the human reliability analysis were determined from two standard industry approaches to cultural assessment: Hofstede’s (1991) cultural factors and Davis’ (1989) technology acceptance model (TAM). The result is called the Culture Adjustment Method (CAM). An example is presented that (1) reviews human reliability assessment with and without cultural attributes for a Supervisory Control and Data Acquisition (SCADA) system attack, (2) demonstrates how country specific information can be used to increase the realism of HRA modeling, and (3) discusses the differences in human error probability estimates arising from cultural differences. This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Automated Feedback and Situation Awareness in Net-Centric C3 Systems at Varying Difficulty Levels 2006 Paper No. 2606 John S. Barnett & Larry L. Meliza U.S. Army Research Institute Orlando, FL Jennifer M. Ross University of Central Florida Orlando, FL Net-centric command, control, and communications (C3) systems provide vital information to military commanders and increase their situation awareness (SA) of the battlespace. However, the amount of information presented and the dynamic nature of that information often makes it difficult to focus attention appropriately. Immediate feedback, in the form of automated alerts, can help direct the user’s attention to important events and can be a valuable tool both for training and during military operations. Research in other areas (aviation) has shown under certain conditions automated alerts improve SA, but under other conditions alerts can interfere with SA. Theory suggests workload may be a factor. The present research sought to determine how automated feedback and workload affects SA for users of net-centric C3 Systems. In the experiment, participants viewed a laptop computer showing a simulated C3 display running a typical combat scenario, and were asked to look for and remember key events. A software application called SHIELD (System to Help Implement and Empower Leader Decisions) provided automated alerts of potentially hazardous events, such as violating unit boundaries or approaching minefields. Each participant completed two trials, one in the experimental condition and one in the control condition, with the order assigned randomly. In the experimental condition, subjects were alerted to certain situations. In the control condition, no alerts were provided. The participant’s SA and workload were measured periodically during each trial. SA and workload scores were compared across the experimental and control conditions. Results showed that immediate feedback increased SA for participants with… This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Next Generation Aircraft Training; Good Pilot –Good Instructor? 2006 Paper No. 2629 Tam Gipson, David Bowman Air Affairs (UK) Ltd Yeovil, Somerset, UK Wg Cdr Ian BraynSmith Flying Training Development Wing RAF Cranwell, Sleaford, Lincs, UK The failure to fully consider the requirements of instructor pilots imposes an unacceptable risk on the quality of training and the success of student pilots. The skills required to ‘fly’ an aircraft are only a sub-set of the skills required to ‘instruct’ and evidence indicates that highly competent pilots do not necessarily make competent instructors. Instructor pilots need to be given training on how to create the most effective learning environment from which students can achieve their highest possible standard. With the introduction of next generation aircraft, the instructor pilot also needs to have the ability to analyse and assess a student’s psychomotor and cognitive skills performance. This paper draws on two separate studies undertaken to investigate the selection, training and development requirements of next generation aircraft instructor pilots. Study one, conducted by Air Affairs (UK) Ltd., analysed the human-human and human-equipment interfaces of instructor pilots, as part of a UK Ministry of Defence (MoD) Training Needs Analysis (TNA) process. This included the development of an Operational Task List (OTL), together with the identification of associated… This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Undergraduate Boot Camp: Getting Experimental Populations Up To Speed 2006 Paper No. 2976 Roberto Champney, Laura M. Milham, Meredith Bell Carroll, Kay M. Stanney, David Jones Design Interactive, Inc. Oviedo, FL A struggle common across many researchers is the definition of a process to systematically train undergraduates to a degree of competence on experimental tasks. This process becomes more critical when paired with the notion that much data resultant from academic research is utilized to either generalize or apply findings to a much larger population. Many researchers have criticized the practice of using undergraduates as experimental participants, claiming that there are characteristics of the undergraduate population which are different than those found in the general population (Ward, 1993). This is of particular concern in military training research in which the goal is to generalize experimental results to military populations. Further, in these settings, it is challenging to make conclusive recommendations regarding training effectiveness of systems when the experimental population may not have the requisite knowledge, skills and attitudes (KSAs) necessary for task accomplishment, or an understanding of the requisite tasks and the motivations behind mission success. This paper presents a method designed to increase the KSAs of undergraduate participants such that they more closely parallel the target population; this is accomplished through the design and implementation of an "Undergraduate Boot Camp". This methodology was applied within the Military Operations on Urbanized Terrain (MOUT) domain, and focused on providing both classroom and practical application training in which a subject matter expert (SME) instructed, demonstrated and evaluated trainee performance. As a next step, training was supplemented by detailed scenario based feedback targeted at developing self and team awareness of task accomplishment. A key aspect of the methodology is to develop a central set of competencies, incorporate them into a pre training advance organizer, developed to support rapid development of a mental model of the… This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Near-Eye Augmented Reality Tower Controller Displays: Human Performance Issues 2006 Paper No. 2518 John W. Ruffner, Ph.D., Roger R. Labbe, DCS Corporation, Alexandria, VA Steve Hoyt, NVIS, Inc. Reston, VA The primary means by which air traffic controllers in airport towers obtain information is by direct head-up, out-the-window (OTW) viewing. However, controllers spend a considerable amount of head-down time looking at flight strips, panel-mounted displays, and other information sources in the tower. The U.S. Air Force recognized that tower controllers may benefit from near-eye augmented reality (NE/AR) display technology to increase head-up OTW time and increase situation awareness (SA), especially during limited visibility conditions. Therefore, the U.S. Air Force Research Laboratory tasked us to develop a prototype NE/AR display that enhances tower controller performance and SA, and that will be accepted and used by tower controllers. We developed or adapted prototypes of (1) a video see-through held-to-head virtual binoculars display (VBD) slaved to an external pan-tilt camera, and (2) an optical see-through head-worn display (HWD). Relevant text (e.g., aircraft identity) and graphic images (e.g., runway outline), are overlaid on real-time, head-tracked video (for the VBD), or on real-time, untracked OTW scenes (for the HWD). We performed usability assessments with Air Force tower controllers using 1) user/task observation, 2) user review of an early physical mockup, 3) rapid prototyping involving user review of airfield movement scenarios with text/graphics overlays, and 4) user trial of a partially functional system. We discuss heuristics for selecting the best display concepts, design challenges and engineering tradeoffs, and the results of our usability engineering and assessment efforts. Finally, we discuss key human performance issues affecting the functionality and controller acceptance of NE/AR displays. This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Training Wayfinding Using Mixed Reality 2006 Paper No. 2581 Ruthann Savage University of Central Florida Orlando, FL Jason Daly, Glenn Martin Institute for Simulation and Training Orlando, FL The Army needs a distributed training environment that can be accessed whenever and wherever required for training and mission rehearsal. This paper describes an experiment to investigate the effectiveness of a prototype of such a system, using mixed reality (MR) presented by the Battlefield Augmented Reality System (BARS), a mobile augmented reality system with a wearable computer, and see-through head mounted display (HMD). The users’ positions were tracked wirelessly, while a graphic representation of their virtual surroundings was displayed on the HMD as they walked. Previous studies have shown virtual environments can be used to train navigation. The addition of natural movement to a type of virtual environment may enhance that training. Sixty participants were randomly assigned to three conditions and memorized directions for a route through a complex area. Participants in the map condition performed three rehearsals of the route by drawing on a printed floor plan of the environment. Participants in the walker condition executed three rehearsal traversals of the route in the actual facility. Those in the MR condition did the same wearing the BARS system which provided a realistic environment, with landmarks, waypoints and virtual footprints. A scaling factor was used, with each step in the MR environment equal to three steps in the real environment, and the environment was broken into “tiles”, much like atlas pages. Participants worked their way through each tile in succession until they completed the entire route. Transfer of training results from testing in the real environment found a significant but weak… This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
High Availability 3D Finally Arrives 2006 Paper No. 2684 John M. Brabbs, Mark S. Sokolik EDS MSCS Troy, MI David M. Korpalski EDS VRC Troy, MI In December of 2004, Adobe announced the release of Acrobat Reader 7.0 and with it the ability to view embedded 3D content within the PDF format. This multi-platform, free upgrade has created a ubiquitous 3D standard that provides the capability to view and interact with 3D data, while avoiding drawbacks that have typically hindered other 3D viewers, such as the need for a plug-in and the lack of distribution. Using the 3D PDF technology, a proof of concept was developed for the US Army PM Stryker, to demonstrate the effectiveness of the PDF as a low cost implementation of the Interactive Electronic Technical Manual (IETM). This application provides insight to the process and cost of this technology and the viability of future integration into training and maintenance systems. This paper will describe the creation process and compare and contrast other 3D delivery platforms versus the PDF, as well as what’s needed to make this 3D capability as commonly used as the format itself. To adequately represent the implementation of this concept this paper will include a demonstration of the Stryker deliverable. This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
AUTOMATING METADATA TAGGING: COMBINING MAN/MACHINE INPUT FOR OPTIMAL OUTPUT 2006 Paper No. 2483 Katrina E. Ricci, John C. Hodak, and Sandra Hughes NAVAIR Orlando Training Systems Division Orlando, FL There is an ever-increasing need to apply metadata to legacy electronic training material as well as to content currently under development. Metadata, or very simply data about data, provide an underlying description of training material. Metadata describe attributes of learning objects including, but certainly not limited to, the content itself, when it was created, who created it, and its intended purpose. This information can allow developers to search and find previously developed content in order to achieve a financial efficiency through updating or reusing existing content. Further, as the future vision of Navy training matures, metadata can help ensure that sailors receive the right training at the right time based on knowledge of an individual sailor’s needs and applicable training material. Metadata are comprised of both objective and subjective data elements. Objective elements are those that are relatively straightforward to identify. They include data such as the developer, the training title, or the revision number of the content. Subjective elements – arguably the more valuable data – more thoroughly describe the training content. However, they are subject to individual interpretation and present a potential time consuming and expensive component to generating metadata. It is very appealing, therefore, to apply automation to the process of generating metadata. Technologies are available to assist in this process. Most notably, the application of a machine learning technology, Latent Semantic Analysis (LSA), can assist in the very arduous task of identifying subjective metadata tags. … This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
ANIMATION IN PERFORMANCE SUPPORT: USE IT OR LOSE IT 2006 Paper No. 2602 John C. Hodak, Katrina E. Ricci, Ph.D., Tyson Griffin NAVAIR Orlando Training Systems Division Orlando, FL Anna Connelly Kaegen Corp. Orlando, FL There is a strongly held belief that the use of animation in multimedia instruction can enhance learning. While the research literature is somewhat mixed, animation, when used along with sound instructional principles, can have a significant impact on learning (Mayer, 2002; Hansen, Narayanan, & Hegarty, 2002). This research effort investigated the use of animations in a performance-aiding environment in order to derive basic guidance for the use of animated graphic material for delivery in electronic performance support applications. Earlier pilot research showed many participants failed to engage animations provided. Thus, the current study investigated not only possible components of animation presentation (animation, text and narration) and their contributions to task performance, but the frequency of use and control of the animation, as well. Ninety participants were assigned to one of the six conditions of a 2 x 3 (User Control by Modality) between-subject experimental design. The two levels of User Control included participant initiated animation or system initiated animation. The three levels of Modality included 1) animation alone, 2) animation with text, or 3) animation with a corresponding vocal narration of the text instruction. Participants were asked to assemble a thirteen piece wooden puzzle by following instructions delivered on a laptop computer. Dependent measures were time to task completion, accuracy of task performance, and frequency of animation use. Individual difference measures such as spatial ability and goal orientation were also collected. … This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
EPSS: Calling for Content Analysis and Information Design! 2006 Paper No. 2914 Jay Winstead, Mary Gehres and Dr. Lynne Monson Dynamics Research Corporation Denver, CO Many today know that EPSS stands for Electronic Performance Support Systems, which provide just-in-time delivery of information with minimal interruption of work flow. But how can EPSS designers achieve these twin goals? Most software applications include online help and other support tools, but the content is often disorganized with poor usability. Such problems are magnified by the complexity of performance support systems. What should EPSS designers do to avoid these pitfalls? How can they leverage solid content analysis and information design to minimize cognitive load on users while maximizing job task effectiveness and efficiency? And which technologies are best for presenting information? When should a task be supported by a WBT module vs. by designing a wizard to simplify the task and obviate the need for instruction? What goes into online help and what doesn’t? What’s the best way to integrate support tools into the computer system? Into the user interface? This paper proposes that information design is the single most important factor in EPSS design, taking precedence over the sophisticated technologies now available in this arena and their admittedly impressive capabilities. An effective framework for information design enables project teams to answer all of the questions posed above. Furthermore, it enables the designers to tie these answers back to the results of user, task, and needs analysis and forward to the selection of delivery technologies and the allocation of content to different presentation methods. The goal of this design approach is to ensure computer system users get meaningful task-based support and training that utilizes best practices in information design and state-of-the-art technologies. … This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
Networked Voice Communications Audio Balancing 2006 Paper No. 2665 Nathaniel M Napoletano Lockheed Martin Akron, Ohio Voice communications is an important part of the U.S. Air Force (USAF) Distributed Mission Operations (DMO) program, managed by the Simulator System Group (SMSG), Aeronautical System Center, Wright-Patterson Air Force Base, Ohio. Before and during DMO missions, coordination intercommunications and simulated radio communications take place between flight simulators and other training assets across the distributed DMO network. During recent missions involving the AWACS center at Tinker AFB, F-15C fighters from Eglin AFB and the F-16 Mission Training Center (MTC) from Shaw AFB, the participants found that variations in voice loudness and timbre interfered with communications intelligibility and fidelity. While networking, stability, and interfacing problems have been routinely addressed in DMO testing with code fixes and adjustments to the standards, the discipline of balancing voice loudness levels involves human perception and requires special consideration. Current alignment procedures are inadequate to address the variation in the diversity of headsets, unpredictable microphone placement, and inconsistent dynamic range compression across the many DMO Federate Systems. This paper defines a common set of audio alignment methods and tools. There is little existing published data establishing common alignment parameters for a simulated radio network. In order to develop these parameters the author performed three separate studies: Variance in Perceived Loudness of Processed and Unprocessed Voice Streams, Rate of Clipping Distortion Events in Live Radio Missions, and A Voice Channel Headroom Study. These studies revealed that differing methods of processing voice signals affect their perceived loudness and require that these signals be adjusted accordingly. The analytical approach from these studies suggests… This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
PCBased Desktop Display versus Immersive HeadMounted Display Flight Simulator Performance 2006 Paper No. 2600 Chris McClernon, Rhoshonda McGruder, Katrina Mumaw, Terence Andre USAF Academy Colorado Springs, CO In the last five years, there has been an increased emphasis on the use of 2D and 3D PCbased flight simulators for flight training. Flight simulators allow users to practice certain skills and to stay current between flights in real aircraft. In addition, affordable PCbased flight simulators may be a viable, costeffective alternative for actual aircraft flight time or fully immersive, high fidelity simulators in the future. The research reported in this study examined the differences between flight simulator training on a limited fieldofview desktop monitor and training with an immersive virtual reality headmounted display (HMD). A betweensubjects design was used with thirty participants. Half of the participants performed designated flight protocols using the desktop monitor trainer and the other half performed the same flight protocols on the HMD immersive trainer. Participants using the desktop monitor performed significantly better than those in the HMD condition on tasks that involved maintaining altitude, pitch, bank, and heading. Possible explanations for these results include the “headsup” nature of an HMD, resulting in less time viewing the aircraft instruments, and less familiarity with an HMD. Further research is needed to study performance differences between simulators with varying levels of fidelity. Research is also needed to study longitudinal effects of flight simulator training on real aircraft flight performance. This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.
User Interface Lessons Learned from Distributed Simulations 2006 Paper No. 2844 Noah Evens, Brooke Whiteford, Geoff Frank, & Rob Hubal Technology Assisted Learning Center, RTI International Research Triangle Park, NC The Army is pursuing distance learning strategies to meet Soldiers’ lifelong learning needs. One cost-effective approach to providing Soldiers with learning by doing is to develop and distribute simulation training systems. The U.S. Army Signal Center & FT Gordon is leading the implementation of this approach through a series of distributed simulations accessible via its University of Information Technology (UIT) portal. This paper reports on the many user interface lessons learned from development of a series of distributed simulation systems, mostly but not solely for the Signal Center, developed over the past six years, as well as lessons al-ready learned from several simulation systems currently being developed. These systems are designed to meet contractual requirements that they be downloadable and usable by Soldiers anywhere in the world with an AKO connection within 15 minutes, but able to run in standalone mode. The paper describes a host of issues dealing with visualization and interactivity, context and usability, navigation and tool use, and policy and technology. The lessons learned include observations, solutions, and suggestions to these issues. The paper provides guidelines for future simulation systems building on a model that considers task affordances and demands, user characteristics, and the nature of the domain. This paper is available on the 2006 I/ITSEC CD ROM. Order it from I/ITSEC'S Website.