Simulation systems that pervade military training, mission rehearsal, and tactical decision making have successfully leveraged advances in computer hardware and M&S software to capture key properties of the represented world. These sophisticated systems present lifelike behavior to the user but are often difficult to use and time-consuming to learn, so that human interaction with simulation-based training applications remains awkward. This is especially true when the training system has been designed from the perspective of the backend software application rather than the human user or the cognitive task in which the user will be engaged. The result is that students are unable to lose themselves in the simulated scenario because the training system itself demands their conscious attention. For a subset of simulation-based applications the solution will involve mixed-initiative natural language dialog that lets the human mentally ‘penetrate’ the user interface to communicate directly with synthetic agents. Spoken natural language dialog in particular, lets the user control the simulation while keeping their eyes, hands, and focus of attention on the exercise and its representation in the simulation. This paper describes a software architecture for integrating mixed-initiative spoken dialog interaction into simulation systems, and illustrates one use of that architecture to integrate a dialog-enabled ITS with the multiplayer online game NeverWinterNights™.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Breaking the Natural Language Barrier in Human-Machine Interaction

Mr. Keith Garfield ,

Mr. Donald A. Washburn ,

Mr. Matthew B. Gerber ,

Mr. Anton P. Kiriwas

University of Central Florida, Institute for Simulation and Training

Orlando, FL 32826-0544

This paper presents a Natural Language Vocal Interaction (NLVI) tool developed for training of Individual Combatants (IC) in immersive virtual environments. A major disadvantage to existing immersive simulations this is that live and synthetic participants cannot interact directly in any significant fashion. Vocal interactions, specifically, are lacking even though they play a major role in normal human interaction. Vocal interaction is particularly important in the areas of Individual Combatant (IC) training. This implies that inclusion of natural language vocal interactions between live and synthetic participants will result in more effective training tools.

The Institute for Simulation and Training (IST) has produced the Voice Federate, which allows for basic command and control to be accomplished vocally. The results of this research showed that teams composed of live and synthetic IC’s can effectively participate in a single training exercise using vocal communications. The Voice Federate provides minimal two-way vocal interaction. Therefore a follow-on project, the NLVI system, was funded to allow research on extending the Voice Federate results. A full description of methods and experimental results from our prototype system will be presented.

In order to perform natural language interaction in a non-scripted fashion, the virtual entities must have a degree of situational awareness and assessment capability. We have provided mechanisms for the synthetic entities to recognize landmarks in the virtual environment based on characteristics (e.g. color or name) and also to work with concepts such as ‘near/far’ or ‘front/back.’ The synthetic entities disambiguate verbal landmark references through use of the characteristics and may query the human user if vocalizations cannot be completely disambiguated.

We will discuss how this research benefits immersive training and embedded training systems. These systems will require hands-free human-machine interactions and focus on increasing machine awareness and autonomy. Results can be generalized to systems utilizing multi-model human-machine interaction.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Employing Cognitive Task Analysis to Define Intelligent Agent System Requirements

John Burns ,

John Barnocky

Janice Giebenrath ,

Dave Grieve

Sonalysts, Inc.

Orlando, FL

Beth Blickensderfer

NAVAIR Orlando, Training Systems Division

Orlando, FL

Requirements identification is a critical step in the development of any system. When the goal is to develop automation software to offload and augment tasks previously done by humans, the requirements identification challenge becomes one of understanding what experts do, why they do it, and when they do it. To accomplish this, a cognitive task analysis (CTA) was conducted to enumerate the what, when, and why of job performance for trainers at the Joint Warfighting Center (JWFC). The results are being used to help build intelligent agent software to aid trainers associated with Category 3 level training. A total of seventeen interviews were completed and the resultant data provided a wealth of information for compilation and review. This included crucial information that helped to define the intelligent agents software. For example, the CTA revealed types of instructional feedback the trainers give and when they give it. In addition to the knowledge used directly for software development, the results also gave insight regarding the practical considerations of using CTA. The purpose of this paper is to define the CTA problem space, discuss different approaches that were considered, and describe the methodology used for conducting the CTA. The paper will conclude with results and lessons learned that demonstrate the applicability of CTA to the project at hand, as well as to the larger community of training systems.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Remote Collaboration: The Next Step in Advanced Learning Delivery

J. M. “Jodi” Johnston

Naval Undersea Warfare Center Division

Keyport, Washington

Chris W. Love

Naval Undersea Warfare Center Division

Keyport, Washington

The Navy’s Revolution in Training challenges developers of learning tools to rethink the foundation of what has previously been accepted as the traditional elements of Operator and Maintenance training. New technologies have made it possible to shift the balance of learning delivery from the schoolhouse to the operating environment. Taking the concept even further, new collaboration tools might alter the concept of how much retained knowledge a sailor must be given in advance, and how much can be delivered to the worksite at the actual time it is needed. This remote knowledge-delivery concept is a potentially powerful approach that blends on-site training and real-time assistance into a true human performance support tool. A remote technical-assistance system was developed by the Naval Undersea Warfare Center (NUWC) Division, Keyport, to explore the premise of “just-in-time” training combined with real-time, on-site assistance through use of a telemaintenance tool that integrates the student and the teacher/subject-matter expert into the system. The foundation for the system was established during collaboration between NUWC Division, Keyport, and the Human Interface Technology Laboratory at the University of Washington. The system has been further developed to bring an expert over-the-shoulder presence to the workspace through the integration of humans and Internet technologies. Wearable computers were selected as the initial workspace tools for confined shipboard use, providing the sailor with the ability to work uninhibited by bulky equipment, while receiving guidance from an expert on the other side of the globe. The current system user interface has been developed to simplify user tools and controls. Planned improvements include the use of voice commands to make the system hands-free. The evolution of this type of human performance support tool could help change the way we think about equipping the sailor with system knowledge to perform required tasks.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

HAND-HELD DIAGNOSTIC ULTRASOUND

TSgt Herman Brooks ,

Dr. Ken Levi

Air Education and Training Command Studies & Analysis Squadron

Randolph AFB, Texas

Air Education and Training Command, Studies and Analysis Squadron (AETC SAS) in conjunction with the 382nd Training Squadron (TRS) at Sheppard AFB, Texas did an assessment of innovative ultrasound technology to improve students’ sonographer skills in the training environment. The focus of this study was the Diagnostic Ultrasound course. The course does use simulators to assist students with these difficult hands-on skills. Unfortunately, there are many limitations to these simulators (unrealistically small anatomical regions for scanning, no patient movement artifacts, no respiratory variations, and none of the typical anatomical variables normally encountered for scanning experience). With the Terason‰ 2000 Handheld Ultrasound System, the students’ hands-on experience is doubled. This ultra-portable system displays its images via PC applications. The unique portability allows the Handheld Ultrasound System to be used in any environment (emergency room, tent, bedside, aircraft, etc.) and rapidly display its images for diagnosis via the hospital’s network or telemedicine channels.

Another advantage of the system’s PC application is the ability to use the ultrasound machine during the didactic portions of the course. The instructor can lecture on the anatomy of interest, then immediately demonstrate how the anatomy may appear, using real-time ultrasound scanning techniques. Two sets of subjects were studied – pre-Terason 2000 and post Terason 2000. The two groups were compared in regard to scan time, Progress Checks, Performance Checks, Performance Times, and Instructor Assists. Student critiques were also elicited for the hand-held unit. Neither in performance nor in academics did the post Terason 2000 class show any improvement. In terms of size, weight, flexibility and cost, however, the advantages of the HHDU were substantial. Also, the Terason 2000 doubled student hands-on scan time.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

APPLICATION OF TECHNOLOGY TO FUTURE ELECTRONIC DOCUMENTATION

Tyson A. Griffin

NAVAIR Orlando Training Systems Division

Orlando, Florida

Dr. Katrina E. Ricci

NAVAIR Orlando Training Systems Division

Orlando, Florida

The current trend to convert traditional paper documentation into electronic form presents a number of opportunities to employ emerging technologies in support of the maintainer through the creation of Interactive Electronic Technical Manuals (IETMs). However, as with the development of any system in which the human is an integral part, the need exists to apply sound, human engineering practices to insure the most effective and useful design.

This paper will provide an overview of several issues associated with the next generation of IETMs in the context of good Human Systems Integration (HSI) and the evolving technologies that may provide solutions. In particular, this paper will address human factors research issues associated with technologies for incorporating device models into IETMs, the use of Latent Semantic Analysis (LSA) for advanced search and navigation within IETMs, and wearable computer technology with spoken, natural language capability for hands-free, on-time performance support. Suggestions and lessons learned are provided along with preliminary research findings.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Automation Integration: Comparing Flightdeck Automation and U.S. Army Digitization

John S. Barnett

and Larry L. Meliza

U.S. Army Research Institute for the Behavioral and Social Sciences

Orlando, Florida

The purpose of this work was to identify similarities between concerns about integrating automated systems into U.S. Army units, known as digitization, and automation concerns in other areas, such as aviation, with the goal of suggesting that integrating automation and automated systems into organizations and work groups follows a common pattern. The researchers compared a taxonomy of automation problems and concerns from commercial aviation with examples of similar behaviors and attitudes observed in U.S. Army units. The observations came from field studies and previous research literature about integrating digitization into U.S. Army units. Once elements in the taxonomy which were unique to aviation were eliminated, the remaining statements were evaluated to identify similarities in U.S. Army digital units. Lists of concerns from each area were compared to find matches. Results showed that a significant percentage of the aviation automation concerns were also digitization concerns. The results suggest there may be a common pattern of attitudes and behaviors towards automation. If this pattern were identified, it could be used to introduce automated systems into work groups more easily. Suggestions for future research are included.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Visual Change Detection in Digital Military Displays

Paula J. Durlach

Army Research Institute

Orlando, Florida

Jessie Y.C. Chen

Army Research Lab/HRED

Orlando, Florida

As the army undergoes its planned transformation, it will rely more and more on humans interacting with complex visual displays. As the complexity of those systems increases, so also does the possibility that important changes in visually presented information will be missed. Research on change detection suggests that people often fail to notice changes in visual displays when they occur at the same time as various forms of visual transients, such as eye blinks or screen flashes, or scene relocation. In this experiment, we investigated change detection using the Force XXI Battle Command, Brigade, and Below display, with no more than 2 map icons ever depicted at one time. Participants were instructed to monitor the display and report changes they noticed as quickly as possible. They were also periodically instructed to conduct certain tasks with the system, such as performing a circular-line-of-sight analysis (CLOS). Changes scheduled for observation included icon appearance and disappearance, changes in icon platform, color/affiliation, or position. We analyzed the observer’s ability to detect these different types of changes, as well as the effect of performing concurrent tasks (such as the CLOS) on change detection. Our dependent measures were speed and probability of change detection. Observers were very good at detecting icon appearance, disappearance and color changes; however, they missed more than 20% of the platform changes. Detection of icon movement was dependent on the length of movement, with detection being particularly poor (more than 25% misses) when the icon was in the periphery of the map and the distance moved was small. There was a dramatic effect of performing a concurrent task. When an icon changed simultaneously with the conclusion of a concurrent task, detection dropped to less than 50%. Possible training or design solutions to overcome this type of detection failure need to be considered as the army transforms to the Future Combat System. For example, inclusion of a dedicated change detection tool could explicitly provide operators with changes that occurred during an interruption, and would greatly assist recovery of situation awareness. Instead of an operator needing to figure out what changed, the system should provide that information.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Providing Feedback on Unit Employment of Vehicular Command, Control and Communication Systems

Larry L. Meliza

U. S. Army Research Institute

Simulator Systems Research Unit

Orlando, FL

Karen J. Lockaby

Northrop Grumman

Mission Systems

Killeen, TX

Bruce C. Leibrecht

Northrop Grumman

Mission Systems

Fort Knox, KY

Command, control, and communication (C3) systems in Army combat vehicles set the stage for improved awareness of the tactical situation, enhancing the capability to implement changes in mission plans to exploit improved awareness. Reaping these benefits requires training in a collective context, because C3 is inherently a collective activity. This requirement makes it necessary for trainers to consider the stream of digital messages among nodes and operator interactions with systems—adding to the observation requirements that already burden collective trainers. Further, collective trainers must address the fact that many leaders view digital training as a detractor to unit training. The U.S. Army’s III Corps identified the need for guidance on how trainers can provide units with diagnostic feedback. The project team used observations, interviews, and literature reviews to identify fifty digital proficiency performance goals arranged into nine topical areas (e.g., apply situational awareness in maneuver decisions) to be addressed by measurement guidance. The measurement guidance developed for each performance goal included a description of its tactical significance, the echelon(s) to be observed, tactical events that should trigger observations, and data to be applied in assessing performance. This approach is designed to help reduce trainer workloads by focusing on high-payoff digital activities. In describing data requirements, the team gave special attention to the possibility of using automated systems to collect and present information regarding the content of digital messages and operator interactions with digital systems to reduce the need for trainers to track these aspects of the situation on their own. The measurement guidance evolved into laminated, pocket-sized products that were distributed to leaders in the 4^th Infantry Division and 1^st Cavalry Division to support digital training. The product is also being used to define the information displays that need to be produced by digital after action review systems addressing digital data streams and operator interactions.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

A Focus on Battle Command: Human System Integration for Future Command Systems

Dr. Carl W. Lickteig ,

Mr. William R. Sanders ,

Dr. Paula J. Durlach

and Dr. James W. Lussier

U.S. Army Research Institute (ARI) at Fort Knox, KY

The Army’s ongoing transformation to Future Combat Systems (FCS) requires an unprecedented alliance of humans and machines. Examples of the alliance include humans working with intelligent agents or “bots” for information processing and decision aiding, and with robotic entities for moving, seeing, and shooting. Particularly for battle command, advanced technologies are expected to help commanders visualize the operation, describe it within their intent, and direct subordinates toward mission accomplishment. However, the “science” of enabling technologies severely lags the “art” of battle command. Creating an alliance that actually improves, and does not impede, battle command is a human systems integration challenge for FCS. To address that challenge, the FCS C2 program is investigating future battle command concepts at the small unit level. The effort, led by DARPA and CECOM with a focus on human performance by the U.S. Army Research Institute (ARI), included an incremental series of command-in-the-loop experiments from October 2001 to March 2003. This paper summarizes research methods and findings across a programmatic series of four exploratory experiments directed at the integration of humans and machines in order to establish new paradigms for command and control. A description of the research environment and experimental design for assessing the command and control performance of a small command group with manned and robotic elements is provided. Findings are based on a battery of subjective measures and highly detailed objective measures of verbal and human-computer interaction. Overall, the methods and results provide an emerging empirical database on the C2 functions and tasks required by an FCS command group in collaboration with prototype C2 systems. More problematic conclusions on workload and training are followed by more promising conclusions on user-based involvement and proactive research, and their potential for solving problems in human system integration.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

THE EFFECT OF BANDWIDTH ON OPERATOR CONTROL OF AN UNMANNED GROUND VEHICLE

Jon French

Micro Analysis and Design

Orlando, FL

Thomas G. Ghirardelli

Jennifer Swoboda

AMSRL-HR-SE

Aberdeen Proving Ground, MD

Unmanned ground vehicles (UGV) are a critical conceptual capability for Future Combat Systems where they will be used for reconnaissance, surveillance and target acquisition. The objective of this study was to ascertain the effect of the relative loss of signal strength or bandwidth, represented by 2, 4, 8 and 16 frames per second (fps) on a UGV operator’s workload and situational awareness. A secondary objective was to measure the effect of different modes of control (joystick, voice command, joystick and voice and completely autonomous or passive control) on UGV control. The UGV was supplied by the Army Research Laboratory - Computer Information Sciences Directorate (ARL-CISD) and the tests were conducted on a field at Aberdeen Proving Ground. Participants were 22 soldiers recruited from the 16th Ordnance Battalion at Aberdeen Proving Ground. Each was randomly assigned to one of the four control device between-group conditions and experienced all four levels of the bandwidth within group conditions. Reduced signal strength had a significant impact on operator performance. The average time to navigate the course increased for 2 and 4 fps compared to 8 and 16 fps. Voice command was more difficult than the other modes of control. Visual and cognitive workload scores also were affected by bandwidth with 8 and 16 fps resulting in the best scores. Finally, the NASA Task Load Index (TLX) workload measure was improved for 4, 8 and 16 fps compared to 2 fps. The modern battlefield poses significant challenges for UGV operations due to loss of signal strength from a variety of sources such as distance, obstacles, multiple UGVs on the same channel or electronic jamming. These results suggest that optimal bandwidth for UGV operation should remain above 8 fps. Also, the passive or completely autonomous condition provided no advantages in terms of target acquisition or workload. These results will provide empirical detail for a computer model of UGV operator performance.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

KC-135 Boom Operator Training Hi-Fidelity Stereoscopic Display Technology Demonstrator

Warren C. Couvillion

Southwest Research Institute

San Antonio, TX

Aaron J. Roberts

Southwest Research Institute

San Antonio, TX

Bruce D. Newell

Eastman Kodak Company

Rochester, NY

KC-135 boom operators are responsible for flying the refueling boom trailing from their aircraft into the refueling receptacle of receiving aircraft. The task requires them to maneuver a refueling knuckle on the end of a 30-ft assembly into the receptacle of the receiver under widely varying lighting and weather conditions. The potential for damage to the receiver is high. Small scratches and dents at best compromise corrosion resistance; larger collisions can reduce stealth characteristics and mission effectiveness of coated aircraft, or even cause fires in receiver cockpits when the refueling tube punctures cockpit glass.

This paper describes a technology demonstrator built around commercial off the shelf (COTS) PC-based image generators and an innovative autostereoscopic image display system to evaluate the effectiveness of depth perception cues to support future boom operator training systems. Using infrared face imaging for eye tracking, two liquid crystal on silicon (LCOS) light-valve based projectors, an arrangement of beam splitters and mirrors mounted to a motion platform, the system steers collimated and focused stereo images at full HDTV resolution directly into each of the moving viewer's eyes. The associated image generator adjusts rendering viewpoints in both channels simultaneously, providing an intensely immersive experience with no perception of a screen or projection surface. Brightness and contrast levels are unprecedented; yet, the system's eye strain and energy exposure are substantially lower than what the viewer would receive from a large CRT or flat-screen.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Virtual Dismounted Soldier Simulation: Human Performance and Training Effectiveness

Bruce W. Knerr

and Donald R. Lampton

U.S. Army Research Institute for the Behavioral and Social Sciences

Orlando, FL

The U.S. Army requires a virtual dismounted soldier simulation capability for training, mission rehearsal, and concept development. To meet that need, the Army Research Institute, the Army Simulation, Training, and Instrumentation Command, and the Army Research Laboratory participated in a four-year research and development program to create a demonstration virtual training system for dismounted small-unit leaders. The concept was that repeated practice on realistic scenarios in the simulator, enhanced by training features and after action reviews, would build decision-making and coordination skills. Computer-controlled or semi-automated agents would represent friendly forces, enemy forces, and civilians. The goal was to produce a training system that was realistic and effective, yet required few support personnel to fill the positions of subordinates and role players. Progress was assessed during annual culminating events, series of exercises conducted with Infantry soldiers at the end of each year to obtain objective and subjective data about system capabilities and training effectiveness. This paper describes the results of the fourth and final culminating event, during which three groups of soldiers each participated in a series of eight tactical scenarios in virtual simulators over a two-day period. Ratings of unit performance and responses to questionnaires covering simulator capabilities and training effectiveness were obtained. The paper briefly reviews the key technological capabilities developed, but focuses primarily on the results of the human performance and training effectiveness assessments. Between 82% and 100% of leaders said that their performance improved as a result of the training, depending on the task. Similarly, observer ratings of squad performance on comparable scenarios were slightly higher at the end of training than at the beginning. Trends across the four years are described. Major accomplishments and remaining challenges are discussed.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Integrated Performance Measurement and Assessment in Distributed Mission Operations Environments: Relating Measures to Competencies

Thomas Carolan

Micro Analysis and Design, Inc.

Boulder, Colorado

Jean MacMillan ,

Eileen B. Entin ,

Rebecca M. Morley

Aptima, Inc.

Woburn, Massachusetts

Brian T. Schreiber ,

Antoinette Portrey

Lockheed Martin Technology Services Group

Mesa, Arizona

Todd Denning

L3 Communications, Link Simulation and Training Division

Mesa, Arizona

Winston Bennett, Jr.

Air Force Research Laboratory, Warfighter Training Research Division

Mesa, Arizona

Ongoing research at the Air Force Research Laboratory has underscored the importance of developing systematic methods for evaluating the impact of advanced training and rehearsal systems for both individual operators and teams. Given the expense associated with these systems, there is a critical need for reliable and valid methods and data to quantify the benefits of such systems. Previous attempts to quantify individual and crew performance have been modestly successful, but have not been integrative and inclusive. This paper describes the development of a competency-based and embedded performance measurement system for tracking individual and team performance in Distributed Mission Operations training events. The paper discusses the measurement challenges that must be addressed if focused assessment and evaluation of distributed training and rehearsal technologies and methods is to be accomplished. The paper describes the integrated approach taken by the research team at the Air Force Research Laboratory to develop and validate process- and outcome-oriented measures.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

REAL - TIME FORCE PROTECTION SIMULATION AND TRAINING SYSTEM USABILITY EVALUATION

Bruce McDonald

McDonald Research Associates

Winter Park, FL

Joseph L. Weeks

USAF Research Laboratory

Mesa, AZ

The Air Force Research Laboratory is conducting research and development of a computer-based simulation capability to support training in decision-making and team coordination for security forces ground operations. Simulation software supports the interaction (over a local area network) of trainees with each other and with computer-generated forces (CGF’s) that simulate behavior and communications of enemy, neutral, and friendly troops and civilians. Radio functions allow multi-channel communication among instructors, trainees, and CGF’s. Trainees can practice decision-making and team coordination in a number of scenarios with varying threat and environmental conditions. Current systems are too costly for training large numbers of security forces because they require an on-site technician to develop simulation exercises, control the exercise, serve as role players, task CGF’s, and support after action reviews. Consequently, design and development of a simulation control interface that can be directly used by instructors and trainees is an important R&D objective. For the security forces simulation capability the goal is to design a control interface that instructors can learn to use in two hours and trainees can learn to use in thirty minutes. To achieve this goal, a Windows-based control interface (with a number of video game features) was adopted as the initial point of departure. Menu options were developed to correspond to the standard mission planning procedures used by security forces and drag-and-drop functions were developed to replace menu options to contribute to usability. We have conducted an evaluation of the initial user interface with school instructors and trainees. The paper describes the emerging control interface, the approach to and outcomes from a field evaluation of the interface to include actual times required for instructors and trainees to learn to use the system and instructors’ acceptability evaluations. Lessons learned shed light on critical human-machine interface design issues for computer-based real-time training simulations.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Developing and Evaluating Advanced Technologies for Military Simulation and Training

Peter Muller

VisiTech, Ltd.

Lansdowne, VA

LT Joseph Cohn

Naval Research Laboratory

Washington, DC

Dr. Denise Nicholson

NAVAIR Orlando

Orlando, FL

As the military continues to focus on reducing the overall cost of training while increasing the efficiency with which this training is provided -without sacrificing quality- Virtual Environment (VE) systems will become an increasingly attractive alternative. While current VE development efforts typically focus on supporting the individual user, rather than a broader, integrated collaborative environment, it is precisely this type of distributed, integrated and cross-platform environment within which the military typically operates and for which such VE systems must be developed. The VIRtual Technologies and Environments (VIRTE) research program was developed, through the Office of Naval Research, to support a solid Science and Technology base from which to expand this narrow development focus. VIRTE’s research thrusts unite elements from the Modeling and Simulation communities with those from the Human Factors and Experimental Psychology fields in order to develop VE systems that are both technologically sound and performance enhancing, within a distributed virtual battlespace.

VIRTE’s component systems are based on real-world operational requirements and are designed to easily transition. The systems include: a Virtual Environment Landing Craft, Air Cushion (VELCAC), a Virtual Environment Advanced Amphibious Assault Vehicle (VEAAAV), and a Virtual Environment Helicopter (VEHelo). These domains were selected precisely because military doctrine for Expeditionary Warfare including Marine Corps Strategy 21 and Sea Power 21 rely on elements from each of the real world vehicle analogues in order to be effective, thus forming a natural collaborative, integrated environment. The virtual systems are designed with the dual purpose of supporting training at the individual level, as well as at the level of distributed, team-based events, operating within a shared synthetic battlespace. This paper will describe the human centric process that was applied to the simulation development, the role of continuous training effectiveness evaluation, and key findings from this program that should be considered for future simulation development efforts.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

THE UK HFI DEFENCE TECHNOLOGY CENTRE: BACKGROUND, RESEARCH & COLLABORATION

Robert J. Stone

University of Birmingham & Virtalis / VP Defence Limited

A major initiative to support the establishment of British Defence Technology Centres (DTCs) was announced in 2002 by Dr Lewis Moonie, Parliamentary Under Secretary of State for Defence (www.mod.uk/dtc/). The aim of the initiative is to establish world-class virtual centres of excellence, each conducting innovative science and technology research contributing to an enhanced UK defence capability, together with a valuable “spin-out” of results into civilian sectors as well. One of the first DTCs to be launched by the UK’s Ministry of Defence (MoD) focuses on Human Factors Integration (HFI) over a broad range of military domains, from synthetic environments to individual/collective training, from human performance metrics in training to multi-cultural forces, and from C4I to future telerobotic systems. The Centre is based on a virtual enterprise (coordinated via a proprietary shared working environment), primed by Aerosystems International and bringing together a consortium of established defence industries and academic institutions – Lockheed Martin, MBDA, SEA, VP Defence, and the Universities of Birmingham, Cranfield and Brunel. This paper will describe the background to the HFI DTC together with the research programme drawn up for the first 2-3 years of the Centre’s existence. International collaboration opportunities will also be described. All DTC programmes are coordinated centrally by Dstl, the MoD’s Defence Science & Technology Laboratory, and in the case of the HFI DTC, Dstl representatives are an integral part of the Centre’s Management and Technical Steering Committees.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Collection and Reuse of Job-Task Analysis (JTA) Information for US Navy C4ISR Integration Purposes

Dee Quashnock, M.S.

Space and Naval Warfare

Systems Command (SPAWAR)

San Diego, CA

Bruce C. Wetherby, Ph.D. CPE

Science Applications

International Corporation

(SAIC)

San Diego, CA

Mike Bennett

Antin, Inc.

Norfolk, VA

The collection of Job-Task Analysis (JTA) information is a central tenant of the Human Performance Systems Model (HPSM) and a cornerstone of the Navy’s Revolution in Training. Critical data elements identified in a JTA are the C4ISR jobs that need to be implemented and staffed; the duties, tasks and subtasks that need to be performed; and various ancillary measures (e.g., task periodicity, frequency and duration) that contribute to the derivation of Manpower, Personnel and Training (MPT) support requirements. While the collection of C4ISR JTA information is most often used to outline proposed training curricula, a myriad of other uses for C4ISR JTA information have been identified, serving as a foundation for Human Systems Integration (HSI). These include the provision of guidance for Human Computer Interface (HCI) design; the development of task-focused technical user documentation; the construction of test and evaluation requirements; the derivation of individual-level, performance assessment instruments; and the potential linkage to organization-relevant C4ISR Navy Mission Essential Tasks (NMET) requirements. The reuse of JTA information for multiple C4ISR purposes and/or products serves to distill, or consolidate, a common operational framework among all parties involved; facilitates multi-modal learning and application; and contributes to improved program effectiveness. Business transformation and process-reengineering factors associated with the implementation of this conceptualization of C4ISR JTA reuse within SPAWAR are presented and discussed, and suggestions for continued refinement and improvement of this business approach are forwarded.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

A Graphical Interface for Managing Multiple Unmanned Aerial Vehicles

Charles J. Cohen, Ph.D.

and Ron Hay

Cybernet Systems Corporation

Ann Arbor, MI 48108

The control and/or command of multiple Uninhabited Air Vehicles (UAVs) within an actual or simulated operational theater requires an interface that allows the user to direct the actions of these UAVs in real-time. The user must, at the very least, have the capability to issue commands that apply to whole UAV groups (“swarms”), as well as the ability to narrow commands to a few or even a single UAV. The user must have the ability to create UAV groups, select and assign targets, and interactively generate and assign routes.

This paper describes an innovative interface for operators to control multiple UAVs in a combat situation. Coupled with the latest off-the-shelf input and output hardware, the software will be an intuitive, real-time, graphical, 3D environment that enhances situation awareness as much as possible without perceptual overload. This system will also be fully compatible with the Mixed Initiative Control for Automa-teams (MICA) Open Experimental Platform (OEP) multiple UAV scenario system.

To control multiple UAVs, there are specific command requirements for the UAVMI system. Relevant issues include:

1. Where and how exactly does the UAV’s autonomy interface with the higher level commands coming from the UAVMI? The solution is to create (or adopt an existing) command language for the individual UAVs.

2. Define a library of “swarm-level commands”. These are commands that affect groups of UAVs.

3. Address the issue of how “swarm-level commands” translate into commands to individual UAVs. For example: if the swarm-level command is for group A to attack enemy group E, then what exactly should a1 (one UAV in group A) do? Attack e1? Attack the closest element in group E? What if that target is already covered by a3?

The areas addressed include group and individual commands, path generation, relative posture, selection methods, viewpoint navigation, and information displays. The control and/or command of multiple Uninhabited Air Vehicles (UAVs) within an actual or simulated operational theater requires an interface that allows the user to direct the actions of these UAVs in real-time. The user must, at the very least, have the capability to issue commands that apply to whole UAV groups (“swarms”), as well as the ability to narrow commands to a few or even a single UAV. The user must have the ability to create UAV groups, select and assign targets, and interactively generate and assign routes.

To control multiple UAVs, there are specific command requirements for the UAVMI system. Relevant issues include:

2. Define a library of “swarm-level commands”. These are commands that affect groups of UAVs.

The areas addressed include group and individual commands, path generation, relative posture, selection methods, viewpoint navigation, and information displays.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Network Centric Warfare: Metrics for Assessing FORCEnet Human Performance Solutions

Robert Breaux

NAVAIR Orlando, Training Systems Division

Orlando, FL

The Navy’s Transformation Roadmap, signed jointly by Secretary of the Navy, Commandant of the Marine Corps and Chief of Naval Operations, details the plans for SeaPower 21 and the Navy’s move into the Information Age. Successfully implementing Network Centric Warfare (NCW) will require a Human-Centered Design (HCD) approach to FORCEnet, on a scale unprecedented within the training community, that will ensure optimal warfighter performance. Thus, a robust set of human performance metrics will be required to guide the evaluation of solutions for design, training, manning, implementation and testing of FORCEnet.

This paper discusses various metrics that are under consideration, how metrics can be developed, how the metrics will be used for assessing system design and training solutions, and recommends an approach for developing additional metrics that will ensure fully qualified sea warriors for implementing FORCEnet in NCW.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Objective Human Performance Measurement in a Distributed Environment: Tomorrow’s Needs

Brian T. Schreiber ,

Eric A. Watz

Lockheed Martin

Mesa, AZ

Winston Bennett, Jr.

Air Force Research Laboratory

Mesa, AZ

Networked simulation development continues its rapid progress, but automated human performance assessment development has been almost completely neglected. Typically, subject matter expert opinions and surveys are used to assess human performance. While subjective ratings for complex simulation environments provide a valuable assessment for overall performance, a comprehensive effectiveness evaluation should include objective measures for both in-simulator and transfer to actual environment assessments. Furthermore, the individual assessment of numerous skills can exceed the attentional resources a subject matter expert has to offer. Thus, an automated objective skill measurement system is required to properly evaluate the training effectiveness of networked simulations. For example, if one wanted to track the amount of time an aircraft has spent within different “range rings” to a threat, an automated tool can quickly and precisely track this information by “listening” to network traffic and calculating distances from the positional information provided by each entity. An automated performance assessment tool could go beyond simple measures such as kill ratios and weapon hit ratios; tracking hundreds of variables, thereby providing objective assessments for individual and team skills quickly and accurately. This paper presents a general methodology for capturing automated objective assessments from a networked simulation environment. Research in this area reveals that, although many objective assessments can be made today, additions to the current DIS/HLA protocol standards would increase the number of opportunities and methodologies used to measure individual and team performance. This paper presents results showing that human performance assessments for networked simulation is possible and advocates for new requirements to enable a standardized, extensive suite of measures. These measures will allow researchers to quantify the amount of learning that has taken place in a networked simulation environment, in terms of both outcome and process measures, and the standardization enables cross-comparison of effectiveness results.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Stereovision System for Shipboard Cranes

Jeff Green

Naval Surface Warfare Center

Carderock Division

Ed Dougherty ,

Ed Savacool

IntelliTrans, LLC

120 West Lancaster Ave, Ardmore, PA

The movement of cargo during a Joint Logistics Over the Shore (JLOTS) operation is a difficult and often hazardous task. The difficulties and hazards increase greatly in conditions of rough weather and high seas. A particular complexity encountered by military cargo handlers is the shipboard crane operator's poor visibility of the working area.

The Stereovision System (SVS) is a teleoperation system designed to provide the crane operator with a high fidelity, stereoscopic view of the working area. The cameras are positioned so that working locations, such as within the hold of the ship or over the side of the ship, that are not visible from the operator's cab can be viewed with a stereoscopic head-mounted display.

A prototype SVS has been installed and tested onboard a military cargo ship to investigate the efficacy of this system for improving the productivity and safety of shipboard crane operations.

The objective of the testing was to measure comparative performance of the crane operator with and without the SVS. Limited performance parameters were measured, including the time to pick/place cargo, and the accuracy of placing the cargo in its target location within that time. Other factors, such as operator fatigue, safety, and ease of operation were also observed. Results showed the system hardware and software worked as expected. In addition, the consensus of operator opinion was that the system helped considerably and showed great potential.

The shipboard installation did not resolve the issue of whether a monocular vision system would be as effective as the stereoscopic system in this application. Also, some negative comments from the operators were associated with the small field of view and the discomfort of wearing the head mounted display. The SVS has been installed at the training facility of the Naval Cargo Handling And Port Group to facilitate operator training, procedure development, evaluations of alternative stereo display, and stereo-vision versus monocular-vision performance.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

Team Performance Modeling in Sustained Simulated C4ISR Operations

Christopher Barnes, 1st Lt, USAF

Air Force Research Laboratory

Warfighter Fatigue Countermeasures

2585 Gillingham Drive

Brooks AFB, TX 78235-5100

Linda R. Elliott

Veridian Engineering

2485 Gillingham Drive, Suite 201

Brooks City-base, San Antonio, TX 78235

Donald L. Harville

Air Force Research Laboratory

Warfighter Training Division

2485 Gillingham Drive

Brooks AFB, TX 78235-5100

Mathieu A. Dalrymple

Veridian Engineering

2485 Gillingham Drive

Brooks AFB, TX 78235-5100

In this paper, we report methodology and preliminary findings focused on the application of multi-level modeling techniques to distinguish effects of sleep loss and task demands on individual and team C4ISR decision making, coordination, and performance over time. We focus our efforts on measurement and modeling. First, we describe aspects of C4ISR scenario development, to ensure (a) psychological fidelity and operational relevance, (b) elicitation and assessment of performance constructs of interest, and (b) equivalence in scenario task demands and difficulty. Sustained operations research is challenged by the need for repeated-measures assessment, while minimizing effects of practice or experience. Second, we describe aspects of cognitive performance based on a standard cognitive test battery. Third, we describe other assessments (e.g. NEO PI personality assessment, moodstate inventory, Stanford Sleepiness Scale, physiological indices) that will be included in an overall approach to modeling fatigue effects, using multi-level hierarchical modeling analyses.

This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website