HUMAN
SYSTEMS INTEGRATION
MATHEMATICAL ALGORITHMS FOR TRAINING EFFECTS
DETERMINATION IN CGF
ESTIMATING CREW READINESS IN DISCRETE EVENT SIMULATIONS
OZ: A
HUMAN-CENTERED COMPUTING COCKPIT DISPLAY
A BENEFITS ANALYSIS FOR INTEGRATING THREE DIMENSIONAL
DISPLAYS IN NAVY COMBATANT INFORMATION CENTERS
IMPROVING CCTT DISMOUNTED INFANTRY TRAINING
SELF-EFFICACY AS A TRAINING EFFECTIVENESS ASSESSMENT
METHODOLOGY AND OPTIMIZATION TOOL
AN INTEGRATED HUMAN MODELING PROCESS FOR THE INTERNATIONAL
SPACE STATION, INTRA-VEHICULAR ACTIVITY
ENHANCEMENT OF THE U. S. ARMY AIRCREW COORDINATION
TRAINING (ACT) PROGRAM
HUMAN SYSTEM INTEGRATION (HSI) TRADE OFF PROCESSES
A THEORY-BASED MODEL OF COGNITIVE WORKLOAD AND ITS
APPLICATIONS
EVALUATING OPERATOR LOADING DURING SYSTEM DESIGN
EFFECTS OF STREAMING VIDEO QUALITY OF SERVICE ON SPATIAL
COMPREHENSION IN A RECONNAISSANCE TASK
COLLABORATIVE NAVIGATION IN REAL AND VIRTUAL ENVIRONMENTS
SIMULATING COMMAND AND CONTROL FLOW IN THE FIRE EFFECTS
COORDINATION CELL
INDIVIDUAL SKILL MEASUREMENT BY MEANS OF A SIMULATED
TEAMMATE IN THE CHEETAH TANK SIMULATOR
VOLKSSIM 2000I: UNIQUE RESEARCH TOOL FOR HUMAN
PERFORMANCE EVALUATION
CORRELATIONAL DATA THAT SUPPORT A CONSTRUCTIVE ASSESSMENT
OF DRIVING SKILLS
RELATIONSHIP BETWEEN MISSION PREPARATION AND PERFORMANCE
DURING COMBAT MISSION TRAINING
DEFINING DIGITAL PROFICIENCY MEASUREMENT TARGETS FOR U.
S. ARMY UNITS
KNOWLEDGE ACQUISITION IN DISTRIBUTED MISSION TRAINING
OPTIMIZATION OF AIRCRAFT CARRIER DESIGN AND OPERATIONS
HUMAN SERGEANTS, ROBOT SOLDIERS: SOLDIER-MACHINE
INTEGRATION ISSUES WITH SEMI-AUTONOMOUS SYSTEMS
|
IMPROVING SOLDIER FACTORS IN HUMAN
PERFORMANCE MODELS Rick Archer, Brett Walters, and Alia Oster Micro Analysis & Design, Inc Boulder, Colorado U. S. Army Research Institute Orlando, Florida This paper describes
work being performed under Phase II of a Small Business Innovative Research
Project (SBIR) for the Army Research Institute. A goal of this effort is to
improve the realism of computer- generated force CGF entities in constructive
simulations. Currently, CGF entity behavior is very predictable and
unrealistic with respect to the natural variability with which humans perform
given varying amounts of aptitude, training, and environmental stressors to
which they would be exposed on a real battlefield. A second goal of the SBIR effort is to develop a set of
algorithms and data structures for including variables such as aptitude,
training, and stressor effects that can be integrated with other types of
available software packages used for developing human performance models. The work is currently progressing on three
main thrusts. One thrust is focused on developing the aptitude algorithms,
learning curves, and stressor algorithms that will eventually influence the
performance variables in the human performance models. Work in support of this
thrust included a review of the literature on individual and team learning
theories, military requirements for human performance modeling and a search
of the literature for empirical data that describes the effects of training
on human performance. Work toward developing the learning curves has also
included the development of a data collection questionnaire for obtaining
estimates from soldiers on how their training affected their proficiency in
combat performance. This questionnaire was administered to platoon leaders
and platoon sergeants from Armor divisions at Fort Riley, Kansas and Fort
Carson Colorado. Data from the questionnaires has been analyzed and used to
develop learning curves for classroom, simulator, and field training effects. A second thrust of the project is the
development of a software tool that will allow a user to enter information
about the training and aptitude of a population of operators in a human
performance model. This information will be used as input to the learning
curves to calculate the appropriate changes to performance variables in the
human performance models. In order to make the tool generalizable for any
type of training or human performance model, we are designing it to be
completely configurable by the user. This feature of the software tool will
allow users that have data to generate their own learning curves use those
learning curve algorithms to affect the performance variables in the models.
A part of this effort was the development of a test bed model that we are
using to test and demonstrate the functionality of the software tool in
correctly modifying performance variables.
The third major thrust of the project is to develop the architecture
for communicating performance variable values between the software tool and
the human performance models. Included in this effort is the selection of an
appropriate entity-based constructive simulation such as ModSAF, OneSAF
testbed, or JointSAF to apply the tool described above. At this time, the
eventual target platform is the OneSAF test bed. However, since all three of
these simulations share the same code base, we have begun this work on the
JointSAF software. One of the major challenges in developing the
communication architecture is in identifying the performance and potential
stressor variables in the simulation that can be modified.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
MATHEMATICAL ALGORITHMS FOR TRAINING
EFFECTS DETERMINATION IN CGF Alia Oster, Brett Walters, Rick Archer, Tony Edgin Micro Analysis & Design, Inc. Boulder, Colorado Advanced
distributed simulations (ADS) along with computer generated forces (CGFs) are
used to provide troops with tactical combat training and to perform research.
Current CGFs behave as perfectly trained troops, their ability to perform
missions to do not vary. This is an inaccurate portrayal of human
performance. If the military cannot model human factors, such as training and
physiological stressors in ADS, they cannot perform trade-off analyses. For
the military to be able to use ADS and CGFs to answer resource allocation and
system design questions, the CGFs have to be effected to be a human
performance model. Micro Analysis & Design, Inc. (MA&D) was awarded a
Phase II SBIR entitled “Improving Soldier Factors in Prediction Models” by
the Army Research Institute (ARI). This goal of this SBIR was to develop a
model that uses training and other performance shaping factors (PSFs) to
affect the abilities of CGFs. This performance effects model incorporates the
benefits of different types of training, the effects of skill decay,
physiological stressors and aptitude. The final model will allow users to
affect a wide range of tasks. It is generalizable to both military and
non-military applications. The military will be able to use it to affect the
performance of CGFs in ADS. Once the model is implemented, the military will
be able to conduct trade-off analyses. They will also be able to better
prepare troops for combat by having them train against opponents of different
skill levels.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
ESTIMATING CREW READINESS IN DISCRETE
EVENT SIMULATIONS Micro Analysis and Design Orlando, FL An
empirically based mathematical algorithm that predicts operator fatigue was
developed for discrete event computer simulations tools. The algorithm
estimates fatigue resulting from extended duty days and fragmented or reduced
sleep. The algorithm would be useful in many human system integration
problems such as predicting which crew duty schedules would produce the least
fatigued operators or which crews would be most rested to complete a
difficult assignment. It might also be used to degrade computer-generated
forces from fatigue to more realistically estimate their effectiveness.
During a Micro Saint simulation of two weeks of maritime activity, fragmented
and reduced sleep produced by the contemporary US Navy watch schedules
resulted in unacceptable fatigue levels over another crew duty schedule.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
OZ:
A HUMAN-CENTERED COMPUTING COCKPIT DISPLAY Institute
of Human and Machine Cognition University
of West Florida Pensacola,
Florida Naval
Aerospace Medical Research Laboratory United
States Navy Pensacola, Florida Introduction: With OZ, a
glance can provide the pilot with all the important flight information. Moreover, OZ directs a pilot’s attention
to flight path deviations when they occur so the pilot does not have to
continually "hunt" for them as he/she does with current instruments.
Conventional contemporary instrument design forces the pilot to view each
instrument sequentially in order to gather information, then remember,
interpret, and integrate it with the information from other instruments. This
process, the ‘instrument scan,’ is a huge workload for the pilot engaged in
instrument flight. OZ’s goal is to
simplify these tasks. Methods: OZ design capitalizes on what the human eye
and brain were designed to see best, quickest, and easiest so that the pilot
is no longer forced to use a sequential instrument scan to gather information.
Furthermore, OZ’s manner of illustrating flight information directly shows
the data, interactions, and aircraft capabilities, rather than requiring the
pilot to recall flight models and key values to calculate performance. In this way, OZ reduces mental
workloads. To evaluate OZ
empirically, it was interfaced with a commercially available Cessna 172
simulator so flight performance with OZ and the Cessna’s conventional
instrumentation was compared in controlled laboratory studies. Results: OZ is described and flight
performance data presented. In a
simple flight task, subjects learned the flight task faster and performed
better with OZ than with conventional instruments and as the flight task
became more challenging, performance deteriorated significantly with the
conventional display but remained unaffected with OZ. When an auxiliary task
was added to the simple flight task, OZ enabled the pilot to perform both
tasks simultaneously, a feat not possible with the conventional display. Discussion: OZ symbology provides an
operationally important increase in information transfer rate and ease of
understanding. These reduce workload requirements and improve flight
performance.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
|
A BENEFITS ANALYSIS FOR INTEGRATING THREE DIMENSIONAL DISPLAYS IN NAVY
COMBATANT INFORMATION CENTERS Instructional Science &Development,
Inc. Pensacola, Florida SPAWAR Systems Center San Diego, California The Department of
Defense is seeking to leverage advanced information technologies to improve
training and job performance of personnel performing in increasingly complex
working environments. Designers of
future command, control, communications, computers, and intelligence (C4I)
systems are now considering using advanced graphical methods to assist the
user in acquiring and maintaining the common tactical picture. The objective
of this study was to determine the benefit for incorporating a 3D perspective
view display and a 3D audio capability into the AN/UYQ-70(V) display system
in the CIC/CDC onboard Navy Ships. A review of relevant
literature reveals inconsistent findings. While most studies note some
benefit for 3D displays, nearly all take an either/or approach, stating that
a 3D display may be better for certain integration tasks, but a 2D display is
superior for absolute measures and close control. Here, we explore the
benefits of a 3D display that does not replace existing tactical displays
onboard Navy combatants, but is incorporated as an additional aid to provide
information in an integrative and intuitive format. A behavioral task
analysis was conducted through the use of a Task Survey, which elicited
responses on criticality, frequency, and difficulty regarding specific
tactical procedures. A cognitive task analysis was conducted using the
Applied Cognitive Task Analysis (ACTA) methodology developed by Klein and
Associates. This method was selected because it allows cues and sources of
information to be derived within the context of situation awareness. The ACTA
methodology was modified to specifically incorporate situation awareness
probe questions. More than 50
personnel provided responses to the task survey and participated in the
cognitive analysis process. Respondents identified a variety of watch
standing/decision making tasks that were critical, frequent, and difficult.
These tasks were then explored for areas where a 3D display and enhanced
communications capability could provide benefit. Almost all of the
respondents perceived a benefit to one or more warfare areas from a 3D
perspective view display. Specific
beneficial 3D display elements include: terrain, air tracks, environmental,
weather, and underwater data. Nearly all of the respondents perceived a
significant to huge benefit for incorporating an enhanced communication
system, which allowed for the monitoring of multiple internal and external
nets simultaneously. Based on the results of this study, SPAWAR Systems
Center has developed a prototype console, which incorporates a 3D perspective
view display and virtual 3D spatial communications system.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
||
|
IMPROVING CCTT DISMOUNTED INFANTRY
TRAINING Lockheed Martin Information Systems Orlando, Florida The
Dismounted Infantry Manned Modules (DIMM) for Squad Leaders and Platoon
Leaders are currently fielded as part of the Close Combat Tactical Trainer
(CCTT) System. The purpose of the DIMM is to train small unit leaders
(Platoon, Squad, and Section leaders) to use Dismounted Infantry tactics
while interacting with the other Army battlefield operational units in CCTTs
Synthetic Natural Environment. The DIMM is intended to train dismounted
leaders without requiring the entire squad or platoon. The
existing CCTT DIMM is based on technology available in the early-1990s.
Unlike the vehicle-manned modules in CCTT, the soldiers must be extensively
trained on the 'trainer unique' aspects of the DIMM for them to perform their
functions on the battlefield. For example, a joystick, mouse and menus are
used to move about the battlefield, change field-of-view, fire a weapon, and
give commands to the other soldiers. The current DIMM runs on an AIX
processor and provides a 180 x 27 degree field-of-view on five 20-inch
monitors. This solution does not provide a man machine interface reflective
of the field environment of a soldier, but the functionality does allow the
leader to control his computer-generated squad or platoon without the use of
a Semi-automated Forces (SAF) operator. A logical question is whether the
interface can be modified to provide a soldier-friendly environment, with
greater immersion, allowing the leader to concentrate on tactics instead of
trainer-unique requirements. A
prototype DIMM system has been developed to both reduce the amount of
trainer-unique training required to use the system and to allow the leaders
to function more closely to the way they do on the battlefield. This paper
documents a prototype effort to drastically change the DIMM interface and
compares the prototype to the existing solution. The prototype allows a Squad
Leader to give voice commands to the soldiers (computer generated) in his
squad and to hear voice responses. Once the Voice technology replaces the
tedious mouse/menu commands as the soldiers’means of giving tactical orders,
the soldier is free to stand up and hold a weapon as he does in the field. In
the prototype, the Squad Leader stands up holding his weapon, looking at an
8’ x 6’ image with a 60 x 45 degree field-of-view. The leader wears a Helmet
Mounted Display displaying a digital map to be used for planning and
navigational purposes. The intent of this effort is to provide better
training for soldiers in the Dismounted Infantry role by providing an
environment where tasks are similar to the same tasks in the field with minimal
trainer-unique knowledge required. This
paper is available on the 2001 I/ITSEC CD ROM.
|
||
|
SELF-EFFICACY AS A TRAINING
EFFECTIVENESS ASSESSMENT METHODOLOGY AND OPTIMIZATION TOOL University of Wyoming Laramie, Wyoming United States Air Force Academy Sonalysts, Inc. Orlando, Florida Naval Air Warfare Center, Training
Systems Division Orlando, Florida One
of the best predictors of performance on a particular task is an individual’s
perception of skill on the task. This perception, called self-efficacy, has
been demonstrated to impact performance on a broad range of physical and
cognitive tasks via changes in motivation, persistence, and affect. The first
systematized implementation of a self-efficacy assessment for a military
application was performed for an immersive virtual environment replication of
the shiphandling task Underway Replenishment (UNREP). Surface Warfare
Officers’ (SWO) self-efficacy was assessed via self-report. Self-report items
were derived from a cognitive task analysis, observation of experts’
performance, and subject matter expert interviews. A final set of
self-efficacy assessment items was validated after pilot testing. The
resulting self-efficacy measure for the UNREP was then administered both
before and after participation in two trials of a virtual environment UNREP
scenario. Participants were moderately skilled UNREP officers. Analyses revealed that participants
reported higher self-efficacy for performing the actual at sea UNREP after
participating in the UNREP virtual environment training task. The final
version of the UNREP Self-Efficacy Scale (URSE) demonstrated excellent
reliability with a Chronbach’s alpha of .97. Perceptual-motor and cognitive
skills have been extensively researched, but self-efficacy appraisals have
been overlooked despite the significant impact they have on task performance.
Further research in this area is warranted to optimize training
interventions, create trainees with resilient personal efficacy for other
training domains, and to aid in the development of realistic human cognitive
models.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
An Integrated Human Modeling Process
for the International Space Station, Intra-Vehicular Activity Terri B. Graham, Janis Hebermehl The Boeing Company Defining
a process for integrating human modeling within the design and verification
activities of the International Space Station (ISS) has proven to be as
important as the simulations themselves. The process developed (1) ensured
configuration management of the required digital mockups, (2) provided
consistent methodology for simulating and analyzing human tasks and hardware
layout, (3) facilitated an efficient method of communicating design
requirements and relaying satisfaction of contract requirements, and (4)
provided substantial cost savings by reducing the amount of late redesign and
expensive mockup tests. Human
simulation is frequently the last step in the design process. Consequently,
the influence it has on product design is minimal and oftentimes being used
as a post-design verification tool. This paper presents an integrated human
modeling simulation process that demonstrates the use of the Boeing Human
Modeling System (BHMS) as a design and training tool for the International
Space Station.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
ENHANCEMENT OF THE U. S. ARMY AIRCREW
COORDINATION TRAINING (ACT) PROGRAM Neal Crossland, Training and Performance Support
Systems Gary Grubb, Center for Team Performance Dynamics Research Corporation Andover, Massachusetts The
Aircrew Coordination Training Enhancement (ACTE) program is a continuation of
the US Army Research Institute’s (ARI) effort to promote applied research and
development of the existing Army Aircrew Coordination Training (ACT) program.
Following implementation of the initial Army ACT program in the mid-90’s,
field commanders and aircrews alike acknowledged the benefit of the
mandatory, one-time training that was received by all aviators within the US
Army aviation community. However, program funding did not provide a mechanism
to effectively sustain high levels of aircrew coordination training. Additionally, experience and skill levels
have decreased significantly because of diminished defense spending and concomitant
reductions in personnel strength levels. The combination of these factors may
have contributed directly to a reversal in Army aviation accident rates that
had been declining since ACT implementation. The goal of the current ACT
enhancement effort is to provide a capability for a web-delivery, interactive
aircrew coordination training system that will provide Army aircrews
worldwide with the knowledge, skill sets, and attitudes that will increase
their safety of flight and mission effectiveness in daily operations. This
paper describes the ongoing training system research and development effort
under the guidance of the US Army Research Institute. The research plan
consists of three major phases –upgrade and sustain the existing ACT program,
refresh and maintain the upgraded ACT program, and deploy advanced ACT
applications. Central to the upgrade and sustainment phase is the process of
designing and developing a prototype interactive computer-based instructional
package that will be exported and evaluated for instructor-mediated training
events. The instructional design employs state-of-the-art technology provided
by programming software such as Macromedia Flash and Dreamweaver Ultra-Dev.
The instructional approach includes refresher modules prerequisite to
scenario-based case study modules that serve as the basis for facilitated
discussions or “hangar talk ”designed to instill and enhance effective crew
coordination skills among aircrew members. An important part of the case
study modules is the on-line evaluation of presented mishap scenarios in the
form of a feedback histogram. The ability of facilitators and students to see
performance variations in scenario situations serves as a trigger for
discussion and internalization of optimal team coordination behaviors. The
long-term goal is to provide ACT for consumption as a mission and aircraft
specific training and performance evaluation system for use by unit
instructors and individual aircrew members.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
HUMAN SYSTEM INTEGRATION (HSI) TRADE
OFF PROCESSES Iain S. MacLeod and Carole D B Deighton PhD Aerosystems International, Inc. Orlando, Florida The
Human System Integration (HIS) process includes a consideration of Human
Factors Engineering, Manpower, Personnel, Training, System Safety, Health
Hazards and Personnel survivability. A lack of consideration of any one of
these elements presents some risk to the fitness-for-purpose of the final
system. In practice a project plan which includes a detailed consideration of
each HSI element presents an unacceptable project overhead with respect to
time, quality and cost. A process for understanding those HSI elements which
present the greatest risk to system effectiveness needs to be determined.
Activities within the Project Management Plan (PMP), or Systems Engineering
Management Plan (SEMP), are then shaped to address those HSI elements, which
present the greatest risk to system effectiveness at a given stage in the
system life cycle. Unfortunately, trade off processes and the associated
trade off rules are not provided by the US HSI initiative or indeed by the UK
equivalent Human Factors Integration Programme. This
paper attempts to address this gap by outlining a trade off process, based on
the established repertory grid technique. It is argued that stakeholders can
use such processes, as can the appointed project team, to maximise project
resources, achieve system requirements and ultimately achieve fitness for
purpose. Moreover, such processes provide a justification to non-specialists
for considering elements defined by the HSI process. The proposed trade off
process can form an important part of an HSI Assistant such as that is being
developed by AeI to promote expertise in the engineering of Human-Systems.
The objectives of the HSI Assistant, its content and its structure are
outlined briefly.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
A THEORY-BASED MODEL OF COGNITIVE
WORKLOAD AND ITS APPLICATIONS Christian Lebiere, Human-Computer Interaction Institute Carnegie Mellon University Pittsburgh, Pennsylvania We
present a model of cognitive workload based on the ACT-R (Adaptive Character
of Thought – Rational) cognitive architecture. That model has been validated
in a synthetic air traffic control task according to a wide range of
behavioral measures. Its cognitive workload predictions are sensitive to
level of task embedding, interaction speed, level of interface decision
support and individual differences. We sketch possible extensions of the
model that support multiple workload dimensions and instantaneous workload
ratings. We also discuss possible applications of this kind of fine-grained
computational models to system design, operator training and selection and
dynamic load balancing.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
EVALUATING OPERATOR LOADING DURING
SYSTEM DESIGN Micro Analysis & Design Inc. Boulder Colorado Defence and Civil Institute of
Environmental Medicine Toronto Canada The
Canadian Defence and Civil Institute of Environmental Medicine (DCIEM),
Systems Modelling Group sponsored the development of a new human performance
modeling approach based on an Information Processing and Perceptual Control
Theory model. The approach combines Hierarchical Goal Analysis with
simulation to predict a human “operator loading, performance and error
production” versus the classic analytic approach of predicting performance
under simulated “task loading”. This paper will describe the implementation
of the theoretical approach within a task network simulator called the
Integrated Performance Modelling Environment (IPME) and discusses its
intended use and benefits.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
EFFECTS OF STREAMING VIDEO QUALITY OF
SERVICE ON SPATIAL COMPREHENSION IN A RECONNAISSANCE TASK Rudolph P. Darken, MAJ Kurt Kempster, & Barry Peterson MOVES Institute, Naval Postgraduate
School Monterey, CA It
has been proposed that if we could configure individual personnel with
micro-video cameras and wireless communications such that they could transmit
a video stream of what they were seeing to a remote observer, this would be
an enormous improvement in reconnaissance and battlefield command and
control. We looked ahead, based on current video and wireless communications
technologies and trends to what we can expect to have available in terms of
streaming video quality of service (QOS) and we used those predictions to
conduct an experiment to determine if this assertion of improvement is true.
Participants viewed a digital video with a data rate associated with a given
transmission technology. They were asked to maintain their orientation by
tracking the position of the camera on a paper floor plan diagram. They were
also asked to identify a number of objects and place them in the correct room
on the floor plan. The results show that participants found all conditions
except the live walkthrough control condition to be extremely difficult with
poor performance on both the spatial orientation task and the object
identification task. Bandwidth does affect error as increased data rate
improves performance. Rapid head rotations seem to be the largest contributor
to disorientation, especially with low data rate video. Our results suggest
that simply supplying video feedback to a remote observer may be useless at
best or possibly damaging at worst. What is needed is not necessarily more
bandwidth, but better interfaces and tools to help observers to remain
oriented such that they can extract what is needed from the video stream.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
COLLABORATIVE NAVIGATION IN REAL AND
VIRTUAL ENVIRONMENTS Barry Peterson, James Boswell and Rudolph P. Darken MOVES Academic Group, Naval Postgraduate School Monterey, California Our
study of collaborative land navigation in the real-world provides input for
our design of the human computer interface of a virtual learning environment.
Study findings reinforce the applicability of the Recognition-Primed
Decision-Making model to the land navigation domain. Also, study of the
interpersonal communication between team members informs our understanding of
the relationship between tutor and student. Finally, we found that knowledge
elicitation based upon narrative form generates valuable descriptive
knowledge quite naturally and that team members exchanged information through
the storytelling medium.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
SIMULATING COMMAND AND CONTROL FLOW IN
THE FIRE EFFECTS COORDINATION CELL
As part of the
Army’s Science and Technology Objective (STO) – Cognitive Engineering of the
Digital Battlefield a human performance simulation model of the Fire Effects
Coordination Cell (FECC) was developed. This model is being used to assess
the impact of personnel, training, material, and organizational factors on
operator efficiency and effectiveness.
Among the performance measures under scrutiny are operator
utilization, the number of tasks performed by the operators, the quality of
decisions made by the operators, and the effects of having a central decision
maker involved in the process of target detection to delivery of effects
(either lethal or non-lethal) within the task flow. In order to obtain
decision data from the model, an information-driven decision making scheme
was developed to assess the interaction of operator training, experience, and
fatigue variables as well as the quality of available information on the
decision making process. This paper will describe the human performance
model, with emphasis on the unique aspect of the embedded decision making
model. The paper will also present results of a preliminary study when this
model was applied to several FECC concepts. The model is
designed into two distinct phases. Phase 1 simulates pre-battle planning from
the time an order is delivered to the start of the battle. Phase 2 of the
model simulates the processes that the operators perform after the beginning
of the battle and continues through mission assessment. The model is designed
to compute and record various statistics of operator performance. The
decision making scheme was implemented in the following way: each operator is
responsible for a variety of information and the model keeps track of the last
time that the operator updated this information. The information is defined
according to the Army’s accelerated decision-making process for developing a
successful plan. Five aspects of a successful plan were identified, and it
was determined that these aspects contained 24 information elements. These
information elements represent the data for which the operator is
responsible. The information quality is a function of the last time the
information element was updated and how susceptible to change the information
may be. This result is then combined with the rating of the operators’
experience, training level, and level of fatigue to obtain a decision score
(i.e., whether the decision was determined to be ‘good’ for the given
situation or not). The decision scores are used to discover points in the
process where mistakes are likely to be made and to identify when
communication between operators is likely to break down. It is anticipated
that this model can be used to identify system design, manning and training
changes that could improve system efficiency and effectiveness.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
INDIVIDUAL SKILL MEASUREMENT BY MEANS
OF A SIMULATED TEAMMATE IN THE CHEETAH TANK SIMULATOR Netherlands Organisation for Applied
Scientific Research (TNO) TNO Human Factors (TNO-HF) Soesterberg, The Netherlands TNO Physics and Electronics Laboratory
(TNO-FEL) The Hague, The Netherlands In
our research project, a simulated teammate was used to determine the
effectiveness of tank simulator training for individual task aspects. This
tank simulator was built to teach a crew how to use an Armoured Air Defence
tank called “Cheetah”. In this tank, a team of two crewmembers carries out
the necessary actions. Besides teamskills, they both need a basic amount of
individual skills to work effectively as a team. To obtain those skills, students are trained in different team
combinations each lesson. During the normal training program, students’
individual skills are evaluated by looking at the team results. Our project
aimed at measuring the individual skills of a crewmember separate from the
team results. We considered two options to measure the skills individually;
the first option used speech recognition and the second a computer-simulated
teammate to work together with a student during the test in the simulator.
After testing, we selected the second option in order to obtain objective
individual results on skill acquisition. A number of trials with the selected
option showed that the students reacted positively to the new test and that
the objectivity of the test was considerably improved. Individual differences
became clear and the need for extra training could be determined more
precisely. Further research issues are discussed in the paper.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
VolksSim 2000I: UNIQUE RESEARCH TOOL
FOR HUMAN PERFORMANCE EVALUATION Systran Federal Corporation Dayton, OH NTI Incorporated Dayton, OH U. S. Army Aeromedical Research Laboratory Fort Rucker, AL Using
an assortment of commercial off the shelf (COTS) products, an open and
flexible architecture, some government furnished equipment (GFE) seats and
controls, and easy to work Sintra . iii expanded PVC structural panels, our
team developed a pair of simulators to support the human performance mission
conducted by the U. S. Army Aeromedical Research Laboratory at Fort Rucker,
AL. These simulators include both helicopter and ground vehicle cockpits,
interfaced to a common simulation computing, image generation, and data
collection “platform. ” While
not unlike many of the simulators seen at this and past I/ITSECs, we also
faced two additional challenges in this development: (1) each simulator
cockpit would be mounted upon a four-foot diameter 3-degree of freedom (DOF)
shake table, thus being subjected to a wide vibration spectra for many hours
at a time, and (2) a broad array, yet not completely defined set of future
experiments meant that a robust toolset would be needed to meet the needs of
future researchers. We
employed a multifunctional team composed of human performance measurement
experts (NTI, Inc.), product designers (Simpkins Design Group, Inc.), and
system architects and integrators (Systran Federal Corp.). A series of
in-process reviews was used to bring all members of the team together, in
order to list, prioritize, review, and fix the evolving requirements for the
simulation.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
CORRELATIONAL DATA THAT SUPPORT A
CONSTRUCTIVE ASSESSMENT OF DRIVING SKILLS Profile Associates Chapel Hill, North Carolina Research Triangle Institute Research Triangle Park, North Carolina We
describe how a constructive PC-based driver assessment part-task trainer
(PTT) can be integrated into driver training. The PTT, developed through
research with law enforcement agencies, gathers data on drivers' scanning and
divided-attention skills and also measures the tendency toward tunnel vision
under stress. The user sits in front of a computer monitor and interacts with
the system through a force-feedback steering wheel and foot pedals. A validation study tested 50 North
Carolina State Highway Patrol cadets before they were evaluated on a closed
circuit driving course. A composite score was derived that reflected the
cadets' scanning and divided-attention skills at five levels of increasing
difficulty. The track testing evaluated behavioral skills as the cadet
completed a 1.5-mile circuit with 11 obstacles; the skills were
decision-making (evasive actions with obstructed visual fields), accuracy of
maneuvers (number of cones hit), and lap times on three consecutive laps.
Consistent with previous research, the PTT scores showed a linear degradation
of skills over the five difficulty levels. Cadets with higher computer scores
showed fewer driving errors on the track, especially on lap 2. The evasive
action exercises were the most sensitive to individual differences. A
replication with 50 additional cadets found a relationship between PTT scores
and track instructor ratings of driving skills. The results show that a
complex and unfamiliar computerized training assessment correlates with
real-world driving skills, particularly when the driver is under pressure. A survey
of trainers employing the PTT suggests improvements in how the system should
be integrated into driver training. Results from an evaluation of its
effectiveness as an assessment tool point to methodologies for focusing
costly on-track (live) training by using relatively low-cost video reality
(constructive) assessment data. Implications for developing computerized and
track assessments of driving skills are discussed.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
Relationship between Mission
Preparation and Performance During Combat Mission Training Anacapa Sciences Air Force Research Laboratory Warfighter Training Research Division Lockheed Martin Although
military doctrine assumes that thorough mission preparation is a prerequisite
for mission success, empirical data are lacking. The present study
investigated the relationship between mission preparation and mission
performance during combat mission training of 11 MC-130P (Combat Shadow)
aircrews from USAF Special Operations Forces squadrons. Two observers
independently rated crew processes and mission performance based n extensive
observations taken (a) during a planning period and (b) while the crews
executed a simulated mission. A statistically reliable, strong relationship
was noted between preparation and performance on a number f indices, with
correlations ranging from .60 to .78. Notable preparation behaviors include
utilizing personnel effectively, establishing a firm timeline, aggressively
questioning a plan’s assumptions, and testing a plan’s logic against possible
contingencies. The paper concludes with a discussion of the characteristics
of effective mission preparation and implications for combat mission
training.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
DEFINING DIGITAL PROFICIENCY
MEASUREMENT TARGETS FOR U. S. ARMY UNITS John S. Barnett and Larry L. Meliza U. S. Army Research Institute for the
Behavioral and Social Sciences Orlando, Florida The
U. S. Army is exploiting the advantages of networked computer systems to
enhance battlefield situation awareness and command and control, a program
known as digitization. Digital
systems, as well as the proceduresfor using these systems, are evolving and
will continue to do so for many years. The goal of the current effort i
to support the evolution of
digitization with measures of digital proficiency that retain their value
across specific hardware and software products. The research identified
high-profile problems in the performance of non-digital units likely to be
addressed by the effective application of digital systems to help soldiers
visualize the battlefield and increase the operating tempo (OPTEMPO) of
units. Data from the U. S. Army Center for Army Lessons Learned (CALL) was
analyzed to identify the more frequently occurring problems in the
performance of non-digital units at the Army’s National Training Center and
Joint Readiness Training Center. These data were used to identify the
mechanisms where digital systems might address each problem and found that
over 92% of the approximately 200 high-profile problems could be addressed by
one or more of over forty mechanisms (e.g., increased situation awareness
makes it possible to use events rather than time to trigger many activities).
Twenty-two skills U. S. Army personnel would need to implement these
mechanisms (e.g., maintain awareness of expected versus actual locations of
friendly units) were identified. This approach defined four linked targets
for digital proficiency measurement: the impacts on high-profile problems in
unit performance, increases in battlefield visualization/OPTEMPO, employment
of digital mechanisms for addressing problems, and proficiency in skills
enabling digital mechanisms.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
KNOWLEDGE ACQUISITION IN DISTRIBUTED
MISSION TRAINING
At
the Air Force Research Laboratory (AFRL) in Mesa, AZ, there is an ongoing
program of research on training in a 4-ship F-16 Distributed Mission Training
(DMT) system. Typically, a team of four pilots comes to the laboratory for a
week-long training exercise. They fly together as a 4-ship team on several
missions designed to provide exposure to a range of combat scenarios. The
missions involve extensive briefing and debriefing sessions in addition to
the time in the simulators. In order to track the effects of training,
several inter-related projects are underway to assess the effectiveness of
training and to compare different training methods. In the project reported
here, we assessed changes in the ways pilots understand important concepts
related to the training. Pilots rated the relatedness of all pairs of 21
concepts from the domain of air-to-air engagements both before and after
training. Measures of the internal consistency (Coherence) of the ratings and
Pathfinder networks were derived from the ratings. Data from a group of the
most experienced pilots (experts) provided a point of reference for the less-
experienced pilots. At the beginning of the week, Coherence was significantly
correlated with previous experience in fighter aircraft suggesting that
providing consistent ratings depends on having a well- developed mental model
of the domain. Also, there was a significant correlation between experience
level and similarity to the expert reference group at the beginning of the
week supporting the general validity of the measurement methods. There was a
significant negative correlation between experience level and change in
similarity to experts from pre-to post training ratings. Greater changes were
found for the least-experienced pilots. As a result of these changes,
correlations with prior experience level were no longer statistically
significant at the end of the week. Further analyses on a group of the least-
experience pilots (novices) lead to similar conclusions. In particular, there
was a significant difference in mean Coherence between the experts and
novices at the beginning of the week but not at the end of the week. Also
novices showed significant pre-to post training increases in Coherence and in
similarity to experts. These measurement methods appear to provide a basis
for evaluating conceptual change. These assessment methods should prove to be
a useful adjunct to performance-based methods of assessing training.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
||||||
|
OPTIMIZATION OF AIRCRAFT CARRIER DESIGN
AND OPERATIONS Naval Air Systems Command Lakehurst Lakehurst, New Jersey Support Systems Associates, Inc. Lakehurst, New Jersey An
advanced Modeling and Simulation (M&S) approach has been developed to
define, analyze, and optimize aircraft carrier (CV) design and operations
utilizing Integrated Definition (IDEF) process modeling and associated
Commercial-Off-the-Shelf (COTS) tools. The IDEF process modeling methodology,
coupled with the interfacing requirements of the process, information, and
product models, identifies opportunities for improvement. The primary
objectives of the effort include an assessment of technology insertion,
integrated automation, Human Systems Integration (HSI), workload and manpower
reduction, and new operational concept development. An
aircraft carrier catapult launch process model has been developed and
simulated. The approach methodology incorporates M&S to replace current
antiquated steam-powered aircraft launch catapults with future technology
systems in the launch process model. This is accomplished by removing the
steam functions from the aircraft carrier launch process model and replacing
them with analogous future- generation technological advancements. In
addition to the advanced technology insertion, various HIS requirements (for
example, skill level, training, safety, task sequencing, etc.) and
opportunities for reducing the number of operator crew workstations and
associated workload will be investigated. Process
M&S will be used to analyze and optimize personnel quantities and skill
mixes by examining workload and operator utilization for the existing and
future launch processes. This approach can be used to assess the impacts to
Fleet operations if manpower is reduced or the impact to manpower allocation
if the operational processes are modified. The next-generation catapult
system may be evaluated by relating its performance to the overall launch
process through appropriate Measures of Performance (for example, aircraft
launch cycle time) and Measures of Effectiveness (for example, sustained
aircraft sortie rate). In addition, launch process catapult recharge and
catapult cycle times may be analyzed with respect to aircraft and pilot
preparation and turnaround times.
This
paper is available on the 2001 I/ITSEC CD ROM.
|
|
HUMAN SERGEANTS, ROBOT SOLDIERS:
SOLDIER-MACHINE INTEGRATION ISSUES WITH SEMI-AUTONOMOUS SYSTEMS John S. Barnett, U. S. Army Research Institute John P. Holmquist, University of Central Florida Orlando, Florida The Army of the future may use future combat systems (FCS) consisting of human soldiers directing semi-autonomous systems –essentially robot soldiers. This integration of human flexibility and adaptability with the power of technology promises to forge a team with remarkable combat power, provided the team can work together efficiently. However, previous experience with automation in aviation has shown human-machine teams may experience unique problems. Automation affects human workload, decision-making, crew interaction and team performance. People’s confidence in the automated systems affects their performance, and they often have difficulty dealing with automation failure. It is conceivable that similar problems, as well as FCS-specific problems, may manifest themselves in future combat systems. Therefore, research is necessary to identify areas where the soldier and the automation do not get along, so that the design process can eliminate, or at least mitigate, these problems. This paper presents some known automation concerns and suggests how research can be used to address these issues prior to fielding.
This
paper is available on the 2001 I/ITSEC CD ROM.
|