EMERging concepts and innovative
technologies
Does Game-Based Learning Work? Results
from Three Recent Studies
Fact or Fiction - Soldiers are
Gamers: Potential Effects on
Training
The Utilization of Low Cost Gaming
Hardware in Conventional Simulation
Eye Limited Resolution Displays: On
The Cusp?
Task Validation of Display
Temporal-Resolution Measurements
Automated Shader Generation using a
Shader Infrastructure
Building Interactive Virtual Humans
for Training Environments
Intelligent Agents: Incorporating Personality into Crowd Simulation
High Fidelity Flight Training Devices
for Training Ab Initio Pilots
DO BETTER MULTI-TASKERS MAKE BETTER
PILOTS?
Use of Haptic Devices to Provide
Contextual Cues in a Virtual Environment for Training
Markerless Augmented Reality based
Cameras using System-on-Chip Technology
Synergistic Classifier Fusion for
Security Applications
Mixed Initiative Team Performance Assessment System (MITPAS) For Training and Operation
2007 Paper No.
7172
DeVry
University
Arlington,
VA
The Department
of Defense (DoD) is faced
with challenges in expanding technology-based solutions that can make Warfighters more efficient, effective, knowledgeable, and
flexible. Of growing important to
the DoD is the potential of
using Commercial Off the Shelf (COTS) game-based learning in the armed forces
for increasing combat readiness.
The recruits of today not only understand technology in everyday use,
they expect it. These young
recruits are “digital natives” who were raised in a digital environment
surrounded by inexpensive, yet highly interactive gaming systems. To get the most from our new “best and
brightest,” new research into game-based learning needs to focus on military
use. The objective of these
projects was to add definitive research in the badly needed area of game-based
learning. Three
research studies were conducted at a national university to examine the
difference in academic achievement between students who did and did not use
video games in learning. Three different video games were added to approximately
half the classes of freshmen Introduction to Business and Technology courses,
3rd year Economics courses, and 3rd year Management courses. Identical testing
situations were used in all courses while data collected included game use, test
scores, gender, ethnicity, and age. ANOVA, chi-squared, and t tests were used to
test game use effectiveness.
Students in classes using the game scored significantly higher means than
classes that did not. There were no significant differences between genders, yet
both genders scored significantly higher with game play. There were no
significant differences between ethnicities, yet all ethnic groups scored
significantly higher with game play. Students 40 years and under scored
significantly higher with game play, while students 41 and older did not. These studies add definitive research in
the badly needed area of game-based learning. The DoD now has studies proving the efficacy of digital
game-based learning and how it can improve learning.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7341
James Belanich, Daniel B.
Horn, Jennifer L.
Solberg
U.S. Army
Research Institute for the Behavioral and Social Sciences
Arlington,
VA
Old Dominion
University & Consortium Research Fellows Program
Norfolk,
VA
George Mason
University & Consortium Research Fellows Program
Fairfax,
VA
PC-based games
are increasingly being used for military training domains. Proponents of
training games argue that younger Soldiers are part of the “digital” generation,
and having grown up playing videogames they will respond positively to their use
in training. However, in a series of research projects we’ve found that these
assumptions may be overstated. This paper covers two research efforts that
analyzed Soldiers’ videogame experience, as well as the impact of trainees’
prior experience on training outcomes. The first project surveyed a total of 777
first-year U. S. Military Academy Cadets who participated in a team tactics
training exercise using America’s Army, over two years. Results across both
years indicated that 60% of Cadets had limited or no videogame experience in the
prior year. Additionally, the amount and type of prior gaming experience
correlated with training outcomes (i.e., performance, training satisfaction,
motivation, and time on task). A second project assessed the frequency that
Soldiers of all ranks play videogames compared to engagement in other
recreational activities. Results
suggest that fewer than 32% of over 10,000 U.S. Army Soldiers surveyed across
various ranks play videogames recreationally on a weekly basis (numbers vary by
rank). For the ranks with the highest frequency (E2-E4),
only…
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7256
Peter
Smith, Lee Sciarini, Denise
Nicholson
Institute for
Simulation and Training Orlando, Fl
The current
convergence between conventional simulation training and game based training is
blurring the lines that once defined each industry. The simulation industry has
already begun to assimilate low cost gaming middleware for graphics, artificial
intelligence, and physics. With the unequivocal success of the integration of
game based software solutions into conventional training
, the next logical step is the utilization of low cost gaming console
hardware within the tool chain.
The most recent generation of gaming console hardware is ripe with
potential uses within simulation and training. The three major game console
manufacturers, Microsoft, Nintendo, and Sony, have each provided open tools that
allow direct access to their various hardware solutions. The potential of these
systems envelop a broad range of uses from deploying fully immersive 3D
simulations and performing computationally intensive multithreaded tasks to
revolutionary paradigms for user interaction. Determining the appropriate
solution for a given situation can be a daunting task for uninitiated
developers, as each solution is unique to its hardware. Given this, training
objectives and requirements must be mapped to the full range of fidelity and
capabilities offered by each solution. A recent study to determine the utility
of low cost gaming hardware in the Deployable Virtual Training Environment
(DVTE) was conducted by a multidisciplinary team from the Applied Cognition and
Training in Immersive Virtual Environments (ACTIVE) Lab at the University of
Central Florida’s Institute for Simulation and Training. This work discusses the
initial findings from the study and provides guidelines for employing the latest
generation of gaming hardware within PC Simulation and Training.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7075
Boeing Training
Systems and Services
St Louis,
MO
As the cost per
pixel of image generation and display hardware has fallen, it has become
economically feasible to consider constructing a wide field of view visual
system having eye-limited resolution.
Many simulation and training visual systems have been fielded over the
years with resolution barely equivalent to legally blind levels of acuity, so it
is certainly a welcome opportunity to improve training capability by bringing
these new technologies to bear. It
is hard to argue that incrementally better resolution is not always
incrementally better, but trade-offs made simply to provide more pixels to the
display may actually reduce overall
image quality. This paper focuses
on identifying those trade-offs and how they must be managed so that truly
eye-limited displays can be fielded.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7274
Air Force
Research Laboratory
Meza,
AZ
Link Simulation
and Training
Mesa,
AZ
We have recently
described a perceptual technique for assessing the temporal resolution of
display systems used in Air Force flight-simulators. That technique was based on
the assessment of the perceived blur of a simple stimulus consisting of a pair
of moving vertical lines. In the present study, we have attempted to validate
our previous technique by correlating it directly with performance on simplified
tasks that may be performed during training in a high-fidelity Air Force flight
simulator. Data were obtained by asking observers to; 1) detect whether or not a
moving F-16 target-aircraft banked as it moved laterally across the observers’
field-of-view and 2) judge whether changes in aircraft pitch resulted in
blurring of the associated moving terrain. The level of moving-image blur was
determined by the length of time that the image was presented during each video
frame (i.e., the project or hold-time) on a digital (DLP) projector. The results
were compared to those obtained using a standard CRT projector, whose effective
hold-time was about one-quarter of that of the lowest DLP hold-time tested. For
both the roll detection task and the aircraft pitch task, results with the DLP
project or did not differ significantly from that obtained with the CRT when the
DLP hold-time was reduced to 5.8 msec. These results are in qualitative
agreement with those obtained using the simpler moving-line test, and suggest
that the latter is a valid measure of display temporal resolution in the context
of flight-simulator applications.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7097
Chris
Coleman, Kevin
Harris, Anthony
Hinton, Anton Ephanov
MultiGen-Paradigm
Richardson,
TX
Programmable
rendering on the GPU (Graphics Processing Unit) utilizing “shader” technology has become a recognized benefit to visual
simulation, providing unprecedented realism and fidelity to synthetic
environments. However, effective use of shaders is a
technological challenge, where implementation of even a trivial vertex or
fragment shader requires manual re-implementation of
the fixed-function pipeline. Furthermore, generation of efficient shaders for existing data is a complicated undertaking that
is compounded by the number of state permutations possible under a modern
graphics API, such as OpenGL, and the desire to extend fixed function state with
advanced rendering techniques.
Therefore, system integrators and run-time providers face significant
challenges in incorporating shader technology while
hiding complexity and maintaining backwards compatibility with existing data.
Preserving the massive investment in existing database and model libraries,
while enhancing system capabilities, is a fundamental concern. Though some shader techniques are required for modeling, others are
typically supplied by the run-time. Fundamentally, it is clearly disadvantageous
to have run-time code contained in shared modeling assets. As a solution to these problems, this
paper introduces a Shader Infrastructure that
automates the building of vertex and fragment shaders
by analyzing the OpenGL state machine. The Shader
Infrastructure is capable of not only dynamic generation of highly efficient
shaders for rendering any legacy data considered valid
by the OpenFlightTM standard, but also extending the
rendering pipeline via Advanced Rendering Techniques. Techniques allow for
implementation and merging of novel GPU-based rendering approaches by injecting
small snippets of shader code into the Shader Infrastructure. The Shader
Infrastructure allows designers to customize portions of the rendering pipeline
and to automatically combine that customization with other rendering techniques,
either fixed-function or shader-based. Therefore, it
significantly simplifies the problem of content management and reuse while
taking full advantage of the advances in the programmable
PC…
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7105
Patrick
Kenny, Arno Hartholt, Jonathan Gratch, William Swartout, David Traum, Stacy Marsella, Diane Piepol
Institute for
Creative Technologies / University of Southern California
Marina Del Rey,
CA
There is a great
need in the Joint Forces to have human to human interpersonal training for
skills such as negotiation, leadership, interviewing and cultural training.
Virtual environments can be incredible training tools if used properly and used
for the correct training application. Virtual environments have already been
very successful in training Warfighters how to operate
vehicles and weapons systems. At the Institute for Creative Technologies (ICT)
we have been exploring a new question: can virtual environments be used to train
Warfighters in interpersonal skills such as
negotiation, tactical questioning and leadership that are so critical for
success in the contemporary operating environment? Using embodied conversational agents to
create this type of training system has been one of the goals of the Virtual
Humans project at the institute. ICT has a great deal of experience building
complex, integrated and immersive training systems that address the human factor
needs for training experiences.
This paper will address the research, technology and value of developing
virtual humans for training environments. This research includes speech
recognition, natural language understanding & generation, dialogue
management, cognitive agents, emotion modeling, question response managers,
speech generation and non-verbal behavior. Also addressed will be the diverse
set of training environments we have developed for the system, from single
computer laptops to multi-computer immersive displays to real and virtual
integrated environments. This
paper will also discuss the problems, issues and solutions we encountered while
building these systems. The paper will recount subject testing we have performed
in these environments and results we have obtained from users. Finally the
future of this type of Virtual Humans technology and training applications will
be discussed.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7151
Sivakumar Jaganathan, Dr. T. L.
Clarke, Dr. D. J.
Kaup, Jitendra Koshti, Dr. L.C.
Malone, Rex Oleson
University of
Central Florida
Orlando,
FL
The simulation
and modeling of crowd behavior has become an active research area in recent
years. This area of research has been applied to a wide variety of domains such
as military, education, training, entertainment and human factors analysis. Our
current research at the Institute for Simulation and Training (IST) which is a
part of University of Central Florida (UCF) focuses on the influence of
individual personality factors on crowd behavior. The objective of this research
is to model and simulate the “Big Five” personality factors and associated
forces using the Helbing-Molnar-Farjas-Vicsek (HMFV) crowd model
within the MASON (Multi-Agent Simulation of Neighborhoods) environment. MASON is
a fast discrete-event multi-agent simulation library with the core of the
programming written in Java. It provides wealth of functionality for many
lightweight simulation needs and the ability to record simulation results for
comparison with real data. The
crowd behavior models are validated by comparison with real-life video recorded
at venues such as churches and sporting events. This comparison is facilitated by use of
in-house software developed using the OpenCV (Open
Computer Vision) library to derive optical flow from the videos. The HMFV model
is being enhanced by the incorporation of personality factors, and the
coefficients associated with the personality factors are being optimized by
comparison against the real-life video using the in house developed software
tool. The results of the comparison
are discussed. This research falls
within the technical area of modeling and simulation of the behavior of
individuals in teams and in crowds and other social groups. It will find
application in a variety of areas such as emergency management of crowds and for
the design of buildings for emergency egress. An additional area of application
of such models of collective human behavior is to identify anomalous crowd
behavior associated with the Improvised Explosive Device (IED) threat.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7467
Sandia National
Laboratories
Albuquerque,
NM
The present
paper is the third in a series that seeks to describe the theories and methods
employed to create engaging learning environments for training Marine and Army
junior leaders, U.S. Special Forces, Civil Affairs, and Psychological Operations
teams to think adaptively. The first publication (I/ITSEC 2005) described a
novel use of a first-person shooter game-based training system (in use since
2004) at the JKF Special Warfare Center and School that focuses on negotiation
skills, cross-cultural communication. The second publication (I/ITSEC 2006)
described the simulation experience design method used to create a “crucible”
experiences that invoke trainee adaptive thinking by forcing crucial choices,
and sharpening one’s focus. The present paper addresses more specifically the
perspectives that have inspired the development of methods (used first for the
U.S. Special Forces and currently for DARPA DARWARS Ambush NK! [Non-Kinetic])
to train adaptive thinking particularly by honing
cultural awareness and metacognitive agility for
non-kinetic engagements. In the
full paper, we discuss perspectives from intercultural communication,
social-process simulation, and metacogntion that have
inspired the approach to overall training architecture and software development
to train adaptive thinking, cultural awareness, and metacognitive agility for multi-player game-based systems.
We describe how game-based training can be designed as consisting of a system of
experiences, and how the design of a reflective role, in-game assessments &
evaluation, and quantitative evaluations in after action reviews enhanced for
non-kinetic engagements, present a unique blend of methods from which to enhance
adaptive thinking. The paper addresses how the instantiated role functionality
and methods can be used by observer controllers, peer trainees, subject matter
or cultural experts, instructors, etc. to provide quantitative feedback of
actions taken, (including communications) as they occur in real-time. We discuss
how our approach instantiates in software a unique role that provides experience
with…
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7007
U.S. Army
Research Institute
Fort Benning, GA
Fifty-two
leaders in the Basic Non-Commissioned Officer Course (BNCOC) at Fort Benning, Georgia, participated in an assessment of two
desk-top computer simulations used to train tactical decision making. Thirteen
leaders trained with the Soldier Visualization Station (SVS) simulation, while
39 others trained with Simulation Field Exercise, or SimFX. Pre-simulation exercise measures included military
and simulation experience, and tactical situation judgment. A questionnaire
administered to leaders following simulation exercises documented their sense of
personal involvement during mission execution and their perceptions of the
training value of the simulations.
Leaders in both groups were assessed individually for their ability to
recognize and implement rapid, sound tactical decisions while serving as a
leader of a light Infantry squad during urban patrol and defense missions.
Results suggested that desk-top simulations can be used to train the tactical
decisions leaders make during exercises that require greater expenditures of
resources. The methods used to train with simulations impacted leaders’ tactical
decision making, their perceptions of the training value of the simulations, and
their ideas about what they learned from the experience. Instructor-facilitated
training resulted in better tactical decision making.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7145
Nickolas D.
Macchiarella, Shawn M.
Doherty
Embry-Riddle
Aeronautical University
Daytona Beach,
Florida
An experimental
flight training program at Embry-Riddle Aeronautical University (ERAU)
investigated the degree to which flight simulation could be used in a training
curricula for ab initio pilots. The experimental group earned an FAA
private pilot’s certificate using a curriculum largely comprised of simulated
flight in flight training devices (FTD); the control group trained exclusively
in airplanes. What sets this
research apart from prior transfer of training (ToT)
work is the high level of simulator usage, (60% FTD-based flight) and the FAA’s
approval for certifying pilots with this methodology. Researchers hypothesized
that optimizing the application of specifically designed FTDs will afford the
simulation related training benefits typically associated with more costly
“graduate level” flight training (e.g., the FAA’s Advanced Qualification Program
[AQP] and military advanced aircraft type ratings) to ab initio flight training for a relatively low cost. At the core of this research
investigation was a classic transfer of training (ToT)
study examining 34 flight tasks.
Transfer can be measured using the transfer effectiveness ratio (TER)
equation. The results from the
amount of transfer from the FTD to actual aircraft flight suggested implications
for both adjustments to the flight training curricula and for specific
modifications to the FTD as applied in an ab-initio
training program. More specifically, these results provided an indication that
added visual fidelity, in terms of graphical 3D artwork, was necessary in the
virtual environment for particular ground reference maneuver tasks. A low fidelity visual scene at low level
flight altitudes provided poor cues for pilots training for ground reference
maneuvers. Additionally, the level
of traffic in the scenario and degree of complexity in simulated airspace
affected transfer to the real world flights.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7193
|
V. Alan
Spiker, Tricia
Mautone, Susan
Fischer Anacapa Sciences,
Inc. Santa
Barbara, CA |
Arizona
State University Mesa,
AZ |
Aircraft
operation involves many facets of multi-tasking (MT), where breakdowns in task
management have serious implications for performance and safety. There is a need
to develop valid, predictive tests of MT ability and provide practical criterion
measures of that performance. Thirty advanced students (15 pilot-copilot pairs)
in a university flight training program were tested in a medium-fidelity King
Air simulator. The 30-min. scenario was designed to task-load both pilot and
copilot during an event-filled instrument approach to an unpublished holding
point, challenging procedure turn, and steep descent with loss of glideslope and worsening weather, culminating in a missed
approach. Two observers independently scored subjects’ performance on 65
MT-relevant behavioral events and rated them on six process dimensions of MT.
Subjects also took a battery of predictive tests, including two
specifically-designed MT tests, and tests of fluid intelligence, processing
speed, and aviation mathematics. The MT tests assessed ability to simultaneously
monitor multiple visual fields and hold multi-dimensional concepts in memory.
The scenario challenged MT for all crews, with 40% of the events showing
evidence of disruption. Inter-observer reliability for MT ratings and sub-event
performance was high (r=.85-.91) and MT test reliability coefficients exceeded
.80. MT criterion measures showed a complex, but fascinating, relationship to
the predictive tests. One test of MT was significantly related to performance
for copilots but not pilots. The other test of MT predicted performance on two
key archival measures of student proficiency (hours to instrument rating, number
of extra flights). The paper closes with study implications for developing other
criterion measures of MT, adapting MT tests for student placement, and
developing MT training programs for “at-risk”
student-pilots.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7227
Amanda Hafich, Jennifer Fowlkes, Patrick Lenihan
CHI Systems,
Inc.
Orlando,
FL
The usefulness
of incorporating haptic devices into virtual
environments (VEs) is becoming an increasingly prevalent area of research in the
training arena. The work described
in this paper investigates the utility of incorporating haptics in a VE for Military Operations in Urban Terrain
(MOUT) training, focusing on two primary research questions: How do haptic
cues interact with and compare to auditory cues for the identification of
task-relevant VE events, and how does the inclusion of haptic cues impact presence using the Witmer & Singer (1998) Presence Questionnaire? To answer
these questions, participants performed a building search task eight successive
times in a desktop VE. Using vibrotactile stimulation applied to the torso, arms and
legs, participants received either no haptic cues,
haptic cues that were approximations of real-world
stimuli (“Natural Haptics”), or haptic cues which were metaphoric signals for real-world
stimuli (“Metaphoric Haptics”). Across the haptics groups, half of the participants were given auditory
cues for the stimuli events (explosions, gunshots, and collisions), and the
other half received no auditory stimuli.
All participants received identical visual cues. Results from 74 participants showed that
participants in the Natural Haptics condition showed
better identification of collision and gunshot events compared to participants
in the other groups, whereas participants receiving auditory cues showed better
identification of explosion events.
Regarding presence, the scores for participants in the Natural Haptics group only increased over trials on the Interface
Quality subscale, and participants in both haptic
groups showed increased scores over trials for the Adaptation and Immersion
subscale. However, there was not a general advantage in presence scores that was
related to haptics, perhaps due to ceiling effects.
Other findings showed that haptics was associated with
increased perceived task demands.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7257
Veronica Teichrieb, João Marcelo X. N. Teixeira, João Paulo S. do M. Lima, Judith Kelner
Federal
University of Pernambuco – UFPE, Computer Science
Center – CIn
Recife, Pernambuco, Brazil
Markerless augmented
reality (MAR) aims to integrate 3D virtual objects into the real environment in
real-time, enhancing the user’s perception and interaction. It differs from
marker based augmented reality (AR) systems basically by the method used to
place virtual objects in the real world. In MAR, the real environment may be
used as a marker that can be tracked in order to position virtual objects.
Generally, the techniques used are based on computer vision, image processing,
and computer graphics, and a major issue related to the field is robust, yet
precise real-time object tracking and registration. Several AR solutions using
general purpose devices have been developed. Such processing is done by
software, making it difficult to obtain real-time results without compromising
resolution and frame rate, and requiring the use of high clock frequencies that
consequently lead to higher costs and power consumption. This paper introduces
MARCam, an FPGA based solution that allows the
development of embedded MAR applications. This framework allows the development
of compact hardware with wearable capabilities for applications requiring user
mobility through unknown environments and real-time dedicated processing. One of
the most promising applications of this technology is guidance in training
systems. MARCam can utilize a Structure from Motion
(SfM) based technique. Instead of relying on
previously obtained information about the real scene, SfM based techniques estimate the camera displacement
without a priori knowledge of the environment. These methods are also able to
retrieve the structure of the scene in real-time, with various levels of detail.
Due to this, it is possible to reconstruct a totally unknown environment on the
fly. MARCam’s architecture is divided into many
circuit modules, each one responsible for a specific task. This way, one can
quickly arrange an assembly of modules and have in short time a fully-working,
dedicated MAR system.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7489
|
United
States Military Academy West
Point, NY |
Mark J.
Embrechts, Boleslaw
K. Szymanski Rensselaer
Polytechnic Institute Troy,
NY |
Unbalanced,
high-dimensional, binary classifications create challenges in a variety of
systems and environments, most notably within physical security and computer
security domains. The imbalance
within these problems consists of a significant majority of the negative
(healthy, non-intruding) class and a minority (unhealthy, intruding) positive
class. Any system that needs
protection from malicious activity, intruders, theft, or other types of breaches
in security must address this type of problem. Given numerical data that represent
observations or instances which require classification, many practitioners apply
state of the art machine learning algorithms to aid in solving unbalanced
classification problems. The
unbalanced and high-dimensional structure of the data can trouble these learning
methods. High-dimensional data poses a ``curse of dimensionality'' which can be
overcome through subspace modeling and intelligent fusion. A fundamental method for evaluation of
the binary classification model is the receiver operating characteristic (ROC)
curve and the area under the curve (AUC), and the intelligent fusion employed
ties directly with the properties of this evaluation method. This work exposes the underlying
statistics involved with ROC curves and leverages these properties to create
synergistic classifier fusion through rankings. Decision ROC charts are a novel
illustration that augment the ROC curve to provide a more complete
representation of the classifier performance. Pseudo-ROC curves, created from
simulated rankings utilizing principles based on the Wilcoxon-Rank sum or Mann-Whitney U statistic, provide novel
insight into the behavior of classifier rankings. The critical finding involves the unique
behavior of rankings for unbalanced classification problems and methods to
capitalize on this behavior to improve classifier accuracy for unbalanced
problems. Arguments presented
include theoretical discussion, proof of principle through simulated classifier
rankings examined with a factorial design, and experimental results with actual
data including host-based and network-based intrusion detection
datasets.
This paper is
available on the 2007 I/ITSEC CD ROM.
Order the paper
from I/ITSEC’s Website.
2007 Paper No.
7366
Feedback and guidance
is an essential component of any learning environment so
that errors in performance can be pointed out and corrected. A crucial part of
this process is the ability to identify gaps in skills or knowledge and assess
the underlying causes. Currently, there is little consistency in how a student
is evaluated across the different training environments they will encounter in
their career, whether in the school, in the field, at home or in simulated
training exercises.
The Joint ADL CoLaboratory, in cooperation with the US Navy, the US Army Research, Development,
and Engineering Command’s Simulation Training Technology Center, and the US
Army’s Program Executive Office for Simulation, Training and Instrumentation, is
funding the development of the Learner Assessment Data Model and Authoring Tools
(LADMAT) project. This project is focused on development of an assessment data
model and associated authoring tools that are capable of capturing complex
assessment data across multiple learning and training systems. Specifically, the
project will integrate assessment capabilities into a live simulation through
the One Tactical Engagement Simulation System and with multiple virtual
simulations using the Gamebryo game engine
and the Delta3D game engine to ensurethat training objectives are being met by the learner.
One key component of this research effort is to test and evaluate this technology as it is integrated into these
dynamic learning environments. This paper will discuss in detail the
complexities of assessing performance, the underlying technologies used in this
project to simplify the assessment process and how they can be used to help
standardize the manner in which the students is assessed throughout their
career….
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007 Paper No.
7398
A. Freedy, E. De Visser, G. Weltman
Perceptronics Solutions,
Inc.
Sherman Oaks,
California
U.S. Army
RDECOM-STTC
Unmanned
vehicles (UVs) are being developed and fielded at an unprecedented rate in
various environments for both civilian and military purposes. Despite the term
“unmanned,” control of such vehicles requires considerable inputs from those
operating the UVs and others in the C2 system. To optimize such mixed initiative teams
we must develop new methodologies and measurements for evaluating and
understanding human-robot team performance. The Mixed Initiative Team
Performance Assessment System (MITPAS) provides such new methodology. MITPAS
consists of models, tools and procedures, including an OneSAF-based simulation environment, with which to measure
the performance of mixed manned and unmanned teams in both training and real
world operational environments.
This paper describes MITPAS and the results of several initial
experiments conducted to validate the measures and gain insight into the effect
of robot competence on overall human-robot team performance, operator trust and
operator situational awareness. Our
initial results are indicative of the type of new insights into human-robot team
behavior that can be gained by combining the measurement power of MITPAS with
realistic simulations of tactical UV operations. Consequently, we are working to
make the readily-customized MITPAS available to other researchers and developers
for use with their simulations, scenarios and special
measures.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.