Research and Development

Real-time color blending of rendered and captured video

The Future of Mixed Reality: Issues in Illumination and Shadows

An Integrated Procedure for Measuring the Spatial and Temporal Resolution of Visual Displays

A Validation Methodology for Human Behavior Representation Models

Opening Up New Possibilities: Simulation-Based Tactics Mining

Adaptive and Modular M&S Configuration for Increased Reusability

Low Cost Virtual Cockpits for Air Combat Experimentation

Advanced Message Routing for Scalable Distributed Simulations

Load Balancing for Distributed Battlefield Simulations: Tradeoffs in Workload and Communications

Training in Virtual Environments:  Experimental Evaluations and Implementation Strategies

Within-Simulator Training Effectiveness Evaluation

Student vs. Software Pacing of Instruction: An Empirical Comparison of Effectiveness

Real-time translation of simulation data across multiple complex terrains

Digital Environment Data: Identifying Anomalies from Source to Final Databases

Myths and Truths of Interactive Volume Graphics

Building a Mobile Augmented Reality System for Embedded Training: Lessons Learned

Genetic Algorithm and Neural Network Hybrids for Controlling Mobile Robots

M&S within the Model Driven Architecture

Using Intelligent Agents to Control Teams of Robotic or Simulated Entities

 

 

Real-time color blending of rendered and captured video

 

Erik Reinhard, Ahmet Oguz Akyuz,

Mark Colbert, Charles E Hughes

School of Computer Science

University of Central Florida

Orlando FL

 

Matthew O’Connor

Institute for Simulation and Training

University of Central Florida

Orlando FL

 

Augmented reality involves mixing captured video with rendered elements in real-time. For augmented reality to be effective in training and simulation applications, the computer generated components need to blend in well with the captured video. Straightforward compositing is not sufficient, since the chromatic content of video and rendered data may be very different such that it is immediately obvious which parts of the composited image were rendered and which were captured.

 

We propose a simple and effective method to color-correct the computer generated imagery. The method relies on the computation of simple statistics such as mean and variance, but does so in an appropriately chosen color space - which is key to the effectiveness of our approach. By shifting and scaling the pixel data in the rendered stream to take on the mean and variance of the captured video stream, the rendered elements blend in very well.

Our implementation currently reads, color-corrects and composites video and rendered streams at a rate of more than 22 frames per second for a 720x480 pixel format. Without color correction, our implementation generates around 30 frames per second, indicating that our approach comes at a reasonably small computational cost.

2004 Paper No. 1502

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

The Future of Mixed Reality: Issues in Illumination and Shadows

 

Charles E. Hughes, Jaakko Konttinen, Sumanta N. Pattanaik 

School of Computer Science 

University of Central Florida 

Orlando, Florida 

 

Mixed Reality (MR) is a blending of real and virtual objects. How well that blending works is critical to a  user’s experience within an MR scenario. The focus of this paper is on the visual aspects of this blending;  other senses such as sound and haptics are covered elsewhere. 

 

Blending the real and virtual realities in MR requires that the virtual objects react properly to changes in  real lighting and that the real react properly to the insertion of virtual lights (e.g., a virtual flashlight). Even  more challenging, virtual objects must cast shadows on  real objects and vice versa.  Making this realistic  means that all such interactions must occur at interactive rates (30+ frames per second). 

 

Our research focuses on algorithmic development and implementation of these procedures on  programmable graphics units (GPUs) found commonly on today’s commodity graphics cards. The  algorithms we develop are tailored to take advantage of the parallel pipeline architecture of GPUs and to  carefully avoid some of the limitations found in currently available versions of these units.

2004 Paper No. 1883

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

An Integrated Procedure for Measuring the Spatial and Temporal Resolution of Visual Displays

 

Marc D. Winterbottom

Air Force Research Laboratory

Mesa, AZ 85296

 

George A. Geri

Link Simulation and Training 

Mesa, AZ 85296 

 

Bill Morgan

The Boeing Company 

Mesa, AZ 85296 

 

Byron J. Pierce

Air Force Research Laboratory 

Mesa, AZ 85296 

 

Spatial and temporal resolution are two of the most fundamental characteristics of visual displays, and yet they are  often incorrectly defined and specified.  In order to address this problem, we have developed techniques for  estimating both spatial and temporal resolution, and we have compared the resulting estimates to data obtained from  perceptual tasks.  The spatial resolution technique is based on a VESA standard (FPDM, Ver. 2.0), and was applied  to several CRT displays.  It was found that the pixel count does not adequately define display resolution when the  former exceeds the bandwidth of the display device.  In addition, the spatial resolution measurements were found to  correlate well with perceptual assessments of the orientation of target aircraft simulated at various distances.  The  temporal resolution technique involved measuring the response of various displays to simple light patterns that could  be flickered at up to 30 Hz.  Data obtained for CRT projectors indicated that temporal artifacts obtained with these  devices are due primarily to the limited frame rate of the image generator, rather than to limitations in the temporal  response of the projectors.  In addition, data obtained from liquid crystal on silicon (LCoS) projectors indicated that  their on- and off-responses are short enough to support 60 Hz simulator frame rates, but that the hold-time used to  maximize image luminance interacts with eye movements to produce temporal artifacts that can reduce the quality  of the displayed imagery.  The results of a perceptual test, based on the perceived separation of moving lines, were  consistent with the measured temporal resolution of the two displays. 

 

All measurement and analysis techniques described here have been implemented in a software package that is  available from AFRL, Mesa, Arizona.

2004 Paper No. 1855

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

A Validation Methodology for Human Behavior Representation Models

 

Lieutenant Colonel Simon R. Goerger, Ph.D.

Colonel Michael L. McGinnis, Ph.D.

United States Military Academy

Department of Systems Engineering

West Point, NY

 

Rudolph P. Darken, D.Sc.

Naval Postgraduate School

Modeling, Virtual Environments & Simulation

Monterey, CA

 

The Department of Defense relies heavily on mathematical models and computer simulations to analyze  and acquire new weapon systems. Models and simulations help decision-makers understand the differences  between systems and provide insights into the implications of weapon system tradeoffs. Given this key  role, the credibility of simulations is paramount. For combat models, this is gained through the verification,  validation, and accreditation process required of DoD analytical models prior to their use in weapon system  acquisition and other studies. The nature of nondeterministic human behavior makes validation of models  of human behavior representation contingent on the judgments of subject matter experts that are routinely  acquired using a face validation methodology. In an attempt to better understand the strengths and  weaknesses of assessing human behavior representation using experts, and the face validation  methodology, the authors conducted experiments to identify issues critical to utilizing human experts for  the purpose of ascertaining ways to  enrich the validation process for models relying on human behavior  representation. The research was limited to the behaviors of individuals engaged in close combat in an  urban environment. This paper presents the study  methodology, data analysis, and recommendations for  mitigating attendant problems with validation of human behavior representation models

2004 Paper No. 1589

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Opening Up New Possibilities: Simulation-Based Tactics Mining

 

Sui Qing, Yeo Ye Chuan, How Khee Yin, Tan Poey Guan

DSO National Laboratories 

20 Science Park Drive, Singapore 118230 

 

Modeling and Simulation (M&S) has been effectively made use of by the military for applications like training and  supporting acquisition decisions. This paper proposes a new technology called Simulation Based Tactics Mining  (SBTM) which will open up new possibilities for military applications of M&S systems.  The goal of SBTM is to  explore the use of machine learning techniques to enable one or more simulated agents to learn novel ways of task  execution through interaction with the simulated environment. For instance, SBTM could enable the military to use  an M&S environment to explore the space of possible tactics for a system of unmanned platforms to engage a system  of ground targets. Another example is to use SBTM on robotic simulation systems to automatically acquire the rules  for a system of multiple robots to search and localize multiple targets. The rules acquired by SBTM from robotic  simulation environment can then be tested on actual physical robots. 

2004 Paper No. 1494

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Adaptive and Modular M&S Configuration for Increased Reusability

 

Nathalie Harrison, Bruno Gilbert, Alfred Jeffrey, Marc Lauzon, Richard Lestage  

Defence R&D Canada - Valcartier

Quebec, Canada

 

In the context of Modeling and Simulation (M&S) proliferation, modelers can take advantage of a Model-Driven  Development (MDD) approach to increase the reuse of their models within different implementation frameworks. 

 

However, this MDD approach can still lead to monolithic M&S components of limited reusability. This paper  focuses on the simulation modeling phase of a MDD approach applied to M&S. It presents a novel XML-based  method to increase the reusability of M&S components through fine granularity. Granular instance-specific data  elements are associated to granular generic model elements. Configurable simulation modeling data include model  parameters, entity composition, scenario composition and log configuration. These elements can be reused through  various models, scenarios and frameworks. The data and the components are assembled at run-time to build a  specific simulation application. This method has been implemented in a M&S process designed for weapon system  modelers who need to integrate their models into different simulation frameworks. The reusability of the resulting  model components and XML data files was demonstrated in instantiating engagement simulations with various  entities dynamically composed of defensive self-protection suites and weapon sub- systems having different  parameters. An adaptive M&S configuration tool was developed to support this flexibility. It allows to create  compositions and to dynamically display new parameters in adapting itself to the content of the XML data files.  Simulation modeling, including scenario creation, is considered to be the entry point of any simulation framework. 

 

However, the model component interoperability is often limited since it is generally related to proprietary file  formats and graphical user interfaces. It is believed that the method proposed in this paper could increase the  interoperability and ease the exchange of models if XML schemas were standardized.

2004 Paper No. 1864

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Low Cost Virtual Cockpits for Air Combat Experimentation

 

MAJ Chien Wei Chia, Ph.D.

Singapore Armed Forces Centre for Military Experimentation (SCME), Future Systems Directorate (FSD),

Ministry of Defence, Singapore

 

The preamble in any experimentation usually involves a stage known as  “screening”  (Montgomery, 2001) or  “discovery”  (Alberts & Hayes, 2002). In general, the idea behind this stage is to identify the major independent  variables that appear to have greater influences on the measures so that subsequent stages could be more focused on  the important variables. This paper describes the research in which an innovative solution for a simulation  environment was found to more cost-efficiently enable air combat discovery experimentation. In essence, the  investigation was to see how the limited 60 degrees field of view imposed by a single display desktop virtual cockpit  system could be circumvented cost efficiently.  The solution in this case was the addition of a 2D display  augmentation that provides information of what would be acquired visually by the pilots up in the air.  The challenge  was to decide the manner in which the information will be portrayed on this display. The human factors literature  provided the guiding light during the research and helped narrow the effort down to an experiment with the Frames  of Reference problem, i.e. deciding whether it should be egocentric or exocentric. With these insights an experiment  was designed to benchmark the performance of the solution. The performances of 6 operational pilots were  compared in four environments  (domed simulator, desktop with no augmentation, desktop with egocentric  augmentation and desktop with exocentric augmentation.) using a full-factorial experiment design with 3 replicates  each. The outcome of the experiment showed that there was no statistical difference between the performance of the  domed simulator and the desktop with the egocentric  augmentation.

2004 Paper No. 1596

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Advanced Message Routing for Scalable Distributed Simulations

 

Brian Barrett

University of Southern California

Marina del Rey, CA

 

Thomas Gottschalk

California Institute of Technology

Pasadena, CA

 

The  Joint  Forces  Command  (JFCOM)  Experimentation  Directorate (J9)'s  recent  Joint  Urban  Operations  (JUO) experiments have demonstrated the viability of Forces Modeling  and  Simulation  in  a  distributed  environment.  The JSAF application  suite,  combined  with  the  RTI-s  communications  system,  provides  the  ability  to  run  distributed simulations  with  sites  located across the  United  States,  from  Norfolk,  Virginia  to  Maui,  Hawaii.  Interest-aware routers are essential  for communications  in  the  large,  distributed  environments,  and  the  current  RTI-s  framework provides such routers connected in a straightforward tree topology. This approach is successful for small  to  medium sized simulations, but faces  a  number of  significant  limitations  for very large simulations  over high-latency,  wide area networks.  In  particular, traffic is  forced through  a  single  site,  drastically increasing  distances  messages  must travel to sites not near the top of the tree.  Aggregate bandwidth is limited  to  the  bandwidth  of  the  site  hosting  the top  router,  and  failures  in  the  upper  levels  of  the  router  tree  can  result  in  widespread  communications  losses throughout the system.

 

To  resolve  these  issues,  this  work  extends  the  RTI-s  software  router  infrastructure  to  accommodate  more sophisticated, general router topologies, including both the existing  tree framework and  a  new generalization of  the fully connected mesh topologies used  in  the  SF  Express ModSAF  simulations  of  100K  fully  interacting vehicles. The new software router objects incorporate the  scalable features of  the  SF  Express design,  while  optionally  using low-level RTI-s objects  to  perform actual site-to-site  communications.  The limitations  of  the  original  mesh  router formalism  have  been  eliminated,  allowing  fully  dynamic  operations.  The  mesh  topology  capabilities  allow aggregate bandwidth and site-to-site latencies to  match  actual network performance. The heavy resource load  at  the root node can now be distributed across routers at the participating sites.

2004 Paper No. 1832

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Load Balancing for Distributed Battlefield Simulations: Tradeoffs in Workload and Communications

 

David R. Pratt, Ph.D.

SAIC

Orlando, FL

 

Amy E. Henninger, Ph.D.

Soar Technology, Inc.

Orlando, FL

 

Load balancing attempts to optimize the utilization of processors in a parallel computing systems, and dynamic load  balancing is a prime candidate for improving the performance of distributed battlefield simulation systems.  Last  year  we  reported on development of test-bed developed to assist in the empirical exploration of a number of  dynamic load balancing heuristics.  Unique to the test-bed was a modification of the classical discrete-event  simulation (DES) scheduling paradigm that enabled  us to determine processor lo ad at the application level.   Experimental runs considered a number of load-balancing heuristics, corroborated results reported by  other  researchers, and provided confidence that our approach is indeed feasible. 

 

Absent from this initial study  was a consideration of cost measures for the system and a recognition of the conflict  between  distributing  workload  evenly  and minimizing communication cost s.  For a load balancing system to be  effective, the cost  of balancing load must be less than the cost  of the status quo.  The cost  is manifested by  the  monitoring, selection, transport, and initialization time, versus the processing and bandwidth requirements. Without  proper monitoring and calibration, it possible to spend more time trying to balance the load than it does actually  processing productive work. In this paper we present the implementation of additional test-bed infrastructure  designed to capture these tradeoffs.  Moreover, we motivate the selection of heuristics to be considered, present the  results of experimental runs with these heuristics, and discuss the implications of the results. 

2004 Paper No. 1638

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Training in Virtual Environments:  Experimental Evaluations and Implementation Strategies

 

Barbara Buck and Bruce Perrin

The Boeing Company 

St. Louis, MO 

 

This paper summarizes a series of research studies conducted over a period of six years in which we investigated the  effectiveness of virtual environments for aircraft maintenance training.  In these studies, we systematically evaluated  numerous aspects of virtual maintenance trainers (VMTs) that may impact training effectiveness, including comparisons of  immersive vs. desktop virtual environments; high vs. low detail graphics; task selection and specific tasking requirements;  and the effects of individual differences on learning effectiveness within virtual environments.  In each study, we conducted  an evaluation of training effectiveness, collecting objective performance measures of declarative knowledge, training  transfer task performance time, and transfer performance errors.   In addition, individual difference measures of spatial  reasoning aptitude, computer/video game experience, and hand tool experience afforded the opportunity to analyze the  impacts of these variables on VMT effectiveness.  Within each study, rigorous experimental protocol in training procedures  and data collection remained constant, allowing us to compare results across the multiple studies.  The paper highlights  significant results from each of these individual studies, as well as generalized findings across all of the studies.  Our studies  showed that interactivity and high graphic detail are important for training effectiveness, and that desktop VE trainers were a  significantly more effective medium than immersive training.  In addition, individual spatial reasoning aptitude is a  significant mediator of virtual training effectiveness when the task would be learned and performed with limited visual  feedback in the real world (e.g., a tactile or “blind” task).  The impact of the spatial processing aptitude is similar for both  hardware and VE-based training, however, when the task is normally learned and performed in the visual field.  This paper  summarizes these findings, as well as other study results, as a series of empirically driven guidelines for the implementation  of virtual environments for aircraft maintenance training.

2004 Paper No. 1635

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Within-Simulator Training Effectiveness Evaluation

 

Sara Elizabeth Gehr

Boeing

Mesa, AZ

 

Brian Schreiber

Lockheed Martin

Mesa, AZ

 

Winston Bennett, Jr.

Air Force Research Laboratory

Mesa, AZ

 

There exists a need to formally assess the training benefits of Distributed Mission Operations (DMO) training on the  performance of F-16 pilots.  DMO training consists of multiplayer networked environments designed to enhance  warfighter competency.  Although many studies have converged on the effectiveness of training pilots in stand  alone systems, very little research has been done on the  effectiveness of DMO training of multiple pilots in  networked simulators.  As Bell and Waag (1998) outlined,  to establish support for the  effectiveness of training,  several different levels of converging support are needed.  A proper approach would involve collecting data from  several sources that taken together will lend support to  the significance of DMO training.  To establish the  effectiveness of networked simulated training, evidence from a variety of sources will be examined, including: (1)  objective indicators of the performance of the pilots acting as a four-ship team engaged in point-defense actions, (2)  ratings of team performance made by subject matter experts (SME), (3) scaling evidence collected using the  Pathfinder paired-comparison methodology, and (4) pilot reactions to DMO as recorded  on rating forms collected.  

 

Although all four types of data should show support for the effectiveness of DMO training, the inclusion of  objective data allows stronger conclusions to be drawn.   Objective data enables quantification of the subjective  opinions and ratings, thereby providing indications of the return on investment (ROI), in terms of increased human  performance, of the training system.  Our current work involves assessing pilots using these methods, and the  results should address the changes in capability of training our warfighters.

2004 Paper No. 1778

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Student vs. Software Pacing of Instruction: An Empirical Comparison of Effectiveness

 

Bruce M. Perrin, Brandt W. Dargue, and Frederick Z. Banks  

Training Systems & Services, Boeing 

St. Louis, MO  63166 

 

The research on how people control their learning processes (e.g., metamemory) indicates that students are sensitive  to the difficulty of the material to be learned and will allocate their study time accordingly.  In general, they will  devote more time to more difficult material.  This research also suggests, however, that students’ control of pace is  not perfect and they do not always increase study time sufficiently for more difficult topics.  As a supplement to  learner-controlled allocation of study time, adaptive learning methods have been proposed that adjust time on a topic  according to student performance.  The relative effectiveness of software vs. student-controlled pacing, however, has  not been empirically evaluated.  The current study provides data relevant to this issue.  We evaluated two adaptive  learning methods, which are based on the capabilities provided by Advanced Distributed Learning  (ADL™ )  Shareable Content Object Reference Model (SCORM®) 2004.  By using performance during training to diagnose  student learning needs, these methods schedule additional practice and review as required.  Our results indicate that  adaptive methods significantly increased student learning, compared to classroom instruction or self-paced,  computer-delivered training.  Although students using the adaptive learning courses did study longer, statistical  analysis indicated that the increase in learning was directly attributable to how study time was allocated to specific  topics, and not just the increase in time spent.  Implications for adaptive learning and ADL SCORM-based training  are discussed, as well as directions for future research.

2004 Paper No. 1671

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Real-time translation of simulation data across multiple complex terrains

 

Matthew Olson

SAIC

Arlington, VA

 

Ben Wise, Ph.D.

SAIC

Vienna, VA

 

Ed Ronan

SAIC

Burlington, MA

 

Wes Braudaway, Ph.D.

SAIC

Arlington, VA

 

We present with this paper the results of an ongoing research and development effort in which we integrate multiple  live and synthetic environments representing complex terrain at different world locations into a single location  environment using real-time entity translation. Our efforts stem from the increased number of US military  engagements in complex terrains like Afghan caves,  Philippine jungles, and Iraqi neighborhoods and the  commensurate need for improvements in training and analysis capabilities for such operations. The combined use of  live ranges and simulations can expand these capabilities,  but the introduction of simulations requires that we  generate from multiple terrain representations, a single simulation environment in which constructive, live, and  virtual entities interact. 

 

Various means of generating this unified environment exist. We discuss currently used general methodologies and  their associated trade-offs. We then describe a method of real-time entity translation that allow integration of  multiple live ranges and simulated models of complex terrains at different world locations to produce a single  location environment. The integration process is rapid  and inexpensive, and it requires no modification of the  simulation tools used or their terrain models.  Run-time processing is computationally inexpensive. 

 

Our real-time method shifts position, velocity, and orientation relative to immediate surroundings for all simulation  data. The translation of orientation is accurate to the  point that small unit, small  arms combat between entities  modeled to exist at different locations on the globe is no different than small unit, small arms combat between  entities modeled in the same terrain. Minimal perceptible degradation of fidelity or realism results from the  translation.

2004 Paper No. 1763

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Digital Environment Data: Identifying Anomalies from Source to Final Databases

 

Robert F. Richbourg, Timothy M. Stone, and George E. Lukes 

Institute for Defense Analyses

Alexandria, VA

 

Digital representations of the environment are being used in a wider spectrum of applications.   Military units deploy to operational  areas  with  a digital  database  of that  area.  This  is one example  of the many  factors  that  have  led the National  Geospatial-Intelligence  Agency  (NGA,  formerly  NIMA)  to adopt  a new  data  production  strategy  focused on providing digital geospatial data in addition to traditional paper map and chart products.   The NGA emphasis on digital geospatial data promises new opportunities for simulation database developers and users.  Simulation systems already  utilize  large-scale,  high-fidelity,  geo-specific  databases  to  execute  joint  experiments  including  urban  area operations.   Expanded  availability  of digital  source  data  can  only  increase  this  trend;  however,  the  new  geospatial data  production  model,  coupled  with  more  rigorous  demands  and  expectations  from  the  operational  community, results in continued tension between data quantity and data quality within a crisis-response  production environment. Processes  developed  and refined  to produce  traditional  maps  and charts  are not sufficient  to meet  the demands  for multi-purpose  digital  geospatial  data.   This  paper  reports  on results  of research  into identifying the  data  anomalies that may arise in such an environment and describes the development of automated tools that can be applied early in the production process to detect those anomalies.

2004 Paper No. 1675

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Myths and Truths of Interactive Volume Graphics

 

Andrew Woo, Paul Halmshaw 

NGRAIN Corporation

Vancouver, BC, Canada

 

Within the 3D graphics industry, there are many myths surrounding the use and effectiveness of volume graphics  (based on voxels). The most common myths–that voxels produce poor image quality and slow performance, require  significant memory and are not  am enable to  compression–prevent many  developers  from  seriously  considering  voxels as a viable solution for their interactive  visualization needs, and reinforce their continued dependence on  standard surface technologies. In this  paper, the authors examine and dispel  the most common myths associated  with volume graphics, by referring to the most current state-of-the-art literature, research and development work.  Having shed light on these misunderstandings,  this  paper will further illustrate that a voxel representation can  efficiently serve many interactive visualization  needs in  areas  such as training applications for dangerous or  complex machinery, dynamic synthetic environment, and battle damage assessment.

2004 Paper No. 1755

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Building a Mobile Augmented Reality System for Embedded Training: Lessons Learned

 

Dennis G. Brown, David Armoza,

Mark A. Livingston, Lawrence J. Rosenblum

Advanced Information Technology

Naval Research Laboratory

Washington, DC

 

Yohan Baillot, Simon J. Julier

ITT Advanced Engineering and Sciences

Alexandria, VA

 

Paul Maassel

ReallaeR, LLC

Port Republic, MD

 

Mobile augmented reality (AR) is a method for providing a “head up display”  to individual dismounted users.  A  user  wears  a miniaturized computer system, tracking sensors, and a see-through graphics display. The system  superimposes three-dimensional spatially registered graphics and sounds onto the user’s perception of the  real  world.  Because information can be presented in a head up and hands free way, it has the potential to revolutionize  the way in which information is presented to individuals. 

 

A mobile AR system can insert friendly, neutral, and enemy computer-generated forces  (CGFs) into the real world  for training  and mission rehearsal  applications.  The CGFs are drawn realistically and properly occluded with  respect to the real  world.  The behaviors of the CGFs  are  generated  from two  Semi-Automated  Forces  (SAF)  systems: JointSAF and OneSAF.  The AR user appears as an  individual combatant entity in the SAF system.  The  AR user's position and orientation are fed to the SAF system, and the state of the SAF entities is reflected in the AR  display. The SAF entities react to the AR user just as they do any other individual combatant entity, and the AR user  interacts with the CGFs in real time. 

In this paper, we document the development of a prototype mobile AR system for embedded training and its usage  in MOUT-like situations.  We discuss the tradeoffs of  the components of the hardware (t racking technologies,  display technologies, computing technologies) and the software (networking, SAF systems, CGF generation, model  construction), and we describe the lessons that have been learned from implementing several scenarios.

2004 Paper No. 1575

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Genetic Algorithm and Neural Network Hybrids for Controlling Mobile Robots

 

Jimmy Secretan

University of Central Florida

Orlando, FL

 

Guy A. Schiavone

Institute for Simulation and Training

Orlando, FL

 

As the hardware capabilities of unmanned battlefield robots, such as Micro Aerial Vehicles (MAVs) and Unmanned  Ground  Vehicles  (UGVs),  increases,  so  to  must  the  intelligence of  the  software  controlling  them.    Genetic  Algorithms  (GAs)  and  Genetic  Programming  (GP)  have  proven  effective  in  preliminary  MAV  and  UGV  simulations  for  evolving  simple  tracking  and  surveillance  behaviors.    However,  the  reactive  approach  that  most  robotic GAs provide falls short of demonstrating a comprehensive range of intelligence.  If for instance, an object  becomes  occluded  from  a  robot's  view,  GAs  usually  must  evolve to  considerable  complexity  before  they  can  effectively handle such situations.  In this paper, we suggest an approach whereby we augment the GA with a neural  network predictor as one of its inputs. The robot’s task consists of following another moving object and maintaining  a certain distance.  The neural network system is trained with the behavior of the robot's intended target, and feeds  this as an input to the GA.  We present simulation results of how well this method achieves its task, as well as  suggestions for adapting these techniques for implementation on advanced mobile cluster computers.

2004 Paper No. 1642

 

 

 

M&S within the Model Driven Architecture

 

Andreas Tolk, Ph.D.,  James A. Muguira 

Virginia Modeling Analysis & Simulation Center (VMASC) 

Old Dominion University, Norfolk, Virginia

 

Currently, standardized distributed simulation systems are likely to follow the High Level Architecture (HLA) standard, not only because it is widely adopted in the United States for DoD applications, but also because it can be seen  as the most matured M&S standard in this domain.  However, the HLA focuses on the implementation level; thus it  cannot solve problems on the conceptual level, such as validation, verification, or composability of models.  The  Discrete Event System Specification (DEV S) deals successfully with  the conceptual challenges, but its application  is limited to the academic community.   The M&S community is still looking for a general method insuring interoperability on all levels, from the syntactic level via the semantic level to the pragmatic/dynamic level, and maybe  even to the most general conceptual level.  To this end, the communication-driven ideas of the HLA must be merged  with the conceptually driven ideas of DEVS, preferably in a commercially viable way. 

 

One candidate is the Model Driven Architecture (MDA), proposed by the Object Management Group (OMG).  The  MDA can best be described as an overarching standard framework merging various middleware solutions and platform independent models of various application domains, using the Unified Modeling Language (UML) as the common concept gluing the various components together. 

 

This paper introduces the concepts of the MDA and shows, how the complementary ideas and methods of the HLA  and DEVS can be merged into a well-defined M&S application domain within the MDA framework, allowing heterogeneous solutions as well as the migration from existing solutions to alternatives.  The focus of this paper is the  proposal of a framework of methods ensuring reusability, composability, and orchestration of events in a heterogeneous M&S and information technology environment, based on the migration of successful and accepted concepts.

2004 Paper No. 1477

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.

 

 

 

Using Intelligent Agents to Control Teams of Robotic or Simulated Entities

 

Scott D. Wood, Jack D. Zaientz  

Soar Technology 

Ann Arbor, MI 

 

DARPA’s vision of the future for armored and mechanized military structure includes the use of mixed teams of  human and robotic forces on a dynamic and rapidly changing battlefield. Successful implementation of this shift will  require autonomous and semi-autonomous robotic forces and a command and control infrastructure that will allow  human, robotic, and mixed teams to be controlled quickly and easily. This infrastructure will need to allow human  commanders to control the robot teams in a similar manner to how they command human teams, that is, in the  language of  the military, not the language of robotic control theory. Furthermore, the human interface for robotic  command and control must simplify the controller’s tasks and automate processes in such a way that the cognitive  workload is reduced, situation awareness is enhanced, and situational control is preserved.  

 

In this paper we present a theoretic approach to creating such a system. The approach is centered on recasting the  robotic control problem as a supervisory control problem, focusing on the human commander as supervisor of an  intelligent command and control (C2) system. To explore and evaluate this approach our team is building a C2  framework of cooperative interface agents that reflect roles found in current command staffs. The intention is to  create a virtual command staff for the robotic controller by embedding these military functions within the system  itself. Although this work is ongoing, the results to date suggest that the approach we have taken will significantly  reduce cognitive and human performance workload, thus improving overall usability and reducing training  requirements. In addition to robotic control, this work opens up a new approach to controlling simulated entities  during experimentation, training and mission rehearsal.

2004 Paper No. 1867

This paper is available on the 2004 I/ITSEC CD ROM.

Order it from I/ITSEC'S Website.