Research and development
Detecting Asymmetric Plans Using Case-Based Reasoning
Verbal Protocol Analysis of UAV Subject Matter Experts
An Approach to the Construction of Multi-agent Models for
Planning
A Formal Approach to Composability
An Experiment in Simulation Coercion
Linking Assessment and Instruction Using Ontologies
“Focused Fidelity”: Applying Hybrid Rendering Technology
to Dynamic Synthetic Environments
Interactive Display of Complex Environments
An Automated Testing Perspective of Graphical User Interfaces
Joint Experimentation on Scalable Parallel Processors
Load Balancing for Distributed Battlefield Simulations:
Initial Results
Real Time with a Red Hat Analysis of Standard Linux for
Real-Time Simulation
An Optimization Methodology to Investigate Operator Impact
on Quality of Service
Beowulf Supercomputing for Mobile Applications
Detecting Asymmetric Plans Using Case-Based ReasoningStottler Henke\ San Mateo, CA CHI Systems Lower Gwynedd, PA In recent years, government agencies have displayed a growing interest in the prospect of detecting the activity of clandestine organizations. The clandestine organization and the government agencies who oppose them are an example of asymmetric warfare, which comes in contrast to traditional notions of armed conflict involving force-on-force scenarios where opposing sides can be measured according to force size, weapon assets, etc. We present an approach to the problem of detecting the execution of mission plans by the unconventional side in asymmetric warfare. The problem is to find threatening patterns of action in a data collection characterized as massive, relational, incomplete, noisy, and corrupt. In this paper we describe Sibyl: a system embodying a case-based reasoning (CBR) approach to automated plan detection. Sibyl features a “spanning case base” that covers the space of theoretical scenarios. Each case is used in a state-space search algorithm to adapt case elements to the data. Sibyl also features a graphical programming language that allows analysts to draw patterns to be found in an evidence database. We describe experimental results obtained for the Russian mafia domain used by DARPA’s Evidence Extraction and Link Discovery (EELD) program. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Verbal Protocol Analysis of UAV Subject Matter ExpertsAir Force Research Laboratory, Warfighter Training Research Division Mesa, Arizona L-3 Communications, Inc. Mesa, Arizona United States Air Force Academy Colorado Springs, Colorado Scientists at the Air Force Research Laboratory’s Warfighter Training Research Division in Mesa, AZ are engaged in a basic research program to advance the state of the art in computational process models of human performance in complex, dynamic environments. Current modeling efforts are focused on developing and validating a fine-grained cognitive process model of the Uninhabited Aerial Vehicle (UAV) operator. The model is implemented in the ACT-R cognitive modeling architecture. The design of the model is inspired by the well-known “Control and Performance Concept” in aviation. The study described here was conducted in order to assess how accurately the model represents the information processing activities of expert pilots as they are flying basic maneuvers with a UAV simulation. The data suggest: (a) pilots verbalize attention to performance instruments more often than control instruments, despite the fact that they generally appear to be using the control and performance concept to fly these maneuvers, (b) the distribution of operator attention across instruments is influenced by the goals and requirements of the maneuver, and (c) although the model is an excellent approximation to the average proficiency level of expert aviators, for an even better match to the process data, the model should be extended to include the use of trim and a meta-cognitive awareness of the passage of time. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
An Approach to the Construction of Multi-agent Models for PlanningEscuela de Informática del Ejército C/ Prim, 4 28004 Madrid (Spain) Universidad Politécnica de Madrid Ciudad Universitaria s/n 28070 Madrid (Spain) Efficient planning has been a challenge for mankind for centuries. Reaching objectives can imply the selection of different approaches but each approach can provide different effects or results. The correct assignment of resources and the study of objectives and scenarios are the keys to making an efficient plan. The field in which planning can be applied is enormous and consequently, a specific solution will be required for any concrete problem. Despite the wide range of problems in which planning is used, it is possible to utilize a common methodology that allows us to reach specific planning objectives for a real problem. In this paper we will introduce a methodology that permits us to construct models in order to solve planning problems in strategic or tactical environments through the use of Artificial Intelligence procedures. The methodology is based upon the Scientific Method and it is divided into several steps: the study of the real system’s characteristics, the specification of aims, the selection of agents, the formulation of the model and the verification and validation phase. The agents are constructed upon the foundation of Artificial Intelligence procedures. Each agent is responsible for carrying out a specific task such as classifying, assigning, fuzzifying or optimizing an intelligent search. The use of agents as efficient actors facilitates us in providing the solution that best fits the objectives for the plan. We have built two prototypes to solve planning in different environments. Within the Tactical Planning Method, we have built a military prototype for Field Artillery planning. Additionally, we have created a prototype for Strategic Planning that presents us with an alternative solution to the well-known ‘Prospective’ technique. The results obtained through the construction of the prototypes allows us to assure that the methodology stated above provides a proven and tested way of solving problems within the planning context. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
A Formal Approach to ComposabilityVirginia Modeling, Analysis and Simulation Center Old Dominion University Suffolk, Virginia Composability is the capability to select and assemble simulation components in various combinations into simulation systems to satisfy specific user requirements. The defining characteristic of composability is the ability to combine and recombine simulation components into different simulation systems. Two types of composability are considered: syntactic and semantic. Syntactic composability is the actual implementation of composability and requires that the composable components be constructed so that their implementation details are compatible for the different configurations that might be composed. In contrast, semantic composability is a question of whether the models that make up the composed simulation system can be meaningfully composed, that is, if their combined computation is valid. There has been significant work on syntactic composability but almost none on semantic composability. We have developed a formal theory of semantic composability that enables the determination, in a formal way, of whether a composition of models is valid. In this paper we provide an overview of the essentials of the theory. Definitions of model, simulation, validity and composability are stated and arguments that the definitions are appropriate for the purpose are given. The apparent disconnect between formal reasoning and semantics is resolved using the definition of validity. Theorems regarding the conditions under which compositions of models are valid and the computational complexity of determining the validity of compositions are described. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
An Experiment in Simulation CoercionDepartment of Computer Science, University of Virginia 151 Engineer’s Way, PO Box 400740 Charlotesville, VA If simulations could be coerced -- literally reshaped -- to conform to requirements different from those for which they were originally designed, many of the challenges facing the simulation community should become less daunting. Success could foster reuse, enable linkages between multi-resolution models, and increase the chances of success for interoperability and composability. We consider coercing as it applies to multi-resolution modeling. Given two simulations of the same phenomenon at different levels of detail, we explore coercing the lower resolution simplified simulation to produce results that correspond satisfactorily with the detailed high resolution simulation. The product is a simulation possessing the speed of the low resolution simulation along with the desired accuracy of the high resolution simulation.
Coercing involves a subject matter expert and optimization. The subject matter expert selects simulation parameters and establishes constraints within which those parameters are allowed to vary without compromising the validity of the simulation. Then, an optimization technique is employed to search through the possible parameter values and to select that set for which the simulation results most closely reflect the ideal; namely, conformance with new requirements stemming from the high resolution model. To explore the concept of coercing simulations, we selected a high resolution three-dimensional bicyclist simulation and a simple simulation of a particle moving in two-dimensional space as the low resolution simulation. For these two simulations, we were able to modify, using optimization, the parameters of the low resolution simulation to follow a route that more accurately reflected the route traced by the three-dimensional bicyclist on a given course. We report on our coercing experience, providing detailed insight into the process we have designed for coercing, and we describe results. Also, we discuss automating additional phases of the coercion process and their integration into our evolving coercion tool, SimEx. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Analysis of Tradeoffs in Modeling Continuous Speech Recognition for Domain Specific Training ApplicationNAVAIR Orlando Orlando, Florida NAVAIR Orlando Orlando, Florida NAVAIR Orlando Orlando, Florida NAVAIR Orlando Orlando, Florida Since the 1980's the underlying technology in speech recognition has been the Hidden Markov Model (HMM), an accurate process to statistically model continuous human speech. A speech model is represented as combination of probabilities associated with both acoustic and language models. Acoustic models estimate the probability associated with postulated sequence of acoustic observations. Language models describe the probability associated with postulated sequence of words and can incorporate both syntactic and semantic constraints of the language. When developing speech recognition for training systems, both acoustic and language models are crafted for the application. Due to the complexity in building a tuned accurate speech recognition application, it is necessary to understand how acoustic and language models affect accuracy. The Speech Technology Group (STG) at NAVAIR Orlando develops acoustic and language models specifically for the Navy Air Traffic Control (ATC) trainers, in contrast to commercial-off-the-shelf speech tools that contain generic acoustic models with limited alterability. The present study evaluates several speech model configurations including word pair (bi-gram) models. The STG, under laboratory conditions, measured the effects of accuracy of the following variables: vocabulary, perplexity, acoustic models, and language models. The findings of this study describe the influence of acoustic and language modeling on speech recognition. These lessons learned provide a better understanding of how speech model parameters influence model accuracy and can be used to more efficiently incorporate speech recognition within training applications, thereby enhancing the learning performance of the war-fighter. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Using the Learning Orientations Model to Design Personalized Environments for Successful Learning on the WebThe Training Place, Inc. Oro Valley, AZ Our national priority is preparing lifelong learners who competently respond to rapid changes/opportunities. We need more reliable personalized learning models that support the way we learn differently--faster, better, smarter, cheaper, continually, and intentionally. If we fail to develop new adaptive learning perspectives, theories, and models that (a) clearly address the more independent nature of online learning and individual learning differences, (b) understand the impact of emotions and intentions on learning, communication, and interaction (c) apply reliable measures to differentiate learning audiences, and (d) adapt realistic adaptive learning solutions to fundamental differences, then our solutions will to continue to be disappointing and the results nonsignificant. The Web offers the perfect technology and environment for more precise adaptive learning because learners can be uniquely identified, relevant content can be specifically personalized, flexibly sequenced, and subsequent response and progress can be monitored, supported, and assessed. Technologically, researchers are making rapid progress realizing the personalized learning dream with object architecture and adaptive technology. The missing link is a truly personal, whole-person understanding with scientific models showing how individuals learn differently (more than just how they process and build knowledge). This study introduces (a) learner-difference profiles describing sources for individual learning differences, (b) research designs and analytical models to support these differences, and (c) strategies for helping researchers tap into the dominant influence of emotions, intentions, and social factors on learning, achievement and performance improvements. These insights offer simple ways to enhance and evaluate contemporary Web instructional designs so that they instill the right habits for continuous learning, achievement, and performance improvement. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Characterizing Trainees in the Cognitive Phase using the Human Performance Knowledge Mapping Tool (HPKMT) and Microgenetic AnalysisNational Center for Research on Evaluation, Standards, and Student Testing (CRESST) University of California, Los Angeles Los Angeles, CA Models of skill acquisition suggest that learners go through three phrases: (1) cognitive phase—when instruction is most effective, errors are frequent, and performance is inconsistent; (2) associative phase—when the learner begins to integrate the parts of the process or domain as a whole, and errors gradually become eliminated; and (3) autonomous phase—when the process becomes more automatic and less moderated by cognition, and there is less interference from outside distracters. In this paper, we will examine the use of the CRESST Human Performance Knowledge Mapping tool (HPKMT) as a tool for microgenetic analysis to characterize learners in the cognitive phase using Marine Corps 2nd Lieutenants going through entry-level marksmanship training. The capability to characterize learners may direct the level of instruction or practice she is given. HPKMT is designed to measure a learner’s knowledge of a domain. Learners express their understanding of a domain by graphically depicting the relations among concepts. Further, the microgenetic analysis methodology provides a finer picture of the learning process by using repeated observations throughout the period of change giving detailed analysis of how and when change occurs. By measuring a Marine’s knowledge of marksmanship during classroom training, dry-fire practice, live-fire practice, and after qualification, we will have observations of their performance on the HPKMT at key stages of their learning. Our results suggest that the HPKMT has the potential to identify four types of learners in the cognitive phase: growing, declining, stable, and erratic. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Linking Assessment and Instruction Using OntologiesUCLA/CRESST Los Angeles, CA In this study we report on a test of a method that uses ontologies to individualize instruction by directly linking assessment results to the delivery of relevant content. Our sample was 2nd Lieutenants undergoing entry-level training on rifle marksmanship. Ontologies are explicit expressions of the concepts in a domain, the links among the concepts, and the governing constraints of these links. We have developed an ontology for the domain of rifle marksmanship. The ontology contains over 160 concepts and over 160 relationships that capture the different types of relations among the concepts (e.g., causal, part-whole, classifying, functional). The content was drawn from Marine field manuals, and interviews with snipers and coaches. Concepts were tagged with instructional content (e.g., definitions, explanations, elaborations, multimedia examples). Relations were tagged with an explanation of why the particular relation holds under particular conditions. Assessment is tied to instruction via influence (Bayesian) networks. Performance on assessment items determines what content is pulled from the ontology for delivery. For example, if a Marine scores poorly on all assessment items related to breathing control, then instructional content tied to the ontology concept “breathing control” (and any linked concepts) could be delivered. Conversely, if a Marine scores low on items that suggest poor knowledge of the shot group associated with poor breathing control, then only a shot group related to breathing might be delivered. Our test of this approach appears feasible and promising. The Bayesian network appeared to be successful in identifying knowledge gaps, and relevant and targeted content was served to Marines. Learning appeared to be occurring at a faster rate over time for Marines who received targeted instruction compared to Marines in a control group. Implications are discussed. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
“Focused Fidelity”: Applying Hybrid Rendering Technology to Dynamic Synthetic EnvironmentsThe HFE Group Ottawa, Ontario, Canada NGRAIN Corporation Vancouver, BC, Canada The challenge of accurately simulating dynamic environments has largely eluded many current technologies. While some success has been achieved in specific focus areas, such as dynamic terrain, current solutions are highly specialized and lack the robustness required to support integration with standards and specifications, such as SEDRIS, which strive to create a unified representation of the environment. NGRAIN™ Corporation and The HFE Group have collaborated on a unique volume-based technology to address the needs of dynamic synthetic environments. The initial focus was on visualization, but the potential to address a much broader scope of environment simulation issues was immediately recognized. Of specific interest was the ability of the volume technology to support a nearly limitless set of attributions on a per-element basis, allowing the environment to be represented as a composition of discrete volumetric elements each with unique properties and relationships – as it exists in real life. Rather than representing an environmental artefact, such as a surface composed of polygons, the environment is represented as a unified collection of volumetric layers. Further, an efficient application programming interface (API) is provided for querying and modifying individual volumetric elements. While this technology represents important advancements in the areas of data repurposing, data storage and visualization, it was also recognized that existing polygonal-based technology could deal with static elements of the environment very well. It therefore did not make sense to “re-create” existing capabilities, but rather to leverage the best of traditional rendering technologies and the volume based technology through a combination of capabilities. This led to the development and maturation of hybrid rendering technology, allowing for the visualization of both polygonal and volumetric representations of the environment in a common rendering space. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Interactive Display of Complex EnvironmentsDepartment of Computer Science University of North Carolina at Chapel Hill Intelligent systems and simulated environments require intuitive interfaces for man machine interaction. One of the most important components of such a system is to interactively display the data in response to the user. Interactivity is defined as at least 20 updates per second for visual display. In this paper, we give a brief overview of our work on interactive isplay of complex datasets. We use algorithms based on visibility culling that do not render the primitives that are not visible to the viewer from a viewpoint. We augment them with level-of-detail representations or multi-resolution modeling techniques, which use different approximations of an object in the scene based on its distance from the viewpoint. We also utilize out-of-core techniques that use a limited memory footprint and predictive schemes to load only a portion of the scene that is used by the underlying rendering algorithm. We also present techniques to integrate these approaches and analyze the trade-offs. We have also developed parallel techniques that utilize multiple CPUs and GPUs (graphics processing units). The resulting systems have been used for interactive walkthrough of complex datasets including a powerplant model composed of 13 million triangles, Double Eagle Tanker model composed of 82 million triangles and Boeing 777 model consisting of more than 470 million triangles. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
An Automated Testing Perspective of Graphical User InterfacesOak Ridge National Laboratory Oak Ridge, TN Florida Southern College Lakeland, FL Software test automation provides hope for testers who face the daunting task of testing large, complex software systems. However, this hope often fades like a sunset as testing progresses from unit testing to system testing. One reason for this is that the interface between the system and its operating environment or the user becomes increasingly more complex as testing progresses from unit testing to system testing. In particular, the graphical user interface (GUI) has proven to be one of the most difficult challenges for test automation. This paper describes the efforts and difficulties of creating and using an automated test harness for the GUI of the U.S. Army’s OneSAF Testbed Baseline (OTB) simulation system. There is a significant amount of effort required for a user to create scenarios with the OTB user interface, even for simple scenarios. From a testing perspective, a test harness is needed to reduce this effort, thereby allowing more testing to be done within a given amount of time. Unfortunately, this generally requires that the test harness be custom made for the system being tested. This paper describes the methodology of creating a custom test harness for the OTB system. The goal in developing this test harness was to automate completely the process of creating scenarios in OTB using the GUI. This allows the tester to focus more on the execution of the scenario and determining its correctness. The test harness that was developed is capable of reading test scenarios provided by the tester, and then automates creating the scenario via the OTB user interface. Only partial test automation was achieved due to irregularities and complexities in the OTB user interface. This paper describes these problems and provides suggestions for improved user interface design strategies that can enhance possibilities for test automation of simulation system GUIs. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Joint Experimentation on Scalable Parallel ProcessorsInformation Sciences Institute, University of Southern California Marina del Rey, California The JESPP project exemplifies the ready utility of High Performance computing for large-scale simulations. J9, the Joint Experimentation Program at the US Joint Forces Command, is tasked with ensuring that the United States’ armed forces benefit from improvements in doctrine, interoperability, and integration. In order to simulate the future battlespace, J9 must expand the capabilities of its JSAF code along several critical axes: continuous experimentation, number of entities, behaviors complexity, terrain databases, dynamic infrastructure representations, environmental models, and analytical capabilities. Increasing the size and complexity of JSAF exercises in turn requires increasing the computing resources available to JFCOM. Our strategy exploits the scalable parallel processors (SPPs) deployed by DoD’s High Performance Computing Modernization Program (HPCMP). Synthetic forces have long run in parallel on inter-networked computers. SPPs are a natural extension of this, providing a large number of processors, inter-connected with a high performance switch, and a collective job management framework. To effectively use an SPP, we developed software routers that replace multicast messaging with point-to-point transmission of interest-managed packets. This in turn required development of a new simulation preparation utility to define the communication topology and initialize the exercise. We also developed tools to monitor processor and network loading and loggers capable of absorbing all of the exercise data. We will report on the results of J9’s December 2002 Prototype Event which simulated more than one million clutter entities along with a few thousand operational entities using 50,000 interest states on a terrain database encompassing the entire Pacific Rim. The exercise was controlled and “fought” from a J9 test bay in Suffolk, VA and the clutter entities were executed on a remote SPP in Los Angeles, CA. We will also present results from the Prototype Event in March 2003, as well as our long-term plans. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Load Balancing for Distributed Battlefield Simulations: Initial ResultsOrlando FL 32826 Orlando, FL SAIC Soar Technology, Inc. Orlando, FL It is well known that the system performance of parallel discrete event simulations depends on the assignment of workload to processors. If one processor is heavily loaded while the others are lightly loaded or idle, the overall performance of the system may be improved by offloading some of the workload to a less loaded processor. Load migration is the means by which this workload is dynamically moved from one processor to another. Central to the successful implementation of load migration is the ability to effectively predict processor loading and to apply domain relevant heuristics to select the source, the entities, and their destination for migration. The modified discrete-event simulation (DES) scheduling paradigm used by battlefield simulations impedes the system’s ability to effectively predict processor loading. In this paradigm, the real-time or scaled real-time systems depend on the busy wait construct to cycle through a time-delay loop and fire an event at the correct time. Because these cycles are consumed during what would otherwise be idle time, the inefficiencies inherent in this type of polling are moot with respect to processing time at the application level. However, at the system level, where processor loading is normally ascertained, since all of the cycles are fully consumed by the application, there is no practical way of predicting the processor loading. In this paper we will present a modification to the DES scheduler used in real-time systems such that it allows for the identification of trends in processor loading. Briefly, this modification will monitor the queue access request so it is possible to generate and analyze the pertinent data. Next, the paper will discuss how items are selected for offloading once the processor has been determined to be in the “overloaded” state. This involves implementing a number of application and system configuration heuristics as they relate to the entity-based simulation domain. Finally, the paper will present our sample implementation and the results of our analyses. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Real Time with a Red Hat Analysis of Standard Linux for Real-Time SimulationThe Boeing Company St. Louis, Missouri There is no doubt that Linux is a full-featured operating system. Its Unix heritage and tuning for use in large server farms has given it traits that any designer of large-scale systems craves: robust, near-crash-proof operation, high performance, native software development environment familiar to any Unix user, standard POSIX APIs, and so on. There is also little doubt that standard Linux distributions, such as those from Red Hat and Debian, are unsuitable for many hard real-time, deeply embedded applications. Embedded and Real-Time Linux extensions fill that niche but at the expense of many great features of standard Linux such as user/kernel protection, memory protection, a complete set of system services, and full device driver availability. However, most man-in-the-loop simulation systems do not have the toughest requirements of hard real-time embedded systems. There is no need for 1000 Hz sample rates, 20 microsecond interrupt response, or deeply embedded operation. Can standard Linux possibly meet the hard real-time requirements of simulation without losing any of the many Linux advantages? The answer is a resounding “Yes”. This paper details the Research and Development effort that resulted in the use of an ordinary Red Hat Linux distribution as the operating system for real-time training simulators. Real-time determinism is the key. Tests were devised to subject various system configurations to a mix of computation and I/O that is typical in a flight simulator. Determinism test results are presented for these types of system variations: single vs. dual processor, diskless vs. disked system, single process vs. multi-process, CPU affinity, the 2.4 vs. the developmental 2.5 Linux kernel, and the effects of the Hyperthreading feature of Intel processors. Certain system combinations were found to be suitable for use in hard real-time simulation host computers. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
An Optimization Methodology to Investigate Operator Impact on Quality of ServiceThe Pennsylvania State University University Park, Pennsylvania The 21st century battle space consists of vastly complex and dynamic environments to which human decision makers must adapt in order to achieve their objective. Moreover, the recent trend of manpower reductions necessitates the implementation of highly automated systems. Therefore, a decision maker must not only function under conditions of target ambiguity, time pressure, and information overload, he/she must also be able to effectively operate somewhat autonomous pieces of machinery. This paper presents the work in progress of a human decision making model for dynamic, time stressed tasks. We build upon a theoretical model of behavioral decision making generally used in judgment analyses in static contexts. For this paper, we focus on the mathematical formulation of the problem and some solution techniques. Our model of the human decision maker seeks to account for noncompensatory (i.e., rule-based) behavior. In general, we present a model that uses inductive inference principles to generate rules in disjunctive normal form that are consistent with human decisions. More specifically, we formulate the problem as a multi-objective linear programming (MOLP) problem. We provide some sensitivity analyses of exhaustive solutions to simple decision problems and propose the use of heuristic algorithms for more complex problems. This paper is available on the 2003 I/ITSEC CD ROM. Order it from I/ITSEC'S Website
|
|
|
Beowulf Supercomputing for
|
|
|