DISTRIBUTED VIRTUAL TRAINING APPLICATIONS FOR EDUCATION OF MAINTENANCE AND SERVICE PERSONNEL

Axel Hintze, Marco Schumann, Stefan Stuering
Department Of Planning And Visualization Techniques
Fraunhofer Iff, Magdeburg, Germany

This paper describes the application of VR-based computer simulation in the field of practical training. It introduces a training application that is currently under development. The training application is designed to reduce the need for practical training with the actual object. The main objective of the project is to develop a general modeling methodology that can be utilized in a wide variety of scenarios, while minimizing the need for programming simulation source code. Added benefits, such as reduction of traveling costs and time needed for travel, can be achieved by the utilization of distributed simulation following the High Level Architecture standard.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

REENGINEERING THE INSTRUCTIONAL SYSTEMS DEVELOPMENT (ISD) PROCESS MODEL TO FACILITATE COST EFFICIENT PROCESSES
 
Dr. Mary F. Bratton-Jeffery and Mr. Kelsey Henderson
Naval Education and Training Professional Development and Technology Center (NETPDTC)

Subject matter experts (SMEs) arrive at NETPDTC to review and revise Training Manuals (TRAMANs) and Advancement Exams. Using the model of the traditional instructional systems development (ISD) process, a Navy Chief SME, working with a team comprised of an instructional designer, instructional developer, a graphic artist, a videographer, a programmer, and an editor, will tackle the ordeal of producing a paper-based product with a shelf life of 5-8 years. The current production process averages 2-3 years. To reduce this burdensome and time-intensive process and to produce courses that remain relevant in the ever-evolving technology of Navy warfighting systems, the Naval Advancement Center (NAC), a department of NETPDTC, re-engineered the design and development process. NAC looked to industry to provide best re-engineering practices and developed the Reusability Architecture. The Reusability Architecture incorporates conceptual and modularized reengineering of the design and development processes associated with training products. It is the warehousing of the lowest common knowledge structures (text or media) in a massive database, which may be manipulated by the training need or the end user. The formulated Reusability Architecture populates a database accurately and rapidly, and facilitates output in a number of formats. By reengineering the design and development process to maximize the utility of databasing knowledge structures, the SMEs can now move fluidly between the two major assignments of TRAMANs and Advancement Exams. The development process, for the non-authoring members of the team, is transparent. This paper presents training course development model concepts and the newly developed reengineered design model, using an example from business' best practices for re-engineering processes. The paper then provides a brief overview of the Reusability Architecture and how SMEs can use it to simultaneously develop and design new courses and Advancement Exams. The new process uses four fewer personnel per team by providing advanced electronic performance support tools that combine many of the previously distinct personnel functions. The Reusability Architecture also provides fully functioning rapid course prototypes in about two minutes, a significant reduction over the previous process.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

DEVELOPING TOTAL TRAINING SOLUTIONS FOR COMPLEX MILITARY AIRCRAFT SYSTEMS

Authors: Bill Powell, Jon Sackett 
GKN Westland Helicopters Limited Yeovil, United Kingdom

Military organisations are demanding an ever-increasing range of training services in support of complex and modern weapons systems. Industry has responded with a number of innovative and cost effective solutions to ensure both affordability and operational effectiveness. These innovative training solutions apply to a wide range of training systems, developed to meet specific training needs, which may be installed within dedicated training facilities. The systems range from desk- top Personal Computer Based Trainers, through to Full Flight Simulators for aircrew, and sophisticated mechanical and electrical systems trainers for maintenance crews. Their effective application not only depends upon a good system design, but also upon the quality of associated courseware, Training Management Information Systems (TMIS) and instruction. Recent experience has shown that the most cost effective solutions make maximum use of low cost devices and part task trainers to ensure that higher cost items, such as flight simulators, are used to their full effect. These solutions accommodate high student throughput and deliver a high standard of training. Training for complex aircraft systems is not only required to support equipment being introduced into service, but must also be delivered and maintained throughout the operational life of the equipment. The delivery and maintenance of training has traditionally been the responsibility of the military operator. However, this situation has changed significantly in recent years, with the Military making ever increasing use of Industry to deliver training. In the UK, military training has recently been contracted under the Private Finance Initiative (PFI), whereby Industry owns the training equipment and delivers training to the Armed Services for the life of the equipment. 

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

THE ARMOR CAPTAINS CAREER COURSE: THE ART AND SCIENCE OF MOUNTED WARFARE VIA THE INTERNET

David E. Robinson and Ronald D. Offutt
A B Technologies, Inc. Alexandria, Virginia

Training the art and science of mounted warfare to its soldiers presents significant challenges for today's Army in an environment of constrained resources, expanding missions, and unit reorganizations. The traditional "best practice" for this learning is a multi-month resident program of instruction, with an alternative correspondence program for those unable to attend resident courses. Understandably, correspondence programs cannot offer the same depth of learning in the "art" of mounted operations as resident courses because the interaction of small group instruction and peer collaboration cannot be duplicated. For various reasons, correspondence courses are often the only viable option for most Reserve Component soldiers, giving them little opportunity to fully develop the range of complex abilities demanded by mounted operations. To address this dilemma the U.S. Army Armor School at Fort Knox, Kentucky, has established a dynamic, innovative program to deliver both the art and science of mounted warfare to a diverse, sophisticated, and widely dispersed student population using learning technologies. The Armor School applied lessons learned from operational tests of various distributed learning methods to create a student-centric, open learning environment that provides a model for future military distributed learning courses addressing similar content. This paper describes the conception, development, implementation, and evaluation of the Armor Captains Career Course, used to qualify Reserve Component officers to command and serve in the mounted arms.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

DOWN THE DIGITAL DIRT ROADS: INCREASING DISTANCE LEARNING ACCESS WITH HYBRID AUDIOGRAPHICS 

Major Michael W. Freeman, EdD
  U.S. Army Forces Command, Fort McPherson, GA
Robert A. Wisher, PhD 
U.S. Army Research Institute, Alexandria, VA
Christina K. Curnow 
George Washington University, Washington, DC
Major Kenneth L. Morris
U.S. Army Reserve Command, Fort McPherson, GA

Distance learning has the potential to enhance individual competency and military unit readiness by delivering learning where needed and when needed. However, this requires the distance learning systems to be highly accessible. Many potential classrooms, learners and organizations, especially those in the Reserve Components of the Armed Forces, don't have direct access to the Information Superhighway. Instead, they live down the equivalent of digital dirt roads. The proliferation of low cost personal computers capable of rendering high quality graphics, adoption of international standards for multimedia conferencing, ubiquity of Internet access and universal telephone service have created the opportunity to deliver scaleable, low cost multimedia instruction down the digital dirt roads with the hybrid audiographics approach. This approach delivers high quality audio over telephone lines and data interaction over packet switched Internet connections, thus distributing the transmission load between two highly accessible but limited bandwidth media. This paper explores the need for hybrid audiographics and describes a study comparing the performance of groups of Reserve Component soldiers taking an introductory course on Information Operations via the following modes: (1) hybrid audiographics, (2) hybrid audiographics with video of instructor and (3) traditional face to face classroom. The study results suggest there is no difference in student learning performance between the hybrid audiographics mode of instruction and the traditional classroom for the Information Operations course. The results also suggest video of the instructor had no effect on learning performance as measured by self assessment or actual test grades. Just as important, the typical transmission costs associated with delivering hybrid audiographics were calculated as approximately 4% of the cost of two-way full motion video using Defense Information System Agency services.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

BEYOND DISTANCE LEARNING

Dr. Donald A. MacCuish, Professor of Distance Learning
Air Command and Staff College
Maxwell AFB, AL
Colonel Edd Chenoweth, Dean of Distance Learning
Air Command and Staff College
Maxwell AFB, AL

For over fifty years the Air Command and Staff College (ACSC) has provided Intermediate Level Professional Military Education (PME) via Distance Learning (DL) to US and Allied Officers and civilians. The ACSC DL program was initiated in 1948. At that time the only feasible way of delivering instruction to a mass audience was by correspondence. How times have changed! Well, for ACSC DL the times will finally change later this year when the last cycle of book-based correspondence students complete their studies. During these five decades technology has had a profound effect on how we have presented instruction to DL students. Today we provide ACSC PME by three modes to over 8,000 active distance learning students worldwide. These three modes are multimedia correspondence, multimedia seminar and cyberseminar. Our program is rigorous and recognized by the American Council of Education, which recommends the granting of up to 27 hours of graduate level credit across four cognate areas upon completion of any of the distance learning programs. In this presentation we will share what we have learned during our evaluation and application of the various technologies we have employed. Included in these technologies are: audio and video tape, live and delayed television, satellite broadcasting, one-way video and two-way audio, inter- and intranet, etc. Our experiences have shaped our thinking. As we have developed our vision of the future we have considered most to the technologies mentioned above. We want to share this vision with you and describe some of the possible mixes of technology we expect to use to make that vision a reality. One of the more profound lessons learned is just because it is cutting edge technology, it doesn't necessarily mean it facilitates learning. We still intend to use books and other printed matter because sometimes it is simply the best method of presentation. Media analysis still applies!

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

NEW INSIGHT FOR TRAINING DEVELOPMENT OF 
21 ST CENTURY ADVANCED WARFIGHTER TRAINING

Conrad G. Bills, Lockheed Martin
Milton E. Wood, Consultant

Distributed Mission Training (DMT) opens a new horizon for advanced warfighter training. At last, the dream of training multiship warfighter interaction in a synthetic combat environment is now achievable. It will also soon be possible to train disparate forces across multiple geographic locations in preparation for our Air Expeditionary Force. With this enhanced training capability, the question becomes, How can training development take advantage of this new capacity for advanced warfighter instruction? The answer must acknowledge the fact that training development for advanced warfighter training is much more than developing master task lists, writing objectives, and meeting qualification standards. While DMT may be the most advanced technology in the world, it requires a shift in training concept to fully utilize its potential. Since DMT learning occurs at high conceptual levels, including the interaction of tacit knowledge and metaskills, cognitive task analysis techniques must be used to identify the intellectual skills and cognitive strategies that complement the motor skills, verbal information and attitudes at the intuitive skill level of the expert combat pilot. A new holistic skill hierarchy must be developed to guide the building blocks of instructional design for DMT. This will make it possible to select instructional strategies that take full advantage of an ability to immerse a warfighter in a full-up, simulated combat environment. Similarly, it will also be possible to design scenarios that can train situational awareness in three-dimensional space, identification of friend or foe, relationships of multiple air and ground entities, and response to known and unknown threats. This will result in a level of readiness never before achieved through combat training. This paper presents a methodology for developing this kind of DMT advanced warfighter training for the 21 st century.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

RESEARCH AND DEVELOPMENT OF INTELLIGENT TUTORING STRATEGIES FOR U.S. NAVAL RECRUITS

Katharine Golas, Ph.D., Claire S. Bartoli, Sherri Miller
Southwest Research Institute
Imelda Idar, Ph.D.
U.S. Navy

Since 1990, the attrition rate of U.S. Navy recruits has increased significantly. While psychological disorders are the most significant cause of attrition for females, males separate primarily for disciplinary and legal offenses. Naval training experts have long realized that academic difficulties are often manifested in problem behaviors that lead to separation. The Navy also recognizes that their operational and training environments are male-dominated, and there is a growing concern that gender integration in training has not been appropriately addressed. To stem attrition, maximize the acculturation process, and align training to address Fleet requirements, the Navy is developing and testing new instructional strategies and pedagogical practices.
The objective of this research effort was to increase the academic success of female recruits in a technical aspect of recruit training, where historically female recruits have performed less satisfactorily than male recruits have. A multimedia Interactive Courseware (ICW) instructional intervention called the Advanced Organizer (AO) was developed and tested in the context of firefighter training, to provide a structure for acquiring and storing technical material. The AO utilizes an intelligent tutoring strategy, where behaviors which reflect a student's cognitive learning style are identified and accommodated.
Data was collected at the Recruit Training Center in Great Lakes, Illinois, with over 1,000 recruits participating in the study. Results indicate that the AO decreased stress and made a significantly positive impact on the academic success of both male and female recruits. 
This project was funded by the U.S. Army Medical Research and Materiel Command, Fort Detrick, Maryland.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

AVIATION MAINTENANCE TRAINING CONTINUUM SYSTEM (AMTCS) SOFTWARE MODULE (ASM): AN INNOVATION IN NAVAL AVIATION MAINTENANCE TRAINING

Virginia Mesenbrink, Rosanne Stanford
Naval Undersea Warfare Center
Keyport, Washington

The Aviation Maintenance Training Continuum System (AMTCS) was established to address fundamental issues with reductions in funding and manpower, and lack of formal standardized technical training for sailors and marines beyond the schoolhouse. The AMTCS is comprised of all training and associated infrastructure required to support Naval Aviation Maintenance training. The AMTCS Program provides training and training management tools in the form of interactive courseware (ICW), computer managed instruction (CMI), computer aided instruction (CAI), and the AMTCS Software Module (ASM) to satisfy just-in-time training requirements for the individual technician, and the organization.

At the heart of the AMTCS is the ASM. The primary objective of the ASM is to enhance the quality and efficiency of training at the Schoolhouse and in the Fleet by providing the capability to identify individual maintenance task requirements, perform real time assessment, identify training deficiencies and provide immediate and focused access to training tools. The ASM is a software application tool designed to track technical training exposure across an individuals military career, validate knowledge/skill for all Navy and Marine Corps aviation maintenance personnel, and provide a feedback system to the decision makers to better manage the training business process. This paper will present the functional capabilities of the ASM and the operational concept of the ASM in Aviation Maintenance schoolhouse and fleet environments.
When fully implemented the ASM will be deployed to all Naval, Marine, and Reserve Aviation Maintenance activities. At present, the ASM is deployed at Naval Aviation Maintenance Training Group Detachments. Next year, the ASM will be implemented in F-14, F/A-18 and E-2/C-2 Fleet Squadrons. This paper will include an analysis on the effectiveness of the ASM based on initial feedback from Schoolhouse and Fleet users.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

ACSC DISTANCE LEARNING - 50 YEARS OF LESSONS LEARNED

Dr. Donald A. MacCuish
Professor of Distance Learning
Air Command and Staff College
Maxwell AFB, AL 36112-6426
Colonel Edd Chenoweth
Dean of Distance Learning
Air Command and Staff College
Maxwell AFB, AL 36112-6426

For over fifty years the Air Command and Staff College (ACSC) has provided Intermediate Level Professional Military Education (PME) via Distance Learning (DL) to US and Allied Officers and civilians. The ACSC DL program was initiated in 1948. At that time the only feasible way of delivering instruction to a mass audience was by correspondence. How times have changed! Well, for ACSC DL the times will finally change later this year when the last cycle of book-based correspondence students complete their studies. During these five decades technology has had a profound effect on how we have presented instruction to DL students. Today we provide ACSC PME by three modes to over 8,000 active distance learning students worldwide. These three modes are multimedia correspondence, multimedia seminar and cyberseminar. Our program is rigorous and recognized by the American Council of Education, which recommends the granting of up to 27 hours of graduate level credit across four cognate areas upon completion of any of the distance learning programs. In this presentation we will share what we have learned during our evaluation and application of the various technologies we have employed. Included in these technologies are: audio and video tape, live and delayed television, satellite broadcasting, one-way video and two-way audio, inter- and intranet, etc. Our experiences have shaped our thinking. As we have developed our vision of the future we have considered most to the technologies mentioned above. We want to share this vision with you and describe some of the possible mixes of technology we expect to use to make that vision a reality. One of the more profound lessons learned is just because it is cutting edge technology, it doesn't necessarily mean it facilitates learning. We still intend to use books and other printed matter because sometimes it is simply the best method of presentation. Media analysis still applies!

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

INSTRUCTIONAL STRATEGIES FOR TRAINING DISMOUNTED INFANTRY IN VIRTUAL ENVIRONMENTS

Donald Ralph Lampton
U.S. Army Research Institute for the Behavioral and Social Sciences (ARI) Simulator Systems Research Unit (SSRU)
Orlando, FL.
James Parsons
University of Central Florida Institute for Simulation and Training
Daniel Paul McDonald, Mar Ester Rodriguez, and James Eastham Cotton 

Human Factors Psychology Program, University of Central Florida
This paper describes the results of the first experiment conducted with the Fully Immersive Team Training (FITT) research system. The objective of the experiment was to examine instructional strategies involving how and when to give guidance during team training with Virtual Environments (VEs). 118 college students participated in the experiment. Two-person teams engaged in search missions in VEs depicting building interiors. The teams were composed of either two participants or a participant and an experimenter's confederate. Before attempting missions all participants studied a printed-text mission training manual that described mission procedures and received training on how to move and interact in VEs. Teams engaged in 1 or 2 practice missions and a test mission. Teams were given guidance either before (demonstration), during (coaching), or after (replay) the first practice mission, or not given any guidance at all (Control group). Performance measures included: speed and accuracy of search, communications, and security procedures. Results indicated that the participants quickly learn to use the FITT interface to move in, and interact with, the VEs; and that performance of mission procedures improved with practice. Relative advantages and disadvantages of the instructional strategies, and lessons learned, are discussed.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

APPLICATION OF COMMERCIAL PERSONAL COMPUTER GAMES TO SUPPORT NAVAL TRAINING REQUIREMENTS: INITIAL GUIDELINES AND RECOMMENDATIONS 

David S. Coleman
Sonalysts, Inc.
Dr. Joan H. Johnston
Naval Air Warfare Center Training Systems Division

The Department of Defense has been implementing a number of commercial PC simulation games to support surface, subsurface, and battle group individual and team tactical training. Many of these low cost games offer realistic and accurate graphical and dynamic depiction's of U.S. and foreign surface ship, air, and submarine platforms. In addition, the sophisticated 3-D visualization, challenging tactical scenarios, and the high level of interactively presented in the current generation of PC games is appealing to today's computer-literate young officers and enlisted personnel, creating a potentially effective learning environment. Conversely, features added to some products to increase entertainment and gamesmanship for the broader commercial audience may misrepresent actual combat situations, conditions, or available information that could adversely impact potential training value. This paper examines the advantages and disadvantages of using commercial PC games for naval training applications and offers recommendations and guidelines for integration of these products into Navy training curricula.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

SIMULATION COUPLED WITH CBT CREATING A COMPREHENSIVE TRAINING TOOL THAT POTENTIALLY INCREASES TRANSFER

Kurt Sand and Jason Schoenfelder
DCS Corporation

Selective use of simulation in CBT increases training effectiveness. Simulations in CBT increase training effectiveness by presenting situations in the same manner as they are experienced in the real world. This increases learner transfer. However, simulations alone are not enough. While simulations allow the user to practice what-if scenarios, they lack content and instruction of proper methods and application. Without content and instruction for the user to access, misconceptions can result. Therefore, an effective CBT should have simulations paired with instruction; preferably, interactive lessons designed to help the learner with specific problems and topics.
An approach has been developed using three different techniques for implementing effective simulations for CBTs: (1) Visual Simulations, (2) Modeling Behavior Simulations, and (3) Re-engineered Application Simulations. The first category uses bitmaps of the graphical user interface to mimic the way the application looks. This is a rudimentary form of simulation. In using this method, a trainer shows a series of actions and computer reactions, thus allowing the learner to see and also manipulate the interface. A second category models the engineering of the original application on another platform. Often requiring reverse engineering to simulate the device behavior, it allows the users to manipulate the interface just as they would in a real situation. It operates and reacts just as the actual system. The third category uses the actual code used to create the original application in the simulation. The users interact with the system the same as they would in the reverse engineered model, but the cost of production of the simulation would be minimal and, in fact, allow the training simulation to be as up to date as the actual application.
In this paper, the benefits and limitations of each type of simulation will be analyzed ending in a proposal matching instructional goals with each form. After all, the goal of training is to attain a desired performance. The range of simulation technologies gives an instructional designer or trainer a choice. It is not always necessary to implement the latest and flashiest of software. By matching effective instructional values with the appropriate technology, training can be more efficient and effective.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

DESIGNING FOR CHANGE: A MODELING AND SIMULATION SYSTEM APPROACH

Michael Companion, Ph.D. and Charles Mortimer
Raytheon Systems Company

Training simulation systems for the 21st century are growing increasingly more complex. They are characterized by being multi-role, i.e., "what do I want to train today", rather than a point design aimed at solving a specific training need. In addition, these systems are no longer viewed as being a final design at Initial Operating Capability (IOC). Instead, they must continue to evolve and adapt to changing requirements over an extended period of time. This paradigm changes the approach to effective system design. This paper discusses an approach to developing a flight simulation system intended to meet a changing training environment.
Changes that drive the evolution of a simulation system design originate from changes in:
o Mission
o Requirements
o Technology

An approach to addressing changing missions is through the mapping of the mission to a Concept of Operations (CONOPS) that describes how all of the components work together to achieve the training objectives. This approach provides a broader perspective of mission needs that highlights the interaction and correlation between components. An approach to addressing changing requirements is through the mapping of system requirements to the system design. This approach supports a flexible modular design. In the past, networking of flight simulation devices was a simple linking of integrated devices. In the future, systems components which are normally integral to a training device, such as electronic combat environment (ECE), natural environment, instructor operator station (IOS), are broken into separate simulation system components that can be flexibly configured to "build" the simulation system to support any training mission need. The end result is a totally modular and distributed simulation architecture in which every component is equivalent to a system in itself. This architecture places greater emphasis on understanding and facilitating the integration, interaction and correlation of the simulation system components.
An approach to addressing changing technology is by mapping of technology to the system architecture and performance requirements. This approach provides the visualization necessary to develop effective long-term technology incorporation by identifying optimum targets of opportunity to maintain concurrency,

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

INFORMATION TECHNOLOGY ADVANCES WILL SUPPORT ADVANCED DISTRIBUTED LEARNING ANYTIME AND ANYWHERE

John J. Morris
Sonalysts, Inc.
Janet M. Weisenford
Naval Air Warfare Center Training Systems Division
Wiley N. Boland, Jr. Ed.D.
Asymetrix Learning Systems, Inc.

Information technology advances will support advanced distributed learning anytime and anywhere. However, similar advances in learning technologies are required to achieve cost-effective readiness and enhanced job performance. Adaptive learning that accommodates mastery differences in individual learners also offers benefits of high media reuse for continuum training - initial, refresher, remedial, and just-in-time instruction and performance aiding. High media reuse also can accrue from multiple courses supporting curricula related by personnel, equipment, or domain/core skills.
The Office of Naval Research (ONR) sponsored a Dual-Use Applications Program (DUAP) through the Naval Air Warfare Center Training Systems Division (NAWCTSD) to further "Artificially Intelligent Tutoring for Advanced Distributed Learning." A competitive procurement resulted in a technical investment agreement with Asymetrix Learning Systems, Inc. and Sonalysts, Inc. to enhance existing technologies and commercialize the resulting product(s).
The technical approach creates and delivers an individualized education plan at run-time. The first level of adaptivity determines "what to teach" by selecting and ordering the presentation of topics (that correspond to learning objectives). Topics are selected based on course definition data consisting of instructional groupings (course, module, lesson, etc.), instruction and testing strategy, and prerequisites, as well as current learner mastery. The second level of adaptivity determines "how to teach" by selecting specific learning objects (that support specific objectives/topics) based on student characteristics, mastery, and instructional history. Learning objects are data files consisting of one or more frames and associated media references that are attributed with objective/topic, detail level, score-based criterion, learner population.
After reviewing instructional issues, the paper also addresses the mechanisms, processes, and lessons learned from the DUAP technical investment agreement including Government goals and objectives. In addition to user-community involvement and program management from NAWCTSD, representatives from the Office of the Secretary of Defense, and Defense Acquisition University participated in working groups to evaluate progress and interim products, and to consider changes in instructional design processes to exploit adaptive learning capabilities.
The paper concludes with the implications for linking of adaptive learning capabilities to simulation-based tutors, embedded performance support, and learning management systems. Specifically, the learner model architecture is compatible with several simulation-based tutors, objective-based scenario generation, and training evaluation tools that have been developed under NAWCTSD training research programs.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

AUTOMATED PRODUCTION OF REUSABLE LEARNING OBJECTS

Susan Codone
Raytheon Systems Company

Creating reusable Interactive Multimedia Instruction (IMI) learning objects for diverse delivery environments is extremely important in today's training systems. Expectations for IMI production require specific standards of reuse, or the ability to combine existing learning objects from varying courses, to create new courseware without re-purposing. In addition, expectations are high for building learning objects once, but delivering them in Internet, Intranet and stand-alone CD-ROM environments. Reusability and variable delivery environments are two of the most important issues to consider when building learning objects. Both are separate, yet intertwined, and must be addressed in order to create true learning objects. Both issues cannot be resolved unless production methods allow standardization in the creation of learning objects, and a procedure to control the functioning of several learning objects linked together as a whole. New COTS technologies are available to produce IMI at a cost and quality only dreamed of previously. As a by-product of this technology, standardization of production techniques allowing reusability and variable delivery environments built into learning objects is now possible.
By using a creative combination of COTS products, anyone can produce cost-effective, reusable, and upgradable learning objects for variable delivery environments. This presentation will cover one such combination of COTS products to produce learning objects in a standardized manner. This combination, which has proven to be successful in several DoD projects, is the use of Allen Communication's Designer's Edge, Asymetrix's Synergy Link, and the Toolbook II Instructor authoring system. This combination allows for the development of reusable learning objects that integrate and function with multiple courseware delivery modes and existing training management system software and hardware. It also enables the creation of new courses using previously developed learning objects.
This presentation will show how instructional designers and courseware developers can design courseware using a highly modular approach, provided by the unique blend of these three COTS products. An entire development process for the creation of learning objects will be presented, to include the building of a visual storyboard using Designer's Edge, and the transfer of Designer's Edge storyboard material into Toolbook II Instructor using Synergy.  Special detail on the use of pre-made templates to achieve standardization in programming will also be presented, along with the use of a control module called a system book that allows for variable delivery environments and the recombination of learning objects.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

THE NAVY ADVANCEMENT CENTER'S REUSABILITY ARCHITECTURE

Mr. Kelsey Henderson and Ms. Judy Hale
Naval Education and Training Professional Development and Technology Center,

Military downsizing and budget reductions compel Department of Defense agencies to seek effective training interventions, which have great applicability at reduced costs. The Navy Advancement Center's Reusability Architecture is one such technology-based intervention. The architecture features a reengineering of the design and development processes associated with training products. Two very important principles, reusability and maintainability, become an integral part of the philosophy and processes used to develop these products. The use of databases and electronic performance support tools allow the warehousing of the lowest common knowledge structures (text or media) which may be manipulated by the training need or training requirement. This architecture is designed to provide increased efficiencies and could reduce much of the redundancy efforts in training materials production costs. Military downsizing and budget reductions compel Department of Defense agencies to seek effective training interventions, which have great applicability at reduced costs. The Navy Advancement Center's Reusability Architecture is one such technology-based intervention. The architecture features a reengineering of the design and development processes associated with training products. Two very important principles, reusability and maintainability, become an integral part of the philosophy and processes used to develop these products. The use of databases and electronic performance support tools allow the warehousing of the lowest common knowledge structures (text or media) which may be manipulated by the training need or training requirement. This architecture is designed to provide increased efficiencies and could reduce much of the redundancy efforts in training materials production costs.
The first product, currently under development, is the Navy Engineering Training Series (NETS). It is a multi-faceted, multi-media interactive training product. When completed, NETS will contain 18 books designed to present basic Navy occupational standards (OCCSTD) related engineering knowledge common to the mechanical engineering ratings. NETS will eliminate the redundancy of content presentation across ratings and will incorporate the information into a single product delivered primarily by CD-ROM with updates via the Internet. Results of the NETS prototype evaluation will be shared with the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) audience designed to present basic Navy occupational standards (OCCSTD) related engineering knowledge common to the mechanical engineering ratings. NETS will eliminate the redundancy of content presentation across ratings and will incorporate the information into a single product delivered primarily by CD-ROM with updates via the Internet. Results of the NETS prototype evaluation will be shared with the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) audience.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

HOW COTS TECHNICAL DEVELOPMENT AND PROGRAM MANAGEMENT CAN SURVIVE IN A MILITARY WORLD

David Beal, Project Manager, David Nemeth, Project Engineer
Advanced Simulation Technology, inc.,
Herndon, VA, USA

The Operations Room Team Trainer (ORTT), provides the Royal Canadian Navy with equipment and operations training in a medium-high fidelity, combat Canadian Patrol Frigate (CPF) environment. 
The ORTT required a communications system able to support one hundred a and seventy operator input/output voice channels with some positions receiving up to 15 simultaneous voice streams. The system required reproduction of several complex communications panels. It also required simulation of the versatile SHips INternal COMmunications system (SHINCOM) along with panel logic functions for several other communications devices.
Cost and schedule imperatives led to a program plan which maximized COTS/NDI techniques to minimize project-specific deliverables. The entire system was created from a small set of sub-system modules allowing custom designs to proceed in parallel with volume production. Design review documents and "custom" SDRLS were submitted on time with minimal support costs. In addition, an innovative ATM-like cell communication structure was developed to transmit both voice and data between the panels and processing nodes of the system.
This paper discusses the technical challenges and requirements of the ORTT Communications Simulation Subsystem, developed by Advanced Simulation Technology under contract to Lockheed-Martin, Canada, and the strategy and techniques used to develop this subsystem within schedule and
budget.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

BENEFITS OF MODERN BATTLEFIELD SIMULATIONS SYSTEMS

Tomaz Savsek
HQ Military Schools, Ministry of Defense
Republic of Slovenia

In the face of global cuts in defense budgets, simulation based training has been recognized as a solution that retains competency while reducing training costs. New tasks for armed forces, new scenarios, multinational structures, and restrictions in both financial and personnel resources including those resulting from environmental considerations require new concepts and solutions in the areas of military training, exercises and planning. The resulting loss of the 'reality' in conventional live exercises due to restrictions in the availability of supporting military personnel and other limitations caused by reductions in training grounds must be compensated for as much as possible through synthetic environment and modern simulation technology. Therefore, computer-based simulations, as training tools for effective planning, have become increasingly important.
Modern simulation systems should not have only one application, but should be used for both training and education and planning and analysis. This is important in relation to cost effectiveness and common databases for such areas as mapping, terrain, vehicle characteristics and tactics. Interoperability and reuse of battlefield simulations require the development of simulation systems which take into consideration the HLA (High Level Architecture). This paper will present some experiences that Slovenian Armed Forces have recognized at the field of battlefield simulations and introduction of HLA concepts in the training of commanders and commander candidates, staff exercises and planning.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

A TRAINING SOLUTION FOR ADAPTIVE BATTLEFIELD PERFORMANCE

Karol G. Ross, Ph.D. US Army Research Laboratory Fort Sill, Oklahoma
James W. Lussier, Ph.D.
US Army Research Institute
Fort Leavenworth, Kansas

To meet evolving operational challenges, we must leverage new strategies to train officers "how to think," as well as "what to think," preparing them to succeed in the face of unexpected events. This paper describes a model used for training adaptive battlefield thinking and the experimental program of instruction to implement and test that model with officers at the US Army Command and General Staff College (CGSC). In a recent project-Army Experiment 6 (AE6)--the challenge to provide a training strategy for adaptive thinking was met by a cooperative effort between the US Army Research Laboratory (ARL) and the US Army Research Institute (ARI). Each organization initially responded to the requirement with proposed approaches that at first glance seemed to be diametrically opposed. The operational question became whether we could integrate the approaches to define, train and measure adaptive performance. The two approaches proposed were a Constructivist Advanced Learning model and the Deliberate Practice model. The Constructivist model, an ecological approach to training, and Deliberate Practice, based on a Behaviorist orientation, were surprisingly complementary. A synthesized approach was developed and implemented as the "Adaptive Thinking Program of Instruction" (AT POI) to train brigade staff decision making during execution.  Eleven Majors from the Advanced Tactics elective, A308, at the CGSC Officer's Course participated in the experimental course in the spring of 1999. The students participated in exercises with a team of highly experienced military experts acting as mentors. The first part of the instruction concentrated on creating a multi-dimensional understanding of the battlefield and actually used a more tradition instructional approach. The second portion of the instruction was in the form of a capstone exercise. It centered on intense deliberate practice of cognitive skills in an environment designed in accordance with the Constructivist model and the Deliberate Practice model. The process to guide the practice was based on the Constructivist model, and it was also congruent with the Deliberate Practice model. Student insight into battlefield situations was supported in both parts of the instruction by use of a consistent set of themes that have been shown to represent expert perception of battlefield situations, and by simulations to enact and display developing situations under discussion. Performance was compared with that of similar students in a control group who did not receive the special training, but who completed the existing advanced tactics elective course during the first half of the AT POI and participated in a traditionally structured capstone exercise during the second half. Performance measurement, consisting of a structured method for eliciting situation assessments, was conducted pre- and post-training for the first half of the course and pre- and post-training for the second half of the course. The performance instrument was adapted from an ARI experimental assessment instrument. Subject matter expert and student assessments of the training were also gathered by means of surveys and interviews. Students who completed the AT POI were found to perform significantly better at adaptive tactical thinking. Better performance was found after the second half of the course only-the intense practice portion. The first half of the course, more traditional in nature, did not produce measurable gains in adaptive thinking. We conclude the paper with recommendations for maintenance of model integrity as this approach is disseminated and with reference to further research and development needed for assessment of adaptive thinking skills.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

INTERACTIVE  EDUCATION FOR THE 21ST CENTURY: THE ARMED FORCES STAFF COLLEGE AND THE EDUCATION OF FUTURE DECISION-MAKERS

Professor John R. Ballard, Ph.D. 
Armed Forces Staff College, Norfolk, Virginia

Raymond D. Kirkwood,
Veridian Inc. Hampton, Virginia
The Armed Forces Staff College (AFSC) has partnered with Veridian Inc. and other organizations to develop and implement an educational system to meet the needs of America's 21st century warfighters. This Joint Professional Military Education (JPME) program integrates current decision-making technologies, an innovative and thought provoking curriculum, web-based databases, and modern command and control systems into a learning system permitting students to experience and learn from a notional, interactive, 24-hour environment. This learning system will simulate the critical actions students may encounter while serving as members of joint military staffs worldwide, thus preparing them intellectually for future challenges. The goal of this learning system is to provide students a realistic and challenging, information-based setting, supporting both long-term retention of joint doctrine and operational art and focused instruction in special areas of emphasis. This paper outlines the results of the initial 2 tests of this new integrated learning system, conducted during the spring and summer trimesters at AFSC.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

MASTER TASK LIST AND CAREER TRAINING PLAN: A TRAINING MANAGEMENT TOOL FOR MANAGERS AT ALL LEVELS

Diane M. Kalivoda
Naval Education and Training
Professional Development and Technology Center
Alfred D. Reynolds
Naval Education and Training
Professional Development and Technology Center

To develop and implement a new strategy for Navy training requires a fundamental change in the process for determining when and how much the Navy educates and trains its workforce. The goal is to blend "just enough" and "just in time" training philosophies in initial training together with a conceptual shift toward apprentice-journeyman-master relationships. This new strategy for training requires a comprehensive cradle to grave assessment of all the knowledge, skills and tasks a sailor must master for any given Navy rating. It allows then to achieve immediate savings just by identifying and eliminating inefficiencies in training. Training is not, however, the sole responsibility of a single agent within the Navy, but a shared responsibility that requires close coordination. The Master Task List and Career Training Plan (MTL/CTP) process provides a vehicle for this coordination. It also provides a process for distinguishing among Apprentice, Journeyman, and Master level tasks, and for determining which tasks are most appropriate for formal, non-resident, fleet  or on-the-job training. Including a process for validating and maintaining a database of these tasks, it provides managers a way to ensure the accuracy of the data and assess the impact of decisions. 
The MTL/CTP process is a broad-based, straightforward detailed analysis. It is based on the premise that experts in the subject-at-hand are the best source of useful solutions to the program issues. Emphasis is on use of expert opinion and common sense rather than complex analysis tools. The process is designed to bring resident expertise to the problem in a forum that is well structured and organized, yet encourages inputs and discussions. It can be conducted either in a workshop environment with Fleet Subject Matter Experts (SMEs) as participants, or in an off-line mode where analysts interact individually with experts to assess the impact of proposed decisions. The synergistic effects of SMEs representing all platforms and major fleet components produce superior results. An added benefit is reduced lead-time for implementing resulting decisions. The MTL/CTP process is very straightforward.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

USING MISSION ESSENTIAL MOES/MOPS FOR EVALUATING EFFECTIVENESS OF DISTRIBUTED MISSION TRAINING

Frank C. Gentner, T. Cliff Tiller, and Paul H. Cunningham 
University of Dayton Research Institute (UDRI) Human Factors Group
Dayton, OH
Winston Bennett, Jr.
Air Force Research Laboratory (AFRL) 
Human Effectiveness DirectorateWarfighter Training Research Division,. Kent Mesa

Demonstrating the mission relevance of advanced training and rehearsal systems and their focus on training and evaluating warfighter needs is best achieved with objective metrics that can highlight mission performance changes. However, for a variety of reasons, it has historically been much easier to evaluate training in the traditional ways, that is, focusing on student evaluations and end-of-course tests as opposed to examining on-the-job behaviors and organizational or mission success. Student evaluations and learning tests are easy to implement but may or may not be explicitly tied to the overall training objectives. Further, these assessments do not provide any indication of the impact of training on job performance or mission effectiveness. Critical reviews found lack of an integrated system for measuring and assessing training performance, over-reliance on subjective measures of performance, and a shortage of valid, reliable, quantitative performance measures of training effectiveness. This paper highlights initial research and data collected to develop an Aircrew Measure of Effectiveness (MOE)/Performance (MOP) Hierarchical Taxonomy capable of assisting training and mission evaluators. The paper details our approach and provides data on sample mission task MOE/MOP decompositions to illustrate how a taxonomic approach can help diagnose actual aircrew mission performance of both individuals and teams. While this approach shows much promise, many technical obstacles need to be overcome before it can be completed and used routinely in an automated form. We highlight and discuss these technical challenges, propose solutions, and provide an agenda for needed research. Implications and potential future applications of the approach are discussed.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

STORYIST CONCEPT DEVELOPMENT TOOL FOR CBT- PROGRAMS

Wilfried Windmüller,
DaimlerChrysler Aerospace Dornier GmbH, Training Systems, 
Germany
Heinz-Bernd Lotz
German Federal Office for Military Technology and Procurement 
Koblenz, Germany

In this paper we describe a tool focusing on analysis, planning and design of CBT programs, called STORYIST. It is a tool for authoring conceptual ideas. The output is not a ready-to-run CBT program but a design from which users can get various multi-platform CBT programs. The process of designing a course usually involves different disciplines and people: problem-domain specialists, teachers, software engineers, psychologists, etc. STORYIST allows the co-operative discussion and design of projects. The overall structure of the CBT program can be improved version by version in a well documented way. Among the distinctive features of STORYIST are: learning goal- centered approach for the development of CBT programs; simultaneous definition of the problem domain, the content of the course and elements of course structure; evolutionary development of the detailed course program structure; reusability of design or design components; producing multi-platform-oriented CBT designs. Authoring with STORYIST actually means to achieve a step by step convergence of learning goals, learning content, and program course structure. The process of convergence passes through three conceptual phases: Conceptual Outline, Detailed Concept and Storyboard (Script). Finally, the author gets a detailed graphic description of the behaviour of the prospective CBT program and he can navigate by sequential or direct links through the whole storyboard. He is able to define a set of multimedia-oriented learning materials for embedding in the CBT programs and he can easily create layouts for learners graphical user interface. Graphical and textual representation and documentation of design components lead to increased performance of the whole course production and follow-up courses. Authors will be able to change the content and the structure easily, following their own ideas, needs, preferences, capabilities and experience.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

PREDICTING NAVAL READINESS BASED ON PERSONNEL VARIABLES

John M. Kenny, PhD
The Pennsylvania State University
State College, Pennsylvania

The ability to accurately measure readiness is a fundamental characteristic of an effective armed force. "Our number one commitment is to the readiness and well being of our men and women in uniform," said President Clinton (Clinton, 1994). However, regardless of the President's often stated commitment to readiness, measuring and maintaining readiness has proven to be an extremely difficult task. Even more alarming, the United States has a track record of being unprepared for its wars (Betts, 1997)
The Status of Resources and Training System (SORTS) is the Department of Defense's automated reporting system that identifies the current level of selected resources and training status of a unit (Joint Chiefs of Staff, 1993c). It assesses a unit's ability to conduct wartime missions. Each unit reports its overall readiness status as well as the status of four resource areas: personnel, equipment and supplies on hand, equipment condition, and training. Based on the evaluation of those four resource areas, the readiness of a unit is reported by assigning "C" levels as shown in Table 1.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

ELECTRONIC LEARNING DELIVERY SYSTEMS: A SELECTION MODEL

Dina E. Widlake
Electronic Data Systems, Inc.
Herndon, Virginia

Selecting the most effective and appropriate delivery system for the successful implementation of a training project is often an overwhelming task. Many of the existing delivery system selection models available to the training professional are outdated, and do not represent the currently available delivery system options. Electronic technology advancements, and their application to training are bringing complex new media choices to the training professional. Distance learning delivery systems represented in this model include satellite networks, teleconferencing, Internet/intranet networks, desktop multimedia, electronic performance support systems (EPSS), transportable audio/video (i.e., systems relying primarily on physically transportable audio- and videotaped materials), collaborative technologies, and the electronically enhanced classroom. 
This model narrows the choices to the system most likely to be educationally, economically, and technologically appropriate. After establishing a working taxonomy, the model recommends four stages of activity. The stages include: 1) The assessment of important overall factors; 2) Rating the importance of many learning system attributes to the training project; 3) Selecting the system that most ideally provides the attributes the model's user has rated important; and finally, 4) Validating the selection by reviewing the final choice against the first stage's overall factors.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

OBJECT-ORIENTED DESIGN APPROACH FOR SIMULATED TACTICAL SIGHTS

Royce McMinn
Lockheed Martin Information Systems, Orlando, Florida
Chris Waguespack
Lockheed Martin Information Systems, Orlando, Florida

This paper describes a design approach to simulate tactical sights within simulators and to facilitate a slaved-sight capability where exercise observers at a Stealth station can observe sight views seen by simulator crews. The design approach exploits object-oriented techniques using Ada95 inheritance and focuses on reusable components shared among system simulators and stealth stations. The approach has benefits including reduced network communication, cost-effective development, and others.
The United Kingdom (UK) Ministry of Defence (MoD) has procured a system, the UK Combined Arms Tactical Trainer (CATT), similar to the Close Combat Tactical Trainer (CCTT), and having a similar slaved-sight requirement. The UK CATT system is different in that there are more simulators and sights than in the CCTT system and CATT has a different image generator (IG). These differences forced a change in approach to simulated sights in CATT. The basic concept for the CATT slaved-sight approach is that the stealth station has the necessary information to construct a slaved-sight view including cursor position, laser range readouts, etc., given the state of the sight. System components with sights that can be slaved need only send state change information to the stealth station and the stealth station constructs the sight view using its local knowledge of the sight definition and the state information. This paper describes the CCTT and CATT designs. For the CATT design, the paper describes the common reusable components, their use across the system, the slaved-sight protocol, overall design features, and benefits.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

ELECTRONIC LEARNING ENVIRONMENTS: DESIGN

Peter C. Riley
Southern Illinois University
Carbondale, IL
Louis C. Gallo
Mediatech, Inc.
Ormond Beach, FL

Pervasive technologies for interactive multimedia uses in education and training have become commonplace. Capabilities of commercially available hardware and software tools are making development issues a problem of the past. However, the purchase and use of technology for technology’s sake has become a widespread problem in educational institutions and military and industry training programs. The chasing of technology in order to be considered one of the "latest and greatest" training and education programs has resulted in an overabundance of facilities and locations that have hardware and software that is improperly designed, configured, and installed. This paper presents an overview of historical significance of technology uses in education and training. It discusses new media technologies currently available to include distance learning and web-based training considerations for electronic multimedia classroom consideration, and examines the plethora of equipment and software available. It also examines the problems and solutions associated with designing an optimum presentation and instructional system. Case studies are used to examine return on investment and expandability and portability issues. A sample checklist is provided for consideration when converting to an electronic multimedia classroom environment.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

DESIGN AND USE OF EFFECTIVE, HIGH-IMPACT, COMPUTER-BASED ACADEMIC COURSEWARE FOR PILOT TRAINING

Terral (Terry) L. Chamberlain
Boeing Military Aircraft and Missiles
Seattle, Washington

It wasn't until simple-to-use multimedia software tools became available, and Personal Computers (PCs) gained a sufficient amount of computing power, that the viability of using PCs for producing and administering self-paced academic courseware sophisticated enough for training pilots became a reality. Such courseware developed for PCs prior to this time, was little more than the old 35mm slide presentation, only with audio narration instead of a stand-up instructor. Unfortunately, with the powerful multimedia PC tools available today, design and implementation of computer-based, self-administered pilot training courseware has not evolved much past a page-turning affair. As a result, the term CBT (Computer Based Training) has gained a bad reputation in the pilot training world rather than the high tech connotation its name implies.
Boeing entered the world of CBT pilot training courseware in 1989 for its commercial airplane pilot training requirements. They have since evolved the development process and courseware effectiveness, which culminated with pilot training CBT courseware for the 777. This courseware is so dynamic and interactive, that 777 pilot trainees are continually amazed at how the drudgery of the ground school portion (the academics) of learning a new airplane has been made into such an enjoyable, yet effective, learning experience.
The techniques learned in Boeing's Commercial Airplane Group are being incorporated into the F-22 academic courseware for both pilot training and the training of airplane maintainers. This paper shows the benefits of self-administered CBT academics for pilot training when the courseware is designed to the strengths of PCs and multimedia software. It also details the philosophy, rules, and techniques to use for making CBT an effective training tool, not only for academics, but also for testing. For testing, it describes how CBT is used to test a student's knowledge and understanding of a subject by using a real-life, interactive, operational format as opposed to a multiple choice, knowledge-only type format.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

ACHIEVING AN ACCEPTABLE RETURN ON INVESTMENT FROM TRAINING TECHNOLOGY

Joseph Sterling Mattoon
Science Applications International Corporation, San Diego, California

Many programs are now using information technology (IT) to deliver training, but in the absence of instructional design principles that match technological capabilities to learning needs, IT systems may fail to improve training effectiveness. One of the most versatile and commonly used IT-based training is interactive multimedia instruction (IMI). The purpose of this paper is to identify general principles for designing IMI that help ensure an acceptable return on investment in this technology. The principles are presented in the form of "General Facilitative Links" (GFLs) that tie technological capability to human learning processes. Examples of GFL application are focused on the type of complex skills that student pilots must develop during aviation training. A substantial proportion of pilot training and other types of complex skill training takes place outside of the operational environment, and this training can directly benefit from IMI when properly designed and implemented. In particular, IMI can be designed with simulation and adaptive components that prepare learners for more advanced training in dedicated simulators (e.g., flight simulators) or with operational equipment. Seven GFLs are identified to guide the design of IMI and take advantage of certain IT capabilities that are not available to conventional training. Each GFL is matched to several "Return on Investment (ROI) Factors" that describe value-added capabilities for training. It is proposed that GFLs will help courseware designers exploit IT capabilities that best

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

THE USE OF IETMS IN TRAINING FOR MAINTENANCE ENGINEERS

Yvonne F. Barnard & Johan B.J. Riemersma
TNO Human Factors Research Institute, Soesterberg, The Netherlands
Robert de Hoog
Dept. of Social Science Informatics, University of Amsterdam
Amsterdam, The Netherlands

In this paper the question is addressed how Interactive Electronic Technical Manuals (IETMs) can be used in the training of maintenance engineers. In order to investigate these questions, we adopted the following approach. We investigated the state-of-the-art concerning IETMs and training. An analysis was made of how an IETM could be used in the current courses for helicopter maintenance engineers. Several training scenarios were investigated: Training embedded in the IETM, IETMs as a source for learning materials, and as a tool in the learning process. Demonstrators were constructed showing how IETMs can be used in training. The electronic manual for a helicopter, available in SGML format, served as the source for the development of demonstrators. The demonstrators show that an instructor can develop tailored instructional materials by (re)-using existing electronic materials with standard presentation and word processing tools.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

INNOVATIONS IN THE APPLICATION OF MULTI-MEDIA WITHIN SOPHISTICATED TRAINING PACKAGES

Authors: Simon Dawes, Margaret James
GKN Westland Helicopters Limited, Yeovil, United Kingdom

Training providers are applying the power of the latest Personal Computers and sophisticated graphics packages to develop multi-media training systems which are able to meet an increasing number of training objectives. These multi-media packages can reduce the cost of complex training equipment packages by reducing the workload requirement, and hence the number, of higher cost training aids. These package are highly flexible and transportable and may also be available over the Internet.
A powerful application of multi-media in this context has been the development of 'Emulations'. In cases where operators and maintainers are required to develop predominately cognitive skills, the physical replication of systems may be replaced by virtual emulation. This technique uses high resolution computer graphics combined with a high-fidelity software model of the system's functionality to represent the equipment and provide effective practical cognitive training.
The UK WAH-64 Apache Training Service makes extensive use of the Worlds' first front end fully interactive 3-D Emulations in its Classroom Maintenance Training System (CMTS). The CMTS includes emulations of the entire WAH-64 aircraft systems to a level sufficient to deliver comprehensive training. The term 'comprehensive' has been used to cover the fact that the emulations will present to the student, through the use of graphics, the same visual stimuli as the real helicopter in terms of the appearance of front panels, LRU's, test equipment displays etc.. They will realistically reproduce the behaviour of the helicopter systems and the test equipment, responding to fault conditions and student input in the same manor as the real helicopter, from a maintainers point of view.
This paper describes equipment emulation and in particular the WAH-64 CMTS, describing how a structured analysis has been used to develop an innovative training equipment solution fully integrated with its classroom environment to deliver a total training solution.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

AN INNOVATIVE APPROACH TO FAMILIARIZATION TRAINING USING LOCALIZATION TECHNOLOGY INTEGRATED WITH INTERACTIVE MULTIMEDIA INSTRUCTION

CAPT. Dave Elliot, CDR Frank Ashton, LT Dave Wood, CDR Jack Burgess, Clementina M. Siders , CDR Chris Cross, CDR Bill Walker, CDR Jonathan Kelchner, CDR Paul Danner

Familiarization training is provided to crewmembers during pre-commissioning and also during crew phasing. During the familiarization training process, the personnel become familiar with the operation of systems, as well with physical location of all system components throughout the ship. These systems include damage control, engineering, weapons systems, communication and fueling systems. Historically, this training process has required knowledgeable, experienced personnel to mentor the crewman. This training is commonly accomplished through the integration of both schoolhouse and shipboard instruction. Unfortunately, with the impact associated with minimally manned ships as well as with the dwindling training dollars, training alternatives are necessary. The Navy and Coast Guard have recognized a similar training need and have combined resources to examine various training solutions.
This paper explores and evaluates a novel concept for providing self-paced, system-level familiarization training customized to the crewman's individual training needs. It also is flexible enough to accommodate individuals with different levels of expertise. This training is accomplished by determining or directing the location of students relative to a on-board system, then stimulating the correct Interactive Media Instruction (IMI) based on that location. Total concept success is dependent on the location technology, the IMI, the movement algorithm and the interaction between these three factors. This approach to familiarization training offers the promise of dramatically improving training effectiveness, while containing costs.
This research explores multiple methods and enabling technologies for determining relative location, and discusses how they may be used to extend current IMI techniques. Military and commercial applications of this procedure will also be discussed.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

PRODUCING COMPUTER LITERACY FOR THE DIGITIZED BATTLESPACE OF THE FUTURE

Karol G. Ross, Ph.D. Keith R. Yoder,
MAJ US Army Research Laboratory US Army Command 
& General Staff College Fort Sill, Oklahoma Center for Army Tactics 
Fort Leavenworth, Kansas

Computer literacy, especially the ability for leaders and staffs to use digital command and control systems, is clearly a decisive skill for success in the future battlespace. What is not clear is how best to achieve it. Relevant research findings support the use of a problem-based, student-centered approach in which the use of computer systems is situated in a context similar to the eventual expected performance setting. Whether this approach could be used in US Army institutional training successfully and within the given time constraint was the operational issue. This paper documents a successful implementation of the approach at the US Army Command and General Staff College (CGSC). The purpose of the new training methodology was to integrate tactics education with training on the Army Battle Command System (ABCS) in a Brigade operations course to produce both computer performance skills and tactical competency using the same amount of time previously devoted to tactics training alone. The methodology for this project consisted of a radical redesign of an existing tactics course, including the integration of a previously separate 40-hour computer operator course. The development also consisted of classroom configuration for maximum participation by the students and facilitation by the instructor. The methodology was based on a Constructivist approach to instruction that is described here to replace two existing courses at CGSC. The new program of instruction has shown that this combined training approach is viable at the introductory level, and we believe that it could be effective in both institutional training and unit training. The results of the course indicated that even students who came into the course with minimal understanding of tactics and no understanding of digitized tactical systems could produce high quality military operational products (such as orders and templates) using digital systems from the 147 hours of training. Our recommendations include
? dissemination of this approach to other officer training
? adapting the approach for operator training patterned on this successful experience with officers
? adapting the approach for "in stride" (just in time) training for units who will have to prepare for a wide variety of operational challenges with minimal train up time
? development of a 40-hour demonstration class for decision makers and potential instructors to allow them to experience the gains in computer literacy that can be obtained with this integrated method as part of the strategy for dissemination of this training approach

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

CREW RESOURCE MANAGEMENT AND MISSION PERFORMANCE DURING MH-53J COMBAT MISSION TRAINING

Jerry Sue Thompson
Raytheon Technical Services Company, Albuquerque, NM
Steven J. Tourville
Lockheed Martin Information Services, Albuquerque, NM
V. Alan Spiker
Anacapa Sciences, Inc., Santa Barbara, CA
Robert T. Nullmeyer
Air Force Research Laboratory
Warfighter Training Research Division, Mesa, AZ

Empirical research is confirming that crew resource management (CRM) should be an integral part of tactical aircrew training. This study evaluated the link between CRM and mission performance for 16 MH-53J rotorcraft aircrews during preparation and execution of a complex combat scenario in a networked training simulation. A strong correlation of .84 was obtained between CRM and mission performance. All CRM categories (e.g., mission evaluation, risk management, situation awareness) were highly correlated with mission performance; however, the categories were differentially predictive of mission performance in the various mission phases (communication was the best predictor in mission preparation, task management in infil/exfil, tactics employment in low-level). This study extends the findings of earlier research with MC-130P aircrews, where comparable CRM-mission performance correlations were obtained. Despite differences in the missions, flight profiles, and capabilities of the two aircraft, a number of key CRM behaviors emerged as common across the two weapon systems. As an example, the best crews in both aircraft immediately identified threats while enroute and classified them as "planned" or "unplanned." There were also some notable differences between the two weapon systems. One example is that the nature of communication among the crewmembers differed , with the MH-53J crews exhibiting more general, crew-wide interaction, while communication within the larger MC-130P crew centered around pairs of crewmembers communicating specific information at specific times. A major implication of this body of research is that a core behavior-based tactical CRM curriculum can be tailored to reflect the tactics and unique demands of each individual weapon system. The networked training mission afforded an excellent opportunity to observe important coordination and communication activities within the larger tactical team context. The results underscore the importance of broader team coordination and the need to formally train these team skills.

This paper is available on the 1999 I/ITSEC CD ROM. Order it from I/ITSEC'S Website

MEASURING C-5 CREW COORDINATION PROFICIENCY IN AN OPERATIONAL WING

V. Alan Spiker
Anacapa Sciences, Inc., Santa Barbara, CA
Steven J. Tourville
Lockheed Martin Information Services, Albuquerque, NM
MAJ John Bragger and TSG Duane Dowdy
Dover AFB, DE
Robert T. Nullmeyer
Warfighter Training Research Division, Air Force Research Laboratory
Mesa, AZ

Air Force Instruction (AFI) 11-290 requires that all operating units use proficiency data to measure the effectiveness of their cockpit/crew resource management (CRM) training programs. In response, the 512th Airlift Wing, Dover AFB and the Air Force Research Laboratory (AFRL) developed an approach whereby the Wing supplied qualified data collectors while AFRL developed process and performance instruments, "calibrated" Wing observers in the use of the instruments, and analyzed the data. Data were collected from 16 C-5 aircrews during a challenging, simulated nighttime airlift mission involving poor weather, post-takeoff landing gear malfunctions, and eventual engine failure. Building on AFRL's established methodology with the MC-130P, two expert observers independently rated CRM proficiency and mission performance using behaviorally anchored, C-5 specific scale elements. A significant correlation (r=.58) was obtained between rated overall CRM proficiency and mission performance, extending the validity of AFRL's approach to another weapon system and mission. The study also yielded a wealth of qualitative data capturing the specific CRM behaviors of successful aircrews (e.g., pilots and flight engineers di