Converting to SCORM: Lessons Learned
Training Content Harvesting:
Walking the Walk
Creating the Workplace Learning
Culture: Consequences of e-Learning implementation
Key Decision Factors for Selecting
e-Learning within UK Defence
M&S for Critical Thinking and
Performance: The Affective Domain
Instructional Design
Revelations: Intelligence, Learning, and
Leaving No One Behind
Design Structures for Intrinsic
Motivation
Rules of Engagement: Fostering Active Learning for Performance
Improvement
Rapid Authoring of Task Knowledge
for Training and Performance Support
Standardizing IRS EPSS by Applying
SCORM and S1000D Lessons Learned
EPSS: Designing for Context
in an Age of Standardization
Lessons Learned From Integrating
Commercial Gaming Technology into an ADL Environment
Digital Warrior: Blending Pedagogy
and Game Technology
Adaptive Thinking & Leadership
Simulation Game Training for Special Forces Officers
Creating JFACC Aces: Utilizing
Cognitive Requirements to Develop Effective Training Simulations
A TC3 Game-based Simulation for
Combat Medic Training
DESIGN OF FLEXIBLE ADVANCED
DISTRIBUTED TRAINING FOR MILITARY AIRCRAFT MAINTENANCE
Where Training Resources Should be
Allocated
Air Superiority Knowledge
Assessment System
Creating Expertise: A Framework to
Guide Simulation-Based Training
A Behaviorally Anchored Assessment
Tool to Measure Tactical Thinking Proficiency
Challenges of Developing Web-Based
Soft Skills Training
FCS Intelligent Structured
Training - Experimental Results and Future Applications
Diagnosing Shortfalls in
War-Gaming Effectiveness: A Model-Based
Approach
Enhancing Decision-Making by Explicitly
Training Battlefield Visualization Skills
A Distance Learning Framework for
an African Defence Force
Breaking Away from the Training
Paradigm through Performance Analysis: An Air Force Case Study
Linking Reusable Competency
Definitions to Learning Activities
Learning to Pull the Thread: Application of Guided Discovery Principles to
the Inquiry Process
Motivation and Online Learning
The Distributed Instructor: Innovator, Rule-Breaker, Negotiator
Automatic Causal Explanation
Analysis for Combined Arms Training AAR
Converting to SCORM: Lessons Learned
Susan Marshall, Damon Regan Joint ADL Co-Lab Patricia Mulligan AFAMS/ESS In this paper, the authors describe the process of converting the Air Force Modeling and Simulation Introductory Course (AFMSIC) to conform to the Sharable Content Object Reference Model (SCORM). SCORM is a collection and harmonization of specifications and standards that define the interrelationship of content objects, data models, and protocols such that objects are sharable across systems that conform to the same model. The Air Force Agency for Modeling and Simulation (AFAMS) collaborated with the Joint ADL Co-Lab to demonstrate the process of converting web-based courseware to conform to SCORM, while collecting lessons learned, resulting in the production of a SCORM 2004 conformant course. The objective of this paper is to explain this conversion process, while sharing lessons learned with the education and training communities implementing SCORM conformant courseware, to align with DoD distributed learning policy. The process involved (1) reviewing organizational concerns; (2) ensuring content remained instructionally sound; (3) defining Sharable Content Objects (SCOs); (4) debating the pros and cons of converting to SCORM 1.2 or 2004; (5) adjusting the design and development to include the Sequencing and Navigation specifications of SCORM 2004; (6) using practical tools; and (7) testing the courseware in the SCORM Conformance Test Suite. 2005 Paper No. 2071 |
|
|
Soar with Eagles: Linking Satellite Networks and Learning
Management Systems to Distribute and Manage ADL
Dr. Air Force Institute for Advanced Distributed Learning Maxwell AFB—Gunter Annex, SMSgt Jose Baquero Headquarters, Second Air Force Keesler AFB, Problems with Internet reliability and limitations to data networks have surfaced as major obstacles to content delivery. Internet connectivity is limited and local base firewalls strip files or block transmissions. Bandwidth is limited and usually dedicated to the operational mission. These technical problems are compounded by the expeditionary nature of today’s Air Force and the resulting challenge to provide learning opportunities to Airmen in deployed locations. The Air Force Institute for Advanced Distributed Learning (AFIADL) partnered with Second Air Force to address these problems. The result was a proof of concept for harnessing the capabilities of three Air Force enterprise assets—the Air Technology Network (a satellite interactive television network), the Air Force Integrated Learning Center (a SCORM-conformant learning management system (LMS), ADL object repository, and electronic customer service center), and the Air Force Expeditionary Classroom (deployed learning labs and resource centers). When fully integrated and implemented, the resulting capability—dubbed the Expeditionary and Global Learning Environment or EAGLE—will provide learning and information on demand. This paper describes the proposed EAGLE vision,
architecture, and fielding plan. Also
presented are the results and lessons learned of a proof-of-concept study to
determine the ability to successfully migrate content and student data
between the master LMS at AFIADL and an LMS at a deployment site via datacasting over the Air Technology Network. This proposal is still conceptual in nature
and has not been approved for final deployment. 2005 Paper No. 2096 |
|
|
Training Content Harvesting:
Walking the Walk
William A. Stembler PhD C2 Technologies William B. Stafford Lead Systems Engineer The theory of reusable training content is currently being brought to reality in the form of two major efforts. The effort of training development teams and technologies is to create content that is reusable and useful in sharable “Chunks”, and the effort of training delivery technologies and organization is to reuse the content as best it can. Currently, there is an imbalance between these two efforts. A much greater effort of organizational study, development time and technological advancement is currently being expended in the creation of re-usable training content, than there is in the actual re-use of that content. The reason for this may be that, though training content is delivered in ways that allow for the inner content, media and objects that make up that training to be re-used independently, most training delivery technologies and efforts only provide for the re-use of the training package or lesson as a whole, including whatever interface elements are attached. This paper explains how C2, along with its partners, in preparation for an Army Distributed Learning contract, have developed a Sharable Content Harvesting environment that allows for the transfer and re-use of not only training “lessons” or Objects (SCOs), but also the training content, media, template and sequencing information that make up the object, independently. This allows for the transfer of the objective based training textual content, media objects and all other information required to assemble a training lesson, from one Learning Content development and delivery system to another. Once transferred, the objects and content that comprise the lesson may be re-used independent of the original interface, in pieces or as a whole. The C2 SCO Harvesting Environment, as it is named, is a significant step towards mending the imbalance between re-usable content development, and the re-use of training content. 2005 Paper No. 2080 |
|
|
Creating the Workplace Learning
Culture: Consequences of e-Learning implementation
Maj Dave Crome BEd (Hons) MSc AGC(ETS) Worthy Down, Hampshire Maj Tony Charles MSc AGC(ETS) Directorate of Individual Training, British Army Upavon, Wiltshire There are few changes more radical within an organisation’s learning culture than the implementation of e-Learning, where traditionally school based study is distributed. Workplace learning requires not only acceptance of e-Learning as a means of training delivery, but a recognition that learning is part of the daily workflow; such a concept requires command support at all levels. Officers, particularly those at platoon or troop command with a responsibility for soldiers and non-commissioned officers, must support e-Learning in the workplace if this is to be achieved, and as such, can be seen as key stakeholders in the creation of an e-Learning or workplace learning culture. These same officers will be the higher commanders, senior staff officers and policy makers within UK Defence in future years. The Military Knowledge Programme, introduced in 2003-04 incorporates over 100 hours of e-Learning for officers within their first 5 years of service and is the largest distributed training programme implemented within UK Defence. Officers studying this programme will have their perceptions of workplace learning shaped by these courses, both in terms of the quality of the product, how they adapt to learning as a workplace activity and the learner support they receive. This paper will examine the experiences of learners on the Military Knowledge Programme. The authors argue that as key stakeholders in workplace learning, the degree of success of this programme amongst Junior Commanders will influence the transformation of the British Army into an organisation where workplace learning is accepted and, if successful, encouraged. 2005 Paper No. 2135 |
|
|
Key Decision Factors for
Selecting e-Learning within
|
|
|
M&S for Critical Thinking
and Performance: The Affective Domain
Mr. David Henderson Ms. Sharon Sloane WILL Interactive, Inc. There are many game-based simulations for combat operations that address shooter recognition and reaction, particularly in urban warfare settings. Now, higher end modeling and simulations are being used to improve warrior’s decision-making, judgment and adaptability under combat stresses. This session explores how the Army and Joint Chiefs of Staff are utilizing virtual experience simulation software to help improve leadership education. New advanced computer-based programs are employing digital game-based learning theory and interactive video with branching storylines to create learning experiences that improve individual’s performance under pressure. The military has adopted this venue to create emotionally immersive programs that address critical areas of human performance traditionally considered too difficult for the simulation world. This session discusses examples of educational simulations in which users become the lead character(s) in realistic combat and peacekeeping operations, make difficult real-time decisions, take real-life risks, and experience potentially life-threatening consequences, all in the safety of cyberspace. This new learning method increases user engagement to provide longer-term retention than has previously been achievable. The session will share how the US Army Field Artillery
School has incorporated a virtual experience simulation based on Lessons
Learned in Discover how the military is now educating even high-level officers using emotionally-engaging virtual experience immersive learning simulations. Experience how technology, art, science, psychology, learning theory and gaming theory are woven together to address areas such as critical-thinking, decision-making, judgment and adaptability. 2005 Paper No. 2379 |
|
|
Instructional Design
Revelations: Intelligence, Learning,
and Leaving No One Behind
Dr. Robert J. Blizek, DSL Scientific Applications International Corporation Today, training requires a fundamental shift that has not
been seen since the advent of the Second World War. The need to train broad cross-sections of
the general population for homeland defense is once again paramount. The objective has spurred a new look at
what we thought we knew about human intelligence, multi-media presentation,
learning, and how we measure the results.
The paper begins with Flynn’s intelligence paradox, the mid-70’s dip
in SAT scores, and data from the 2005 Paper No. 2043 |
|
|
Design Structures for Intrinsic
Motivation
Richard H. Swan Education as a whole does little to activate students’ intrinsic motivation. Thus, students see education as something they have to do rather than something they want to do. The discussion of intrinsic motivation often centers around games. While games are engaging and motivational, they are not always the most appropriate solution. In some ways, the issue of engagement and games obscures the more fundamental issue: perception of relevance. This paper explores the role of motivation in establishing relevance; the range of general types of learning experience, and; a design structures that can be applied to any experience type that is derived from industries that rely on designing experiences, such as computer games, motion pictures, literature, etc. These structures and some of their implications for instruction are briefly discussed. 2005 Paper No. 2181 |
|
|
Rules of Engagement: Fostering Active Learning for Performance
Improvement
Jo MacDonald Intelligent Decision Systems, Inc. The new millennium has seen significant changes in the operational environment and in the nature of the enemy. Training must likewise change to prepare our military to meet the challenges associated with this new paradigm. One of the current Department of Defense initiatives in education and training is the move to convert instructor-led courses to distributed learning. However, creating distributed learning involves more than the mere migration of content to online delivery. Effective distributed learning engages learners through study and investigation within authentic contexts; it encourages the growth of learner responsibility, initiative, decision making, and intentional learning. This is active learning, which has its roots in constructivist theory. This paper discusses what makes learning active and provides strategies for fostering active learning in an asynchronous distributed learning situation. These strategies, based on research and the author's own considerable experience in designing online instruction, provide for individual and collaborative engagement while the learner practices the skills needed to develop expertise in solving problems and in performing tasks in the learning domain. They also include strategies for learner support and feedback to ensure that the active learning process results in appropriate and accurate knowledge construction for performance in the operational environment—in air, on land, and at sea. Integrating active learning facilitated by collaborative interactions and expert guidance will result in curriculum that promotes individual development as well as teamwork, providing the unique cognitive skills necessary to meet the challenges of a rapidly changing and increasingly complex joint operational environment. It is a strategic approach to education and training that will enable the military to realize the goal of One Team, One Fight, One Training Future. 2005 Paper No. 2139 |
|
|
Rapid Authoring of Task
Knowledge for Training and Performance Support
John L. Mohammed Stottler Henke Associates, Inc. Barbara Sorensen Air Force Research Laboratory James Ong & Jian Li Stottler Henke Associates, Inc. Intelligent tutoring systems evaluate student performance and provide coaching and feedback during and/or after exercises. Intelligent job aids help users execute procedures by providing step-by-step instructions. These systems use computable task representations that specify appropriate actions at each step. These knowledge representations must be expressive enough to enable detailed, context-sensitive guidance and feedback, handle the wide range of situations and anomalies that might occur, and accurately assess the various possible actions the student might take. Yet, these representations must also enable easy and rapid knowledge entry and maintenance of large collections of procedures and training scenarios. This paper describes an intelligent job aid and integrated simulation-based tutoring system developed for the Air Force to help satellite operators carry out complex command plans. These systems use hierarchical, object-oriented task representations that enable rapid authoring by non-programmers while supporting sophisticated job aiding and student performance evaluation. For example, the tutoring scenario editor enables the instructor to create an initial solution template by demonstrating a correct sequence of actions. The instructor can generalize this template, so the tutoring system can recognize alternate orderings of actions, alternative sets of actions that accomplish the same task, and conditional actions that are appropriate in certain situations. The job aid helps users execute procedures by presenting
step-by-step instructions using HTML-formatted text and graphics, hyperlinks,
and embedded graphical user interface components. It enables gradual
automation by presenting instructions to the operator for some steps while
automating other steps by computing values, interpreting data, recommending
actions, and sending and receiving information with other systems and
databases. Looping and branching enable the software to execute some steps repeatedly
or only when certain conditions are true. A graphical overview of the steps’
hierarchical organization and flow-of-control helps operators and procedure
authors quickly review and understand the procedure and maintain context
during execution. 2005 Paper No. 2022 |
|
|
Standardizing IRS EPSS by
Applying SCORM and S1000D Lessons Learned
Steven A. Kerschenbaum VERTEX Solutions, Inc. Tim Tate Because of the nature of its business and the authoritative and complex nature of its Internal Revenue Manual (the IRM), many Internal Revenue Service (IRS) business units have found themselves developing ad-hoc Electronic Performance Support Systems (EPSS) to help staff navigate and apply its directives effectively. Unfortunately, this has led to difficult to maintain Web systems that are completely decoupled from the trusted source document (the IRM), and content that cannot be leveraged or even discovered by other business units with potentially similar needs for both Web-based Training (WBT) and EPSS. For these reasons, the IRS began to investigate applying the effective standardization principles and approaches promoted by the Sharable Content Object Reference Model (SCORM) to developing EPSS. In conjunction with the ADL Co-Lab and other industry participation, the IRS began the arduous task of applying a SCORM-like, object-oriented rigor to EPSS content development. Using these principles, the IRS defined several Performance Support Objects or “SPOs,” along with their corresponding metadata elements based on industry standards such as the Dublin Core Metadata Initiative. While the IRS successfully demonstrated facilitated EPSS development using these structures and a collaborative authoring tool (an LCMS), electronic access to the IRM itself has emerged as the principle challenge and ultimate goal for enterprise-wide use at IRS. Traditionally published in hardcopy, the IRM requires both structural and technological transformations for it to become a “trusted electronic source” for all IRS needs, including EPSS. This led the IRS to examine S1000D and how it might help facilitate the transformation of the IRM to a truly sharable electronic content source. This paper will review the process followed by the IRS to decompose its Performance Support Objects using a SCORM-like approach, its successes in standardizing facilitated EPSS development, its challenges in maintaining and leveraging its predominantly print-based trusted source (the IRM), and its goals for applying S1000D principles for transforming the IRM for us with EPSS, WBT, and traditional print. 2005 Paper No. 2237 |
|
|
EPSS: Designing for
Context in an Age of Standardization
Janet J. Cichelli SI International, Inc. Customizing electronic performance support environments to meet individual worker needs is one of the cornerstones of today’s human performance paradigm. By its very definition, a performance support solution should not be a templated, reusable construct. Instead it must provide the worker with a highly customized, personalized, and dynamic view of the work context and task progression. Performance support design is essentially context design. Context design explicitly draws upon cognitive science as a basis for understanding how people think about and approach work. Content management and standardization efforts have gained increasing interest over the past several years. Fueled by the federal Advanced Distributed Learning (ADL) efforts to manage and reuse learning content, efforts have now expanded to focus on standardization of technical support data for IETMs and EPSS. While content objects may be designed to meet all the ADL goals of reusability, durability, interoperability, and maintainability, they fail if they cannot be assembled and applied in a way that facilitates and improves human performance. A new model has emerged that combines highly customized context design (i.e., human performance design) with standardized content design (i.e., SCORM or S1000D). This model is fundamental to the design and development of next-generation EPSS solutions. It allows computers to do what they do well and humans to do what they do well. Computers can store, automate, organize, track, locate, and categorize large repositories of content. But designing for human performance still requires human creativity and knowledge. When this model has been employed, drawing upon standardized “task objects” and other support artifacts contained in a content repository, cost savings are realized through reusability and durability across related systems without compromising critical aspects of human usability and performance. This paper describes the new design model and how it is being defined, formalized, and moved into standard practice. It also specifically highlights three military EPSS projects and illustrates how the elements of the design model can be applied to help achieve the goal of organizing and presenting technical information within a work context that drives performance. 2005 Paper No. 2110 |
|
|
Lessons Learned From Integrating
Commercial Gaming Technology into an ADL Environment
Cindy Carlisle Joint ADL
Co-Laboratory Brent Smith Engineering &
Computer Simulations The CSTT is a government funded program that is currently being used to demonstrate the effective use of ADL technologies for the military. It provides training and simulated practice to help our National Guard Bureau’s Civil Support Teams – Weapons of Mass Destruction (CST-WMD) train in their mission to support local and state authorities at domestic Chemical, Biological, Radiological, Nuclear and Explosive (CBRNE) incident sites. The CSTT program uses distributed simulations and instructionally sound courseware to help keep their skills current for individuals and teams with regard to interagency operations and decision-making. This training improves the essential aspects of civil-military interoperability and mission planning. The Joint Advanced Distributed Learning (JADL) Co-Laboratory, in partnership with the US Army Research, Development and Engineering Command (RDECOM), the National Guard Bureau (NGB) and the US Army Program Executive Office for Simulation, Training and Instrumentation (PEOSTRI), has funded the development of the CSTT prototype and its associated technology to allow it to reach large numbers of people quickly and to provide simulated experiences that transfer efficiently into high levels of performance in an actual emergency. Skillful management of these technologies has proven to be an important element in the teaching and learning process. To ensure the quality of instruction, an in-depth instructional systems design was performed to develop the instructional specifications for the CSTT. This paper will look in detail at the changing requirements in the architecture and design of the CSTT from initial research, through the prototype development process and into the foreseeable challenges associated with production. It will discuss the role that Advanced Distributed Learning technologies and commercial gaming technologies have played in the development of this application and will identify the current state of these technologies in supporting this role, as well as lessons learned on working with and integrating these technologies. 2005 Paper No. 2077 |
|
|
Digital Warrior: Blending
Pedagogy and Game Technology
Aubrey White, John Stawasz, Sheilagh O’Hare University of Texas Institute for Advanced Technology Dr. Roy Jenevein, Dr. Don Fussell University of Texas
Computer Sciences Dept. & IC Institute Darrell Woelk Telcordia Technologies The military services face a revolutionary “Transformation
Process” in order to meet global threats that are dynamic, innovative and
adaptive. The Institute for Advanced Technology at The University of Texas at
2005 Paper No. 2206 |
|
|
Adaptive Thinking &
Leadership Simulation Game Training for Special Forces Officers
Elaine M. Raybourn Sandia National Laboratories MAJ Ed Deagle, Kip Mendini Jerry Heneghan Virtual Heroes Inc. Complex problem solving approaches and novel strategies employed by the military at the squad, team, and commander level are often best learned experientially. Since live action exercises can be costly, advances in simulation game training technology offer exciting ways to enhance current training. Computer games provide an environment for active, critical learning. Games open up possibilities for simultaneous learning on multiple levels; players may learn from contextual information embedded in the dynamics of the game, the organic process generated by the game, and through the risks, benefits, costs, outcomes, and rewards of alternative strategies that result from decision making. In the present paper we discuss a multiplayer computer game simulation created for the Adaptive Thinking & Leadership (ATL) Program to train Special Forces Team Leaders. The ATL training simulation consists of a scripted single-player and an immersive multiplayer environment for classroom use which leverages immersive computer game technology. We define adaptive thinking as consisting of competencies such as negotiation and consensus building skills, the ability to communicate effectively, analyze ambiguous situations, be self-aware, think innovatively, and critically use effective problem solving skills. Each of these competencies is an essential element of leader development training for the U.S. Army Special Forces. The ATL simulation is used to augment experiential learning in the curriculum for the U.S. Army JFK Special Warfare Center & School (SWCS) course in Adaptive Thinking & Leadership. The school is incorporating the ATL simulation game into two additional training pipelines (PSYOPS and Civil Affairs Qualification Courses) that are also concerned with developing cultural awareness, interpersonal communication adaptability, and rapport-building skills. In the present paper, we discuss the design, development, and deployment of the training simulation, and emphasize how the multiplayer simulation game is successfully used in the Special Forces Officer training program. 2005 Paper No. 2370 |
|
|
Creating JFACC Aces: Utilizing
Cognitive Requirements to Develop Effective Training Simulations
Holly C. Baxter, Ph.D. Klein Associates
Inc. James R. Lunsford MÄK Technologies New training simulations are being developed at an
increasingly rapid rate to improve skills ranging from piloting and driving
proficiency to learning tactical decision making and logistics. While these
training simulations often have high physical fidelity (the landscape,
buildings, vehicles, and people look real), they often lack cognitive
fidelity (the decisions people are making are not realistic). Developers
commonly have a clear picture of the “what” that needs to be accomplished
such as what tasks need to be trained. However, developers rarely address the
“why” (why is this decision challenging?) and “how” (what information do you
need to make this decision?) components in their simulations. This study used
Cognitive Task Analysis (CTA) and Decision Requirements Tables (DRTs) to fill this gap. Through CTA we identified several
cognitive training requirements of Joint Force Air Component Commanders (JFACCs). The results of CTA interviews with experienced JFACCs were analyzed and the critical decisions they
identified, such as managing and prioritizing their activities associated
with the air tasking order (ATO) battle rhythm or ATO cycle, were evaluated. JFACCs will often engage in numerous ATO planning
processes concurrently, and given the limited time they have, they must be
able to prioritize these critical tasks. DRTs were
then used by the developers to create a training simulation that
realistically replicated the stressful decision-making environment within a 2005 Paper No. 2019 |
|
|
A TC3 Game-based Simulation for
Combat Medic Training
Sandy Fowler US Army RDECOM-STTC Brent Smith Engineering &
Computer Simulations CSM David J. Litteral The modern battlefield dictates that US Army combat medics,
also known as 91Ws, be capable of operating
effectively and independently as members of highly dispersed and mobile combat
formations. These Soldiers must have
the necessary skills to accurately assess and initially treat a wide variety
of life-threatening illnesses and injuries. US Army medical personnel are
incorporating lessons learned from operations in Many of these lessons learned are being incorporated in the
16-week, 91W10 Healthcare Specialist course administered by the US Army Medical
Department (AMEDD) Center and School at Fort Sam Houston, TX. In addition to
using lessons learned to modify course content, the Army has also introduced
the use of simulation technologies into the program of instruction to improve
the overall effectiveness of the training.
The 91W10 Healthcare Specialist – Tactical Combat Casualty Care (TC3)
prototype trainer is an example of this type of technology advancement. Developed in partnership by the AMEDD
Center & School and the US Army Research, Development and Engineering
Command, Simulation and This paper will
look in detail at the changing requirements of the combat medic and will
document the results of a four month architecture design / requirements study
performed by the RDECOM-STTC. It will discuss the role that Advanced
Distributed Learning and commercial game technologies may play in the
development of the TC3 courseware application. It will also identify the current state of
these technologies in supporting additional opportunities for improving
combat medic training. 2005 Paper No. 2017 |
|
|
DESIGN OF FLEXIBLE ADVANCED
DISTRIBUTED TRAINING FOR MILITARY AIRCRAFT MAINTENANCE
Colonel A.C.J. (Ton) Bernards MA Royal Dr. W.F.S. (Willem) Hylkema Royal Major Inge Hopland
and Captain Erik Qvam Royal Norwegian Air Force Training center Kjevik The Royal Netherlands Air Force (RNLAF) and the Royal Norwegian Air Force (RNoAF) have designed a new training program for F-16 maintenance using a constructivistic approach. The training uses a learner centered approach. The training development is based on a new interpretation of ISD. The core of the training is an emulation of the F-16. This emulation creates a synthetic maintenance environment in which learning tasks are executed. CBT and classroom training are used to create a mental model necessary, for the students to execute learning tasks. The new, to be developed training environment, should be so flexible that it can be used within different kind of learning, for example group training, self paced learning, classroom delivery, mentoring, mixed mode learning, certification, performance support and distributed training. Using SCORM guidelines will make it possible to connect the learning objects (ELO’s and TLO’s) to a LMS. A script is developed, based on SCORM Simple Sequencing, which describes the complete training design. This script will be used to guide the training development and production process. During this process the production of CBT will be outsourced and managed by the RNLAF and RNoAF. The script will also be translated into an eletronic textbook for students and an electronic syllabus for instructors. The script will be the “bedrock” on which the training rests. In this paper we will discuss the design of the training program and the integration of the different approaches, as conceived by the RNLAF and RNoAF. The central questions addressed are: (1) What elements are necessary to design flexible blended learning which can be used in different learning environments? (2) Is is possible to translate mixed mode learning design in IMS Simple Sequencing? (3) How do we make sure that there is compliance between pedagogical, SCORM, “shipping and handling” and the training design? (4) How do we make sure that there is compliance between training design, development and execution? 2005 Paper No. 2018 |
|
|
Where Training Resources Should
be Allocated
J. (Jay) Bahlis, Ph.D., P.Eng. BNH Expert Software
Inc. Steven J. Tourville, Ph.D. Lockheed Martin STS Proper allocation of training budgets and resources requires fundamental and important business decisions. Unless the organization’s training function is a revenue-generating business, the demand for training most likely exceeds capacity and budgets are likely viewed as an expense – which is continually scrutinized. As a result, resources need to be focused on those initiatives that are important and which can maximize the training investment and demonstrate value. Otherwise, limited resources may be improperly invested in programs that end up having minimal impact on the organization’s missions/goals. How can training program funding decisions be optimized? Should money and resource allocations be equally spread across programs? Should the focus be on a few programs, with eLearning or blended solutions being favored to spread the investment? How can spending on the training function be prioritized in advance to deal with the constant challenge of new products, regulations and initiatives that require training? Managing training budgets and resources should be no different than managing any other investment, including information technology. The issue is how much time, money and resources are required to run programs versus the benefits generated in return. To identify benefits, links should be established between training activities and missions/goals – to define why training is needed in the first place. Training should address specific performance deficiencies needed to achieve unit/organizational goals and, as a result, the success and importance of training will not be measured by the skills and competencies that are being developed, but by the impact of the newly acquired skills on "performance". This paper presents a process for capturing the costs and benefits of instructor-led, eLearning, and blended solutions, assessing the impact of training initiatives on performance, missions and goals, identifying reasons behind success/failure, and maximizing training investment by redirecting resources to those activities that generate the greatest impact. 2005 Paper No. 2124 |
|
|
Evaluating Army Structured
Professional Forums: Innovations in Understanding and Assessing Effectiveness
Dr. Anna T. Cianciolo Global Information Systems Technology, Inc. Charles G. Heiden Human Resources Research Organization LTC Mike Prevou Dr. Joseph Psotka U.S. Army Research Institute As the rate of change in the operational environment outpaces the development of doctrine and schoolhouse instruction, Army leaders must direct their own development in order to adaptively and professionally meet the challenges brought on by Army transformation. Army Structured Professional Forums (SPFs), powered by advances in collaborative toolsets and multimedia presentation software, provide a means for self-development and professional growth. Systematic assessment of the functioning and impact of SPFs is critical for ensuring that SPF activity is aligned with organizational goals and producing return on investment. Assessing the impact of an SPF requires identification of those performances that activity in the forum can reasonably be expected to influence. In this paper, we present an approach for evaluating the functioning of SPFs and identify several candidate metrics for assessing the impact of SPFs. These serve as a foundation for organizing an effectiveness analysis of SPFs, and provide a general method for assessing SPF impact at the individual, unit, and organizational level. 2005 Paper No. 2137 |
|
|
Air Superiority Knowledge
Assessment System
Sara Elizabeth Gehr Boeing Christopher C. Metz Rickard Consulting Group, Inc. Brian T. Schreiber Lumir Research Institute Winston Bennett, Jr. Air Force Research Laboratory The US Air Force is developing a multifaceted approach to assessing knowledge and measuring mission effectiveness and performance in military training and rehearsal environments. This paper will present recent results from a field study using new knowledge assessment technology - the Air Superiority Knowledge Assessment System (ASKAS). ASKAS is a system that has been developed to enable measurement of a pilot’s mastery of Mission Essential Competencies (MECsSM). The pilots watch a series of mission oriented combat engagements which are actual air combat scenarios recorded from simulators, including audio communication between the pilots and the controller. After approximately two minutes of playback, the engagement stops, and the pilot is asked a series of questions. The questions are presented first, followed by the responses, allowing response times to be measured. The questions assess the pilot’s knowledge about the events of the preceding vignette, specifically addressing the MECsSM that are necessary for a successful mission. This process is repeated until the end of the engagement. The paper will present recent results, which include both percent correct and reaction time as a measure of the pilot’s situational knowledge and mastery of the MECsSM. Individual scores are compared to other pilot scores with different amounts of experience; more experienced pilots should have higher scores. We will discuss operational implementation challenges with ASKAS, the assessment protocol developed for field use, and possible future uses for this tool. For example, if a pilot takes the test before and after training events such as Distributed Mission Operations (DMO), training effectiveness can be measured, and will be indicated by higher scores after training. This knowledge assessment system could also be used to pinpoint a pilot’s knowledge gaps and outline areas that may need more attention in training. 2005 Paper No. 2228 |
|
|
Creating Expertise: A Framework
to Guide Simulation-Based Training
Karol G. Ross, Jennifer K. Phillips, Gary Klein Klein Associates
Inc. Joseph Cohn Naval Research
Laboratory The US Armed forces, and increasingly its coalition allies, continue to embrace simulation technologies as a solution to many training challenges. As these applications broaden in scope, it will no longer prove effective to use high-end systems to train all tasks. Rather, it will become critical to match training requirements to specific training technologies. Simulation tools such as Tactical Decision-Making Simulations (TDSs) seem to offer substantial benefits as modalities for training complex cognitive skills including planning and decision making, while keeping development and procurement costs significantly lower than those associated with high-end systems. However, the training community lacks a means of ensuring that utilization of these technologies for cognitive training will enhance battlefield performance. To aid TDS developers in designing tools that support the acquisition of expertise, a framework was generated to describe the evolution from beginner to expert in ill-structured environments. The framework reflects the notion that effective training enables learners to move toward a state of expertise, where good decisions are made quickly and automatically despite environmental ambiguity and chaos. Thus, the framework describes 1) five levels of proficiency and characteristics of learners at each stage; 2) the process by which individuals transition from one stage to the next; and 3) indicators of proficiency that enable diagnosis of a learner’s current stage. Two main assertions anchor the framework. First, in cognitively complex domains such as tactical thinking, learning consists of developing and refining mental models which enable individuals to accurately size up situations and apply action scripts to accomplish the objectives. Second, mental model development relies differentially on a range of training environments and techniques depending on the learner’s current proficiency level. Since the framework has implications for training in ill-structured environments regardless of specific domain content, it should prove adaptable across services and even across alliances. 2005 Paper No. 2221 |
|
|
A Behaviorally Anchored
Assessment Tool to Measure Tactical Thinking Proficiency
Jennifer K. Phillips, Jennifer Shafer, Karol G. Ross, Ph.D , Donald A. Cox. Klein Associates Inc. Scott Shadrick US Army Research Institute There is an ongoing need in the Army to enhance soldiers’ and leaders’ battlefield thinking skills, especially as Future Combat Systems technologies are introduced. To improve cognitive task performance, soldiers and leaders must engage in deliberate practice in context-rich environments, including training scenarios and simulations, field exercises, and actual combat situations. An important aspect of training tactical thinking skills that still needs development, however, is assessment. How do we know that cognitive skills are improving across experiences and over time? Diagnostic assessment will allow us to evaluate training effectiveness and tailor training as expertise develops. The goal of this project was to develop a technique to
assess tactical thinking skills. In previous projects, we created a
Behaviorally Anchored Rating Scale for tactical thinking (T-BARS) based on
eight themes. The initial T-BARS attempted to capture the progression of
tactical skill development, but was developed for
limited internal use. The goal of this project was to develop the T-BARS by
expanding and validating it to make it more robust and broadly useful. We
collected feedback from combat arms Captains, Majors, and Lieutenant Colonels
to vignettes provided by the U.S. Army Research Institute. The vignettes were
developed as part of their Think Like a Commander training program. We used
these responses to improve the T-BARS by adding behavioral anchors,
clarifying and proportioning the scale for each of the eight themes, and
iteratively testing the T-BARS to reach a goal of 80% inter-rater agreement.
The final product was a tool designed to accurately rate the development and
quality of tactical thinking. T-BARS in its validated form can be used both
as a diagnostic assessment and as a methodology for training effectiveness
evaluations. 2005 Paper No. 2444 |
|
|
Challenges of Developing
Web-Based Soft Skills Training
Valerie Hainley Imedia.it, Inc. The Human Intelligence Collector (HC) Web-based training (WBT) helps US Army soldiers acquire skills in questioning Enemy Prisoners of War (EPW). This interactive, media rich courseware makes learning complex soft skills more fun and less intimidating. After a pretest to determine the learner’s skill level, the learner is immersed in one of eight realistic scenarios. The course has three skill levels. Level 1 is practice and offers scaffolding for every error. Learners must pass a Level 2 scenario which only offers scaffolding after three errors. The Level 3 scenario is optional and offers no scaffolding. Learners practice their questioning skills in a simulated questioning booth where they must question a prisoner in order to collect information to satisfy the commander’s Priority Intelligence Requirements (PIR). The learner, who plays the role of the HC, asks questions of the EPW who, in turn, responds to the questions via programmed video responses. Just as in real life, the prisoner sometimes refuses to answer the questions. The learner must then change the questioning approach to gain the willing participation of the prisoner. The prisoner only resumes answering when the learner chooses the correct approach. If the learner asks questions in the wrong sequence or has difficulty in formulating a question, a virtual aide offers assistance. This aide provides scaffolding depending on the learner’s skill level. A detailed After Action Review is provided at the end of each scenario. One of the many challenges encountered in developing the courseware was how to manage the 5,000 plus video responses and equal number of questions required for the eight scenarios. Also, the course design needed to include techniques to help the learners with some of the more complex skills such as identifying control and repeat questions and PIR indicators. These and other challenges were overcome with creative design decisions and complex programming. 2005 Paper No. 2131 |
|
|
FCS Intelligent Structured
Training - Experimental Results and Future Applications
Randy Jensen, Coskun Tasoluk Stottler Henke Associates, Inc. LTC Mike Sanders,
Henry Marshall In support of the Army’s objective of developing embedded training for the Future Combat System (FCS), Army Research, Development & Engineering Command (RDECOM) sponsored the development of a technology demonstration and experiment with an integrated architecture linking intelligent evaluation mechanisms with their Command and Control Vehicle (C2V) testbed. Automated evaluation methods based on Intelligent Tutoring Systems (ITS) techniques are applied to monitor simulation and testbed events, and deliver feedback during scenario execution via messages published to the simulation environment. The logic for these intelligent evaluations is captured in hierarchical agent behaviors, and indexed to training principles identified by subject matter experts. This paper summarizes the findings from this effort, including technical methods as well as the results of experiments with human test subjects to measure the effectiveness of the system for training. Specifically, the improvement in performance over time among test subjects receiving automated feedback was contrasted with subjects receiving other forms of instruction such as an instructor-led after action review. These results provide a basis for the discussion of the way forward with FCS embedded training. Ultimately, with the application of automated training methods such as embedded ITS and structured training, a major potential benefit is the ability to train in settings where human instructors are not present or available. The work building the demonstration system sheds light on several areas of potential future work in support of developing full scale Intelligent Structured Training systems to realize these benefits. In addition to further validation of the approach, significant developmental areas include integration and compatibility with simulation common components like OneSAF that are likely to be used by the FCS program, scenario authoring tools to streamline the process for subject matter experts, and rapid behavior definition methods for simulated opposing forces to heighten realism and enhance training benefits. 2005 Paper No. 2189 |
|
|
Diagnosing Shortfalls in
War-Gaming Effectiveness: A
Model-Based Approach
Dr. Anna T. Cianciolo Global Information
Systems Technology, Inc. William R. Sanders U.S. Army Research
Institute Advances in instructional technology have significantly
increased the range of learning environments available to geographically
distributed learners. Distributed combat readiness training need not be
limited to individual skills now that students can gather in virtual spaces
to train on collective skills critical to mission success. However,
technological advancement has outpaced the capability to formally assess the
quality of collective skills and to diagnose team performance deficits--a
capability that is key to transforming learning
environments into true instructional systems.
In our research, we explored a range of methods for assessing the
collective skills of U.S. Army National Guard officers in training as they
war-gamed in a virtual tactical operations center. Our intent was to capture
aspects of war-gaming performance that would (a) indicate the level of
collective skill development; and (b) reveal the causes for team performance
shortfalls. We based the design of our assessments on a psychological model
of war-gaming that we developed using the results of a comprehensive
cognitive task analysis. Our model comprises the psychological constructs
associated with effective war-gaming processes and outcomes as well as with
key determinants of war-gaming effectiveness. In collaboration with
instructors, we administered our assessments to National Guard officers
enrolled in the distance-learning version of the Armor Captains’ Career
Course, taught through the U.S. Army Armor School at 2005 Paper No. 2215 |
|
|
Enhancing Decision-Making by Explicitly
Training Battlefield Visualization Skills
Jack D. Zaientz, Lisa Scott Holt, Scott D. Wood Soar Technology,
Inc. Christopher G.
Healey, Robert St. Amant Laura Strater, John Hyatt SA Technologies,
Inc. As the vision of network-centric warfare (NCW) becomes a reality in the Future Force, it is clear that trading steel for information represents a true revolution in military affairs. This places increasing demands on individual soldiers and commanders to incorporate this additional information into the military decision-making process (MDMP). To be truly useful for decision-making, new information must be integrated into the warfighter’s mental image of the current situation and future events, and be accurately related to the mission. This process, called battlefield visualization, is often more of an art than a science and is not accorded the same level of training rigor as other aspects of military decision-making. In this paper, we will present an approach for improving
the quality of military decision-making instruction by explicitly training
the basic visualization skills underlying battlefield visualization. This approach is based on a careful
analysis of the information and cognitive skill requirements for a
company-level cordon and search in a Military Operations in Urban Terrain
(MOUT) environment. In addition, the
approach includes deliberate pedagogical strategies to engage trainees in
active, guided practice and to prompt them to self-explain their
actions. The goal is to produce
visualization training (consisting of curricular materials and a digital
visualization training tool) that transfers to the battlefield regardless of
whether the soldier is using a new command and control system or grease
pencils on an acetate sheet. While
this work is ongoing, our current framework suggests that this approach will
demonstrate the effectiveness and utility of human-computer visualization for
military decision-making training and represent a significant step forward in
digital instruction technique and product. 2005 Paper No. 2241 |
|
|
A Distance Learning Framework
for an African Defence Force
Colonel (Dr) Annémarie van der Walt (neé Viljoen) Foundation Centre Manager, The diversity of the South African Society is reflected in
the South African National Defence Force (SANDF),
not only in terms of the rich variety of culture, but also in accessibility
to resources. To cater for all
potential learners it became clear through research that a continuum from
paper-based to electronically delivered learning systems has to be developed. One key enabler is the policy and
supporting guidelines and doctrine that were developed to enable the design
and development of contextualized distance learning systems in training
units. The result is a framework for
the unique challenges faced by a defence force in The research comprised questionnaire studies, structured interviews, benchmarking internally, nationally and internationally. Matched against the characteristics of the target group and the ETD system capabilities, the framework provides for the development of mixed mode delivery systems, development of distance learning skills, facilitator profiling and skilling. Interactive learning material development guidelines include the diversity from paper-based materials to electronically delivered material supported by a customized infrastructure. The significance of the framework is in its development for
the African context where a rich diversity is found in capabilities and
attributes. 2005 Paper No. 2024 |
|
|
Breaking Away from the Training
Paradigm through Performance Analysis: An Air Force Case Study
SMSgt Christopher Allen and TSgt David Chick 367th TRSS IMI Flight Our computer based training development organization, the 367th Training Support Squadron Interactive Multimedia (IMI) Flight, spent many years building products without completely understanding the performance problem. This approach caused a lot of frustration both on the production line and with our customers. The instructional designers and analysts were frustrated because they could never seem to get focused on truly what needed to be trained, and the customers were frustrated with the roughly 24-month development time needed to create these massive unfocused courses. In 2001 a group of individuals from within the organization started a change initiative that ended in the creation of what we call Performance Analysis. This process provides the production line with the information needed to concentrate efforts on bridging the validated knowledge and skill gap using a tactical approach. This change in turn has brought our development timeline down from roughly 24 months to 6 months. In addition to getting products quicker, customers also get the benefit of fully understanding the other components associated with their given organizational problem. This data allows for more effective decision making. This paper discusses the journey our organization has undertaken to get from a status quo training development house to what we feel to be a relevant, world-class and cost effective, Air Force organization. 2005 Paper No. 2082 |
|
|
Linking Reusable Competency
Definitions to Learning Activities
Geoffrey Frank, Donald Gemeinhardt RTI International Claude Ostyn Ostyn Consulting The lack of standardization of competency records hampers enterprise integration efforts, preventing organizations from linking their personnel databases to their training and assessment efforts. This lack of standardization leads to a greater risk level to their personnel and to the decisions these personnel must make at all levels. This also negates any immediate assessment of skilled personnel selection from high risk tasks to high risk decision making. This application effects many civilian organizations but is particularly applicable to many of the common Joint Environments DOD faces today. Automation of this linkage and creation of this process can reduce corporate costs and automatically provide the personnel databases with the assessment records and improve the documentation of personnel skills. Furthermore, an audit trail linking the assessment records of its employees to competencies desired by the enterprise is a valuable form of corporate knowledge and also valuable information for proving the fairness of promotions and salary increases. Training systems can increase their value to their sponsoring organization by supporting this linkage. Training systems can also employ this information to customize the learning for the individual based on gap analysis of the available evidence as compared with the desired evidence of the individual’s competency. The IEEE Learning Technology Standards Committee is developing a standard for reusable competency definitions to enable effective exchange of worker competency information. This standard is based on an existing IMS specification for which there is existing practice. The standard is designed to achieve reuse by combining reusable component competency definitions and referencing existing catalogs of job descriptions, skills, knowledge, assessments, etc. This paper presents a scenario that shows how to characterize competencies in terms of the U.S. Army’s existing catalog of Military Occupational Specialties (MOS), critical tasks, and performance measures. The paper also describes how the assessment capabilities of SCORM 2004 can define policy in terms of alternative means of demonstrating competency and how a SCORM-compliant simulation can supply records needed to support a claim of competency. The scenario describes how two soldiers and their supervisor interact with a system using the standard to select assessment methods and tailor training for the soldiers. 2005 Paper No. 2056 |
|
|
Learning to Pull the
Thread: Application of Guided
Discovery Principles to the Inquiry Process
Frank L. Greitzer, Douglas M. Rice, Sharon L. Eaton, and Michael C. Perkins Pacific Northwest National Laboratory Investigation of direct causes is a fundamental component of inquiry and analysis tasks that require skilled observations, logical thinking, and a persistent search for a complete understanding of the events. The need to cultivate such skills and persistence is a major challenge for diverse disciplines from accident investigation to forensics to intelligence analysis. In this context, persistence means to keep pulling the threads of evidence until a sufficient understanding of cause-effect relationships has emerged. The training challenge is rooted in fundamental questions about performance measurement and instruction: Can we effectively instill the required skills and persistence by merely informing learners through traditional classroom instruction? Or would such cognitive skills and persistence be better developed and refined through carefully crafted experience-based training? In instructional systems design terminology, this question may be phrased as a choice between receptive/directive instructional architectures that focus on ASK and TELL approaches versus approaches that emphasize SHOW and DO. The latter, more interactive instructional approaches emphasize active learning and performance assessment. We suggest that active, performance-based paradigms such as scenario-based and guided-discovery learning approaches may provide more effective solutions. By immersing the learner in appropriate interactive scenarios, we can ascertain through actual performance the extent to which the learner demonstrates the objective knowledge or skills. We have previously reported on an application of guided-discovery principles to develop web-based awareness training for security inquiry officials. The purpose of this paper is to report on subsequent research that employs guided-discovery scenarios to enhance the learner’s evidential reasoning process through practice in following threads to identify direct causes. Implications for inquiry/analysis and cognitive skills training are discussed. 2005 Paper No. 2073 |
|
|
Motivation and Online Learning
LtCdr Geir Isaksen, and Pål Andre Ramberg This paper is based on a post-graduate thesis which received an A at the Institute for Adult Education (VOX) spring 2004 and looks at which measures can be facilitated such that the teaching principle of motivation is optimized during development of online learning for the Norwegian Defence(NoD). These measures are collected in a checklist to ensure pedagogical quality and focus on student motivation. This list has become a standard piece of the information available to NoD courseware developers and is included in NoD`s methodology for developing elearning. By looking at the Didactical Relational Theory (DRT), well-known principles of learning and variables affecting success with online learning, the checklist helps to ensure that student motivation is optimised in all NoD online courses. The checklist contains the following important aspects tied
to achieving student motivation: Objectives and goals, involvement, feedback,
emotions, socialisation and self-efficacy. During
the last couple of years the checklist has been successfully used during the
development of a number of online courses. These courses include both hardskills courses such as application- and computer
systems training and soft-skills courses such as education in laws of armed
conflict. 2005 Paper No. 2118 |
|
|
The Distributed Instructor: Innovator, Rule-Breaker, Negotiator
Linda R. McCafferty UNITECH Marcia R. Gibson National Institute
for Aerospace An innovator is someone who introduces something new. Even though the concept of E-learning and distance education may have been around for a long time, the rules for the distributed learning (DL) instructor are constantly changing. Facilitation via a telephone or posting PowerPoint slides on a website seemed simple enough. However, now the DL instructor may incorporate desktop conferencing, voice over Internet protocol (VoIP), synchronous chat, asynchronous threaded discussion, or courseware downloaded on a personal digital assistant (PDA). The real innovation for the DL instructor is to use the technology to further the learning process and not impede it. The DL instructor must come up with new ways to create lasting learning, learn to bend or stretch the traditional academic paradigm to achieve lasting learning, and to negotiate with people, equipment, and concepts to ensure lasting learning. In the paper The Revolution of Blended Training: The Distributed Instructor (2004 Paper No.
1550), there was a promise for a follow-up paper once the new DL course
described was validated and implemented. Qualitative and quantitative
analyses were performed on the student, instructor, and observer data from
the validation. The purpose of this
paper is to share those lessons learned and provide prescriptive instructions
for individuals needing to transform themselves into distributed learning
instructors in all environments, to include business, academia, and the
military. This paper will provide
direction and guidance for creative ways for the DL instructor to manage
scheduling, feedback, motivation, development, problems, discipline, and the
lack of face-to-face interaction with the learner. 2005 Paper No. 2167 |
|
|
Automatic Causal Explanation
Analysis for Combined Arms Training
|
|
|