EDUCATION
‘Open-Source’ Exploitation: The Real Cost of Effective
Learner Management
Implementing New Educational Technology for 21st Century
DoD Leadership Development
Taskonomy vs. Taxonomy: Human-Centered Knowledge Management
Design
A Companion Agent for Automated Training Systems
Learning Styles Again: Where is Empirical Evidence?
Instructional Environments - Characterising Training
Requirements and Solutions to Maintain the Edge
Singapore
Armed Forces Core Values Storytelling Project
Using Digital Storytelling to Stimulate Discussion in
Army Professional Forums
Competency-Based Training and Grading for the RNLAF
Fighter Controllers
Overcoming Decision Making Bias: Training Implications for
Intelligence and Leadership
The Training and Retention of
Selected FBCB2 Operator Skills
The Impact of Combat Identification Training on Combat
Identification Skills
British Army E-Learning: Mathematics Skills Revision
– Game On!
Integrating Didactic and
Experiential Training: Round Pegs in Square Holes?
2007
Paper No. 7069
|
Educational
and Training Services (Army) Upavon,
United Kingdom |
Educational
and Training Services (Army) Aldershot,
United Kingdom |
In the networked learning environment,
where interoperability poses significant challenges, it is accepted that a ‘one
size fits all’ Learning Management System (LMS) is unlikely and the landscape
is a network of interoperable systems, delivering content and gathering data to
common standards. Despite the complexity
of the training environment, a single need prevails: Cost-effective learner management that
delivers courseware to a distributed population, which gathers data and
provides collaborative environments.
Open-source software is distributed under licence
and allows end-users to run the software, modify and re-distribute the source
code. Defence
is justifiably hesitant about adoption of open source and this paper
acknowledges that this concept remains in its infancy. However, the requirement to deliver content
and record achievement is now upon us:
In short, our appetite to develop and invest in courseware has raced ahead
of our ability to deliver effectively to the end user. The UK training community is justifiably
impatient to realise the economic benefits ADL, but
continues to invest in development, with little evidence to base continued
optimism. The authors conclude that
costs must be minimised to attract training sponsors
to ADL and as such the predominant expense should be the courseware, rather
than expensive learner management with spiralling licence fees. A
model for sustainable courseware delivery via open source is proposed. This is scalable; however the pilot is
restricted to delivering SCORM Courseware, with data managed by a single
training sponsor. Costs of maintaining an open-source system are identified and skills to develop
expertise for thorough-life exploitation.
Although this paper is aimed at Defence, the
authors provide an overview for the cost-conscious organisation
to develop courseware and manage the learner: fully harnessing the
opportunities of ‘Open Source’.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7408
|
Next
Generation Leaders, Inc. Culver
City, California |
Information
Sciences Institute, USC Marina
del Rey, California |
In educating emerging leaders to meet
the challenges of tomorrow’s non-traditional conflicts, the DoD must take advantage of new pedagogical and
technological methods and venues that provide the learner with perceived risk
reduction during education processes. The authors discuss how budding
commanders must deeply and effectively experience geopolitical, historical,
sociological and psychological material to improve their risk analyses and
management to produce decisiveness in complex, diverse situations. An
environment is described where they can engage regularly with lower thresholds
for taking risks: emotional, intellectual, social and (virtual) physical. This
will drive them to truly expand their “live” knowledge base. This paper sets
out how High Performance Computing (HPC) is the catalytic enabler for creating
complex innovative learning environments in which young leaders can most
thoroughly engage with the dynamic situations that they must master to be most
effective. The ability of HPC to manage manifold complex factors will allow the
DoD to create learning
modules that recognize and ameliorate the elements of risk-taking that the
learner undergoes when faced with new knowledge. Didactic instruction should be
almost entirely provided by this advance in computer-aided education, with the
live instructor focusing on the role of coach and guide for the preparation
before, and reflection after, the use of the virtual learning environment.
There is a valuable cadre of highly experienced leadership instructors who are
skilled in integrating didactic material with successful field experience. The DoD can develop the technology to
leverage the capabilities of those few instructors to make their talents
universally available by capturing their input for HPC-enabled virtual learning
environments. The goal is to radically alter
instructional interfaces to enhance vital pedagogical processes and thereby
improve educational outcomes in fundamental and transformational ways.
Documented support for the stated propositions and detailed analyses based on
experience are set forth.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7295
|
SI
International, Inc. Rockville,
Maryland |
US
Army Defense Ammunition Center McAlester,
Oklahoma |
The main goal of knowledge-based systems
is to provide workers with actionable information in support of real-world
activities. This includes connecting people with answers, experts with less
experienced workers, and delivering knowledge to help solve problems, make
decisions, or complete critical work tasks. Many knowledge-based systems and
tools, including knowledge portals and performance support systems, have been
developed but are simply underutilized because users are unable to locate the
information they need. This is largely because these systems are organized more
like an encyclopedia structure—alphabetically or thematically—and less like a human activity structure. What is needed is an organizational and
design method to effectively capture, store, and deliver actionable
information. Taxonomic structures are appropriate when there is no context. In
a hardware store, for example, hammers are in the hammer section, and nails are
in the nail section. This organization is based upon a
taxonomy. It works well for stores, libraries, dictionaries, and most
Web sites. But once a goal has been established and an activity has begun, then
a task-based taxonomy, or a “taskonomy”, is more
effective. This is why, in the real world, carpenters keep their nails with
their hammers. This paper discusses the
concepts and framework of a taskonomy and how it is
being applied at the Army’s Defense Ammunition Center to enable mission
performance and information superiority. The paper describes how a taskonomy categorizes tasks into a common format and
language. It discusses how this approach serves as the foundation for learning,
exercise, and workforce performance objectives to ensure that targeted, timely
and relevant training and knowledge resources are in place. Finally, the paper
addresses how a taskonomy serves as an important
framework for evaluation and assessment of performance.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7121
Eric F.T. Buiël PhD and Jan Lubbers MSc
TNO Defence, Security and Safety
The Netherlands
TNO Defence,
Security & Safety has a long history of applied research in the area of
automated simulator-based training by means of Computer-Assisted Instruction
(CAI). Traditionally, a CAI system does not enable a true dialogue between the
learner and the virtual instructor. Most frequently, the system acts like a
human expert, and authoritatively provides feedback and ways to improve the
task performance. In this paper, we describe an educational agent that enables
a dialogue between the learner and the agent. The agent is called the companion
agent. It acts like a virtual co-learner, for example by deliberating about new
operational measures after a situation-change. The agent operates on the same
authority level as the learner, and is therefore less threatening than a
traditional virtual instructor. We believe companion agents are typically
useful in modern, constructive learning situations where learners can take
control of their own learning process. Potential applications of companion
agents lie within the civil area (for example a civil tunnel operator during
tunnel surveillance training) and the military area (for example embedded
training in tactical surveillance).
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7426
Ellen S. Menaker, PhD, CPT Susan L. Coleman, PhD, CPT
Intelligent
Decision Systems, Inc.
Centreville, VA
The purpose of this review was (a) to
distinguish what has been empirically proven about learning/cognitive styles
from what has popular appeal alone, and (b) to discuss implications for the
design of effective and efficient learning experiences for all potential
learners. Lack of Empirical Evidence No
single theory unites the literature on learning styles (BECTA, 2005; Coffield, Mosley, Hall,& Ecclestone, 2004a; McLoughlin,
1999; Stahl, 1999). While it is likely that learners differ in many ways, there
is question as to whether these style differences are stable across all
situations or whether they vary according to task and environment; and how or
if addressing these styles has an impact on learning. The issue of matching a learner to a
particular instructional strategy has been fraught with a lack of research
demonstrating a relationship between instruction design of learning materials
or teaching styles and learning outcomes (BECTA, 2005; Evans &
Sadler-Smith, 2006; Hattie, 1999; McLoughlin, 1999;
Spoon & Schell, 1998). These issues have further been clouded by
commercialization efforts that focus on intuitive appeal of models and
instruments over psychometric rigor (Coffield et al.,
2004b). Conclusions The
lack of empirical evidence linking learning/cognitive styles to learning
outcomes suggests a rethinking of this topic. The impact of prior knowledge of
the learner and nature of the subject matter are stronger partners in the
decisions relating to what and how to provide instruction. Opponents and
proponents of learning/cognitive styles research have suggested that efforts
should focus on examining (a) constructs rather than models and (b) the impact
of associated strategies on learning outcomes. Research on prior knowledge,
cognitive load, motivation, and metacognition offers
promising insights into how to prepare learners to achieve desired learning and
performance goals and to become lifelong learners.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7497
Jonathan Pike, Dr John Huddlestone
Cranfield University
Bedford, England
With the ever increasing complexity of warfighting systems in the Network Centric Warfare era, and
the changing nature of the threats we face exemplified by the emergence of
asymmetric warfare, maintaining the edge and achieving force transformation is
ever more challenging. One of the many
aspects of this challenge is ensuring that we identify the most appropriate
training solutions for our warfighters. Current models provide guidance at the lower
levels of individual training episodes but are found wanting in critical areas
such as collective training and when trying to inform choices and make business
cases when developing new capabilities.
To address this problem we are developing a comprehensive and rigorous
model of instructional environments, instructional methods and the nature of
training tasks themselves in order to elicit a rigorous yet accessible method
for identifying training solutions which meet the key characteristics of the
demanding training problems that we face, whatever their scale. This paper
reports on the outcome of the first stage of this work, which is the
development of a comprehensive and robust model of the instructional
environment. The model embraces actors,
communication modes and channels, methods of encoding of stimuli and responses and
the characterisation of key resources used in the
instructional process. It forms the foundation for the characterisation
of instructional methods and media, essential for the determination of key
correspondences to the requirements of training problems which have to be
supported if force transformation is to be achieved.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7061
Quek Pek Noi, LTC Karuna
Ramanathan
SAFTI Military
Institute
Singapore Armed
Forces
Mountain Quest
Institute
Frost, West
Virginia
This paper introduces Singapore Armed
Forces (SAF) Core Values Storytelling Project, which aims to develop a
systematic storytelling approach in order to operationalize
Singapore Armed Forces Core Values (SCV) across the three Services of SAF. SAF is a largely national service military
force. Therefore, it is critical for SCV
to be rationalised and embraced by SAF leaders as
they entered SAF with personal values developed during their civilian lives and
nurtured over years of personal experience.
Under the project, key themes on SCV are identified and extracted based
on interpretations from senior SAF leaders and personal stories of real-life
operational experience from SAF field personnel using a generic SCV
storytelling template jointly developed by SAF and Mountain Quest
Institute. A storytelling methodology is
conceptualized and developed to provide processes, tools and methods to
facilitate thematic knowledge mobilization and knowledge transfer of SCV
embedded in the stories. Plans are in
the pipeline to develop generic lesson plans for SAF instructors and trainers
in SAF Schools and SAF Units to deliver SCV in training and education,
and to translate the key themes into a storyboard for an electronic application
to automatically elicit and categorize more personal stories on SCV in the
long-term. As personal stories on SCV
are continuously created and shared throughout their period of service, SAF
leaders as citizens of Singapore are understanding and
embracing core values that will help build and sustain the nation-state in the
future.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7280
Intelligent
Automation, Inc.
Rockville,
Maryland
US Army Research
Institute
Fort
Leavenworth, Kansas
Dr. Thomas D. Wason, Preetam
Maloor
Intelligent
Automation, Inc.
Rockville,
Maryland
Teaching a complex skill set, such as
leadership, requires interaction with realistic environments. LEED (Leadership
Education through Evolutionary Design) is a training simulation in which
platoon leaders learn and practice leadership skills through interactive dramas
related to crowd management. LEED uses
Interactive Pedagogical Drama and autonomous agents to create interactive
instruction. The learner interacts with
simulated characters in a story-based setting where the plot changes as the
learner’s actions affect the story. LEED builds on previous research using an
intelligent agent-based simulation infrastructure to model role-playing
characters simulating crowd behavior and military responses during MOUT (Military
Operations on Urban Terrain) operations. This paper will describe the use of
agent-based crowd simulation architecture as an instructional medium, as well
as provide a detailed description of the instructional design.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7043
Online communities of practice cannot
achieve organizational impact without member activity – discussion that leads
to knowledge sharing, social network development, and strengthening of the
professional community (Cianciolo, Heiden, Prevou, & Psotka, 2005; Schwietzer, 2003).
Several guidelines exist for facilitating ongoing, “organic” discussion, which
stems from requests for information made by forum members. However, there do
not exist formal guidelines for designing and using
“built” opportunities to participate in conversation (Dixon, Allen, Burgess, Kilner, & Schwietzer, 2005).
Built opportunities for discussion, such as book reviews, storytelling, and
polling, may foster the impact of online discussion forums through tacit
knowledge sharing and shared learning experiences (Dixon et al., 2005). The
present research explored the design and development of digital stories as
built opportunities for discussion in Army professional forums. Digital stories
are the combination of narrative, visuals, and audio in a short vignette that
communicates a particular message, such as one’s values, knowledge, or visions
for the future (Denning, 2005; Freidus & Hlubinka, 2002). Lessons learned about the characteristics
of effective digital stories, how to support the storytelling process, and the
type of discussion generated from digital stories will be presented.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7340
Jelke van der Pal, Berber Nicolai
National
Aerospace Laboratory NLR
Amsterdam, The Netherlands
Air Operations
Control Station, RNLAF
Nieuw Milligen, The Netherlands
The Fighter Controller training staff of
the Royal Netherlands Air Force (RNLAF) regularly seeks to improve training to
enhance the transfer of training to the operational environment and to reduce
attrition of students. Recently the Fighter Controller training has been
redefined using a competency perspective. The competency-based approach, as
tailored for the RNLAF, ensures that the competencies identified are consistent
to the Four Component Instructional Design (4C/ID) model. This approach
provides a hierarchic model of competencies and skills, enabling a close
relation to training design decisions, including performance assessment and
grading. The paper will describe and discuss the competency analysis, the
method, the results and its practical applicability.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7063
Lieutenant
Colonel Karuna Ramanathan, Colonel Sukhmohinder
Singh
SAF Center of
Leadership Development, SAFTI Military Institute
Singapore Armed
Forces
Instructors in the Singapore Armed
Forces (SAF) use the Competency-Based Learning (CBL) Methodology to provide
feedback to their trainees on their leadership skills through coaching. The
recording and transfer of such feedback is also facilitated through the use of
mobile technologies such as Personal Digital Assistants. The paper presents how SAF schools use CBL
methodology, as it outlines the recently completed SAF Competency-Based
Learning Project. The paper also highlights a trial implementation carried out
by the SAF instructors in using mobile technology to assist them to develop
their trainees’ leadership skills through this project.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7074
Gary E. Riccio, Michael Lerario, Blaise
Cornell d’Echert
The Wexford
Group
Columbus, GA
Frederick J. Diedrich, Jason Sidman, Alexandra Geyer, Tad Brunyé
Aptima, Inc.
Woburn, MA
The contemporary operational environment
(COE) is rapidly evolving – what worked recently may not work now and probably
will not work next week. As the Army operates increasingly as an expeditionary
force, it will become even more important to share lessons learned quickly and
to promulgate best practices widely. Training requirements are therefore also
becoming increasingly dynamic and complex.
Instructors must develop timely teaching points that rest on solid
doctrinal foundations while addressing current issues. Accordingly, through the
sponsorship of the Army Research Institute and the Office of the Secretary of
Defense, we are exploring educational strategies and technologies that can
close the gap between the classroom and the operational environment. The goal
is to provide capabilities that support interactions between students and
outside experts in the Contemporary Operational Environment under the control
of a Small Group Instructor. The central idea is to enable students to relate
classroom concepts to current issues through the guidance of outside experts.
In this manuscript we describe the problem, the theoretical background for our
approach, and results from our efforts to identify requirements.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7103
|
RTI
International Research
Triangle Park, NC |
Carnegie
Mellon University Pittsburgh,
PA |
Mercyhurst
College Erie,
PA |
Recent military and homeland security
events have made public officials, the media, and the public skeptical of the
conclusions produced by intelligence analysts. Expert panels investigating the
causes of intelligence failures some-times list biases in judgment and decision
making during analysis as a contributing factor. Experimental findings from
cognitive and social psychology and decision science are typically cited to
make this inference. Decision making or analytic biases may indeed influence
intelligence products, but findings from the bias literature may be
over-generalized. Given that individuals can easily be biased, are flawed
thinking and judgments inevitable? Conversely, can analysts be trained to
understand and detect their biases, and use that knowledge in applying
heuristics capable of counteracting biases, to minimize mistakes in judgment?
This paper addresses training approaches that can influence the mental
processes that decision makers follow during the intelligence-producing task.
After a brief literature review of decision making bias, analytic methods are
described and training interventions outlined that might mitigate biases in
real-world analytic situations. Finally, the training approaches that have
influenced development of leadership training are described where, again,
awareness of potential biased reasoning is necessary for decision makers
engaged in critical warfighting tasks.
This paper is
available on the 2007 I/ITSEC CD ROM.
Order the paper
from I/ITSEC’s Website.
2007
Paper No. 7210
|
U.S.
Army Research Institute Fort
Benning, Georgia |
Bruce
Leibrecht,
Richard Wampler Northrop
Grumman Corp. Columbus,
Georgia |
It is commonly believed that digital
skills are highly perishable, yet little empirical data exist to document just
how perishable these digital skills are.
Skill decay for Force XXI Battle Command Brigade and Below (FBCB2) was
investigated with 54 infantry captains who attended a two-day FBCB2
familiarization course. Operator
knowledge and skills were measured immediately after the course and again eight
weeks later. During the eight-week
retention interval, none of the participants had an opportunity to practice
with an FBCB2 system. At baseline,
individuals did not do as well on the knowledge test (avg. score 40%) as they
did on the hands-on test (avg. score 72%).
After the eight-week retention interval, there was no significant
overall decline in performance on the knowledge test, but this may have been
due to a floor effect. On the other
hand, there was a small (10%) but significant forgetting of operator
skills. Decay of knowledge and skills
was not uniform across individual items.
On the knowledge test, forgetting was most acute for a specific fact
(the packet mode message size limit in bytes – 45% decline). On the operator skill test, auto-centering
the icon on the map, creating a route, and creating an address group all showed
significant declines of 20% or more while creating and sending free-text
messages and using the line-of-sight tool showed virtually no decay. Measures of training, experience, and
knowledge were all examined as possible mediators of skill decay but were not
found to have a very large impact on proficiency scores. In summary, these findings suggest that
digital skills are not uniformly prone to decay and therefore training can be
made more effective and efficient by targeting the most perishable skills. Different methods for improving skill retention
are discussed.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7252
US Joint Forces Command has listed
prevention of fratricide and combat identification as one of their top five
most urgent unfulfilled operational concerns.
In recognition of the urgency of this training shortfall, NATO Allied
Command Transformation embarked on an investigation into the effectiveness of a
prototype computer-based combat identification training system under the
Coalition Combat Identification (CCID) Advanced Concept Technology Demonstrator
(ACTD).Under the cognizance of Joint Experimentation Evaluation and Analysis
(JEEA) staff officers from NATO ACT, the prototype combat identification
training system (CITS) completed an operational evaluation during a live
exercise, Exercise Urgent Quest, in September 2005. Derived from commercial-off-the-shelf
software, CITS is a multi-lingual, multi-spectral, distributive training system
that embodies recognition training as well as operational applications in
facilitating intelligence gathering, surveillance and target tracking. Under
this evaluation initiative, the participating nations included Canada, Denmark,
France, Italy, Sweden, UK and USA.
Following the success of Exercise Urgent Quest, CITS evaluation was
extended to cover pre-deployment training to Spanish and French units of the
land component of the NATO Response Force in 2006. In this paper, the authors
will briefly review the charter of NATO ACT and methodology employed in the
assessment process, provide an overview of the technology employed in CITS, and
highlight the results of the evaluation.
This 2-year controlled, independent study provides coalition commanders
with an objective assessment as to the benefits to be gleaned from recognition
training.
This paper is available on the 2007
I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7272
Directorate of
Individual Training (Army), Trenchard Lines,
Upavon, Wiltshire, UK
This paper describes a synchronous
e-Learning Mathematics Skills Revision (MSR) course, developed for students of
the Royal Electrical and Mechanical Engineers (REME). The courseware takes
about 10 hours to complete and is SCORM conformant. MSR has 2 parts: in the
first, students work individually, but with optional on-line instructor
(e-Moderator) support, at courseware comprising 9 remedial tutorials, each
incorporating multiple choice practice and assessments; in the second, students
work in a collaborative and competitive game format, again facilitated by an
on-line instructor, where they apply their recently revised mathematics skills
to a virtual world simulation of a military logistics problem. MSR has been
evaluated with some 240 students with positive results. Drawing upon this
detailed quantitative and qualitative evaluation, an number of issues are
explored, including: (1) the locus of learning efficacy, with particular
reference to student self-esteem and motivation; (2) instructional design
constraints imposed by SCORM conformancy; (3)
e-Moderator skills and on-line facilities; (4) e-Learning courseware
production, using a combination of in-house and out-sourced suppliers; (5)
e-Pedagogy and Serious Games, where it is argued that the latter actually add
little or nothing to e-Pedagogy, notwithstanding the success of MSR in
particular, and current worldwide interest and heightened expectations for
Serious Games in general.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7417
H. Chad Lane, Mark G. Core, David Gomboc, Ashish Karnavat, & Milton Rosenberg
Institute for Creative Technologies University
of Southern California
Marina del Rey, CA USA
We describe
some key issues involved in building an intelligent tutoring system for the
ill-defined domain of interpersonal and intercultural skill acquisition. We
discuss the consideration of mixed-result actions (actions with pros and cons),
categories of actions (e.g., required steps vs. rules of thumb), the role of
narrative, and reflective tutoring, among other topics. We present these ideas in the context our
work on an intelligent tutor for ELECT BiLAT, a
game-based system to teach cultural awareness and negotiation skills for
bilateral engagements. The tutor provides guidance in two forms: (1) as a coach that gives hints and feedback
during an engagement with a virtual character, and (2) during an after-action
review to help the learner reflect on their choices. Learner activities are
mapped to learning objectives, which include whether the actions represent
positive or negative evidence of learning. These underlie an expert model,
student model, and models of coaching and reflective tutoring that support the
learner. We describe several other cultural and interpersonal training systems
that situate learners in goal-based social contexts that include interaction
with virtual characters and automated guidance. Finally, our future work
includes evaluations of learning, expansion of the coach and reflective
tutoring strategies, and integration of deeper knowledge-based resources that
capture more nuanced cultural aspects of interaction.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7012
|
Jardon
& Howard Technologies, Inc. Orlando,
FL |
Psychology
Dept. FAU Boca
Raton, FL |
Naval
Air Warfare Center Training Systems Division Orlando,
FL |
The technical capability to design
interactive and adaptive learning environments is maturing rapidly. In this paper, we report on an empirical
investigation that used a variant of the wizard-of-oz methodology to identify
guidelines for the design of effective simulated tutors within this type of
environment. Thirty-six subjects
completed a 4 hour curriculum on simple direct current (D.C.) circuits while
interacting with an unseen human tutor via a chat interface. Student learning,
calculated using pre-test and post-test scores, were significantly correlated
with the amount that the students talked, as measured by the average number of
words per student turn. Also, the use of
open-ended questions resulted in a statistically significant increase in the
length of student utterances. Tutor
strategies that were significantly positively predictive of student learning
gains included reinforcing correct responses by restating the answer and
remediating incorrect responses by proposing activities likely to help the
student figure out the correct answer by him or herself. We suggest that the strategies that proved
effective for human tutors are good candidates for incorporation into a
simulated tutor. The paper ends with a
discussion of the implications for the design of simulated tutors, the
limitations of the current research, and additional research requirements.
This paper is
available on the 2007 I/ITSEC CD ROM.
Order the paper
from I/ITSEC’s Website.
2007
Paper No. 7201
|
Virginia
Travers,
R. Bruce Roberts,
Jacob Marks Joint
ADL Co-Lab Orlando,
FL |
Christopher
Guin,
Ray Tomlinson BBN
Technologies Cambridge,
MA |
In
recent years, the training community has shown an increased interest in
techniques for integrating distributed learning and simulation-based training.
BBN, together with its sponsor, the Joint ADL Co-Lab, has implemented an
architecture that supports both traditional, SCORM-based training, and
experiential, simulation-based training. This architecture allows integration
of traditional didactic and experiential content, makes it easier for content
developers to integrate new training content, improves ease-of-use for the
student, and improves training value. The architecture focuses on two
components: the SCORM Run-Time Environment, and a new Distributed Training
Event Coordination Service (DTECS). While the SCORM Run-Time Environment
launches and records student performance on traditional, didactic content, the
DTECS coordinates configuring, launching, and reporting results from
simulation-based training systems. This separation of function allows the DTECS
to handle the unique requirements of simulation-based training. The
architecture uses the W3C web services standards (including XML, SOAP, and Web
Services Description Language) to provide communications among the SCORM
Run-Time Environment, the DTECS, and the training systems. The use of web
services provides an open, reliable, and extensible distributed architecture.
This architecture supports the development of independent components, such as
user databases, content repositories, and assessment engines, which can be
easily integrated with existing components through web services. The
integration of didactic and experiential training is supported not only in the
architecture, but also in the selection and configuration of training content.
A student’s performance on didactic content informs the selection and
configuration of experiential training systems. Likewise, the student’s
performance in experiential training drives the selection of new or remedial
didactic content. This coupling improves the training value for students by
providing opportunities to practice what they have just learned…
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7066
|
Shawn
A. Weil,
Gilbert Mizrahi Aptima,
Inc. Woburn,
MA |
Joint
ADL Co-Lab Orlando,
FL |
BBN
Technologies Cambridge,
MA |
Training is a fundamental activity for
military readiness. To reach and maintain a high level of performance,
individuals must be taught the information and then offered the opportunity to
practice the skills critical to their roles and responsibilities. In many
instances, didactic instruction (either classroom or computer-based) is
provided separately from experiential instruction. Warfighters
today have the benefits of computer technology that can be used to facilitate
instruction—both didactic and experiential—in the same environment. However,
the separation between these two types of training—now exemplified by
e-learning (didactic) and PC-based simulation (experiential)—remains largely
divided. This is an unfortunate circumstance that does not allow warfighters to “train as they operate.” To address this
apparent gap, the Joint Advance Distributed Learning (JADL) Co-lab is exploring
ways in which didactic and experiential learning approaches can be synthesized.
The current version of SCORM (Sharable Content Object Reference Model) supports
many types of curriculum sequencing, but has a number of architectural and
pedagogical limitations when didactic/experiential integration is the desired
end result. This is in part a consequence of the differences between e-learning
and simulation, including the overall objectives of training, the clarity of
performance measurement, the necessity of “roles” in instruction, and the
degree of non-linearity implied by the instructional technique. The incongruity
of the approaches requires careful thought as to effective integration. This
paper presents a number workflows (or templates) which illustrate the variety
of instructional possibilities inherent in integrated didactic/experiential
training. These contextually-anchored use cases are intended to guide
instructional designers as they increase the degree of integration in
computer-supported instruction. The implications of these workflows onto the
SCORM 2004 standard and the JADL 2010 Integrated Prototype Architecture (IPA)
are discussed.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7471
Northern
Illinois University
DeKalb, IL
Learnitec, Inc.
Madison, WI
The Sequencing and Navigation module of
SCORM 2004 (3rd edition) is one of its most powerful features, but also one of
its most complex. With the addition of the extended sequencing capability,
SCORM (Sharable Content Object Reference Model) has become suitable for the
design and implementation of complex and realistic training systems. With its
new sequencing and navigation features, it has become easier to build systems
to teach technical topics such as mathematics where a highly sequential
approach is required. However, the
system must be carefully designed and implemented for each content object to be
delivered at the right time based on the student’s performance. Many users have
been unable to benefit from the full generality of the Sequencing and
Navigation standard due to the number of options available for each SCO, the
complex interrelationships between options, and the need to provide appropriate
sequencing criteria not only for individual SCOs for but for higher-level
aggregations of SCOs. In this paper, we
attempt to provide a new level of understanding of the Sequencing and
Navigation standard through the use of a top-down diagram-based approach,
concentrating on procedural topics. We give a comprehensive set of examples
showing a variety of cases that occur in procedural systems.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7008
|
US
Joint Forces Command Suffolk,
VA |
Concurrent
Technologies Corporation Suffolk,
VA |
The Joint Knowledge Development and
Distribution Capability has the responsibility to
integrate the various individual training services with other knowledge,
education, and training services. This integration enables Joint training to be
managed as a continuum by allowing different content sources to be used by
trainers to reinforce and amplify each other.
There are two semantic loci around which the Joint content can be
organized – the individual warfighter and the tasks the warfighter carries out
within the battlespace. Each locus supports a star-topology information
network. Around the individual warfighter, we organize characterizing
information such as, “What is my job?”
“What are my roles and responsibilities within my unit?” “Where am I in my career?” “What is my experience?” Around the tasks, we organize information
such as, “What training is available for this task?” “Who is supposed to carry it out?” “What is the corporate knowledge about this
task?” JKDDC has characterized this
learning in three areas – on the job experience, collective training, and
individual training – to merge the information networks. The resulting structure supports the
integration of the different knowledge services to develop individual training
solutions for the warfighter. Additionally, this network allows crosswalking
requirements against capabilities. Examples of the types of integration that
are supported by the information network are:
• When a new lesson learned is published, we can identify which
personnel have been recently trained in that area and forward it to them. • If
a commander has to cross-fill a Joint billet, he can identify who are the best
trained people for that billet within his organization. • When a warfighter
receives new orders to a Joint billet, they can determine what training they
need and from where it is provided. • Exercise planners can select prerequisite
or post-exercise mitigation training options from a list targeted to the
training objective of concern.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7501
COL James C. Markley, Dr. Millie J. Abell HQ,
US Army TRADOC
Fort Monroe, VA
As the Army moves forward with implementing
the Army Force Generation (ARFORGEN) Model, the US Army Training and Doctrine
Command (TRADOC) must respond to the
Army Force Generation (ARFORGEN) Model, the changing
requirements of the operational force for trained Soldiers and
leaders.To better position TRADOC to meet the requirement for trained
and adaptive leaders, GEN Wallace, Commanding General (CG) TRADOC, directed his
staff to look at hoe TRADOC could take advantage of the latest developments in
proven adult learning strategies and training technology to shorten the time
leaders spend in TRADOC schools.
As a result, TRADOC is
in the process of piloting a learning model that has the
following components: (1) an increased use of technology-based instruction; (2)
Saturday training: (3) fast tracking, and (4) use of Guided Experimental
Learning (GEL) as the instructional design and delivery strategy. Use of
technology, Saturday training, and fast tracking provide greater efficiency in
training, while GEL ensures training will be at least as effective as existing
instruction despite a reduction in training time. In addition, CG TRADOC
directed his staff to ensure the model incorporates strategies that make
students more powerful learners. TRADOC is now identifying
learning-how-to-learn strategies for integration into the learning model’s
pilot courses. TRADOC is piloting the new learning model within two Captians Career Courses in fiscal years 07-08, with the
TRADOC Analysis Center serving as its independent evaluator.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7400
Consultant
In order to help learners acquire strong
knowledge structures or mental models of performance and systems, they must be
exposed to instructional messages and environments which convey the complexity
of the models they are intended to assimilate. Many instructional strategies
and media focus on helping learners build and develop these mental models.
However, before strategies and media can be selected, instructional designers
must analyze content into structures which are similar to the mental models the
learner will build. Key to analyzing
and building these structures is identifying the relationships between systems,
environments and performance. The common structures for defining these
relationships are task and learning objective statements. These structures identify a performance
requirement for a system within a given environment. These statements, by their
very nature though, are static and discrete and do not encompass all the
dynamic relationships which exist in real-world operation. Modeling approaches can be used to help
instructional designers develop strong and complete knowledge structures of
content. The very essence of modeling helps identify key relationships between
systems and the environments they exist within. Unfortunately, modeling methods
are not commonly associated with traditional instructional systems design.
Rather, modeling skill sets are more associated with systems analysis,
intelligent tutoring and computer science. This paper identifies and introduces
modeling methods for instructional designers to capture the relevant
performance, systems, and environmental knowledge for representations in
training.
This paper is
available on the 2007 I/ITSEC CD ROM.
Order the paper
from I/ITSEC’s Website.
2007
Paper No. 7044
Command Performance
Research, Inc.
Champaign, IL
The same factors that create a greater
need for U.S. Army Soldiers and leaders to learn quickly make it more difficult
to receive instruction via the traditional Army education system. Lifelong
Learning Centers (LLCs) leverage advanced technologies to enable anytime,
anywhere learning throughout the career of Army personnel (Wilson & Helms,
2003). In the present research, a comprehensive, generalizable
framework was developed to conceptualize the effectiveness of LLCs and was used
to conduct a formative evaluation of a pilot LLC situated at the U.S. Command
and General Staff College. The framework, based on public sector program
evaluation techniques, was a useful tool for capturing LLC functioning, from
the acquisition of resources to the achievement of organizational impact. The
evaluation findings indicated that although the pilot LLC was technologically
stable and cost-effective, system adoption rates were lower than criterion due
to human factors involved in collective information management and
organizational change. Recommendations for further validating the LLC
assessment framework and enhancing LLC implementation are provided.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.
2007
Paper No. 7283
David H. Olwell, Jean M. Johnson, Jarema
M. Didoszak
USN Naval
Postgraduate School
Monterey,
California
Chief Of Naval Operations (N1ZC)
Washington, DC
The Defense Modeling and Simulation
Coordination Office presented the Naval Postgraduate School with an enormous
challenge in 2006: design and deliver an
educational program by 2008, segmented by career specialties, for 20,000 or
more acquisition professionals, focusing on the effective use of modeling and
simulation in acquisition. The
acquisition workforce is central to force transformation, and education is the
key to transforming that workforce. This
paper describes the processes, lessons learned to date, and assessment plan for
this project. We applied a systems
engineering approach to the problem of curricular design. The resulting solution consists of four
spirals. The first spiral focused on defining the problem and ascertaining
requirement. We developed our analysis
based on factors such as our market segmentation of the acquisition workforce,
the current resources available, the state of the modeling and simulation body
of knowledge, the desired educational outcomes for each market segment, and the
gaps that existed between those outcomes and the existing resources. At each step in the process, we involved key
stakeholders from the acquisition, test and evaluation and training
communities. The results of the first
spiral are the requirements generated, and subsequent lessons learned; they
will be the focus of our discussion herein.
We will also briefly summarize the other spirals, which are currently
underway, that involve architecting the curricula in spiral two, course design
and testing in spiral three, and delivery and assessment of the curriculum for
spiral four.
This paper is available on the
2007 I/ITSEC CD ROM.
Order the paper from I/ITSEC’s Website.