HUMAN SYSTEMS INTEGRATION
Automated Performance Assessment
for DD(X) Post-Watch Debrief
Evaluating Training Effectiveness
During Training System Development
The Cognitive Cockpit – State of
the Art Human-System Integration
Team Exposure Stress Training
(TEST): An Approach for Reducing Stress
Team Training with Simulated
Teammates
Facilitating Leadership in a
Global Community: A Training Tool for
Multicultural Team Leaders
A Framework for Applying HSI Tools
in Systems Acquisition
Simulation and Network-Centric
Emergency Response
Current and Future Net-Centric C3:
Usage and Preferences
Measuring Situation Awareness for
Dismounted Infantry Squads
Human Factors in Air Force Flight Mishaps: Implications for Change
Automated Performance Assessment
for DD(X) Post-Watch Debrief
Ruth A. Wienclaw, Ph.D. Wienclaw Associates Paul E. Van Hemel, Ph.D. Raytheon Company In potentially life-and-death situations as may occur aboard the DD(X), it is essential to develop sound performance assessment measures, to devise and implement appropriate and thorough feedback mechanisms, and to employ feedback earlier rather than later in the process. The technology of the 21st century offers many methods to do these tasks. However, just because technology exists does not mean that its application will be useful or even that it will not have a negative impact on performance. Further, the incorporation of many technologies currently cannot be justified by their return on investment. Automated data collection and performance assessment capabilities for the DD(X) are categorized into three levels. Level 1 capabilities include automated data collection for after action review. These data will be analyzed in light of the sailor’s performance history and flagged at the watch supervisor’s station. This level may also include the ability to bookmark data and limited ability for ad hoc inquiries. Level 2 capabilities include more detailed tracking to determine why performance degradation occurs. This level may also include more advanced input technology such as voice recognition, eye tracking, and basic cognitive modeling as well as more information on the components of teamwork. Level 3 capabilities are those not feasible from either a cost or technological standpoint at the present time, including thorough cognitive modeling, ability to have immediate answers to ad hoc inquires, or inclusion of data on all components of teamwork. Whereas it is both reasonable and feasible to include such capabilities in the DD(X), they will have to be included in stages, incorporated only as technologies mature and human behavioral performance models improve. Each inclusion will need to be based on rigorous cost-benefit tradeoff analysis to ensure that the added feature’s benefits will yield an acceptable return on investment. 2005 Paper No. 2456 |
|
|
Evaluating Training
Effectiveness During Training System Development
Dennis J. Folds, Ph.D. Georgia Institute of Technology Conclusive evidence for the effectiveness of training cannot be obtained until training actually takes place, and performance of trainees after training can be compared to their pre-training baseline. Performance-based specifications for training systems, however, are often worded in such terms as “shall provide effective training”, and thereby put a burden of proof on the developer to demonstrate that these requirements are satisfied before the system is accepted for use. A common test method used in these situations is to allow a few presumed subject matter experts to view a demonstration of the training system and then provide ratings and comments on whether the system “provides effective training.” Positive ratings are offered as proof that the requirement is met. The validity of this approach is obviously suspect. A more robust test method is needed during training system development. An analytical approach to evaluating training system effectiveness can be applied during design and development. Although such an approach does not ensure that the training system will be used effectively, it can esta |