High-Stakes Psychomotor Ability Assessment: A Military Selection Case Study of Practice Effects in Airplane Tracking Tasks
Saved in:
| Title: | High-Stakes Psychomotor Ability Assessment: A Military Selection Case Study of Practice Effects in Airplane Tracking Tasks |
|---|---|
| Language: | English |
| Authors: | Christopher Draheim (ORCID |
| Source: | Cognitive Research: Principles and Implications. 2025 10. |
| Availability: | Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ |
| Peer Reviewed: | Y |
| Page Count: | 26 |
| Publication Date: | 2025 |
| Sponsoring Agency: | U.S. Naval Research Laboratory (NRL) (DOD) Office of Naval Research (ONR) (DOD) |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Aviation Education, Flight Training, Military Training, Military Personnel, High Stakes Tests, Simulation, Psychomotor Skills, Personnel Selection, Drills (Practice), Video Games, Prediction |
| DOI: | 10.1186/s41235-025-00672-z |
| ISSN: | 2365-7464 |
| Abstract: | Aviation selection tests are high-stakes assessments designed to identify candidates capable of succeeding in demanding flight environments. Most branches of the US military incorporate both content-based and process-based assessments to evaluate prior knowledge and reasoning ability, respectively. A challenge with high-stakes process tests is that their validity requires participant naivety, which is increasingly difficult to maintain in the modern internet era. As such, these high-stakes tests must be continuously evaluated to ensure the most valid, reliable, and cost-effective selection procedures are employed. To address this, we examined practice effects in the psychomotor airplane tracking tasks of the Navy's Aviation Selection Test Battery (ASTB). We had 146 Naval Flight Students and 119 enlisted Sailors perform the ASTB's psychomotor battery six times across two days. Results revealed large practice effects, shifting in rank ordering of individuals across attempts, and that psychomotor performance had not stabilized even by the sixth attempt. Prior action video gaming and flight simulator gaming experience correlated with psychomotor performance, with some evidence that improvements were related, albeit weakly, to either gaming experience or initial performance. Finally, correlations between psychomotor performance and eventual naval flight training scores were stable across the six attempts, but simulations indicated predictive validity can range widely if participants have differing levels of practice from one another. Overall, these findings indicate that the psychomotor component of the ASTB is a valuable inclusion to the Naval Flight School selection process but also could benefit from further refinement. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1499074 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Aviation selection tests are high-stakes assessments designed to identify candidates capable of succeeding in demanding flight environments. Most branches of the US military incorporate both content-based and process-based assessments to evaluate prior knowledge and reasoning ability, respectively. A challenge with high-stakes process tests is that their validity requires participant naivety, which is increasingly difficult to maintain in the modern internet era. As such, these high-stakes tests must be continuously evaluated to ensure the most valid, reliable, and cost-effective selection procedures are employed. To address this, we examined practice effects in the psychomotor airplane tracking tasks of the Navy's Aviation Selection Test Battery (ASTB). We had 146 Naval Flight Students and 119 enlisted Sailors perform the ASTB's psychomotor battery six times across two days. Results revealed large practice effects, shifting in rank ordering of individuals across attempts, and that psychomotor performance had not stabilized even by the sixth attempt. Prior action video gaming and flight simulator gaming experience correlated with psychomotor performance, with some evidence that improvements were related, albeit weakly, to either gaming experience or initial performance. Finally, correlations between psychomotor performance and eventual naval flight training scores were stable across the six attempts, but simulations indicated predictive validity can range widely if participants have differing levels of practice from one another. Overall, these findings indicate that the psychomotor component of the ASTB is a valuable inclusion to the Naval Flight School selection process but also could benefit from further refinement. |
|---|---|
| ISSN: | 2365-7464 |
| DOI: | 10.1186/s41235-025-00672-z |
