TY - JOUR
T1 - A Novel Approach Using Non-Experts and Transformation Models to Predict the Performance of Experts in A/B Tests
AU - Stranger, Phillip
AU - Judmaier, Peter
AU - Rottermanner, Gernot
AU - Rokitansky, Carl Herbert
AU - Szilagyi, Istvan Szilard
AU - Settgast, Volker
AU - Ullrich, Torsten
N1 - Publisher Copyright:
© 2024 by the authors.
PY - 2024/7
Y1 - 2024/7
N2 - The European Union is committed to modernising and improving air traffic management systems to promote environmentally friendly air transport. However, the safety-critical nature of ATM systems requires rigorous user testing, which is hampered by the scarcity and high cost of air traffic controllers. In this article, we address this problem with a novel approach that involves non-experts in the evaluation of expert software in an A/B test setup. Using a transformation model that incorporates auxiliary information from a newly developed psychological questionnaire, we predict the performance of air traffic controllers with high accuracy based on the performance of students. The transformation model uses multiple linear regression and auxiliary information corrections. This study demonstrates the feasibility of using non-experts to test expert software, overcoming testing challenges and supporting user-centred design principles.
AB - The European Union is committed to modernising and improving air traffic management systems to promote environmentally friendly air transport. However, the safety-critical nature of ATM systems requires rigorous user testing, which is hampered by the scarcity and high cost of air traffic controllers. In this article, we address this problem with a novel approach that involves non-experts in the evaluation of expert software in an A/B test setup. Using a transformation model that incorporates auxiliary information from a newly developed psychological questionnaire, we predict the performance of air traffic controllers with high accuracy based on the performance of students. The transformation model uses multiple linear regression and auxiliary information corrections. This study demonstrates the feasibility of using non-experts to test expert software, overcoming testing challenges and supporting user-centred design principles.
KW - air traffic management
KW - statistics
KW - transformation models
KW - user evaluation
KW - user study
UR - http://www.scopus.com/inward/record.url?scp=85199591831&partnerID=8YFLogxK
U2 - 10.3390/aerospace11070574
DO - 10.3390/aerospace11070574
M3 - Article
AN - SCOPUS:85199591831
SN - 2226-4310
VL - 11
JO - Aerospace
JF - Aerospace
IS - 7
M1 - 574
ER -