AFOSR Study Evaluates Trust in Autonomous Systems Published May 7, 2015 By Molly Lachance Air Force Office of Scientific Research ARLINGTON, Va. -- A case study funded by the Air Force Research Laboratory's (AFRL) basic research directorate, the Air Force Office of Scientific Research, has quickly transitioned to AFRL's Human Performance Directorate (AFRL/RH), where researchers are now using it in the field to examine how pilots respond to a new form of flight autonomy. In collaboration with NASA and the Air Force Flight Test Center, Dr. Nhut Tan Ho, a professor of mechanical engineering at California State University, Northridge led the 2012-2014 study, which looked at how cultural and organizational factors affect trust of and reliance on an automated system. The study focused specifically on the Automatic Ground Collision Avoidance System (Auto-GCAS), which the Air Force integrated into the fleet in September 2014 after decades of development. The system is intended to protect pilots by taking temporary command of an aircraft and switching into an automatic recovery mode when it detects imminent impact with the ground. As the Auto-GCAS system neared implementation, Ho and his team were sampling test pilots, engineers, managers, and operational pilots to identify lessons learned from technology development, determine real-world perspectives on trust evolution, and examine potential influences on trust. At the end of the study, Ho provided a set of questions for researchers to use to create more trustable automation, which is now being used in an AFRL field study for the Air Force Flight Test Center, the F-16 System Program Office Safety Office, the Air Combat Command Safety Office, the AFRL Air Systems Directorate, and F-35 Collision Avoidance Technology. This four year study plans to examine operational pilot trust, development, and evolution in Auto-GCAS and identify and document user experience, concerns, impact, and benefits of the technology as they emerge. "Understanding how operators respond to and rely on a technology like Auto-GCAS is as important as the operation of the technology itself," said AFOSR Program Officer Dr. Benjamin Knott. "This case study and the follow-on field study are helping the Air Force improve the interface between operators and autonomous systems." In the first phase of the field study, researchers will visit Hill Air Force Base, Eielson Air Force Base, Aviano Air Base, Kunsan Air Base, Osan Air Base, Shaw Air Force Base, and a number of Air National Guard bases. Additionally, a large survey was released in April, which will assess perceptions about Auto-GCAS as well as opinions toward novel forms of autonomy. "As a result of the field study, we hope to be able to provide recommendations to the Air Force that enhance performance and mitigate factors that undermine trust calibration," said Dr. Joseph Lyons, Technical Advisor, in the Human Trust and Interaction Branch at AFRL/RH. Auto-GCAS received some good press in February, when the system was credited with saving its first F-16 from certain collision with the ground in Syria (http://aviationweek.com/defense/ground-collision-avoidance-system-saves-first-f-16-syria). Researchers hope to see more articles like the one mentioned above as they work to improve trust in and reliance on the autonomous systems that could ultimately save pilots' lives. __________________________________ AFOSR continues to expand the horizon of scientific knowledge through its leadership and management of the Air Force's basic research program. As a vital component of the Air Force Research Laboratory, AFOSR's mission is to discover, shape and champion basic science that profoundly impacts the future Air Force. To stay up-to-date on the latest AFOSR happenings, please join us on Facebook at http://www.facebook.com/AFOSR or follow us on Twitter at http://www.twitter.com/AFOSR.