AFRL Researchers Fine-Tune F-35 Speech Recognition System

  • Published
  • By John Schutte
  • AFRL/HE
WRIGHT-PATTERSON AIR FORCE BASE, Ohio --  Due in part to testing and analysis conducted at AFRL, the F-35 Lightning II will be the first US fighter aircraft equipped with a speech recognition system capable of "hearing" a pilot's spoken commands to manage various aircraft subsystems, such as communications and navigation. AFRL researchers are in the process of collecting data and recommending various improvements geared towards ensuring optimal performance of the F-35's new speech recognition system as it undergoes future operational tests.

Currently, pilots must press buttons, flip switches, or glance at instruments to obtain status information. The new system, SRI International's DynaSpeak® product, not only simplifies pilot workload but increases safety and efficiency, since it allows pilots to remain focused on flying the aircraft and scrutinizing the combat environment. To fine-tune the new capability, researchers are using digital recordings from in-flight tests aboard the NF-16D variable-stability in-flight simulator test aircraft (VISTA) to pinpoint problem phrases--those that could be difficult to interpret or are otherwise ambiguous--and recommend alternatives that are more easily recognized. They are also tweaking error parameters and adjusting the system's sensitivity for peak speech recognition performance with minimal errors.

SRI International developed DynaSpeak as a highly accurate system for use in noisy environments--specifically, for embedded devices such as personal digital assistants, in-car navigation systems, and avionics systems. It is speaker-independent, meaning that a pilot can use the system without first "training" it to perceive his or her voice, a process taking up to an hour with previous experimental systems. SRI International is working with integrating contractor Adacel Systems, Inc., to tailor the system for the F-35 Joint Strike Fighter's airborne environment.

Leveraging resources at the US Air Force Test Pilot School (Edwards Air Force Base, California), the AFRL team worked with six pilots whose primary mission for the effort was flying VISTA test bed sorties to collect speech recognition data. The resultant digital recordings enabled the team to study troublesome phrases, as well as the impact of environmental factors on acoustic characteristics. While ambient cockpit noise degrades the quality of spoken commands entering the recognition system, which could cause errors of misinterpretation or misunderstanding, acceleration (i.e., g) forces up to approximately 6 g have little affect on the system's performance. If proven successful on the F-35, this baseline speech recognition system could be advanced for incorporation into aircraft such as the F-22 Raptor and future unmanned air vehicle platforms.