An official website of the United States government
Here's how you know
A .mil website belongs to an official U.S. Department of Defense organization in the United States.
A lock (lock ) or https:// means you’ve safely connected to the .mil website. Share sensitive information only on official, secure websites.

C-17 first to fully employ 'design of experiments' test strategy

  • Published
  • By Lt. Col. Sandy Burr
  • Air Force Flight Test Center Public Affairs
EDWARDS AIR FORCE BASE, Calif. (AFMCNS) --  The 418th Flight Test Squadron C-17 test team recently became the first to fully employ a test strategy called design of experiments, or DOE, to a C-17 Globemaster III pilot-in-the-loop oscillation test.

"Design of Experiments," said Dr. George Kailiwai, Air Force Flight Test Center technical advisor, "is a robust, powerful test strategy in which we deliberately control the inputs or factors of a test in such a way so that we can deduce their relationships, if any, with the outputs or responses."

After extensive test planning with the team's handling qualities engineers, the test team chose to compare the performance of a new flight control software program with the existing one by having three pilots fly three different maneuvers while the C-17 had two different flap configurations, Dr. Kailiwai said.

The test team recorded each test pilot's pilot-in-the-loop oscillation rating score after each run and analyzed the results.

To arrive at a score, the pilot used a decision tree. The score related to the tendency of the flight control system to put the pilot in a potentially hazardous or out-of-control situation while performing specific flight maneuvers, Dr. Kailiwai said. The flight test was conducted in the "blind," as the test pilots didn't know which flight control software program was activated, and the test runs were conducted in a random order.

"The test team randomized the test runs to minimize the effects from nuisance variables such as learning curve and the 'halo effect,'" Dr. Kailiwai said. The test pilots instantly recognized the benefits of randomizing the test runs and readily accepted randomization as an essential feature in the DOE test strategy.

"Although we teach and employ DOE at the Test Pilot School, and we've used DOE as a test strategy at our Air Force Electronic Warfare Evaluation Simulator facility in Fort Worth (Texas), this is the first time we've employed the DOE 12-step test process to a flying and handling qualities test project," Dr. Kailiwai said.

As with all test programs, the team approach is key to success.

Greg Hutto, 53rd Wing at Eglin Air Force Base, Fla., provided the DOE training. Bill Kitto, 95th Communications Group, Capt. Bryon McClain, 31st Test and Evaluation Squadron, 2nd Lt. Matt Timbrook, 412th Test Wing, and Madelene Vega of the 418th Flight Test Squadron, assisted with the analysis of the C-17 flight test data.