Defense Department program offices are struggling in testing and evaluation, particularly when it comes to systems reliability and the cybersecurity of systems across the military, according to a new report.
The Office of the Director, Operational Test and Evaluation's annual report stated that despite work toward improving reliability testing in more than 90 DoD systems, a "poor track record of meeting system reliability requirements" still pervades. Furthermore, testing on dozens of DoD systems revealed continued shortfalls in assuring cybersecurity, Michael Gilmore, director of operational test and evaluation (DOT&E), wrote in the report released Jan 20.
Gilmore underscored the cyber threat as "serious a threat to U.S. military forces as the air, land, sea and undersea threats represented in operational testing for decades."
Resource: Read the Report
Cybersecurity testing of more than 40 systems in 2014 showed the need for improvements in order to assure secure and resilient cyber capabilities, Gilmore wrote. "One important conclusion from my 2014 review of DOD programs was that operational testing still finds exploitable cyber vulnerabilities that earlier technical testing could have mitigated. My review of these systems also identified the need to increase the participation of network defenders and assessment of mission effects during threat-representative, adversarial assessments."
Among the cited vulnerabilities:
- unnecessary network services or system functions;
- misconfigured, unpatched, or outdated software;
- passwords violating Pentagon policy.
Specific platforms affected by systems-facing vulnerabilities the report detailed include the Army's Warfighter Information Network-Tactical (WIN-T), the Navy's Joint High Speed Vessel and the Freedom class of Littoral Combat Ship.
Gilmore also noted a shortfall in cyber personnel in the military, and called for increased "adversarial assessments" on DoD systems that would boost understanding of and improve how operators fight through attacks. He recommended that DoD ramp up the use of cyber ranges, closed environments and assessments of day-to-day operations live networks.
Beyond cybersecurity, reliability in systems and in the ways systems are tested and evaluated also proved to be another pressure point. Gilmore found increased use of reliability-focused policy guidance – used by more than 80 percent of the 90 programs surveyed – but despite that, there was no measureable improvements in reliability outcomes.
The use of guidelines to improve reliability also varied across the services.
"The Army has been a leader at implementing reliability policy and the use of reliability growth planning curves, and while the Air Force has caught up considerably, many Navy programs have yet to adopt these methods," Gilmore wrote.
Overall, what's critical is that improved operational testing and evaluation continue despite increasingly shrinking budget dollars, Gilmore stressed.
"What constitutes adequate operational testing under realistic combat conditions is determined not by fiscal constraints, but by our war plans and the threats we face—the enemy (always) gets a vote," he wrote. "It would therefore, be a false economy and a disservice to the men and women we send into combat to make arbitrary budget-driven reductions to either developmental or operational testing."
