In January, the Pentagon’s chief weapons tester said the Department of Defense should stop deploying its new network security platform, known as Joint Regional Security Stacks.

JRSS is a suite of equipment that is expected to improve security through firewall functions, intrusion detection and prevention, enterprise management, and virtual routing. It is expected to centralize and standardize network security into regional architectures, making it easier to share information on threats.

But the annual report from the DoD’s Office of Operational Test and Evaluation questioned the effectiveness of the new system.

“The [chief information officer] and the services should discontinue deploying JRSS until the JRSS demonstrates that it is capable of helping network defenders to detect and respond to operationally realistic cyberattacks,” the annual report read.

Speaking at West 2018 in San Diego, California, Col. Gregory Griffin, the JRSS program manager at the Defense Information Systems Agency, said that while the agency would incorporate fixes to many of the problems the report identified, he disagreed with the report’s ultimate recommendation.

“We do disagree to pause or halt the program at this time,” he said. Instead, he suggested, it’s impossible to know how the program will fare against true threats until it’s deployed and faces those attacks.

Among the recommendations from the Pentagon’s weapon tester was that DISA’s program office use realistic testing results to improve configurations, training and procedures.

The report said that during a test last year the version of the program in use by the Air Force did not help protect the network.

“The Army, Air Force and other DoD components continue to deploy JRSS to operational DoD networks, despite testing that demonstrates JRSS technology integration, training, and service and agency processes are not able to protect networks from cyberattacks,” the report read.

Griffin said nine of the 11 stacks for nonclassified networks within the continental United States are operational. The two remaining stacks could be operational this summer. A total of 20 nonclassified stacks worldwide could be up and running by the end of fiscal 2019. Another 25 stacks for classified networks are also planned.