Pentagon pushes ahead on autonomous AI systems

The Pentagon has announced a new initiative to develop and deploy thousands of autonomous systems across multiple domains within the next two years. The program, dubbed Replicator, is meant to counter China’s military mass with mass of its own, but with a twist: the US drones will be small, smart, cheap, and many.

Replicator is overseen by Deputy Secretary of Defense Kathleen Hicks and Vice Chairman of the Joint Chiefs of Staff Adm. Christopher Grady, with support from Doug Beck, director of the Defense Innovation Unit. Hicks said the program was spurred by China’s current “biggest advantage” militarily, "which is mass. More ships. More missiles. More people."

“To stay ahead, we’re going to create a new state of the art — just as America has before — leveraging attritable, autonomous systems in all domains — which are less expensive, put fewer people in the line of fire, and can be changed, updated or improved with substantially shorter lead times,” Hicks said at the National Defense Industrial Association’s Emerging Technologies conference on Monday.

Hicks pointed to the war in Ukraine, in which cheap, often commercial drones have proven indispensable on the battlefield for reconnaissance, targeting, and attacks. Russia too, she said, appeared to have a similar mass before launching its invasion last February. However, this program is squarely focused on China. Hicks called this moment a "generational challenge to American society."

“We’ll counter the [People’s Liberation Army’s] mass with mass of our own, but ours will be harder to plan for, harder to hit, and harder to beat. With smart people, smart concepts, and smart technology, our military will be more nimble, with uplift and urgency from the commercial sector,” she said.

Replicator’s goal of fielding small drones in high numbers and on a rapid timeline echoes calls from former DIU director Mike Brown for the Pentagon to better leverage commercial innovation to deliver capability at scale — an approach he called a “hedge strategy.” House appropriators have backed that idea in their fiscal 2025 defense spending bill. The legislation would allocate $1 billion toward establishing a DIU-managed hedge portfolio made up of low-cost drones, agile communication and computing modes and AI capabilities.

But Replicator also raises some hard questions about the ethics and safety of lethal autonomous weapons. Hicks said the Pentagon is intent on ensuring that humans remain “in control” of the systems, and that the department is updating its policy on the use of AI in military operations. However, some experts argue that the pace and scale of AI development may outstrip the ability of humans to oversee and intervene in the actions of autonomous systems, especially in complex and dynamic environments.

Moreover, the reliability and robustness of AI systems may be compromised by the lack of rigorous testing and evaluation before deployment. As Breaking Defense reported, the Pentagon’s chief weapons tester, Robert Behler, has warned that the department is not adequately testing AI systems, and that there is a need for more realistic and adversarial scenarios to assess their performance and limitations.

The advancement of AI in warfare is likely to pose some major moral and technical challenges in the near future. The Pentagon’s Replicator program may be a bold and innovative move to counter China, but it also requires careful and responsible oversight to ensure that the US does not lose its edge or its values in the pursuit of AI supremacy.