How Did US Marines Outsmart Advanced Military AI Detection Systems Using Cardboard Boxes?

During a field test documented in former Army Ranger and Pentagon official Paul Scharre’s book Army of None, US Marines demonstrated a critical vulnerability in an advanced AI surveillance system by successfully bypassing its detection algorithm. The exercise, which involved a DARPA-designed system, required the Marines to approach a highly sophisticated AI sensor from a distance and physically touch it without triggering an alert. While the system was designed to detect human movement with high precision, the Marines defeated it using unconventional, low-tech methods.

Instead of relying on standard military camouflage or stealth technology, the Marines employed tactics that mimicked what some have called “cartoon logic.” Two Marines hid beneath a large cardboard box and crept toward the sensor, while others used erratic movements such as continuous somersaults or disguised themselves as a bush. The AI system failed to detect any of them, highlighting significant gaps in how machine learning models process visual data and the inherent limitations of AI when faced with unexpected human behavior.

The Mechanics of the Failure

To understand why the AI failed to detect the Marines, it helps to understand how computer vision models are trained to recognize objects. AI does not “see” the world the way a human does; it relies on pattern recognition based on its training data.

• Training Data Bias: The AI system was trained on large datasets of standard human movement, such as walking, running, or crawling. It learned to identify the specific biomechanical patterns and silhouettes associated with these actions.
• Out-of-Distribution Inputs: Because the AI had never been trained on data featuring humans moving while hidden under a cardboard box or performing continuous somersaults, these actions fell outside its recognized parameters.
• Lack of Contextual Logic: A human guard would immediately recognize a moving cardboard box as suspicious. The AI, however, lacked common sense and contextual awareness. It simply scanned for the specific pixel arrangements of a walking human, found none, and registered the environment as clear.

Broader Implications for AI Reliability

This incident has become a widely discussed case study in the technology and defense sectors because it exposes foundational challenges in AI safety and deployment.

• Algorithmic Brittleness: The event demonstrated that highly advanced AI systems can be surprisingly fragile. A system optimized for complex, high-tech threats can be completely neutralized by a low-tech, unforeseen anomaly.
• Adversarial Vulnerabilities: Adversaries do not need sophisticated cyber-weapons to defeat AI. Simple, physical-world tactics that exploit the blind spots in an AI’s training data can render expensive surveillance systems useless.
• The Importance of Red Teaming: This test underscored the critical need for “red teaming” — the practice of actively trying to break a system using human ingenuity. Developers cannot anticipate every edge case in a laboratory setting; physical testing against creative adversaries is required to find real-world flaws.
• Human-Machine Teaming: The failure highlights why fully autonomous systems still carry significant risk. Human operators are still needed to provide the contextual understanding and adaptive reasoning that current AI models lack.

Summary

The incident where US Marines bypassed a DARPA-developed AI sensor using cardboard boxes and other low-tech tactics serves as a clear reminder of the limitations of artificial intelligence. While modern computer vision systems are highly capable within their specific training parameters, they lack human intuition and common sense reasoning. This field test proved that until AI can dynamically adapt to entirely novel situations, human ingenuity will remain a critical factor in both exploiting and safeguarding automated systems.