Artificial intelligence (AI) holds immense promise for modern warfare, offering the potential to transform military operations and enhance decision advantage on the battlefield. From sophisticated intelligence analysis to optimizing logistics, the United States Marine Corps (USMC) recognizes AI as a transformative technology. AI offers extensive operational benefits, but like any powerful tool, its implementation also introduces new vulnerabilities and expands the attack surface that adversaries can exploit.

The New Threat Landscape: When AI Becomes a Target

Adversaries are not idle in the face of AI advancements. They are increasingly utilizing AI to automate attacks, enhance phishing and social engineering tactics, and develop malware that can adapt and evade traditional detection methods. For the USMC, this means anticipating that “[adversaries will attempt] to disrupt, degrade, deny, deceive, or defeat our AI systems.”

These threats can be broadly categorized into three areas:

• Adversarial AI: Refers to attacks that directly undermine the AI itself, impacting its functionality or how well it performs.
• Target Systems Analysis: Traditional attacks against the broader system or infrastructure, designed to prevent the Marine Corps from leveraging AI effectively.
• Systems Overmatch: It describes situations where a peer adversary leverages their AI to achieve superiority against friendly AI systems.

Deep Dive: Adversarial AI – Understanding the “How”

Adversarial AI is a particularly insidious threat because it targets the very features that make AI and machine learning useful: how the AI system uses data inputs to act and learn from results to improve its performance. BreakPoint Labs specializes in simulating these advanced threats to harden your AI systems.

Here are some key techniques adversaries might employ, and how BreakPoint Labs’ AI Red Teaming can expose these vulnerabilities:

• Data Poisoning: Imagine an AI system trained to identify enemy vehicles. With data poisoning, adversaries inject false information into training datasets. “Even… minor changes can cause AI systems to misclassify targets or misinterpret critical signals, directly affecting mission success.” This can degrade the model’s operational quality, jeopardize its security, or steer its outputs toward detrimental outcomes. BreakPoint Labs’ AI Red Teaming rigorously tests for these vulnerabilities, mimicking adversarial data injection to identify weaknesses in your training pipelines and model integrity.
• Prompt Injection: This involves crafting specific inputs (prompts) that manipulate a Large Language Model (LLM) into unintended or unauthorized actions, effectively hijacking its function. This can lead to sensitive data disclosure or the generation of malicious content. A significant threat, especially for Retrieval Augmented Generation (RAG) systems, is “indirect prompt injection,” where malicious instructions are hidden within external data retrieved by the AI. Our Red Teaming services include advanced prompt engineering techniques to bypass model intent and constraints, ensuring your LLMs are resilient against such attacks.
• Sensitive Information Disclosure: AI models can inadvertently leak confidential data. This could happen through training data memorization, where the model remembers sensitive information from its training phase, or through vulnerabilities in inference processes, GPU memory, or system logs. BreakPoint Labs’ Red Teaming probes for potential exposure of sensitive information or training data and tests the model’s resistance to extraction attacks.
• Malicious Content Generation: Adversaries can compel AI to “generate harmful, toxic, [or] misleading” content. This could include creating large volumes of disinformation aimed at undermining model performance or even generating unsafe code. Our experts assess model defenses against bypassing guardrails or exfiltration protections, and simulate interactions such as hate speech, toxicity, or egregious conversation detection to uncover vulnerabilities.
• Physical Perturbations: Adversaries can make minor, often imperceptible, changes to an object in the physical world that dramatically affect an AI system’s perception. This is similar to camouflage, but designed to fool machines rather than humans. BreakPoint Labs’ Red Teaming assesses how AI models react to unexpected or adversarial inputs, ensuring robustness under perturbation.

Beyond AI-Specific Attacks: The Broader Cybersecurity Context

It’s not just about direct AI attacks. Like any complex system, AI implementations rely on underlying infrastructure. An adversary can still leverage traditional cyberattack methods (Target Systems Analysis) to disrupt AI capabilities. For instance, exploiting vulnerabilities in cloud infrastructure, denying necessary data through signature management or shutting down civilian networks can cripple AI-enabled processes. BreakPoint Labs provides comprehensive system evaluation, examining system-wide vulnerabilities, supply chain vulnerabilities, deployment pipelines, and data security, integrating seamlessly with your existing cybersecurity frameworks.

The USMC’s Proactive Stance: Building a Resilient AI Force

The USMC is not only aware of these threats but is actively developing strategies to counter them. Their AI Implementation Plan is designed to mature the Service into a 21st Century fighting force that innovates and integrates AI into warfighting functions and business processes. BreakPoint Labs directly supports these initiatives by providing the crucial “Red Teaming LLMs” capabilities needed to “establish and update AI test, evaluation, validation, and verification processes.”

Key initiatives that BreakPoint Labs can help organizations mitigate these threats include:

• Expanding Education and Training: BreakPoint Labs offers specialized training for GenAI Red Teaming and security personnel about new threats and opportunities presented by AI and LLMs, regularly updating content to include the latest developments and threat landscapes.
• Developing Intelligence and Counterintelligence Requirements: We assist in threat modeling, considering how attackers might exploit AI applications and aligning these scenarios with your highest-priority risks.
• Establishing AI Test, Evaluation, Validation, and Verification Processes: BreakPoint Labs provides a structured blueprint for GenAI Red Teaming, defining specific steps, techniques, and objectives to rigorously test an organization’s security measures.
• Developing Operational Data Management Systems: Our Red Teaming includes assessing data risks, such as intellectual property extraction, copyright violations, PII/sensitive data recovery, and training data reconstruction.
• Incorporating Cybersecurity and Information Assurance: We emphasize a holistic approach to Red Teaming in four areas: model evaluation, implementation testing, infrastructure assessment, and runtime behavior analysis, ensuring comprehensive security throughout the AI lifecycle.

Conclusion: Securing the Future of Warfighting

The integration of AI into military operations is inevitable and necessary for maintaining a competitive edge. However, this advancement demands a robust and proactive cybersecurity posture. BreakPoint Labs focuses on understanding the unique vulnerabilities of AI systems, especially LLMs, RAG, and MCP, and implementing comprehensive threat assessment and mitigation strategies. Our goal is to empower organizations to evolve into an AI-enabled force, well-prepared to tackle future conflicts with greater readiness and effectiveness. This commitment to responsible and innovative AI use is a testament to a dedication to mission success.

By understanding the specific AI-centric threats/weaknesses and by continuously testing and hardening AI systems against these attacks, military forces and any organization leveraging AI in critical applications can ensure their AI provides a decisive edge rather than an exploitable vulnerability. In the age of autonomous systems and machine-speed warfare, securing AI is not just a technical challenge; it’s a mission imperative. A hardened AI advantage isn’t built by chance. It’s built by design. Organizations that adopt a threat-informed, resilience-focused approach to AI security today will be the ones best positioned to lead tomorrow. We’re committed to helping them succeed at BreakPoint Labs.

In our previous posts, we introduced the foundations of cATO and explored the pivotal role of Continuous Monitoring (ConMon) in keeping federal systems secure and compliant. Now, we’re focusing on another crucial element of cATO: Active Cyber Defense (ACD). ACD is a proactive strategy designed to help agencies like the Department of Defense (DoD) and the Army combat increasingly sophisticated cyber threats.

What is cATO, and Why Does It Matter?

If you’re in the federal or defense world, you know the importance of an Authorization to Operate (ATO). It’s essentially a green light for systems to run, confirming they meet required security standards. However, the traditional ATO process is slow and rigid. You might secure approval today, but your system could face new vulnerabilities six months later. This is where cATO comes in.

cATO ensures systems maintain continuous compliance, allowing for faster deployment of technologies without sacrificing security. Unlike traditional ATOs, cATO emphasizes real-time risk assessments, continuous monitoring, and adaptive security measures, aligning with agile development practices.

The ACD-cATO Connection

ACD is a critical requirement for achieving and maintaining cATO for the DoD and its components. To qualify, organizations must demonstrate three key competencies:

1. Continuous monitoring of Risk Management Framework (RMF) controls
2. Active cyber defense (ACD) to respond dynamically to threats
3. Use of an approved DevSecOps reference design for secure software development and supply chain management

What is Active Cyber Defense (ACD)?

Think of ACD as moving from a “set it and forget it” strategy to one that is proactive, adaptive, and predictive. It’s not just about deploying tools—it’s about staying vigilant, hunting for threats, and even countering adversaries when necessary.

ACD aligns with cATO’s focus on continuous compliance. By adopting strategies outlined in NIST SP 800-53 and leveraging methodologies like the MITRE ATT&CK Framework, agencies can build robust defenses tailored to their mission requirements and threat landscapes.

Key Components of ACD

ACD relies on actionable intelligence to predict, identify, and counteract threats. To effectively implement the ACD pillar of cATO, several key components must be put in place.

Let’s break it down:

1. Threat Intelligence and Analysis

• Indicators of Compromise (IOCs): Unique identifiers like IP addresses or file signatures tied to known threats.
• Behavioral Analytics: Using machine learning to analyze unusual patterns in network traffic and flag potential risks.
• Collaborative Intelligence: Sharing threat data with other agencies, as outlined by CISA Cybersecurity Guidelines, to strengthen collective defenses.

2. Automated Threat Response: When a threat is detected, speed is key. This involves having robust mechanisms for detecting and mitigating threats as they emerge rather than relying on periodic scans or updates. The DoD memo on cATO emphasizes that “systems must be able to show a real, or near real-time ability to deploy appropriate countermeasures to thwart cyber adversaries” 1,2.

3. Hunt Teams and Incident Response: Automation doesn’t replace the need for skilled personnel. Hunt teams are the boots on the ground, proactively looking for signs of malicious activity and taking action to stop it. They analyze threat data, assess risks, and respond to incidents, guided by the latest intelligence. Agencies can quickly contain and recover from breaches with a well-defined incident response plan.

4. Deception Technology: Deception technology is a fascinating piece of the ACD puzzle. It creates decoys or traps within the network, drawing attackers away from real assets. By luring attackers into a controlled environment, agencies can gather intelligence on their tactics and buy themselves extra time to respond.

5. Continuous Monitoring and Real-Time Analytics: ACD must be integrated with the Continuous Monitoring (ConMon) pillar of cATO. This integration ensures that security data collected from various environments (development, test, and production) is used to inform and power active incident response. The flow of information between these systems is crucial for making real-time risk decisions and responding to threats efficiently.

The Army’s cATO Initiative

A recent SIGNAL media article by Evan Lynch highlights the U.S. Army’s implementation of the cATO framework, marking a significant shift in how the Army monitors and addresses cybersecurity risks. This initiative aligns closely with the principles of ACD we’ve been discussing and offers valuable insights for organizations implementing ACD and working towards cATO.

Key aspects of the Army’s cATO framework include:

1. Continuous Risk Monitoring: The Army’s cATO initiative leverages software and artificial intelligence to constantly observe and analyze cybersecurity risks, moving beyond periodic assessments.
2. Maturity-Based Evaluation: Instead of a checklist approach, the Army assesses the maturity of systems to determine their readiness for cATO.
3. Efficiency and Agility: By adopting cATO, the Army aims to streamline operations, allowing for quicker task completion and reducing administrative burdens.

The Army is leading by example with its adoption of cATO. Two of the key systems under this initiative include:

1. Nett Warrior: A tactical tool connecting field leaders to real-time network data for decision-making in combat.
2. Gabriel Nimbus: A big data platform facilitating analytics and multi-domain operations.

Leonel Garciga, the Army CIO, states this development marks a major shift in how Army officials monitor and address threats. He explained, “It’s really about understanding the risk of the software you’re delivering as opposed to our more compliance-based culture that we have today… We’re really focused on the threat-based understanding of what the environment looks like and making decisions on software we’re building based on the risks that we’re creating with that software.” 4

Best Practices for Implementing ACD

To earn and maintain cATO, organizations need to demonstrate they can handle threats as they emerge—not just during a biannual review.

• Integration with Cybersecurity Service Providers (CSSPs): A critical capability for effective ACD includes integrating CSSP sensors and tools to detect vulnerabilities and provide real-time threat intelligence. Select tools that integrate well with your existing infrastructure and provide real-time threat intelligence.
• Conduct Regular Assessments: Use external penetration tests and red team exercises to identify and remediate weaknesses.
• Maintain Security Testing and Documentation: Track system changes and assess their impact on the defense posture, ensuring continuous compliance.
• Incident Response Plan: A detailed incident response plan is key for effective ACD. This plan should include policies, procedures, defined roles, and training for all personnel involved in incident response.
• Provide Comprehensive Training: Ensure your team is trained on ACD principles and practices, including using security automation tools and incident response procedures.

Takeaways

We’re excited about the potential of Active Cyber Defense in cATO. ACD is an essential capability for achieving and maintaining cATO. It’s the linchpin of a security strategy that’s as dynamic as the threats it faces. The DoD is actively putting these principles into action, demonstrating that the concepts of ACD and cATO are not just theoretical but are actively being implemented at the highest levels of defense.

We see it as a game-changer for our clients to effectively integrate cutting-edge threat intelligence, advanced virtualization technologies, automated orchestration tools, and cloud-based security solutions within DevSecOps platforms. All of these come together to create a robust, dynamic defense system that’s always on, always learning, and always one step ahead of the bad guys.

If you’re interested in learning more about Continuous Authorization to Operate (cATO) and how to implement it effectively for your organization, contact BreakPoint Labs. Our team of experts can guide you through the process and help you develop a robust cybersecurity strategy tailored to your specific needs.

References

1 GovCIO Media & Research. (2022, February 4). DOD Releases New Continuous ATO Initiative for ‘Active’ Cybersecurity. https://govciomedia.com/dod-releases-new-continuous-ato-initiative-for-active-cybersecurity/

2 U.S. Department of Defense. (2022, February 3). Continuous Authorization To Operate (cATO). https://media.defense.gov/2022/Feb/03/2002932852/-1/-1/0/CONTINUOUS-AUTHORIZATION-TO-OPERATE.PDF

3 FedTech Magazine. (2022, September 9). ATO to cATO Cybersecurity Transition in The Federal Government. https://fedtechmagazine.com/article/2022/09/understanding-transition-authorization-operate-continuous-ato-perfcon

4. Lynch, E. (2024, May 28). U.S. Army Officials Launch New Way to Constantly Monitor Risks. SIGNAL Media. https://www.afcea.org/signal-media/cyber-edge/us-army-officials-launch-new-way-constantly-monitor-risks