Annual Security Tests Cannot Keep Pace With AI

The Decision in Front of You

Most companies test their security once a year: an outside penetration test, an audit sample, and a remediation plan that runs into the next quarter. The decision is whether that cycle still matches the speed of the people attacking you.

Mandiant's M-Trends 2026 report, drawn from its 2025 investigations, put the mean time to exploit a vulnerability at an estimated negative seven days, meaning exploitation typically began before a fix was available ¹. Exploits were the most common way into a network for the sixth year running, at 32% of intrusions ¹. VulnCheck found that 28.96% of the vulnerabilities it confirmed as exploited in 2025 were attacked on or before the day they were publicly disclosed, up from 23.6% in 2024 ².

Defenders are moving the other way. Coverage of Verizon's 2026 Data Breach Investigations Report says exploitation of vulnerabilities was the initial access route in 31% of breaches, that organizations fully fixed only 26% of the critical flaws on CISA's known-exploited list, and that the median time to fix rose from 32 to 43 days ³.

What's Happening

AI has shortened every step of an attack

In mid-September 2025 Anthropic detected a campaign it attributed with high confidence to a Chinese state-sponsored group, tracked as GTG-1002 ⁴. The attackers pointed Anthropic's Claude Code tool at roughly 30 targets, including technology firms, banks, chemical manufacturers, and government agencies, and succeeded in a small number of cases ⁴. The AI did an estimated 80% to 90% of the work, making thousands of requests, often several per second ⁴.

Malware now calls AI models mid-attack. Google's Threat Intelligence Group reported in November 2025 that PROMPTSTEAL, used by the Russian government-backed group APT28 against Ukraine, asks a language model to write the commands it runs on victim machines, the first such malware Google had seen in live operations ⁵.

Vulnerability discovery has changed most. In April 2026 Anthropic said an unreleased model, Claude Mythos Preview, had found thousands of high-severity vulnerabilities, including a 27-year-old flaw in OpenBSD, and warned that such capabilities would soon spread beyond defenders ⁶. CrowdStrike's 2026 Threat Hunting Report says frontier AI is collapsing exploit windows and found China-nexus groups exploiting flaws 24 hours after a working proof of concept appeared ⁷.

The trend predates these tools. Mandiant put the average time to exploit at 63 days in 2018 and 2019 and 5 days in 2023 ⁹. Rapid7 counted 146 exploited high- and critical-severity vulnerabilities in 2025, double the 71 of 2024 ⁸.

Four cases from 2024 and 2025

GTG-1002, 2025. The campaign above is the first documented case of an AI agent doing most of an intrusion's technical work ⁴. No cost has been published. A few operators ran reconnaissance and exploitation against 30 organizations in parallel, with people stepping in at four to six decisions per campaign ⁴. The agent sometimes invented credentials, which Anthropic called an obstacle to fully autonomous attacks ⁴. The lesson is pace: one small team now works at the scale of many.

Arup, 2024. An employee in the Hong Kong office of the engineering firm Arup joined a video call with what appeared to be the company's chief financial officer and other colleagues ¹⁰. All were deepfakes. He made 15 transfers totaling HK$200 million, about US$25.6 million, before checking with the UK head office ¹⁰. The approval steps in a payment process are attack surface and need testing like any system.

Nx "s1ngularity," August 2025. An attacker exploited a flaw in an automated build workflow of Nx, a popular open-source build tool, stole its publishing token, and released malicious versions to the npm registry on August 26, 2025 ¹¹. Installing them ran a script that collected credentials and posted them to a public repository in the victim's own GitHub account ¹¹. The script also ran the developer's AI coding tools, Claude Code, Gemini CLI, and Amazon Q, with flags that skip permission prompts, to hunt for secrets ¹². A build pipeline flaw is as exploitable as a product flaw, and AI tools widen what a bad package can reach.

Package hallucination, 2025. Researchers at the 2025 USENIX Security Symposium generated 576,000 code samples with 16 models and found that at least 5.2% of packages suggested by commercial models, and 21.7% by open-source models, did not exist ¹³. An attacker who publishes malware under an invented name can wait for an AI assistant to install it, a technique called slopsquatting. Code scanning has to confirm every dependency is real and approved.

Why the Usual Approach Fails

An annual penetration test is a snapshot. It tells you what a skilled team found during two or three weeks of one year. A flaw introduced the week after the test can stay invisible for eleven months, and the attackers described above need days.

Consider an illustration, the author's estimate. A company's internet-facing systems pick up one serious exploitable flaw a week, each fixed 30 days after discovery. If only the annual test finds them, the average flaw stays open about 212 days, or 11,050 days of exposure across 52 flaws. Quarterly scanning cuts the average to about 76 days. Weekly scanning with a seven-day fix target brings it to 10.5 days and the total to 546, about one-twentieth of the annual figure.

Several habits widen the gap.

Fixes are ranked by severity score, though many flaws rated critical are never attacked. CISA's Known Exploited Vulnerabilities catalog, usually called KEV, lists flaws confirmed as attacked in the wild, and CISA recommends that every organization use it as an input to patching priority ¹⁴. It held 1,766 entries at the time of writing ¹⁴. In June 2026 CISA replaced its earlier directive, BOD 22-01, with BOD 26-04, which sets federal agency deadlines by four factors: whether the system is exposed to the internet, whether the flaw is on the KEV list, whether exploitation can be automated, and how much control an attacker would gain ¹⁵. The tightest tier requires a fix within three days plus a check for signs the system was already compromised ¹⁵. The directive binds only federal civilian agencies, but its logic applies to any company.

The inventory is incomplete. A test covers only listed systems, and a forgotten site or a project team's cloud service is often missing. BOD 26-04 requires agencies to tag every publicly exposed asset for this reason ¹⁵.

AI is outside the scope. Test plans rarely include the chatbot connected to the customer database, the AI coding tools on developer laptops, or the packages those tools install.

The Framework: The Exposure Loop

The Exposure Loop treats security testing as a control that runs continuously. Each stage has an owner and a clock, and the loop restarts every time code, infrastructure, or a vendor changes.

flowchart LR
    A[Inventory<br/>exposed systems<br/>code and AI] --> B[Scan<br/>code, dependencies<br/>and live systems]
    B --> C[Prioritize<br/>by evidence<br/>of exploitation]
    C --> D[Fix<br/>on a clock<br/>set by exposure]
    D --> E[Verify<br/>retest and report<br/>to leadership]
    E --> A

Inventory. Keep a live list of everything reachable from the internet, every code repository, every third-party package, and every AI system, model, and agent in use. NIST's Cybersecurity Framework 2.0 starts with this: asset inventories in its Identify function, and supplier risk under a new Govern function ¹⁶.

Scan. Code and dependency scanning run on every change, before code ships. Active scanning probes exposed systems weekly from the outside, as an attacker would. Web applications are tested against OWASP's Application Security Verification Standard, version 5.0.0 since May 2025 ²¹, using its Web Security Testing Guide ²². AI applications are tested against the OWASP Top 10 for LLM Applications 2025, which starts with prompt injection and covers supply chain, poisoning, and excessive agency ²³.

Prioritize. Rank findings by evidence of attack. A flaw on the KEV list, on an exposed system, that can be exploited automatically goes first ¹⁴¹⁵. Severity scores break ties.

Fix. Set fix deadlines by tier and publish them. A reasonable private-sector version of the federal model is three days for exploited flaws on exposed systems, two weeks for other exploited flaws, and the normal release cycle for the rest. Where a fix is not available, remove the system from the internet or put a compensating control in front of it.

Verify. Retest every fix. A few times a year, commission human penetration tests of the most valuable systems and AI applications, aimed at what automation misses, such as business logic and approval flows. Report open exposure to leadership monthly.

Each stage maps to a standard auditors recognize

For AI, NIST's AI Risk Management Framework (AI 100-1) sets four functions, Govern, Map, Measure, and Manage ¹⁸, and its 2024 Generative AI Profile (AI 600-1) adds risks specific to generative systems ¹⁹. NIST AI 100-2 E2025 gives testers and auditors shared names for attacks such as poisoning and evasion ²⁰, and MITRE ATLAS catalogs real attacker tactics against AI systems ²⁴.

Regulators are asking for the same evidence

Public companies in the United States must disclose a material cybersecurity incident on Form 8-K within four business days of deciding it is material, and must describe in the annual 10-K how they assess and manage cyber risk and how the board oversees it ²⁸. A monthly exposure report gives the board accurate material for that filing.

For companies building or deploying AI in Europe, under the EU AI Act as amended on July 27, 2026, transparency obligations apply from August 2, 2026, and obligations for standalone high-risk systems move to December 2, 2027, with AI embedded in regulated products following on August 2, 2028 ²⁹. ISO/IEC 42001, published in 2023, gives companies a certifiable management system for AI that sits beside ISO/IEC 27001 ²⁶²⁷.

Costs, Risks, and Tradeoffs

A continuous program costs more than an annual test: licenses, weekly triage, and engineering time spent on fixes. The exposure model above shows what that buys.

How to Act

Questions to Ask Your Team

1. How many systems do we expose to the internet, and how do we know the list is complete?
2. How many flaws on CISA's exploited list are open on those systems today, and for how long?
3. What is our median time to fix an exploited flaw, and what deadline have we published?
4. Which AI applications and agents do we run, and when were they last tested for prompt injection?
5. Which AI coding tools do developers use, and what credentials can those tools reach?
6. Could a convincing video call from our CFO move money without a second check?
7. What would we write in our 10-K about how we manage cyber risk, and what evidence supports it?

Closing

Attackers now exploit weaknesses in days, sometimes before a fix exists, and AI is making them faster. A program that looks once a year leaves those weaknesses open for months. Fund continuous testing that scans every change, fixes exploited flaws on a published clock, tests AI systems like any other application, and reports open exposure to the board monthly. Mapped to NIST, OWASP, and ISO, the same work gives auditors and regulators the evidence they expect.