The idea of artificial intelligence taking over jobs has been a hot topic for years, but in the world of cybersecurity, the shift is happening faster than many experts predicted. Recent findings from the UK’s AI Security Institute (AISI) show that advanced AI models are not just assisting human professionals, but are beginning to perform specific security tasks at a pace that rivals a trained expert. This progress raises a critical question: how exactly are these models getting better, and what does it mean for the humans who currently hold these roles? The answer lies in how researchers measure capability using an ai cybersecurity benchmark, and the results are startling.
Way 1: Accelerating the Doubling Period of Task Competence
The first and perhaps most dramatic way AI models are improving is through the sheer speed at which their capability is growing. Researchers track this growth by looking at how quickly the “doubling period” shrinks. A doubling period is the amount of time it takes for the model’s effective task length to double. In plain language, if an AI could handle a 10-minute task today, after one doubling period it could handle a 20-minute task.
In late 2024, AISI estimated this doubling period was about 8 months. By February 2026, that estimate had been cut nearly in half to 4.7 months. Then, with the arrival of newer models like Anthropic Mythos Preview and OpenAI GPT-5.5, the pace accelerated even further. The recalculated doubling time is now well under 4.7 months. This is not a slow, steady climb. It is a rapid acceleration that catches even the researchers off guard.
What This Means for Real-World Tasks
To visualize this, consider a simulated corporate network attack called “The Last Ones,” which involves 32 separate steps. When an older model, Opus 4.6, was tested in early 2026, it could only manage 22 of those 32 steps. The latest Mythos Preview checkpoint, however, solved the entire 32-step chain in six out of ten attempts. It also cracked a previously unsolved seven-step challenge targeting an industrial control system, known as “Cooling Tower,” in three out of ten tries.
This is not just about doing more steps. It is about handling more complex, multi-layered attacks that require reasoning, adaptation, and persistence. The doubling period is not just a number on a chart. It translates directly into the AI’s ability to take on longer and more sophisticated attack chains without human intervention.
The Role of Token Budgets in This Growth
A key detail in these benchmarks is the concept of a “token budget.” Tokens are the units of text or code that a language model processes. In the AISI tests, models are given a budget of 2.5 million tokens to complete a task. This is a significant but finite resource. If the token budget were unlimited, the AI might perform even better. The fact that models are improving within a fixed budget makes the acceleration even more impressive. They are learning to be more efficient with their resources, not just bigger.
Way 2: Narrowing the Gap Between Simulated and Real-World Environments
A common criticism of AI benchmarks is that they are too easy. They often take place in a controlled, simulated environment where the rules are clear and the variables are limited. The real world is messy. Networks have legacy systems, strange configurations, and unpredictable user behavior. The second way AI models are improving is by closing the gap between these simulated tests and actual, defended systems.
The ai cybersecurity benchmark used by AISI is explicitly narrow. It measures performance on specific security tasks, not general hacking ability. However, the progress is real. The models are not just memorizing patterns from a training set. They are learning to chain together multiple steps, such as reverse-engineering a Windows service binary to access encrypted credentials, escalating privileges through token impersonation, and recovering cryptographic keys to access command-and-control services.
A Cautionary Tale from the Curl Project
One real-world data point comes from the curl project, a widely used open-source tool for transferring data. Developers asked the latest Mythos model to find vulnerabilities in the curl codebase. The result was modest: the AI found just one confirmed vulnerability. This is a useful reminder that the models are not yet omniscient. They can fail on real-world code that is messy, undocumented, or requires deep contextual understanding that a human developer might have built over years.
Yet, even this single finding is significant. A few years ago, a model finding zero vulnerabilities was the norm. Now, it can find at least one. The trajectory suggests that as the models improve, their hit rate on real-world code will rise. The gap is narrowing, but it has not disappeared.
The 80 Percent Reliability Threshold and Its Risks
Why do researchers focus on the 80 percent reliability mark? This threshold is a practical balance. It is high enough to show meaningful capability, but low enough that the missing 20 percent is a serious concern. In cybersecurity, a 20 percent failure rate is not acceptable for critical tasks. A model that misses a vulnerability one in five times could lead to a catastrophic breach. This is why the human-in-the-loop model remains essential. The AI can handle the bulk of routine triage, but the human expert must review the edge cases and handle the failures.
The risk is that organizations, eager to cut costs, might rely too heavily on the AI and ignore the 20 percent gap. This would be a mistake. The benchmark is a tool for understanding capability, not a license to remove human oversight entirely.
Way 3: Converging Benchmarks Across Different Skillsets
The third way AI models are improving is through the convergence of different measurement frameworks. AISI is not the only organization tracking this progress. The nonprofit research house METR has been measuring AI performance on broader software engineering tasks. Their findings align closely with AISI’s cybersecurity-specific data.
METR found a consistent doubling time of about 4.2 months on software tasks since late 2024. With the latest Mythos Preview checkpoint, that number is closer to 4 months. The fact that two independent organizations, using different tests and different metrics, are arriving at similar conclusions is powerful evidence that the trend is real. It is not a fluke of a single benchmark design.
What This Convergence Means for Hiring and Training
For a cybersecurity manager at a mid-size company, this convergence has practical implications. If the AI can handle software engineering tasks at a similar pace to security tasks, then the tools available for automating incident response, patch management, and vulnerability scanning are likely to improve rapidly. A manager considering a pilot program for an AI triage tool can look at these converging benchmarks with more confidence. The AI is not a gimmick. It is a rapidly maturing tool.
You may also enjoy reading: Europe First to Authorize Moderna Combo mRNA Vaccine.
For a recent graduate considering a career in cybersecurity, the picture is more complex. The entry-level tasks that used to be the training ground for junior analysts, such as log review and basic threat hunting, are exactly the kinds of tasks AI is getting good at. This does not mean there are no jobs. It means the jobs are changing. A graduate will need to focus on skills that AI cannot yet replicate, such as strategic thinking, cross-team communication, and handling novel attack patterns that have no precedent in training data.
The Economic Ripple Effects
The accelerating pace of capability also has economic implications. If the doubling period continues to shrink, the cost of performing a specific security task with AI will drop far below the cost of employing a human. This could reshape industry hiring, pushing companies to invest more in AI infrastructure and less in large security teams. Venture capitalists funding AI startups are watching these numbers closely. A halving of doubling times signals a massive shift in what is possible, and where money should flow.
However, there is a counterpoint. The AISI itself warns that the benchmark does not tell us how the pace of progress will evolve, when AI will reach any particular capability threshold, or how these capabilities will translate against defended, real-world systems. The economic effects are real, but they are not guaranteed to follow a straight line.
What the Missing 20 Percent Means for Human Experts
Throughout this discussion, the 80 percent reliability figure appears repeatedly. It is important to understand what this number does and does not mean. It does not mean the AI is 80 percent as smart as a human. It means that for a specific task, within a specific time window, the AI matches the human outcome 80 percent of the time. The remaining 20 percent includes cases where the AI gets stuck, makes a wrong assumption, or misses a critical detail.
This is where human experts remain irreplaceable. The human role shifts from being the primary doer to being the supervisor, the validator, and the fallback. A security researcher who curates open-source projects like curl, for example, might use the AI to automate routine bug fixes and vulnerability scans. But the researcher still needs to review the AI’s output, test the fixes, and decide which findings are worth pursuing. The AI handles the grunt work. The human handles the judgment.
Token Limits as a Hidden Constraint
Another factor that affects the 80 percent figure is the token budget. The ai cybersecurity benchmark currently caps models at 2.5 million tokens. If that cap were removed, the AI might perform better on the hardest tasks, potentially raising its reliability above 80 percent. However, uncapped models also introduce new risks. They could consume enormous amounts of compute power, generate irrelevant output, or even go down rabbit holes that a human would avoid. The token cap is not just a limitation. It is also a safety measure.
As models improve, researchers will need to decide whether to raise the token budget, remove it entirely, or develop new ways to constrain AI behavior without limiting its effectiveness. This decision will have a direct impact on how quickly AI can replace human roles in cybersecurity.
Looking Ahead: The Hybrid Model of Cybersecurity
The evidence from AISI and METR paints a clear picture. AI models are getting better at replacing cybersecurity pros on specific, well-defined tasks. The pace of improvement is accelerating, with task length doubling on the order of months, not years. However, the ai cybersecurity benchmark also reveals the limits of current models. They are not general-purpose hackers. They are specialized tools that excel in narrow domains.
The most likely future is a hybrid model. AI handles the bulk of routine triage, vulnerability scanning, and basic incident response. Human experts focus on complex attacks, strategic planning, and the 20 percent of cases where the AI falls short. This hybrid model is not a compromise. It is a force multiplier. It allows human experts to do more with less, and it gives organizations a level of security that would be impossible with either humans or AI alone.
The key for professionals in the field is to stay informed about these benchmarks. Understanding the ai cybersecurity benchmark and what it measures is the first step in adapting to a changing landscape. The models are improving. The question is no longer if they will take on more cybersecurity work, but how quickly, and how we prepare for the shift.
