AI-Based vulnerability discovery in the age of Anthropic, Mythos, and others: A turning point or just a fleeting trend?
The use of AI-based systems is steadily increasing. This applies not only to businesses but also to individuals, threat actors, and researchers. Anthropic recently made a particularly strong impression with announcements surrounding its latest AI model, Mythos. “Too dangerous for the public,” the media headlined.
And it seemed to be coming true, as special meetings were arranged between Anthropic, Federal Reserve Chair Jerome Powell, Treasury Secretary Scott Bessent, and the CEOs of major U.S. banks to discuss potential threats.
The tool certainly seems efficient, as it finds 27-year-old security holes in OpenBSD, hundreds of vulnerabilities in Firefox, and countless bugs in the Linux kernel.
Does this now spell the end of software security in enterprises? Are we even facing a paradigm shift – or just a hyped-up trend?
Status Quo – how security vulnerabilities are discovered today
The discovery of bugs is almost as old as computer systems themselves. The first discovery dates back to 1947 – though it was likely an actual insect in the literal sense of the word.
Since then, the detection of such flaws has changed significantly. Today, we rely on traditional methods such as penetration testing, bug bounty programs, and static and/or dynamic code analysis. However, the verification of such vulnerabilities has so far always been carried out by humans.
Naturally, such methods eventually reach their limits. People generally do not scale linearly with increasing workloads, and more complex vulnerabilities require a great deal of time. Some security vulnerabilities may also arise solely due to the complexity of a company’s software landscape and require extensive troubleshooting.
The rise of AI-powered vulnerability discovery
This is where AI comes into play. AI models do what human resources cannot:
- Analyzing massive codebases? Done in seconds, minutes, or hours instead of days, weeks, or months.
- Code review at the semantic level? That’s no challenge for the tool either. Logical errors are detected alongside syntax errors.
- How can these vulnerabilities be exploited in practice? Mythos and Co. provide the right answer to this as well; even if conventional chatbots are reluctant, with a little persuasion, they’ll provide it right away.
AI is thus no longer just a toy but offers the potential for an explosive increase in the number of vulnerabilities found. Furthermore, searching for and discovering vulnerabilities is no longer the exclusive domain of trained professionals. AI makes discovery child’s play.
New risks: AI tools in the wrong hands
But this is precisely where the danger lies. While more discovered vulnerabilities mean more secure software in the long term, they also mean a larger attack surface for companies in the short term.
AI helps malicious actors generate exploits even faster. It also helps attackers.
The technical knowledge required drops significantly when programming skills are no longer needed to discover security vulnerabilities. Today, all it takes is writing a suitable prompt. Practical experience shows that this is already the case. Mass scans and attack automation, for example, were already used earlier this year in a campaign targeting numerous Fortinet firewalls. A suspected Russian-speaking attacker compromised hundreds of companies in the process—all without any technical expertise.
As an attacker, should I target the human vulnerability instead of systems? No problem. AI easily provides me with phishing emails. And all of this without spelling errors, featuring perfectly crafted lures.
So is the fragile balance between attackers and defenders now tipping?
Hype cycle: Is it a turning point or just hype?
The answer to this question is not clear-cut. As is so often the case, it lies somewhere in between.
The fact is – yes: AI models like Mythos are capable of discovering vulnerabilities faster and more effectively than most humans could. Vulnerabilities that will be exploited even faster and more frequently in the future. The barriers to expertise, money, and time are continuously lowering. This means more and more threat actors can take advantage of them. For malicious actors with advanced technical expertise (Advanced Persistent Threat Groups), on the other hand, it’s becoming easier to discover security vulnerabilities, move undetected within a network, or automate even larger campaigns.
However: AI is far from detecting all bugs, nor does it do so immediately. Even though many articles suggest that zero-days can be found “for just a few dollars,” this represents the best-case scenario. Generating reliable exploits usually requires more than one attempt, so the costs will likely run into the thousands rather than just $50. It should also be noted that zero-day vulnerabilities have been actively exploited for years, often just hours after their initial disclosure. This trend is not new. Even today, attackers are often faster than many patch cycles, as they do not have to account for the impact on production systems. So why resort to unknown security vulnerabilities when I can use exploits for already known ones?
Conclusion: A turning point, yes, but with caveats
There is no denying that a structural shift is taking place in the IT security landscape. However, this is not a complete upheaval. Rather, it is an acceleration of existing trends. AI merely changes the speed and scale at which new vulnerabilities are discovered and existing ones exploited, but not their fundamental principles. Similar to the introduction of static and dynamic code analysis, it should be understood as a new tool in the attacker’s arsenal.
Practical implications
What does this mean specifically for me and my company?
- Transparency as part of the IT strategy is gaining relevance: No company can effectively protect an asset it doesn’t know exists. The use of AI acts here more than ever as a catalyst. Not only are agents, chatbots, and the like assets themselves, but they also generate them at a high frequency.
Inventorying and, in particular, ensuring the up-to-date status of one’s own asset inventories is one of the core challenges of the future. - The war of AI models: Attackers are already deploying their new “toys” today, albeit with varying degrees of efficiency and effectiveness so far. To avoid being hopelessly outmatched in the arms race between attackers and defenders in the future, companies must address the question today of how AI can meaningfully improve their own information security processes.
We have already observed that known and unknown security vulnerabilities will be actively exploited even more rapidly and on a much larger scale in the future. In the future, therefore, the question will no longer be whether attackers will penetrate a company’s network, but how they can be isolated in a timely and, above all, effective manner to protect critical resources. In addition to serving as a tool for threat actors, AI can help address gaps in the detection of and response to such attacks. Anomaly detection already helps identify unusual network traffic and alert defenders. Extended Detection and Response (XDR) already supports the automated execution of measures to defend against cyberattacks. AI thus supports incident response teams in optimizing their own processes and workflows by building new tool chains.
AI attacks are currently still “in their infancy.” It is reasonable to assume that the complexity and frequency of attacks will increase until AI ultimately attacks external systems fully automatically and adapts attack patterns to defenders’ measures in real time. To be prepared for such attacks, companies will also need AI models for defense in the future, unless they wish to resort to disconnecting the entire company from the internet during every attack. - The human factor is gaining relevance in the age of AI: It may sound paradoxical, but people and their skills are now more relevant than ever before. They are simultaneously the main point of entry and the final decision-maker.
Humans as a risk: phishing, vishing, quishing. People remain a popular target today. Social engineering is becoming increasingly sophisticated. While at the start of the millennium it was emails asking for help for a “prince in distress,” today employees must be wary of AI-generated calls from their department head—should he have left enough voice samples on YouTube during his last symposium.
However, to ensure that staff do not immediately reveal all company secrets in such a scenario, they must be regularly trained on the existence of such attacks—both in theory and in practice. Even more important, however, is creating a corporate culture in which such attacks are reported.
Humans as the final decision-makers: As much as AI can already accomplish today, it is also sometimes unreliable. Probabilistic is simply not deterministic. In cases where AI cannot or should not be trusted, it is therefore essential that humans serve as the final decision-makers. This can only succeed, however, if staff can build and expand their expertise early on so they can make the correct decision in precisely those moments. In addition to security engineers, the role of the AI operator is therefore gaining relevance in the field of information security. - Secure AI adoption: But it’s not just external AI that can pose a risk within your own company. Your own agents, chatbots, and the like can also become weapons in the event of a cyberattack.
They often have extensive privileges, are powerful, and are rarely adequately monitored. The Adopt (A), Defender (D), Govern (G) framework helps companies secure the implementation.
In this context, a general distinction must be made between two types of AI: models that have been actively introduced into the company (Trusted AI) and those that have not undergone a formal approval process (Shadow AI).
Shadow AI encompasses all AI-based assets that have been introduced into the company intentionally (by employees or malicious actors) or unintentionally and must therefore be treated in the same way as vulnerabilities and insider threats, unless they can be transferred to a Trusted AI.
Author: Robin Enste