AI companies have introduced strict guardrails and verification programs to prevent their models from being used for cyberattacks. However, many cybersecurity researchers argue these restrictions are making it harder for legitimate security professionals to identify and fix software vulnerabilities before attackers exploit them.
The debate intensified after the U.S. imposed export restrictions on Anthropic’s AI models, Mythos and Fable, following reports that their security safeguards could be bypassed. While some restrictions have since been eased, Anthropic continues to limit access to certain advanced models through a verification process. Similar programs are also offered by OpenAI, allowing approved cybersecurity researchers to access models with fewer restrictions.
Many offensive security researchers believe these safeguards are too restrictive. They argue that the same AI capabilities used to identify security flaws can also help developers fix them, making it difficult to separate defensive and offensive use cases.
Security researcher Mark Dowd criticized AI companies for deciding what security research should or should not be allowed, saying such decisions are often arbitrary. Dowd, known for discovering and selling zero-day vulnerabilities to government agencies, believes these restrictions limit valuable cybersecurity research.
Chris Anley, Chief Scientist at NCC Group, explained that AI plays an important role in validating whether a discovered software flaw can actually be exploited. If a model refuses to assist because of built-in safeguards, researchers lose an important tool for confirming vulnerabilities before they can be patched.
According to Anley, cybersecurity tools naturally serve both offensive and defensive purposes. He compared AI to a hammer, noting that while it can be used to build something useful, it can also be misused. Because of these limitations, some researchers turn to open-source AI models that have few or no usage restrictions.
Paolo Stagno, Chief Technology Officer at Crowdfense, also criticized the strict controls, arguing that AI companies often treat security professionals as though they require constant supervision. While his team uses commercial AI models for reverse engineering software, they rely on locally hosted open-source models for vulnerability research to avoid exposing sensitive information to cloud-based systems.
Other researchers take a different approach. Security researcher Giuseppe Cali said AI is useful for understanding code and building supporting tools, but he still prefers to discover and develop exploits manually. He believes the core process of finding vulnerabilities remains a task best handled by experienced researchers.
Some organizations say the restrictions significantly reduce the usefulness of commercial AI models. One anonymous researcher at a smartphone component manufacturer said the models frequently refuse to respond whenever they detect security-related requests, making them ineffective for vulnerability research.
Chris Thompson, CEO of cybersecurity firm RemoteThreat and founder of Offensive AI Con, said researchers often spend more time working around inconsistent AI guardrails than conducting actual security research. He noted that responses can vary from one day to the next, even within approved access programs.
As a result, many cybersecurity professionals are increasingly adopting open-source AI models, including Chinese-developed models such as GLM, which can run locally without approval processes or usage restrictions.
Thompson warned that overly restrictive policies could drive legitimate researchers away from AI systems developed by U.S. companies and toward unrestricted foreign alternatives. Instead of imposing tighter controls, he believes AI developers should expand responsible access programs while holding users accountable for misuse.
According to researchers, as AI-powered cyber threats continue to grow in speed and sophistication, providing legitimate security professionals with effective AI tools will become increasingly important for defending digital infrastructure.
