Anthropic tightens safeguards to prevent AI-powered phishing and data theft by hackers

The report, Detecting and countering misuse of AI: September 2026, details cyber operations identified and disrupted by Anthropic between December 2025 and August 2026, in which threat actors used the company's Claude AI models.

The actors included suspected state-sponsored groups, financially motivated criminals and politically motivated individuals.

Anthropic said AI's role in cyber operations is becoming increasingly autonomous, with Claude being used not only as an assistant but also to execute or coordinate parts of attacks.

In several cases, multi-agent AI systems carried out reconnaissance, exploitation and data exfiltration, while humans largely remained involved in selecting targets and reviewing stolen information.

One of the cases detailed in the report involved a Russian state-nexus espionage actor, identified by Anthropic as GTG-20006, which used AI-driven workflows to automate operations from infrastructure development and phishing to maintaining access to compromised systems and stealing data.

The group used a toolkit that included Windows-based implants, a mobile exploitation kit, a credential-stealing tool targeting passwords stored in browsers, and a phishing platform designed to imitate government organisations.

The report said the actor also used AI to drive its phishing operations.

They developed AI-driven workflows to research then register domains and then configure the hosting infrastructure used to send phishing emails, Anthropic said.

The workflows were also used to send phishing emails and monitor command-and-control channels for successful compromises. Anthropic identified more than 20 organisations targeted in the group's planning, reconnaissance and live operations.

These included government ministries, defence and intelligence bodies, embassies, diplomatic missions, think tanks and defence-industrial companies.

The report also highlighted financially motivated attacks in which Claude was used during large-scale data theft and supply-chain compromises.

In one attack on a software-as-a-service (SaaS) provider, hackers extracted data belonging to around 200 downstream customer organisations and obtained more than 2,100 Azure AD token sets spanning over 40 corporate tenants in about 34 hours. Anthropic said AI agents performed nearly all of the work.

In another SaaS compromise, an attacker exploited a cross-site scripting vulnerability, escalated privileges and eventually exfiltrated data from thousands of downstream customer organisations.

Claude was used to help understand developer and authentication APIs, create privileged tokens and build tools for bulk data exports and cross-tenant data collection.

Anthropic said the findings show that AI is allowing attackers to operate faster and across more targets with fewer resources, while increasingly taking over tasks that previously required skilled human operators.

It said, While this is a danger, the risk from AI adoption is more pronounced across the cyber kill chain, where adversaries can operate faster, across a broader and deeper surface area, with fewer resources.

Anthropic said it has taken steps to disrupt such malicious activity by banning accounts linked to threat actors and strengthening its systems to identify similar attacks in the future.

In one cyber operation, the company said it banned accounts associated with the actors and deployed additional monitoring to detect and ban related activity.

The company also said it uses findings from its investigations to improve safeguards against future misuse.

In another section, Anthropic said, We banned the accounts associated with these operations and are mapping their wider footprints, adding that it was tracking the actors digital signatures to prevent future misuse and incorporating the findings into new safeguards and model training. (ANI)