AI & Developer Tools
What are the security risks of AI coding assistants?
AI coding assistants introduce security risks including data leakage to model providers, insecure code generation, package hallucination (where AI suggests non-existent packages that attackers then register), unvetted MCP server connections, and autonomous command execution on developer workstations. With 84% of developers using AI coding tools, these risks affect virtually every engineering organization and operate below the visibility of EDR and CI/CD security tools.
What Is / Definition
AI coding assistant security risks are the set of threats introduced when developers use AI-powered tools like GitHub Copilot, Claude Code, Cursor, Windsurf, and similar assistants for software development. These risks go beyond the well-known concern about AI generating insecure code — they include data exfiltration, supply chain manipulation, autonomous execution, and uncontrolled external service connections.
The adoption curve makes this urgent. Eighty-four percent of developers now use AI coding tools. GitHub Copilot users complete 126% more projects per week. Microsoft and Google report 25% of their code is AI-assisted. This is not a fringe practice — it is the default mode of software development.
The risk categories span five areas:
Data leakage. AI coding assistants send code context, file contents, and environment information to model providers for inference. When developers use personal API keys rather than enterprise accounts, this data flows outside organizational governance. Even with enterprise accounts, the scope of what the model ingests — including adjacent files, terminal output, and environment variables — is broader than most security teams realize.
Package hallucination. AI models sometimes suggest packages that do not exist. Researchers have demonstrated that attackers can identify these hallucinated package names and register them on npm or PyPI, creating real packages with malicious payloads. When another developer follows the same AI suggestion, they install the attacker's package. With 3,180+ malicious packages confirmed on npm and PyPI in 2025, the supply chain is already primed for this attack.
Autonomous execution. Tools like Claude Code do not just suggest code — they execute commands, install packages, create files, and modify configurations directly on the developer's workstation. Default configurations are often permissive, and the AI agent operates with the developer's full system permissions.
MCP server connections. AI assistants using the Model Context Protocol can connect to external servers for additional context — databases, document stores, communication tools. With 13,000+ MCP servers on GitHub, developers can connect unvetted external services without IT visibility.
Prompt injection. Attackers embed instructions in repositories, documentation, or code comments that manipulate the AI agent's behavior — causing it to execute malicious commands, install compromised packages, or exfiltrate data.
Why It Matters
AI coding assistant security risks matter because they introduce a new class of threats that existing security infrastructure was not designed to detect. For CISOs and security leaders, the challenge is that blocking AI tools is not a viable option. The competitive pressure to adopt AI is real — organizations that restrict AI coding tools risk losing engineering talent and falling behind competitors. The question is not whether to allow AI coding assistants, but how to govern their use. The data leakage risk has immediate regulatory implications. If a developer using a personal API key sends proprietary source code to an AI model provider, that code has left the organization's control. For companies in regulated industries — finance, healthcare, energy — this can constitute a data breach even without malicious intent. Package hallucination is a supply chain risk that scales with AI adoption. A 2024 study found that commercial AI models hallucinate package names in approximately 5% of code suggestions. Across an organization with hundreds of developers making dozens of AI-assisted code changes daily, the probability of encountering a hallucinated (and potentially weaponized) package name is significant. The autonomous execution risk is the most novel. Previous generations of developer tools were passive — they analyzed code, highlighted errors, suggested completions. AI coding agents actively install packages, run commands, and modify the development environment. When Claude Code executes `npm install` to satisfy a dependency, it triggers the same supply chain risks as a manual install — but at machine speed, without the developer reviewing each package. MCP server connections compound all of these risks by extending the AI agent's reach beyond the local machine to external services. Each connection is a potential data exfiltration path that operates outside EDR and network security monitoring.
How It Works
AI coding assistant security risks operate through several distinct mechanisms, each exploiting a different aspect of how AI tools interact with development environments. Data flow to model providers. When a developer opens a file and asks an AI assistant for help, the tool sends context to the model API. This typically includes the current file, adjacent files, terminal output, and sometimes environment configuration. The data flows over HTTPS to the model provider's infrastructure. Enterprise plans offer data retention controls, but developers using personal API keys bypass these entirely. Security teams often have no visibility into which developers are using personal versus enterprise AI accounts. Package hallucination attack chain. The AI model generates code that includes an `import` or `require` statement referencing a package that does not exist in any registry. The developer (or the AI agent autonomously) attempts to install this package, gets a \"not found\" error, and moves on. An attacker who has identified common hallucinated package names registers them on npm or PyPI with malicious payloads. The next developer — or AI agent — that encounters the same hallucination successfully installs the attacker's package. The malicious code executes immediately via postinstall hooks. Autonomous command execution. AI coding agents like Claude Code operate in an agentic mode where they execute shell commands directly. The agent might run `pip install pandas`, `npm install express`, `brew install redis`, or any other command to complete a task. Each command runs with the developer's permissions. If the agent installs a compromised package, the malicious postinstall script has the same access as the developer — including SSH keys, cloud credentials, and CI/CD tokens. MCP server exploitation. An MCP server connected to an AI assistant has access to the conversation context and can inject instructions into the AI's responses. A malicious or compromised MCP server can manipulate the AI agent into executing commands, reading sensitive files, or installing specific packages. This is prompt injection via infrastructure rather than via text. Prompt injection via repositories. Hidden instructions in code comments, README files, or configuration files can manipulate AI coding assistants when they process those files as context. An attacker who can contribute to a repository (even via a seemingly benign pull request) can embed instructions that cause the AI to behave maliciously when a developer uses the AI assistant on that codebase.