AI Coding Assistants Chase Phantoms and Destroy Real User Data Through Simple File-System Traps

The intersection of generative artificial intelligence and software engineering continues to present unexpected risks, highlighting how mundane operating system behaviors can combine with agentic hallucinations to produce catastrophic data loss. In July 2025, a routine session involving the Gemini CLI software development tool turned destructive when an autonomous coding assistant, tasked with organizing a directory of experimental files, quietly obliterated critical data. The incident—documented in GitHub issue #4586 and subsequently analyzed by security and technology researchers—demonstrates a fundamental vulnerability in how Large Language Model (LLM) agents interact with local file systems. Rather than executing a complex, exotic exploit, the failure stemmed from a classic command-line quirk: a failed directory creation command paired with a traditional file-moving utility that silently renames targets when destination folders do not exist.

The Chronology of an Automated Disaster

The sequence of events began on July 21, 2025, when a product manager initiated a session using Gemini CLI version 0.1.13, powered by the gemini-2.5-pro model. Operating on a Windows environment without an active sandboxing barrier, the user issued a straightforward request: rename the working directory from claude-code-experiments to AI CLI experiments and transfer its contents into a newly established folder.

Recognizing its technical limitations, the AI agent correctly noted that it could not rename the active directory in which it was currently executing. To circumvent this, the model attempted to create a new directory one level up in the file hierarchy using a standard mkdir command. However, this preliminary command failed silently.

Crucially, the Gemini CLI agent did not verify the success of the directory creation. Instead, the model hallucinated a successful system state and proceeded to execute individual move commands for each file within the directory. In Windows command-line environments (cmd.exe), executing a move operation when the destination directory does not exist triggers a legacy behavior: instead of transferring the source file into a new folder, the operating system renames the source file to match the missing destination name.

Consequently, the first file transferred during the operation was renamed to match the intended folder title. As subsequent move operations were executed for the remaining files, each new item was written directly over the previous one. By the end of the automated sequence, multiple distinct experiment files had been sequentially overwritten, leaving only the final file in place of the entire directory structure. The session concluded with a stark, apologetic message from the agent: "I have failed you completely and catastrophically."

Understanding the Underlying Operating System Mechanics

The mechanics behind the incident are not unique to Windows, nor are they exclusive to generative AI tools; rather, they reflect decades-old operating system design decisions that assume human operators are actively monitoring terminal outputs.

In Microsoft’s cmd.exe and standard batch script environments, running a move command toward a non-existent path without explicit error-checking creates a specific overwriting hazard. When executed from within a batch file, the command-line interpreter bypasses standard interactive confirmation prompts, executing the overwrite automatically. While subsequent updates to the Gemini CLI shifted execution pathways to PowerShell—which handles missing target directories with an error rather than a silent overwrite—the fundamental vulnerability remains embedded in numerous command-line interfaces.

A parallel risk exists within POSIX-compliant environments utilizing standard Unix and Linux shell utilities. The traditional mv utility behaves similarly when invoked within loops or iterative scripts. If an agent or a developer iterates through a list of files, executing individual move instructions targeting a non-existent destination directory, the system executes a rename operation on the initial file. Subsequent iterations attempt to move the next file to the same destination path, triggering overwrites depending on local flag configurations.

While manual command-line users typically notice immediate errors or review terminal output, autonomous agents lack an innate contextual awareness of filesystem anomalies unless explicitly programmed with verification steps. When an agent encounters an unverified command failure, it often relies on parametric probability to assume the operation succeeded, compounding the error with subsequent tool calls.

Industry Implications for Agentic Software Development

The July 2025 incident highlights broader architectural challenges facing the rapidly growing market of autonomous coding assistants. As software development workflows increasingly incorporate agents capable of executing shell commands, reading codebases, and modifying local environments, the margin for error narrows significantly.

Software engineers and product security specialists have increasingly emphasized that autonomous agents operate without a theory of mind regarding physical data loss. Unlike human programmers, who possess an intuitive understanding of permanence and file-system hierarchies, LLM agents treat file paths and terminal outputs as linguistic tokens to be manipulated based on training data. When a tool call fails, the model’s objective function encourages it to press forward toward task completion rather than pausing to diagnose systemic failures.

Furthermore, the lifecycle of the software involved underscores the rapid evolution and retirement cycles within the AI tooling ecosystem. Records indicate that Google subsequently replaced the Gemini CLI framework with alternative tooling, such as the Antigravity CLI, by mid-2026. However, the foundational interaction patterns between LLM orchestrators and local shells remain largely unchanged across various third-party coding agents.

Mitigation Strategies and Defensive Engineering

To prevent similar catastrophic data loss events, security architects and tool builders advocate for a multi-layered defense strategy that does not rely on an AI model’s ability to self-correct or notice subtle terminal outputs.

First, enforcing strict script execution controls is critical. Incorporating strict error handling—such as utilizing set -eu in Bash environments or implementing explicit conditional logic to check error levels in Windows batch scripts—ensures that script execution halts immediately upon the failure of a prerequisite command like mkdir.

Second, developers should mandate the use of explicit flags and structured commands designed to prevent ambiguity. For example, utilizing GNU mv with the target directory flag (-t) forces the utility to treat the destination strictly as a directory, triggering an immediate error if the path does not exist rather than defaulting to a rename-and-overwrite operation. Similarly, adopting flags that diagnose and reject silent overwrites prevents iterative destructive loops.

Third, engineering teams emphasize the necessity of programmatic read-after-write verification. Autonomous agents should be explicitly instructed to verify the success of file-system operations by listing destination directories, comparing file counts, and validating checksums before reporting task completion to the user.

Finally, relying on automated checkpointing and version control systems provides a vital safety net. Modern development environments and agent frameworks increasingly integrate automatic snapshotting—utilizing Git worktrees, shadow repositories, or specialized IDE review desks—to ensure that every agent turn can be cleanly reversed. By maintaining isolated task branches and recording turn-by-turn diffs, developers can mitigate the risks associated with autonomous tool execution, transforming potential data disasters into easily reversible routine operations.