Google has moved “computer use” from a specialized model to Google Gemini 3.5 Flash, making agent-style control of browsers, applications and desktop workflows a built-in feature rather than a separate product. This means that Gemini can now see and interact with user interfaces, reason about what’s on a computer screen, and take direct action. A senior scientist at Google DeepMind recently warned that large-scale AI agents create incentives “for bad people to do bad things.”
Developers can now create agents that do more than just call APIs. They can automate GUI-only workflows, such as testing software, filling out forms, navigating dashboards, or using legacy applications without API access. This reduces bottlenecks for automation and expands what AI agents can realistically do in production.
If the software has a graphical user interface (GUI) but no API, an AI agent can still use it. Agents can be asked to log into a dashboard, export yesterday’s SEO reports to a spreadsheet, compare them with last week’s data, and email a summary to the user. The workflow is managed in natural language instead of relying on custom scripts to connect the dashboard, spreadsheet, and email.
What this means for SEO
SEO tools could become much more agentic in the near future. Instead of just surfacing data, AI could connect to Google Search Console, audit sites, crawl a site with Screaming Frog, extract specific data points for comparison, and run repetitive optimization workflows.
For site owners, this also implies that another set of AI agents can act as “visitors,” which could affect how site owners interpret site interactions and engagement signals for site and sales optimization.
AI agents will be attacked
Google announcement is pretty optimistic, but the “security best practices” document it links to is worth paying attention to, because failing to get this part right can lead to theft and other poor user experiences.
THE document explain :
“Computer use presents unique security and operational risks because a model acting on behalf of a user may encounter untrusted content on screens or make errors when performing actions.”
This “unreliable content on screens” may refer to the “traps” set for AI agents that Google DeepMind’s senior scientist has warned about.
Google recommends seven best practices for this new AI agent:
1. Human in the Loop (HITL):
Enforce user confirmation: When the security response says require_confirmation (or a legacy security decision requires it), ask the user for approval.
Provide custom security instructions: Implement a custom system instruction to define and enforce your own security limits.2. Secure execution environment:
Run your agent in a secure environment and in sandbox mode to limit its potential impact. This could be a sandboxed virtual machine (VM), a container (e.g. Docker), or a dedicated browser profile with limited permissions.3. Disinfection of entrances:
Clean up all user-generated text in prompts to mitigate the risk of unintended instructions or rapid injection. This is a useful layer of security, but does not replace a secure execution environment.4. Content guardrails:
Use content security guardrails and APIs to evaluate user input, tool input and output, and agent responses for relevance, rapid injection, and jailbreak detection.5. Allowed lists and blocked lists:
Implement filtering mechanisms to control where the model can navigate and what it can do. A blacklist of banned websites is a good place to start, while a more restrictive green list is even more secure.6. Observability and logging:
Maintain detailed logs for debugging, auditing, and incident response. Your client should record prompts, screenshots, template-suggested actions (function_call), security responses, and any actions ultimately performed by the client.7. Environmental management:
Make sure the GUI environment is consistent. Unexpected pop-ups, notifications, or layout changes can disrupt the template. If possible, start from a known and clean state for each new task.
Beware of websites full of traps
As attack surfaces increase, the more likely it is that attackers will seek to exploit them. This means that as the number of AI agents on the web proliferates, hackers will focus on exploiting them. Websites become the battlefield from which attackers launch attacks against AI agents.
A senior scientist at Google DeepMind recently stated that Bad actors are already setting traps to steal money from humans by targeting their AI agents.
This is not an exaggeration. This month, a cybersecurity expert in California suffered illicit credit card charges from Anthropic’s AI agent Claude. According to the article, he appears to have downloaded a Skills.md file that may contain an AI agent trap.
The article reports:
“…he found a problematic add-on connected to Claude called a “skill”, similar to a plug-in. “It basically told Claude to try to purchase different types of gift accounts on my stored information. It was therefore by using the digital wallet which was on my computer that Claude began to make these purchases…”
Site owners may need tighter controls on bots and the ability to identify hackers who have hidden quick injection instructions on their sites. But that’s not something website owners are looking for, making the problem worse for users who use AI agents like the one Google just released.
Learn more: Google DeepMind: AI agent traps are already stealing money
Featured image by Shutterstock/blocberry
