Randomized field experiment reveals that Google AI insights reduce organic clicks to external websites by 38% on queries where they appear, while self-reported search satisfaction remains almost unchanged when summaries are removed.
THE working document by researchers at the Indian School of Business and Carnegie Mellon University was published on SSRN this month. Authors Saharsh Agarwal and Ananya Sen describe it as the first randomized field experiment to test how AI insights affect user behavior in a real-world browsing environment.
How the experiment worked
Agarwal and Sen created a Chrome extension that randomly assigned 1,065 American participants to one of three groups. People were recruited from Prolific and used Chrome on desktop. They also had to meet minimum browsing history thresholds, so the sample reflected active desktop Chrome users rather than all Google users.
The control group viewed Google Search normally. A “Hide AIO” group asked the extension to remove real-time AI previews. A third group was redirected to Google’s AI mode for all searches. The study took place over two weeks per participant between January and February 2026.
The researchers pre-recorded the experiment with the AEA ECR Registry before data collection. More than 95% of users in the Hide AIO group detected no changes during the study.
What the researchers discovered
AI previews appeared on 42% of queries, and removing them increased outbound clicks from 0.38 to 0.61 per search. They reduced outbound organic clicks by 38% on triggered queries, with zero-click search increasing from 54% to 72%.
The effects were strongest when AI previews appeared at the top of the page, which occurred 85% of the time. Removing AI previews at the top almost doubled outbound clicks, but lower clicks had no effect.
Sponsored clicks and search frequency remained stable, indicating a substitution between AI previews and organic visits.
Discovering the user experience
The final survey used a Likert scale of 1 to 5 to rate participants’ research experience. Responses from the control and Hide AIO groups were nearly identical across all measures, including satisfaction, information quality, and ease of finding information.
The researchers wrote that the insights into AI “diverting traffic away from publishers without providing measurable improvements to the user experience.»
AI mode comparison
Participants directed to AI mode had lower outbound click rates, higher zero click rates, and lower final satisfaction compared to other groups.
The authors note that these results are exploratory, as higher attrition, some uninstallation of the extension, or finding workarounds may have influenced the results.
Why it matters
Independent measures of the impact of AI insights on traffic have been mostly correlational. Research on the bench We found that users click 8% of the time with AI previews, compared to 15% without. Ahrefs analyzed GSC data and reported a 58% drop in click-through rate for top-ranking pages when the AI insights appeared.
This experiment adds a different approach by randomly assigning users whether or not to see the AI previews, thereby isolating the causal effect.
Liz Reid, vice president of Google complaints AI insights reduce “bouncing clicks” but provide no data to support user benefit. Agarwal and Sen’s paper tested a related question with a randomized design, finding no measurable change in satisfaction or ease of information seeking.
Looking to the future
The document is a draft on the SSRN and has not been peer-reviewed. The authors will add more results and we will provide an update if the results change.
