Adding FAQ sections to a set of test pages improved AI citations. Their removal caused citations to drop.
This reversion differentiates between correlation and causation, and almost no team measuring AI research today can produce it.
This standard of proof anchored the latest SEJ webinar with seoClarity’s Mark Traphagen, VP of Product Marketing and Training, Mihir Naik, Senior Product Manager, AI, and Suraj Lalchandani, Senior IT Project Manager. Their main point: “Viewability scores tell you if you showed up. Page-level performance and split testing tell you if what you did actually matters.”
The session presented the split testing methodology of seoClarity enterprise customers running on ChatGPT, Claude, Perplexity, Gemini and Google AI surfaces: how create a set of golden prompts covering the funnelhow to build a control group when LLMs don’t allow you to A/B test, and where the new Google Search Console AI data fits.
They also shared the results of three real customer tests, including the one change that moved the quotes and two results that no one in the room predicted.
Look at it full webinar on demand for the complete testing methodology.
Can you finally see AI search visibility in Google Search Console?
For a subset of sites, yes. On June 3, Google launched dedicated Search Console reports for AI Previews and AI Mode, showing page by page how often each URL appears in Google’s AI search features.
Lalchandani called it the biggest metric upgrade that AI research testing has received. “This has been the hardest thing to measure in AI search. Everyone was sampling. Everyone was inferring. But now Google is giving it to you.”
First-party data straight from the source carries a different level of trust than any third-party tool. But the team was blunt about the limitations: The new reports only cover part of what an AI research testing program needs, and ChatGPT, Claude, and Perplexity still require structured third-party monitoring.
During the session, the team maps exactly what gaps the new reports fill, which ones they leave open, and the platform-by-platform benchmark for what each AI engine can explore and render.
Action Item: Check Search Console for new AI reports, then see where first-party data matches your testing program before building around it.
Which prompts should you test first in AI search?
The ones where you’re almost winning. The team creates a set of golden prompts covering the full AI search funnel, from awareness to retention, with each prompt labeled by stage, then sorts each prompt into levels based on the brand’s current position in the AI response.
Level 1 prompts are the easy wins. As Lalchandani says: “You’re relevant, but the AI just hasn’t been given a URL to link to. »
Level 2 is the heaviest, and one set of prompts is removed from the tests entirely, a decision that surprised many participants.
The sequence is deliberate: early victories buy the political capital needed to conduct tougher tests later. The session covers how to create and label the golden prompt set, how the levels are defined and the tracking unit that associates each prompt with the exact page you want to cite.
How to run a split test on an LLM?
You can’t split live traffic 50-50, so instead you create a control group: a set of correlated pages that acts as a noise filter against model updates and algorithmic changes.
“Without a control group, every result would be guesswork,” Lalchandani said. “With just one, you can distinguish a real victory from the background noise. »
Timing is the discipline that most teams ignore. The methodology defines a specific reference period before any changes are implemented and a minimum testing window afterward, as AI search doesn’t respond overnight like traditional SEO sometimes does. Cut the window and, in Lalchandani’s words, “you could read noise.”
Each test yields one of three results, and each tells you something about your hypothesis. The full session covers how to construct the correlated control group, exact baseline and test windows, and how to read the three results.
Look at it full webinar on demand to get the complete test setup.
The FAQ Test That Proved Causality and Two Tests That Didn’t
seoClarity applied the same methodology for three clients and got three very different results, which is exactly the point.
The FAQ test was clearly a victory. With approximately 1,000 prompts being measured, adding FAQ sections to test pages increased citations compared to the control, and they remained high while the change was in effect. Then the team reverted the change. “Citations went down. That’s the second half of the evidence. Not that citations just went up when we added FAQs, but they went down when we removed them. That’s causation, not correlation.”
The other two tests, one on meta descriptions and the other on list formatting, ended very differently, and the reasons why provide lessons for anyone about to invest in either tactic. Find out how the two tests went in full session.
Naik’s framework: Every outcome is a victory because you have evidence rather than guesswork. This is more than most AI research teams have today.
The session also presents schema and markdown test planstwo of the most debated AEO issues today, plus a set of rapid structural testing for high-value models you will be able to run in a few weeks.
Q&A: Most useful questions from the webinar
Q: How do you measure AI authority when there is no authority measure of its own?
“AI’s authority basically depends on how much the model trusts you as a source for that topic. I don’t think there’s a clear number or a single number, but there are a few signals that you can stack up to give you some sort of working picture.”
Lalchandani named four stackable signals, starting with sharing quotes on your top prompts and consistency across engines, because “consistency across engines simply means you become the authoritative source in your category for specific types of questions.” He goes over all four, and how to follow them, during the full session.
Q: Can AI bots read answers to FAQs hidden behind bendable buttons?
“Foldable can mean a lot of different things. It’s how you make it foldable.”
It depends entirely on the implementation: one common configuration keeps the collapsed FAQs fully readable for AI search engines and Google, and another makes the content invisible to both, because “even Google won’t click on your site.” Lalchandani explains what’s what in the recording, with his ongoing advice attached: “If you’re not sure about something, test it. It takes effort, but it will give you a sure answer.”
Q: What is the ROI of an AI citation that does not generate referral traffic?
“You want to be quoted because you control what answer is actually going to appear.”
Even without a click, Naik explained, your cited page shapes the narrative inside the answer, especially in comparison queries where citations do the heavy lifting of positioning both brands. The question shifts from traffic to representation: are your USPs correctly highlighted, is the comparison correct, are inaccuracies surfacing. Lalchandani added an example warning from a real restaurant customer that shows exactly what happens when AI can’t reach your content, explained in detail in the recording.
Q: Is traditional SEO still a factor in the evolution of AI search?
“Absolutely. It’s fundamental. It’s the foundation.”
Traphagan noted that seoClarity’s oldest clients, those with well-optimized content and technically sound sites, also achieve the best results in AI search, with AI optimization as an added layer. Lalchandani added: “When we test with our clients, we rarely, if ever, find a situation where something works for SEO and doesn’t work for AI search. »
Watch the full webinar
The on-demand recording contains everything the summary remembers: the golden prompt set construction, level definitions, control group construction with exact baseline and test windows, platform-by-platform crawler benchmark, meta description and list results, and schema and markdown test plans. Register to watch the full session on demand.
