AI crawlers are reading websites thousands of times for every customer they send back. Cloudflare’s July 2025 data showed Anthropic’s bot crawling 38,000 pages for each referred page visit, the highest imbalance among major AI players. That single ratio is the clearest early signal of where commerce discovery is moving in 2026: from a search results page that hands a customer to your site, to an agent that reads, weighs, and decides on the customer’s behalf.
McKinsey projects that $750 billion in US revenue will funnel through AI-powered search by 2028. Half of consumers already use it, and 44 percent of those users call it their primary and preferred source of insight. The founders who restructure their catalogs, trust signals, and measurement for agent retrieval now will own the next decade of commerce visibility. The ones who wait will spend the cycle catching up.
The Third Inversion of Commerce Discovery
Commerce discovery has flipped twice in the last 17 years and is flipping a third time. The first flip was the social era, when community-driven discovery rerouted traffic away from the search-and-display playbook. The second was the SEO era, when content depth and link equity decided which brands customers ever found. The third is the agent era, and it is happening right now. Agents read claims, weigh evidence, and choose. The brand that cannot back its claims gets skipped without warning.
The crawlers are showing up first. Cloudflare tracked a 24 percent year-over-year increase in AI and search crawling in June 2025, slowing to 4 percent in July, and training now drives nearly 80 percent of AI bot activity, up from 72 percent a year ago, per the July 2025 crawler-to-referral data set.
The transaction shift is already forecast. McKinsey’s AI Discovery Survey of 1,927 US consumers, fielded in August 2025, found about half of users intentionally seek out AI-powered search engines, and a majority call it the top digital source for buying decisions. McKinsey projects $750 billion in US revenue will move through that channel by 2028, per the $750 billion AI search revenue projection. Unprepared brands may see a 20 to 50 percent decline in traffic from traditional search channels. The inversion is not coming. It is here.
- 38,000:1: Anthropic’s crawl-to-referral ratio, July 2025 (Cloudflare)
- 58%: drop in click-through rate for top-ranking pages since AI Overviews launched (Ahrefs, 300,000 keywords)
- 0.664: Spearman correlation between brand web mentions and AI Overview visibility (Ahrefs, 75,000 brands)
- $750 billion: US revenue projected to flow through AI-powered search by 2028 (McKinsey)
- 20 to 50%: projected decline in traditional search traffic for unprepared brands (McKinsey)
Why SEO Alone Will Not Get You Cited
The click economy is shrinking. Ahrefs analyzed 300,000 keywords using Google Search Console data, comparing December 2023 (before AI Overviews) to December 2025, per the 300,000-keyword click-through rate study. Out of every 100 clicks that once went to top-ranking pages, Google now keeps 58.
Backlinks are no longer the load-bearing signal. Ahrefs studied 75,000 brands and ranked the correlations between off-site and on-site factors and AI Overview brand visibility, per the 75,000-brand AI visibility correlation study. Brand web mentions topped the list at 0.664 on the 0 to 1 Spearman scale. Brand anchors came in second at 0.527, then brand search volume at 0.392. The number of backlinks correlated at 0.218, and Domain Rating at 0.326. Off-site text signals beat off-site links by roughly three to one.
Mention density is winner-take-all. Brands in the top quartile for web mentions averaged 169 AI Overview mentions, more than 10 times the next quartile’s 14. Brands in the bottom 50 percent of web mentions barely register, averaging 0 to 3 AI Overview mentions. Ahrefs found 26 percent of the 75,000 brands it studied had zero mentions in AI Overviews. If a brand is not being discussed across the web already, AI is unlikely to start citing it now.
The consumer side is already behaving like the new ranking. Google AI Overviews appeared for 21 percent of all keywords in a 146 million search-result-page study, and 46.4 percent of queries with seven or more words. The shift is not a fringe behavior: half of users in McKinsey’s survey call AI-powered search their primary and preferred source of insight, ahead of traditional search at 31 percent, brand or retailer sites at 9 percent, and review sites at 6 percent. The old funnel is being rerouted, not replaced.
| Signal type | SEO era weight | AI era correlation (Ahrefs 75K) |
|---|---|---|
| Brand web mentions (unlinked) | Low | 0.664 |
| Brand anchors | Moderate | 0.527 |
| Brand search volume | Supportive | 0.392 |
| Backlinks / Domain Rating | High | 0.218 / 0.326 |
| Paid search ads | Direct driver | 0.216 |
Audit Your Discovery Stack
The first shift is also the most uncomfortable. Most founders cannot name their top three discovery channels by source without opening a dashboard. The shift from SEO-driven traffic to agent-mediated discovery is invisible to anyone who has not built the map. The audit is the whole game, because every reallocation that follows depends on it.
The discipline is to draw two pictures and put them side by side. Map every path a customer took to reach your site in 2023, then map the same paths for 2026. The crawlers show up first, the transactions come later, and most operators are not measuring the share of their inbound discovery that is now agent-mediated. Even the dominant incumbent is not insulated, as detailed in how Alphabet’s search moat is cracking in the AI era. If the agent share is climbing at your discovery stack and you have no instrumentation for it, you are flying blind into a transition that has already started.
Restructure Your Product Data So an Agent Can Read It
The catalogs founders wrote in 2018 were built for human browsers. Marketing copy. Image-first galleries. Prose reviews. An agent reading on behalf of a customer cannot use any of it. Agents need claims they can test, prices they can verify in real time, and provenance they can cite back to the source.
McKinsey’s analysis puts a number on the gap: in many categories, a brand’s own sites comprise only 5 to 10 percent of the sources that AI search references. The rest is affiliates, user-generated content, third-party review sites, and Reddit threads. Your own product pages are no longer the only store window, and for many categories they are not even the main one.
The calculus is short. An agent will not recommend a product it cannot defend. Whatever percentage of your catalog an agent could not defend if asked is the percentage of your brand that agent commerce will quietly route around. The fix is to publish prices, specs, and claims in a structured, machine-readable form, with timestamps, sources, and revision histories attached. Schema markup is no longer an SEO nice-to-have. It is the wire format for agent retrieval.
Rebuild Your Trust Signals for Agent Retrieval
Search engines rewarded backlink profile and keyword coverage for 15 years. Agents evaluate a different stack across three dimensions, and a brand that scores on none of them reads to an agent as marketing copy, not as evidence.
The first dimension is provenance. Every claim on your site needs a verifiable trail: where the data came from, when it was last validated, and who else can confirm it. A claim with no provenance is a claim an agent has no way to defend, and an agent that cannot defend a claim will not cite it. The second dimension is willingness to be wrong. Brands that publish only what they got right read to an agent as marketing. Brands that publish a public record of what they predicted, audited, and revised read as a research operation.
The cost of skipping that distinction is concrete. In an experiment published in Ahrefs’ Q1 2026 AI Search Benchmark Report, a researcher invented a fake company and planted false claims online. The claims were repeated as truth across nearly every AI tool tested, even though an official company FAQ explicitly denied them. When the training corpus is contaminated, agents do not adjudicate. They amplify.
The third dimension is third-party density, and the data on it is the most direct of the three. Ahrefs’ study of 75,000 brands found mentions of a brand across independent authoritative sources correlate with AI citation at 0.664 on a 0 to 1 scale. Backlinks correlate at 0.218. The math has already moved. The budget that used to fund link-building should be redirected toward earned third-party coverage, published methodology, and original research that other sites cite.
The shift is not optional. McKinsey’s research is blunt: in major categories like credit cards, hotels, electronics, and apparel, top brands can be absent from some answers across top AI-powered search platforms, including Google AI Overview. Strong brands in 2024 are not safe citations in 2026. Visibility is now something a brand has to earn from agents, not something the brand’s prior reputation confers.
Traditional brand strength is no indicator a brand is ready to compete in the new world of AI-powered search. Visibility is not guaranteed.
McKinsey & Company, “New front door to the internet: Winning in the age of AI search,” October 16, 2025.
Instrument the Feedback Loop, Then Reallocate
The final shift is operational, and almost no one is doing it. In 2010, every operator I worked with had a SERP position dashboard. The 2026 equivalent is a monthly AI citation report: define a query set, run it across the four major retrieval surfaces (Google AI Overviews, ChatGPT, Perplexity, and Claude), and track which queries return your brand and which do not. Cohort the queries by buying stage, with research queries, comparison queries, and purchase-intent queries tracked separately. The loop turns the shifts above into a permanent operating discipline.
The reallocation follows the loop. The crawlers are showing up first and the transactions come later, so spend on third-party coverage and published methodology now, because that is what agents will cite in 2027 and 2028. Spend on structured product data now, because that is the wire format agents read. Defer the spend on tactics built only for the old ranking economy. Cloudflare’s data shows the crawlers are already inside the perimeter. The founders who instrument the loop and reallocate first will own the next cycle of commerce search visibility. The ones who wait will spend the cycle catching up.
Frequently Asked Questions
What is AI search visibility and why does it matter?
AI search visibility is whether a brand shows up in the answers that AI assistants and AI Overview panels produce, not just in the ten blue links below them. McKinsey’s August 2025 survey of 1,927 US consumers found 50 percent of users now intentionally seek out AI-powered search, and 44 percent of those users call it their primary source of insight. If an agent does not cite your brand, the customer it is shopping for never sees you.
What is the crawl-to-click gap?
The crawl-to-click gap is the imbalance between how often an AI bot reads a website and how often it sends a human visitor back. Cloudflare measured Anthropic’s bot crawling 38,000 pages for every page visit it referred in July 2025, the highest imbalance among major AI players. The gap matters because it shows who is consuming your content and who is not returning the traffic that sustained the open web.
Which signals most affect whether AI cites a brand?
Ahrefs’ study of 75,000 brands ranked brand web mentions highest, at 0.664 on a 0 to 1 scale. Brand anchors came in at 0.527, branded search volume at 0.392, and backlinks at 0.218. Off-site text signals beat link metrics by roughly three to one.
How do I audit my discovery stack?
Map every path a customer took to reach your site in 2023 and the same paths in 2026. Most founders cannot name their top three discovery channels by source without checking a dashboard. The audit is the foundation for every reallocation that follows.
How do I track AI citations each month?
Define a fixed query set, run it across the major retrieval surfaces (Google AI Overviews, ChatGPT, Perplexity, and Claude), and log which queries return your brand and which do not. Cohort the queries by buying stage so you can see whether you are losing research queries, comparison queries, or purchase-intent queries. Treat the output as the 2026 version of a SERP position dashboard.
