S01E01: SEO in the Age of AI Agents

  • Level L2 (intermediate)|~13-17 min
  • no prerequisites
  • published 2026-11-13|verified 2026-11-09
  • seogeoai-searchpublishing

Search results changed shape: for many queries, an AI answer now sits above the blue links (the organic, unpaid results). This lesson walks the measured evidence, then the parts of SEO that still decide whether AI agents find, cite, and send you readers.

Not familiar with SEO terms? Read this first — SEO for engineers is the working-vocabulary primer (CTR, impressions, referrers); the glossary carries the course's own concepts.

The click collapsed

Start with the two measurements every number in this course leans on.

🏷️ New term AI Overview· glossary

An AI-generated answer Google displays above the web links on a results page, built from retrieved web sources and shown with links to the pages that support it. Also the model for the answer boxes other engines show. In this course: the thing that swallows the click.

Pew Research Center's behavioral panel (900 US adults, 68,879 searches, March 2025, published July 2025) watched real browsing. When Google showed an AI Overview, users clicked a web result in 8% of visits. Without one, they clicked in 15%. Half the clicks disappeared on the spot. 1% of AI-Overview visits clicked a link inside the AI answer itself. 26% of those visits ended with no click at all, against 16% without a summary.

Two ranking-side measurements point the same direction. They use different denominators, so compare the ratios, not the levels. Ahrefs (published February 2026, 300,000 keywords in two matched samples, desktop, Search Console data) compared December 2023 with December 2025. On the raw two-year slide, the position-1 result's click-through rate (CTR: the share of people who see a result and click it; the primer carries the full term) fell from 7.6% to 3.9% for informational keywords without an AI Overview, and from 7.3% to 1.6% for keywords where an AI Overview now sits above position 1, a 78% drop. Ahrefs' published effect size is deeper arithmetic than the raw slide: 58% is the 1.6% measured against a counterfactual forecast of about 3.7%, what those keywords would have reached if the AI Overview had never rolled out.

Advanced Web Ranking (published September 2026, over 1.75 million US results pages per month, all intents) measured the same squeeze from the other denominator. Position-1 click-through is 9.69% on pages with an AI Overview against 20.02% across all results pages. That is a 52% shortfall. The hit does not stop at position 1: position 2 fell from 10.36% to 3.69%. By either denominator, ranking first under an AI Overview means roughly half the click-through the same result would command without one.

Read the pattern carefully. Clicks did not slide down the page to position 5. They evaporated into the answers.

Table 1. Where an AI-Overview session ends: measured shares of visits (Pew Research Center, July 2025; US behavioral panel).
Outcome in an AI-Overview sessionShare of visits
Click a web result anywhere on the page8%
Click a web result with no AI Overview present15%
Click a link inside the AI answer1%
Session ends without any click26% (against 16% without a summary)
💡 Where the sessions end

The 10-point shift matters. Pew's panel: sessions that clicked any result fell from 15% to 8% when an AI Overview appeared, and sessions that ended with no click at all rose from 16% to 26%. A quarter of answer-page sessions now end inside the results page; no result is visited at all. No amount of ranking recovers a session that never leaves Google. Recovery has to come from somewhere else: being inside the answer.

The fall stopped in early 2026

Mid-2025 projections extrapolated the decline continuing into 2026, with the projectors themselves flagging their figures as directional floors, not predictions. It did not go to zero. Seer Interactive re-measured in April 2026: organic click-through rate on AI-Overview queries climbed from 1.3% (December 2025) to 2.4% (February 2026), an 85% rebound in two months, approaching the January-2025 baseline of 3.2%. Queries without an AI Overview ended the same stretch higher than they started: 2.8% in January 2025, 3.8% by February 2026.

🏷️ New term Citation· glossary

Your page being linked by name inside the AI answer. Cited is the state this whole course plays for: it is the position that pays clicks back at a premium, while merely ranking on the same page pays nothing.

The rebound leaves organic CTR at 2.4% against the 3.2% pre-AI-Overview baseline: a quarter down, not recovered. But the shakeout has settled into a market shape, and the shape pays a premium to one position: results cited inside the AI answer. Through 2025, a citation delivered two to five times the click-through of uncited results on the same page (Seer, 2025 tracking). The April 2026 update of the same study (53 brands, 5.47 million tracked queries, 2.43 billion impressions, an impression being a page shown in results whether clicked or not; both working terms live in the primer) sharpened it: cited pages earn +120% more organic clicks per impression than uncited pages on the same answer-bearing results page. Per million impressions: about 33,500 clicks with no AI Overview at all, 20,700 with a citation, and 9,400 uncited (Figure 1): all three all-position averages, every result on the page folded into one rate, a different denominator from the position-1 studies above. One honesty clause from the same study: a citation brings you most of the way back, not all the way: cited pages still trail no-answer pages by 38%.

Bar comparison of organic clicks per one million informational impressions: no AI Overview about 33,500 clicks, with a citation about 20,700, without a citation about 9,400. Source: Seer Interactive, April 2026.
Figure 1. Organic clicks per 1,000,000 informational impressions (Seer Interactive, April 2026; 53 brands, 5.47M tracked queries). The cited bar is more than double the uncited bar; the No-Answer bar above shows what no citation ever pays back.
💡 Two eras, one sentence each

Before AI Overviews, SEO played for the click. After them, SEO plays for the citation, and the citation pays its own clicks back at a premium.

AI referrals: rare but pre-sold

🏷️ New term AI referral· glossary

A visit whose referrer is an AI assistant (chatgpt.com, perplexity.ai, gemini.google.com, claude.ai, and friends), meaning the reader came from an AI conversation that linked your page. Small in volume, different in intent.

Two studies and one aggregation agree on the shape, not the size. The size depends on which pair of numbers you take. Orbit Media's study (July 2026, 97 business-to-business websites, 29 million visits, 364,000 conversions — where a conversion means a visitor taking the action the site wants, such as becoming a lead) found AI-referred visitors converting well above visitors from ordinary search. ChatGPT sent 2.1% of its visitors into a lead action against Google Search's 0.5%: about four times on those endpoints, where the study's published headline across all AI sources says three. Ahrefs' aggregation (2025) points the same direction from the signup side, with a much wider spread: AI visitors made up about 0.5% of measured traffic but 12.1% of signups, a 24-fold share gap measured on a different funnel with different selection. One fact survives every pairing: the volume is small in every measurement; about one visit in 200 came from an AI source.

Why the gap? Two live explanations. A reader arrives from an AI answer already briefed: the agent summarized the options and picked your page for one specific question. Equally possible: AI assistants recommend familiar brands more often, so the visitors arrive pre-convinced by prior brand exposure. That is selection bias, and the studies are correlational; they cannot rule it out. Treat the multipliers as magnitude, not gospel. Either way the direction survives: a rare visitor who arrives already convinced beats a frequent visitor who arrives indifferent.

What Google's own guide says still works

Google's own AI-optimization guide (updated July 2026, the authoritative statement in this space) closes two debates at once.

First: AI features are rooted in core ranking (Google's main search-ranking systems, the ones that order ordinary results). There is no second algorithm to game. Second: the guide explicitly dismisses a list of fashionable tactics, among them llms.txt (a proposed site file that tells AI crawlers how to read the site), "special AI markup", chunking pages specifically for AI (splitting a page into pieces so retrieval systems digest it), and AI-keyword variations. Not one of them does what its promoters claim.

What Google says still works is unglamorous: crawlability (search systems and AI agents can fetch every page without hurdles), first-hand content that is not commodity filler, clear information structure, links between related pages. This course teaches those four, because they are the load-bearing walls. The guide also looks forward: a section on agentic experiences describes browser agents that inspect screenshots, DOM structure, and the accessibility tree, and links a best-practices guide for sites serving them. Google treats agent access as a first-class concern, and episode 3 builds directly on that.

💡 Receipts, not promises

This course runs on the site that published it. Peripatos measured (November 2026 snapshot, zero promotion): about 1,630 unique visitors in 30 days, organic-search clicks near zero, nearly all inbound traffic machine traffic; a single site with no comparison group, so read that as a demo of machine-readability, not as evidence that machine-readable content attracts agents. Those are unflattering numbers. They stay in the course on purpose: the lessons here describe a site that gets found by agents but not yet by search. Every recommendation in this course is one the site itself must satisfy.

The static-site floor

Everything above is the argument. This is the checklist it rests on. For a static-first site on this course's reference stack (Astro, a static-site framework, on Cloudflare, a host), the floor is ten items long, all verified against official documentation as of this lesson's date.

Table 2. The static-site floor: the rule and why it survives (each verified against official documentation, 2026-11-09).
ItemThe ruleWhy it survives
SitemapEmit sitemap-index.xmlCrawlers and AI search both discover from it
Site URLSet site in the site configThe sitemap integration requires it to emit
robots.txtAllow-all, plus a Sitemap: lineThe allow-all admits AI crawlers by default
AI crawlersAllow the search crawlers and user-triggered fetchers: OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-UserSearch crawlers govern answer inclusion; user-triggered fetchers serve the live prompt in the citation audit and are not robots.txt-governed (OpenAI documents that user-initiated fetches "robots.txt rules may not apply"); those are edge controls, not robots.txt controls; GPTBot/ClaudeBot blocks affect training only, not citations
CanonicalsOne absolute canonical per page, consistent trailing slashNo duplicate-page ambiguity to mis-cite
RedirectsAt the host, never meta-refreshLink equity (the ranking value a link carries) and crawls follow real redirects
FreshnessUpdate substance; expose real lastmodFake freshness is detectable and worthless
HeadingsStrict H1 to H2 to H3 orderOne study correlates heading order with citations; the mechanism is not shown
Structured dataJSON-LD: Course, Article, BreadcrumbListRich-result eligibility on top of ranking; FAQ rich results are retired
Search generative AI controlIn Search Console, verify the control is not set to ExcludeExclude removes the property from AI Overviews and AI Mode entirely; Include is the default state
🏷️ New term Search generative AI control· glossary

A Search Console setting with three states — Include, Exclude, and Inherit from parent — governing whether your site's links and content can appear in Google's AI features (AI Overviews, and AI Mode — Google's conversational answer-first search mode). Include is the default state for every property, so a site that never touched Search Console is already eligible; the failure mode is Exclude. An excluded site's content cannot appear in those features, no matter how good the site is.

Three refinements keep the checklist honest. Skip changefreq and priority in sitemaps: Google ignores both. FAQ rich results are dead: they stopped appearing on May 7, 2026 and the reports went in June, so write the prose if it helps readers and skip the markup. And one setting sits at the bottom of the table because older checklists predate it: Google requires a site to be included in Search generative AI features before AI Overviews or AI Mode can show it. But the Search generative AI control's Include state is the default for every property, so the checklist action is not enrolling, it is verifying that neither your property nor a parent property has been set to Exclude, as Table 2's last row records.

One stack-specific caution: on Cloudflare, edge-level AI-bot blocking rules reject crawlers before robots.txt is ever read, so an allow-all robots.txt alone is not sufficient: check the dashboard's bot controls allow the same search crawlers and user-triggered fetchers.

And treat any heading-structure statistic you meet on social media with care: the strongest widely-shared one (cited-page heading order) is a correlation from a single content-marketing study, not a controlled experiment.

💡 Boring on purpose

The floor is boring on purpose. Nothing in the table is clever, and that is the point: every item is either verified against Google's documentation or explicitly part of their recommended stack. Clever tactics rot; the floor moves slowly, but it does move: FAQ markup died and the Search generative AI control arrived within the same year.

Measure citations, not clicks

🏷️ New term Citation audit· glossary

Whether generative engines cite you is a measurement you own: an eval you design — prompts a reader would actually type, the engines your readers use, a cadence that fits — run against live answers, recording cited or not. One pass is one sample of a nondeterministic system; the trend across passes is the signal. The only instrument in Google's ecosystem that measures citation share directly.

The old scoreboard (rankings, click-through rate) hides the new game. The scoreboard for this course has three instruments, all free, all day-one.

  1. Search Console with the Search generative AI control from the floor: its Generative AI performance report isolates impressions arriving from AI Overviews and AI Mode, by page, country, date, and device. The report gives page-level impressions only, no query breakdown (dimensions are pages, countries, dates), which is exactly why the citation audit below owns the query side. Clicks are not in that report (they stay merged into the main Search results report), and be precise about what referrer segmentation can recover: clicks out of the standalone assistants arrive with their own host in the referrer (on desktop web; in-app mobile webviews often strip the referrer, landing those visits as direct traffic), but clicks out of AI Overviews and AI Mode arrive as plain Google traffic, the same as organic; referrer logs cannot split those. Never attribute to Search Console what only your own analytics can see.
  2. Analytics segmented by AI referrer (chatgpt.com, perplexity.ai, gemini.google.com, claude.ai, copilot.microsoft.com); tiny absolute numbers with conversion attached beat anonymous totals.
  3. The citation audit: an eval you design and run yourself against live engine answers, recording cited or not. It is the only instrument in Google's ecosystem that measures citation share directly (Bing Webmaster Tools' AI Performance report plays the same role for Copilot); one pass is one sample, and the trend across passes is the measurement. The exercise below runs the minimal version.
⚠️ Instrumentation caveat

Google has stated Search Console over-counted impressions from May 13, 2025 through April 27, 2026 (clicks were unaffected), so impression totals from that window read high. Two consequences follow. Absolute per-impression rates computed inside the window read low: they are floors, not flattery. And within-the-window ratios share the inflated denominator, so they largely cancel; Seer Interactive's premium study took both of its tracking waves inside the window, which is why its cited-versus-uncited ratios are the part to trust and its absolute clicks-per-million figures are the part to read as conservative floors.

Judge the site the way the earlier sections argue you should: citation share and conversion, not raw traffic. A static site with five agents citing it and a healthy conversion rate is winning: five agents are five independent channels no traffic collapse can switch off. A site with ten times the traffic and no citations is running a museum.

What you now know

  • An AI Overview above position 1 cuts that result's clicks roughly in half (58% by Ahrefs' counterfactual comparison, 52% by AWR's cross-page comparison), and a quarter of answer-page sessions end with no click at all.
  • The clicks did not move; they evaporated. The recovery position is being cited inside the answer: about 20,700 clicks per million impressions against 9,400 uncited, a premium that still trails no-answer pages by 38%.
  • AI-referred visitors convert several times better than ordinary search visitors: correlational evidence, with brand familiarity as the rival explanation. Treat the multipliers as magnitude.
  • There is no second algorithm: core ranking, crawlability, and the ten-item floor carry everything. The one Google-specific switch worth checking is the Search generative AI control: Exclude pulls a property out of AI Overviews and AI Mode entirely, while Include is the default state.

Knowledge check

📖 This knowledge check is an interactive quiz. Open this page in a browser to take it. Reader mode shows only the prose.

Sources & further reading

  • Google users are less likely to click on links when an AI summary appears in the results Pew Research Center · research summary · verified 2026-11-09

    The behavioral-panel source for every session-level number in section 1 (900 US adults, 68,879 searches, March 2025).

  • AI Overviews reduce clicks: update Ahrefs · blog study · verified 2026-11-09

    The February-2026 measurement behind the 58% figure — actual position-1 CTR of 1.6% against a counterfactual of about 3.7% (December 2023 vs December 2025, desktop, two matched samples).

  • Optimizing your website for generative AI features on Google Search Google Search Central · official guide · verified 2026-11-09

    The official stance this lesson trusts: what works, what is dismissed, and the Search generative AI control (updated July 10, 2026).

  • AIO impact on Google CTR: 2026 update Seer Interactive · industry study · verified 2026-11-13

    The v3 study (Apr 24, 2026): the rebound numbers, the citation premium with its honest 38% clause, and the full methodology (53 brands, 5.47M tracked queries). Page 404s with a trailing slash — cite it bare.

  • AI traffic conversion rates: 97 B2B websites and 29M visits Orbit Media (Andy Crestodina) · study · verified 2026-11-09

    The largest public conversion study behind the conversion section (July 2026).

  • Organic CTR study Advanced Web Ranking · CTR study · verified 2026-11-13

    The September-2026 position-by-position measurement behind the 52% figure (over 1.75 million US results pages per month).

  • AI SEO statistics Ahrefs · aggregation · verified 2026-11-09

    The 2025 aggregation behind the signup-side conversion numbers (0.5% of traffic, 12.1% of signups).

  • GEO: Generative Engine Optimization Aggarwal et al. · KDD 2024 paper · verified 2026-11-09

    The peer-reviewed content-level evidence outside Google's guide: measurable lifts from inline citations, quotations, and statistics (+28%, +41%, +31% respectively on position-adjusted visibility; 10k-query benchmark). Full treatment in lesson 2.

🧭 Practice

Your turn. Run the minimal citation audit, the exercise that shows the principle in one sitting: write a handful of prompts in your readers' own words, run them in ChatGPT, Perplexity, and Gemini, write down cited or not for every prompt-engine pair, and record the date. Engines answer nondeterministically — one pass is one sample of a nondeterministic system — so what counts is the technique you keep from here: design the eval (which prompts, which engines, how often) your readers' behavior actually calls for, widen it where this minimal version was thin, vary the prompt phrasing from pass to pass, and let the trend across passes be the measurement. That trend is your episode-one baseline; episode 2 builds the fixes on top of it.

See also